o    1
o    2
o    3
o    4
o    5
o    Not important
o    Fairly important
o    Very important
·         13
  1. If predicted results demonstrate the original aim of the forecaster, does that (in your view) increase or negate the credibility of the forecast used?*
e.g. If an organisation uses market forecasts to investigate the need for a new product, and then concludes that a new product is needed, does that give extra credibility to the original prediction or do you see this a self-fulfilling prophecy?
o    1
o    2
o    3
o    4
o    5
o    Reduces credibility
o    No effect
o    Increases credibility
·         14
If commercial ventures, which are based on forecast results, prove to be successful, does that increase the credibility of the forecast used?*
e.g. If an organisation uses market forecasts to outline a business case for a new product, and then is successful with this new product in the marketplace, do you think the success demonstrates the accuracy of the original prediction, or do you think other factors may have played a more important role in the success?
o    1
o    2
o    3
o    4
o    5
o    Never
o    Occasionally
o    Very often
·         15
How important is it to you that a forecast can be extended to address other purposes outside of the original aim that it was intended for?*
e.g. If a weather forecast could also give you details of solar power generation levels, would this make the forecast more valuable in your view, or would this detract from the main purpose of the forecast?
o    1
o    2
o    3
o    4
o    5
o    Not important
o    Fairly important
o    Very important
·         16
How important is it for credibility that simulations tackle real-world problems as opposed to laboratory tests?*
e.g. If a transport forecast uses simplified assumptions to predict traffic flows between cities (such as the assumption that the overall global population level does not change with time), would the results of this forecast still represent, in your view, a valid indication of the future?
o    1
o    2
o    3
o    4
o    5
o    Not important
o    Fairly important
o    Very important
·         17
How important is it to you that forecasts are targeted at a specific industry, domain, or condition (as opposed to a more generic prediction)?*
e.g. Would you see a Bristol-based weather forecast as being more reliable than the BBC World Service weather forecast? (i.e. does a more specific context help to justify the predictions being made?)
o    1
o    2
o    3
o    4
o    5
o    Not important
o    Fairly important
o    Very important
·          “
Forecasting methods
Continue
press ENTER
·         18
How credible do you find it when analogies are used in research to support the outcome of forecasts?*
e.g. If a city planning department uses observations of ant colonies and termite mounds to explore pedestrian movements and behaviours within city development concepts, do you think this is a valid analogy?
o    1
o    2
o    3
o    4
o    5
o    Never credible
o    Occasionally credible
o    Very credible
·         19
How convincing do you find the analogies commonly chosen in the media to explain the assumptions made and methods adopted in scientific research?*
e.g. When scientific information is presented on the news, can you see how the analogies presented from natural events, biology, sociology, etc. matches the approaches taken by the researchers?
o    1
o    2
o    3
o    4
o    5
o    Not convincing
o    Occasionally convincing
o    Very convincing
·         20
How important do you find it to see the rationale behind the selected assessment criteria for future predictions?*
e.g. If you are told that the main concern for a new technology in the future is its carbon dioxide emissions, (as opposed to the use of rare earth minerals to produce the technology, etc.), are you convinced that this is the main concern to focus on?
o    1
o    2
o    3
o    4
o    5
o    Not important
o    Fairly important
o    Very important
·         21
Do you feel that popular media and press predictions of the future generally use the right criteria to measure future changes?*
e.g. Would a car magazine describing the energy needs of future electric vehicles in terms of 'miles per gallon' give you a clear impression of future energy consumption?
o    1
o    2
o    3
o    4
o    5
o    Never
o    Occasionally
o    Very often
·         22
If a forecast is shown to be the product of a collaborative effort does that increase your confidence in the results?*
e.g. Do you think that group consultation would help to improve the realism of a market forecast compared to those developed in isolation?
o    1
o    2
o    3
o    4
o    5
o    Reduces confidence
o    No effect
o    Increases confidence
·         23
If there are sporadic gaps present in the data sets used to generate a forecast, do you still think it is possible to generate meaningful and helpful results?*
e.g. As exact migration numbers between countries are often not fully known, do you think migration forecasts are still meaningful? (i.e. do you think that missing information always presents a critical obstacle to producing helpful analysis?)
o    1
o    2
o    3
o    4
o    5
o    Never
o    Occasionally
o    Very often
·         24
How effective do you find sensitivity studies at improving your confidence in a simulation?*
e.g. If published results from a transport forecast include analysis of the impact on predictions of varying key model parameters (such as national GDP and the price of oil) does this give you more confidence in the accuracy of forecasts produced?
o    1
o    2
o    3
o    4
o    5
o    Not effective
o    Moderately effective
o    Very effective
·         25
Do you think that the accuracy of predicted trends at a global (or macro) level is improved by forecasting patterns that emerge at a microscopic level?*
e.g. Do you think that it would be possible to build up a more accurate picture of global weather patterns from combining all localised weather forecasts into a single model, or would this lead to increased errors in predictions? (i.e. would the 'sum of all parts' be equivalent to 'the whole', or would there be significant differences if you looked at forecasting on two different scales?)
o    1
o    2
o    3
o    4
o    5
o    Very unlikely
o    Unsure
o    Very likely
·         26
How many alternative methods do you think forecasts should be compared against to demonstrate sufficient credibility?*
e.g. How many alternative climate models would you like to observe reproducing the same results as your own climate model to have confidence that your forecast of climate change was realistic?
Choose as many as you like
0 (no comparison necessary)
1 to 4
5 to 9
10+
No fixed number (dependant on method being trialled)
Ok
press ENTER
·         27
How regularly do forecast results need to be compared against results obtained using alternative approaches to provide you with confidence that the prediction method used is still the most appropriate for the task?*
e.g. If a population growth forecast based on current migration flows is generated every year, providing a 10 year prediction of a nations’ changing demographics, how often do you think this annual forecast needs to be compared to growth projections based on future labour force estimates (or any other new approaches) to give you confidence that the migration flow approach is still the most appropriate method?
o    1
o    2
o    3
o    4
o    5
o    After several decades
o    After several years
o    Annually
·         28
How long should it take to gain an understanding of the modelling principles behind a forecast for the results produced to be considered easy to reproduce and verify in a commercial environment?*
e.g. If it was possible to learn how an economic forecasting model works in a short period of time, would that suggest to you that methodological errors would be identified and resolved fast enough to make the technique commercially useful?
o    1
o    2
o    3
o    4
o    5
o    Order of months
o    Order of weeks
o    Order of days
·         “
Supporting information provided with forecasts
Continue
press ENTER
·         29
In your view, how important is comprehensively referenced background documentation (traceability) to the overall credibility of a forecast?*
e.g. If an economic forecast presents plausible market results but does not provide sufficient references to explain the current market conditions and the types of forecasts that have been produced in the past, do you still find value in the conclusions?
o    1
o    2
o    3
o    4
o    5
o    Not important
o    Fairly important
o    Very important
·         30
How important do you think it is to see evidence of a rigorous methodology to provide confidence in the results of a forecast?*
e.g. If a climate forecast suggests environmental changes that appear credible, but there is no clear indication of the logic behind the current predictions, then are the conclusions presented still worthwhile to you?
o    1
o    2
o    3
o    4
o    5
o    Not important
o    Fairly important
o    Very important
·         31
How important do you think it is to see evidence of well-defined model boundaries to provide confidence in the results of a forecast?*
e.g. If an economic forecast suggests market changes that appear credible, but there is no clear indication of the limits of validity (i.e. conditions in which the model will not produce meaningful results), then are the conclusions presented still worthwhile to you? (for example, a forecast of European economic growth patterns may not be valid for Chinese markets where a different political and economic system exists)
o    1
o    2
o    3
o    4
o    5
o    Not important
o    Fairly important
o    Very important
·         32
How important do you think it is to see evidence of the iterative development of forecasting models to provide confidence in the results produced?*
e.g. If an election forecast suggests political changes that appear credible, but there is no clear indication of how the forecasting model has evolved into its current form based on the results of previous elections, then are the conclusions presented as valuable to you?
o    1
o    2
o    3
o    4
o    5
o    Not important
o    Fairly important
o    Very important
·         33
How important do you think it is to see evidence that a range of different sources, opinions, and perspectives have been taken into account when generating forecasts to provide confidence in the results produced?*
e.g. If a technology forecast suggests that new innovative technologies are likely to become commercially viable in the near future, but there is no clear indication of opinions provided by industry and technology experts, manufacturers, or members of the public being reviewed, then are the conclusions presented as valuable to you?
o    1
o    2
o    3
o    4
o    5
o    Not important
o    Fairly important
o    Very important
·         34
In your view, how important is complete traceability of all data sources to the overall credibility of a forecast?*
e.g. If a market forecast presents plausible results but does not provide the sources for all of the market data used, do you still find value in the conclusions?
o    1
o    2
o    3
o    4
o    5
o    Not important
o    Fairly important
o    Very important
·         35
How important do you think it is to see evidence of sensitivity studies being conducted on a model's initial conditions (i.e. the original state of the environment being modelled) to provide confidence in the final results of the forecast?*
e.g. If a climate forecast presents significant changes resulting from one assumed average global temperature, but does not examine the effect of different starting temperatures, do you still find value in the conclusions?
o    1
o    2
o    3
o    4
o    5
o    Not important
o    Fairly important
o    Very important
·         36
In your view, how important is the demonstration of a rigorous software-checking process to the overall credibility of a computer-generated forecast?*
e.g. If a population growth forecast presents plausible results but does not provide detailed evidence of checks carried out on the computer code used to run the simulation, do you still find value in the conclusions?
o    1
o    2
o    3
o    4
o    5
o    Not important
o    Fairly important
o    Very important
·         37
How effective do you find the inclusion of recommendations for future modelling developments in improving the credibility of forecasting techniques?*
e.g. If a traffic forecast acknowledges current weaknesses in incorporating severe weather effects and outlines future refinements to improve this aspect of the simulation, does this give you confidence that the basic forecasting assumptions are sound and will ultimately lead to a reliable forecast?
o    1
o    2
o    3
o    4
o    5
o    Not effective
o    Moderately effective
o    Very effective
·         “
Using hindsight as a guide to foresight
Continue
press ENTER
·         38
In your view, does the examination of past scenarios give a good indication of uncertain future conditions?*
i.e. How reliable do you find references to the past as an indication of the future?
o    1
o    2
o    3
o    4
o    5
o    Never
o    Occasionally
o    Very often
·         39
Do you need to see evidence of accurate forecasts being produced by existing applications of a given modelling technique before you are convinced of the method's viability?*
i.e. Would you consider using a new forecasting technique in your day-to-day work without examples of the benefits demonstrated in other applications?
o    1
o    2
o    3
o    4
o    5
o    Never
o    Occasionally
o    Always
·         40
How accurately should a simulation be able to reproduce demonstrated results in order to provide you with confidence that future forecasts will also be useful?*
e.g. If a product demonstrator is manufactured that provides real-life performance estimates of a new technology, how closely do you think a computer-generated simulation should be able to emulate these results to be considered useful?
o    1
o    2
o    3
o    4
o    5
o    "Ball park" accuracy is good enough
o    Close consistency is required
o    Exact numbers are required
·         41
Does the use of a historic test sample to calibrate models provide you with increased confidence that any forecasts produced by the model would accurately predict future disruptions?*
e.g. Does the ability to reproduce historic patterns observed in financial markets give you increased confidence that an economic forecasting method will be accurate for predicting sudden market crashes in the future?
o    1
o    2
o    3
o    4
o    5
o    Reduces confidence
o    No effect
o    Increases confidence
·         42
How many historical scenarios should a forecast be compared against to demonstrate a high level of confidence in predicting disruptions?*
Choose as many as you like
0 (no correlation with the past)
1 to 4
5 to 9
10+
No fixed number (dependant on environment being forecast)
Ok
press ENTER
·         43
How accurately should a forecasting model be able to reproduce historic trends in order to give you confidence that future predictions will be useful?*
e.g. If a population forecast can approximate the changing global demographics over the past 20 years, and whether these populations are increasing or decreasing overall, is this accuracy sufficient (in your opinion) to use as a basis for future decisions on population-related environmental policies (such as legislation on resource consumption)?
o    1
o    2
o    3
o    4
o    5
o    "Ball park" accuracy is good enough
o    Close consistency is required
o    Exact numbers are required
·         “
Generating useful information
Continue
press ENTER
·         44
Is it more important for a long-term forecast to provide indicative general trends for future scenarios, or specific rankings of different possible outcomes?*
e.g. If a technology forecast is produced to examine the possible evolutions of existing products over the next 20 years, is it necessary to be able to predict rankings of specific technologies against each other in order to make valuable investment decisions, or are general trends sufficient in this long-term timeframe?
o    1
o    2
o    3
o    4
o    5
o    General trends
o    Unsure
o    Specific probabilities
·         45
How extensively do the root causes of predicted events need to be known in order to provide you with a valuable insight into expected future changes?*
e.g. How much certainty would you need of the root cause of predicted increases in natural gas prices in order to make a decision on how to heat your home in the future?
o    1
o    2
o    3
o    4
o    5
o    Causes do not need to be known
o    Causes should be estimated
o    Causes should be proven
·         46
How extensively do you feel you rely on pattern recognition to assess the credibility of a prediction?*
e.g. If a weather forecast suggests the occurrence of familiar/recognisable pattern of weather at a particular time of year (such as summer storm fronts) does that suggest an increased likelihood of accuracy to you in model predictions?
o    1
o    2
o    3
o    4
o    5
o    Never
o    Occasionally
o    Very often
·         47
How important is it to you that forecasts are able to capture the appearance of previously undiscovered patterns or events?*
e.g. If a market forecast identifies a new emerging phenomena (such as a previously unobserved type of consumer behaviour), does this increase the significance to you of the results being produced by this market forecast?
o    1
o    2
o    3
o    4
o    5
o    Not important
o    Fairly important
o    Very important
·         48
If new patterns are discovered within a forecast that cannot be easily explained using existing knowledge does this reduce your view of the credibility of the overall results?*
e.g. If a new weather phenomena (such as El Nino) is predicted that cannot easily be traced back to known climate science, does the uncertainty associated with the predicted change in weather patterns decrease your confidence in the credibility of the original forecast?
o    1
o    2
o    3
o    4
o    5
o    Reduces confidence
o    No effect
o    Increases confidence
·         49
How confident are you generally in the ability of human judgement to filter out and eliminate 'background noise' from forecasts whilst preserving critical information?*
e.g. When examining predictions of fluctuating financial markets, do you think humans are effective at correctly identifying the most important information to analyse, and subsequently the most important information to use for guiding investments?
o    1
o    2
o    3
o    4
o    5
o    Not confident
o    Moderately confident
o    Very confident
·         50
How confident are you generally in the ability of automated processes (i.e. computers) to filter out and eliminate 'background noise' from forecasts whilst preserving critical information?*
e.g. When examining predictions of fluctuating financial markets, do you think computers are effective at correctly identifying the most important information to analyse, and subsequently the most important information to use for guiding investments?
o    1
o    2
o    3
o    4
o    5
o    Not confident
o    Moderately confident
o    Very confident
·         51
Computer simulations are now often used in disaster and emergency planning exercises. Do you believe that current virtual modelling techniques are sophisticated enough to be credible for realistic contingency planning in conditions that cannot be recreated in a laboratory environment?*
e.g. In your opinion, could a computer model generate a reliable representation of a large-scale emergency for rapid decision and contingency planning purposes, that could not be tested otherwise?
o    1
o    2
o    3
o    4
o    5
o    Very unlikely
o    Unsure
o    Very likely
·         52
Do you think that the results of simulations play as important a role in guiding effective strategic decision-making as other information gathering approaches?*
e.g. In your view would a computer-generated model of the introduction of a new product to market  provide the necessary breadth and quality of information, compared to the results of a focus group analysis, to directly steer the outcome of commercial decisions?
o    1
o    2
o    3
o    4
o    5
o    Less important
o    Equally important
o    More important
·         “
Real-world behaviours
Continue
press ENTER
·         53
How important do you think it is to represent human factors in predictions of the future?*
e.g. If a transport forecast includes an assessment of the value of comfort to passengers, and the influence this has on their travel choices, do you think this will produce a more realistic prediction of future traffic levels?
o    1
o    2
o    3
o    4
o    5
o    Not important
o    Fairly important
o    Very important
·         54
In your opinion, can computer simulations provide a useful representation of human individuality?*
e.g. If a city planning office uses computational behavioural models to predict pedestrian and vehicle movements through a new city development, do you think this will provide them with an improved understanding of how the constructed developments will actually be used?
o    1
o    2
o    3
o    4
o    5
o    Never
o    Occasionally
o    Very often
·         55
In your opinion, can computer simulations provide a useful representation of cultural and organisational behaviours?*
e.g. If a market forecast uses computer-generated models to represent organisations and their commercial strategies, do you think this will provide an improved understanding of how and when real-life organisations will compete and collaborate?
o    1
o    2
o    3
o    4
o    5
o    Never
o    Occasionally
o    Very often
·         56
How important is it to you that simulations are able to identify the impact of changes beyond the main field of interest for the forecast?*
e.g. If an energy forecast predicts that crude oil and natural gas sources will become scarcer in the future, leading to a rise in supply costs, and that this will have a direct impact on the economies of nations heavily dependent on oil and gas exports, does the identification of these ‘knock-on effects’ increase the usefulness for you of the original energy forecast?
o    1
o    2
o    3
o    4
o    5
o    Not important
o    Fairly important
o    Very important
·         57
If a forecasting technique can be shown to be useful for predicting multiple criteria of interest, does that increase your confidence that the results produced for the main area of focus are realistic?*
e.g. If a transport forecast is used for predicting both traffic congestion levels and vehicle emissions, do you think that the traffic forecast is more likely to produce realistic results, or does this introduce additional uncertainty?
o    1
o    2
o    3
o    4
o    5
o    Reduces confidence
o    No effect
o    Increases confidence
·         58
The 'Butterfly Effect' describes the ability of tiny variations in environmental conditions to drastically change the outcome of large-scale events. Do you think that forecasts which include these local variations would provide a more valuable explanation of real-life chaotic behaviours?*
e.g. Do you think that if a weather forecast was detailed enough to distinguish between very subtle environmental variations it would be able to accurately identify the causes of extreme weather events (that may otherwise have been undetected), or would this only introduce additional uncertainty?
o    1
o    2
o    3
o    4
o    5
o    Very unlikely
o    Unsure
o    Very likely
·         “
Forecasting disruptions
Continue
press ENTER
·         59
If a disruptive event is predicted in the future, does having knowledge of the likely extent and localisation of the impact provide you with increased confidence that the disruption is likely to occur?*
e.g. If an economic forecast provides details of the businesses that would be most affected by a potential market crash, and the depth of the market impact, does this increase the probability, in your opinion, that the forecast disruption is realistic?
o    1
o    2
o    3
o    4
o    5
o    Reduces confidence
o    No effect
o    Increases confidence
·         60
When a disruptive event occurs, do you think it is more important to accurately forecast the magnitude of the disruption or the time taken to adjust to the changes made?*
e.g. If a technology forecast predicts that an emerging technology (such as 3D printing) will soon revolutionise a wide range of industries, do you think it is more important from a commercial point of view to understand the scale of the changes that will be made, or the time that will be required for the changes to be implemented?
o    1
o    2
o    3
o    4
o    5
o    The magnitude of the disruption
o    Equally important
o    The time taken to adjust
·         61
Do you think that by mapping the propagation of disruptive events beyond their initial field of origin the realism of current forecasting techniques would be improved?*
e.g. If an air traffic forecast identifies the spill-over effects from the sudden closure of a major airport on to the road and railway networks, does this increase the realism of the traffic forecast, or does this only introduce additional uncertainty?
o    1
o    2
o    3
o    4
o    5
o    Very unlikely
o    Unsure
o    Very likely
·“
That's the last question! Thank you for taking part in this survey, and providing an insight into your views on forecasting future disruptions. You can now submit your results on the next screen
Continue
press ENTER
Ok
press ENTERpress SHIFT + ENTER press ENTER
Create your own Typeform...
0 of 61 answered
Submit
press ENTER
Review
press ENTER
Never submit passwords! - Report abuse
Thanks for completing this Typeform Now create your own — it's free, easy & beautiful
Create a Typeform
press ENTER
Create a Typeform
press ENTER