Bill's books

  • As an aerospace engineer on Apollo, an Air Force officer, a Silicon Valley manager, professor of technology & innovation, and president of TechCast, I have always been fascinated with the revolutionary power of technological change driving us into a high-tech global order. My work is devoted to helping all of us -- especially leaders in business and government -- figure out where this profound transition is heading, what it all means, and how we can get there. Bill Halal.
  • Testing Forecast Repeatability: Before and After Data on the Move from TechCast.org to TechCastGlobal.com

     

    TechCast’s recent move from its 6th generation website (www.TechCast.org) to its new 7th gen site (www.TechCastGlobal.com) offered a rare opportunity to test the repeatability of forecast data.

    As the move was approaching, we captured one of the last data sets from the old site on Jan 29, 2014. The extensive background data framing each forecast (research breakthroughs, applications, new ventures, adoption levels, etc. organized into trends) was transferred to the new site, but we decided to drop the old expert estimates and have experts enter new estimates from scratch.  Below is a summary of forecasts from the Jan 29 data (Before) and the most recent data (After) on Oct 13, 2014.


    FORECAST

    BEFORE

    AFTER

    CHANGE
    3-D Printing 2017 2017 0
    Cloud/Grid 2016 2017 +1
    Space Tourism 2015 2018 +3
    Intelligent Web 2017 2018 +1
    Climate Control 2022 2018 -4
    E-Government 2017 2019 +2
    Global Brain 2017 2019 +2
    Fuel Cell Cars 2019 2019 0
    Biometrics 2018 2020 +2
    Virtual Education 2018 2020 +2
    E-Healthcare 2018 2021 +3
    Intelligent Interface 2019 2021 +2
    Green Economy 2020 2021 +1
    Internet of Things 2020 2021 +1
    Hybrid Cars 2019 2021 +2
    Synthetic Biology 2023 2022 -1
    Small Vehicles 2023 2022 -1
    Virtual Reality 2019 2022 +3
    Intelligent Cars 2021 2022 +1
    Commercial Space 2023 2023 0
    Power Storage 2020 2023 +3
    Precision Farming 2021 2023 +2
    Water Purification 2028 2023 -5
    AI 2024 2024 0
    GMO Crops 2021 2024 +3
    Nanotechnology 2022 2024 +2
    Smart Grids 2024 2025 +1
    Aquaculture 2025 2025 0
    Modular Buildings 2026 2025 +1
    Replacement Organs 2026 2026 0
    Personalized Medicine 2025 2026 +1
    Smart Robots 2026 2027 +1
    Alternative Energy 2025 2027 +2
    Next Gen Computing 2025 2027 +2
    Cancer Cure 2029 2029 0
    High-Speed Rail 2030 2029 -1
    Thought Power 2025 2030 +5
    Gene Therapy 2029 2031 +2
    Organic Farming 2025 2031 +6
    Solar Satellites 2036 2031 -5
    Humans On Mars 2037 2033 -4
    Moon Base 2035 2034 -1
    Neurotechnology 2031 2034 +3
    Child Traits 2034 2035 +1
    Life Extension 2040 2037 -3
    Star Travel 2073 2055 -18

    MEAN CHANGE = +.38 years

    Note:  Forecasts are for varying adoption levels.

    Results

    This simple test is a good way to check repeatability of a research method. It is often thought that such results are “anchored” by the existing forecast data, which is another way of saying experts are “biased” by the present results. The resulting mean error of .38 years seems remarkably small, especially considering that the average forecast has a time horizon of at least 10 years out.

    Repeatability is not the same as accuracy, of course, and that’s where our annual accuracy studies come in, We have found from previous studies that TechCast accuracy is on the order of +3/-1 years at about ten years out. That is, there is a tendency for experts to be over optimistic by about 3 years and under optimistic by about 1 year. This tendency toward optimism is well-reported in the literature on forecasting. We call this “forecast creep” – the tendency for forecasts to slowly creep into the future by about 3 years over a ten year horizon.

    Since the elapsed time between our Before and After data is about 9 months (Jan to Oct), forecast creep probably accounts for significant portion of the .38 years error. We also note that another form of the anchoring likely accounts for this repeatability. In our system of collective intelligence, the background data provides an empirical foundation of knowledge which experts use to make their estimates, thus anchoring the results to an accurate knowledge base.

    Conclusions

    This simple test demonstrates that the TechCast system is remarkably repeatable and robust. The results also confirm other studies we have done comparing results from two different groups of experts, which also seemed remarkably similar. Considering that the expert panel changes over time, and the conditions affecting forecasts also change constantly, these results support the utility of collective intelligence forecasting. There may be a small zone of error, but pooling knowledge is a great way to get good answers to tough questions.

     

    Share

    October 20, 2014   No Comments