Enabling effective and fast decision-making in organisations: forecasting with the Theta Method [REF2021]

Description of impact

It is well-known that adopting best practice benchmarking delivers improvements in business performance as well as generating broad-based changes in organisational thinking and action. Research at Bangor University by Nikolopoulos has led to analytical advances in the Theta Method, resulting in significant economic savings and efficiencies and the method being widely used across regional, national and international businesses and governments. Nikolopoulos’ successful development of his Theta Method is widely accepted as the benchmark to beat in most empirical forecasting investigations in supply chain and financial forecasting. Consequently, it is used by large multi-national organisations (e.g., Uber, Bosch, and Amazon) to benchmark their in-house developments. The world-leading analytic software house SAS consistently benchmark the in-house development they produce against the performance of the Theta Method. According to SAS Product Marketing Manager and Senior Manager Advanced Analytics SAS R&D: “The Theta method is well-recognized in the forecasting literature for its strong performance in the M3 (and M4) forecasting competitions. ... It is currently on our product roadmap to have the method included among the built-in methods that are automatically provided in the software” [5.1]. Moreover, Nikolopoulos’s Theta Method is implemented in the most widespread R statistical platform, among 9 other forecasting methods and has as of May 2020 over 6,500,000 downloads [5.2]. Forming part of the umbrella R package ‘Forecast’ it has achieved global usage and reach [5.2]. Nikolopoulos’s Theta model developments serve Uber Taxi’s needs because Uber operates in the real, physical world, with its many actors of diverse behaviours and interests, physical constraints, and unpredictability. Physical constraints, like geographic distance and road throughput move forecasting from the temporal to spatio-temporal domains. Theta’s fast and accurate marketplace forecasting enables Uber to accurately predict user supply and demand in a spatio-temporal fine granular fashion to direct driver-partners to high demand areas before they arise, thereby increasing their trip count and earnings. Since 2017 the leading international company Uber has been using Nikolopoulos’s research on the Theta Method. In a public document Uber states: “When the underlying mechanisms are not known or are too complicated…..the Theta method performs very well. In fact, the Theta method won the M3 Forecasting Competition and we also have found it to work well on Uber’s time series” [5.3]. Furthermore, Staff Data Scientist at Uber describes the successful use of the Theta method in Uber as follows: “In Uber we have an internal interactive service providing a forecast on receiving a time series input. It is integrated with some graph library and our own fast implementation of Theta, Holt-Winters, and a simple AR algorithms. It is a popular service, allowing users to visually explore data enriched with a forecast” further stating that with “interactive situation[s], the speed is almost as important as accuracy and here, the Theta method fits very well the need” [5.4]. Uber also applies Nikolopoulos’s Theta model developments in their Omphalos (Parallel and Language-Extensible Time Series Backtesting) Tool to generate error estimates and cross-validate their forecasts [5.5]. Uber’s Omphalos’ backtesting tool leverages aggregate data about user preferences to improve rider and driver experiences while optimising driver and Uber earnings in more than 600 cities worldwide with over 5,000,000 drivers. Uber Staff Data Scientist states “Theta is one of the algorithms in an ensemble used for lower frequency (weeks, months) financial forecasting. So here Theta is appreciated not for speed, but the diversity it adds. Additionally, in Uber we have hundreds of thousands of real-time time series that are monitored at high frequency. e.g. one minute. The anomalies detected raise alerts routed to appropriate engineers” [5.4]. Uber further state “these time series forecasts are essential to much of our decision making, from marketplace optimization and cost modelling to hardware capacity planning” [5.5]. Considering the above testimonial statements, a conservative estimate of the annual Uber savings from employing the Theta Method at over approximately USD750,000,000 (04-2020) per annum since 2018. Research in 2019 shows Nikolopoulos’ Theta Method yields benefits in terms of a 1.42% improvement in forecasting accuracy [3.5]. It is well established in the practitioner and academic literature that a 1% improvement in forecasting accuracy can lead up to 20% improvement in efficiency savings. However, a far more moderate rule of thumb according to Forbes [5.6] show savings at 2.5%. Even taking Forbes’ moderate figure of 2.5% we would expect an increase of 3.55% (1.42 x 2.5) in efficiency savings for Uber. Uber has 5,000,000 drivers delivering 6,900,000,000 charged trips in 2019, and 18,700,000 trips per day on average in Q1 2020, generating income of USD364 (04-2020) per month per driver [5.7]. Thus, the 1.42% improvement in forecasting accuracy at Uber is estimated to yield overall efficiency savings of USD775,320,000 (04-2020) (0.0355 x 364 x 12 x 5,000,000) per calendar year. Bosch has implemented Nikolopoulos’ Theta Methods since 2018. Bosch also uses the Theta model as a benchmark in developing their in-house forecasting applications. Chief Data Officer of Bosch, the Expert in Predictive Analytics and Forecasting Algorithms attests: “Theta method(s) is a very fast and accurate method. Since it is a benchmark difficult to beat, we use the Theta method(s) as a benchmark and also as an essential element when we estimate forecast combinations. Applications range from time series forecasting of demand and sales (such as Power Tools demand forecasting), to financial controlling and forecasting, and more advanced applications” [5.8]. Similarly, impact of Nikolopoulos’ research also applies to Amazon Web Services (AWS). AWS is one of Amazon's strongest revenue segments, generating USD35,030,000,000 (12-2019) in 2019 net sales. AWS ranks as one of the most popular public cloud infrastructure and platform services running applications worldwide, ahead of Azure and Google cloud services [5.9] and major global organisations (such as NASA, Netflix, Expedia, Samsung) operate their businesses via AWS. According to testimonial from Manager, Machine Learning Science division at AWS, Amazon Web Services “used [Nikolopoulos’] variations of the Theta method” and that their in-house forecasting methods are often benchmarked against the Theta Method [5.10]. As global companies have demonstrated by adopting Bangor University’s Theta Method, it is widely used in their forecasting applications primarily because its forecasting speed and accuracy lead to economic and resource efficiencies. The results of the 2018 M4 competition affirm Theta as a world-leading forecasting method, with results demonstrating it is the most accurate method assessed across the M4 datasets, through an extensive evaluation over 100,000 real-life time series [5.11].

Impact Summary for the General Public

Nikolopoulos’ research focuses on Predictive Analytics employing the Theta forecasting method he initially developed. His empirical research at Bangor University has led to significant efficiency savings and enhanced decision-making for multinational companies by improving forecasts. Nikolopoulos’ forecasting models have proven accuracy and superior computational speed in international competitions. Uber has applied Nikolopoulos’ Theta models worldwide since 2017 for their financial forecasts with an efficiency saving of over approximately USD750,000,000 (04-2020) per annum. BOSCH and Amazon Web Services use Theta for forecasting product demands and sales. The wider use of Nikolopoulos’ Theta Method within commerce is highlighted by over 6,500,000 downloads of the implementation of the Theta Method within the freeware R package.

Description of the underpinning research

Accurate and timely forecasting are constant requirements to ensure efficient and effective operational and strategic decision-making that ensure organisations prosper. Since joining Bangor University in 2010, Nikolopoulos made substantive advancements to his Theta Method, maintaining it as a world-leading forecasting method. Key elements include the factors (theta lines) that influence the forecast, extending the theoretical underpinnings of the method to multivariate time series and incorporating hybrid approaches integrating machine-learning techniques [3.1, 3.2, 3.3, 3.4].

Nikolopoulos’ research since 2010 further developed his Theta Method to gain a widely recognised reputation as the leading authority in the forecasting field on delivering accurate and fast forecasts while tackling large very high frequency real-time data streams [3.1, 3.2, 3.3]. This is particularly true when demand is challenging to predict and where wastage can be high such as in the case of taxi services. An accurate short-term forecast yields efficiency savings such as better inventory positions, less stock, less stock-outs and therefore less inventory costs. However, forecasting methods also need to be fast and ‘computationally cheap’ to tackle very high frequency real-time data (e.g. Uber). Nikolopoulos’ Theta Method is a time series univariate [3.1], and multivariate [3.2] forecasting method. Theta Method is internationally recognised as a leading forecasting method delivering this balance between statistical accuracy and computational cost [3.3, 3.6]. In 2000 the Theta Method was the winner of M3, the third international Makridakis Competition with over 3000 time series and 24 different methods competing. As the winner of the M3 competition Nikolopoulos’s Theta Method was chosen as a benchmark in the 2018 M4 competition. The Theta Method used in the M4 competition incorporated advancements to the method based on Nikolopoulos’s Bangor University research.

The results of the 2018 M4 competition affirm Theta as a world-leading forecasting method, with results demonstrating the Theta Method to be the most accurate method assessed across the M4 datasets. A 2020 article [5.11] further states “Theta provides more accurate PFs (point forecasts) than Comb (combination methods), ARIMA and ETS, despite utilizing a simple approach to forecasting”. Further, of the M4’s more advanced benchmarks, the Theta Method is the more accurate and faster benchmark method of the M4 forecasting competition in 2018, through an extensive evaluation over 100,000 real-life time series, outperforming ARIMA and ETS forecasting methods [5.11].

Impact status	Closed
Impact date	1 Aug 2013 → 30 Jun 2020
Category of impact	Economic