Claims:CLAIMS
We claim:

1. A server system, comprising:
a communication interface;
a memory comprising executable instructions; and
a processor communicably coupled to the communication interface and the memory, the processor configured to cause the server system to perform at least in part to:
obtain a set of time-series data associated with an entity from one or more data sources,
analyze the set of time-series data to identify time-series characteristics of the set of time-series data,
input the time-series characteristics of the set of time-series data to a deep neural network associated with a deep reinforcement learning model,
determine at least one sub-optimal candidate ensemble subset of trained time-series forecasting models by the deep neural network,
evaluate the at least one sub-optimal candidate ensemble subset of the trained time-series forecasting models to generate an optimal ensemble subset of the trained time-series forecasting models for the set of time-series data based, at least in part, on a tree search model, and
apply the optimal ensemble subset of the trained time-series forecasting models to the set of time-series data to predict future sample values of the set of time-series data.

2. The server system as claimed in claim 1, wherein the tree search model is implemented based on a Monte Carlo tree search (MCTS) model.

3. The server system as claimed in claim 2, wherein the deep neural network is trained by integration of the deep reinforcement learning model and the MCTS model, and wherein the deep reinforcement learning model is trained based, at least in part, on historical time-series data associated with the entity.

4. The server system as claimed in claim 3, wherein, to determine the at least one sub-optimal candidate ensemble subset of the trained time-series forecasting models, the server system is further caused at least in part to:
determine a state of the deep reinforcement learning model based, at least in part, on selection of an initial ensemble subset of the trained time-series forecasting models;
set an action space of the deep reinforcement learning model, the action space comprising addition of at least one time-series forecasting model in the initial ensemble subset of the trained time-series forecasting model, and/or change in hyper-parameter values of time-series forecasting models included in the initial ensemble subset of the trained time-series forecasting models;
calculate Q-values corresponding to state-action pairs formed by the state and actions based on the deep neural network; and
select an action based, at least in part, on the calculated Q-values to determine the at least one sub-optimal candidate ensemble subset of the trained time-series forecasting model.

5. The server system as claimed in claim 4, wherein the server system is further caused at least in part to select the initial ensemble subset of the trained time-series forecasting models from all possible ensemble subsets of the trained time-series forecasting models based, at least in part, on the time-series characteristics of the set of time-series data.

6. The server system as claimed in claim 5, wherein the initial ensemble subset of the trained time-series forecasting models includes a time-series forecasting model with dual LSTM based neural network architecture when the time-series characteristics of the set of time-series data indicates intermittency greater than a threshold condition.

7. The server system as claimed in claim 3, wherein for training the deep reinforcement learning model and the MCTS model, the server system is caused to at least:
access the historical time-series data associated with the entity from the one or more data sources;
define an initial state of the deep reinforcement learning model by selecting a set of time-series forecasting models with a particular ensemble method;
define an action space of the deep reinforcement learning model, the action space comprising the set of time-series forecasting models with all possible hyper-parameter values;
sample actions for simulations based on a predetermined forecasting criterion using the MCTS model;
simulate the sampled actions using the MCTS model for training the deep neural network;
calculate reward values corresponding to the sampled actions based, at least in part, on a reward function;
identify sub-optimal actions to train the deep neural network that leads to a positive reward value, the trained deep neural network configured to determine sub-optimal candidate ensemble subsets of the trained time-series forecasting models for predicting future values of a particular time-series data; and
optimize the sub-optimal actions by the MCTS model to determine the optimal ensemble subset of the trained time-series forecasting models.

8. The server system as claimed in claim 7, wherein a depth of the MCTS model is truncated by a computational constraint.

9. The server system as claimed in claim 7, wherein a reward function is based on a root mean square error (RMSE) gain.

10. A computer-implemented method, comprising:
obtaining, by a server system, a set of time-series data associated with an entity from one or more data sources;
analyzing, by the server system, the set of time-series data to identify time-series characteristics of the set of time-series data;
inputting, by the server system, the time-series characteristics of the set of time-series data to a deep neural network associated with a deep reinforcement learning model;
determining, by the server system, at least one sub-optimal candidate ensemble subset of trained time-series forecasting models by the deep neural network;
evaluating, by the server system, the at least one sub-optimal candidate ensemble subsets of the trained time-series forecasting models to generate an optimal ensemble subset of the trained time-series forecasting models for the set of time-series data based, at least in part, on a tree search model; and
applying, by the server system, the optimal ensemble subset of the trained time-series forecasting models to the set of time-series data to predict future sample values of the set of time-series data.
, Description:
FORM 2
THE PATENTS ACT 1970
(39 of 1970)
&
The Patent Rules 2003
COMPLETE SPECIFICATION
(refer section 10 & rule 13)

TITLE OF THE INVENTION:
ARTIFICIAL INTELLIGENCE BASED METHODS AND SYSTEMS FOR AUTOMATED TIME-SERIES FORECASTING MODELS

APPLICANT(S):

Name:

Nationality:

Address:

MASTERCARD INTERNATIONAL INCORPORATED

United States of America

2000 Purchase Street, Purchase, NY 10577, United States of America

PREAMBLE TO THE DESCRIPTION

The following specification particularly describes the invention and the manner in which it is to be performed.

DESCRIPTION
(See next page)

ARTIFICIAL INTELLIGENCE BASED METHODS AND SYSTEMS FOR AUTOMATED TIME-SERIES FORECASTING MODELS

TECHNICAL FIELD
The present disclosure relates to artificial intelligence processing systems and, more particularly to, electronic methods and complex processing systems for determining an optimal ensemble set of time-series forecasting models for analyzing time-series data of an entity.

BACKGROUND
Automated time-series forecasting is an important step in many industries especially in retail and financial services where a large number of time-series patterns need to be predicted on a weekly or monthly basis for financial planning and analysis. There are many models or techniques available for predicting time-series data. A forecasting model or technique that is appropriate for one type of time-series data might not be the best choice for another. Therefore, it is vital to select and use a suitable time-series forecasting model when evaluating time-series data to make predictions for future values. For example, conventional time-series forecasting models (such as, ARIMA, exponential smoothing, Vanilla-LSTM) may not give an accurate forecast for the intermittent time-series data due to the many zero values in the intermittent time-series data.
In current approaches, forecasts are generally made using a number of time-series forecasting models and the best time-series forecasting model is selected based on the error on a validation dataset. However, due to a very large search space of hyper-parameters in the time-series forecasting models, it is a very tedious task to perform a complete search by various combinations of time-series forecasting models with varying hyper-parameters. Consequently, the time-series forecasting models end up being sub-optimal for the time-series data.
Additionally, in the current approaches, a large number of time-series forecasting models are evaluated and the best time-series forecasting model is selected based on a validation period. However, due to the large number of time-series forecasting models, it is impossible to evaluate all possible ensembles of time-series forecasting models as these are computationally costly to evaluate.
Thus, there exists a need for a technical solution for determining an optimal time-series forecasting model for the time-series data with better prediction accuracy.

SUMMARY
Various embodiments of the present disclosure provide systems, methods and electronic devices for determining the optimal ensemble set of time-series forecasting models for analyzing time-series data.
In an embodiment, a server system is disclosed. The server system includes a communication interface, a memory including executable instructions and a processor communicably coupled to the communication interface and the memory. The processor configured to cause the server system to at least in part obtain a set of time-series data associated with an entity from one or more data sources. The server system is further caused to analyze the set of time-series data to identify time-series characteristics of the set of time-series data. The server system is further caused to input the time-series characteristics of the set of time-series data to a deep neural network associated with a deep reinforcement learning model. The server system is further caused to determine at least one sub-optimal candidate ensemble subset of trained time-series forecasting models by the deep neural network. The server system is further caused to evaluate the at least one sub-optimal candidate ensemble subset of the trained time-series forecasting models to generate an optimal ensemble subset of the trained time-series forecasting models for the set of time-series data based, at least in part, on a tree search model. The server system is further caused to apply the optimal ensemble subset of the trained time-series forecasting models to the set of time-series data to predict future sample values of the set of time-series data.
In another embodiment, a computer-implemented method is disclosed. The computer-implemented method performed by a server system includes obtaining a set of time-series data associated with an entity from one or more data sources. The computer-implemented method includes obtaining a set of time-series data associated with an entity from one or more data sources. The computer-implemented method further includes analyzing the set of time-series data to identify time-series characteristics of the set of time-series data. The computer-implemented method includes inputting the time-series characteristics of the set of time-series data to a deep neural network associated with a deep reinforcement learning model. The computer-implemented method includes determining at least one sub-optimal candidate ensemble subset of trained time-series forecasting models by the deep neural network. The computer-implemented method includes evaluating the at least one sub-optimal candidate ensemble subset of the trained time-series forecasting models to generate an optimal ensemble subset of the trained time-series forecasting models for the set of time-series data based, at least in part, on a tree search model. The computer-implemented method includes applying the optimal ensemble subset of the trained time-series forecasting models to the set of time-series data to predict future sample values of the set of time-series data.

BRIEF DESCRIPTION OF THE FIGURES
For a more complete understanding of example embodiments of the present technology, reference is now made to the following descriptions taken in connection with the accompanying drawings in which:
FIG. 1 is an example representation of an environment, related to at least some example embodiments of the present disclosure;
FIG. 2 is a simplified block diagram of a server system, in accordance with one embodiment of the present disclosure;
FIG. 3 is a block diagram representation of a deep reinforcement learning (DRL) model, in accordance with an example embodiment of the present disclosure;
FIG. 4 is a block diagram representation of a deep neural network of the deep reinforcement learning (DRL) model along with Monte-Carlo tree search (MCTS) agent, in accordance with an example embodiment of the present disclosure;
FIG. 5 is a flow chart for training the deep reinforcement learning model and the MCTS agent, in accordance with an example embodiment of the present disclosure;
FIG. 6 is a flow chart for predicting an optimal ensemble subset of time-series forecasting models for a given time-series data, in accordance with an example embodiment of the present disclosure;
FIG. 7 represents a simplified block diagram of dual Long Short Term Memory (LSTM) based neural network architecture for forecasting intermittent time-series data, in accordance with an embodiment of the present disclosure; and
FIG. 8 is a simplified block diagram of a server, in accordance with one embodiment of the present disclosure.
The drawings referred to in this description are not to be understood as being drawn to scale except if specifically noted, and such drawings are only exemplary in nature.

DETAILED DESCRIPTION
In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be apparent, however, to one skilled in the art that the present disclosure can be practiced without these specific details.
Reference in this specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. The appearance of the phrase “in an embodiment” in various places in the specification is not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Moreover, various features are described which may be exhibited by some embodiments and not by others. Similarly, various requirements are described which may be requirements for some embodiments but not for other embodiments.
Moreover, although the following description contains many specifics for the purposes of illustration, anyone skilled in the art will appreciate that many variations and/or alterations to said details are within the scope of the present disclosure. Similarly, although many of the features of the present disclosure are described in terms of each other, or in conjunction with each other, one skilled in the art will appreciate that many of these features can be provided independently of other features. Accordingly, this description of the present disclosure is set forth without any loss of generality to, and without imposing limitations upon, the present disclosure.
OVERVIEW
Automated time-series forecasting is an important problem in many industries especially in retail and financial services. A large number of time-series data may have to be predicted on a weekly or monthly basis. Currently, the approaches usually forecast future sample values using a number of time-series forecasting models and the best one is selected based on the error on a validation dataset. However, due to a very large search space of hyper-parameters in these time-series forecasting models, it is not possible to do a complete search and the models end up being sub-optimal. Additionally, when the dataset is huge and the number of time-series forecasting models is more, trying out all the time-series forecasting models with all possible ensemble methods on the huge dataset is practically impossible.
In view of the foregoing, various example embodiments of the present disclosure provide methods, systems, user devices, and computer program products for selecting a time-series forecasting model or an optimal ensemble set of time-series forecasting models for analyzing time-series data. The time-series data may be received from data sources such as a plurality of servers. A plurality of time-series forecasting models may be utilized, out of which an optimal ensemble subset of time-series forecasting models may be used to forecast the future sample values.
In an example, the present disclosure describes a server system that is configured to train a deep reinforcement learning model and a Monte-Carlo tree search (MCTS) agent to predict future sample values using an optimal ensemble subset of time-series forecasting models. The server system includes at least a processor and a memory. The server system may obtain a set of time-series data form one or more data sources (for example, datacenters, time-series data repository, etc.). The server system is configured to analyze the set of time-series to identify time-series of the set of time-series data. The time-series characteristics may be defined in terms of seasonality, seasonal strength, number of peaks, length of time-series data, bias/level, functional trend (for example, multiplicative or addictive), outlier detection, etc.
In one embodiment, the server system is configured to access trained time-series forecasting models from a database. The trained time-series forecasting models are trained based on various historical time-series data.
During execution phase, the server system is configured to input the time-series characteristics of the set of time-series data to a deep neural network associated with a deep reinforcement learning (DRL) model. The deep neural network is trained by integration of the DRL model and the MCTS model. The DRL model is trained based, at least in part, on historical time-series data associated with the entity.
The DRL model is configured to determine at least one sub-optimal candidate ensemble subset of the trained time-series forecasting models by the deep neural network. In particular, the server system is configured to determine a state of the deep reinforcement learning model based, at least in part, on an initial ensemble subset of the trained time-series forecasting models. The initial ensemble subset of the trained time-series forecasting models is selected from all possible ensemble subsets of the trained time-series forecasting models. In one embodiment, the initial ensemble subset of the trained time-series forecasting models includes at least one time-series forecasting model with dual LSTM based neural network architecture when the time-series characteristics of the set of time-series data indicate intermittency greater than a threshold condition.
The server system is configured to set an action space of the DRL model, where the action space includes addition of at one time-series forecasting model in the initial ensemble subset of the trained time-series forecasting models, change in hyper-parameter values of time-series forecasting models in the initial ensemble subset, change ensemble method, etc. Thereafter, the server system is configured to calculate or predict Q-values corresponding to state-action pairs formed by the state and actions using the deep neural network. The server system is configured to select an action based on the predicted Q-values and epsilon greedy policy methods to determine the at least one sub-optimal candidate ensemble subset of the trained time-series forecasting model.
In particular, the server system is configured to calculate a reward value corresponding to the selected action based, at least in part, on a reward function. The reward function is based on a discounted Root Mean Squared Error (RMSE) that is derived by adding/removing a time-series forecasting model to the initial subset of time-series forecasting models, or by changing ensemble method. The ensemble subsets of time-series forecasting model having Q-values higher than a threshold value are determines as sub-optimal candidate ensemble subsets of time-series forecasting models.
In one embodiment, the server system is configured to evaluate the at least one sub-optimal candidate ensemble subset of the trained forecasting models based, at least in part, on a tree search model. The tree search model is implemented using a Monte-Carlo tree search (MCTS) model. The MCTS model is configured to determine an optimal ensemble subset of the trained time-series forecasting models.
Once the optimal ensemble subset of the trained time-series forecasting models is determined for the set of time-series data, the server system is configure to apply the optimal ensemble subset of the trained time-series forecasting models to the set of time-series data to predict future sample values of the set of time-series data.
In one embodiment, the deep reinforcement learning model is trained based, at least in part, on historical time-series data associated with the entity.
Various embodiments of the present disclosure offer multiple advantages and technical effects. For instance, the present disclosure provides an automated system for determining an optimal ensemble set of time-series forecasting models for analyzing time-series data. The present disclosure provides methods for intelligently picking the hyper parameters of a time-series forecasting model and reducing the number of iterations required to find the best time-series forecasting models, thereby reducing the search space for searching the hyper parameters by drastically and improving prediction accuracy. Furthermore, the present disclosure provides a significantly more robust solutions because of handling simultaneous/concurrent processor execution (such as applying an optimal ensemble subset of time-series forecasting models over the same input time-series data, simultaneously).
Additionally, the present disclosure improves accuracy on time-series where there is a lot of intermittent behavior in the time-series data by applying a dual LSTM neural network based architecture over the time-series data.
Various example embodiments of the present disclosure are described hereinafter with reference to FIGS. 1 to 8.
FIG. 1 illustrates an exemplary representation of an environment 100 related to at least some example embodiments of the present disclosure. Although the environment 100 is presented in one arrangement, other embodiments may include the parts of the environment 100 (or other parts) arranged otherwise depending on, for example, determining an optimal ensemble set of time-series forecasting models for a particular time-series data, etc. The environment 100 generally includes a plurality of entities, for example, an automated time-series forecasting ("ATSF") server 102, a plurality of servers 108a, 108b, 108c (collectively represented as server 108), and time-series data repositories 110, each coupled to, and in communication with (and/or with access to) a network 106. The network 106 may include, without limitation, a light fidelity (Li-Fi) network, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a satellite network, the Internet, a fiber optic network, a coaxial cable network, an infrared (IR) network, a radio frequency (RF) network, a virtual network, and/or another suitable public and/or private network capable of supporting communication among the entities illustrated in FIG. 1, or any combination thereof.
Various entities in the environment 100 may connect to the network 106 in accordance with various wired and wireless communication protocols, such as Transmission Control Protocol and Internet Protocol (TCP/IP), User Datagram Protocol (UDP), 2nd Generation (2G), 3rd Generation (3G), 4th Generation (4G), 5th Generation (5G) communication protocols, Long Term Evolution (LTE) communication protocols, or any combination thereof. For example, the network 106 may include multiple different networks, such as a private network made accessible by the plurality of servers 108a-108c, separately, and a public network (e.g., the Internet etc.)./
The ATSF server 102 (interchangeably referred as “server system” throughout the description) is configured to perform one or more of the operations described herein. In general, the server system 102 is configured to determine the best ensemble subset of time-series forecasting models for time-series data of an item. The server system 102 is a separate part of the environment 100, and may operate apart from (but still in communication with, for example, via the network 106) the plurality of servers 108a, 108b and 108c, the data repositories 110, and any third party external servers (to access data to perform the various operations described herein). However, in other embodiments, the server system 102 may actually be incorporated, in whole or in part, into one or more parts of the environment 100, for example, the server 108a. In addition, the server system 102 should be understood to be embodied in at least one computing device in communication with the network 106, which may be specifically configured, via executable instructions, to perform as described herein, and/or embodied in at least one non-transitory computer-readable media.
In one embodiment, the ATSF server 102 is configured to receive datasets from multiple time-series data sources including, for example, time-series data repositories 110, the plurality of servers 108a-108c (for example, application server, web server, media server, etc.), and other suitable data sources not shown in FIG. 1. Alternatively or in addition, the ATSF server 102 can be directly coupled to time-series data sources, as opposed to, via the network 106. In some other implementations, the ATSF server 102 can include a time series monitoring system (not shown in FIG. 1) to capture time-series data points.
In an example, the time-series data repositories 110 may store time-series data having data points associated with a feature of an item or an article. The time-series data repositories 110 may implement relational and non-relational databases, big data stations, file systems, and other suitable applications storing datasets with time-series data points and other relevant data affecting a time-series.
In one embodiment, the ATSF server 102 is in communication with an ATSF database 104. The ATSF database 104 may store trained models and algorithms required for the ATSF server 102 to perform one or more of the operations described herein, for example, to determine the best ensemble subset of time-series forecasting models for time-series data of an item. The ATSF database 104 may also store a plurality of trained time-series forecasting models. The ATSF database 104 may be incorporated in the ATSF server 102, or may be an individual entity connected to the ATSF server 102, or may be a database stored in a cloud storage.
In some other instances, each time-series observation or sample can include a vector of values; these types of time-series data are known as multivariate time series. In some cases, time-series data is uniformly spaced over time according to a constant scale or spaced time interval e.g., year, month, day, hour, second, and so forth. Examples of uniformly spaced time-series data include, but not limited to, monthly indices of industrial production of a given country, annual per capita gross domestic product for a group of countries, daily sales of a given product, and other suitable types of uniformly spaced time-series. In some other cases, time-series data points are not uniformly spaced over time but instead, are distributed unevenly depending on when an event occurs. These types of time-series are generally called uniformly spaced time series. Examples of uniformly spaced time-series data include time-series describing: traffic incidents on highways, weather condition, natural disasters such as earthquakes and floods, and other suitable types of unevenly spaced time series. In one example, intermittent time-series data may contain several periods of zero value. Examples of the intermittent time-series data include time-series describing spend transactions of a payment card, retail sale data, aviation data, etc. In case of the spend transactions of the payment card, intermittency occurs near to the end of payment card’s lifecycle.
In one embodiment, the ATSF server 102 is configured to determine the best ensemble subset of time-series forecasting models by utilizing reinforcement meta learning algorithm which efficiently searches model parameters and the best ensemble subset of time-series forecasting models using Monte-Carlo Tree Search (MCTS) model.
The number and arrangement of systems, devices, and/or networks shown in FIG. 1 are provided as an example. There may be additional systems, devices, and/or networks; fewer systems, devices, and/or networks; different systems, devices, and/or networks; and/or differently arranged systems, devices, and/or networks than those shown in FIG. 1. Furthermore, two or more systems or devices shown in FIG. 1 may be implemented within a single system or device, or a single system or device shown in FIG. 1 may be implemented as multiple, distributed systems or devices. Additionally, or alternatively, a set of systems (e.g., one or more systems) or a set of devices (e.g., one or more devices) of the environment 100 may perform one or more functions described as being performed by another set of systems or another set of devices of the environment 100.
FIG. 2 is a simplified block diagram of a server system 200, in accordance with an embodiment of the present disclosure. The server system 200 is similar to the server system 102. In some embodiments, the server system 200 is embodied as a cloud-based and/or SaaS-based (software as a service) architecture.
The server system 200 includes a computer system 202 and a database 204. The computer system 202 includes at least one processor 206 for executing instructions, a memory 208, a communication interface 210, and a user interface 216 that communicate with each other via a bus 212.
In some embodiments, the database 204 is integrated within computer system 202. For example, the computer system 202 may include one or more hard disk drives as the database 204. A storage interface 214 is any component capable of providing the processor 206 with access to the database 204. The storage interface 214 may include, for example, an Advanced Technology Attachment (ATA) adapter, a Serial ATA (SATA) adapter, a Small Computer System Interface (SCSI) adapter, a RAID controller, a SAN adapter, a network adapter, and/or any component providing the processor 206 with access to the database 204. In one embodiment, the database 204 is configured to store a plurality of trained time-series forecasting models 232.
Examples of the processor 206 include, but are not limited to, an application-specific integrated circuit (ASIC) processor, a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a field-programmable gate array (FPGA), and the like. The memory 208 includes suitable logic, circuitry, and/or interfaces to store a set of computer-readable instructions for performing operations. Examples of the memory 208 include a random-access memory (RAM), a read-only memory (ROM), a removable storage drive, a hard disk drive (HDD), and the like. It will be apparent to a person skilled in the art that the scope of the disclosure is not limited to realizing the memory 208 in the server system 200, as described herein. In another embodiment, the memory 208 may be realized in the form of a database server or a cloud storage working in conjunction with the server system 200, without departing from the scope of the present disclosure.
The processor 206 is operatively coupled to the communication interface 210 such that the processor 206 is capable of communicating with a remote device 218 such as, the plurality of servers 108a-108c, or communicated with any entity connected to the network 106 (as shown in FIG. 1).
It is noted that the server system 200 as illustrated and hereinafter described is merely illustrative of an apparatus that could benefit from embodiments of the present disclosure and, therefore, should not be taken to limit the scope of the present disclosure. It is noted that the server system 200 may include fewer or more components than those depicted in FIG. 2.
In one embodiment, the processor 206 includes a data pre-processing engine 220, a feature extraction engine 222, a model training engine 224, a deep reinforcement learning (DRL) agent 226, a Monte Carlo Tree Search (MCTS) agent 228, and a prediction engine 230. It should be noted that components, described herein, can be configured in a variety of ways, including electronic circuitries, digital arithmetic and logic blocks, and memory systems in combination with software, firmware, and embedded technologies.
The data pre-processing engine 220 includes suitable logic and/or interfaces for receiving a set of time-series data associated with an entity from one or more data sources such as the plurality of servers 108a-108c. Examples of the time-series data may include, but not limited to, financial indicator values of one or more items, weather data, etc. In some implementations, the data pre-processing engine 220 executes one or more preprocessing operations on the received time-series data. Examples of pre-processing operations performed by the data pre-processing engine 220 include normalization operations, splitting of datasets, merging of datasets, and other suitable preprocessing operations.
The feature extraction engine 222 includes suitable logic and/or interfaces for analyzing the set of time-series data to determine which time-series characteristics the time-series data exhibits. The time-series characteristics may include, but not limited to, statistical properties such as mean, standard deviation, range and skewedness. The time-series characteristics could also include seasonality, seasonal strength, number of peaks, length of time-series data, bias/level, functional trend (for example, multiplicative or addictive), outlier detection, etc. In general, identifying certain time-series characteristics may be complex and require a fair degree of analysis. For example, the seasonality can be detected by using the Fourier transform of autocorrelation. The functional trend can be detected through regression analysis methods.
In some embodiments, the feature extraction engine 222 may determine time-series characteristics of multiple seasonality for instance, weekly and annual seasonality. In such a case, a set of forecasting models that are constrained to handle only one seasonality can be trained and tested multiple times, once for each of the determined seasonality characteristics.
The model training engine 224 includes suitable logic and/or interfaces for training a plurality of time-forecasting models using training datasets associated with the entity. The plurality of time-forecasting models may include, but not limited to, Autoregressive Moving Average (ARMA) model, Exponential smoothing (EM) model, nonlinear Regression Models, non-regression based models, and non-parametric regression based models, etc. The model training engine 224 may be configured to continually update the time-series forecasting models 232 based on the forecasts predicted by the time-series forecasting models 232.
The DRL agent 226 includes suitable logic and/or interfaces for determining sub-optimal candidate ensemble subsets of the trained time-series forecasting models that are used for predicting future values of the time-series data. The DRL agent 226 is configured to implement a deep reinforcement learning (DRL) model with a deep neural network. In one embodiment, the DRL agent 226 is configured to train the DRL model based on historical time-series data of the entity. The policy space of the DRL model may include the plurality of time-series forecasting models with all possible hyper-parameters as well as all possible transformations available in the time-series data.
In order to express the use of reinforcement learning in determining the best ensemble set of time-series series forecasting models for prediction, the present disclosure explains theoretical models of deep Q learning, the Markov Decision Process (MDP) with reference to FIG. 3 in more detail. It would be apparent to those skilled in the art that several of deep reinforcement learning models may be applied to accomplish the spirit of the present disclosure. In general, the goal of the DRL model is to learn a policy (control strategy) that maximizes the expected return (cumulative, discounted reward).
The MCTS agent 228 includes suitable logic and/or interfaces for determining the optimal ensemble subset of the trained time-series forecasting models from the various sub-optimal candidate ensemble subsets of the trained time-series forecasting models based on Monte-Carlo tree search (MCTS) model.
In one embodiment, the DRL agent 226 is configured to train the deep neural network in an off-line manner using the DRL model and MCTS model. The state information (i.e., which is defined based on the time-series characteristics and initial ensemble subset of time-series forecasting models) is input into the DRL agent 226 in real-time. The deep neural network of the DRL agent 226 is trained by repeating episodes of start-to-finish simulation. The DRL agent 226 simulates some actions i.e., apply an ensemble subset of time-series forecasting models on the time-series data, and, upon observing the consequences of its actions, learns to alter its own behavior in response to rewards (i.e., RMSE error) received. The DRL agent 226 is configured to observe a state S(t) at time step t and transition to a new state S(t+1) by taking an action A(t) in the state S(t). The MCTS agent 228 is used to speed up the offline training process. The deep neural network may generate sub-optimal actions since the policy space for the DRL model is too large.
The term "sub-optimal" herein refers to actions that may not be the ideal or optimal ensemble subset of the time-series forecasting model for the time-series data. In an analogy for game theory, the sub-optimal actions may represent one or more possible moves that have been identified as "good" or likely to lead to a winning outcome. Because of the uncertainty in the system and the vast number of possibilities, the DRL model may be unable to identify a single "optimal" action. However, based on prior simulations (or records of historical time-series data), the DRL model is configured to identify the possible next steps for the current state that will lead to an efficient or beneficial outcome. The sub-optimal actions are input below into the MCTS agent 228 to refine the determination and identify an "optimal" ensemble subset of the time-series forecasting models.
The sub-optimal actions are fed into the MCTS agent 228. The MCTS agent 228 speeds up the search for near optimal polices in offline training phase. The MCTS agent 228 balances exploration and exploitation based on the available computation resources (for example, CPU time). In real-time, the MCTS agent 228 performs continuous rollout utilizing the time-series characteristics of the time-series data etc. The rollout continues to search for an optimal ensemble subset of the time-series forecasting models. In other words, the MCTS agent 228 samples actions on the sub-optimal ensemble subsets of the time-series forecasting models to determine the optimal ensemble subsets of the time-series forecasting models for the particular time-series data.
The main objective of the DRL agent 226 and the MCTS agent 228 is to find the best ensemble subset of the time-series forecasting models with the least RMSE error, thus improving the prediction accuracy.
The prediction engine 230 includes suitable logic and/or interfaces for applying the optimal ensemble subset of the time-series forecasting models over the time-series data to predict the future samples of the time-series data. Once the optimal ensemble subset of the time-series forecasting models is determined by the MCTS agent 228, the optimal ensemble subset of the time-series forecasting models is provided to the prediction engine 230. The prediction engine 230 may apply the optimal ensemble subset of the trained time-series forecasting model to the time-series data and determine the corresponding future values based on the optimal ensemble subset of the trained time-series forecasting models.
FIG. 3 is a block diagram representation of a deep reinforcement learning (DRL) model 300, in accordance with an embodiment of the present disclosure. As shown in the FIG. 3, the DRL model 300 involves two entities, i.e., an agent 302 (similar to the DRL agent 226) and an environment 304, that interact with each other. The agent 302 is an entity that makes decision of selection of an ensemble subset of time-series forecasting models, and the environment 304 may be set to feedback a reward value depending upon the root mean square error (RMSE) after an inclusion or exclusion of a time-series forecasting model into the ensemble subset of time-series forecasting models. The DRL model 300 implements Markov Decision Process (MDP). The MDP may be represented by a five-tuple , where,
1) S is a State Space, which includes a set of environmental states that the agent 302 may perceive. Herein, at any time t, a state of the DRL model refers a state of an ensemble subset of the trained time-series forecasting models selected for the time-series data.
2) A is an Action Space, which includes a set of actions that the agent 302 may take on each state of the environment. Herein, the set of actions includes addition or exclusion of a particular time-series forecasting models of a plurality of time-series forecasting models to an ensemble subset for predictions with all possible hyper-parameters as well as possible transformations available on the time-series.
3) R is a reward function and R(s, a, s’) represents a reward that the agent 302 obtains from the environment 304 when the action ‘a’ is performed on the state s and the state is changed to state s’. Herein, the reward function consists of discounted root mean square error (RMSE) gain by performing an action.
4) T is a state transition function and T(s, a, s’) may represent a probability of executing action ‘a’ on state ‘s’ and moving to state s'.
5) ? is a constant factor where ?=0.95, based on which the reward is calculated.
Further, the MDP methods may also require following parameters to define:
6) Episode: One episode represents application of a particular ensemble subset of the time-series forecasting models over the time-series data.
7) Policy Space: A stochastic policy outputs the possibility of an action for every time step t. Herein, the policy space of the DRL model may include the plurality of time-series forecasting models with all possible hyper-parameters as well as all possible transformations available in the time-series data.
In the process of interaction between the agent 302 and the environment 304 in the MDP, the agent 302 senses that the environment state at time t is 'st'. Based on the environment state 'st', the agent 302 may select an action 'at' from the action space A to execute. After the environment 304 receives the action selected by the agent 302, it returns corresponding reward signal feedback Rt+1 to the agent 302 and transfers to new environment state st+1, and waits for the agent 302 to make a new decision. In the process of interacting with the environment 304, the goal of the agent 302 is to find a plurality of sub-optimal policies based on the reward function by finding the ensemble subset of time-series forecasting models whose RMSE value is the least.
In one embodiment, the policy space may include different types or various combinations of time-series forecasting models as shown below in the table (1).
Type Variations
Homogeneous Different execution of time-series forecasting models using different hyper-parameters
Different execution of time-series forecasting models using different optimization criteria (e.g., MAE, MAPE, AIC, etc.)
Different execution of time-series forecasting models with external factors, and using additive versus multiplicative decompositions
Heterogeneous Different type of time-series forecasting models on the same dataset
Neural vs Statistical approaches
Hybrid approaches (ES-RNN)
Spectral approaches (NBEATS)
Table 1
The total reward is also called as Q-value denoted using the following equation:
Q(s,a)=r(s,a)+? max??Q(s^',a)?….Eqn. (1)
The above equation states that the Q-value yielded from being at state s and performing action a is equal to the immediate reward r(s, a) plus the highest Q-value possible from the next state s', and Y is a discount factor which controls the contribution of rewards further in the future. In other words, the Q(s, a) is a cumulative reward value of rewards generated in the subsequent learning optimization when the agent 302 executes the action 'a' in the state 's'.
Further, in the deep reinforcement learning model 300, a neural network architecture is utilized to approximate Q value-function. The state is given as the input and the Q-values of all possible actions are generated as the output.
Based on the above deep reinforcement learning model 300, the server system 200 provided by the present disclosure extracts time-series data features from the time-series data received from a plurality of data sources and predicts an ensemble subset of time-series forecasting models which would be best suitable for predictions of the corresponding time-series data. Then, the server system 200 iteratively updates the policy space by using the deep reinforcement learning model 300, to finally learn the optimal ensemble prediction strategy using the MCTS agent (not shown) step by step.
FIG. 4 is a block diagram representation 400 of a deep neural network 402 of the deep reinforcement learning (DRL) model along with an MCTS agent 408, in accordance with an example embodiment of the present disclosure.
As mentioned above, in deep reinforcement learning, in the process of interacting with the environment, the goal of the agent 302 is to find an optimal strategy such that it receives the biggest cumulative reward in any state s and any time step t. In some example embodiments, the above objective may be achieved using a Q-value function approximation algorithm. In other example embodiments, the foregoing objectives may also be implemented by using other deep reinforcement learning algorithms such as a strategy approximation algorithm, which is not limited herein.
In one embodiment, the deep reinforcement learning model may include one or more neural networks. As shown in the FIG. 4, the deep neural network 402 includes an input layer, multiple hidden layers, and an output layer. In one example, the deep neural network 402 may include LSTM based neural network architecture to determine the best ensemble subset of time-series forecasting models for predictions of a particular time-series data.
More illustratively, during the training, the DRL agent runs a plurality of simulations by selecting ensemble subsets of time-series forecasting models that are to be applied over the time-series data, randomly, and identifies reward values corresponding to the plurality of simulations using a reward function.
The deep neural network 402 is utilized to approximate the Q-value function. The MDP in the deep reinforcement learning model includes the state space S and the action space A, wherein an ensemble subset of time-series forecasting models and the time-series data characteristics of the time-series data belong to the current state, and addition or exclusion of one or more new time-series forecasting models to the ensemble subset belongs to the action space A.
Since finding the best ensemble set of time-series forecasting models for a particular time-series is difficult and exact solutions and subject to a very large search space of hyper-parameters in the time-series forecasting models are nearly impossible to find, the server system 200 is configured to utilize a deep reinforcement learning along with Monte-Carlo tree search (MCTS) model.
In one example, the problem of finding the best ensemble set of time-series forecasting models with the least RMSE error can be expressed as a pruning of an ensemble of different time-series forecasting models in the following manner:
F={F1, F2, F3,….}…..Eqn. (2)
,where F represents the all possible combinations of ensemble subsets of time-series forecasting models with different hyper-parameters.
In one embodiment, states of the DRL model are defined based on a current ensemble subset of F and a particular time-series forecasting model (fi) under evaluation by the MCTS model. The MCTS model provides an efficient and quick real time search to identify the optimal action from the two possible actions (e.g., A=?¦?(include(f_i ),exclude(f_i))?).
In one embodiment, the input to the deep neural network 402 is a current state 404 determined based on a selection of an initial ensemble subset of the trained time-series forecasting models and time-series characteristics of the time-series data. In some embodiments, the initial ensemble subset of the trained time-series forecasting model must include a time-series forecasting model with the dual LSTM based neural network architecture (as shown in the FIG. 7) when the time-series characteristics of the time-series data indicates high variability or intermittency greater than a threshold condition. The current state 404 is changed when an action such as addition of a new time-series forecasting model or exclusion of an existing time-series forecasting model to the initial ensemble subset is selected. The reward value may be calculated based on an infinite horizon discounted model. In one embodiment, the reward value may be based on multiple different values and may be determined using an algorithm that weighs different values differently. The reward function (see, Eqn. (3)) may reflect inverse RMSE values after an inclusion or exclusion of a time-series forecasting model within the initial ensemble subset of the time-series forecasting models.
R_t=r_(t+1)+?r_(t+2)+?= ?_(k=0)^8¦??^k r_(t+k+1) ?…….Eqn. (3)
The DRL model provides learning upon observing the consequences of selected actions and alters its own behavior in response to rewards received. The DRL agent including the deep neural network 402 receives a state s(t) at timestep t and performs an action at state s(t). When the DRL agent selects the action, the state gets transitioned to a new state s(t + 1) based on the current state and the chosen action. The new state s(t + 1) is returned to the agent to await a further action. The state is a sufficient statistic of the simulation and includes information for the DRL agent to suggest an optimal action at that time. The optimal action may not end up leading to the optimal reward due to the complexity of the large search space in the plurality of time-series forecasting models.
In particular, the processor 206 is configured to determine RMSE values under various types of ensemble methods (such as, Mean, Median, Ordinary Least Square (OLS), LAD (Least Angle Deviation), BATES-GRANGER approach, constrained least squares, etc.) and identify the ensemble method with minimum RMSE gain that is then appropriately discounted to get the reward value.
In one embodiment, the processor 206 is configured to use more than one technique to evaluate forecast accuracy of time-series forecasting models or to define the reward function. Further forecast accuracy techniques include computing mean absolute percentage errors (MAPE), mean absolute scaled errors (MASE), normalized root mean square error (NRMSE), coefficient of variation (CV), mean of forecasted values (MFV), standard deviation of forecasted values (SFV), and other suitable measures for the assessment of accuracy of forecasted data points.
The output of the deep neural network 402 represents predicted Q-values (i.e., Q value-action 1 406a, Q value-action 2 406b….Q value-action n 406n) for each state-action pair. The reward function may be the root mean squared error of the predicted Q-value and the target Q-value. To the extent the predicted Q value from the deep neural network 402 differs from the target Q-value, various training techniques, (such as, back propagation, stochastic gradient descent, etc.,) may be employed to adjust various weights associated with the deep neural network 402 to reduce the loss values.
Here, the Q value-action 1 406a corresponds to a sub-optimal candidate ensemble subset of the trained time-series forecasting models with minimum RMSE error among other Q values. In one embodiment, sub-optimal candidate ensemble subsets of the time-series forecasting models are input to the MCTS agent 408. The sub-optimal candidate ensemble subsets of the time-series forecasting models may represent time-series forecasting models in form of features output by the DRL model that suggests the next step that will lead to a winning solution. The features include a low dimensionality than the input state data. The low dimensionality limits the size of the search. However, because the space is large and there are a large number of possibilities, the DRL model may be unable to identify an optimal action. The MCTS agent 408 assists the DRL model in identifying and selecting the next step. Exploration of hyper-parameters and adding or exclusion of time-series forecasting models is handled through the MCTS agent 408 which randomly samples the hyper-parameter space on a particular forecasting algorithm as well as the different types of algorithms.
The MCTS agent 408 includes iteratively building a search tree until a predefined computational budget, for example, a time, memory or iteration constraint is reached. At that point, the search is halted and the best performing root action is returned. Each node in the search tree represents a state and directed links to child nodes represent actions leading to subsequent states. The MCTS agent 408 includes at least four steps that are applied at each search iteration. The steps are selection, expansion, simulation, and back-propagation. For an initial selection, the MCTS agent 408 uses the sub-optimal ensemble subsets of the time-series forecasting models provided by the DRL agent and randomly sample the hyper-parameter space as specified by an administrator on a particular time-series forecasting model as well as the different types of algorithms.
The MCTS agent 408 uses multiple iterations to estimate the value of each state in a search tree. Each node of the tree represents a state. Moving from one node to another simulates an action or actions performed by the system. At the end of the simulation, an optimal ensemble subset (see, 410) of time-series forecasting models may be identified by tracing the path along the selected nodes of the tree using back-propagation. For the MCTS algorithm, as more simulations are executed, the search tree grows larger and the relevant values become more accurate. A policy used to select actions during search is also improved over time, by selecting children with higher reward values. The policy converges to a near-optimal policy and the evaluations converge to a stable value function.
FIG. 5 represents a flow chart 500 for training the DRL model and the MCTS agent 408, in accordance with an embodiment of the present disclosure. The sequence of operations of the flow chart 500 may not be necessarily executed in the same order as they are presented. Further, one or more operations may be grouped and performed in the form of a single step, or one operation may have several sub-steps that may be performed in parallel or in a sequential manner.
At 502, the server system 200 accesses historical time-series data associated with an entity from one or more data sources such as the plurality of servers 108a-108c as training data. The historical time-series data may include weekly, monthly or yearly recorded data associated with the entity arranged in order of time. In an example, the plurality of servers 108a-108c may be payment servers, data center servers, cloud servers, etc.
At 504, the server system 200 accesses pre-trained time-series forecasting models from the database 204. For example, there are 10 different time-forecasting (TF) models M1, M2….M10 stored in the database 204. In one example, the pre-trained time series forecasting models may include Autoregression (AR), Autoregressive Moving Average (ARMA), Moving Average (MA), Seasonal Autoregressive Integrated Moving-Average (SARIMA), Seasonal Autoregressive Integrated Moving-Average with Exogenous Regressors (SARIMAX), Simple Exponential Smoothing (SES), Autoregressive Integrated Moving Average (ARIMA), Vector Autoregression (VAR), Vector Autoregression Moving-Average (VARMA), Vector Autoregression Moving-Average with Exogenous Regressors (VARMAX), Holt Winter’s Exponential Smoothing (HWES), and the like.
At 506, the server system 200 trains the DRL model along with the MCTS model based, at least in part, on time-series characteristics of the historical time-series data. The training of the DRL model is performed at steps 506a-506g.
At 506a, the server system 200 defines an initial state of the DRL model by selecting a set of time-series forecasting models (for example, M1 and M2) with a particular ensemble method, in random manner. The set of time-series forecasting models with the particular ensemble method is also called as initial ensemble subset of the time-series forecasting models and are applied over the training data. In one embodiment, the processor 206 is configured to include a time-series forecasting model with the dual LSTM based neural network architecture in the initial ensemble subset of the trained time-series forecasting models when the time-series characteristics indicate time-series data as intermittent or high variability. In other words, the processor 206 is configured to include the time-series forecasting model with dual LSTM based neural network architecture in the initial ensemble subset while defining the initial state. Detailed explanation of the dual LSTM based neural network architecture is provided with reference to FIG. 7.
At 506b, the server system 200 defines an action space of the DRL model as suggested by a policy space of the DRL model. The policy space may include the pre-trained time-series forecasting models with all possible hyper-parameters as well as all possible transformations available in the time-series data. The action space may include a plurality of actions such as, addition/exclusion of a time-series forecasting model into the initial ensemble subset, changing hyper-parameters of the set of time-series forecasting models (e.g., M1, M2), and changing ensemble method. In one embodiment, when the time-series characteristics of the time-series data indicate intermittency greater than the threshold condition, the processor 206 is configured to mask out all other invalid actions (such as, selection of ARIMA model or an ensemble subset of conventional time-series forecasting models, etc.) in the action space during the action selection process of policy gradient methods. In other words, the policy space for the intermittent time-series data with high variability may be truncated to inclusion of the time-series forecasting model with the dual LSTM based neural network architecture in the initial ensemble subset while defining the initial state.
After defining the state and action spaces, at 506c, the server system 200 samples the actions for simulation based on a predetermined forecasting criterion using the MCTS model. In general, the MCTS model balances exploration and exploitation based on the available computation resources (for example, CPU time). Exploration of hyper parameters and adding time-series forecasting model are handled through a Monte-Carlo tree search (MCTS) model which randomly samples the hyper parameter space as specified by an administrator on a particular forecasting algorithm as well as the different types of algorithms.
At 506d, the server system 200 simulates the sampled actions using the MCTS model for training the deep neural network of the DRL model.
At 506e, the server system 200 calculates reward values corresponding to the sampled actions based on a predefined reward function. The reward function is based on RMSE gain obtained after performing an action. The reward function may be described based on the state and action pair associated with the samples actions. The reward value may be denoted by a function that is defined to maximize the prediction accuracy of the time-series forecasting models. Therefore, higher the reward value for a particular state-action pair, higher would be the accuracy in prediction of that particular ensemble subset of time-series forecasting models associated with the state-action pair. For each action, the server system 200 identifies both the reward values of the MCTS evaluations and the eventual outcome. In other words, the server system 200 is configured to record immediate reward values and cumulative reward values of each action.
At 506f, the server system 200 identifies sub-optimal actions (i.e., ensemble subsets of the time-series forecasting models) to train the deep neural network 402 that leads to a positive reward value. For reinforcement learning, a feedback mechanism is used to improve the performance of the deep neural network 402. The DRL model collects reward in the Monte-Carlo tree search (MCTS) at different states, the deep neural network 402 maps these states to their corresponding values based on the reward collected. By comparing the reward values, the deep neural network 402 can decide which states are more favorable and generate policy that leads to high reward-value states. To train the deep neural network 402, the encoded input states are fed forward to generate the outputs which will in turn be compared to the target values given by the MCTS model. The errors between the generated outputs and the target values are then propagated back to update the weights of the neural networks.
At 506g, the server system 200 optimizes the sub-optimal actions by the MCTS model to identify an optimal ensemble subset of the time-series forecasting models. The MCTS model uses a tree search model to balance between exploration and exploitation of the tree based on specific domains and problems. The MCTS model provides an online search mechanism to identify which of the sub-optimal actions should be implemented. When the algorithm starts, the tree is formed only by the root node that represents the initial state. During the selection step, the tree is navigated from the root until a threshold number of computations have been reached. In every one of the action decisions, the MCTS model balances between exploitation and exploration. The MCTS model chooses between taking an action that leads to states with the best outcome found so far, and performing a move to go to less explored states, respectively. In other words, the server system 200 is configured to take greedy actions to minimize the sum of RMSE (root mean squared error) of sub-optimal time-series forecasting models, only limited by the number of computations user is willing to perform to achieve the best ensemble of time-series forecasting models. The computations should be stopped if the RMSE gain has dropped below a threshold value. Thus, a depth of the MCTS model is truncated by a computational constraint.
At the end of the simulation, an optimal ensemble set of time-series forecasting models may be identified by tracing the path along the selected nodes of the tree. For the MCTS model, as more simulations are executed, the search tree grows larger and the relevant values become more accurate. A policy used to select actions during search is also improved over time, by selecting children with higher reward values. The policy converges to a near-optimal policy and the evaluations converge to a stable value function.
In one embodiment, the server system 200 stores the optimal ensemble subset of time-series forecasting models to be applied to the time-series data in the database 204.
FIG. 6 represents a flow chart 600 for predicting an optimal ensemble subset of time-series forecasting models for a given time-series data, in accordance with an embodiment of the present disclosure. The sequence of operations of the flow chart 600 may not be necessarily executed in the same order as they are presented. Further, one or more operations may be grouped and performed in form of a single step, or one operation may have several sub-steps that may be performed in parallel or in a sequential manner.
At 602, the server system 200 obtains a set of time-series data associated with an entity from one or more data sources. The data sources may be a plurality of servers such as the servers 108a-108c.
At 604, the server system 200 analyzes the set of time-series data to identify time-series characteristics of the set of time-series data. The time-series characteristics may include, but not limited to, statistical properties such as mean, standard deviation, range and skewedness. The time-series characteristics could also include seasonality, seasonal strength, number of peaks, length of time-series data, bias/level, functional trend (for example, multiplicative or addictive), outlier detection, etc. The feature extraction engine 222 may be configured to perform feature extraction by analyzing the set of time-series data.
At 606, the server system 200 inputs the time-series characteristics of the set of time-series data to a neural network associated with a deep reinforcement learning model. The time-series characteristics of the set of time-series data identified by the feature extraction engine 222 may be provided to the deep DRL agent 226. The deep DRL agent may include a neural network configured to receive the time-series characteristics and predict ensemble subsets of time series forecasting models.
At 608, the server system 200 predicts at least one sub-optimal candidate ensemble subset of the trained time-series forecasting models by a deep neural network. The DRL agent 226 including the deep neural network predicts a plurality of sub-optimal candidate ensemble subsets of the time-series forecasting models. Each sub-optimal candidate ensemble subset may include a combination of time-series forecasting models with various hyper-parameters and an ensembling method.
At 610, the server system 200 evaluates the at least one sub-optimal candidate ensemble subset of the trained time-series forecasting models to generate or determine an optimal ensemble subset of the trained time-series forecasting models for the set of time-series data based on a tree search model. The server system 200 may utilize the MCTS agent 228 to evaluate the plurality of sub-optimal candidate ensemble subsets of the time-series forecasting models. The MCTS agent 228 is configured to generate an optimal ensemble subset of the trained time-series forecasting models for the set of time-series data, based on the evaluation.
At 612, the server system 200 applies the optimal ensemble subset of the trained time-series forecasting models to the set of time-series data to predict future sample values of the set of time-series data. The prediction engine 230 in the server system 200 may apply the time-series forecasting models present in the optimal ensemble subset of the trained time-series forecasting models to the set of time-series data over the time-series data to determine the future sample values of the set of time-series data.
FIG. 7 represents a simplified block diagram of dual Long Short Term Memory (LSTM) based neural network architecture 700 for forecasting intermittent time-series data, in accordance with an embodiment of the present disclosure. The dual LSTM based neural network architecture 700 may include, but not limited to, an encoder 702, a first decoder 708, and a second decoder 710. The dual LSTM based neural network architecture 700 is configured to separate intermittency and actual data value predictions apart. The first decoder 708 is configured to learn sequence of presence and absence of values of input time-series data. In other words, the first decoder 708 is configured to output sequence of one and zero values indicating whether next prediction of the time-series data is zero or not. The second decoder 710 is configured to learn time-series trends and seasonality of the input time-series data.
As shown in the FIG. 7, dual pair of LSTM encoder-decoder structure facilitates end-to-end mapping between an ordered multidimensional input sequence of intermittent time-series data and its matching output sequence representing future time-series data. Although only three unrolled LSTM blocks in the encoder, and first and second decoders are shown in FIG. 7, it will be appreciated that any number of LSTM blocks may be used (e.g., corresponding to the number of time-series data points that are selected for use).
The encoder 702 receives time-series data 704 associated with past time-intervals as an input and outputs an encoder vector 706 which is used as input for the first and second decoders. The first and second decoders represent dual decoder architecture that predicts both intermittency and values simultaneously. Based on the encoder output 706, the first decoder 708 predicts intermittency in a future time series. Based on the encoder output 706, the second decoder 710 predicts values for the future time series. Based on the predicted intermittency and values, the first and second decoders forecast the future time series. The future time series is zero at intermittent instances and corresponds to the predicted values at non-intermittent instances.
More particularly, a series of the LSTM blocks 702a, 702b and 702c is fed with multi-dimensional vectors representing time-series data points which appear at different time stamps. The input to the LSTM blocks at each time stamp is simply time-series data points appeared at that timestamp. Based on the time-series data 704 as the input, the encoder 702 generates encoder states 706 as an encoder output.
The LSTM blocks 708a, 708b and 708c in the first decoder 708 learn intermittency characteristics (i.e., pattern of occurrences of zero values) of the time-series data 704 based on the encoder output 706 and predicts intermittency in future time-series data in form of an output [Z1, Z2, Z3] (see, 712). The LSTM blocks 710a, 710b and 710c in the second decoder 710 learns data value pattern of the time-series data 704 based on the encoder output 706 and predicts values for the future time-series data (i.e., trends and seasonality) in form of an output [Y1, Y2, Y3] (see, 714).
Thereafter, the outputs 712 and 714 are combined to predict future time-series data 716. In other words, the output 712 of the first decoder 708 acts as a filter over the output 714 of the second decoder 710 and results in intermittent time-series forecast.
FIG. 8 is a simplified block diagram of a server 800, in accordance with one embodiment of the present disclosure. In one embodiment, the server 800 is an example of one of the plurality of servers 108a-108c as described in FIG. 1. Examples of the server 800 include, but are not limited to, data center server, cloud server, issuer server, acquirer server, and a payment server. The server 800 is an example of one of the servers 108a-108c shown and explained with reference to FIG. 1. The server 800 includes a computer system 802 and a database 804. The computer system 802 includes at least one processor 806 for executing instructions. Instructions may be stored in, for example, but not limited to, a memory 808. The processor 806 may include one or more processing units (e.g., in a multi-core configuration).
The processor 806 is operatively coupled to a communication interface 810 such that computer system 802 is capable of communicating with a remote device 812 such as the ATSF server 102, time-series data repositories 110, etc. For example, the communication interface 810 may receive a request to access time-series data from the ATSF server 102.
The processor 806 may also be operatively coupled to the database 804. The database 804 is any computer-operated hardware suitable for storing and/or retrieving data, such as, but not limited to, time-series data associated with the server 800 including server logs associated with the plurality of components of the server 800, physical data such as the temperature, voltage, associated with the server 800, etc. The database 804 may include multiple storage units such as hard disks and/or solid-state disks in a redundant array of inexpensive disks (RAID) configuration. The database 804 may include a storage area network (SAN) and/or a network attached storage (NAS) system.
In some embodiments, the database 804 is integrated within computer system 802. For example, computer system 802 may include one or more hard disk drives as the database 804. In other embodiments, the database 804 is external to the computer system 802 and may be accessed by the computer system 802 using a storage interface 814. The storage interface 814 is any component capable of providing the processor 806 with access to the database 804. The storage interface 814 may include, for example, an Advanced Technology Attachment (ATA) adapter, a Serial ATA (SATA) adapter, a Small Computer System Interface (SCSI) adapter, a RAID controller, a SAN adapter, a network adapter, and/or any component providing processor 806 with access to the database 804.
In one embodiment, the processor 806 is configured to automatically send time-series data to the remote device 812 at predefined time intervals. The processor is configured to receive a notification regarding the future values of the time-series data, from the remote device 812.
The components of the server 800 provided herein may not be exhaustive, and the server 800 may include more or fewer components than those depicted in FIG. 8. Further, two or more components may be embodied in one single component, and/or one component may be configured using multiple sub-components to achieve the desired functionalities. Some components of the server 800 may be configured using hardware elements, software elements, firmware elements and/or a combination thereof.
Although the invention has been described with reference to specific exemplary embodiments, it is noted that various modifications and changes may be made to these embodiments without departing from the broad spirit and scope of the invention. For example, the various operations, blocks, etc., described herein may be enabled and operated using hardware circuitry (for example, complementary metal oxide semiconductor (CMOS) based logic circuitry), firmware, software and/or any combination of hardware, firmware, and/or software (for example, embodied in a machine-readable medium). For example, the apparatuses and methods may be embodied using transistors, logic gates, and electrical circuits (for example, application specific integrated circuit (ASIC) circuitry and/or in Digital Signal Processor (DSP) circuitry).
Particularly, the server system 200 and its various components may be enabled using software and/or using transistors, logic gates, and electrical circuits (for example, integrated circuit circuitry such as ASIC circuitry). Various embodiments of the invention may include one or more computer programs stored or otherwise embodied on a computer-readable medium, wherein the computer programs are configured to cause a processor or computer to perform one or more operations. A computer-readable medium storing, embodying, or encoded with a computer program, or similar language, may be embodied as a tangible data storage device storing one or more software programs that are configured to cause a processor or computer to perform one or more operations. Such operations may be, for example, any of the steps or operations described herein. In some embodiments, the computer programs may be stored and provided to a computer using any type of non-transitory computer readable media. Non-transitory computer readable media include any type of tangible storage media. Examples of non-transitory computer readable media include magnetic storage media (such as floppy disks, magnetic tapes, hard disk drives, etc.), optical magnetic storage media (e.g. magneto-optical disks), CD-ROM (compact disc read only memory), CD-R (compact disc recordable), CD-R/W (compact disc rewritable), DVD (Digital Versatile Disc), BD (BLU-RAY® Disc), and semiconductor memories (such as mask ROM, PROM (programmable ROM), EPROM (erasable PROM), flash memory, RAM (random access memory), etc.). Additionally, a tangible data storage device may be embodied as one or more volatile memory devices, one or more non-volatile memory devices, and/or a combination of one or more volatile memory devices and non-volatile memory devices. In some embodiments, the computer programs may be provided to a computer using any type of transitory computer readable media. Examples of transitory computer readable media include electric signals, optical signals, and electromagnetic waves. Transitory computer readable media can provide the program to a computer via a wired communication line (e.g., electric wires, and optical fibers) or a wireless communication line.
Various embodiments of the invention, as discussed above, may be practiced with steps and/or operations in a different order, and/or with hardware elements in configurations, which are different than those which, are disclosed. Therefore, although the invention has been described based upon these exemplary embodiments, it is noted that certain modifications, variations, and alternative constructions may be apparent and well within the spirit and scope of the invention.
Although various exemplary embodiments of the invention are described herein in a language specific to structural features and/or methodological acts, the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as exemplary forms of implementing the claims.