FIELD OF THE INVENTION
Embodiments of the present invention generally relate to the field of weather
forecasting, and, more particularly, to systems and methods for forecasting
weather by using data analytics and machine learning.
BACKGROUND OF THE INVENTION
It is well known that real-time weather forecast data is accessible to businesses
and developers through Application Programming Interfaces (APIs), provided
by weather services such as Weather Underground, Accuweather and
Forecast.io. This weather forecast data becomes a crucial variable to many
businesses and interests, and ultimately determines huge financial gains and
losses across all industries. As a result, this forecast data needs to be as
accurate as possible and must have reliable data availability. Due to the
inherent complexity of meteorology, weather forecasts contain a degree of
uncertainty and are rarely perfect.
In addition, it is also evident that forecasts have improved over time, due to
improvements made to Numerical Weather Prediction Models (NWPs),
increased observational data and greater computational power. However,
forecasts still contain errors. These uncertainties vary from one weather API
provider to the next, due to differences in each providers methods and
strategies, such as, for example:
(i) the use of different NWP models; and,
(ii) different observational data; updates times; etc.
3
Consequently, as a result, the accuracy of each provider varies, and these
accuracies are also location dependent, weather variable dependent and use
case dependent.
Furthermore, weather APIs from individual weather services provide forecast
data with reasonable accuracies, and are constantly attempting to improve their
forecast output data and provide greater accuracies to their customers. This
includes:
• Using an ensemble of NWP models and find the best combination of
these NWP model outputs.
• Using greater amount of observational data and ensuring the data is
clean and valid.
Additionally, the accuracies of Weather APIs are limited to the NWP models
that feed into the providers own ensemble model. This means any errors in the
NWP models will also be present in the Weather API output, depending on the
degree which each NWP model is weighted. Observational data is an important
input to NWP models. Some regions contain more observational data than
others. For example, India lacks observations as compared with the spread of
observations in USA and Europe. This effects the model outputs, especially
impacting the accuracy of short-term forecasting.
However, present Weather APIs are not location-specific. Instead, the same
methodologies and strategies are deployed in a global model. This means that
the model parametrization schemes may be applicable to some regions, but
not to others. These parametrizations are often set to suit regions with their
greatest customer base (e.g. USA, Europe), and so the parametrizations are
unsuitable for other regions. This leads to large biases in various locations
4
globally. As a result of the above points, output from Weather APIs tend to
contain bias and this bias fails to get adjusted in the short-term. This causes
over/under - forecasting to be present and persist over several hours.
Accordingly, there remains a need in the art for innovative, novel, collaborative
and interactive solutions providing systems and methods for forecasting
weather by using data analytics and machine learning.
SUMMARY OF THE INVENTION
In accordance with an embodiment of the present invention, a method for
forecasting weather by using data analytics and machine learning includes the
steps of, retrieving a weather forecast information from a plurality of sources for
one or more locations to create a plurality of datasets; applying machine
learning algorithms on the plurality of datasets; comparing the weather forecast
information retrieved from the plurality of sources with observational data
obtained from the one or more locations to create a data analytics based
forecast; and, combining the weather forecast information retrieved from the
plurality of sources with the data analytics based forecast to create a weather
forecast combiner output.
In accordance with an embodiment of the present invention, the method further
includes the step of calculating at least one error in the weather forecast
combiner output by comparing the weather forecast combiner output with the
observational data obtained from the one or more locations. In use, the at least
one error is likely to occur in future.
5
The embodiments of the present disclosure have several features, no single
one of which is solely responsible for their desirable attributes. Without limiting
the scope of the present embodiments as expressed by the claims that follow,
their more prominent features will now be discussed briefly. After considering
this discussion, and particularly after reading the section entitled “Detailed
Description”, one will understand how the features of the present embodiments
provide advantages, which include providing systems and methods for
forecasting weather by using data analytics and machine learning.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 illustrates a flow diagram of a method for forecasting weather by using
data analytics and machine learning, according to an embodiment of the
invention;
FIG. 2 illustrates a system architecture of a system for identification and
verification of a product, according to an embodiment of the invention; and,
FIG. 3A and FIG. 3B illustrate pictorial representation of results produced by
systems and methods for identification and verification of a product, according
to an embodiment of the invention.
DETAILED DESCRIPTION OF THE INVENTION
Various embodiments of the present invention are disclosed herein below,
which relate to systems and methods for forecasting weather by using data
analytics and machine learning.
Generally, it is well known that there are many resources for providing a service
of very accurate, short-term weather forecast data, with global coverage and
6
client-specific. This is provided in various formats via the internet - through an
API, via FTP, or via downloadable data files. This weather forecast data has
been tried and tested in the energy industry, such as for use in electricity load
forecasting. However, it’s capabilities are not limited to the energy industry and
is an important source of accurate weather forecast data for numerous
business needs, as described hereinbelow.
Various systems and methods in accordance with multiple embodiments of this
invention are novel and inventive because they optimize weather forecasts for
a specific latitude and longitude, by using real-time observational data and
applying machine-learning algorithms to provide highly accurate weather
forecast data.
Those of ordinary skills in the art will appreciate that not even a single solution
offered in prior art discloses the concept of applying machine learning
algorithms to multiple Weather APIs to improve upon individual Weather API
sources in a location-specific manner. It will be further appreciated that the use
of machine learning algorithms requires advanced knowledge of how these
algorithms works and how they can be applied to datasets. These algorithms
are complex and require extensive testing and training to give successful
outcomes. The optimization of very-short-term forecasts, by predicting the
future error in the forecast, is an innovative solution to an obvious problem. The
error in forecasts is not constant and tends to have little pattern or repetition.
As a result, predicting the future error in the forecast cannot be judged by eye
and is only possible through advanced statistical techniques in combination
with novel and inventive aspects, as presented herein.
7
In accordance with multiple embodiments of the present invention, the systems
and methods as disclosed herein are aimed at providing multiple features to
the users, such as, for example, but not limited to, increasing the accuracy of
weather forecasts, particularly during extreme weather events.
Those of ordinary skills in the art will appreciate that the accuracy of weather
forecasts is increased by combining the output of multiple weather service
Application programming interface (hereinafter referred to as the “APIs”) and
applying machine learning protocols (algorithms, applications etc.) to learn how
much weighting to give each weather forecast provider, and hence improving
upon the data provided by one single provider.
Various embodiments of the present invention are further aimed at providing
systems and methods to the users that are capable of providing solutions for
one or more specific locations required and for each weather variable of
concern.
In accordance with multiple embodiments of the present invention, the systems
and methods as disclosed herein disclose the novel and inventive aspect of a
‘weather forecast combiner’, which is further configured for very-short-term
forecasting (such as, for example, but not limited to, 3 hours ahead) by
comparing the recent weather forecast combiner output with the latest
observational data for that same location. Subsequently, embodiments of the
present invention predict the error in the forecast output for future timeblocks
(such as, for example, but not limited to, the next 3 hours), thereby allowing the
weather combiner output to be adjusted for these future timeblocks, as
described hereinbelow.
8
In accordance with an embodiment of the present invention, the Weather
Forecast Combiner as disclosed hereinbelow functions in a manner such that
the weather forecasts of multiple APIs from various global weather services are
collected for the location of interest. Machine learning algorithms are applied to
these datasets in order to understand how each individual provider compares
with the observational data of the same location. Large historic datasets of
these forecasts are used to train the model and the algorithms learn how much
weighting to give each individual forecast provider in future outputs of the
combined forecasts, as described below.
In accordance with an embodiment of the present invention, a very-short-term
optimization of Weather Forecast Combiner works in a manner such that the
recent output from the Weather Forecast Combiner is compared against the
recent observational data, in order to predict the likely error that will occur in
the future (next 3 hours) output from the Weather Forecast Combiner. This is
achieved through linear regression of the recent error data, as described below.
FIG. 1 illustrates a flow diagram of a method for forecasting weather by using
data analytics and machine learning, according to an embodiment of the
invention.
In accordance with an embodiment of the present invention, a method for
forecasting weather by using data analytics and machine learning includes the
steps of, retrieving a weather forecast information from a plurality of sources for
one or more locations to create a plurality of datasets; applying machine
learning algorithms on the plurality of datasets; comparing the weather forecast
information retrieved from the plurality of sources with observational data
obtained from the one or more locations to create a data analytics based
9
forecast; and, combining the weather forecast information retrieved from the
plurality of sources with the data analytics based forecast to create a weather
forecast combiner output.
In accordance with an embodiment of the present invention, the method further
includes the step of calculating at least one error in the weather forecast
combiner output by comparing the weather forecast combiner output with the
observational data obtained from the one or more locations. In use, the at least
one error is likely to occur in future.
In accordance with an embodiment of the present invention, the at least one
error is calculated through linear regression of recent error data. In use, the
weather forecast combiner output and the at least one error are transmitted
through a wired or wireless link to a communication network.
FIG. 2 illustrates a system architecture of a system for identification and
verification of a product, according to an embodiment of the invention.
In accordance with an embodiment of the present invention, a system 200 for
forecasting weather by using data analytics and machine learning includes at
least one processor; and, a plurality of computer-executable components for
execution in the at least one processor including: a retrieving component 202
for retrieving a weather forecast information from a plurality of sources (2011,
2012, 2033…) for one or more locations to create a plurality of datasets (2041,
2042, 2043…); a machine learning component 206 for applying machine
learning algorithms on the plurality of datasets (2041, 2042, 2043…); a
comparison component 208 for comparing the weather forecast information
retrieved from the plurality of sources (2011, 2012, 2033…) with observational
10
data obtained from the at least one location to create a data analytics based
forecast; and, a weather combiner component 210 for combining the weather
forecast information retrieved from the plurality of sources (2011, 2012, 2033…)
with the data analytics based forecast to create a weather forecast combiner
output.
In accordance with an embodiment of the present invention, the plurality of
computer-executable components of the system further includes an error
calculator component 212 for calculating at least one error in the weather
forecast combiner output by comparing the weather forecast combiner output
with the observational data obtained from the one or more locations. In use, the
at least one error is likely to occur in future.
In accordance with an embodiment of the present invention, the error calculator
component 212 calculates the at least one error through linear regression of
recent error data.
In accordance with an embodiment of the present invention, the plurality of
computer-executable components of the system further includes a
communication component for transmitting the weather forecast combiner
output and the at least one error through a wired or wireless link to a
communication network, which encompasses communication networks such
as local-area networks (LAN) and wide-area networks (WAN). LAN
technologies include Fiber Distributed Data Interface (FDDI), Copper
Distributed Data Interface (CDDI), Ethernet/IEEE 802.3, Token Ring/IEEE
802.5 and the like. WAN technologies include, but are not limited to, point-topoint
links, circuit switching networks like Integrated Services Digital Networks
11
(ISDN) and variations thereon, packet switching networks, and Digital
Subscriber Lines (DSL).
Those of ordinary skills in the art will appreciate that systems and methods as
disclosed by various embodiments of the present invention may require an
exemplary environment for implementing various aspects of the invention,
including a computer. The computer includes a processing unit, a system
memory, and a system bus. The system bus couples system components
including, but not limited to, the system memory to the processing unit. The
processing unit can be any of various available processors. Dual
microprocessors and other multiprocessor architectures also can be employed
as the processing unit. The system bus can be any of several types of bus
structure(s) including the memory bus or memory controller, a peripheral bus
or external bus, and/or a local bus using any variety of available bus
architectures including, but not limited to, 8-bit bus, Industrial Standard
Architecture (ISA), Micro-Channel Architecture (MSA), Extended ISA (EISA),
Intelligent Drive Electronics (IDE), VESA Local Bus (VLB), Peripheral
Component Interconnect (PCI), Universal Serial Bus (USB), Advanced
Graphics Port (AGP), Personal Computer Memory Card International
Association bus (PCMCIA), and Small Computer Systems Interface (SCSI).
The system memory includes volatile memory and non-volatile memory. The
basic input/output system (BIOS), containing the basic routines to transfer
information between elements within the computer, such as during start-up, is
stored in non-volatile memory. By way of illustration, and not limitation, nonvolatile
memory can include read only memory (ROM), programmable ROM
(PROM), electrically programmable ROM (EPROM), electrically erasable ROM
12
(EEPROM), or flash memory. Volatile memory includes random access
memory (RAM), which acts as external cache memory.
By way of illustration and not limitation, RAM is available in many forms such
as synchronous RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM
(SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM
(ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM).
In accordance with an embodiment of the present invention, the Computer also
includes removable/nonremovable, volatile/non-volatile computer storage
media.
It is to be appreciated that the exemplary environment as disclosed herein
describes an intermediary between users and the basic computer resources
described in suitable operating environment.
FIG. 3A and FIG. 3B illustrate pictorial representation of results produced by
systems and methods for identification and verification of a product, according
to an embodiment of the invention. Those of ordinary skills in the art will
appreciate that systems and methods as disclosed by various embodiments of
the present invention provide significant advantages over prior art, such as, for
example, but not limited to, reducing the inaccuracies of presently available
solutions and reducing the forecast errors, especially in the very short-term time
horizon (next 5 hours), providing 100% data availability and reliability,
increasing the accuracy of weather forecasts, particularly during extreme
weather events by combining the output of multiple weather service APIs and
applying machine learning algorithms to learn how much weighting to give each
weather forecast provider, and hence improving upon the data provided by one
single provider.
13
Conditional language used herein, such as, among others, “can,” “could,”
“might,” “may,” “e.g.,” and the like, unless specifically stated otherwise, or
otherwise understood within the context as used, is generally intended to
convey that certain embodiments include, while other embodiments do not
include, certain features, elements and/or steps. Thus, such conditional
language is not generally intended to imply that features, elements and/or steps
are in any way required for one or more embodiments or that one or more
embodiments necessarily include logic for deciding, with or without author input
or prompting, whether these features, elements and/or steps are included or
are to be performed in any particular embodiment. The terms "comprising,"
"including," 'having," and the like are synonymous and are used inclusively, in
an open-ended fashion, and do not exclude additional elements, features, acts,
operations, and so forth. Also, the term "or" is used in its inclusive sense (and
not in its exclusive sense) so that when used, for example, to connect a list of
elements, the term "or" means one, some, or all of the elements in the list.
While there has been shown and described the preferred embodiment of the
instant invention it is to be appreciated that the invention may be embodied
otherwise than is herein specifically shown and described and that, within said
embodiment, certain changes may be made in the form and arrangement of the
parts without departing from the underlying ideas or principles of this invention
as set forth in the Claims appended herewith. Therefore, the appended claims
are to be construed to cover all equivalents falling within the true scope and
spirit of the invention.
14
Claims
We claim,
1. A method for forecasting weather by using data analytics and machine
learning, said method comprising the steps of:
retrieving a weather forecast information from a plurality of sources for
at least one location to create a plurality of datasets;
applying machine learning algorithms on said plurality of datasets;
comparing said weather forecast information retrieved from said plurality
of sources with observational data obtained from said at least one location to
create a data analytics based forecast; and,
combining said weather forecast information retrieved from said plurality
of sources with said data analytics based forecast to create a weather forecast
combiner output.
2. The method as claimed in Claim 1, wherein said method further
comprises the step of calculating at least one error in said weather forecast
combiner output by comparing said weather forecast combiner output with said
observational data obtained from said at least one location, said at least one
error being likely to occur in future.
3. The method as claimed in Claim 2, wherein said at least one error is
calculated through linear regression of recent error data.
15
4. The method as claimed in Claim 3, wherein said weather forecast
combiner output and said at least one error are transmitted through a wired or
wireless link to a communication network.
5. A system 200 for forecasting weather by using data analytics and
machine learning, said system comprising:
at least one processor; and,
a plurality of computer-executable components for execution in said at
least one processor, said plurality of components comprising:
a retrieving component 202 for retrieving a weather forecast
information from a plurality of sources (2011, 2012, 2033…) for at least
one location to create a plurality of datasets (2041, 2042, 2043…);
a machine learning component 206 for applying machine learning
algorithms on said plurality of datasets (2041, 2042, 2043…);
a comparison component 208 for comparing said weather
forecast information retrieved from said plurality of sources (2011, 2012,
2033…) with observational data obtained from said at least one location
to create a data analytics based forecast; and,
a weather combiner component 210 for combining said weather
forecast information retrieved from said plurality of sources (2011, 2012,
2033…) with said data analytics based forecast to create a weather
forecast combiner output.
6. The system as claimed in Claim 5, wherein said plurality of computerexecutable
components of said system further comprises an error calculator
16
component 212 for calculating at least one error in said weather forecast
combiner output by comparing said weather forecast combiner output with said
observational data obtained from said at least one location, said at least one
error being likely to occur in future.
7. The system as claimed in Claim 5, wherein error calculator component
212 calculates said at least one error through linear regression of recent error
data.
8. The system as claimed in Claim 7, wherein said plurality of computerexecutable
components of said system further comprises a communication
component for transmitting said weather forecast combiner output and said at
least one error through a wired or wireless link to a communication network.