FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENT RULES, 2003
COMPLETE SPECIFICATION
(See Section 10 and Rule 13)
Title of invention:
METHOD AND SYSTEM FOR MANAGING A BIDIRECTIONAL
CHARGING AT AN ELECTRIC VEHICLE (EV) CHARGING STATION
Applicant
Tata Consultancy Services Limited
A company Incorporated in India under the Companies Act, 1956
Having address:
Nirmal Building, 9th floor,
Nariman point, Mumbai 400021,
Maharashtra, India
Preamble to the description
The following specification particularly describes the invention and the
manner in which it is to be performed.
2
TECHNICAL FIELD
[001] The disclosure herein generally relates to the field of bidirectional
charging at an electric vehicle (EV) charging station, and, more particularly, to a
method and system for bidirectional charging at an electric vehicle (EV) charging
5 station with an energy model that uses electricity bought from the day-ahead market
for charging the fleet and uses the intra-day market for arbitrage with arbitrage.
BACKGROUND
[002] Recent years have seen an exponential growth in the adoption of
10 electric vehicles (EV) for last-mile deliveries. EV fleets are of particular interest to
e-commerce companies since they not only curtail vehicular emissions but can also
reduce the cost of operations; both of which are key metrics for sustainable business
operations. Because profit margins are quite thin in the e-commerce business,
companies seek revenue opportunities while optimizing costs. In this regard, EV
15 fleets can function as a distributed energy resource (DER). The energy demands of
EV fleets are reasonably predictable due to the repeating nature of last-mile
deliveries and daily distance traveled. The consolidated energy stored in the batteries
of the EVs at a fleet-scale is, therefore, an arbitrage opportunity in the electricity
markets (such as day-ahead or intra-day) or can provide ancillary services to the grid
20 operator; without impacting operations. Here, the opportunity to generate additional
revenue arises when the cost of charging is less than the benefit of discharging. Grid
interaction via the wholesale electricity market presents an opportunity for EVs to
charge when the electricity price is low and vice versa. However, fleet managers
need to operate such a system within the constraints of the installed captive charging
25 infrastructure and delivery requirements of the fleet. The charging operation
constraints arise due to (𝑖) fewer charging points than EVs (on account of high capital
expenditure and efforts to recover it as the earliest by better asset utilization); (𝑖𝑖)
capacity limitation of the grid connection line to the charging facility; and (𝑖𝑖𝑖)
scheduled black-outs or brown-outs (that are typical in developing economies).
30 Delivery fulfillment needs all EVs in the fleet to be charged adequately for them to
complete the assigned trip routes, and deal with any associated uncertainties that
3
could arise in transit. Furthermore, they would need to return to the depot and prepare
(i.e., load goods and charge) for their next scheduled departures. In addition,
charging/discharging takes much longer than re-fueling existing non-EVs.
Electricity can be traded (buy/sell) in the wholesale markets (e.g., day-ahead, and
5 intra-day). In a day-ahead market, electricity can be traded a day in advance; while
an intra-day market allows trading time up to (typically) an hour before the delivery
time. Trading in both the markets provides opportunity to hedge the risk of volume
and price uncertainties. There is a large body of existing work that has explored
various models to transact energy between EV fleets and day-ahead, intra-day
10 markets. One model is to trade energy in the day-ahead market and adjust the
deviation in day-ahead energy commitments (that arise due to uncertainty in EV
charging demand) through intra-day transactions. However, given that the energy
demand of last-mile EV delivery fleets is reasonably predictable, this model does not
capitalize well on the greater price volatility of the intra-day market to derive better
15 returns; as most of the energy commitments are already made in the day-ahead
market, and therefore, leaves limited room for major trades in the intra-day market.
Another model is to fulfill the fleet energy demand through the grid or other reliable
captive sources (e.g., battery bank), while offering to trade energy only in the intraday market. However, this approach does not capitalize on the day-ahead market
20 dynamics that could potentially fetch energy at lower costs. Managing the
charging/discharging operation using the proposed energy transaction model
requires EVs to be assigned to compatible, available charging points in time. This
problem is non-trivial due to the following reasons: (𝑖) there is limited charging
flexibility due to constraints on the available supply capacity and length of vehicle
25 stay in the depot; (𝑖𝑖) there is limited trading flexibility due to day-ahead
commitments that needs to be prioritized before intra-day transactions; (𝑖𝑖𝑖) there is
propagation of planning errors/deviations when vehicles do multiple trips in a day;
(𝑖𝑣) there is a higher degree of computation complexity to obtain a fleet-level plan
with heterogeneous vehicles (with different battery capacities) and charger types
30 (with different power ratings and connectors); (𝑣) there is electricity price volatility
in the energy markets whose trends differ across day-ahead and intra-day market
4
segments. All of these factors make the planning decision hard and finding a decision
support mechanism for large-scale EV (bidirectional) charging operation (consisting
of hundreds of vehicles and chargers) becomes even harder.
5 SUMMARY
[003] Embodiments of the present disclosure present technological
improvements as solutions to one or more of the above-mentioned technical
problems recognized by the inventors in conventional systems. For example, in one
embodiment, a method of managing bidirectional charging at the electric vehicles
10 (EVs) charging station is provided. The method includes, receiving, via a day-ahead
planning module, a trip plan of a fleet of the EVs, wherein the fleet of the EVs
comprising a plurality of assigned vehicles for a day-ahead trip and a plurality of
available vehicles at the EV charging station. The day ahead planning module also
receives a charge plan of the EV charging station, and price of electricity of a day
15 ahead market. The method further includes, preparing, a day-ahead charge schedule
for each of the plurality of assigned vehicles via the day-ahead planning module,
wherein preparing the day-ahead schedule involves selecting, from the trip plan, the
plurality of vehicles assigned for the day-ahead trip. Further the module assesses a
state S୲
ഥ of the system at each time-step by scheming the plurality of assigned
20 vehicles and a plurality of chargers available at the EV charging station for charging
the plurality of assigned vehicles. The module then allocates, by a learning agent
(LA3_D), the plurality of assigned vehicles to the plurality of chargers available and
observing the state 𝑆𝑡 of the system at each time-step, wherein an allocation of a
vehicle among the plurality of assigned vehicles to a charger among the plurality of
25 chargers is an action At of the learning agent. The module iteratively, transitions to
a next time-step, and continue allocating the plurality of assigned vehicles to the
plurality of chargers and receives a reward for the action of the learning agent,
wherein the reward trains a Graph Neural Network (GNN) to generate the day ahead
schedule for each of the plurality of assigned vehicles. The method further includes,
30 preparing, the intra-day schedule by an intra-day planning module, wherein
preparing the intra-day schedule involves receiving the day-ahead schedule of each
5
of the plurality of assigned vehicles generated by the day-ahead planning module
and the price of electricity of an intra-day market. The module identifies available
time slots for the intra-day schedule from the day-ahead schedule at each time-step
by executing a greedy algorithm to determine feasibility of possible assignment of
5 suitable charger to the available vehicle to derive charger-EV pairing. The module
scans through a plurality of infeasible conditions and identifies feasible time slots
for the assignment of a charger from the plurality of chargers to the available vehicle.
The module discharges the available vehicle at the charger from the plurality of
chargers and trading back the energy in the intra-day market; and applies a priority
10 function to prioritize the plurality of available EVs wherein the priority function is a
weighted sum of an individual priority components. The module then iteratively,
prioritize the vehicle allocation until all the available vehicles for the intra-day
discharging gets the charger for trading back the energy by way of discharging. The
method further includes, scoring, the bidirectional charging by obtaining a cost
15 incurred by the day-ahead planning module in charging the plurality of vehicles in
the day-ahead market and profit generated by the intra-day planning module by
discharging the plurality of vehicles in the intra-day market.
[004] In another aspect, a system for a bidirectional charging at the electric
vehicles (EVs) charging station is provided. The system includes at least one
20 memory storing programmed instructions; one or more Input /Output (I/O)
interfaces; and one or more hardware processors, a day-ahead planning module and
an intra-day planning module, operatively coupled to a corresponding at least one
memory, wherein the system is configured to receive, via a day-ahead planning
module, executed by the one or more hardware processors, a trip plan of a fleet of
25 the EVs, wherein the fleet of the EVs comprising a plurality of assigned vehicles for
a day-ahead trip and a plurality of available vehicles at the EV charging station. The
day ahead planning module also receives a charge plan of the EV charging station,
and price of electricity of a day ahead market. The system is configured to prepare,
via the one or more hardware processors, a day-ahead charge schedule for each of
30 the plurality of assigned vehicles via the day-ahead planning module, wherein
preparing the day-ahead schedule involves selecting, from the trip plan, the plurality
6
of vehicles assigned for the day-ahead trip. Further the module assesses a state S୲
ഥ of
the system at each time-step by scheming the plurality of assigned vehicles and a
plurality of chargers available at the EV charging station for charging the plurality
of assigned vehicles. The module then allocates, by a learning agent (LA3_D), the
5 plurality of assigned vehicles to the plurality of chargers available and observing the
state 𝑆𝑡 of the system at each time-step, wherein an allocation of a vehicle among
the plurality of assigned vehicles to a charger among the plurality of chargers is an
action At of the learning agent. The module iteratively, transitions to a next timestep, and continue allocating the plurality of assigned vehicles to the plurality of
10 chargers and receives a reward for the action of the learning agent, wherein the
reward trains a Graph Neural Network (GNN) to generate the day ahead schedule
for each of the plurality of assigned vehicles. The system is configured to prepare,
via the one or more hardware processors, an intra-day planning module, wherein
preparing the intra-day schedule involves receiving the day-ahead schedule of each
15 of the plurality of assigned vehicles generated by the day-ahead planning module
and the price of electricity of an intra-day market. The module identifies available
time slots for the intra-day schedule from the day-ahead schedule at each time-step
by executing a greedy algorithm to determine feasibility of possible assignment of
suitable charger to the available vehicle to derive charger-EV pairing. The module
20 scans through a plurality of infeasible conditions and identifies feasible time slots
for the assignment of a charger from the plurality of chargers to the available vehicle.
The module discharges the available vehicle at the charger from the plurality of
chargers and trading back the energy in the intra-day market; and applies a priority
function to prioritize the plurality of available EVs wherein the priority function is a
25 weighted sum of an individual priority components. The module then iteratively,
prioritize the vehicle allocation until all the available vehicles for the intra-day
discharging gets the charger for trading back the energy by way of discharging. The
system is configured to score, via the one or more hardware processors, the
bidirectional charging by obtaining a cost incurred by the day-ahead planning
30 module in charging the plurality of vehicles in the day-ahead market and profit
7
generated by the intra-day planning module by discharging the plurality of vehicles
in the intra-day market.
[005] In yet another aspect, a computer program product including a nontransitory computer-readable medium having embodied therein a computer program
5 for managing bidirectional charging at the electric vehicles (EVs) charging station is
provided. The computer readable program, when executed on a computing device,
causes the computing device to receive, via a day-ahead planning module, executed
by the one or more hardware processors, a trip plan of a fleet of the EVs, wherein
the fleet of the EVs comprising a plurality of assigned vehicles for a day-ahead trip
10 and a plurality of available vehicles at the EV charging station. The day ahead
planning module also receives a charge plan of the EV charging station, and price of
electricity of a day ahead market. The computer readable program, when executed
on a computing device, causes the computing device to prepare, via the one or more
hardware processors, a day-ahead charge schedule for each of the plurality of
15 assigned vehicles via the day-ahead planning module, wherein preparing the dayahead schedule involves selecting, from the trip plan, the plurality of vehicles
assigned for the day-ahead trip. Further the module assesses a state S୲
ഥ of the system
at each time-step by scheming the plurality of assigned vehicles and a plurality of
chargers available at the EV charging station for charging the plurality of assigned
20 vehicles. The module then allocates, by a learning agent (LA3_D), the plurality of
assigned vehicles to the plurality of chargers available and observing the state 𝑆𝑡 of
the system at each time-step, wherein an allocation of a vehicle among the plurality
of assigned vehicles to a charger among the plurality of chargers is an action At of
the learning agent. The module iteratively, transitions to a next time-step, and
25 continue allocating the plurality of assigned vehicles to the plurality of chargers and
receives a reward for the action of the learning agent, wherein the reward trains a
Graph Neural Network (GNN) to generate the day ahead schedule for each of the
plurality of assigned vehicles. The computer readable program, when executed on a
computing device, causes the computing device to prepare, via the one or more
30 hardware processors, an intra-day planning module, wherein preparing the intra-day
schedule involves receiving the day-ahead schedule of each of the plurality of
8
assigned vehicles generated by the day-ahead planning module and the price of
electricity of an intra-day market. The module identifies available time slots for the
intra-day schedule from the day-ahead schedule at each time-step by executing a
greedy algorithm to determine feasibility of possible assignment of suitable charger
5 to the available vehicle to derive charger-EV pairing. The module scans through a
plurality of infeasible conditions and identifies feasible time slots for the assignment
of a charger from the plurality of chargers to the available vehicle. The module
discharges the available vehicle at the charger from the plurality of chargers and
trading back the energy in the intra-day market; and applies a priority function to
10 prioritize the plurality of available EVs wherein the priority function is a weighted
sum of an individual priority components. The module then iteratively, prioritize the
vehicle allocation until all the available vehicles for the intra-day discharging gets
the charger for trading back the energy by way of discharging. The computer
readable program, when executed on a computing device, causes the computing
15 device to score, via the one or more hardware processors, the bidirectional charging
by obtaining a cost incurred by the day-ahead planning module in charging the
plurality of vehicles in the day-ahead market and profit generated by the intra-day
planning module by discharging the plurality of vehicles in the intra-day market.
20 BRIEF DESCRIPTION OF THE DRAWINGS
[006] The accompanying drawings, which are incorporated in and
constitute a part of this disclosure, illustrate exemplary embodiments and, together
with the description, serve to explain the disclosed principles:
[007] FIG. 1 illustrates an exemplary block diagram of a system for
25 managing bi-directional charging, according to some embodiments of the present
disclosure.
[008] FIG. 2 illustrates a diagram of the steps executed by a fleet operator
for preparing bidirectional charging plan day-ahead and intra-day charging of
electric vehicle fleet, according to some embodiments of the present disclosure.
9
[009] FIG. 3 is a functional block diagram of sequential constraint
application in the day-ahead charge planning for managing the fleet of electronic
vehicles, according to some embodiments of the present disclosure.
[010] FIG. 4 is a functional block diagram of sequential constraint
5 application in the intra-day charge planning for managing the fleet of electronic
vehicles, according to some embodiments of the present disclosure.
[011] FIG. 5 illustrates an architecture of the system of FIG. 1 for managing
bi-directional charging, according to some embodiments of the present disclosure.
[012] FIG. 6 illustrates a flow diagram depicting a method for managing
10 bi-directional charging, according to some embodiments of the present disclosure.
[013] FIG. 7 is a graph depicting reward function of a training of the
learning agent, according to some embodiments of the present disclosure.
DETAILED DESCRIPTION OF EMBODIMENTS
15 [014] Exemplary embodiments are described with reference to the
accompanying drawings. In the figures, the left-most digit(s) of a reference number
identifies the figure in which the reference number first appears. Wherever
convenient, the same reference numbers are used throughout the drawings to refer to
the same or like parts. While examples and features of disclosed principles are
20 described herein, modifications, adaptations, and other implementations are possible
without departing from the scope of the disclosed embodiments.
[015] There exist multiple approaches for effectively managing the
available power in the electronic vehicles at the charging stations. Exact methods
such as mathematical programming, constraint programming, propositional
25 satisfiability take extremely long computational time for large problem sizes.
Heuristic methods are generally used in practice due to their relatively fast execution
time at scale but yield low-quality solutions due to their myopic decision nature. The
present disclosure employs a bidirectional charging mechanism along with utilizing
power price fluctuation along the length of the day. In the first part, the disclosed
30 model utilizes a day-ahead planning module that prepares EV charging plan in
advance based on EV delivery schedules for the next day. Simultaneously, in the
10
second-part, the disclosed model utilizes an intra-day planning module that estimates
available EVs at the station along with the available power in each EVs and suggests
to prepare next delivery EVs by charging-discharging actions.
[016] In the day-ahead planning module, the present disclosure utilizes
5 agent-based learning methods to solve such large-scale charging decision problems
using a model learned over several problem instances collected in the past; as
opposed to exact methods and heuristics that solve only a single specific problem
instance. Learning agents are able to generalize solutions with reasonable solution
accuracy (compared to exact methods) by discovering repeatable patterns. The
10 learning agent (LA3_D) learns to optimize charging decisions for day-ahead
electricity market. The agent is based on a sequential decision-making model that
observes the state of the system (that includes vehicles; chargers; trip lengths and
energy demand; departure deadline; electricity price); takes a control action of
assigning vehicles to chargers at particular charging rates and at specific time slots;
15 and obtains a reward for the respective action. The reward encodes the operation
objective of cost-effective charging, but without delays that impact departure of
vehicles from the depot. This form of reinforcement guides the agent to explore
better control actions, and eventually obtains an approximate solution that is close to
the optimal strategy.
20 [017] In the intra-day planning module, the present disclosure utilizes a
heuristic approach (GH_I) that makes greedy decisions in intra-day electricity
market. Because intra-day market prices are more volatile and have much shorter
clearing windows, decisions have to be taken quickly. So, a system based on priority
order for planning charging/discharging actions in the intra-day market is built.
25 [018] Referring now to the drawings, and more particularly to FIG. 1
through FIG. 7, where similar reference characters denote corresponding features
consistently throughout the figures, there are shown preferred embodiments and
these embodiments are described in the context of the following exemplary system
and/or method.
11
[019] FIG. 2 illustrates a diagram of the steps executed by a fleet operator
for preparing bidirectional charging plan day-ahead and intra-day charging of
electric vehicle fleet, according to some embodiments of the present disclosure.
[020] In an embodiment, the system 100 includes one or more processors
5 104, communication interface device(s) or input/output (I/O) interface(s) 106, and
one or more data storage devices or memory 102 operatively coupled to the one or
more processors 104. The one or more processors 104 that are hardware processors
can be implemented as one or more microprocessors, microcomputers,
microcontrollers, digital signal processors, central processing units, state machines,
10 graphics controllers, logic circuitries, and/or any devices that manipulate signals
based on operational instructions. Among other capabilities, the processor(s) are
configured to fetch and execute computer-readable instructions stored in the
memory. In the context of the present disclosure, the expressions ‘processors’ and
‘hardware processors’ may be used interchangeably. In an embodiment, the system
15 100 can be implemented in a variety of computing systems, such as laptop
computers, notebooks, hand-held devices, workstations, mainframe computers,
servers, a network cloud and the like.
[021] The I/O interface (s) 106 may include a variety of software and
hardware interfaces, for example, a web interface, a graphical user interface, and the
20 like and can facilitate multiple communications within a wide variety of networks
and protocol types, including wired networks, for example, LAN, cable, etc., and
wireless networks, such as WLAN, cellular, or satellite. In an embodiment, the I/O
interface(s) 106 can include one or more ports for connecting a number of devices to
one another or to another server.
25 [022] The memory 104 may include any computer-readable medium known
in the art including, for example, volatile memory, such as static random-access
memory (SRAM) and dynamic random-access memory (DRAM), and/or nonvolatile memory, such as read only memory (ROM), erasable programmable ROM,
flash memories, hard disks, optical disks, and magnetic tapes. In an embodiment, the
30 memory 102 may include a database or repository. The memory 102 may comprise
information pertaining to input(s)/output(s) of each step performed by the
12
processor(s) 104 of the system 100 and methods of the present disclosure. In an
embodiment, the database may be external (not shown) to the system 100 and
coupled via the I/O interface 106. In an embodiment, the memory 102 includes a
database 108, a day-ahead planning module 110 and an intra-day planning module
5 112. The database 108 stores a complete schedule of all the electric vehicles (EVs)
tagged for day-ahead or intra-day charging along with their delivery schedules. The
day-ahead planning module 110 prepares EV charging plan in advance, based on
EV delivery schedules for the next day. The intra-day planning module 112 estimates
available EVs at the station along with the available power in each EVs and suggests
10 to prepare next delivery EVs by charging-discharging actions. The memory 102
further includes a plurality of modules (not shown here) comprises programs or
coded instructions that supplement applications or functions performed by the
system 100 for executing different steps involved in the process of managing fleets
of electric vehicles through bidirectional charging with arbitrage. The plurality of
15 modules, amongst other things, can include routines, programs, objects, components,
and data structures, which perform particular tasks or implement particular abstract
data types. The plurality of modules may also be used as, signal processor(s), node
machine(s), logic circuitries, and/or any other device or component that manipulates
signals based on operational instructions. Further, the plurality of modules can be
20 used by hardware, by computer-readable instructions executed by the one or more
hardware processors 104, or by a combination thereof. The plurality of modules can
include various sub-modules (not shown).
[023] FIG. 2 illustrates a diagram of the steps executed by a fleet operator
for preparing bidirectional charging plan electric vehicle fleet.
25 [024] As illustrated in FIG. 2, the fleet operator needs to manage a fleet of
EVs for last-mile deliveries and can also trade energy (that is stored in the EV
batteries) with the energy market without comprising delivery operations. Electricity
wholesale market (such as day-ahead or intra-day) is the single source of energy
available to the operator for fulfilling energy requirements of the entire fleet. EVs
30 are rostered to do multiple delivery trips in a day. This timetable is prepared in
advance, where vehicles are assigned to designated routes along with their departure
13
deadlines. EVs must leave the depot on time; complete their respective delivery trips;
return back to the same depot; and then resume the subsequent trip. At step 202, the
fleet operator receives the day-ahead market price 204 and EV roster plan 206. At
step 208, based on the EV roster plan 206, the fleet operator gets an estimate of
5 already assigned EVs. These already assigned EVs are the once for which trips are
planned well in advance with all the estimate of duration of run as well as the distance
to be covered. At step 210, the estimate of assigned vehicle is fed to the day-ahead
planning module 110. At step, 212, the day-ahead planning module 110 assigns
specific chargers to the assigned EVs. Therefore, effective management of EVs for
10 charging can be done through day-ahead planning module 110. At step, 214, the fleet
operator extracts an information about available EVs. The available EVs are the ones
which are not assigned for any upcoming trip but are expected to be ready and
prepared for any unforeseen trip. At step 216, the intra-day market price of the
electricity 218 is fed to the intra-day planning module 112 receives the and
15 accordingly, the intra-day planning module 112 prepares the cost-effective chargingdischarging plan 220. The fleet operator receives intra-day electricity prices are only
a few hours before the market opens for business. It is, therefore, essential for the
fleet operator to obtain a cost-effective charge-discharge plan in order to ensure
timely delivery operations; which gets all EVs charged to the right battery levels by
20 their departure time and can efficiently arbitrage by (dis)charging with the energy
markets.
[025] FIG. 3 is a functional block diagram of sequential constraint
application in the day-ahead charge planning for managing the fleet of electronic
vehicles, according to some embodiments of the present disclosure.
25 [026] As illustrated in FIG. 3, the day-ahead charge planning is modeled to
minimize the charging cost of all vehicles in the fleet using electricity prices from
the day-ahead energy market. The day-ahead charge planning is modeled by
implementing certain intrinsic as well as extrinsic constraints. The sequential
application of these constraints directs the model for guided assistance of charge
30 utilization among vehicles in the EV fleet. It is given by:
14
𝒰஽ = 𝑚𝑖𝑛
ቄఈ(೔,ೕ)
ವ ,௥(೔,ೕ,ೖ)
ವ ,ఋ೔
,௤೔
ವ,ఒ೔ቅ
൞ා ෎ ෍ ෌ 𝑝
஽(𝑡)
௞∈ℛ ௝∈஼
௜∈𝒱
௧∈𝒯
∗ ቀ𝑟(௜,௝,௞)
஽
⋅ ℛ(𝑘)ቁ + 𝒰
஽
ൢ (1)
[027] As illustrated in FIG. 3, at step 302, the penalty term 𝒰
ୈ
in equation
Eq. (1) is the cost of not meeting the energy demand of vehicles by their departure
5 deadlines is a first constraint introduced in the model. Further, at step 304, vehicles
switching chargers across consecutive charging sessions is another constraints. It is
given by:
𝒰
஽
= ቊ(∑ (𝜆௜) ௜∈𝒱 )
ଶ + ൬෍ ෌ ൫𝛿௜(𝑡)൯
௜∈𝒱
௧∈𝒯
൰
ଶ
ቋ
భ
మ
(2)
10 Further, Eq. (1) is sequentially subject to the following constraints:
෍ ቀ 𝛼(௜,௝)
஽ (𝑡)ቁ
௜∈𝒱
≤ 1 ∀ 𝑗 ∈ 𝐶 (3)
[028] As depicted at step 306, it is not possible for any vehicle in the fleet
to charge at more than one charger simultaneously. This condition is enforced by
15 constraint (3).
0 ≤ 𝑞௜
஽
(𝑡) ≤ 𝑄௜
∀𝑖 ∈ 𝒱, ∀𝑡 ∈ 𝒯 (4)
[029] At step 308, at any given time, constraint (4) ensures that the state of
charge (SoC) of a vehicle does not exceed its respective battery capacity.
𝑞௜
஽(𝑡) + (𝑀 ⋅ 𝜆௜) ≥ 𝑒௜ 20 (𝑡) ∀ 𝑖 ∈ 𝒱, ∀𝑡 ∈ 𝒯 (5)
[030] It is also necessary that the SoC of every vehicle is more than energy
required to complete its assigned trip at any given time. This condition is fulfilled by
constraint (5) at step 310. However, in the case of a delay in vehicle departure (𝜆𝑖 =
25 1), a large constant 𝑀 is used to enforce this condition.
෍ [{1 − 𝜓௜(𝑡)} ⋅ 𝛼(௜,௝)
஽
(𝑡)]
௝∈஼
− 𝜆௜ < 1 ∀ 𝑖 ∈ 𝒱, ∀𝑡 ∈ 𝒯 (6)
15
[031] Constraint (6) at step 312 guarantees that a vehicle cannot charge
when it is not in the depot. The respective vehicle does not leave the depot at its
original departure time when it is delayed (𝜆𝑖 = 1).
෍ [{1 − 𝜇(௜,௝)} ⋅ 𝛼(௜,௝)
஽
(𝑡)]
௝∈஼
< 1 ∀ 𝑖 ∈ 𝒱, ∀𝑡 ∈ 𝒯 (7)
5
[032] Constraint (7) at step 314 ensures that a vehicle can charge only at a
compatible
charger
൛𝑅௝
min
⋅ 𝛼(௜,௝)
஽ (𝑡)ൟ ≤ ൛𝑟(௜,௝,௞)
஽ (𝑡) ⋅ ℛ(𝑘)ൟ ≤ ൛𝑅௝
max
⋅ 𝛼(௜,௝)
஽ (𝑡)ൟ 𝑖 ∈ 𝒱,𝑗 ∈ 𝒞, 𝑘 ∈ ℛ,𝑡 ∈ 𝒯
10 (8)
[033] Constraint (8) at step 316 enforces that the charging rate is within the
system limit of each charger in the depot.
෍ ቀ 𝑟(௜,௝,௞)
஽ (𝑡)ቁ
௞∈ℛ
≤ 1 ∀ 𝑖 ∈ 𝒱,𝑗 ∈ 𝐶,𝑡 ∈ 𝒯 (9)
15 [034] Constraint (9) at step 318 ensures that only one charging rate is
chosen at a charger for a vehicle at any given time.
𝑞௜
஽(𝑡 + 1) = 𝑞௜
஽(𝑡) + ൣ൛𝜓௜(𝑡) ⋅ 𝑟(௜,௝,௞)
஽ (𝑡) ⋅ ℛ(𝑘)ൟ − ൛𝐻൫1 − 𝜓௜(𝑡)൯ ⋅ (1 − 𝜆௜)ൟ൧ ,
∀𝑖 ∈ 𝒱, ∀𝑗 ∈ 𝒞, ∀𝑘 ∈ ℛ
(10)
20
[035] Constraint (10) at step 320 captures the change in vehicle SoC level
from time t to (𝑡 + 1). The term {𝜓𝑗 (𝑡)· r(୧,୨,୩)
ୈ
(𝑡)· R(𝑘)} represents the amount of
energy charged at time 𝑡. The term {𝐻(1 −𝜓𝑗 (𝑡) · (1 −𝜆𝑖 )} describes the amount of
energy discharged during a vehicle run; with the exception of a delay in vehicle
25 departure from the depot.
෍ ൣ{1 − 𝜙௜(𝑡)} ⋅ 𝛼(௜,௝)
஽ (𝑡)൧
௜∈஼
≤ 1∀𝑖 ∈ 𝒱, ∀𝑡 ∈ 𝒯 (11)
[036] The condition that the chargers are not utilized when they are down
for maintenance is ensured by constraint (11) at step 322.
𝑞௜
஽(0) = 𝑞௜
௜௡௜௧ 30 (12)
16
[037] The SoC of every vehicle in the fleet must be set at the start of the
planning horizon. It is ensured by constraint (12) at step 324.
𝛿௜(𝑡) = ෍ 𝛼(௜,௝)
஽ (𝑡 − 1)
௝∈஼,௝∈஼ି{௝}
⋅ 𝛼(௜,௝)
஽ (𝑡)∀𝑖 ∈ 𝒱,𝑡 ∈ {2. .𝒯} (13)
5
[038] Constraint (13) at step 326 accounts for the vehicle shifts. It is
calculated as the number of switching instances of a vehicle with respect to the
charger in consecutive time steps. The final output parameters of the day-ahead
charge planner are the decision variables α(୧,୨)
ୈ
(𝑡) and r(୧,୨,୩)
ୈ
(𝑡) for all 𝑡 ∈ T. They
10 are given as input parameters to the intra-day charge-discharge planning module. All
other parameters related to electricity markets, vehicles, and chargers are reset. The
notations used in the above equations as well as in foregoing discussion are presented
in Table-1
Table-1
15
Symbol Meaning
t Time index
Sets
V = {𝑣1, ..., 𝑣𝐼 } set of 𝐼 EVs (at depot)
C = {𝑐1, ..., 𝑐𝐽 } set of 𝐽 chargers (at depot)
R = {𝑟1, ..., 𝑟𝐾 } set of 𝐾 charging rates
D = {𝑑1, ..., 𝑑𝐿 } set of 𝐿 discharging rates
T = {𝑡1, ..., 𝑡𝑇 } set of 𝑇 scheduling intervals
Charger Parameter
𝜙j (t) 1 : if charger 𝑗 is available at time 𝑡
0 : otherwise
R୨
୫୧୬
, R୨
୫ୟ୶ min. & max. charging rate (amp) of charger j
D୨
୫୧୬
, D୨
୫ୟ୶
, min. & max. discharging rate (amp) of charger j
Vehicle Parameter
17
a(୧,୨)
(ୈ)
(t)
1: if EV 𝑖 is charging at charger 𝑗 at time 𝑡 in day-ahead
market;
0: otherwise
a(୧,୨)
(୍)
(t) 1: if EV 𝑖 is charging at charger 𝑗 in interval 𝑡 in intra-day
market;
0: otherwise
r
(୧,୨,୩)
(ୈ)
(t)
1: if EV 𝑖 is charging at charger 𝑗 at charging rate R(𝑘) at time 𝑡 in
day-ahead market;
0: otherwise
r
(୧,୨,୩)
(୍)
(t) 1: if EV 𝑖 is charging at charger 𝑗 at charging rate R(𝑘) at time 𝑡 in
intra-day market;
0: otherwise
d(୧,୨,୪)
(୍)
(t)
1: if EV 𝑖 is discharging at charger 𝑗 at charging rate D(𝑙) at time 𝑡
in intra-day market;
0: otherwise
𝛿𝑖 (𝑡) 1: if EV 𝑖 switched chargers between (𝑡 − 1) & 𝑡;
0: otherwise
𝜆𝑖 1: if EV 𝑖 does not depart on time; 0: otherwise
𝑞𝑖 (𝑡) 𝐷 SoC of EV 𝑖 at time 𝑡 in day-ahead market;
𝐼
𝑞𝑖 (𝑡)
SoC of EV 𝑖 at time 𝑡 in intra-day market
Energy Market Parameters
𝑝ˆ𝐷 (𝑡) Forecast of day-ahead electricity market price at time t
𝑝ˆI (𝑡) Forecast of intra-day electricity price at time t
Decision Variables
α(୧,୨)
ୈ
(t) 1: if EV 𝑖 is charging at charger 𝑗 at time 𝑡 in day-ahead market;
0: otherwise
α(୧,୨)
୍
(t) 1: if EV 𝑖 is charging at charger 𝑗 in interval 𝑡 in intra-day market; 0:
otherwise
r(୧,୨,୩)
୍ୈ (t) 1: if EV 𝑖 is charging at charger 𝑗 at charging rate R(𝑘) at time 𝑡 in
day-ahead market; 0: otherwise
r(୧,୨,୩)
୍
(t) 1: if EV 𝑖 is charging at charger 𝑗 at charging rate R(𝑘) at time 𝑡 in
intra-day market; 0: otherwise
d(୧,୨,୪)
୍
(t) 1: if EV 𝑖 is discharging at charger 𝑗 at charging rate D(𝑙) at time 𝑡 in
intra-day market; 0: otherwise
𝛿𝑖 (𝑡) 1: if EV 𝑖 switched chargers between (𝑡 − 1) & 𝑡; 0: otherwise
𝜆𝑖 1: if EV 𝑖 does not depart on time; 0: otherwise
q୧
ୈ
(t) SoC of EV 𝑖 at time 𝑡 in day-ahead market;
18
[039] FIG. 4 is a functional block diagram of sequential constraint
application in the intra-day charge planning for managing the fleet of electronic
vehicles, according to some embodiments of the present disclosure.
5 [040] As illustrated in FIG. 4, the intra-day charge planning is modeled to
maximize the revenue through energy arbitrage. This implies minimizing the cost of
energy bought through charging and maximizing the cost of energy sale through
discharging using intra-day electricity prices. The intra-day charge planning is
modeled by implementing certain intrinsic as well as extrinsic constraints. The
10 sequential application of these constraints directs the model for guided assistance of
charge utilization by balancing through charging-discharging mechanism wherein
based on sequential application of constraints, model identifies EVs for charging and
the EVs for discharging. It is given by:
𝒰ூ = (𝑚𝑖𝑛)൛𝛼(௜,௝)
ூ
, 𝑟(௜,௝,௞)
ூ
, 𝑟(௜,௝,௞)
ூ
, 𝛿௜
, 𝑞௜
ூ
, 𝜆௜ൟ
⎩
⎪
⎨
⎪
⎧෎ ෍ ∑ ∑ ∑ ௝∈஼ ௟∈𝒟 ௞∈ℛ
௜∈𝒱
௧∈𝒯
. 𝑝
ூ(𝑡)
∗ ቀ𝑟(௜,௝,௞)
ூ
⋅ ℛ(𝑘) − 𝑑(௜,௝,௟)
ூ
⋅ 𝒟(𝑙)ቁ + 𝒰
ூ
⎭
⎪
⎬
⎪
⎫

15 (14)
[041] As illustrated in FIG. 4, at step 402, the penalty term 𝓊ഥ
୍
in equation
Eq. (14) is the cost of vehicles switching chargers across consecutive charging
sessions. It is given by:
𝒰
ூ
= ൜෍ ൫ ෌ ൫ 𝛿௜(𝑡)൯
௜∈𝒱
൯
௧∈𝒯
20 ൠ (15)
Eq. (14) is subject to the following constraints:
෍ ൛𝛼(௜,௝)
஽ (𝑡) + 𝛼(௜,௝)
ூ (𝑡)ൟ
௜∈𝒱
≤ 1∀𝑗 ∈ 𝐶, (16)
25
[042] Constraint (16) at step 404 ensures that the intra-day charging
decisions are made only in available time slots and do not override the day-ahead
schedules.
q୧
୍ (t) SoC of EV 𝑖 at time 𝑡 in intra-day market
19

0 ≤ 𝑞௜
ூ(𝑡) ≤ 𝑄௜ ∀𝑖 ∈ 𝒱, ∀𝑡 ∈ 𝒯,
𝑞௜
ூ(𝑡) ≥ 𝑒௜(𝑡) ∀𝑖 ∈ 𝒱, ∀𝑡 ∈ 𝒯,
(17) & (18)
[043] Constraints (17), (18) at steps 406 and 408 are similar to constraints
5 (4), (5) respectively; but for meeting the SoC requirements for intra-day schedule.

෍ ൣ{1 − 𝜓௜(𝑡)} ⋅ 𝛼(௜,௝)
ூ (𝑡)൧
௝∈஼
< 1 ∀𝑖 ∈ 𝒱, ∀𝑡 ∈ 𝒯,
෍ ൣ൛1 − 𝜇(௜,௝)ൟ ⋅ 𝛼(௜,௝)
ூ (𝑡)൧
௝∈஼
< 1 ∀𝑖 ∈ 𝒱, ∀𝑡 ∈ 𝒯,
(19) & (20)
10 [044] Constraints (19), (20) at steps 410 and 412 are similar to constraints
(6), (7) respectively; but for meeting the vehicle availability and vehicle-charger
compatibility conditions for intra-day schedule.

൛𝑅௝
௠௜௡
⋅ 𝛼(௜,௝)
ூ (𝑡)ൟ ≤ ൛𝑟(௜,௝,௞)
ூ (𝑡) ⋅ ℛ(𝑡)ൟ ≤ ൛𝑅௝
௠௔௫
⋅ 𝛼(௜,௝)
ூ (𝑡)ൟ
∀𝑖 ∈ 𝒱, ∀𝑗 ∈ 𝐶, ∀𝑘 ∈ ℛ, ∀𝑡 ∈ 𝒯,
൛𝑅௝
௠௜௡
⋅ 𝛼(௜,௝)
ூ (𝑡)ൟ ≤ ൛𝑑(௜,௝,௞)
ூ (𝑡) ⋅ 𝒟(𝑡)ൟ ≤ ൛𝑅௝
௠௔௫
⋅ 𝛼(௜,௝)
ூ (𝑡)ൟ
∀𝑖 ∈ 𝒱, ∀𝑗 ∈ 𝐶, ∀𝑙 ∈ 𝒟, ∀𝑡 ∈ 𝒯,
(21) & (22)
15 [045] Constraints (21), (22) at steps 414 and 416 enforce that the charging
and discharging rates are within the system limit of each charger in the depot.
෍ ቀ𝑟(௜,௝,௞)
ூ (𝑡)ቁ
௞∈ℛ
+ ෍ ቀ𝑑(௜,௝,௟)
ூ (𝑡)ቁ
௟∈𝒟
≤ 1∀𝑖 ∈ 𝒱, ∀𝑗 ∈ 𝐶, ∀𝑡 ∈ 𝒯, (23)
[046] Constraint (23) at step 418 ensures that either charging or discharging
20 happens at a charger at any given time, and that only one charging or discharging
rate is chosen for the respective operation.

𝑞௜
ூ(𝑡 + 1) = 𝑞௜
ூ(𝑡) + {𝑟(௜,௝,௞)
஽
(𝑡) + 𝑟(௜,௝,௞)
ூ
(𝑡)} ⋅ ℛ(𝑘)} −
൛𝑑(௜,௝,௟)
ூ (𝑡) ⋅ 𝒟(𝑙)ൟ − ൛𝐻൫1 − 𝜓௜(𝑡)൯ൟ
∀𝑖 ∈ 𝒱, ∀𝑗 ∈ 𝒞, ∀𝑘 ∈ ℛ, ∀𝑙 ∈ 𝒟,
(24)
[047] Constraint (24) at step 420 is used to update the SoC level from time
𝑡 to (𝑡 + 1), taking into account the day-ahead energy commitment and the
25 charging/discharging done using intra-day market.
20
෍ ൣ{1 − 𝜙௜(𝑡)} ⋅ 𝛼(௜,௝)
ூ (𝑡)൧
௜∈஼
≤ 1 ∀𝑖 ∈ 𝒱, ∀𝑡 ∈ 𝒯,
𝑞௜
ூ(0) = 𝑞௜
௜௡௜௧ ∀𝑖 ∈ 𝒱,
𝛿௜(𝑡) = ෍ 𝛼(௜,௝ )
ூ (𝑡 − 1)
௝∈஼,௝∈஼ି{௝}
⋅ 𝛼(௜,௝)
ூ (𝑡)
∀𝑖 ∈ 𝒱, ∀𝑡 ∈ {2. . 𝒯}.
(25), (26) & (27)
[048] Constraints (25), (26), (27) at steps 422, 424 and 426 ensure charger
availability; initialize the starting SoC of each vehicle; account for the vehicle shifts
5 across chargers. They are similar to constraints (11), (12), (13) respectively, but for
the intra-day schedule.
[049] According to an embodiment of the present disclosure, many of the
constraints defined above in the optimization routines are non-linear. Even though
they can be linearized using additional binary variables, the modified problem is still
10 computationally expensive and hard to solve optimally. (𝑖𝑖) Solving the problem on
a scale is even more difficult. Each decision time-step results in (𝐼 ∗ 𝐽 ∗ 𝐾) decision
variables. Therefore, even for a small problem size of 𝐼 = 5; 𝐽 = 3; 𝐾 = 𝐿 = 16 and 𝑇
= 96 time steps (considering 15 min decision interval), the decision space is
significantly large with 5760 decision variables. This space grows exponentially as
15 the problem scales with more vehicles, chargers, and charging/discharging rates;
which lead to extremely long computational delays that take days to get an optimal
solution. To overcome the above challenges and maintain a reasonable solution
accuracy, the present disclosure proposes an approximate method to optimize the
problem of handling large constraints using a learning agent.
20 [050] FIG. 5 illustrates an architecture of the system of FIG. 1 for managing
bi-directional charging, according to some embodiments of the present disclosure.
[051] As illustrated in FIG. 5, vehicle raw features 502, operation raw
features 504 and V-O arc raw features 506 are input to the heterogeneous graph
neural network (HGNN). The vehicle raw features 502 comprises of bi (t), a binary
25 value indicating if vehicle 𝑣𝑖 needs charging (1) or not (0) at time-step t; |Nt (vi)|,
num. of neighboring operation nodes of vehicle node 𝑣𝑖 at time-step t; Ei(t), required
trip SoC of vehicle vi at time-step t; and T୧
ୢ
(t), duration left for vehicle vi to depart
21
from the depot at t. The operation raw features 504 includes bjk (t), binary value
indicating if operation okj (charger k; charging rate j) is available (1) or not (0) at t,
|Nt (ojk)|, number of neighboring vehicle nodes of operation node ojk and P෢ୈ(t),
forecast of day-ahead electricity market price at time t. The V-O arc raw features 506
5 include δi(t), total number of vehicle shifts. The input to GNN 508 architecture is
processed in two-stage embedding that efficiently encode the varying size
heterogeneous graph G𝑡 and obtain a fixed-dimensional embedding of size ሬd⃗. The
two-stage embedding comprises of (𝑖) Vehicle embedding 510 and (ii) Operation
embedding 512. In the vehicle embedding 510 the neighbors of a vehicle node 𝑣𝑖 in
10 G𝑡 are a set of operation nodes N𝑡(𝑣𝑖 ) of different importance levels. In order to
learn these relationships, an attention mechanism is applied. Since each 𝑣𝑖 is
connected with a neighboring O𝑗𝑘 with only one 𝑒(𝑖,𝑗,𝑘); the corresponding feature
vectors of each O𝑗𝑘 ∈ N𝑡 (𝑣𝑖) and 𝑒(𝑖,𝑗,𝑘) are concatenated to obtain 𝜎𝑖𝑗𝑘 = [𝜎𝑗𝑘
∥𝜑𝑖𝑗𝑘 ] ∈ R4 . The attention coefficients 𝑒𝑖𝑗𝑘 for vehicle node to neighboring
15 operation nodes is given as:
𝑒௜௝௞ = LeakyReLU൫𝐚
ୃൣ𝐖ଵ
𝒱𝜗௜ ∥ 𝐖ଵ
ை𝜎௜௝௞൧൯ (28)
where Wଵ
୚𝜗i ∈ R
ୢሬ⃗ ୶ ସ
and Wଵ
୓∈ R
ୢሬ⃗ ୶ ସ
are the respective linear transformations for
vehicle and operation nodes; a ∈ Rଶୢሬሬሬሬሬ⃗
; 𝜗𝑖 ∈ R4
. The attention coefficient eതതననത for
20 vehicle node to itself is given as:
𝑒௜௜ = LeakyReLU൫𝐚
ୃൣ𝐖ଵ
𝒱𝜗௜ ∥ 𝐖ଵ
𝒱𝜗௜൧൯ (29)
[052] The normalized attention coefficients aതijk ∀ eതijk is obtained by using
the softmax function. The aggregate embeddings 𝑣ˆ𝑖 ∈ R
ୢഥ
of vehicle node 𝑣𝑖 is given
25 as:
𝑣௜ = 𝛼௜௜𝐖ଵ
𝒱𝜗௜ + ෍ 𝛼௜௝௞𝐖ଵ
ை𝜎௜௝௞
௢ೕೖ∈ே೟(௩೔
)
(30)
[053] The operation embedding 512 uses the same architecture as that of
vehicle embedding with the exception of the features. The feature vector of each 𝑣𝑖
∈ 𝑁𝑡(𝑜𝑗𝑘) is extended by concatenating it with the corresponding edge as: 𝜗𝑖𝑗𝑘 =
22
[𝜗𝑖||𝜑𝑖𝑗𝑘] ∈ R5 . The attention coefficients eොijk for operation node to neighboring
vehicle nodes is calculated as:
𝑒௜௝௞ = LeakyReLU൫𝐚
ୃൣ𝐖ଶ
𝒱𝜗௜௝௞ ∥ 𝐖ଶ
ை𝜎௝௞൧൯ (31)
Where Wଶ
୚𝜗i ∈ R
ୢሬ⃗ ୶ ହ
and Wଶ
୓∈ R
ୢሬ⃗ ୶ ଷ 5 are the respective linear transformations for
vehicle and operation nodes; a ∈ Rଶୢሬሬሬሬሬ⃗
; 𝜎𝑗𝑘 ∈ R4
. The attention coefficient eොjj for
operation node to itself is:
𝑒௝௝ = LeakyReLU൫𝐚
ୃൣ𝐖ଶ
ை𝑜௝௞ ∥ 𝐖ଶ
ை𝑜௝௞൧൯ (32)
10 [054] The normalized attention coefficients aොiijk ∀ eොijk and aොjj ∀ eොjj is
obtained by using the softmax function. The aggregate embeddings oොjk ∈ R
ୢሬ⃗
of
operation node 𝑜𝑗𝑘 is given as:
𝑜௝௞ = 𝛼௝௝. 𝐖ଶ
ை𝜎௝௞ + ෍ 𝛼௜௝௞
௩೔∈ே೟൫௢ೕೖ൯
. 𝐖ଶ
𝒱
. 𝜗௜௝௞ (33)
15 [055] The vehicle embedding 510 and operation embedding 512 are stacked
and passes through pooling layers 514 to derive state embeddings 516. The final
embeddings vො
୧
(୐)
and oො
୨୩
(୐)
are obtained by passing vො୧
and oො୨୩ through 𝐿 GNN layers
of identical structure. Mean pooling is applied after these 𝐿 layers to obtain the
vehicle and operation embedding sets separately, each of ሬd⃗- dimension. It is then
concatenated to obtain the single state embedding set 516 ℎ𝑡 ∈ R
ଶୢሬሬሬሬሬ⃗
20 of graph state
G𝑡 . It is given as:
ℎ௧ = ቈ
ଵ
|ை|෍ ൫ 𝑜௝௞
௅ ൯
௢ೕೖ∈ை
∥
ଵ
|𝒱|෌ ( 𝑣௜
௅ )
௩೔∈௏
቉ (34)
[056] The state embedding 516 is then pass through multilayer perceptron
25 (MLP)518. The MLP 518 triggers respective action At. The action 𝐴𝑡 is the vehicle
to charging operation assignment at time-step 𝑡. It is selected either at the start of an
episode when all vehicles are unassigned, or when any vehicle is waiting for
assignment. This action is derived in the following manner. For each of the feasible
actions {𝑣𝑖, 𝑜 𝑗𝑘} at 𝑡, the corresponding vehicle, operation and state embeddings
23
are concatenated, and given to a policy network to get a priority index of those
actions that can be selected at state S୲
ഥ. The assignment of vehicle 𝑣𝑖 to charger 𝑐𝑗 at
charging rate 𝑟𝑘 is chosen as the action with the highest priority value. The process
is repeated until the energy demand for vehicle 𝑣𝑖 is fulfilled or there are no more
5 feasible actions for vehicle 𝑣𝑖. The environment is updated along with the reward for
the assigned action.
[057] FIG. 6 illustrates a flow diagram depicting a method for managing
bi-directional charging, according to some embodiments of the present disclosure.
[058] The steps of the method 600 of the present disclosure will now be
10 explained with reference to the components or blocks of the system 100 as depicted
in FIG. 1 through FIG. 7. Although process steps, method steps, techniques or the
like may be described in a sequential order, such processes, methods, and techniques
may be configured to work in alternate orders. In other words, any sequence or order
of steps that may be described does not necessarily indicate a requirement that the
15 steps be performed in that order. The steps of processes described herein may be
performed in any order practical. Further, some steps may be performed
simultaneously.
[059] At step 602 of the method 600, the one or more hardware processors
104 are configured to receive, via day-ahead planning module 110, a trip plan of a
20 vehicle fleet comprising a plurality of vehicles assigned for a day-ahead charge and
a plurality of vehicles marked for an intra-day charge. The trip plan also includes
vehicle specification, number of trips assigned, distance of each trip and the status
of power available in the vehicle. Further, the day-ahead planning module 110
includes a charge plan of the EV charging station, and price of electricity of a day
25 ahead market. Based on above inputs, at step 604 of the method 600, the day-ahead
planning module 110 prepares a day-ahead charge schedule for each of the plurality
of assigned vehicles. The day-ahead schedule preparation comprises selecting, from
the trip plan, the plurality of vehicles assigned for the day-ahead trip and assessing a
state S୲
ഥ of the system at each time-step by scheming the plurality of assigned
30 vehicles and a plurality of chargers available at the EV charging station for charging
the plurality of assigned vehicles. The state of the system is defined by all the features
24
based on which the day ahead planning module 110 executes possible allocation of
EVs to the chargers from the plurality of chargers at the charging station. The
allocation is executed by a learning agent (LA3_D) wherein the plurality of assigned
vehicles is allocated to the plurality of available chargers as the learning agent
5 observes the state 𝑆𝑡 of the system at each time-step and takes an At. The action of
the learning agent is allocation of the charger to the assigned vehicle. The learning
agent (LA3_D) iteratively, transitions to a next time-step, and continue allocating
the plurality of assigned vehicles to the plurality of chargers; and receives a reward
for the action of the learning agent, wherein the reward trains a Graph Neural
10 Network (GNN) to generate the day ahead schedule for each of the plurality of
assigned vehicles. The reward function for each action A୲
തതത = {vi, ojk} choose the
reward R୲ାଵ തതതതതത is given as a weighted sum of four components:
𝑅௧ାଵ = −൛𝐴ଵ ⋅ 𝑝
஽(𝑡) ⋅ 𝑟(௜,௝,௞)
஽
⋅ ℛ(𝑘)ൟ + ቊ𝐴ଶ ⋅ ቆ
𝐸௜(𝑡)
𝑇௜
ௗ(𝑡)
ቇቋ − {𝐴ଷ ⋅ 𝛿௜(𝑡)} − {𝐴ସ ⋅ 𝜆௜}
(35)
15 [060] The first term is the cost of charging, and a negative reward is given
for reducing this factor. The charging urgency is captured by the second term, and a
positive reward is given to actions that prioritize such assignments. The negative
reward of the third and fourth terms, respectively, prevent vehicles from shifting
chargers and incurring delays in their departure schedules. 𝐴1, 𝐴2, 𝐴3, 𝐴4 are non20 negative weight coefficients. The training of the learning agent (LA3_D) involves
obtaining policies using the proximal policy optimization (PPO) technique. It is a
type of actor-critic algorithm consisting of two neural networks 𝜋𝜃 and 𝜋𝜔. The
actor (or policy function) network 𝜋𝜃 learns the actions to be taken for an observed
state, while the critic (or value function) network 𝜋𝜔 learns to evaluates actions taken
25 by the actor based on the given policy. Both 𝜋𝜃 and 𝜋𝜔 have the same network
structure with two hidden layers and one activation layer. 𝜋𝜃 takes h୲
෡ as input and
gives a list of probabilities as output, with one probability per action selected at that
state. Note that h୲
෡ = [ vෝన ∥ σෞ఩୩ ∥ℎ𝑡] is obtained by concatenating the corresponding
operation, vehicle, and state embedding sets for each feasible action. 𝜋𝜔 takes h୲
෡ as
30 input and gives a single number representing the estimated value of the action. 𝜋𝜃
25
optimizes the policy according to the value given by 𝜋w. The learning agent was
trained using randomly generated datasets generated using specifications given in
the work by Garg et. al. Electricity prices from the day-ahead and intra-day energy
markets (and other related parameters) are taken from Turkey EPIAS. The average
5 reward received by the agent during the training phase is shown in FIG. 7. It is
observed that the agent learns aggressively over 60K episodes and stabilizes
thereafter.
[061] At step 606 of the method 600, the one or more hardware processors
104 are configured to receive, via intra-day planning module 112, the day-ahead
10 schedule of each of the plurality of assigned vehicles generated by the day-ahead
planning module 110. The intra-day planning module 112 further receives price of
an electricity of an intra-day market. Based on these inputs, the intra-day planning
module 112 identifies available time slots for the intra-day schedule from the dayahead schedule at each time-step by executing a greedy algorithm (GH_I) to
15 determine feasibility of possible assignment of suitable charger to the available
vehicle to derive charger-EV pairing. At each time-step 𝑡 ∈ T, the algorithm
determines the feasibility of possible assignments (vehicle 𝑣𝑖 → charger 𝑐𝑗 →
charging rate 𝑟𝑘 or discharging rate 𝑑𝑙) using a masking scheme. An assignment is
considered infeasible if satisfies any of the following conditions:
20 • vehicle 𝑣𝑖 is not available at 𝑡 (refer to Eq. 16 and Eq. 19)
• vehicle 𝑣𝑖-charger 𝑐𝑗 is not compatible for charging or discharging (refer to Eq. 20)
• charging rate 𝑟𝑘 and discharging rate 𝑑𝑙 are not within the system limit of charger
𝑐𝑗 (refer to Eq. 21 and Eq. (22))
• charging rate 𝑟𝑘 and discharging rate 𝑑𝑙 are both non-zero at charger 𝑐𝑗 at 𝑡 (refer
25 to Eq. 23)
• charger 𝑐𝑗 is not available at 𝑡 (refer Eq. 25)
[062] From this list, the algorithm chooses an assignment according to a
priority function. It is a weighted sum of individual priority components that capture
the: (a) urgency of a vehicle to charge; (b) cost of charging and discharging; (c)
30 difference between the forecasted intra-day electricity cost at time 𝑡 and the average
forecasted cost of electricity for the entire day; (d) maximizes the rate of charging or
26
discharging. The process is iterated until all participating vehicles get an assignment,
or no more chargers are available. Therefore, the intra-day planning module 112
scans through a plurality of infeasible conditions and identifies feasible time slots
for the assignment of a charger from the plurality of chargers to the available vehicle.
5 The assignment facilitates discharging of the available vehicle at the charger from
the plurality of chargers and trades back the energy in the intra-day market. The
greedy heuristic further applies a priority function to prioritize the plurality of
available EVs wherein the priority function is a weighted sum of an individual
priority components. The intra-day planning modules 112, iteratively, prioritizes the
10 vehicle allocation until all the available vehicles for the intra-day discharging get the
charger for trading back the energy by way of discharging.
[063] At step 608 of the method 600, the one or more hardware processors
104 are configured to score the bidirectional charging by obtaining a cost incurred
by the day-ahead planning module in charging the plurality of vehicles in the day15 ahead market and profit generated by the intra-day planning module by discharging
the plurality of vehicles in the intra-day market.
[064] The written description describes the subject matter herein to enable
any person skilled in the art to make and use the embodiments. The scope of the
subject matter embodiments is defined by the claims and may include other
20 modifications that occur to those skilled in the art. Such other modifications are
intended to be within the scope of the claims if they have similar elements that do
not differ from the literal language of the claims or if they include equivalent
elements with insubstantial differences from the literal language of the claims.
[065] The bidirectional (two-way) charging management is a decision
25 control problem that aims to reduce the cost of operating an EV fleet using optimal
charging/discharging strategies for energy procurement and sale. The present
disclosure is an approach for the solving this problem at-scale by combining a
learning model for day-ahead planning with a heuristic method for intra-day
scheduling. The combined planning model gives a 17-23% cost reduction over cases
30 that do not arbitrage energy. The embodiments of present disclosure herein address
unresolved problem of effective utilization of power in the electricity market with an
27
arbitrage wherein conscious planning is done through day-ahead planning module to
buy electricity to charge the assigned vehicles and intra-day module planning module
to trade back the electricity by discharging available vehicles.
[066] It is to be understood that the scope of the protection is extended to
5 such a program and in addition to a computer-readable means having a message
therein; such computer-readable storage means contain program-code means for
implementation of one or more steps of the method, when the program runs on a
server or mobile device or any suitable programmable device. The hardware device
can be any kind of device which can be programmed including e.g., any kind of
10 computer like a server or a personal computer, or the like, or any combination
thereof. The device may also include means which could be e.g., hardware means
like e.g., an application-specific integrated circuit (ASIC), a field-programmable
gate array (FPGA), or a combination of hardware and software means, e.g., an ASIC
and an FPGA, or at least one microprocessor and at least one memory with software
15 processing components located therein. Thus, the means can include both hardware
means, and software means. The method embodiments described herein could be
implemented in hardware and software. The device may also include software
means. Alternatively, the embodiments may be implemented on different hardware
devices, e.g., using a plurality of CPUs.
20 [067] The embodiments herein can comprise hardware and software
elements. The embodiments that are implemented in software include but are not
limited to, firmware, resident software, microcode, etc. The functions performed by
various components described herein may be implemented in other components or
combinations of other components. For the purposes of this description, a computer25 usable or computer readable medium can be any apparatus that can comprise, store,
communicate, propagate, or transport the program for use by or in connection with
the instruction execution system, apparatus, or device.
[068] The illustrated steps are set out to explain the exemplary
embodiments shown, and it should be anticipated that ongoing technological
30 development will change the manner in which particular functions are performed.
These examples are presented herein for purposes of illustration, and not limitation.
28
Further, the boundaries of the functional building blocks have been arbitrarily
defined herein for the convenience of the description. Alternative boundaries can be
defined so long as the specified functions and relationships thereof are appropriately
performed. Alternatives (including equivalents, extensions, variations, deviations,
5 etc., of those described herein) will be apparent to persons skilled in the relevant
art(s) based on the teachings contained herein. Such alternatives fall within the scope
of the disclosed embodiments. Also, the words “comprising,” “having,”
“containing,” and “including,” and other similar forms are intended to be equivalent
in meaning and be open ended in that an item or items following any one of these
10 words is not meant to be an exhaustive listing of such item or items or meant to be
limited to only the listed item or items. It must also be noted that as used herein and
in the appended claims, the singular forms “a,” “an,” and “the” include plural
references unless the context clearly dictates otherwise.
[069] Furthermore, one or more computer-readable storage media may be
15 utilized in implementing embodiments consistent with the present disclosure. A
computer-readable storage medium refers to any type of physical memory on which
information or data readable by a processor may be stored. Thus, a computerreadable storage medium may store instructions for execution by one or more
processors, including instructions for causing the processor(s) to perform steps or
20 stages consistent with the embodiments described herein. The term “computerreadable medium” should be understood to include tangible items and exclude carrier
waves and transient signals, i.e., be non-transitory. Examples include random access
memory (RAM), read-only memory (ROM), volatile memory, nonvolatile memory,
hard drives, CD ROMs, DVDs, flash drives, disks, and any other known physical
25 storage media.
[070] It is intended that the disclosure and examples be considered as
exemplary only, with a true scope of disclosed embodiments being indicated by the
following claims

We Claim:
1. A processor implemented method for managing a bi-directional charging at an
electric vehicle (EV) charging station, wherein the method comprising:
receiving (602,) via a day-ahead planning module executed by one or more hardware
5 processors,
a trip plan of a fleet of the EVs, wherein the fleet of the EVs comprising
a plurality of assigned vehicles for a day-ahead trip and a plurality of
available vehicles at the EV charging station,
a charge plan of the EV charging station, and
10 price of electricity of a day ahead market;
preparing (604), a day-ahead charge schedule for each of the plurality of assigned
vehicles via the day-ahead planning module executed by one or more hardware
processors, wherein preparing the day-ahead schedule comprises:
selecting, from the trip plan, the plurality of vehicles assigned for the day15 ahead trip;
assessing a state S୲
ഥ of the system at each time-step by scheming the plurality
of assigned vehicles and a plurality of chargers available at the EV charging
station for charging the plurality of assigned vehicles;
allocating, by a learning agent (LA3_D), the plurality of assigned vehicles to
20 the plurality of chargers available and observing the state 𝑆𝑡 of the system at
each time-step , wherein an allocation of a vehicle among the plurality of
assigned vehicles to a charger among the plurality of chargers is an action
At of the learning agent;
iteratively, transitioning to a next time-step, and continuing allocating the
25 plurality of assigned vehicles to the plurality of chargers; and
receiving a reward for the action of the learning agent, wherein the reward
trains a Graph Neural Network (GNN) to generate the day ahead schedule for
each of the plurality of assigned vehicles;
30
preparing (606), the intra-day schedule by an intra-day planning module executed by
the one or more hardware processors, wherein preparing the intra-day schedule
comprises:
receiving the day-ahead schedule of each of the plurality of assigned vehicles
5 generated by the day-ahead planning module;
receiving price of electricity of an intra-day market;
identifying available time slots for the intra-day schedule from the day-ahead
schedule at each time-step by executing a greedy algorithm to determine
feasibility of possible assignment of suitable charger to the available vehicle
10 to derive charger-EV pairing;
scanning through a plurality of infeasible conditions and identifies feasible
time slots for the assignment of a charger from the plurality of chargers to the
available vehicle;
discharging the available vehicle at the charger from the plurality of chargers
15 and trading back the energy in the intra-day market; and
applying a priority function to prioritize the plurality of available EVs
wherein the priority function is a weighted sum of an individual priority
components;
iteratively, prioritizing the vehicle allocation until all the available vehicles
20 for the intra-day discharging gets the charger for trading back the energy by
way of discharging; and
scoring (608), the bidirectional charging by obtaining a cost incurred by the dayahead planning module in charging the plurality of vehicles in the day-ahead market
and profit generated by the intra-day planning module by discharging the plurality
25 of vehicles in the intra-day market.
2. The method as claimed in claim 1, wherein the trip plan includes vehicle
specification, number of trips assigned, distance of each trip and the status of
power available in the vehicle.
30
31
3. The method as claimed in claim 1, wherein the charge plan includes number
of chargers available at the charging station and the charger maintenance
time.
4. The method as claimed in claim 1, wherein the - state S୲ 5 ഥ at time-step 𝑡 is
defined as a graph ꓖt = (ꓦഥ, Ɛ୲), wherein the set of nodes is denoted by ꓦഥ =
ꓦ ∪ O, wherein V denotes the plurality of vehicles in the fleet and O denotes
the plurality of chargers at the charging station.
10 5. The method as claimed in claim 1, wherein the action 𝐴𝑡 is the vehicle to
charging operation assignment at time-step 𝑡 and for each of the feasible
actions {𝑣𝑖, 𝑜𝑗𝑘 } at 𝑡, the corresponding vehicle, operation and state
embeddings are concatenated, and given to a policy network to get a priority
index of the actions selected at state S୲
ഥ .
15
6. The method as claimed in claim 1, wherein the GNN comprises of a twostage embedding process to efficiently encode the varying size heterogeneous
graph G𝑡 and obtain a fixed-dimensional embedding of size d, ሬሬ⃗ and wherein
the two-stage embedding includes (i) a vehicle node embedding, and (ii) an
20 operation node embedding.
7. A system (100), comprising:
a memory (102) storing instructions;
one or more communication interfaces (106); and
25 one or more hardware processors (104) coupled to the memory (102) via the
one or more communication interfaces (106), wherein the one or more
hardware processors (104) are configured by the instructions to:
receive, via a day-ahead planning,
a trip plan of a fleet of the EVs, wherein the fleet of the EVs comprising
30 a plurality of assigned vehicles for a day-ahead trip and a plurality of
available vehicles at the EV charging station,
32
a charge plan of the EV charging station, and
price of electricity of a day ahead market;
prepare, a day-ahead charge schedule for each of the plurality of assigned vehicles
via the day-ahead planning module, wherein preparing the day-ahead schedule
5 comprises:
selecting, from the trip plan, the plurality of vehicles assigned for the dayahead trip;
assessing a state S୲
ഥ of the system at each time-step by scheming the plurality
of assigned vehicles and a plurality of chargers available at the EV charging
10 station for charging the plurality of assigned vehicles;
allocating, by a learning agent (LA3_D), the plurality of assigned vehicles to
the plurality of chargers available and observing the state 𝑆𝑡 of the system at
each time-step , wherein an allocation of a vehicle among the plurality of
assigned vehicles to a charger among the plurality of chargers is an action
15 At of the learning agent;
iteratively, transitioning to a next time-step, and continuing allocating the
plurality of assigned vehicles to the plurality of chargers; and
receiving a reward for the action of the learning agent, wherein the reward
trains a Graph Neural Network (GNN) to generate the day ahead schedule for
20 each of the plurality of assigned vehicles;
prepare the intra-day schedule by an intra-day planning module, wherein preparing
the intra-day schedule comprises:
receiving the day-ahead schedule of each of the plurality of assigned vehicles
generated by the day-ahead planning module;
25 receiving price of electricity of an intra-day market;
identifying available time slots for the intra-day schedule from the day-ahead
schedule at each time-step by executing a greedy algorithm to determine
feasibility of possible assignment of suitable charger to the available vehicle
to derive charger-EV pairing;
33
scanning through a plurality of infeasible conditions and identifies feasible
time slots for the assignment of a charger from the plurality of chargers to the
available vehicle;
discharging the available vehicle at the charger from the plurality of chargers
5 and trading back the energy in the intra-day market;
applying a priority function to prioritize the plurality of available EVs
wherein the priority function is a weighted sum of an individual priority
components; and
iteratively, prioritizing the vehicle allocation until all the available vehicles
10 for the intra-day discharging gets the charger for trading back the energy by
way of discharging; and
score the bidirectional charging by obtaining a cost incurred by the day-ahead
planning module in charging the plurality of vehicles in the day-ahead market and
profit generated by the intra-day planning module by discharging the plurality of
15 vehicles in the intra-day market.
8. The system as claimed in claim 7, wherein the trip plan includes vehicle
specification, number of trips assigned, distance of each trip and the status of
power available in the vehicle.
20
9. The system as claimed in claim 7, wherein the charge plan includes number
of chargers available at the charging station and the charger maintenance
time.
10. The system as claimed in claim 7, wherein the - state S୲ 25 ഥ at time-step 𝑡 is
defined as a graph ꓖt = (ꓦഥ, Ɛ୲), wherein the set of nodes is denoted by ꓦഥ =
ꓦ ∪ O, wherein V denotes the plurality of vehicles in the fleet and O denotes
the plurality of chargers at the charging station.
30 11. The system as claimed in claim 7, wherein the action 𝐴𝑡 is the vehicle to
charging operation assignment at time-step 𝑡 and for each of the feasible
34
actions {𝑣𝑖, 𝑜𝑗𝑘 } at 𝑡, the corresponding vehicle, operation and state
embeddings are concatenated, and given to a policy network to get a priority
index of the actions selected at state S୲
ഥ .
5 12. The system as claimed in claim 7, wherein the GNN comprises of a two-stage
embedding process to efficiently encode the varying size heterogeneous graph
G𝑡 and obtain a fixed-dimensional embedding of size d, ሬሬ⃗ and wherein the twostage embedding includes (i) a vehicle node embedding, and (ii) an operation
node embedding.