SYSTEM AND METHOD FOR SESSION RESILIANCY AT
GEO-REDUNDANT GATEWAYS
CROSS REFERENCE TO RELATED APPLICATIONS
This patent application claims priority to U.S. Provisional Patent
Application Serial No. 61/454,328, entitled GEO-REDUNDANCE IN A SERVING
GATEWAY, filed March 18, 201 1, which is herein incorporated by reference in its
entirety.
This patent application is related to simultaneously filed U.S. Patent
Applications No. (Attorney Docket No. ALU/809350), entitled
SYSTEM AND METHOD FOR SESSION RESTORATION AT GEOREDUNDANT
GATEWAYS, and No. (Attorney Docket No.
ALU/809431 ) , entitled SYSTEM AND METHOD FOR FAILOVER HANDLING AT
GEO-REDUNDANT GATEWAYS, both of which are herein incorporated by
reference in their entireties.
FIELD OF THE INVENTION
The invention relates generally to managing network resources and, more
specifically but not exclusively, adapting operations associated with a system
router such as a Serving Gateway (SGW).
BACKGROUND
A wireless network, illustratively a Long Term Evolution (LTE) network,
may comprise groups of mobile telephones or other user equipment (UE)
communicating with one or more eNodeBs, which communicate with one or more
Serving Gateways (SGWs), which communicate with a Packet Data Network
(PDN) Gateway (PGW), which communicates with fixed networks such as IP
Multimedia Subsystem (IMS) access networks or core networks. Additionally, the
LTE network includes various network elements such as Mobility Management
Entities (MMEs), a Policy and Charging Rules Function (PCRF), a network
management system (NMS) and so on.
In a failure scenario where a Serving Gateway (SGW) loses connectivity
with other nodes in the network (e.g., due to network disconnection, power
failure, or even a triggered behavior based on partial failures), a backup SGW
must take over operations. This should be accomplished in an intelligent manner
to avoid unreasonable spiking in resource utilization while continuing to meet
reasonable user/subscriber expectations.
When the primary SGW fails, all of the packets destined for the failed
SGW are dropped. In addition, the MME will lose path management states
associated with the failed SGW and will need clean up all its active sessions.
This will cause the active UEs to re-connect to the network through the backup
SGW or an alternate SGW. Similarly, the PGW will lose its path management
state to the SGW, and will clean up session state towards the IMS subsystem (all
UEs are active on the PGW and into the network). With the active UEs reattaching,
their state will be restored to the PGW and the IMS subsystem.
However, since the majority of UEs are idle at any given moment, at the
time of the primary SGW failure the MME will not reach out to the idle UEs to
clean up their sessions. This is because the first step to cleaning up the idle UE
sessions is to page each of the idle UEs, which is prohibitively expensive. If an
idle UE is not cleaned up, there is no way for a network-initiated call to reach it
because no network entity knows where in the network it is currently located.
Moreover, the IMS sub-system cannot find the UE and no entity is actively
encouraging the UE to re-identify itself. The consequence is significant as the
UE will not be reachable for up to an hour or two, depending on various timers.
This is unacceptable for users.
BRIEF SUMMARY
Various deficiencies of the prior art are addressed by the present invention
of a method, system and apparatus for managing a backup service gateway
(SGW) associated with a primary SGW such as configured in a geo-redundant
pair. One embodiment provides a backup SGW operating in a slave mode
periodically receiving from the primary SGW at least a portion of corresponding
UE session state information; and in response to a failure of the primary SGW,
entering a master mode of operation and assuming management of IP addresses
and paths associated with the primary SGW; and in response to receiving control
or data plane traffic associated with a UE, generating a Downlink Data
Notification (DDN) message adapted to inform an MME that the UE is in a live
state.
BRIEF DESCRIPTION OF THE DRAWINGS
The teachings of the present invention can be readily understood by
considering the following detailed description in conjunction with the
accompanying drawings, in which:
FIG. 1 depicts an exemplary communication system benefiting from an
embodiment;
FIG. 2 depicts an exemplary Serving Gateway (SGW) router architecture
suitable for use in communication system of FIG. 1;
FIG. 3 depicts a flow diagram of a session state backup method according
to an embodiment;
FIG. 4 depicts a flow diagram of a resilient session state restoration
method according to an embodiment;
FIG. 5 depicts a flow diagram of a method providing resilient session state
restoration in response to an ingress control signal on S 11 or S5 for an idle UE;
FIG. 6 depicts a flow diagram of a method providing resilient session state
restoration in response to an ingress control signal on S 11 or S5 for an active
UE;
FIG. 7 depicts a flow diagram of a method providing resilient session state
restoration in response to an ingress data signal on S 1-u for an active UE;
FIG. 8 depicts a flow diagram of a method providing resilient session state
restoration in response to an ingress control signal on S 11 or S5 for an active
UE;
FIG. 9 depicts a flow diagram of a method providing resilient session state
restoration in response to an ingress data signal on S5 for an idle UE; and
FIG. 10 depicts a high-level block diagram of a general purpose computer
suitable for use in performing the functions described herein with respect to the
various embodiments.
To facilitate understanding, identical reference numerals have been used,
where possible, to designate identical elements that are common to the figures.
DETAILED DESCRIPTION OF THE INVENTION
The invention will be primarily described within the context of a Long Term
Evolution (LTE) network in which Service Gateway (SGW) redundancy such that
both active and idle subscribers are transitioned from a failed SGW to a backup
SGW.
Although primarily depicted and described herein within the context of
providing management and backup functions within a 4G LTE wireless network,
it will be appreciated that the management and backup functions depicted and
described herein may be utilized within other types of wireless networks (e.g., 2G
networks, 3G networks, WiMAX, etc.), wireline networks or combinations of
wireless and wireline networks. Thus, the various network elements, links and
other functional entities described herein with respect to an LTE network may be
broadly construed to identify corresponding network elements, links and other
functional entities associated with various other types of wireless and wireline
networks.
Part of the invention rests in the recognition of the inventors that the
dramatically increasing size of wireless networks in particular leads to specific
network management problems that are not properly addressed by existing
solutions. In particular, it was recognized by the inventors existing solutions
scaled poorly and failed to address the reality that subscriber equipment may be
in various steady states (such as Idle or Active states), or in various transitional
states (such as progressing between call flows, moving between an Idle state
and an Active state, engaged in a handover from one eNodeB to another,
creating a dedicated bearer, destroying a PDN session and so on). Furthermore,
subscriber traffic may be flowing to or from the subscriber in any one of the study
or transitional states.
FIG. 1 depicts an exemplary wireless communication system including
management and backup / protection functions according to an embodiment.
Specifically, FIG. 1 depicts an exemplary wireless communication system 100
that includes a plurality of User Equipment (UEs) 102, a Long Term Evolution
(LTE) network 110, IP networks 130, and a network management system (NMS)
140. The LTE network 110 supports communications between the UEs 102 and
IP networks 130. The MS 140 is configured for supporting various management
functions for LTE network 110. The configuration and operation of LTE networks
will be understood by one skilled in the art.
The exemplary UEs 102 are wireless user devices capable of accessing a
wireless network, such as LTE network 110. The UEs 102 are capable of
supporting control signaling in support of the bearer session(s). The UEs 102
may be mobile phones, personal digital assistants (PDAs), computers, tablets
devices or any other wireless user device.
The exemplary LTE network 110 includes, illustratively, two eNodeBs 111
and 2 (collectively, eNodeBs 111) , two Serving Gateways (SGWs) 1 2 and
1122 (collectively, SGWs 112), a Packet Data Network (PDN) Gateway (PGW)
113, a Mobility Management Entity (MME) 114, and a Policy and Charging Rules
Function (PCRF) 115. The eNodeBs 111 provide a radio access interface for
UEs 102. The SGWs 112, PGW 113, MME 114, and PCRF 115, as well as other
components which have been omitted for purposes of clarity, cooperate to
provide an Evolved Packet Core (EPC) network supporting end-to-end service
delivery using IP.
The eNodeBs 111 support communications for UEs 102. As depicted in
FIG. 1, each eNodeB 111 supports a respective plurality of UEs 102. The
communication between the eNodeBs 111 and the UEs 102 is supported using
LTE-Uu interfaces associated with each of the UEs 102.
The SGWs 112 support communications for eNodeBs 111 using,
illustratively, respective S 1-u interfaces between the SGWs 112 and the
eNodeBs 111. The S 1-u interfaces support per-bearer user plane tunneling and
inter-eNodeB path switching during handover.
As depicted in FIG. 1, SGW 11 supports communications for eNodeB
1111 and SGW 1122 supports communications for eNodeB 1112. In various
protection/backup embodiments, SGW 1 2 is also capable of supporting
communications for eNodeB 1112 and SGW 1122 is also capable of supporting
communications for eNodeB 11 .
The PGW 113 supports communications for the SGWs 112 using,
illustratively, respective S5/S8 interfaces between PGW 113 and SGWs 112 . The
S5 interfaces provide functions such as user plane tunneling and tunnel
management for communications between PGW 113 and SGWs 112, SGW
relocation due to UE mobility, and the like. The S8 interfaces, which may be
Public Land Mobile Network (PLMN) variants of the S5 interfaces, provide inter-
PLMN interfaces providing user and control plane connectivity between the SGW
in the Visitor PLMN (VPLMN) and the PGW in the Home PLMN (HPLMN). The
PGW 113 facilitates communications between LTE network 110 and IP networks
130 via an SGi interface.
The MME 114 provide mobility management functions in support of
mobility of UEs 102. The MME 114 supports the eNodeBs 111 using,
illustratively, respective S 1-MME interfaces which provide control plane protocols
for communication between the MME 114 and the eNodeBs 111.
The PCRF 115 provides dynamic management capabilities by which the
service provider may manage rules related to services provided via LTE network
110 and rules related to charging for services provided via LTE network 110 .
As depicted and described herein with respect to FIG. 1, elements of LTE
network 110 communicate via interfaces between the elements. The interfaces
described with respect to LTE network 110 also may be referred to as sessions.
The LTE network 110 includes an Evolved Packet System/Solution (EPS). In one
embodiment, the EPS includes EPS nodes (e.g., eNodeBs 111, SGWs 112,
PGW 113, MME 114, and PCRF 115) and EPS-related interconnectivity (e.g., the
S* interfaces, the G* interfaces, and the like). The EPS-related interfaces may
be referred to herein as EPS-related paths.
The IP networks 130 include one or more packet data networks via which
UEs 102 may access content, services, and the like.
The MS 140 provides management functions for managing the LTE
network 110. The MS 140 may communicate with LTE network 110 in any
suitable manner. In one embodiment, for example, MS 140 may communicate
with LTE network 110 via a communication path 141 which does not traverse IP
networks 130. In one embodiment, for example, MS 140 may communicate with
LTE network 110 via a communication path 142 which is supported by IP
networks 130. The communication paths 141 and 142 may be implemented
using any suitable communications capabilities. The MS 140 may be
implemented as a general purpose computing device or specific purpose
computing device, such as described below with respect to FIG. 10.
FIG. 2 depicts an exemplary Serving Gateway (SGW) router architecture
suitable for use in communication system of FIG. 1. Specifically, FIG. 1 depicts a
router 200 operating as a SGW such as SGW 112 depicted above with respect to
FIG. 1. The router 200 communicates with various network elements (not
shown) via a network 110, such as the network 110 depicted above with respect
to FIG. 1. It will be appreciated by those skilled in the art that the specific
topology depicted herein with respect to the SGW 200 may be modified while
maintaining the basic SGW functionality.
The SGW 200 is depicted as including a plurality of input output (I/O)
cards 2 10-1 , 210-2 and so on up to 2 10-N (collectively I/O cards 2 10), a switch
fabric 220 and a control module 230. The control module 230 controls the
operation of the I/O cards 2 10 and switch fabric 220 by respective control signals
CONT. The control module 230 also performs various SGW functions as
described herein.
Each of the I/O cards 2 10 includes a plurality of ingress ports, egress
ports, controllers and so on (not shown) which operate to convey packets
between the network 110 and the switch fabric 220. Packets received at a
particular ingress port of an I/O card 2 10 may be conveyed to the switch fabric
220 or back to the network 110 via an egress port of the same I/O card 2 10 or a
different I/O card 2 10 . Routing of packets via the I/O cards 2 10 is accomplished
in a standard manner according to routing data provided by the control module
230
The switch fabric 220 may comprise any standard switch fabric such as
electrical, optical, electro-optical, MEMS and the like.
The control module 230 receives configuration data, routing data, policy
information and other information pertaining to various SGW operational and
management functions from a network manager (not shown), such as the
network management system (NMS) 140 discussed above with respect to FIG.
1. The control module 230 also provides configuration data, status data, alarm
data, performance data and other information pertaining to operational and
management functions to the network manager.
The control module 230 comprises an I/O module 231 , a processor 232
and memory 233. The memory 233 is depicted as including software modules,
instantiated objects and the like to provide a SGW manager 233SGWM, a
backup and recovery manager 23BARM, session data 233SD, router data
233RD and other functions/data 2330. The control module 230 may be
implemented as a general purpose computing device or specific purpose
computing device, such as described below with respect to FIG. 9 .
The SGW manager 233SGWM operates to manage the various Serving
Gateway (SGW) functions as known to those skilled in the art and further
described herein.
The backup and recovery manager 23BARM operates to manage the
backup and recovery functions described herein with respect to the various
embodiments. For example, such backup and recovery functions may be
different depending upon whether the SGW is operating as a primary or active
SGW, a secondary or backup SGW, or both. Generally speaking, the various
embodiments contemplate the transport and storage at a backup SGW of some
or all of the session related data associated with user equipment or mobile
devices for subscribers supported by the active SGW, such that rapid recovery of
both active and idle sessions may be provided to such subscribers.
The session data 233SD comprises session data associated with user
equipment or mobile devices for subscribers. If the SGW is operating as a
primary or active SGW, then the session data 233SD may comprise information
supporting the user equipment or mobile devices for subscribers forth by the
primary or active SGW. If the SGW is operating as a secondary or backup SGW,
then the session data 233SD may comprise a portion of the session data
associated with one or more primary or active SGWs supported by the backup
SGW.
The routing data 233RD comprises routing information associated with the
packet or traffic flows to be processed by the SGW, such as for processing
packet or traffic flows received at ingress ports that are to be routed toward
appropriate egress ports within the context of basic routing functions of the SGW.
The routing data 233RD may include routing tables, protection or fault recovery
information and so on.
The other functions/data 2330 comprises programs, functions, data
structures and the like operative to perform the various functions described
herein with respect to standard SGW operations as well as SGW operations
according to various embodiments which are not explicitly attributed to other
management or data entities.
Backup SGW Selection and Geo-Redundant Pairing
The MME may be alerted to the failure of an SGW by nodes or network
elements adjacent to the failed SGW. These adjacent nodes or network elements
may independently take corrective action to re-establish connectivity through a
previously assigned backup SGW, through a backup SGW identified by the
MME, or through some other routing means.
In various embodiments, a specific backup SGW is assigned to one or
more primary or active SGWs within the network by, illustratively, the network
management system (NMS). A selected backup SGW may be the SGW most
geographically proximate to a primary or active SGW. Moreover, some primary or
active SGWs may operate as backup SGWs to other primary or active SGWs.
In various embodiments, a specific backup SGW is selected after a failure
of a primary or active SGW. In these embodiments the backup SGW may be
selected based on various criteria, including some or all of geographic proximity
to the failed SGW, DNS response criteria, path management verification criteria,
session loading, and various other criteria. In various embodiments, the selection
of a backup SGW is made by the MME from, for example, a pool of SGWs
available to the particular MME which is drawn upon to provide a backup SGW in
the event of one of the pooled SGWs fails.
In one embodiment, SGWs 112 are geographically proximate each other
such that may be used to form a geo-redundant pair of SGWs. Generally
speaking, traffic and data flows from UEs 102 of a particular eNodeB 111 are
primarily routed to the PGW 113 via a particular SGW, the particular SGW
functioning as a primary or working SGW with respect to the voice and data
traffic from the eNodeB. That is, one of the SGWs is configured as a working or
primary node while the other is configured as a protection or backup node. In a
normal state of operations (i.e., no failure), the working node operates to process
calls flows and data flows from, illustratively, a plurality of eNodeBs while the
protection node operates to back up the working node in case of a failure of the
working node.
In one embodiment, first SGW 112i operates as a primary or working
SGW with respect to voice and data traffic from the first eNodeB 111 , while the
second SGW 1122 operates as a secondary or backup SGW with respect to
voice and data traffic from the first eNodeB 111 .
In one embodiment, second SGW 1122 operates as a primary or working
SGW with respect to voice and data traffic from the second eNodeB 1112, while
the first SGW 112 operates as a secondary or backup SGW with respect to
voice and data traffic from the second eNodeB 1112.
In one embodiment, the first and second SGWs 112 operate as primary or
working SGWs with respect to voice and data traffic from their own one (or more)
respective eNodeBs, and secondary or backup SGWs with respect to voice and
data traffic from the one (or more) eNodeBs associated with the other SGW.
Various embodiments discussed herein are directed toward rapidly
restoring sessions, voice and data traffic, and various other management
information or contexts associated with such UEs 102 in response to a failure of
the primary working SGW. In particular, to provide rapid and efficient
protection/backup functions between the SGWs, various embodiments
contemplate several levels of redundant storage of session state information
associated with user equipment to enable rapid transition to the backup SGW
without significant impact to subscriber experience. In particular, session state
information redundancy enables both the MME 114 and PGW 113 to maintain
state information for idle subscriber UE such that active sessions may be rapidly
reestablished and the subscriber experience and enhanced.
Resilient Restoration of User Sessions at a Backup SGW
Within the context of "transferring" support of UEs 102 and/or eNodeBs
111 from a failed or failing SGW 112 to a backup SGW 112, full survivability of
user sessions may not always be achievable. However, the various embodiments
discussed herein are adapted to promote fast restoration of services utilizing ondemand
restoration of services while maintaining a low synchronization overhead
between active and backup SGWs.
On-demand restoration of services is where a backup SGW only
processes sessions that are requesting activity. On an SGW in active use, there
may be a large number of idle sessions that do not require immediate restoration.
Over a period of time, these sessions become active, and at that time, it
becomes necessary to reconnect those sessions. With this just-in-time
restoration approach, the network is not overburdened with signaling overheads
for sessions that are not active.
Low synchronization overhead is where data synchronization operations,
session state updates and the like between a primary SGW and its backup SGW
are kept to a minimum. Typically, there is significant traffic between an active
SGW and a MME directed toward various functions such as keeping track of
sessions that are becoming active, going idle, or handing over from one eNodeB
to another. These activities happen so frequently that it is a significant burden to
communicate all these changes between the active and backup SGWs.
Generally speaking, the various embodiments utilize only the knowledge of which
sessions existed on the active SGW at the time of the failure.
The various methodologies and techniques described herein provide a
mechanism by which both control and data planes of user sessions on a primary
SGW may be restored via a backup SGW in response to a failure of the primary
SGW. Various embodiments of the session restoration mechanism described
herein address three components; namely, ( 1 ) IP address survivability, (2) path
management continuity, and (3) session restoration.
IP address survivability is the process of ensuring that network elements
connected to the backup SGW continue to be able to access the IP address(es)
of the failed SGW throughout the transfer process to the backup SGW.
In some embodiments, IP address survivability is implemented using a
virtual IP address, such as through the use of VRRP (layer 2 approach) or
anycast IP address (layer 3 approach).
In some embodiments, IP address survivability is implemented by having
the active and backup SGW advertise the same IP address, wherein the active
SGW advertising the IP address with a highly preferred metric while the backup
SGW advertises the IP address with a non-preferred or "poisoned" metric. In
these embodiments, any network elements choosing between the advertise IP
addresses will always choose that of the active SGW since this address is highly
preferred. When the active SGW fails and the only valid IP address is that
advertised by the backup SGW, then network elements will choose the backup
SGW to send all data plane and control plane traffic.
Path management continuity is the process of ensuring that network
elements with path management to the failed SGW maintain continuity through
the transfer process to the backup SGW. In some embodiments, the active SGW
engages in a periodic path management relationship with various other network
elements (e.g., MME, eNodeB, PGW). Each path management instance is
identified by a Restart Counter that is sent in an Echo Request. If this number
changes, it signifies that the network element has been restarted (because of a
reboot or an administrative action that brought the network element down and
back up).
When the backup SGW takes over, it receives path management Echo
Requests and responsively transmits Echo Replies. In addition, the backup
SGW sends Echo Requests and field Echo Replies. For every peer, the backup
SGW will know the received Restart Counter at the active SGW. In this manner,
if a Restart Counter from a peer changes then the backup SGW may
responsively clean up sessions associated with that peer. In various
embodiments, when the backup SGW sends Echo Requests it will also send the
Restart Counter that the active SGW used to send. In this manner, peers of the
active SGW will not clean up sessions.
Resilient session restoration is the process of identifying a session that is
down or inactive, and restoring the identified session as soon as possible through
the backup SGW. In resilient session restoration, the active SGW conveys
enough information about each UE so that the backup SGW can restore both
control and data planes associated with the UE session. This means that the
backup SGW is not only aware of the active SGWs UEs, but processes control
messages for those UEs as well as forwarding data plane traffic for those UEs.
Resilient session restoration within the context of, illustratively, an LTE
network may provide active processing times of UE within 10 ms while
minimizing the impact of network elements of a loss of a primary SGW. Various
techniques also provide low synchronization overhead between primary and
backup SGWs, no change to idle UE processing, maintaining UE IP address for
active and idle UEs and maintaining the charging session.
A resilient session restoration phase executes whenever there is activity
on a session. The goal is to recover information to establish a downlink path to
the UE. This implies that the network elements involved in the signaling and
maintenance of the session continue to hold on to the sessions so they can
communicate with their peers. It is noted that a UE's session state, apart from its
downlink Tunnel Endpoint Identifier (DL TEID), generally stays constant. In
effect, the Idle UE session state is a relatively invariant portion of the session
state of a UE. Thus, by keeping the UE in Idle Mode the main restoration effort is
directed toward the downlink TEID.
After an active SGW failure scenario, all traffic, whether data plane or
control plane, will be routed to the backup SGW. When data traffic arrives on the
S5-u interface of the backup SGW, it will arrive on a tunnel having a tunnel
endpoint identifier (TEID) that has been programmed in the data plane of the
backup SGW. Since the UE state is maintained as Idle Mode, the normal
behavior of the SGW is to transmit a Downlink Data Notification message to
inform the MME to page the UE, and to return the downlink TEID and eNodeB for
the UE. If the UE is actually in Idle Mode, then the MME will page the UE and reestablish
the downlink path. If the UE is active, then the MME does not have to
page the UE, but will instead provide to the SGW the existing downlink TEID for
the eNodeB that the UE is attached to. For data arriving on the S1-u interface of
the backup SGW, the uplink data path has already been programmed and the
data forwarding can be completed. It is noted that this operation is available to
both and active UE and an idle UE. Downlink return traffic will trigger the
Downlink Data Notification as discussed above.
If a control message arrives on the S5-c interface, then the backup SGW
will forward the message. If the MME does not send Modify Bearer Requests,
then the SGW is aware that the UE was in active state and sends a Downlink
Data Notification to the MME in order to trigger the MME to send a Modify Bearer
Request with the downlink TEIDs. If the UE was idle, then the MME will
automatically send the Modify Bearer Requests. If a control message arrives
from the MME, then it is either bringing the UE out of idle (the SGW doesn't have
to do anything), sending an idle mode TAU (the SGW doesn't have to do
anything), or it is a callflow that requires the UE not to be in idle state (the SGW
throws the message away and sends a Downlink Data Notification, eliciting the
Modify Bearer Request from the MME).
In various embodiments, the TEID spaces used by the active and backup
SGWs are disjoint to ensure that no collisions occur between what has already
been programmed on the backup SGW and the UEs that it is backing up.
Generally speaking, the restoration procedure uses information
communicated between active and backup SGWs, such as ( 1 ) path management
Restart Counters and IP addresses of each peer known to the active SGW; and
(2) all UE session state information known to the active SGW, except for the
downlink TEIDs to the various eNodeBs.
FIG. 3 depicts a flow diagram of a session state backup method according
to one embodiment. The method includes portions adapted for use in a primary
SGW and portions adapted for use in a backup SGW, such as the SGWs 112
described above with respect to FIGS. 1-2.
Generally speaking, the method 300 of FIG. 3 is adapted to store at a
backup SGW enough information about each UE 102 supported by an active
SGW to enable the backup SGW to take at least limited actions, such as
identifying sessions that are down or inactive, and restoring the identified
sessions as soon as possible through the backup SGW. The active SGW
conveys enough information about each UE to enable the backup SGW to
restore both the control and data planes associated with the UE sessions. In this
manner, the various network elements involved in the signaling and maintenance
of the UE session will continue to view the session as alive and communicate
with their peers accordingly.
At step 310, at least one alternative or backup SGW is determined for a
primary SGW. That is, for one or more of the SGW's within a network operating
as a primary or active SGW, at least one backup SGW is determined. Referring
to box 3 15, the backup SGW may be determined with respect to location,
configuration, capacity or other factors associated with the primary and/or backup
SGW. The determination may be made by inter-SGW negotiation, such as within
the context of a discovery, configuration or optimization process among
neighboring SGWs. The determination may also be made by a network
manager, such as the network manager 140 described above with respect to
FIG. 1. Other entities and/or determination methodologies may be used.
In various embodiments, determination of an alternate or backup SGW for
a primary SGW is performed automatically based on one or more of the following
selection criteria: DNS response times, path management verification times,
session loading and the like. In various embodiments the criteria is also used by
the MME to select a new primary SGW for new call setups.
At step 320, the active and backup SGWs are initialized as needed, the
primary and backup roles are allocated among the SGWs, communications
between the primary and backup SGWs are established, and at least the primary
SGW begins to advertise its IP address.
Referring to box 325, the processes at step 320 includes some or all of
establishing an inter-SGW communication channel (ISCC) with an inter-SGW
communication protocol (ISCP) for conveying the events needs to be established
between the active and backup SGWs, defining one or more IP survivability
mechanisms to be used, defining the relevant events that will be conveyed from
the primary SGW to the backup SGW, determining the range of Tunnel Endpoint
Identifiers (TEIDs) that the active SGW will use, sharing peer address and restart
counter information and the like.
In various embodiments, during initialization the active SGW identifies
itself and requests identification of the backup SGW. After verifying that the
peering is between properly configured SGWs, the active SGW declares that it is
going to take the active role. When peering is agreed upon, the active SGW
begins to advertise its IP addresses for the S 1-u, S 11, S5-c and S5-u interfaces.
In normal operation, the active SGW "owns" the IP address on the S 11, S5-c, S5-
u and S 1-u interfaces. The active SGW also shares a TEID range that it will use,
so that the backup SGW can refrain from using that range.
In various embodiments, the active SGW shares with the backup SGW a
local Restart Counter for the SGW, where only one Restart Counter is
maintained for all protocols within the SGW. In some embodiments, the active
SGW shares a Peer IP address and Restart Counter pair, for each peer that the
active SGW communicates with. In these embodiments, as peers periodically,
go, the active SGW communicates this information to the backup SGW. This
information is typically not change in a stable network.
At step 330, a primary SGW transmits session state information
associated with the mobile devices supported by the primary SGW to at least one
corresponding backup SGW. That is, as it processes UE related messages, the
active SGW identifies session-state relevant events for the UE and
communicates this information to the backup SGW.
Referring to box 335, the session state information may be transmitted at
predetermined intervals such as after a predetermined number of seconds or
minutes. The session state information may also be transmitted after the
occurrence of one or a predetermined number of relevant subscriber events. A
relevant subscriber event comprises, illustratively, a Create Session Event, a
Create Bearer Event, a Delete Session Event, and/or a Delete Bearer Event.
Generally speaking, a relevant subscriber events for purposes of a session
restoration embodiment comprises any event that results in the creation or
destruction of a user session, such as given in the following examples:
Create Session Event: When a new session is created, new control TEIDs
are allocated to the S5 interface towards the PGW. If this is the first session for
the UE, then a new control TEID is allocated to the S 11 interface towards the
MME. At the completion of the Create event, data plane TEIDs for the default
bearer are also assigned for traffic ingressing or egressing the SGW on the S5-u,
and ingressing the SGW on the S 1-u interface from the eNodeB.
Create Bearer Event: When a new dedicated bearer is created, new data
plane S5-u TEIDs are allocated for traffic ingressing or egressing the SGW, and
ingressing the SGW on the S 1-u interface.
Delete Session Event: when a session is deleted, the session needs to be
deleted from the backup SGW, and deprogrammed from the data plane.
Delete Bearer Event: When a dedicated bearer is deleted, the bearer
context needs to be deleted from the backup SGW, and deprogrammed from the
forwarding plane.
The frequency of change is based on the frequency of setup/teardown of
PDN sessions and dedicated bearers. However, this is not as frequent as the
events that arrive at the SGW that modify the state of the sessions and bearers.
The state information primarily comprises session state of the UE that does not
change significantly during the life of the session.
In some embodiments, to avoid an incorrect assessment by the backup
SGW that the active SGW has indeed failed, the active SGW periodically sends
keep alive messages to the backup SGW in case there are few relevant events
to convey.
At step 340, at each backup SGW the session state information
transmitted from one or more primary SGWs supported by the backup SGW is
stored. Referring to box 345, in one embodiment, the stored session data is
sufficient to re-create control and data planes for UE sessions.
FIG. 4 depicts a flow diagram of a session state restoration method
according to one embodiment. Specifically, FIG. 4 depicts a method 400 adapted
for use in a gateway operating as an alternate or backup gateway, such as an
alternate or backup SGW in an LTE network having stored thereon session state
information such as described above with respect to FIG. 3 .
At step 410, a gateway such as a SGW operating as a backup SGW is
initialized and a communications path to a primary SGW established, illustratively
in accordance with steps 310-325 of the method 300 as described above with
respect to FIG. 3.
At step 420, the backup gateway receives and stores UE state information
pertaining to UE supported by the active SGW until such time as a failure of the
primary SGW is indicated. Referring to box 425, a primary SGW failure may be
indicated via an explicit failure indication, a timeout of a neighboring node alive
indicator, a peer counter timeout and the like. Such an indication may be due to
an actual failure of the primary SGW or some other condition, such as a
maintenance condition associated with the primary SGW or an overload
condition associated with the primary SGW.
At step 430, after a primary SGW failure, the backup gateway assumes
the IP addresses and path management duties of the failed primary gateway. In
addition, the UEs associated with the failed SGW are maintained in an idle state.
Referring to box 435, the backup gateway may begin to advertise IP addresses
with a preferred criteria such that control plane and data plane traffic and packets
are routed to the backup gateway.
At step 440, when data plane or control plane traffic associated with a UE
session arrives at the backup SGW (i.e., backup SGW Ingress Data Plane or
Ingress Control Plane triggered), the backup SGW responsively generates a
Downlink Data Notification (DDN) IMSI message for the MME to restore S1-u
DownLink (DL) paths. Referring to box 445, the backup SGW generates the DDN
(IMSI) message in response to a Network generated control plane or data plane
traffic, UE generated data plane traffic, control messages on S 11, data traffic on
S 1-u or S5-u and so on.
In response to the DDN (IMSI) message, the MME operates to process
idle mode UEs by (a) performing an IMSI paging function; (b) detaching the UE
while providing a selected re-attach code; and c) performing an IMSI attach if the
UE is in idle mode. The MME operates to process active or connected mode UEs
by (a) performing a detach; and (b) performing an IMSI attach. Further, the
backup SGW forwards a Delete Session Request to the PGW, which
responsively cleaned up UE state anomalies via PCRF and IMS. Since the UEs
are maintained in idle state (per step 430), the MME detaches and reattaches the
UEs thus maintaining data plane and 12 plane integrity of the sessions supported
thereby.
Various resilient session state restoration embodiments are illustrated in
more detail below with respect to FIGS. 5-1 0 . It will be appreciated by those
skilled in the art that various figures depicted herein provide only illustrative
embodiments and may be modified in keeping with the various teachings
discussed herein. Each of these FIGS. 5-9 depicts various signals passed among
a UE 102 (e.g., via an eNodeB 111) , a MME 114, a backup SGW 112 and a
PGW 113 for different resilient session state restoration scenarios such as
described herein with respect to, illustratively, FIGS. 1-4.
FIG. 5 depicts a flow diagram of a method providing resilient session state
restoration in response to an ingress control signal on S 11 or S5 for an idle UE.
FIG. 6 depicts a flow diagram of a method providing resilient session state
restoration in response to an ingress control signal on S 11 or S5 for an active
UE.
FIG. 7 depicts a flow diagram of a method providing resilient session state
restoration in response to an ingress data signal on S 1-u for an active UE.
FIG. 8 depicts a flow diagram of a method providing resilient session state
restoration in response to an ingress control signal on S 11 or S5 for an active
UE.
FIG. 9 depicts a flow diagram of a method providing resilient session state
restoration in response to an ingress data signal on S5 for an idle UE.
Thus, state information is synchronized between the master and slave
SGWs/nodes. Generally speaking, state information is synchronized when a
session is created or deleted. Synchronization may occur at other times as well.
The state data synchronized between the SGWs generally comprises the UE
data available or associated with the created or deleted session DL TEID. The
synchronized data tends to be the relatively stable UE session data; that is, the
data that does not tend to change over time as the UE interacts with the network,
whereas the DL TEID will change over time such as when the UE moves
between eNodeBs, base stations and the like. The various embodiments provide
a mechanism whereby the DL TEID is recovered after UE failover between two
SGWs or nodes. In this manner, by avoiding the synchronization of DL TEID
and/or other dynamics data, the resource utilization associated with the state the
restoration process between the two SGWs or nodes is minimized.
Failover may be triggered by the occurrence of a predefined number of
errors, plant maintenance, catastrophic failure or any other reason. On the
occurrence of a failover event, the slave SGW or node acquires address and
path management responsibilities associated with the UE sessions previously
supported by the failed master SGW or nodes. Since the DL TEID is not
synchronized, the slave SGW or node controller will assume that each UE is in
an idle state.
In response to receiving control plane or data plane traffic associated with
a particular UE, the slave SGW or node will send a downlink data notification
(DDN) message adapted to cause the MME to retrieve the UE from its idle state.
If the UE is truly in an idle state, the MME will find or page the UE and enable the
UE to properly respond to the control plane of data plane traffic. If the UE is not
in an idle state, the MME will provide a response to the slave SGW or node
indicating that the UE is not in an idle state as well as providing the DL TEID
associated with the UE. Upon receiving this information, the slave SGW or node
will consider the UE to be active and use the DL TEID to support the UE-related
control plane for data plane traffic. The slave SGW or node will buffer data as
necessary until the UE is page by the MME or otherwise functionally reattached
to the network. Various protocol back pressure mechanism will activate as
necessary to try and preserve session related data. Generally speaking, resilient
session state restoration is a mechanism for preserving connections across a
failure.
The various embodiments described herein generally contemplates that
session state information and/or other information associated with a primary
SGW is stored at a backup SGW for use in implementing a failover mechanism.
However, in various embodiments such information may be stored at multiple
backup SGWs and/or at one or more network elements that are not SGWs. The
stored session state information and/or other information associated with the
primary SGW is retrieved by the backup SGW as part of the failover mechanism.
Various embodiments are modified to use one or more additional
mechanisms for accelerating the resilient session restoration process. One
mechanism for accelerating the session restoration process comprises the use of
a predefined IE on the first few Echo Requests transmitted from SGW to the
MME to indicate that the backup SGW has taken over. The MME responsively
accelerates the recovery of downlink TEIDs of active sessions instead of waiting
for data plane notification on the S5-u, or for control messages on S 11 and S5-c.
One mechanism for accelerating the session restoration process comprises
periodically passing a list of active sessions from the active SGW to the backup
SGW so that the backup SGW can proactively start populating those sessions
with their downlink TEIDs and bring them to active state faster. These and other
mechanisms may be used individually or in any combination to improve or
accelerate the session restoration process.
Synchronizing state information between the primary and backup SGWs,
as well as a frequency of such synchronization depends on various factors, such
as network topology, available resources, desired speed of restoration and the
like.
As an example, a system such as adapted for an LTE network utilizing
General Packet Radio System (GPRS) Tunneling Protocol or GTP may
synchronize some or all of the state information pertaining to GTP information,
path management information and Rf (charging session) related information
associated with various sessions or UE.
State-related GTP information may comprise, illustratively,
UpLink/DownLink (UL/DL) Fully Qualified Tunnel End Point Identifiers (FTEIDs),
control FTEIDs for S 11 and S5-c, data FTEIDs for S1-u and S5-u, ULI and the
like. State-related path management information may comprise, illustratively,
restart counters for S 11, S1-u and S5 and the like. State-related Rf information
may comprise, illustratively, origin state, RAT and the like (roughly 5 12B per
APN).
Synchronization/update frequency may be predetermined, periodic nature
and/or related to the various network events.
In various embodiments, primary and backup SGWs are synchronized
when sessions are created and/or destroyed, such as synchronizing eight
GTP/Rf messages for a session create event, six GTP/Rf messages for a
session destroy event, and two IMCP messages for a session create/destroy
event.
In various embodiments, primary and backup SGWs are synchronized
when bearers are created and/or destroyed, such as synchronizing six GTP/Rf
messages for a better create event, six GTP/Rf messages for a bearer destroy
event, and two synchronization messages for a bearer Create/Destroy event.
In various embodiments, primary backup SGWs are synchronized in
response to or network configuration events like MME relocations, such as
synchronizing four GTP/Rf messages and two synchronization messages for an
MME relocation event.
In various embodiments, dual IP addresses are used on S 11 and S5
where one addresses local in one addresses backup. The local IP addresses are
used to retain existing sessions at the backup SGW, while the backup IP
addresses are used for new sessions, sessions transferred from the failed or
failing primary SGW, control traffic associated with the failed or failing primary
SGW and so on. Specifically, the IP address allocation even to the backup SGW
is split into two portions (which may or may not be the same size), where a first
portion is used for existing data and control plane traffic at the backup SGW, and
a second portion is used for data and control plane traffic associated with the
failed or failing SGW. In this manner, collisions are avoided as session support
moves from the primary SGW to the backup SGW. That is, a backup SGW
becoming active SGW utilizes an active SGW range of IP addresses. In this
manner, conflicts are avoided and support for sessions may be transferred
between SGWs on a per-range basis with respect to their IP addresses. In
various embodiments, failure suppression is employed, one other embodiments it
is not employed.
Thus, two (or more) service gateways (SGWs) or nodes may be operating
as a geo-redundant pair and may be denoted as a primary/backup or
working/protect gateways or nodes. The primary or working SGW or node
operates in a master mode, while the backup or protect SGW(s) or node(s)
operate in a slave mode. In the event of a failure of the primary or working SGW,
the backup or protect SGW(s) begin operating in the master mode. In this
situation, the UEs and their sessions are "failed over" to the slave(s). When the
failed primary or working SGW/node becomes operational again, it may be
necessary to return or failover the newly sessions from the backup or protect
SGW back to the primary or working SGW/node.
In a master mode of operation, the master SGW/node advertises route
data that is preferable to the route data advertised by the slave(s) SGW such that
any node wishing to send traffic will select the master as the route for that traffic.
To ensure that this happens, the slave SGW may for example advertise
"poisoned" route data; namely, route data that would never be selected for use
do to its high cost or some other negative parameter.
FIG. 10 depicts a high-level block diagram of a general purpose computer
suitable for use in performing the functions described herein with respect to the
various embodiments. In particular, the architecture and functionality discussed
herein with respect to the general-purpose computer is adapted for use in each
of the various switching and communication elements or nodes discussed herein
with respect to the various figures; namely, the UEs 102, eNodeBs 111, SGWs
112, PGW1 13, MMEs 114, PCRF1 15, and network management system 140. It
will be appreciated that some of the functionality discussed herein with respect to
describe general purpose computer may be implemented in various network
elements or nodes, and/or a network operations center (NOC) or network
management system (NMS) operative to configure and manage elements within
the network.
As depicted in FIG. 10, system 1000 comprises a processor element 1002
(e.g., a CPU), a memory 1004, e.g., random access memory (RAM) and/or read
only memory (ROM), a packet processing module 1005, and various input/output
devices 1006 (e.g., storage devices, including but not limited to, a tape drive, a
floppy drive, a hard disk drive or a compact disk drive, a receiver, a transmitter, a
speaker, a display, an output port, and a user input device (such as a keyboard,
a keypad, a mouse, and the like)).
It will be appreciated that computer 1000 depicted in FIG. 10 provides a
general architecture and functionality suitable for implementing functional
elements described herein and/or portions of functional elements described
herein. Functions depicted and described herein may be implemented in
software and/or hardware, e.g., using a general purpose computer, one or more
application specific integrated circuits (ASIC), and/or any other hardware
equivalents.
It is contemplated that some of the steps discussed herein as software
methods may be implemented within hardware, for example, as circuitry that
cooperates with the processor to perform various method steps. Portions of the
functions/elements described herein may be implemented as a computer
program product wherein computer instructions, when processed by a computer,
adapt the operation of the computer such that the methods and/or techniques
described herein are invoked or otherwise provided. Instructions for invoking the
inventive methods may be stored in fixed or removable media, transmitted via a
data stream in a broadcast or other signal bearing medium, transmitted via
tangible media and/or stored within a memory within a computing device
operating according to the instructions.
While the foregoing is directed to various embodiments of the present
invention, other and further embodiments of the invention may be devised
without departing from the basic scope thereof. As such, the appropriate scope
of the invention is to be determined according to the claims, which follow.
What is claimed is:
1. A method for managing a backup service gateway (SGW)
associated with a primary SGW, the method comprising:
in a slave mode of operation, periodically receiving from the primary SGW
at least a portion of corresponding UE session state information; and
in response to a failure of the primary SGW, entering a master mode of
operation, said master mode of operation comprising:
assuming management of IP addresses and paths associated with
said primary SGW; and
in response to receiving control or data plane traffic associated with
a UE, generating a Downlink Data Notification (DDN) message adapted to
inform an MME that the UE is in a live state.
2 . The method of claim 1, wherein said slave mode of operation
further comprises advertising route data adapted to avoid selection of said
backup SGW by other network elements, and said master mode of operation
further comprises advertising preferred route data adapted to encourage
selection of said backup SGW by other network elements.
3 . The method of claim 1, wherein said master mode of operation
further comprises reestablishing data plane support for UE sessions using
downlink tunnel endpoint identifier (DL TEID) information received from said
MME.
4 . The method of claim 1, wherein the UE session state information
comprises an identification of each of a plurality of mobile devices supported by
the primary SGW.
5 . The method of claim 1, wherein session state information
associated with a UE is received in response to one or more of a corresponding
Create Session Event, Delete Session Event, Create Bearer Event and Delete
Bearer Event.
6 . The method of claim 1, wherein session state information
associated with a UE is received at predetermined intervals or after a
predetermined number of subscriber events.
7 . The method of claim 1, wherein said backup SGW is associated
with a set of local IP addresses that does not conflict with IP addresses IP
addresses associated with said primary SGW, wherein said backup SGW in said
slave mode managing only said backup SGW local set of IP addresses, and said
backup SGW in said master mode managing both sets of local IP addresses.
8 . An apparatus for use in a service gateway (SGW) adapted to
backup a primary SGW, the apparatus comprising:
a processor configured for managing the backup SGW, the processor
causing the backup SGW to operate in one of a slave mode of operation and a
master mode of operation;
said backup SGW, in said slave mode of operation, periodically receiving
from the primary SGW at least a portion of corresponding UE session state
information and, in response to a failure of the primary SGW, entering a master
mode of operation;
said backup SGW, in said master mode of operation, assuming
management of IP addresses and paths associated with said primary SGW and,
in response to receiving control or data plane traffic associated with a UE,
generating a Downlink Data Notification (DDN) message adapted to cause inform
an MME that the UE is in a live state.
9 . A computer readable medium including software instructions which,
when executed by a processer, perform a method for managing a backup service
gateway (SGW) associated with a primary SGW, the method comprising:
in a slave mode of operation, periodically receiving from the primary SGW
at least a portion of corresponding UE session state information; and
in response to a failure of the primary SGW, entering a master mode of
operation, said master mode of operation comprising:
assuming management of IP addresses and paths associated with
said primary SGW; and
in response to receiving control or data plane traffic associated with
a UE, generating a Downlink Data Notification (DDN) message adapted to
cause inform an MME that the UE is in a live state.
10 . A computer program product, wherein a computer is operative to
process software instructions which adapt the operation of the computer such
that computer performs a method for managing a backup service gateway
(SGW) associated with a primary SGW, the method comprising:
in a slave mode of operation, periodically receiving from the primary SGW
at least a portion of corresponding UE session state information; and
in response to a failure of the primary SGW, entering a master mode of
operation, said master mode of operation comprising:
assuming management of IP addresses and paths associated with
said primary SGW; and
in response to receiving control or data plane traffic associated with
a UE, generating a Downlink Data Notification (DDN) message adapted to
cause inform an MME that the UE is in a live state.