Abstract: The disclosed system captures and processes network packets in a network, utilizing a high-capacity packet capturing unit (212) that directly captures up to two hundred thousand TCP data packets per second from a network interface card. A data aggregation unit (214) with optical taps and packet brokers filters. A native streaming framework streams data to a centralized streaming framework, while a packet soft parser (216-1) extracts user information and manages data during the centralized streaming framework downtime. A packet writer (216-2) enriches and compresses data for optimized storage, and multiple broker topics ensure efficient data handling and management. FIGURE 3
FORM 2
THE PATENTS ACT, 1970 (39 of 1970) THE PATENTS RULES, 2003
COMPLETE SPECIFICATION
(See section 10; rule 13)
TITLE OF THE INVENTION
SYSTEM AND METHOD FOR PACKET CAPTURING AND SOFT PARSING
APPLICANT
JIO PLATFORMS LIMITED
of Office-101, Saffron, Nr. Centre Point, Panchwati 5 Rasta, Ambawadi, Ahmedabad -
380006, Gujarat, India; Nationality : India
The following specification particularly describes
the invention and the manner in which
it is to be performed
RESERVATION OF RIGHTS
[0001] A portion of the disclosure of this patent document contains material,
which is subject to intellectual property rights such as, but are not limited to, copyright, design, trademark, integrated circuit (IC) layout design, and/or trade dress protection, belonging to Jio Platforms Limited (JPL) or its affiliates (herein after referred as owner). The owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all rights whatsoever. All rights to such intellectual property are fully reserved by the owner.
TECHNICAL FIELD
[0002] The present disclosure generally relates to a field of
telecommunications. More particularly, the present disclosure relates to a system and a method for packet capturing and soft parsing.
BACKGROUND
[0003] The following description of related art is intended to provide
background information pertaining to the field of the disclosure. This section may include certain aspects of the art that may be related to various features of the present disclosure. However, it should be appreciated that this section be used only to enhance the understanding of the reader with respect to the present disclosure, and not as admissions of prior art.
[0004] The advent of 5G Standalone (SA) cellular networks has introduced
new paradigms in a communication technology, significantly increasing a speed
and responsiveness of wireless networks. As these advanced networks proliferate,
a management of network resources and an optimization of user equipment (UE)
operations become increasingly critical. A particular area of concern within this
technology sphere is an Automatic Neighbor Relation (ANR) algorithm, an
essential component for maintaining robust and efficient network connectivity.
[0005] Prior arts in this field have focused on facilitating seamless
communication between UEs and the network infrastructure. The ANR algorithm, a sophisticated feature within the Fifth Generation (5G) Standalone (SA) networks, is designed to streamline a management of cell relations by automating a detection and registration of neighboring cells. However, field observations have revealed a persistent issue: the UEs often report multiple measurement reports containing a same Evolved Universal Terrestrial Radio Access Network (EUTRA) Physical Cell Identifiers (PCIs). This repetition leads to inefficiencies both in network operations and a UE performance.
[0006] The problem is twofold. Firstly, duplicate measurement reports can
overwhelm the network's processing capacity, leading to data congestion and potential delays in network response. Secondly, and perhaps more critically, these redundancies have a detrimental impact on a battery life of the UEs. With each superfluous measurement report sent, the UE expends unnecessary energy, which could otherwise be conserved for essential communications.
[0007] Recognizing the limitations of current technologies, there is a clear
need for an improved approach to UE management within the ANR algorithm. The goal is to enhance the network's ability to handle these measurements without compromising the performance and battery efficiency of the UE. This background lays the foundation for the need for advancements in the field of the 5G SA technology, specifically in ae development of more sophisticated algorithms that can address these challenges.
OBJECTS OF THE PRESENT DISCLOSURE
[0008] It is an object of the present disclosure to provide a system and a
method that uses continuous network packet capturing to analyze past sessions or
flows.
[0009] It is an object of the present disclosure to provide a system and a
method that uses on-demand capturing of network packets to trace a particular flow,
monitor a particular subscriber or monitoring a network traffic flow of a complete
application.
[0010] It is an object of the present disclosure to provide a system and a
method that handles of an internet protocol (IP) fragmentation, and a Transmission Control Protocol (TCP) segmentation and Layer 2 (L2) to Layer 7 (L7) protocol level soft parsing for desired search criteria.
LIST OF REFERENCE NUMERALS
100 - Network Architecture
102-1, 102-2…102-N - Users
104-1, 104-2…104-N - User Equipments (UEs)
106 - Network
108 - System
110 – Entity
112 - Centralized Server
202 - One or more processor(s)
204 - Memory
206 - Interface(s)
208 - Processing Engine(s)
212 – Packet capturing unit
214 – Data aggregation unit
216 – Packet streaming unit
216-1 – Packet soft parser
216-2 – Packet writer
218 - Database
1100 - Computer System
1110 - External Storage Device
1120 - Bus
1130 - Main Memory
1140 - Read Only Memory
1150 - Mass Storage Device
1160 - Communication Port(s)
1170 - Processor
SUMMARY
[0011] In an exemplary embodiment, a system for capturing and processing network packets in a network. The system includes a memory and a processing engine coupled to the memory. The processing engine further includes a packet capturing unit configured to capture a plurality of network packets flowing at a predefined rate, based on at least one network function. The processing engine further includes a data aggregation unit configured to cooperate with the packet capturing unit for collecting the plurality of network packets to filter and aggregate the plurality of network packets based on a set of predefined parameters to generate a plurality of filtered network packets. The processing engine further includes a packet streaming unit configured to retrieve the plurality of filtered network packets in response to filtering and aggregating the plurality of network packets. The packet streaming unit further includes a packet soft parser configured to parse at least one information from each of the plurality of filtered network packets to generate a plurality of parsed network packets. The system further includes a database for configured to store the plurality of parsed network packets along with the at least one information. The plurality of parsed network packets along with the at least one information is further transmitted and stored in a centralized database. [0012] In some embodiments, the packet streaming unit further includes a packet writer configured to retrieve the plurality of parsed network packet and the at least one information from the centralized database based on one or more pre-defined topics to generate a compressed file.
[0013] In some embodiments, the packet soft parser is configured to archive the plurality of parsed network packets in a local memory when a connection with the centralized database is lost.
[0014] In some embodiments, the packet soft parser is configured to send the plurality of parsed network packets archived in the local memory to the centralized database upon restoring the connection.
[0015] In some embodiments, the packet capturing unit is configured to capture the
plurality of network packets from the at least one network function by employing
at least one of a port mirroring approach, an optical tapping approach, and a direct
capturing approach.
[0016] In some embodiments, the direct capturing approach includes receiving
plurality of packets from said at least one network function using a network
interface card.
[0017] In some embodiments, the processing engine is further configured to
support a Generic Routing Encapsulation (GRE) and Encapsulated Remote
Switched Port Analyzer (ERSPAN) encapsulation parsing for mirrored traffic in
the port mirroring approach. GRE is a tunneling protocol developed by Cisco that
encapsulates a wide variety of network layer protocols inside virtual point-to-point
connections. GRE is used to create point-to-point connections over an IP network.
ERSPAN is a Cisco-specific protocol that extends the capabilities of the Switched
Port Analyzer (SPAN) and Remote SPAN (RSPAN) by allowing the mirroring of
traffic across Layer 3 networks. ERSPAN encapsulates mirrored traffic in GRE
packets, which are then transported over an IP network to a destination analyzer.
[0018] In some embodiments, the at least one information includes a source
Internet Protocol (IP), a destination IP, a port number, and an application layer
payload marker.
[0019] In some embodiments, the set of predefined parameters includes an Internet
Protocol (IP), a port, and a Virtual Local Area Network (VLAN).
[0020] In some embodiments, each of the plurality of network packets is a
Transmission Control Protocol (TCP) data packet.
[0021] In some embodiments, the pre-defined rate is two hundred thousand packets
per second.
[0022] In some embodiments, the data aggregation unit includes at least one optical
Traffic Access Point (TAP). The TAP may be a hardware device, or a logical unit
used in network monitoring and security applications to capture and replicate
network traffic. The TAPs are used to passively monitor network data without
interfering with the actual traffic flow, ensuring accurate and reliable data capture for analysis.
[0023] In some embodiments, the packet soft parser is configured to support a Hypertext Transfer Protocol Version 2 (HTTP2).
[0024] In some embodiments, the packet writer is configured to write data into each compressed file, wherein each compressed file is corresponding to a specific broker topic.
[0025] In another exemplary embodiment, a method for capturing and processing network packets in a network is disclosed. The method includes capturing a plurality of network packets flowing at a pre-defined rate, based on at least one network function. The method includes collecting the plurality of network packets to filter and aggregate the plurality of network packets based on a set of predefined parameters to generate a plurality of filtered network packets. The method includes retrieve the plurality of filtered network packets in response to filtering and aggregating the plurality of network packets. The method includes parsing at least one information from each of the plurality of filtered network packets to generate a plurality of parsed network packets. The method includes storing the plurality of parsed packets along with the at least one information in a database. The plurality of parsed network packets along with the at least one information is further transmitted and stored in a centralized database.
BRIEF DESCRIPTION OF THE DRAWINGS
[0026] The accompanying drawings, which are incorporated herein, and
constitute a part of this disclosure, illustrate exemplary embodiments of the disclosed methods and systems in which like reference numerals refer to the same parts throughout the different drawings. Components in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the present disclosure. Some drawings may indicate the components using block diagrams and may not represent the internal circuitry of each component. It will be appreciated by those skilled in the art that disclosure of such
drawings includes the disclosure of electrical components, electronic components or circuitry commonly used to implement such components.
[0027] FIG. 1 illustrates an example network architecture for implementing
a system, in accordance with an embodiment of the present disclosure.
[0028] FIG. 2 illustrates an example block diagram of a system configured
for capturing and processing network packets in a network, in accordance with an
embodiment of the present disclosure.
[0029] FIG. 3 illustrates an example block diagram of a system architecture
configured for capturing and processing network packets in a network, in
accordance with an embodiment of the present disclosure.
[0030] FIG. 4 illustrates a block diagram of a managed switch with porting,
in accordance with an embodiment of the present disclosure.
[0031] FIG. 5 illustrates an example system architecture of an Encapsulated
Remote Switched Port Analyzer (ERSPAN), in accordance with an embodiment of
the present disclosure.
[0032] FIG. 6A illustrates a flow chart of one exemplary method of a packet
capturing and a soft parsing, in accordance with embodiments of the present
disclosure.
[0033] FIG. 6B illustrates a flow chart of another exemplary method of a
packet capturing and a soft parsing, in accordance with embodiments of the present
disclosure.
[0034] FIG. 7 illustrates a flow diagram of an exemplary packet capturing
process, in accordance with embodiments of the present disclosure.
[0035] FIG. 8 illustrates an exemplary packet soft parser, in accordance
with embodiments of the present disclosure.
[0036] FIG. 9 illustrates an exemplary packet writer microservice, in
accordance with embodiments of the present disclosure.
[0037] FIG. 10 illustrates a flow diagram of an exemplary method of
capturing and processing network packets in a network, in accordance with
embodiments of the present disclosure.
[0038] FIG. 11 illustrates a computer system in which or with which the
embodiments of the present disclosure may be implemented.
[0039] The foregoing shall be more apparent from the following more
detailed description of the disclosure.
DETAILED DESCRIPTION OF THE INVENTION
[0040] In the following description, for the purposes of explanation, various
specific details are set forth in order to provide a thorough understanding of embodiments of the present disclosure. It will be apparent, however, that embodiments of the present disclosure may be practiced without these specific details. Several features described hereafter can each be used independently of one another or with any combination of other features. An individual feature may not address any of the problems discussed above or might address only some of the problems discussed above. Some of the problems discussed above might not be fully addressed by any of the features described herein. Example embodiments of the present disclosure are described below, as illustrated in various drawings in which like reference numerals refer to the same parts throughout the different drawings.
[0041] The ensuing description provides exemplary embodiments only, and
is not intended to limit the scope, applicability, or configuration of the disclosure. Rather, the ensuing description of the exemplary embodiments will provide those skilled in the art with an enabling description for implementing an exemplary embodiment. It should be understood that various changes may be made in the function and arrangement of elements without departing from the spirit and scope of the disclosure as set forth.
[0042] Specific details are given in the following description to provide a
thorough understanding of the embodiments. However, it will be understood by one of ordinary skill in the art that the embodiments may be practiced without these specific details. For example, circuits, systems, networks, processes, and other components may be shown as components in block diagram form in order not to
obscure the embodiments in unnecessary detail. In other instances, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments.
[0043] Also, it is noted that individual embodiments may be described as a
process that is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe the operations as a sequential process, many of the operations can be performed in parallel or concurrently. In addition, the order of the operations may be re-arranged. A process is terminated when its operations are completed but could have additional steps not included in a figure. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, its termination can correspond to a return of the function to the calling function or the main function.
[0044] The word “exemplary” and/or “demonstrative” is used herein to
mean serving as an example, instance, or illustration. For the avoidance of doubt, the subject matter disclosed herein is not limited by such examples. In addition, any aspect or design described herein as “exemplary” and/or “demonstrative” is not necessarily to be construed as preferred or advantageous over other aspects or designs, nor is it meant to preclude equivalent exemplary structures and techniques known to those of ordinary skill in the art. Furthermore, to the extent that the terms “includes,” “has,” “contains,” and other similar words are used in either the detailed description or the claims, such terms are intended to be inclusive like the term “comprising” as an open transition word without precluding any additional or other elements.
[0045] Reference throughout this specification to “one embodiment” or “an
embodiment” or “an instance” or “one instance” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Thus, the appearances of the phrases “in one embodiment” or “in an embodiment” in various places throughout this specification are not necessarily all referring to the same embodiment.
Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0046] The terminology used herein is to describe particular embodiments
only and is not intended to be limiting the disclosure. As used herein, the singular
5 forms “a”, “an”, and “the” are intended to include the plural forms as well, unless
the context indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other
10 features, integers, steps, operations, elements, components, and/or groups thereof.
As used herein, the term “and/or” includes any combinations of one or more of the associated listed items. It should be noted that the terms “mobile device”, “user equipment”, “user device”, “communication device”, “device” and similar terms are used interchangeably for the purpose of describing the invention. These terms
15 are not intended to limit the scope of the invention or imply any specific
functionality or limitations on the described embodiments. The use of these terms is solely for convenience and clarity of description. The invention is not limited to any particular type of device or equipment, and it should be understood that other equivalent terms or variations thereof may be used interchangeably without
20 departing from the scope of the invention as defined herein.
[0047] As used herein, an “electronic device”, or “portable electronic
device”, or “user device” or “communication device”, “computing device” or “user equipment” or “device” refers to any electrical, electronic, electromechanical, and computing device. The user equipment is capable of receiving and/or transmitting
25 one or parameters, performing function/s, communicating with other user
equipment, and transmitting data to the other user equipment. The user equipment may have a processor, a display, a memory, a battery, and an input-means such as a hard keypad and/or a soft keypad. The user equipment may be capable of operating on any radio access technology including but not limited to Internet
30 Protocol (IP)-enabled communication, Zig Bee, Bluetooth, Bluetooth Low Energy,
11
Near Field Communication, Z-Wave, Wi-Fi, Wi-Fi direct, etc. For instance, the
user equipment may include, but not limited to, a mobile phone, a smartphone,
virtual reality (VR) devices, augmented reality (AR) devices, a laptop, a general-
purpose computer, a desktop, a personal digital assistant, a tablet, a computer, a
5 mainframe computer, or any other device as may be obvious to a person skilled in
the art for implementation of features of the present disclosure.
[0048] Further, the user equipment may also comprise the “processor” or “a
processing engine” includes processing unit, wherein processor refers to any logic circuitry for processing instructions. The processor may be a general-purpose
10 processor, a special purpose processor, a conventional processor, a digital signal
processor, a plurality of microprocessors, one or more microprocessors in association with a DSP core, a controller, a microcontroller, Application Specific Integrated Circuits, Field Programmable Gate Array circuits, any other type of integrated circuits, etc. The processor may perform signal coding data processing,
15 input/output processing, and/or any other functionality that enables the working of
the system according to the present disclosure. More specifically, the processor is a hardware processor.
[0049] As portable electronic devices and wireless technologies continue to
improve and grow in popularity, the advancing wireless technologies for data
20 transfer are also expected to evolve and replace older generations of technologies.
In a field of wireless data communications, a dynamic advancement of various generations of cellular technology are also seen. The development, in this respect, has been incremental in the order of a second generation (2G), a third generation (3G), a fourth generation (4G), and now a fifth generation (5G), and more such
25 generations are expected to continue in the forthcoming time.
[0050] While considerable emphasis has been placed herein on the
components and component parts of the preferred embodiments, it will be
appreciated that many embodiments can be made and that many changes can be
made in the preferred embodiments without departing from the principles of the
30 disclosure. These and other changes in the preferred embodiment as well as other
12
embodiments of the disclosure will be apparent to those skilled in the art from the disclosure herein, whereby it is to be distinctly understood that the foregoing descriptive matter is to be interpreted merely as illustrative of the disclosure and not as a limitation.
5 [0051] Embodiments herein relate to a system and a method for capturing
and processing network packets in the network. The capturing of the network packets is done based on at least one network function by employing at least one of a port mirroring approach, an optical tapping approach, and a direct capturing approach.
10 [0052] FIG. 1 illustrates an exemplary network architecture (100) in which
or with which a system (108) for capturing and processing network packets in a network is implemented, in accordance with embodiments of the present disclosure.
[0053] In FIG. 1, the network architecture (100) includes one or more
computing devices or user equipments (104-1, 104-2…104-N) associated with one
15 or more users (102-1, 102-2…102-N) in an environment. A person of ordinary skill
in the art will understand that one or more users (102-1, 102-2…102-N) may be individually referred to as the user (102) and collectively referred to as the users (102). Similarly, a person of ordinary skill in the art will understand that one or more user equipments (104-1, 104-2…104-N) may be individually referred to as
20 the user equipment (104) and collectively referred to as the user equipments (104).
A person of ordinary skill in the art will appreciate that the terms “computing device(s)” and “user equipment” may be used interchangeably throughout the disclosure. Although three user equipment (104) are depicted in FIG. 1, however any number of the user equipment (104) may be included without departing from
25 the scope of the ongoing description.
[0054] In an embodiment, the user equipment (104) includes smart devices
operating in a smart environment, for example, an Internet of Things (IoT) system.
In such an embodiment, the user equipment (104) includes, but is not limited to,
smart phones, smart watches, smart sensors (e.g., mechanical, thermal, electrical,
30 magnetic, etc.), networked appliances, networked peripheral devices, networked
13
lighting system, communication devices, networked vehicle accessories, networked
vehicular devices, smart accessories, tablets, smart television (TV), computers,
smart security system, smart home system, other devices for monitoring or
interacting with or for the user (102) and/or an entity (110), or any combination
5 thereof. A person of ordinary skill in the art will appreciate that the user equipment
(104) may include, but is not limited to, intelligent, multi-sensing, network-connected devices, that can integrate seamlessly with each other and/or with a central server or a cloud-computing system or any other device that is network-connected.
10 [0055] In an embodiment, the user equipment (104) includes, but is not
limited to, a handheld wireless communication device (e.g., a mobile phone, a smart phone, a phablet device, and so on), a wearable computer device(e.g., a head-mounted display computer device, a head-mounted camera device, a wristwatch computer device, and so on), a Global Positioning System (GPS) device, a laptop
15 computer, a tablet computer, or another type of portable computer, a media playing
device, a portable gaming system, and/or any other type of computer device with wireless communication capabilities, and the like. In an embodiment, the user equipment (104) includes, but is not limited to, any electrical, electronic, electro-mechanical, or an equipment, or a combination of one or more of the above devices
20 such as virtual reality (VR) devices, augmented reality (AR) devices, a laptop, a
general-purpose computer, a desktop, a personal digital assistant, a tablet computer, a mainframe computer, or any other computing device, wherein the user equipment (104) may include one or more in-built or externally coupled accessories including, but not limited to, a visual aid device such as a camera, an audio aid, a microphone,
25 a keyboard, and input devices for receiving input from the user (102) or the entity
(110) such as touch pad, touch enabled screen, electronic pen, and the like. A person of ordinary skill in the art will appreciate that the user equipment (104) may not be restricted to the mentioned devices and various other devices may be used.
[0056] In an embodiment, the user equipment (104) is configured to
30 communicate with the system (108) through a network (106). In an embodiment,
14
the network (106) includes at least one of the 5G network, a Sixth Generation (6G)
network, or the like. The network (106) enables the user equipment (104) to
communicate with other devices in the network architecture (100) and/or with the
system (108). The network (106) includes a wireless card or some other transceiver
5 connection to facilitate this communication. In another embodiment, the network
(106) may be implemented as, or include any of a variety of different communication technologies such as a wide area network (WAN), a local area network (LAN), a wireless network, a mobile network, a Virtual Private Network (VPN), an Internet, a Public Switched Telephone Network (PSTN), or the like.
10 [0057] In another exemplary embodiment, the centralized server (112) may
include or comprise, by way of example but not limitation, one or more of: a stand¬alone server, a server blade, a server rack, a bank of servers, a server farm, hardware supporting a part of a cloud service or system, a home server, hardware running a virtualized server, one or more processors executing code to function as a server,
15 one or more machines performing server-side functionality as described herein, at
least a portion of any of the above, some combination thereof.
[0058] FIG. 2 illustrates an example block diagram (200) of a system (108)
configured for capturing and processing network packets in a network, in
accordance with an embodiment of the present disclosure. FIG. 2 is explained in
20 conjunction with FIG. 1.
[0059] In an embodiment, the system (108) may include one or more
processor(s) (202). The one or more processor(s) (202) may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, logic circuitries, and/or any devices that
25 process data based on operational instructions. Among other capabilities, the one
or more processor(s) (202) may be configured to fetch and execute computer-readable instructions stored in a memory (204) of the system (108). The memory (204) may be configured to store one or more computer-readable instructions or routines in a non-transitory computer readable storage medium, which may be
30 fetched and executed to create or share data packets over a network service. The
15
memory (204) may comprise any non-transitory storage device including, for example, volatile memory such as a random-access memory (RAM), or a non-volatile memory such as an erasable programmable read only memory (EPROM), a flash memory, and the like.
5 [0060] In an embodiment, the system (108) may include an interface(s)
(206). The interface(s) (206) may comprise a variety of interfaces, for example,
interfaces for data input and output devices (I/O), storage devices, and the like. The
interface(s) (206) may facilitate communication through the system (108). The
interface(s) (206) may also provide a communication pathway for one or more
10 components of the system (108). Examples of such components include, but are not
limited to, a processing engine (208) and a database (218). Further, the processing engine (208) may include a packet capturing unit (212), a data aggregation unit (214), and a packet streaming unit (216). The packet streaming unit (216) may further include a packet soft parser (216-1) and a packet writer (216-2).
15 [0061] In an embodiment, the processing engine (208) may be implemented
as a combination of hardware and programming (for example, programmable instructions) to implement one or more functionalities of the processing engine (208). In examples described herein, such combinations of hardware and programming may be implemented in several different ways. For example, the
20 programming for the processing engine (208) may be processor-executable
instructions stored on a non-transitory machine-readable storage medium and the hardware for the processing engine (208) may include a processing resource (for example, one or more processors), to execute such instructions. In the present examples, the machine-readable storage medium may store instructions that, when
25 executed by the processing resource, implement the processing engine(s). In such
examples, the system (108) may include the machine-readable storage medium storing the instructions and the processing resource to execute the instructions, or the machine-readable storage medium may be separate but accessible to the system and the processing resource. In other examples, the processing engine (208) may
30 be implemented by electronic circuitry.
16
[0062] Initially, the packet capturing unit (212) is configured to capture a
plurality of network packets flowing at a predefined rate based on at least one
network function. The pre-defined rate, for example, may be two hundred thousand
packets per second. In an embodiment, each of the plurality of network packets is
5 a TCP data packet. Examples of the at least one network function may include, but
is not limited to, a Cloud-native Network Function (CNF) and a Cloud-native Network Function Component (CNFC). The packet capturing unit (212) is configured to capture the plurality of network packets from the at least one network function by employing at least one of a port mirroring approach, an optical tapping
10 approach, and a direct capturing approach. The port mirroring, also known as
SPAN (Switched Port Analyzer), is a non-invasive technique where a switch or router copies network packets seen on one port (or an entire VLAN) and sends them to another port where the data can be analyzed. In examples, system or administrator can choose specific ports or VLANs to monitor traffic. The optical
15 tapping involves physically intercepting and copying optical signals traveling
through fiber-optic cables. In examples, optical splitters or couplers are used to split the light signal into two or more paths, allowing one path to continue to its intended destination while the other path carries the copied signal to a monitoring device. The direct capturing approach includes capturing network traffic directly from the
20 network medium without relying on network devices or infrastructure for copying
or mirroring. The direct capturing approach involves placing a network capture device, such as a network tap or a network interface card (NIC), directly on the network segment of interest to capture packets as they pass through.
[0063] Further, in order to capture the plurality of network packets, the data
25 aggregation unit (214) configured to cooperate with the packet capturing unit (212)
for collecting the plurality of network packets to filter and aggregate the plurality
of network packets based on a set of predefined parameters. The set of predefined
parameters may include, but is not limited to, an IP, a port, and a Virtual Local Area
Network (VLAN). The data aggregation unit (214) filters the plurality of network
30 packets to generate a plurality of filtered network packets. The data aggregation
unit (214) includes at least one optical Traffic Access Point (TAP). The at least one
17
TAP is configured to access and monitor the network traffic passing through an optical fiber link without interrupting or affecting a network traffic flow.
[0064] Further, the packet streaming unit (216) is configured to retrieve the
plurality of filtered network packets in response to filtering and aggregating the
5 plurality of network packets. The packet streaming unit (216) further includes the
packet soft parser (216-1) and the packet writer (216-2). The packet soft parser
(216-1) configured to parse at least one information from each of the plurality of
filtered network packets to generate a plurality of parsed network packets. The at
least one information may include, but is not limited to, a source IP, a destination
10 IP, a port number, and an application layer payload marker. Further, the plurality
of parsed network packets along with the at least one information may be stored in
the database (218). In other words, the packet soft parser (216-1) is configured to
archive the plurality of parsed network packets in the database (218) when a
connection with a centralized database is lost. Further, the packet soft parser (216-
15 1) is configured to send the plurality of parsed network packets archived in the
database (218) to the centralized database upon restoring the connection. With
reference to FIG. 1, the centralized server may be hosted on the centralized server
112. In some embodiment, when the connection with the centralized server is stable,
the plurality of parsed network packets along with the at least one information may
20 be directly transmitted and stored in the centralized database. The packet soft parser
(216-1) is configured to support a Hypertext Transfer Protocol Version 2 (HTTP2),
a diameter, a Packet Forwarding Control Protocol (PFCP), Signaling Transport
(SIGTRAN), a Dynamic Host Configuration Protocol (DHCP), a Session Initiation
Protocol (SIP), and a Remote Authentication Dial-In User Service (RADIUS).
25 [0065] The packet writer (216-2) is configured to retrieve the plurality of
parsed network packets and the at least one information from the centralized database based on one or more pre-defined topics to generate a compressed file. The one or more pre-defined topics may include, but are not limited to, protocols (e.g., HTTP, TCP, User Datagram Protocol (UDP), etc.), communication channels
30 (e.g., internal, external), application types (e.g., email, web browsing), or any other
18
relevant criteria. The packet writer (216-2) is further configured to write data into
each compressed file. In an embodiment, each compressed file is corresponding to
a specific broker topic. In an embodiment, the specific broker topic is a predefined
category, or a channel used for organizing and categorizing the data. In this context,
5 the specific broker topic may likely represent different types of network traffic or
specific topics of interest. Each compressed file generated by the packet writer (216-2) is associated with a particular broker topic, indicating a type or a category of data contained within each compressed file.
[0066] Although the FIG. 2 shows exemplary components of the system
10 (108), in other embodiments, the system (108) may include fewer components,
different components, differently arranged components, or additional functional components than depicted in the FIG. 2. Additionally, or alternatively, one or more components of the system (108) may perform functions described as being performed by one or more other components of the system (108).
15 [0067] FIG. 3 illustrates an example block diagram (300) of a system
architecture configured for capturing and processing network packets in a network, in accordance with an embodiment of the present disclosure. FIG. 3 is explained in conjunction with FIGS. 1 - 2.
[0068] In FIG. 3, the presented embodiment describes a multifaceted
20 framework designed for intricate task of capturing the plurality of network packet,
essential for analyzing data flow, the network traffic flow, and network
performance. The presented system architecture is versatile, offering three distinct
methods for capturing the plurality of network packets: through the optical tapping
approach, the port mirror sessions, and direct acquisition from a host machine. Each
25 method is tailored for different scenarios, providing a comprehensive solution for
network monitoring.
[0069] The system, denoted as (108), is equipped with a continuous
capturing feature. Such continuous capturing is crucial for retrospective analysis,
allowing network administrators to review past sessions and data flows, thereby
30 gaining insights into network events that occurred previously.
19
[0070] In one aspect, network elements are represented by Evolved Packet
Core (EPC) nodes at 302. The EPC is a framework for providing converged voice
and data on a 4G Long-Term Evolution (LTE) network and 5G core networks. It is
a core network architecture that enables various EPC nodes to communicate and
5 process user data and control messages. The EPC is designed to support a 4G LTE
network access system and is equivalent of a General Packet Radio Service (GPRS) core network for 3G networks. For example, the EPC nodes includes a MME (Mobility Management Entity), a PGW (i.e., Packet Data Network (PDN) Gateway), a Diameter Routing Agent (DRA), an Access and Mobility Management
10 Function (AMF), a Session Management Function (SMF), a Policy Control
Function (PCF), a PCRF (Policy and Charging Rules Function), and the like. The MME is a key control node in the LTE (Long Term Evolution) core network, specifically in the Evolved Packet Core (EPC). The MME is responsible for handling signaling and control functions to support mobility, session management,
15 and authentication for users. The PGW is a critical component of the LTE EPC.
The PGW serves as the interface between the EPC and external packet data networks, such as the internet or other IMS (IP Multimedia Subsystem) services. The DRA is a network element that facilitates the routing of Diameter signaling messages within an LTE network. Diameter is a protocol used for authentication,
20 authorization, and accounting (AAA). The AMF is a component in the 5G core
network (5G Core, or 5GC) for handling signaling and control for access and mobility management, supporting user equipment (UE) registration, connection, and mobility. The SMF is a component in the 5G core network responsible for session management, including the establishment, modification, and termination of
25 user data sessions. The PCF in the 5G core network is responsible for policy
management and it determines policies for QoS, charging, and access control based on operator-defined rules and user profiles. The PCRF is a component in the LTE EPC responsible for making real-time policy decisions and controlling the charging rules for each user session.
30 [0071] Within this system (108), a packet collection layer (i.e., the packet
capturing unit 212) is established, comprising optical taps 304 (i.e., the optical
20
tapping approach). The optical taps 304 are devices placed on physical network
links—for instance, between an application host and a Top-of-Rack (TOR) switch
or from the TOR to a Nexus switch. Their role is to intercept and collect network
packets without interrupting the flow of network traffic. This non-invasive
5 approach is instrumental in maintaining network integrity while gathering data.
[0072] Further to this, the system (108) incorporates aggregators at a Virtual
Switching System (VSS) aggregation and filtering layer 306 (same as the data
aggregation unit (214)), also known in an industry as Packet Brokers. These
aggregators perform a critical function of data aggregation and filtering. They can
10 sift through the plurality of network packets captured, to select relevant data based
on the set of pre-defined parameters such as an IP, i.e., IP address, a port number, and a VLAN, i.e., a VLAN tag. This selective process ensures that only pertinent information is forwarded to the next stage, preventing data overload and enhancing efficiency.
15 [0073] The system 108 is further configured to perform capturing process
308, i.e., the capturing of the plurality of network packets, which operates according to predefined configurations. It conducts an initial level of parsing and filtering, extracting valuable information like source and destination IP addresses, port numbers, and identifiers for application layer payload. These pieces of information
20 are the metadata that offer insight into the nature and direction of network traffic.
[0074] In one embodiment, the system (108) is configured for capturing and
processing network packets in the network (e.g., the wireless communication
network). In particular, the system (108) includes the packet capturing unit (212)
configured to capture the plurality of network packets flowing at the pre-defined
25 rate, e.g., a rate of up to two hundred thousand packets per second. The plurality of
network packets includes TCP data packets, and the packet capturing unit (212) is further configured to bypass a LINUX kernel by direct capturing from a network interface card.
[0075] In one aspect, the aggregator, i.e., the data aggregation unit (214)
30 includes optical taps for collecting the plurality of network packets for data
21
aggregation and filtering based on the set of predefined parameters including the
IP, the port, and the VLAN, and for directing aggregated data to the packet
capturing unit (212). The data aggregation unit (214) performs traffic aggregation
by combining traffic from multiple network segments, links, or devices into a single
5 stream or a set of streams based on the parameters including, but not limited to IP,
the port, and the VLAN. The data aggregation unit (214) filters the aggregated
network packets by selectively forwarding only the traffic or network packets that
is relevant to specific monitoring or security tools. In example, the filtering criteria
may be the same as the set of predefined parameters. The data aggregation and
10 filtering are performed to generate the plurality of filtered network packets.
[0076] In another aspect, a native streaming framework integrated with the
packet capturing unit (212) is configured to stream the plurality of filtered network packets into a centralized streaming framework.
[0077] In another aspect, as packet parsing, flow identification, and display
15 step 310, the packet soft parser (216-1) is configured for retrieving raw packet data
(i.e., the plurality of) from a centralized streaming framework, i.e., the centralized
database, and performing soft parsing to extract user identifiable information, i.e.,
the at least one information, and for archiving the plurality of parsed network
packets into the compressed file when the centralized streaming framework is
20 unavailable, and for reading from the compressed file and pushing the plurality of
parsed network packets along with the at least one information back into the centralized streaming framework upon re-establishment of the connection.
[0078] In one aspect, a packet writer microservice, i.e., the packet writer
(216-1) is configured for retrieving data (i.e., the plurality of parsed network
25 packets along with the at least one information) from the centralized streaming
framework based on the one or more pre-defined topics, enriching the data with inventory information, and writing the enriched data into a highly compressed fixed file format, i.e., the compressed file in the specific broker topic.
[0079] In one aspect, multiple broker topics listened to by the packet writer
30 microservice for handling data from various sources and for implementing rollover
22
policies based on time intervals, number of records, or a compressed file size for efficient data storage management.
[0080] Concluding the capturing process, the extracted raw packet data,
along with its corresponding metadata, i.e., the plurality of parsed packets along
5 with the at least one information, is transmitted to a centralized packet streaming
framework cluster. This centralized packet streaming framework cluster serves as a repository and a processing centre for the data, facilitating further analysis and decision-making processes for network management.
[0081] FIG. 4 illustrates a block diagram (400) of a managed switch with
10 port mirroring (402), in accordance with an embodiment of the present disclosure.
FIG. 4 is explained in conjunction with FIGS. 1 – 3.
[0082] In FIG. 4, the system (108) is adeptly configured to utilize a network
switch's capabilities for creating a mirror session. This process involves designating one or more ports, i.e., monitoring ports (source) (404), e.g., ‘A’, ‘B’, and ‘C, and
15 an analysis port (destination) (406), e.g., ‘D’, specifically for the purpose of
mirroring network traffic. When a packet (i.e., a network packet) traverses the switch, a duplicate is created and sent to the designated mirror port. Connected to this port is typically a specialized monitoring system whose function is to analyze the network traffic flowing through the switch. This setup is particularly crucial for
20 network security and performance monitoring.
[0083] The system (108) extends its capabilities to encompass both a Local
Switched Port Analyzer (SPAN) and an Encapsulated Remote SPAN (ERSPAN),
i.e., an ERSPAN encapsulation parsing. These are methods employed for directing
a copy of network traffic to an analysis device. The Local SPAN replicates traffic
25 from one or more interfaces to another interface within the same switch, which is
limited to a single local network. In contrast, the ERSPAN allows for the mirroring of traffic to a remote switch within the same L2 domain, expanding the scope beyond a single switch.
[0084] The ERSPAN presents a significant advancement as it supports L3
23
IP routing, enabling a redirection of mirrored traffic across different network
segments. This functionality is particularly advantageous for organizations
operating over dispersed geographic locations. It allows for centralized traffic
analysis by routing mirrored traffic through multiple network layers and across
5 various physical sites. However, it should be noted that the ERSPAN is model
specific and may not be available on all network switch models.
[0085] For instance, consider a network architecture spanning across
different cities or even countries. Using the ERSPAN, a network administrator in a
central location can analyze traffic from switches deployed anywhere within the
10 organization's network without the need for local monitoring at each site. This
centralized analysis can lead to more efficient network management and quicker response times to network issues, irrespective of the physical location of a network traffic source.
[0086] FIG. 5 illustrates a system architecture (500) designed for ERSPAN
15 operations within the network, in accordance with an embodiment of the present
disclosure. FIG. 5 is explained in conjunction with FIGS. 1 -4. The system (108) is
adept at facilitating on-demand capturing, a feature that allows for a precise tracking
of specific data flows, individual subscribers, or a comprehensive network traffic
associated with an application (e.g., a web application, a mobile application, and
20 the like) suite. This capability is particularly advantageous for detailed network
traffic analysis and diagnostics.
[0087] In one embodiment, a Command Line Interface (CLI) or a Graphical
User Interface (GUI) 502 is used by the network administrators or users to interact with the system (108). It serves as a point of control for network management tasks.
25 [0088] Elastic Load Balancers 504 ELB-A (Active) and ELB-S (Standby)
are configured to distribute the network traffic coming from the CLI/GUI 502. Load balancers distribute network or application traffic across multiple servers to enhance the performance, reliability, and scalability of service applications. The elastic load balancers are a service that is configured to automatically distribute
30 incoming application traffic across multiple targets, providing fault tolerance, high
24
availability, and scalability.
[0089] Packet Capture (PC) Gateways (Active) 506-1 and 506-2, both are
marked as active. These PC Gateways (Active) 506-1 and 506-2 are likely
responsible for managing the plurality of network packets capturing process,
5 possibly aggregating and filtering data before it is processed or analyzed. The state
'Active' may suggest a redundancy or load-sharing system where two gateways are operational at the same time to ensure continuous packet capturing without loss of data.
[0090] PC Agents (508-1, 508-2, 508-3 up to 508-N), which could be
10 software or hardware agents, are configured for the actual capturing of the plurality
of network packets on the network. Each agent sends the captured data to the PC
Gateways (506-1, 506-2) for processing. The structure suggests a distributed system
where the PC Agents (508-1, 508-2, 508-3 up to 508-N) capture the data and send
it to the PC Gateways (506-1, 506-2). The ELB 504 components would balance the
15 load between these gateways to ensure efficient data processing and to handle
potential high traffic volumes. The CLI/GUI 502 allows for the administration and monitoring of this process. The overall design aims to provide a robust and scalable network management solution.
[0091] The system (108) harnesses RESTful APIs, which afford a scalable
20 and stateless interaction model for applications to initiate packet capture requests
as needed. An artificial intelligence (AI) powered scheduler is integrated within the
system, programmed to autonomously capture packets and register callbacks in
anticipation of potential network issues, thereby enhancing proactive fault
management. The RESTful APIs is an architectural style for designing networked
25 applications that use HTTP requests to perform CRUD (Create, Read, Update,
Delete) operations on resources.
[0092] This system (108) is engineered to accommodate both Virtual
Extensible LAN (VxLAN) and Generic Routing Encapsulation-Encapsulated
Remote Switched Port Analyzer (GRE-ERSPAN) encapsulated packets. Such a
30 design permits the capturing and subsequent processing of mirrored traffic,
25
allowing for extensive network monitoring across diverse network topologies. The
VXLAN is a network virtualization technology that allows creation of a logical
overlay network on top of a physical IP network, extending Layer 2 segments over
a Layer 3 infrastructure. The GRE is a tunneling protocol used to encapsulate a
5 wide variety of network layer protocols inside virtual point-to-point links over an
IP network. Encapsulated Remote Switched Port Analyzer (ERSPAN) extends the capabilities of GRE by enabling remote port mirroring over an IP network.
[0093] Scalability is an attribute of the system (108), enabling horizontal
expansion based on the volume and nature of the network traffic associated with
10 various network functions or the aggregate of network nodes. The system (108)
supports both LINUX kernel-based capturing and a kernel bypass mode, optimizing packet capturing efficiency in accordance with deployment specifications.
[0094] Data dissemination within the system (108) is meticulously
organized, with support for directing captured data to an array of topics or
15 consumers, based on the originating source IP and destination IP. This targeted
approach to data streaming ensures that relevant information is channelled to appropriate endpoints for efficient consumption and processing.
[0095] An effective traffic filtering mechanism is a cornerstone of the
system (108), utilizing Berkeley Packet Filter (BPF) expressions, such as 'tcp dst
20 port 80 or 8080', to selectively capture traffic. This granular capture is not only
efficient but also allows for the conservation of computational resources and reduction of data redundancy.
[0096] Additionally, the system (108) is equipped with sophisticated
parsing capabilities for various network protocol layers. It is capable of extracting
25 and processing key packet attributes such as Source IP, Destination IP, Source and
Destination Ports, and TCP Sequence Numbers. This enriched data parsing facilitates in-depth analysis and further processing of network traffic.
[0097] FIG. 6A illustrates a flow chart 600A of an exemplary method of a
packet capturing and a soft parsing, in accordance with an embodiment of the
26
present disclosure. FIG. 6A is explained in conjunction with FIGS. 1 - 5. The flow chart (600A) depicts various application layer protocols implemented for soft parsing support for tagging packets with search criteria.
[0098] At step (602A) involves reading network packet data of a network
5 packet of the plurality of network packets as sent by a capturing process. The
reading is performed by the packet capturing unit (212), which can handle network packets at a very high rate and bypasses the standard LINUX kernel, capturing directly from a network interface card.
[0099] At step (604A) includes constructing an HTTP2 network packet (or
10 another TCP user protocol packet) using a TCP payload marker. A construction is
a part of the soft parsing process that the system (108) uses to identify and handle different types of the plurality of network packets.
[00100] At step (606A) includes checking if a complete HTTP2 frame (or
frame of any TCP user protocol) has been received for this connection. This is to
15 ensure that a full message is available for processing before proceeding.
[00101] At step (608A), if no complete frame is received, the network packet
is discarded, as it cannot be processed further. This step helps in maintaining the efficiency of the system by not processing incomplete or partial data.
[00102] Once a complete frame is detected, the system (108) enables
20 processing for packets coming over this connection at step (610A). This step may
involve setting certain flags or initiating specific processes to handle the incoming packets.
[00103] The system (108) checks if there are any network packets pending
to be processed for this connection at step (612A). This might refer to network
25 packets that have been queued up while waiting for a complete frame to be detected.
[00104] Further, at step (614A) involves checking for sequence number
matches for in-order of the network packets. This is important for ensuring that the network packets are processed in the correct order, which is critical for TCP data packets.
27
[00105] The system (108) begins soft parsing, which involves analyzing the
raw packet data (i.e., the plurality of filtered network packets) to extract information without fully decoding the protocol at step (618A). The soft parsing may include identifying user identifiable information within the network packets.
5 [00106] The system (108) calculates an identification number (ID) and
compares it with a network packet length at step (620A). The identification may include validating the integrity of the network packets or for identifying specific network packets within the stream.
[00107] If the network packet is part of a segmented message, it is saved in
10 cache for further processing at step (622A). Meaning that the network packet is part
of a larger message that has been divided into smaller segments.
[00108] If a single network packet contains a complete message or multiple
frames, this frame is skipped at step (624A). It could mean that this network packet does not require further processing because it is already complete.
15 [00109] The system (108) parses HTTP2 fields or fields from other L7
protocols at step (626A).
[00110] Finally, the system (108) tags the HTTP2 field or L7 protocol fields
in the network packet and inserts the data into Hadoop Distributed File System
(HDFS) at step (628A), which suggests the data is stored in a big data ecosystem
20 for further analysis or processing.
[00111] FIG. 6B is a flow chart (600B) of another exemplary method of a
packet capture and a soft parser in a network, in accordance with an embodiment
of the present disclosure. FIG. 6B is explained in conjunction with FIGS. 1 – 6A.
In this exemplary method, an application layer payload is sent into multiple TCP
25 Packets to parse by doing segmentation reassembly. Parsed fields are tagged with
all TCP segments for further analysis. For example, large HTTP2 Frames segmented into multiple TCP Packets. Frame Stream ID/SUPI to be tagged with all the network packets.
[00112] Initially, at step (602B) the initiation of the packet capturing unit that
28
is configured to capture network packets at a very high rate and bypasses the standard LINUX kernel for direct capturing from a network interface card.
[00113] At step (604B), the system (108) specifies the network device (such
as a network interface card) on which network packets capturing will occur.
5 [00114] At step (606B), a chosen device (i.e., the user equipment (104)) is
opened for sniffing, which means the system (108) begins to monitor the network traffic passing through the chosen device.
[00115] At step (608B), the system (108) applies filtering criteria to capture
only specific types of traffic, such as TCP packets to a particular port, as part of the
10 data aggregation unit (214) which filters based on the IP, the port, and the VLAN.
[00116] At step (610B), the packet capturing unit (212) starts an actual
network packet sniffing process by executing ae primary loop of the pcap library, capturing network packets flowing up to the pre-defined rate.
[00117] At step (612B), the system (108) parses an Ethernet header of each
15 packet to determine a type of payload it contains, which could be IPv4 or IPv6.
[00118] After determining the Ethernet payload type, the system (108) parses
the IP header to further understand the type of payload at step (614B).
[00119] At step (616B), the system (108) checks for the network packets that
may contain protocols other than TCP, such as GRE-ERSPAN.
20 [00120] If the payload type is determined to be TCP at step (616B), the
system (108) proceeds to parse the TCP header at step (618B).
[00121] For TCP packets, the system (108) parses the TCP headers,
including source and destination ports and sequence numbers, as part of the soft parsing to extract user-identifiable information at step (618B).
25 [00122] At step (620B), the system (108) calculates a marker for the TCP
payload, which is used for identifying the beginning of the payload data for soft parsing.
[00123] Finally, at step (622B), the network packet data is streamed,
29
potentially into the centralized streaming framework for further processing. The native streaming framework integrated with the packet capturing unit (212) may be responsible for this streaming.
[00124] If the payload type is determined to be GRE at step (624B), the
5 system (108) parses the GRE and ERSPAN protocols, supporting the encapsulation
parsing for mirrored traffic at step (628B).
[00125] If the payload type is not determined to be the GRE at step (624B),
the system (108) checks for other protocols at step (626B).
[00126] When handling GRE packets, the system (108) parses the GRE and
10 ERSPAN protocols as per the packet capturing unit (212) configuration at step
(628B). Further, in response to parsing, the step (618B) to (622B) are re-executed.
[00127] FIG. 7 illustrates an example packet capturing process (700), in
accordance with embodiments of the present disclosure. FIG. 7 is explained in conjunction with FIG. 1 – 6B.
15 [00128] As illustrated in FIG. 7, in an embodiment, the system (108) may
include 2 lakh packets per second capturing capability. Considering TCP with data packet.
[00129] The system (108) offers robust support for the parsing of GRE-
ERSPAN encapsulated traffic, enabling efficient analysis of mirrored network data.
20 [00130] Integrated within the system (108) is the native streaming framework
that facilitates the real-time streaming of captured network data.
[00131] The system (108) is equipped to dispatch streamed data to an array
of topics or consumer endpoints, classified based on the source and destination IP addresses.
25 [00132] Comprehensive parsing functionality within the system (108)
includes the extraction of source and destination IPs, as well as source and destination ports, complemented by the parsing of TCP sequence numbers for advanced data processing.
30
[00133] Future enhancements to the system (108) are expected to incorporate
support for the parsing of VxLAN encapsulated packets, particularly when packets are in transit from Top of Rack (TOR) to Nexus switches.
[00134] The system (108) circumvents the limitations of the LINUX
5 capturing library by directly capturing packets from the network interface card
(NIC), thereby bypassing the kernel and eliminating bottlenecks.
[00135] Flexibility is a hallmark of the system (108), which supports both
on-demand packet capturing for targeted data acquisition and live packet capturing for continuous monitoring.
10 [00136] On-demand packet capturing can be customized through a variety of
filters, including the names of network functions (CNF, CNFC), specific transport protocols and ports, as well as the duration of capture or the total number of packets to be captured.
[00137] Scheduled capturing is a feature that enhances the system’s
15 capability, allowing for the automated capture of network packets based on
predetermined schedules.
[00138] The system (108) is designed with FTP functionality, enabling the
transfer of captured packet data in pcap file format, which is essential for data archival and analysis.
20 [00139] Improvement to the system's operational intelligence include the
addition of logging features and counters, which aid in the detailed monitoring and management of network data.
[00140] The system (108) is also configured for the development of a Packet
Analyzer, which will allow for the capturing of packets based on various L2 through
25 L7 protocol fields. This future functionality will enable the system (108) to capture
network packets associated with specific identifiers such as SUPI, MSISDN, DIAMETER Session-ID, or MAP Transaction IDs, further tailoring the packet capturing process to specific user or session data.
[00141] FIG. 7 illustrates an advanced packet capturing process (700),
31
designed to operate within the parameters of modern telecommunications networks
as indicated in the embodiments of the present disclosure. FIG. 7 is explained in
conjunction with FIGS. 1 – 6B. This packet capturing process where the packets
are captured from network functions 702 and the processes up to two hundred
5 thousand packets per second. This throughput is significant when considering the
handling of TCP data packets, which are fundamental to the reliable transmission of data across network connections.
[00142] A prominent feature of the system (108) is its support for Generic
Routing Encapsulation-Encapsulated Remote Switched Port Analyzer (GRE-
10 ERSPAN) encapsulation parsing at step 704. This feature is implemented for
managing mirrored traffic, wherein data packets are duplicated across the network
for monitoring or analysis purposes. Furthermore, the system (108) integrates a
native streaming framework, an advanced streaming platform that efficiently
handles the transmission of the captured data.
15 [00143] The system (108) adeptness extends to the distribution of the data to
a multitude of topics or consumers, categorized based on the source IP and the destination IP. This allows for a more organized and targeted approach to data handling, ensuring that information reaches its intended recipients effectively at step 706.
20 [00144] Parsing capabilities within the system (108) are robust, offering
detailed analysis of key packet components such as Source IP, Destination IP, Source and Destination Ports, and TCP Sequence Numbers. This meticulous parsing is fundamental to the subsequent stages of data processing.
[00145] Future enhancements to the system (108) include the ability to parse
25 the network packets encapsulated with Virtual Extensible LAN (VxLAN)
technology, which is particularly relevant when data packets traverse from Top of Rack (TOR) switches to Nexus switches, indicating a layered network infrastructure, at block 708.
[00146] The system (108) circumvents potential limitations associated with
32
the LINUX capturing library by adopting a method of capturing packets directly from the network interface card (NIC), effectively bypassing the kernel. This approach is indicative of a more direct and unencumbered data capturing methodology.
5 [00147] Moreover, the system (108) offers versatility in its capturing modes,
with provisions for both on-demand and live packet capturing. On-demand
capturing can be tailored based on various filters such as component name (e.g., a
Cloud-native Network Function (CNF)), component name for containers (e.g. a
Cloud-native Network Component (CNFC)), specific Transport Protocols, Ports,
10 the duration of the capture, or the number of network packets to be captured. This
mode is particularly useful for targeted data acquisition, such as capturing data related to a specific service or user.
[00148] The system (108) further provides support for transferring the
captured packet data in the pcap file format using File Transfer Protocol (FTP),
15 adding another layer of functionality for data management. Additionally, it is
equipped with logging capabilities and counters to aid in monitoring and analysis tasks.
[00149] In essence, the system (108) is constructed to deliver a
comprehensive solution for network traffic capturing and analysis, providing a
20 multifaceted approach to handling the complexities of modern network
management.
[00150] FIG. 8 illustrates an exemplary packet soft parser (800), in
accordance with an embodiment of the present disclosure. FIG. 8 is explained in conjunction with FIGS. 1 – 7.
25 [00151] As illustrated in FIG. 8, in an embodiment, the system (108) may
include the packet soft parser (216-1) that retrieves raw packet data (i.e., the plurality of filtered network packets) from the packet streaming unit (216), typically stored in a packet streaming framework topic, at step 802. It soft parses the data at step 804, extracting user identifiable information such as SUPI, Timestamp, and
33
Stream ID and such metadata at step 806. It then pushes the raw packet data back
into the packet streaming unit (216), but this time on a different topic, ensuring
availability of the extracted user details at step 804. The packet soft parser (216-1)
archives the data into the compressed file, preventing data loss when the packet
5 streaming framework is unavailable and safeguarding it until the packet streaming
unit (216) is reachable. Once the connection to the packet streaming unit (216) is re-established, the packet soft parser reads the archived data (i.e., the plurality of parsed data packets along with the at least one information) from the file and pushes it back into the packet streaming framework, ensuring seamless data transfer.
10 [00152] The packet soft parser (216-1) may be deployed in an Active-
Standby (M: N) mode to enhance reliability and fault tolerance at step 808. The deployment comprises multiple instances, where one instance actively processes data while the others act as backups. In case of issues or failures, the standby instances can seamlessly take over processing duties. This configuration ensures
15 uninterrupted operations by minimizing the risk of downtime or data loss.
[00153] FIG. 9 illustrates an exemplary packet writer microservice (900), in
accordance with an embodiment of the present disclosure. FIG. 9 is explained in conjunction with FIGS. 1 – 8.
[00154] As illustrated in FIG. 9, in an embodiment, the system (108) may
20 include a packet writer (same as the packet writer (216-2)) that retrieves data from
the packet streaming framework based on specified topics at step 902. This data
may be then enriched by incorporating inventory data and is subsequently written
into a fixed file format that is highly compressed, optimizing storage efficiency at
step 904. The packet writer may listen to multiple network packets streaming
25 framework topics, allowing it to handle data from multiple sources simultaneously.
The data is written into multiple files (i.e., the compressed files), each
corresponding to a specific topic at 906. To maintain data integrity and prevent loss,
the packet writer implements robust rollover policies for the generated files. These
rollover policies can be based on various parameters, including time intervals,
30 number of records, or file size. When the defined conditions are met, the packet
34
writer rolls over to a new compressed file at 908, ensuring efficient management of
data storage. The packet writer microservice (i.e., the packet writer) may ensure
data consistency even if any instance is down. This resilience may be achieved by
distributing the workload across multiple instances, allowing the system (108) to
5 seamlessly continue writing data without interruption or data loss, thus maintaining
overall system reliability.
[00155] FIG. 10 illustrates a flow diagram (1000) of a method for capturing
and processing network packets in a network, in accordance with an embodiment of the present disclosure. FIG. 10 is explained in conjunction with FIGS. 1 – 8.
10 [00156] Initially, at step 1002, the plurality of network packets flowing at the
predefined rate may be captured based on at least one network function. The pre-defined rate, for example, may be two hundred thousand packets per second. In an embodiment, each of the plurality of network packets is the TCP data packet. Examples of the at least one network function may include, but is not limited to, a
15 Cloud-native Network Function (CNF) and a Cloud-native Network Function
Component (CNFC). Further, the plurality of network packets may be captured based on the at least one network function by employing at least one of the port mirroring approach, the optical tapping approach, and the direct capturing approach. In the port mirroring approach, the GRE and ERSPAN supports
20 encapsulation parsing for mirrored traffic. The port mirroring is a technique used in
networking to send a copy of network packets seen on one port (or an entire VLAN) to another port where the data can be analyzed. Further, the direct capturing approach includes receiving the plurality of network packets from the at least one network function using the network interface card.
25 [00157] Further, in order to capture the plurality of network packets, at step
1004, the plurality of network packets may be collected to filter and aggregate the plurality of network packets based on the set of predefined parameters. The set of predefined parameters may include, but is not limited to, the IP, the port, and the VLAN. The plurality of network packets is filtered to generate the plurality of
30 filtered network packets. At step 1006, the plurality of filtered network packets may
35
be retrieved in response to filtering and aggregating the plurality of network packets.
[00158] Further, at step 1008, at least one information may be parsed from
each of the plurality of filtered network packets to generate a plurality of parsed
5 network packets. The at least one information may include, but is not limited to, the
source IP, the destination IP, the port number, and the application layer payload marker. Further, at step (1010) the plurality of parsed network packets along with the at least one information may be stored in a database (same as the database 218). In other words, the plurality of parsed network packets may be archived in the
10 database (218) when a connection with a centralized database is lost. Further, the
plurality of parsed network packets archived in the database (218) are sent to the centralized database upon restoring the connection. With reference to FIG. 1, the centralized server may be hosted on the centralized server (112). In some embodiment, when the connection with the centralized server is stable, the plurality
15 of parsed network packets along with the at least one information may be directly
transmitted and stored in the centralized database.
[00159] In addition, the plurality of parsed network packets and the at least
one information may be retrieved from the centralized database based on one or
more pre-defined topics to generate the compressed file. The one or more pre-
20 defined topics may include, but are not limited to, protocols (e.g., HTTP, TCP, User
Datagram Protocol (UDP), etc.), communication channels (e.g., internal, external),
application types (e.g., email, web browsing), or any other relevant criteria. Further,
data is written into each compressed file corresponding to a specific broker topic.
In an embodiment, the specific broker topic is a predefined category, or a channel
25 used for organizing and categorizing the data. In this context, the specific broker
topic may likely represent different types of network traffic or specific topics of
interest. Each compressed file generated is associated with a particular broker topic,
indicating a type or a category of data contained within each compressed file.
[00160] FIG. 11 illustrates an example computer system (1100) in which or
30 with which the embodiments of the present disclosure may be implemented.
36
[00161] As shown in FIG. 11, the computer system (1100) may include an
external storage device (1110), a bus (1120), a main memory (1130), a read-only
memory (1140), a mass storage device (1150), communication port(s) (1160), and
a processor (1170). A person skilled in the art will appreciate that the computer
5 system (1100) may include more than one processor and communication ports. The
processor (1170) may include various modules associated with embodiments of the
present disclosure. The communication port(s) (1160) may be any of an RS-232
port for use with a modem-based dialup connection, a 10/100 Ethernet port, a
Gigabit or 10 Gigabit port using copper or fiber, a serial port, a parallel port, or
10 other existing or future ports. The communication ports(s) (1160) may be chosen
depending on a network, such as a Local Area Network (LAN), Wide Area Network (WAN), or any network to which the computer system (1100) connects.
[00162] In an embodiment, the main memory (1130) may be a Random
Access Memory (RAM), or any other dynamic storage device commonly known in
15 the art. The read-only memory (1140) may be any static storage device(s) e.g., but
not limited to, a Programmable Read Only Memory (PROM) chip for storing static information e.g., start-up or basic input/output system (BIOS) instructions for the processor (1170). The mass storage device (1150) may be any current or future mass storage solution, which can be used to store information and/or instructions.
20 Exemplary mass storage solutions include, but are not limited to, Parallel Advanced
Technology Attachment (PATA) or Serial Advanced Technology Attachment (SATA) hard disk drives or solid-state drives (internal or external, e.g., having Universal Serial Bus (USB) and/or Firewire interfaces).
[00163] In an embodiment, the bus (1120) may communicatively couple the
25 processor(s) (1170) with the other memory, storage, and communication blocks.
The bus (1120) may be, e.g. a Peripheral Component Interconnect PCI) / PCI
Extended (PCI-X) bus, Small Computer System Interface (SCSI), Universal Serial
Bus (USB), or the like, for connecting expansion cards, drives, and other
subsystems as well as other buses, such a front side bus (FSB), which connects the
30 processor (1170) to the computer system (1100).
37
[00164] In another embodiment, operator and administrative interfaces, e.g.,
a display, keyboard, and cursor control device may also be coupled to the bus
(1120) to support direct operator interaction with the computer system (1100).
Other operator and administrative interfaces can be provided through network
5 connections connected through the communication port(s) (1160). Components
described above are meant only to exemplify various possibilities. In no way should the aforementioned exemplary computer system (1100) limit the scope of the present disclosure.
[00165] In an advanced embodiment of a system designed for the meticulous
10 capturing and intricate processing of network packets within a sophisticated
wireless telecommunications network, the system is composed of several integral components functioning in concert to ensure a seamless management and discerning analysis of network traffic.
[00166] Central to the operational efficacy of this system is a highly capable
15 packet capturing processor. This processor boasts an impressive proficiency,
capturing network packets at an expedited rate of up to two hundred thousand
packets per second. The specialization of this processor extends to its handling of
TCP data packets, characterized by its innovative approach to bypass the traditional
LINUX kernel. It achieves this by engaging directly with the network interface card
20 (NIC), thus sidestepping the commonly encountered constraints and inefficiencies
of kernel-dependent packet capture methods.
[00167] In synergy with the packet capturing processor is a data aggregation
module, meticulously engineered to encompass optical taps and packet brokers. The
optical taps are strategically positioned to harvest network packets from the
25 network's physical links, functioning as meticulous collectors of data. The packet
brokers, on the other hand, serve a pivotal role in the aggregation and discerning filtration of the amassed data. They employ selective criteria such as IP addresses, port designations, and VLAN tags to refine the data before its conveyance to the packet capturing processor.
30 [00168] Embedded within the system's architecture is a native streaming
38
framework, deftly integrated to facilitate the streaming of the captured and
methodically aggregated data into a centralized streaming framework. The
centralized streaming framework, operating as the nerve centre for data streams,
orchestrates the orderly organization and methodical retrieval of data for
5 subsequent processing.
[00169] A nuanced packet soft parser is also integral to this system. It
retrieves raw packet data ensconced within the centralized streaming framework and conducts 'soft parsing.' This process is a judicious compromise between the depth of traditional parsing and the necessity for efficiency, adeptly identifying and
10 extracting user-identifiable information. Should the centralized streaming
framework become temporarily inaccessible, the packet soft parser astutely archives the data into a file, ensuring the preservation of information. Upon restoration of the cluster's connectivity, the parser diligently reintroduces the archived data into the centralized streaming framework, ensuring continuity of data
15 flow.
[00170] Additionally, the system is equipped with a packet writer
microservice. This service adeptly retrieves data from the centralized streaming
framework, selecting based on precise topics. It then augments this data with
supplementary inventory information, subsequently inscribing it into files that are
20 formatted in a highly compressed, fixed file format. This process not only optimizes
storage utilization but also ensures the integrity and continuity of the data.
[00171] To adeptly manage the voluminous data, the packet writer
microservice attentively listens to an array of broker topics. It capably handles data
streams from a multitude of sources and implements meticulously crafted rollover
25 policies. These policies are astutely designed to dictate the initiation of new file
writing, triggered by a set of predefined conditions such as elapsed time intervals, the accumulation of records, or the expansion of file size.
[00172] This embodiment delineates a comprehensive and robust solution for
network packet capturing and processing, delivering an indispensable toolkit for
30 telecommunications networks to diligently monitor, analyze, and govern network
39
traffic with unparalleled precision and steadfast reliability.
[00173] While considerable emphasis has been placed herein on the preferred
embodiments, it will be appreciated that many embodiments can be made and that
many changes can be made in the preferred embodiments without departing from
5 the principles of the disclosure. These and other changes in the preferred
embodiments of the disclosure will be apparent to those skilled in the art from the disclosure herein, whereby it is to be distinctly understood that the foregoing descriptive matter to be implemented merely as illustrative of the disclosure and not as limitation.
10
ADVANTAGES OF THE PRESENT DISCLOSURE
[00174] The present disclosure provides a system and a method that provides
continuous network packets capturing to analyze past sessions or flows.
[00175] The present disclosure provides a system and a method that uses on-
15 demand capturing of the network packets to trace a particular flow, monitor a
particular subscriber, or monitoring a traffic flow of a complete application.
[00176] The present disclosure provides a system and a method that handles
of an IP fragmentation, a TCP segmentation, and L2 to L7 protocol level soft parsing for desired search criteria.
20
40
WE CLAIM:
1. A system for capturing and processing network packets in a network, comprising:
a memory (204); a database (210); and
a processing engine (208) coupled to the memory (204), the processing engine (208) comprising:
a packet capturing unit (212) configured to capture a plurality of network packets flowing at a pre-defined rate based on at least one network function;
a data aggregation unit (214) communicatively coupled with the packet capturing unit (212) and configured to collect the plurality of network packets to filter and aggregate the plurality of network packets based on a set of predefined parameters to generate a plurality of filtered network packets;
a packet streaming unit (216) configured to retrieve the plurality of filtered network packets in response to filtering and aggregating the plurality of network packets, wherein the packet streaming unit (216):
a packet soft parser (216-1) configured to parse at
least one information from each of the plurality of filtered
network packets to generate a plurality of parsed network
packets; and
the database (210) configured to store the plurality of parsed
network packets along with the at least one information, wherein the
plurality of parsed network packets along with the at least one information
is further transmitted and stored in a centralized database.
2. The system as claimed in claim 1, wherein the packet streaming unit (216)
further includes:
a packet writer (216-2) configured to retrieve the plurality of parsed network packets and the at least one information from the centralized database based on one or more pre-defined topics to generate a compressed file.
3. The system as claimed in claim 1, wherein the packet soft parser (216-1) is configured to archive the plurality of parsed network packets in the database when a connection with the centralized database is lost.
4. The system as claimed in claim 3, wherein the packet soft parser (216-1) is configured to send the plurality of parsed network packets archived in the database to the centralized database upon restoring the connection.
5. The system as claimed in claim 1, wherein the packet capturing unit (212) is configured to capture the plurality of network packets based on the at least one network function by employing at least one of a port mirroring approach, an optical tapping approach, and a direct capturing approach.
6. The system as claimed in claim 1, wherein the direct capturing approach
includes receiving the plurality of network packets from the at least one
network function using a network interface card.
7. The system as claimed in claim 1, is further configured to support a Generic
Routing Encapsulation (GRE) and Encapsulated Remote Switched Port
Analyzer (ERSPAN)) encapsulation parsing for mirrored traffic in the port
mirroring approach.
8. The system as claimed in claim 1, wherein the at least one information includes a source Internet Protocol (IP), a destination IP, a port number, and an application layer payload marker.
9. The system as claimed in claim 1, wherein the set of predefined parameters includes an Internet Protocol (IP), a port, or a Virtual Local Area Network (VLAN).
10. The system as claimed in claim 1, wherein each of the plurality of network packets is a transmission control protocol (TCP) data packet.
11. The system as claimed in claim 1, wherein the pre-defined rate is in a range of fifty thousand packets per second to four hundred thousand packets per second.
12. The system as claimed in claim 1, wherein the data aggregation unit (214)
comprises at least one optical Traffic Access Point (TAP).
13. The system as claimed in claim 1, wherein the packet soft parser (216-1) is configured to support a Hypertext Transfer Protocol Version 2 (HTTP2).
14. The system of claim 1, wherein the packet writer (216-2) is configured to write data into each compressed file, and wherein each compressed file is corresponding to a specific broker topic.
15. A method for capturing and processing network packets in a network, the method comprising the steps of:
capturing (1002), by a processing engine (208), a plurality of network packets flowing at a pre-defined rate, based on at least one network function;
collecting (1004), by the processing engine (208), the plurality of network packets to filter and aggregate the plurality of network packets based on a set of predefined parameters to generate a plurality of filtered network packets;
retrieving (1006), by the processing engine (208), the plurality of filtered network packets in response to filtering and aggregating the plurality of network packets;
parsing (1008), by the processing engine (208), at least one information from each of the plurality of filtered network packets to generate a plurality of parsed network packets; and
storing (1010), by the processing engine (208), the plurality of parsed network packets along with the at least one information in a database, wherein the plurality of parsed network packets along with the at least one information is further transmitted and stored in a centralized database.
| # | Name | Date |
|---|---|---|
| 1 | 202321045152-STATEMENT OF UNDERTAKING (FORM 3) [05-07-2023(online)].pdf | 2023-07-05 |
| 2 | 202321045152-PROVISIONAL SPECIFICATION [05-07-2023(online)].pdf | 2023-07-05 |
| 3 | 202321045152-FORM 1 [05-07-2023(online)].pdf | 2023-07-05 |
| 4 | 202321045152-DRAWINGS [05-07-2023(online)].pdf | 2023-07-05 |
| 5 | 202321045152-DECLARATION OF INVENTORSHIP (FORM 5) [05-07-2023(online)].pdf | 2023-07-05 |
| 6 | 202321045152-FORM-26 [13-09-2023(online)].pdf | 2023-09-13 |
| 7 | 202321045152-FORM-26 [05-03-2024(online)].pdf | 2024-03-05 |
| 8 | 202321045152-FORM 13 [08-03-2024(online)].pdf | 2024-03-08 |
| 9 | 202321045152-AMENDED DOCUMENTS [08-03-2024(online)].pdf | 2024-03-08 |
| 10 | 202321045152-Request Letter-Correspondence [03-06-2024(online)].pdf | 2024-06-03 |
| 11 | 202321045152-Power of Attorney [03-06-2024(online)].pdf | 2024-06-03 |
| 12 | 202321045152-Covering Letter [03-06-2024(online)].pdf | 2024-06-03 |
| 13 | 202321045152-CORRESPONDANCE-WIPO CERTIFICATE-07-06-2024.pdf | 2024-06-07 |
| 14 | 202321045152-ENDORSEMENT BY INVENTORS [12-06-2024(online)].pdf | 2024-06-12 |
| 15 | 202321045152-DRAWING [12-06-2024(online)].pdf | 2024-06-12 |
| 16 | 202321045152-CORRESPONDENCE-OTHERS [12-06-2024(online)].pdf | 2024-06-12 |
| 17 | 202321045152-COMPLETE SPECIFICATION [12-06-2024(online)].pdf | 2024-06-12 |
| 18 | 202321045152-ORIGINAL UR 6(1A) FORM 26-020724.pdf | 2024-07-05 |
| 19 | Abstract1.jpg | 2024-07-12 |
| 20 | 202321045152-FORM 18 [30-09-2024(online)].pdf | 2024-09-30 |
| 21 | 202321045152-FORM 3 [07-11-2024(online)].pdf | 2024-11-07 |