DESC:TECHNICAL FIELD
The present disclosure relates generally to data processing. More particularly, the present disclosure relates to a system, and a method for text summarization.
BACKGROUND
Summarization is a process of shortening a set of data computationally, to create a subset that represents the most important or relevant information within the original content. Text summarization refers to a technique of shortening long pieces of a text without losing a context of the text. The objective of text summarization is to create a coherent and fluent summary having only the main points outlined in the document.
Automatic text summarization is a common problem in machine learning and natural language processing (NLP). However, the state of art text summarization systems require enormous training datasets for context-based text summarization. Processing the enormous datasets require high computational resources and storage capabilities of the system. Further, such systems do not provide an effective and coherent text summary with appropriate readability.
Thus, a system that provides effective readability and coherent text summarization without exploiting enormous datasets is an ongoing effort, and demands a need for improvised technical solution that overcomes the aforementioned problems.
SUMMARY
In an aspect of the present disclosure, a system is disclosed. The system includes a user device and a server. The user device is configured to facilitate a user to input text to be summarized. The server is coupled to the user device. The server includes processing circuitry configured to pre-process the text to generate a plurality of embeddings. The processing circuitry is further configured to generate first data by applying a 2-dimensional (2D) Fast Fourier Transform (FFT) technique applied on the plurality of embeddings. The processing circuitry is further configured to generate second data by applying a multi-head self-attention technique to the plurality of embeddings. The processing circuitry is further configured to generate third data by convolution of the first data and the second data. The processing circuitry is further configured to generate fractality data such that the fractality data comprising a fractal dimension of each word of the text. The processing circuitry is further configured to generate the fourth data by applying feed forward technique, 2D FFT technique, and one or more addition and normalization on the fractality data. The processing circuitry is further configured to generate fifth data by convolution of the third data and the fourth data. The processing circuitry is further configured to generate a plurality of summaries by decoding the fifth data.
In some embodiments of the present disclosure, to pre-process the text, the processing circuitry is configured to generate a plurality of pre-processed words by lower casing of each word of the text and addition of spacing between each word and every punctuation mark of the text.
In some embodiments of the present disclosure, to apply the 2D FFT technique, the processing circuitry is configured to determine a Discrete Fourier Transform (DFT) twice. To generate the first data, the processing circuitry is configured to perform one or more addition and normalization and a feed forward function on the plurality of embeddings.
In some embodiments of the present disclosure, the processing circuitry is configured to generate a Sentence Relation Graph (SRG) from the plurality of embeddings such that the processing circuitry assigns each sentence of a plurality of sentences as a node of the SRG.
In some embodiments of the present disclosure, the processing circuitry is configured to establish an edge between any two nodes of the SRG when a cosine similarity value between the two nodes exceeds a predefined threshold.
In some embodiments of the present disclosure, the processing circuitry is configured to apply a graph convolution network technique on the SRG to compute a plurality of sentence embeddings.
In some embodiments of the present disclosure, the processing circuitry is further configured to generate a contractive loss for each summary of the plurality of summaries.
In some embodiments of the present disclosure, the processing circuitry is configured to select a summary of the plurality of summaries, based on the contractive loss value of each summary of the plurality of summaries.
In an aspect of the present disclosure, a method is disclosed. The method includes a step of receiving, by way of a user device, text to be summarized. The method further includes a step of pre-processing, by way of processing circuity of a server that is coupled to the user device, the text to generate a plurality of embeddings. The method further includes a step of generating, by way of the processing circuitry, first data by applying a 2-dimensional (2D) Fast Fourier Transform (FFT) technique applied on the plurality of embeddings. The method further includes a step of generating, by way of the processing circuitry, second data by applying a multi-head self-attention technique to the plurality of embeddings. The method further includes a step of generating, by way of the processing circuitry, third data by convolution of the first data and the second data. The method further includes a step of generating, by way of the processing circuitry, fractality data such that the fractality data comprising a fractal dimension of each word of the text. The method further includes a step of generating, by way of the processing circuitry, fourth data by applying feed forward technique, 2D FFT technique, and one or more addition and normalization on the fractality data. The method further includes a step of generating, by way of the processing circuitry, fifth data by convolution of the third data and the fourth data. The method further includes a step of generating, by way of the processing circuitry, a plurality of summaries by decoding the fifth data.
BRIEF DESCRIPTION OF DRAWINGS
The above and still further features and advantages of aspects of the present disclosure becomes apparent upon consideration of the following detailed description of aspects thereof, especially when taken in conjunction with the accompanying drawings, and wherein:
FIG. 1 illustrates a block diagram of a system for text summarization, in accordance with an exemplary aspect of the present disclosure;
FIG. 2 illustrates a block diagram of a server of the system of FIG. 1, in accordance with an embodiment of the present disclosure; and
FIG. 3A and 3B illustrate a flow chart of a method for text summarization, in accordance with an exemplary aspect of the present disclosure.
To facilitate understanding, like reference numerals have been used, where possible, to designate like elements common to the figures.
DETAILED DESCRIPTION
Various aspects of the present disclosure provide a system, and a method for summarization of a text. The following description provides specific details of certain aspects of the disclosure illustrated in the drawings to provide a thorough understanding of those aspects. It should be recognized, however, that the present disclosure can be reflected in additional aspects and the disclosure may be practiced without some of the details in the following description.
The various aspects including the example aspects are now described more fully with reference to the accompanying drawings, in which the various aspects of the disclosure are shown. The disclosure may, however, be embodied in different forms and should not be construed as limited to the aspects set forth herein. Rather, these aspects are provided so that this disclosure is thorough, and fully conveys the scope of the disclosure to those skilled in the art. In the drawings, the sizes of components may be exaggerated for clarity.
It is understood that when an element or layer is referred to as being “on,” “connected to,” or “coupled to” another element or layer, it can be directly on, connected to, or coupled to the other element or layer or intervening elements or layers that may be present. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items.
The subject matter of example aspects, as disclosed herein, is described with specificity to meet statutory requirements. However, the description itself is not intended to limit the scope of this patent. Rather, the inventor/inventors have contemplated that the subject matter might also be embodied in other ways, to include different features or combinations of features similar to the ones described in this document, in conjunction with other technologies. Generally, the various aspects including the example aspects relate to the system, and the method for summarization of the text.
As mentioned, there remains a need for a system that provides effective readability and coherent text summarization without exploiting enormous datasets. The present aspect, therefore: provides a system, and a method for summarization of a text providing an effective readability and coherent text summarization, without utilizing a large training dataset. Preferably, the system may be trained on a training data obtained from Scientific too long didn’t read (SciTLDR) dataset including 1992 train samples and 618 test samples.
The aspects herein and the various features and advantageous details thereof are explained more fully with reference to the non-limiting aspects that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the aspects herein. The examples used herein are intended merely to facilitate an understanding of ways in which the aspects herein may be practiced and to further enable those of skill in the art to practice the aspects herein. Accordingly, the examples should not be construed as limiting the scope of the aspects herein.
FIG. 1 illustrates a block diagram of a system 100 for text summarization, in accordance with an exemplary aspect of the present disclosure. In an aspect of the present disclosure, the system 100 of FIG. 1 may include a user device 102 and a server 104 may be communicatively coupled to each other via a communication network 106. In other aspects of the present disclosure, the user device 102 and the server may be communicably coupled through separate communication networks established therebetween.
The user device 102 may be configured to facilitate a user to input data, receive data, and/or transmit data within the system 100. Examples of the user device 102 may include, but are not limited to, a desktop, a notebook, a laptop, a handheld computer, a touch sensitive device, a computing device, a smart-phone, and/or a smart watch. It will be apparent to a person of ordinary skill in the art that the user device 102 may include any device/apparatus that is capable of manipulation by the user.
In an exemplary aspect of the present disclosure of FIG. 1, the user device 102 may include a user interface 110, a processing unit 112, a device memory 114, a summary console 116 and a communication interface 118. The user interface 110 may include an input interface for receiving inputs from the user. Examples of the input interface of the user interface 110 may include, but are not limited to, a touch interface, a mouse, a keyboard, a motion recognition unit, a gesture recognition unit, a voice recognition unit, or the like. Some aspects of the present disclosure are intended to include or otherwise cover any type of the input interface including known, related art, and/or later developed technologies. In some aspects of the present disclosure, the user device 102 by way of the input interface may be configured to receive a text to be summarized. Preferably, the user device 102 may receive the text in English language. A length of the text may be in a range of 500 to 3000 words.
The user interface 110 may further include an output interface for displaying (or presenting) an output to the user. Examples of the output interface of the user interface 110 may include, but are not limited to, a digital display, an analog display, a touch screen display, a graphical user interface, a website, a web page, a keyboard, a mouse, a light pen, an appearance of a desktop, and/or illuminated characters. In some aspects of the present disclosure, the user device 102 by way of the output interface may be configured to display a summary of the text. Preferably, the user device 102 may display the summary of the text in English language. The length of the summary of the text may be in a range of 2-4 lines.
The processing unit 112 may include suitable logic, instructions, circuitry, interfaces, and/or codes for executing various operations, such as the operations associated with the user device 102, or the like. Further, the processing unit 112 may be configured to control one or more operations executed by the user device 102 in response to the input received at the user interface 110 from the user. Examples of the processing unit 112 may include, but are not limited to, an application-specific integrated circuit (ASIC) processor, a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a field-programmable gate array (FPGA), a Programmable Logic Control unit (PLC), and the like. Some aspects of the present disclosure are intended to include or otherwise cover any type of the processing unit 110 including known, related art, and/or later developed processing units.
The user device 102 may further include the device memory 114, configured to store the logic, instructions, circuitry, interfaces, and/or codes of the processing unit 112, data associated with the user device 102, and data associated with the system 100. Examples of the device memory 114 may include, but are not limited to, a Read-Only Memory (ROM), a Random-Access Memory (RAM), a flash memory, a removable storage drive, a hard disk drive (HDD), a solid-state memory, a magnetic storage drive, a Programmable Read Only Memory (PROM), an Erasable PROM (EPROM), and/or an Electrically EPROM (EEPROM). Some aspects of the present disclosure are intended to include or otherwise cover any type of the first memory 112 including known, related art, and/or later developed memories.
In some aspects of the present disclosure, the user device 102 may further include the summary console 116 configured as a computer-executable application, to be executed by the processing unit 112. The summary console 116 may include suitable logic, instructions, and/or codes for executing various operations and may be controlled by the server 104. The one or more computer executable applications may be stored in the device memory 114. Examples of the one or more computer executable applications may include, but are not limited to, an audio application, a video application, a social media application, a navigation application, or the like.
In some aspects of the present disclosure, the user device 102 may further include a communication interface 118, configured to enable the user device 102 to communicate with the server 104 over the communication network 106. Examples of the communication interface 118 may include, but are not limited to, a modem, a network interface such as an Ethernet card, a communication port, and/or a Personal Computer Memory Card International Association (PCMCIA) slot and card, an antenna, a radio frequency (RF) transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a coder-decoder (CODEC) chipset, a subscriber identity module (SIM) card, and a local buffer circuit. It will be apparent to a person of ordinary skill in the art that the communication interface 116 may include any device and/or apparatus capable of providing wireless or wired communications between the user device 102 and the server 104. In some aspects of the present disclosure, the communication interface 118 may be configured to send the text to the server 104. The communication interface may further be configured to receive the summary of the text from the server 104.
The server 104 may be a network of computers, a software framework, or a combination thereof, that may provide a generalized approach to create the server implementation. Examples of the server 104 may include, but are not limited to, personal computers, laptops, mini-computers, mainframe computers, any non-transient and tangible machine that can execute a machine-readable code, cloud-based servers, distributed server networks, or a network of computer systems. The server 104 may be realized through various web-based technologies such as, but not limited to, a Java web-framework, a .NET framework, a personal home page (PHP) framework, or any web-application framework. The server 104 may be maintained by a storage facility management authority or a third-party entity that facilitates service enablement and resource allocation operations of the system 100. The server 104 may include the processing circuitry 120 and one or more memory units 122a-122m (hereinafter, collectively referred to and designated as “Database 122”).
The processing circuitry 120 may include suitable logic, instructions, circuitry, interfaces, and/or codes for executing various operations, such as user matching based on interests or the like. The processing circuitry 120 may be configured to host and enable the summary console 116 running on (or installed on) the user device 102 to execute the operations associated with the system 100 by communicating one or more commands and/or instructions over the communication network 106. Examples of the processing circuitry 120 may include, but are not limited to, an ASIC processor, a RISC processor, a CISC processor, a FPGA, and the like. The processing circuitry 120 may be configured to perform various operations of the system 100.
The processing circuitry 120 may be configured to pre-process the text to generate a plurality of embeddings. In some aspects, the processing circuitry 120 may be configured to generate a plurality of pre-processed words by lower casing of each word of the text and addition of spacing between each word and every punctuation mark of the text. Further, the processing circuitry 120 may be configured to generate a numeric matrix form of each pre-processed word of the plurality of pre-processed. In some aspects, the numeric matrix form of the pre-processed words may be referred to as the plurality of embeddings of the text.
The processing circuitry 120 may further be configured to generate first data by performing a 2-Dimensional (2D) Fast Fourier Transform (FFT) on the plurality of embeddings. In some aspects, the processing circuitry 120 may be configured to perform the 2D FFT on the plurality of embeddings by calculating a Discrete Fourier Transform (DFT) twice (i.e., one along a hidden dimension and the other along a sequence dimension). The processing circuitry 120, upon performing the 2D FFT, may further be configured to perform one or more addition and normalization, and may perform a feed forward function to generate the first data.
Furthermore, the processing circuitry 120 may be configured to generate second data by applying a multi-head self-attention function to the plurality of embeddings. In some aspects, the processing circuitry 120, applying a multi-head self-attention function to the plurality of embeddings, may be configured to perform one or more addition and normalization, and may perform a feed forward function to generate the second data. Furthermore, the processing circuitry 120 may be configured to generate third data by convolution of the first data and the second data.
Furthermore, the processing circuitry 120 may be configured to generate a Sentence Relation Graph (SRG) from the plurality of embeddings. In some aspects of the present disclosure, the processing circuitry 120 may be configured to segregate the text into a plurality of sentences. The processing circuitry 120 may be configured to assign / designate each sentence of the plurality of sentences as a node of the SRG. The processing circuitry 120 may further be configured to establish an edge between any two nodes of the SRG when a cosine similarity value between the two nodes exceeds a predefined threshold. Preferably, the predefined threshold value may be 0.25.
Furthermore, the processing circuitry 120 may be configured to assign a weight to each edge that may be equal to the cosine similarity value of each edge. In some aspects of the present disclosure, the processing circuitry 120 may be configured to generate a Scientific Bidirectional Encoder Representations from Transformers (SciBERT) embedding for each sentence of the plurality of sentences. SciBERT is used as it is pre-trained on scientific papers and generates rich embeddings. The processing circuitry 120 may further be configured to re-initialize a node feature for each node or sentence. In some aspects of the present disclosure, the processing circuitry 120 may be configured to generate the SRG based on the values of the nodes and edges determined hereinabove. In some aspects of the present disclosure, the processing circuitry 120 may further be configured to apply a graph convolution network on the SRG to compute a plurality of sentence embeddings, which astutely embodies the context of neighboring sentences into the embedding of the sentence of interest, providing a comprehensive “sentence view” of the text.
Furthermore, the processing circuitry 120 may be configured to generate fractality data. In some aspects of the present disclosure, the fractality data may include a fractal dimension of each word of the text. In some aspects of the present disclosure, the processing circuitry 120 may utilize a box counting method to calculate fractal dimension of each word. The box counting method may allow an identification of repeating words within the text, as well as any variations or tweaks in the words (i.e., words of concern). The mathematical formulation for the fractal dimension through box-counting follows power-law and hence transforms to a Linear Regression on a log-log scale. In an exemplary scenario, the fractal dimension can be determined as: log N = log c + D log (1/s), where ‘N’ denotes a number of boxes that are impacted by the words of concern, ‘D’ denotes a dimensionality of the fractal, ‘s’ denotes the box size, and ‘c’ denotes a constant. In some aspects of the present disclosure, the processing circuitry 120 may be configured to determine one or more shuffled words and one or more non shuffled words from the text. The processing circuitry 120 may further be configured to determine the fractal dimension (FracDim) by determining a slope of a line, that can be formulated as: FracDim = Ds(word) / Dns(word), where Ds(word) and Dns(word) are dimensions of the one or more shuffled and dimensions of the one or more non-shuffled words, respectively. The sequence of words in a sentence plays a significant role in determining a meaning of the text. Typically, less important words may be uniformly distributed throughout the document, while important words may be clustered in certain areas. In some aspects of the present disclosure, the processing circuitry 120 may be configured to determine a measure ‘x’ before a shuffling of the text, and a measure ‘y’ after the shuffling of the text. the processing circuitry 120 may further be configured to measure a degree of fractality based on a comparison of the measure ‘x’ before the shuffling of the text, and the measure ‘y’ after the shuffling of the text. As a clustering of important words is disrupted after shuffling, the value of ‘y’ may be greater than ‘x’. However, a distribution of less important words may remain unchanged.
Furthermore, the processing circuitry 120 may be configured to perform, on the fractality data, the feed forward function, the 2D FFT, and the one or more addition and normalization, to generate fourth data. In some aspects of the present disclosure, the processing circuitry 120 may be configured to perform a fusion of the fractality data and the GCN to formulate a unified and robust model. By exploiting the fractal dimension of words, the processing circuitry 120 may identify prominent keywords within the text. The processing circuitry 120 may further be configured to calculate a prevalence of one or more top fractal words in each sentence of the text. Furthermore, based on the prevalence of the one or more top fractal words, the processing circuitry 120 may be configured to assign weights to the corresponding graph sentence embeddings. In some aspects, higher count of the one or more top fractal words in each sentence may result in higher weight of the embeddings corresponding to the SRG. Furthermore, the processing circuitry 120 may be configured to apply a weighing mechanism on the SRG processed through the GCN, and may result in an overlaid fractality data on the embeddings induced by the GCN. In some aspects, the overlaid fractality data may result in generation of one or more sentences containing one or more pivotal keywords with a higher influence value.
Furthermore, the processing circuitry 120 may be configured to generate fifth data by convolution of the third data and the fourth data. The processing circuitry may be configured to generate a plurality of summaries by decoding the fifth data. Further, the processing circuitry 120 may be configured to generate a contractive loss for each summary of the plurality of summaries. In an exemplary aspect, the contractive loss (Loss) for each summary of the plurality of summaries may be determined by mapping the text with each summary of the plurality of summaries using a non-deterministic target distribution technique and may be formulated as: Loss = ?_i¦?_(j>i)¦?max?(0,S(C_j )-S(C_i )+ß_ij)?, where Cj and Ci are two different summaries generated by the system 100, ß_ij is a difference in a rank of the two different summaries, S function calculates a word overlap between the two different summaries. Furthermore, the processing circuitry 120 may be configured to select a summary of the plurality of summaries, based on the contractive loss value of each summary of the plurality of summaries.
The database 122 may be configured to store the logic, instructions, circuitry, interfaces, and/or codes of the processing circuitry 120 for executing various operations. The database 122 may be further configured to store therein, data associated with users registered with the system 100. The data associated with the users may include, but is not limited to, training and testing datasets, instructions, and the like. Some aspects of the present disclosure are intended to include and/or otherwise cover any type of the data associated with the users registered with the system 100. Examples of the database 122 may include but are not limited to, a ROM, a RAM, a flash memory, a removable storage drive, a HDD, a solid-state memory, a magnetic storage drive, a PROM, an EPROM, and/or an EEPROM. In some aspects, a set of centralized or distributed network of peripheral memory devices may be interfaced with the server 104, as an example, on a cloud server.
The communication network 106 may include suitable logic, circuitry, and interfaces that may be configured to provide a plurality of network ports and a plurality of communication channels for transmission and reception of data related to operations of various entities (such as the user device 102 and the server 104) of the system 100. Each network port may correspond to a virtual address (or a physical machine address) for transmission and reception of the communication data. For example, the virtual address may be an Internet Protocol Version 4 (IPV4) (or an IPV6 address) and the physical address may be a Media Access Control (MAC) address. The communication network 106 may be associated with an application layer for implementation of communication protocols based on one or more communication requests from the user device 102 and the server 104. The communication data may be transmitted or received, via the communication protocols. Examples of the communication protocols may include, but are not limited to, Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), Simple Mail Transfer Protocol (SMTP), Domain Network System (DNS) protocol, Common Management Interface Protocol (CMIP), Transmission Control Protocol and Internet Protocol (TCP/IP), User Datagram Protocol (UDP), Long Term Evolution (LTE) communication protocols, or any combination thereof.
In an aspect of the present disclosure, the communication data may be transmitted or received via at least one communication channel of a plurality of communication channels in the communication network 106. The communication channels may include, but are not limited to, a wireless channel, a wired channel, a combination of wireless and wired channel thereof. The wireless or wired channel may be associated with a data standard which may be defined by one of a Local Area Network (LAN), a Personal Area Network (PAN), a Wireless Local Area Network (WLAN), a Wireless Sensor Network (WSN), Wireless Area Network (WAN), Wireless Wide Area Network (WWAN), a metropolitan area network (MAN), a satellite network, the Internet, a fiber optic network, a coaxial cable network, an infrared (IR) network, a radio frequency (RF) network, and a combination thereof. Aspects of the present disclosure are intended to include or otherwise cover any type of communication channel, including known, related art, and/or later developed technologies.
FIG. 2 illustrates a block diagram of the server 104 of the system 100 of FIG. 1, in accordance with an embodiment of the present disclosure. The server 104 may further include a network interface 200 and an input/output (I/O) interface 202. The processing circuitry 120, the database 122, the network interface 200, and the I/O interface 202 may be configured to communicate with each other by way of a first communication bus 204.
The processing circuitry 120 may include a data exchange engine 206, a registration engine 208, an authentication engine 210, a data processing engine 212, and a notification engine 214.
The data exchange engine 206, the registration engine 208, the authentication engine 210, the data processing engine 212, and the notification engine 214 may be configured to communicate with each other by way of a second communication bus 216. It will be apparent to a person skilled in the art that the server 104 is for illustrative purposes and not limited to any specific combination of hardware circuitry and/or software.
The data exchange engine 206 may be configured to enable transfer of data from the database 122 to various engines of the processing circuitry 120. The data exchange engine 206 may be further configured to enable the transfer of data and/or instructions from the user device 102 to the server 104. Specifically, the data exchange engine 206 may facilitate the processing circuitry 120 to receive the text from the user device 102.
The registration engine 208 may be configured to enable the user to register into the system 100 by providing registration data through a registration menu (not shown) of the summary console 116 that may be displayed by way of the user device 102.
The authentication engine 210 may be configured to fetch the registration data of the user. Specifically, the data exchange engine 206 may facilitate the authentication engine 210 to fetch the registration data of the user and authenticate the registration data of the user. The authentication engine 210, upon successful authentication of the registration data of the user, may be configured to enable the user to log-in or sign-up to the system 100.
In some aspects of the present disclosure, the authentication engine 210 may enable the user to set the password protection for logging-in to the system 100. In such a scenario, the authentication engine 210 may be configured to verify a password entered by the user for logging-in to the system 100 by comparing the password entered by the user with the set password protection. In some aspects of the present disclosure, when the password entered by the user is verified by the authentication engine 210, the authentication engine 210 may enable the user to log-in to the system 100. In some other aspects of the present disclosure, when the password entered by the user is not verified by the authentication engine 210, the authentication engine 208 may facilitate to generate a signal for the notification engine 214 to generate a login failure notification for the user.
The data processing engine 212 may be configured to receive the input text from the user device 102. The data processing engine 212 may be configured to pre-process the text to generate a plurality of embeddings. In some aspects, the data processing engine 212 may be configured to generate a plurality of pre-processed words by lower casing of each word of the text and addition of spacing between each word and every punctuation mark of the text. Further, the data processing engine 212 may be configured to generate a numeric matrix form of each pre-processed word of the plurality of pre-processed. In some aspects, the numeric matrix form of the pre-processed words may be referred to as the plurality of embeddings of the text. The data processing engine 212 may further be configured to generate first data by performing a 2-Dimensional (2D) Fast Fourier Transform (FFT) on the plurality of embeddings. In some aspects, the data processing engine 212 may be configured to perform the 2D FFT on the plurality of embeddings by calculating a Discrete Fourier Transform (DFT) twice (i.e., one along a hidden dimension and the other along a sequence dimension). The data processing engine 212, upon performing the 2D FFT, may further be configured to perform one or more addition and normalization, and may perform a feed forward function to generate the first data. Furthermore, the data processing engine 212 may be configured to generate second data by applying a multi-head self-attention function to the plurality of embeddings. In some aspects, the data processing engine 212, applying a multi-head self-attention function to the plurality of embeddings, may be configured to perform one or more addition and normalization, and may perform a feed forward function to generate the second data. Furthermore, the data processing engine 212 may be configured to generate third data by convolution of the first data and the second data. Furthermore, the data processing engine 212 may be configured to generate a Sentence Relation Graph (SRG) from the plurality of embeddings. In some aspects of the present disclosure, the data processing engine 212 may be configured to segregate the text into a plurality of sentences. The data processing engine 212 may be configured to assign / designate each sentence of the plurality of sentences as a node of the SRG. The data processing engine 212 may further be configured to establish an edge between any two nodes of the SRG when a cosine similarity value between the two nodes exceeds a predefined threshold. Preferably, the predefined threshold value may be 0.25. Furthermore, the data processing engine 212 may be configured to assign a weight to each edge that may be equal to the cosine similarity value of each edge. In some aspects of the present disclosure, the data processing engine 212 may be configured to generate a Scientific Bidirectional Encoder Representations from Transformers (SciBERT) embedding for each sentence of the plurality of sentences. SciBERT is used as it is pre-trained on scientific papers and generates rich embeddings. The data processing engine 212 may further be configured to re-initialize a node feature for each node or sentence. In some aspects of the present disclosure, the data processing engine 212 may be configured to generate the SRG based on the values of the nodes and edges determined hereinabove. In some aspects of the present disclosure, the data processing engine 212 may further be configured to apply a graph convolution network on the SRG to compute a plurality of sentence embeddings, which astutely embodies the context of neighboring sentences into the embedding of the sentence of interest, providing a comprehensive “sentence view” of the text. Furthermore, the data processing engine 212 may be configured to generate fractality data. In some aspects of the present disclosure, the fractality data may include a fractal dimension of each word of the text. In some aspects of the present disclosure, the data processing engine 212 may utilize a box counting method to calculate fractal dimension of each word. The box counting method may allow an identification of repeating words within the text, as well as any variations or tweaks in the words (i.e., words of concern). The mathematical formulation for the fractal dimension through box-counting follows power-law and hence transforms to a Linear Regression on a log-log scale. In an exemplary scenario, the fractal dimension can be determined as: log N = log c + D log (1/s), where ‘N’ denotes a number of boxes that are impacted by the words of concern, ‘D’ denotes a dimensionality of the fractal, ‘s’ denotes the box size, and ‘c’ denotes a constant. In some aspects of the present disclosure, the data processing engine 212 may be configured to determine one or more shuffled words and one or more non shuffled words from the text. The data processing engine 212 may further be configured to determine the fractal dimension (FracDim) by determining a slope of a line, that can be formulated as: FracDim = Ds(word) / Dns(word), where Ds(word) and Dns(word) are dimensions of the one or more shuffled and dimensions of the one or more non-shuffled words, respectively. The sequence of words in a sentence plays a significant role in determining a meaning of the text. Typically, less important words may be uniformly distributed throughout the document, while important words may be clustered in certain areas. In some aspects of the present disclosure, the data processing engine 212 may be configured to determine a measure ‘x’ before a shuffling of the text, and a measure ‘y’ after the shuffling of the text. the data processing engine 212 may further be configured to measure a degree of fractality based on a comparison of the measure ‘x’ before the shuffling of the text, and the measure ‘y’ after the shuffling of the text. As a clustering of important words is disrupted after shuffling, the value of ‘y’ may be greater than ‘x’. However, a distribution of less important words may remain unchanged. Furthermore, the data processing engine 212 may be configured to perform, on the fractality data, the feed forward function, the 2D FFT, and the one or more addition and normalization, to generate fourth data. In some aspects of the present disclosure, the data processing engine 212 may be configured to perform a fusion of the fractality data and the GCN to formulate a unified and robust model. By exploiting the fractal dimension of words, the data processing engine 212 may identify prominent keywords within the text. The data processing engine 212 may further be configured to calculate a prevalence of one or more top fractal words in each sentence of the text. Furthermore, based on the prevalence of the one or more top fractal words, the data processing engine 212 may be configured to assign weights to the corresponding graph sentence embeddings. In some aspects, higher count of the one or more top fractal words in each sentence may result in higher weight of the embeddings corresponding to the SRG. Furthermore, the data processing engine 212 may be configured to apply a weighing mechanism on the SRG processed through the GCN, and may result in an overlaid fractality data on the embeddings induced by the GCN. In some aspects, the overlaid fractality data may result in generation of one or more sentences containing one or more pivotal keywords with a higher influence value. Furthermore, the data processing engine 212 may be configured to generate fifth data by convolution of the third data and the fourth data. The processing circuitry may be configured to generate a plurality of summaries by decoding the fifth data. Further, the data processing engine 212 may be configured to generate a contractive loss for each summary of the plurality of summaries. In an exemplary aspect, the contractive loss (Loss) for each summary of the plurality of summaries may be determined by mapping the text with each summary of the plurality of summaries using a non-deterministic target distribution technique and may be formulated as: Loss = ?_i¦?_(j>i)¦?max?(0,S(C_j )-S(C_i )+ß_ij)?, where Cj and Ci are two different summaries generated by the system 100, ß_ij is a difference in a rank of the two different summaries, S function calculates a word overlap between the two different summaries. Furthermore, the data processing engine 212 may be configured to select a summary of the plurality of summaries, based on the contractive loss value of each summary of the plurality of summaries.
The notification engine 214 may be configured to facilitate generation of one or more notifications corresponding to the system 100. The one or more notifications may be presented to the user by way of the user device 102. It will be apparent to a person skilled in the art that the aspects of the present disclosure are intended to include and/or otherwise cover any type of notification generated by the system 100 and/or presented to the user by the system 100, without deviating from the scope of the present disclosure.
The database 122 may be configured to store data corresponding to the system 100. The database 122 may be segregated into one or more repositories that may be configured to store a specific type of data. The database 122 may include an instructions repository 218, a user data repository 220, and a summary repository 222.
The instructions repository 218 may be configured to store instructions data corresponding to the server 104. The instructions data may include data and metadata of one or more instructions corresponding to the various entities of the server 104 such as the processing circuitry 120, the network interface 200, and the I/O interface 202. It will be apparent to a person skilled in the art that the aspects of the present disclosure are intended to include and/or cover any type of instructions data of the server 104, and thus must not be considered as a limitation of the present disclosure.
The user data repository 220 may be configured to store user data of the system 100. The user data may include data and metadata of the data of authenticated users that may be registered on the system 100. The user data repository 220 may further be configured to store partial data and/or partial metadata of the user data corresponding to the users that may fail to register and/or authenticate on the system 100. Furthermore, the user data repository 220 may be configured to store the set of inputs received from the user by way of the user device 102. It will be apparent to a person skilled in the art that the aspects of the present disclosure are intended to include and/or otherwise cover any type of the user data and/or metadata of the user data of the system 100, and thus must not be considered as a limitation of the present disclosure.
The summary repository 222 may be configured to store the plurality of summaries. Specifically, the summary repository 222 may be configured to store the plurality of summaries corresponding to the text such that a target summary may be selected from the plurality of summaries.
FIG. 3A and 3B illustrate a flow chart of a method 300 for text summarization, in accordance with an exemplary aspect of the present disclosure.
At step 302, the system 100, by way of the user device 102, may receive the text to be summarized.
At step 304, the system 100, by way of the processing circuitry 120, may pre-process the text to generate the plurality of embeddings.
At step 306, the system 100, by way of the processing circuitry 120, may generate the first data by performing the 2-Dimensional (2D) Fast Fourier Transform (FFT) on the plurality of embeddings.
At step 308, the system 100, by way of the processing circuitry 120, may generate the second data by applying the multi-head self-attention function to the plurality of embeddings.
At step 310, the system 100, by way of the processing circuitry 120, may generate the third data by the convolution of the first data and the second data.
At step 312, the system 100, by way of the processing circuitry 120, may generate the SRG from the plurality of embeddings.
At step 314, the system 100, by way of the processing circuitry 120, may generate the fractality data including the fractal dimension of each word of the text by training the SRG with the GCN,
At step 316, the system 100, by way of the processing circuitry 120, may generate the fourth data by applying the 2D FFT to the fractality data.
At step 318, the system 100, by way of the processing circuitry 120, may generate the fifth data by convolution of the third data and the fourth data.
At step 320, the system 100, by way of the processing circuitry 120, may generate the plurality of summaries by decoding the fifth data such that the processing circuitry 120 may be further configured to select a target summary from the plurality of summaries.
At step 322, the system 100, by way of the processing circuitry 120, may determine the mapping between the text and the plurality of summaries.
At step 324, the system 100, by way of the processing circuitry 120, may generate the contractive loss value for each summary of the plurality of summaries.
At step 326, the system 100, by way of the processing circuitry 120, may select the summary of the plurality of summaries, based on the contractive loss value of each summary of the plurality of summaries.
The system 100 enables the fractality data subsequently added to an encoder layer of the processing circuitry 120. The system 100 generates static embeddings fused with fractality added to the encoder layer's output. Since the dimensions aren't the same (the dimensions of the graph depend on the number of sentences in the document), the system 100 incorporates dense layers to keep the dimension consistent. The system 100 incorporates 2D FFT to boost the performance further and compensate for not making it a trainable parameter. The decoder remains unaltered. The training regime is contrastive learning. Thus, the system 100 provides effective readability and coherent text summarization without exploiting enormous datasets.
The foregoing discussion of the present disclosure has been presented for purposes of illustration and description. It is not intended to limit the present disclosure to the form or forms disclosed herein. In the foregoing Detailed Description, for example, various features of the present disclosure are grouped together in one or more aspects, configurations, or aspects for the purpose of streamlining the disclosure. The features of the aspects, configurations, or aspects may be combined in alternate aspects, configurations, or aspects other than those discussed above. Rather, as the description reflect, inventive aspects lie in less than all features of a single foregoing disclosed aspect, configuration, or aspect.
Moreover, though the description of the present disclosure has included description of one or more aspects, configurations, or aspects and certain variations and modifications, other variations, combinations, and modifications are within the scope of the present disclosure, e.g., as may be within the skill and knowledge of those in the art, after understanding the present disclosure. It is intended to obtain rights which include alternative aspects, configurations, or aspects to the extent permitted, including alternate, interchangeable and/or equivalent structures, functions, ranges or steps, whether or not such alternate, interchangeable and/or equivalent structures, functions, ranges or steps are disclosed herein, and without intending to publicly dedicate any patentable subject matter.
As one skilled in the art will appreciate, the system 100 includes a number of functional blocks in the form of a number of units and/or engines. The functionality of each unit and/or engine goes beyond merely finding one or more computer algorithms to carry out one or more procedures and/or methods in the form of a predefined sequential manner, rather each engine explores adding up and/or obtaining one or more objectives contributing to an overall functionality of the system 100. Each unit and/or engine may not be limited to an algorithmic and/or coded form, rather may be implemented by way of one or more hardware elements operating together to achieve one or more objectives contributing to the overall functionality of the system 100. Further, as it will be readily apparent to those skilled in the art, all the steps, methods and/or procedures of the system 100 are generic and procedural in nature and are not specific and sequential.
Certain terms are used throughout the description to refer to particular features or components. As one skilled in the art will appreciate, different persons may refer to the same feature or component by different names. This document does not intend to distinguish between components or features that differ in name but not structure or function. While various aspects of the present disclosure have been illustrated and described, it will be clear that the present disclosure is not limited to these aspects only. Numerous modifications, changes, variations, substitutions, and equivalents will be apparent to those skilled in the art, without departing from the spirit and scope of the present disclosure, as described hereinabove. ,CLAIMS:1. A system (100) comprising:
a user device (102) configured to facilitate a user to input text to be summarized;
a server (104) coupled to the user device (102), the server (104) comprising:
processing circuitry (120) configured to:
pre-process the text to generate a plurality of embeddings;
generate first data by applying a 2-dimensional (2D) Fast Fourier Transform (FFT) technique applied on the plurality of embeddings;
generate second data by applying a multi-head self-attention technique to the plurality of embeddings;
generate third data by convolution of the first data and the second data;
generate fractality data such that the fractality data comprising a fractal dimension of each word of the text;
generate fourth data by applying feed forward technique, 2D FFT technique, and one or more addition and normalization on the fractality data;
generate fifth data by convolution of the third data and the fourth data; and
generate a plurality of summaries by decoding the fifth data.

2. The system (100) as claimed in claim 1, wherein to pre-process the text, the processing circuitry (120) is configured to generate a plurality of pre-processed words by lower casing of each word of the text and addition of spacing between each word and every punctuation mark of the text.

3. The system (100) as claimed in claim 1, wherein to apply the 2D FFT technique, the processing circuitry (120) is configured to determine a Discrete Fourier Transform (DFT) twice and wherein to generate the first data, the processing circuitry (120) is configured to perform one or more addition and normalization and a feed forward function on the plurality of embeddings.

4. The system (100) as claimed in claim 1, wherein the processing circuitry (120) is configured to generate a Sentence Relation Graph (SRG) from the plurality of embeddings such that the processing circuitry (120) assigns each sentence of a plurality of sentences as a node of the SRG.

5. The system (100) as claimed in claim 4, wherein the processing circuitry (120) is configured to establish an edge between any two nodes of the SRG when a cosine similarity value between the two nodes exceeds a predefined threshold.

6. The system (100) as claimed in claim 4, the processing circuitry (120) is configured to apply a graph convolution network technique on the SRG to compute a plurality of sentence embeddings.

7. The system (100) as claimed in claim 1, wherein the processing circuitry (120) is further configured to generate a contractive loss for each summary of the plurality of summaries.

8. The system (100) as claimed in claim 7, wherein the processing circuitry (120) is configured to select a summary of the plurality of summaries, based on the contractive loss value of each summary of the plurality of summaries.

9. A method (300) comprising:
receiving (302), by way of a user device (102), text to be summarized;
pre-processing (304), by way of processing circuity (120) of a server (104) that is coupled to the user device (102), the text to generate a plurality of embeddings;
generating (306), by way of the processing circuitry (120), first data by applying a 2-dimensional (2D) Fast Fourier Transform (FFT) technique applied on the plurality of embeddings;
generating (308), by way of the processing circuitry (120), second data by applying a multi-head self-attention technique to the plurality of embeddings;
generating (310), by way of the processing circuitry (120), third data by convolution of the first data and the second data;
generating (314), by way of the processing circuitry (120), fractality data such that the fractality data comprising a fractal dimension of each word of the text;
generating (316), by way of the processing circuitry (120), fourth data by applying feed forward technique, 2D FFT technique, and one or more addition and normalization on the fractality data;
generating (318), by way of the processing circuitry (120), fifth data by convolution of the third data and the fourth data; and
generating (320), by way of the processing circuitry (120), a plurality of summaries by decoding the fifth data.

10. The method (300) as claimed in claim 9, wherein to pre-process the text, the processing circuitry (120) is configured to generate a plurality of pre-processed words by lower casing of each word of the text and addition of spacing between each word and every punctuation mark of the text.