[01] The present invention generally relates to social networking via message
applications technologies and more particularly relate to a method and a system for
generating recommendation of stickers in a group chat using reinforcement learning.
BACKGROUND
[02] Nowadays, internet communication via messaging or chat applications enable
users to visually express their thoughts or emotions through graphical content like
stickers, emoticons, animations, graphic user interfaces (GIFs), etc. One of the most
popularly used graphical content for such communication is the stickers. Generally, a
sticker represents a combination of an artwork with a stylized text. In many
messaging applications, the stickers are used by users to convey their expressions
either along with a text message or in place of a text message. Such stickers are
available as packs that the users download from an online application store and add to
a palette of their messaging applications. This allows the users to use the stickers and
share the stickers in chats, such as chat between two users or group chat among many
users for communicating with one another. Moreover, the stickers allow the users to
save time in typing a text, and continue the conversation.
[03] However, in case of a group chat, some users may face difficulty in searching
relevant stickers from a large number of stickers. The search for the relevant stickers
may be a time-consuming and cumbersome task for the users. Moreover, the users
may not find relevant stickers based on their preferences. For example, the users may
be chatting on a topic “Christmas holiday” and the users may prefer stickers related to
a season or theme of the Christmas holiday. Some users may prefer stickers of certain
characters, e.g. a Santa Claus and some users may prefer the stickers of Christmas
trees. In case the users do not find stickers of their preferences, then the users may
3
lose interest to engage in the group chats. This may prevent the users from
participating in the group chats, resulting in users dropping out of the group chat.
[04] Therefore, in view of the above problems, there is a need to provide a technical
solution that enables users to select relevant stickers in a group chat in an efficient
and feasible manner.
SUMMARY
[05] Various embodiments are provided for generating recommendation of stickers in
a group chat. As used herein, the group chat corresponds to a group chat session
between a plurality of users via a chat application interface. The chat application
interface may include, but is not limited to, an application interface of an instant
messaging (IM) application, a social media chat application, a web-based messaging
application and/or the like. In many example scenarios, the plurality of users having
the group chat via the chat application interface may be hesitant to participate in the
group chat due to difficulty in finding relevant context or stickers. For instance, one
user may be hesitant or not interested in typing a lengthy text message, and another
user may either be reluctant or may find it difficult to search for the relevant stickers
from a large number of stickers available in the chat application interface. The
present invention provides an efficient and user friendly assistance to the users to
respond in the group chat.
[06] Various embodiments are provided for generating the recommendation stickers
based on a context of the group chat between the plurality of users using
reinforcement learning technique. The context may include conversations related to
one or more topics in textual format or other formats like audio, video, and stickers.
Additionally, the context may include information such as geographical locations of
the plurality of users, time-zone information associated with the plurality of users,
season related information associated with the plurality of users, weather related
4
information associated with the plurality of users, or the like. For example, context of
a group chat may correspond to a festival topic. The users in the group chat may
belong to different geographical location, time-zones, weather conditions, or the like.
In some embodiments, the context is represented as user states of the respective users.
Typically, a user state is a state-vector which is encoded based on a sequence of
exchanged content, such as stickers, that corresponds to chat specific information
represented by state variables. The user states are encoded into the user state vectors
based on a federated learning model. The federated learning model enables individual
encoding of the user states locally at corresponding user equipment of the users. The
user state vectors are shared to a centralized server system.
[07] In an embodiment of the present invention, the user equipment of the users are
associated with a learning agent, such as a reinforced learning (RL) agent, hereinafter
“RL agent”. Each RL agent associated with each of the user equipment independently
runs to locally encode the user states (i.e. the context of the group chat and
experiences comprising historical state, actions, or rewards of each of the RL agents)
into the user state vectors. In a similar manner, the server system is associated with a
RL agent.
[08] The server RL agent guides the RL agents in the user equipment regarding
direction of emotion or expression in the group chat. The server system collects the
user state vectors and updates a policy function of the server RL agent. The policy
function herein provides values for behavior or actions of the server’s RL agent. The
server RL agent takes an action, i.e. generation of expression (e.g. sticker) suggestion
vector based on the updated policy function. The server system communicates the
suggestion vector to each of the RL agents in the user equipment. Each of the user
equipment appends the expression suggestion vector with its own state vectors to
obtain a final state vector for generating a recommendation of stickers. The
suggestion vector is appended until an updated suggestion vector is received from the
server system. The collection of the user state vectors by the server system and
5
sending of the suggestion vector to the user equipment, is asynchronously
communicated between the server system and the user equipment. In some
embodiments, the user state vectors are concatenated with encoded experiences of
corresponding RL agents in the user equipment. Each encoded experience of each of
the corresponding RL agents comprises historical information related to user states,
actions of each of the RL agents, and rewards obtained by each RL agent.
[09] Further, the system determines a group level context and a conversational
direction of chats (i.e., a chat trajectory) based on the user state vectors and the
expression suggestion vector. The group level context is a higher level understanding
of the group chat by the server RL agent. The server system estimates a
conversational direction for future chats to be sent to intended-users based on the
determined group level context and the chat trajectory corresponding to conversation
exchanged between the users. The expression suggestion vector is generated based on
group level context and the estimated conversational direction for future chat. . Each
RL agent receives the expression suggestion vector based on the group level context
and the estimated conversational direction for future chat (hereinafter referred to
future chat trajectory), on the basis of which one or more stickers are recommended
to corresponding user via the chat application interface. Further, the RL agent
receives a reward when a sticker from the recommended one or more stickers is
selected by the user. The user exchanges the selected sticker with other users during
conversation in the group chat. When the selected sticker is exchanged, a new
sequence of recent exchanges of stickers represented by a new user state is generated
for each of the users.
[010] After sending the selected sticker, recommendation of stickers for next
response of the user in the group chat, such as follow-up sticker, is generated for the
user by the RL agent. The RL agent decides for the recommendation of the follow-up
stickers by encoding the new user state based on a look-up table comprising a set of
stickers. The look-up table may correspond to a hash map that maps each sticker and
6
a word to an expression embedding space, using a similarity metric, e.g. a cosine
similarity metric. Additionally, the look-up table may comprise weights for each
sticker in the set of stickers. The RL agent distributes weights on the set of stickers
for the recommendation of the follow-up stickers. This process corresponds to
Markov Decision Process (MDP). The MDP provides a policy function that
determines policies corresponding to behavior or actions of the RL agents. The
policies depend on the user states, i.e. the context of the group chat, parameters such
as weights assigned to each recommended sticker, and action taken by the user, such
as the sticker selected by the user among the recommended stickers. The MDP also
provides a value function that provides value of each action of the RL agents. The
policy function and value function controls the RL agent. The RL agent generates
next action of generating the recommendation of follow-up stickers when the policy
function and the value function are updated. In some embodiments, the policy
function and the value function are updated after a pre-defined time interval. After the
pre-defined time interval, the chat trajectories from recent exchanges of stickers are
used to update the policy and value functions. For instance, the policy and value
functions are updated using a multi-agent policy gradient algorithm. The multi-agent
policy gradient may comprise multiple critic network formed by each of the user
equipment and one actor network for actions of the RL agents in the user equipment.
Each critic network learns a value function of each of RL agents. The actor network
learns to take action for each user of the user equipment based on estimated critic
values (i.e. reward or feedback) provided by the critic network. Further, the critic
network and the actor network are updated with all agent experiences based on the
feedback. Further, in some embodiments, the policy and value functions are updated
in a sequential manner. In some other embodiments, the policy and value functions
may be updated in an episodic manner on basis of a pre-defined time interval, such as
time delta-T and/or thread of chat content among the users in the group chat. In some
embodiments, an episode of a chat may correspond to a start till an end of a chat
conversation on a topic. The policy and value functions may be updated when the
7
chat conversation of the particular topic ends. In an alternate manner, the end of the
episode may be determined heuristically when there is no sticker exchange for some
time in the chat conversation.
[011] Accordingly, embodiments of the present invention disclose a method for
recommending stickers in a group chat. The method includes receiving, using a
federated learning model, user state vectors associated with a plurality of user states
of a plurality of users in a group chat, wherein the plurality of user states correspond
to a sequence of stickers exchanged in the group chat. The method includes
determining a group level context and a conversational direction for future chat of the
group chat based on the user state vectors using a reinforcement learning (RL) model.
The method includes generating an expression suggestion vector based on the group
level context and the conversational direction for future chat of the group chat. The
method further includes sending the expression suggestion vector to respective RL
agent in each user equipment of each of the plurality of users.
[012] Accordingly, embodiments of the present invention disclose a system for
recommending stickers in a group chat. The system comprises a memory configured
to store a set of instructions to be executed by a processor. The processor is
configured to receive, via a communication interface, user state vectors associated
with a plurality of users states of a plurality of users in a group chat using a federated
learning model. The plurality of user states correspond to a sequence of recent past
exchange of a chat content comprising sticker. The processor is configured to
determine a group level context and a conversational direction for future chat of the
group chat based on the user state vectors using a reinforcement learning (RL) model.
The processor is configured to generate an expression suggestion vector based on the
group level context and the conversational direction for future chat of the group chat.
The processor is further configured to send, via the communication interface, the
expression suggestion vector to respective RL agent in each user equipment of each
of the plurality of users.
8
[013] The foregoing summary is illustrative only and is not intended to be in any
way limiting. In addition to the illustrative aspects, embodiments, and features
described above, further aspects, embodiments, and features will become apparent by
reference to the drawings and the following detailed description.
BRIEF DESCRIPTION OF THE DRAWINGS
[014] FIG. 1 illustrates an environment representation depicting a plurality of
users in a group chat, in accordance with an example embodiment of the present
invention;
[015] FIG. 2 shows the system of FIG. 1 for generating a recommendation of
stickers in a group chat, in accordance with an example embodiment of the present
invention;
[016] FIG. 3 shows a method flow diagram for generating a recommendation of
stickers in a group chat, in accordance with an example embodiment of the present
invention; and
[017] FIG. 4 illustrates an exemplary representation of user-interfaces (UIs) of
corresponding user equipment of the plurality of users displaying recommendation of
stickers to the plurality of users in a group chat, in accordance with an example
embodiment of the present invention.
DETAILED DESCRIPTION
[018] In the following description, for purposes of explanation, numerous specific
details are set forth in order to provide a thorough understanding of the present
disclosure. It will be apparent, however, to one skilled in the art that the present
disclosure may be practiced without these specific details.
[019] Throughout the following description, numerous references may be made
regarding servers, services, or other systems formed from computing devices. It
should be appreciated that the use of such terms is deemed to represent one or more
9
computing devices having at least one processor configured to or programmed to
execute software instructions stored on a computer readable tangible, non-transitory
medium or also referred to as a processor readable medium. For example, a server
can include one or more computers operating as a web server, data source server, or
other type of computer server in a manner to fulfill described roles, responsibilities,
or functions. Within the context of this document, the disclosed modules are also
deemed to comprise computing devices having a processor and a non-transitory
memory storing instructions executable by the processor that cause the device to
control, manage, or otherwise manipulate the features of the devices or systems.
[020] The embodiments are described herein for illustrative purposes and are
subject to many variations. It is understood that various omissions and substitutions
of equivalents are contemplated as circumstances may suggest or render expedient
but are intended to cover the application or implementation without departing from
the spirit or the scope of the present disclosure. Further, it is to be understood that the
phraseology and terminology employed herein are for the purpose of the description
and should not be regarded as limiting. Any heading utilized within this description is
for convenience only and has no legal or limiting effect.
[021] The term “sticker” used herein throughout the description may include a
character-driven illustration representing a message, an action or emotion for using in
a chat, on a messaging application, a social media application, or the like.
[022] FIG. 1 illustrates an environment representation (100) for generating a
recommendation of stickers to users, such as a user (102A), a user (102B) and a user
(102C) of a group (106) in a group chat, in accordance with an example embodiment.
The users (102A-102C) communicate with each other in the group chat using
corresponding user equipment, such as a user equipment (104A), a user equipment
(104B) and a user equipment (104C). Some examples of the user equipment (104A104C) include, but are not limited to, a desktop computer, a laptop, a tablet, a phablet,
a smart mobile phone, a personal digital assistant (PDA) or any such device obvious
10
to a person skilled in the art. Each user equipment, i.e. the user equipment (104A), the
user equipment (104B) and the user equipment (104C), includes an application
interface (not shown in FIG. 1) hosted by a server (108). The application interface
may include application interface associated with a chat application, an instant
messaging (IM) application, a messaging application in a social media network,
and/or the like.
[023] In an illustrative example scenario, the users (102A-102C) are engaged in
the group chat using the application interface in each of the user equipment (104A104B). Further, the application interface associated with each of the user equipment
(104A-104C) connects with the server (108) via a network (112). The network (112)
may include suitable logic, circuitry, and interfaces that may be configured to provide
a plurality of network ports and a plurality of communication channels for
transmission and reception of data. Each network port may correspond to a virtual
address (or a physical machine address) for transmission and reception of the
communication data. For example, the virtual address may be an Internet Protocol
Version 4 (IPv4) (or an IPv6 address) and the physical address may be a Media
Access Control (MAC) address. The network (112) may be associated with an
application layer for implementation of communication protocols based on one or
more communication requests from at least one of the one or more communication
devices. The communication data may be transmitted or received, via the
communication protocols. Examples of such wired and wireless communication
protocols may include, but are not limited to, Transmission Control Protocol and
Internet Protocol (TCP/IP), User Datagram Protocol (UDP), Hypertext Transfer
Protocol (HTTP), File Transfer Protocol (FTP), ZigBee, EDGE, infrared (IR), IEEE
802.11, 802.16, cellular communication protocols, and/or Bluetooth (BT)
communication protocols.
[024] Examples of the network 112 may include, but is not limited to, a wireless
channel, a wired channel, a combination of wireless and wired channel thereof. The
11
wireless or wired channel may be associated with a network standard which may be
defined by one of a Local Area Network (LAN), a Personal Area Network (PAN), a
Wireless Local Area Network (WLAN),, Wireless Area Network (WAN), Wireless
Wide Area Network (WWAN), a Long Term Evolution (LTE) network, a plain old
telephone service (POTS), and a Metropolitan Area Network (MAN). Additionally,
the wired channel may be selected on the basis of bandwidth criteria. For example, an
optical fiber channel may be used for a high bandwidth communication. Further, a
coaxial cable-based or Ethernet-based communication channel may be used for
moderate bandwidth communication.
[025] The users (102A-102C) may exchange texts, stickers, audio, video and
content of other formats in the group chat of the group (106) using the application
interfaces in the respective user equipment (104A-104C). In an example scenario, one
user, such as the user (102A) is actively engaged in the group chat of the group (106).
The other users (102B) and (102C) may take time to respond or may be reluctant to
respond in the group chat. For instance, the (102B) may be manually searching for an
appropriate sticker in the application interface and the user (102C) may not be
interested in either texting or finding any relevant sticker in an instant to respond
back. In such scenario, the server (108) provides a recommendation of stickers for the
users (102A-102C). The server 108 includes a system (110) that facilitates the
recommendation of stickers to the users (102A-102C). In one example embodiment,
the system (110) includes an RL model that facilitates the recommendation of stickers
using federated learning technique. The recommendation of stickers is generated
based on context of group chat of the group (116). The context may further comprise
information related to the users (102A-102C), such as geographical locations of the of
users (102A-102C), time-zone information associated with the users (102A-102C),
season related information associated with the users (102A-102C), weather related
information associated with the users (102A-102C), or the like. The information may
12
be stored in a database, such as a database (114) of FIG. 1. The database (114) may
be embodied within the server (108) or may be associated with the server (108).
[026] In some embodiments, the system (110) allocates an RL agent to each of the
user equipment (104A-104C). In one example embodiment, each RL agent is
allocated when the users (102A-102C) download or access chat application interfaces
(hosted by the server (108)) in respective user equipment (104A-104C). Each RL
agent analyzes a sequence of exchanges of chat contents comprising stickers in the
group chat. Each sequence of the exchanged stickers corresponds to a user state that
represents state variables specific to a chat context of the group (106). Moreover, the
sequence of exchanged chat content corresponds to a recent history of exchanged
stickers. For instance, the user (102A) shares a sticker (i.e. ‘hi’) in the group chat to
which the users (102B) and (102C) respond with stickers ‘hello’ and ‘ola!’,
respectively. The exchange of stickers (‘hi’, ‘hello’, and ‘ola!’) form the sequence of
exchanged chat content. The user states are encoded into user state vectors using a
federated learning model. According to some embodiments, each RL agent of the
user equipment 104A-104C locally encodes corresponding user state of each of the
users (102A-102C) into a user state vector. Each user state of a user (i.e., the user
(102A), the user (102B) and the user (102C)) corresponds to a semantic or vector
representation of a sticker (i.e. from the sequence of the exchanged stickers) that is
encoded into a user state vector. The user state vectors may be concatenated with
encoded experiences (or experience vectors) of corresponding RL agents in the user
equipment (104A-104C). Each encoded experience of each of the corresponding RL
agents comprises historical information related to user states, actions of each of the
RL agents, and rewards obtained by each RL agent.
[027] The RL agents send the user state vectors of the users (102A-102C) to the
system (110) via the network (112). The system (110) receives the user state vectors
and updates a policy function of an RL model or RL agent in the system (110) to
generate expression (i.e. sticker) suggestion vector. The system (110) determines a
13
group context and chat trajectory for estimating future chat direction in the group chat
based on the encoded user states. The expression suggestion vector is shared to each
of the RL agents based on the group context and the estimated future chat direction
(i.e. future chat trajectory).
[028] Further, the RL agents generate a recommendation of stickers using the
received suggestion vector. Each of the RL agents concatenates the suggestion vector
with its corresponding user state vector of each of the users (102A-102C) to obtain a
final state vector. The recommendation of stickers is generated based on the final
state vector. The recommended stickers are displayed to each of the users (102A102C) (via the chat application interfaces). When each of the users (102A-102C)
selects a sticker from the recommendation of stickers, corresponding RL agent is
rewarded. The users (102A-102C) exchange the selected stickers to each other in the
group chat and a new sequence of exchanges of stickers is generated. The new
sequence of exchanges of stickers provides a new user state for each of the users
(102A-102C). The RL agents decide for a recommendation of follow-up stickers for
next responses of the users (102A-102C) based on Markov Decision Process (MDP).
The MDP provides a policy function and a value function for determining policies
corresponding to behavior or actions of the RL agents and provides value of each
action of the RL agents, respectively. The policies are determined based on the user
states, such as the context of the group chat, weights assigned to each sticker in the
recommendation, selection of one or more stickers from the recommendation of
stickers by the users (102A-102C), or the like. When the policy function and the
value function are updated, the RL agents generate the recommendation of follow-up
stickers. The policy function and the value function are updated based on a predefined time interval. Additionally, the recommendation of follow-up stickers is
generated by encoding the new user states using a look-up table. The look-up table
stores a set of stickers for the recommendation of follow-up stickers as well as
weights for each sticker in the set of stickers. For example, the look-up table may
14
include a hash map that maps each sticker and a word to an expression embedding
space using a similarity metric, such as a cosine similarity metric. In some
embodiments, the follow-up stickers are mapped into the set of stickers using a
similarity metric, such as a cosine similarity metric. The RL agents distribute weights
on the set of stickers for the recommendation of the follow-up stickers. In some other
embodiments, the policy and value functions are updated in an episodic manner based
on experiences of the RL agents and reward or feedback for the RL agents.
[029] FIG. 2 shows a block diagram of the system (110) for generating a
recommendation of stickers to the users (102A-102C) in a group chat, in accordance
with an example embodiment of the present invention. The system (110) may
comprise a processing means such as processor (204), storage means such as memory
(202), and a communication means such as communication interface (206). The
processor (204) may retrieve computer program code instructions that may be stored
in the memory (202) for execution of the computer program code instructions.
[030] The memory (202) is configured to store a set of instructions to be executed
by the processor (204) for generating the recommendation of stickers. In some
embodiments, the set of instructions in the memory (202) may include stores software
modules, such as contextual encoding module (208), a reinforcement learning module
(210) and a sticker recommendation module (212). The contextual encoding module
(208) includes program codes for understanding group level contextual information
of a group chat and determining chat trajectories in the group chat based on the user
state vectors of the users (102A-102C) in the group chat. (210) In some embodiment,
the RL model receives user states vectors with encoded experiences of each of the
corresponding RL agents in the user equipment (104A-104C). The reinforcement
learning module (210) includes a policy function of the RL model. The policy
function provides values for taking actions of server RL agent of the RL model, such
as generation of expression suggestion vector for each RL agent associated with
respective user equipment. The expression suggestion vector is generated by updating
15
a policy function of the RL model. Additionally, the reinforcement learning module
(210) includes a value function for controlling (e.g. updating) the RL model based on
the user state vectors. The policy function determines actions of each of the RL
agents in generating the recommendation of stickers (i.e. the suggested expression)
and the value function determines values for actions of each of the RL agents. The
sticker recommendation module (212) stores suggested expressions or stickers for the
group chat, a look-up table for the group chat, weights of each sticker recommended
to users of a group chat, information related to rewards for the RL agents in the user
equipment (104A-104C).
[031] The memory (202) may be non-transitory and may include, for example,
one or more volatile and/or non-volatile memories. In other words, for example, the
memory (202) may be an electronic storage device (for example, a computer readable
storage medium) comprising gates configured to store data (for example, bits) that
may be retrievable by a machine (for example, a computing device like the processor
(204)). The memory (202) may be configured to store information, data, content,
applications, instructions, or the like, for enabling the system (110) to carry out
various functions in accordance with an example embodiment of the present
invention. For example, the memory (202) could be configured to buffer input data
for processing by the processor (204). As exemplarily illustrated in FIG. 2, the
memory (202) may be configured to store instructions for execution by the processor
(204). As such, whether configured by hardware or software methods, or by a
combination thereof, the processor (204) may represent an entity (for example,
physically embodied in circuitry) capable of performing operations according to an
embodiment of the present invention while configured accordingly. Thus, for
example, when the processor (204) is embodied as an ASIC, FPGA or the like, the
processor (204) may be specifically configured hardware for conducting the
operations described herein. Alternatively, as another example, when the processor
(204) is embodied as an executor of software instructions, the instructions may
16
specifically configure the processor (204) to perform the algorithms and/or operations
described herein when the instructions are executed. However, in some cases, the
processor (204) may be a processor specific device (for example, a mobile terminal or
a fixed computing device) configured to employ an embodiment of the present
invention by further configuration of the processor (204) by instructions for
performing the algorithms and/or operations described herein. The processor (204)
may include, among other things, a clock, an arithmetic logic unit (ALU) and logic
gates configured to support operation of the processor (204).
[032] The processor (204) may be embodied in a number of different ways. For
example, the processor (204) may be embodied as one or more of various hardware
processing means such as a coprocessor, a microprocessor, a controller, a digital
signal processor (DSP), a processing element with or without an accompanying DSP,
or various other processing circuitry including integrated circuits such as, for
example, an ASIC (application specific integrated circuit), an FPGA (field
programmable gate array), a microcontroller unit (MCU), a hardware accelerator, a
special-purpose computer chip, or the like. As such, in some embodiments, the
processor (204) may include one or more processing cores configured to perform
independently. A multi-core processor may enable multiprocessing within a single
physical package. Additionally, or alternatively, the processor (204) may include one
or more processors configured in tandem via the bus to enable independent execution
of instructions, pipelining and/or multithreading. Additionally, or alternatively, the
processor (204) may include one or more processors capable of processing large
volumes of workloads and operations to provide support for big data analysis. In an
example embodiment, the processor (204) may be in communication with a memory
(202) via a bus for passing information among components of the system (110).
[033] The communication interface (206) may comprise input interface and output
interface for supporting communications to and from any component with which the
system (110) may communicate. The communication interface (206) may be any
17
means such as a device or circuitry embodied in either hardware or a combination of
hardware and software that is configured to receive and/or transmit data to/from a
communications device. In this regard, the communication interface (206) may
include, for example, an antenna (or multiple antennae) and supporting hardware
and/or software for enabling communications with a wireless communication
network. Additionally, or alternatively, the communication interface (206) may
include the circuitry for interacting with the antenna(s) to cause transmission of
signals via the antenna(s) or to handle receipt of signals received via the antenna(s).
In some environments, the communication interface (206) may alternatively or
additionally support wired communication. As such, for example, the communication
interface (206) may include a communication modem and/or other hardware and/or
software for supporting communication via cable, digital subscriber line (DSL),
universal serial bus (USB) or other mechanisms.
[034] The processor (204) is configured to execute the set of instructions in the
memory (202) for facilitating the recommendation of stickers. Via the communication
interface (206), the processor (204) receives the user state vectors (i.e. state vectors of
the users (102A-102C)) from the RL agents. The processor (204) performs a group
level encoding of the user state vectors using the contextual encoding module (208) to
determine a group level context and a conversational direction of chats (i.e., a chat
trajectory). The group level context and the chat trajectory are used to estimate a
conversational direction for future chat (for example a direction or context of a future
response or responses) to be sent to intended RL agents in the user equipment (104A104C). The processor (204) generates the expression suggestion vector based on the
group level context and the conversational direction for future chat using the RL
model. Further, the system (110) sends the expression suggestion vector to the RL
agents in the user equipment (104A-104C) via the communication interface (206).
Each RL agents of the user equipment (104A-104C) generates a recommendation of
stickers using the expression suggestion vector. Each of the RL agents appends the
18
expression suggestion vector to corresponding user state vectors to obtain a final state
vector for generating the recommendation of stickers.
[035] The RL agents receive rewards when the users (102A-102C) select stickers
from the recommendation of stickers. The rewards are stored in the sticker
recommendation module (212). The RL agents generate a recommendation of followup stickers for future responses in the group chat when the policy function and the
value function in the reinforcement learning module (210) are updated.
[036] FIG. 3 shows a method flow diagram (300) for generating a
recommendation of stickers in a group chat, in accordance with an embodiment of the
present invention. The method (300) includes steps involved in generating the
recommendation of stickers. The method (300) starts at step (302), and ends at step
(314).
[037] At step (304), user state vectors associated with a plurality of users states of
a plurality of users, such as the users (102A-102C) in a group chat, are received using
a federated learning model. The plurality of user states corresponds to a sequence of
stickers exchanged in the group chat. The federated learning model enables encoding
of the user states locally at the user equipment (104A-104C). In one example
embodiment, each user state of the users (102A-102C) is locally encoded into a user
state vector by corresponding RL agent in each of the user equipment (104A-104C).
The RL agents of the user equipment (104A-104C) send the user state vectors to the
system (110). In some embodiments, each of the user state vectors comprises
historical information related to at least one of an action, a state and a reward of
respective RL agent in each of the user equipment (104A-104C).
[038] At step (306), a group level context and a conversational direction for future
chat of the group chat is determined based on the user state vectors using a
reinforcement learning (RL) model. The group level encoding is performed by the
processor (204) using the contextual encoding module (208) of the system (110).
19
[039] Next, at step (308), an expression suggestion vector is generated based on
group level contexts and the conversational direction for future chat (i.e. future chat
trajectory) of the group chat. The chat trajectory provides future chat direction for
chats to be sent to any of the intended users (102A-102C). In some embodiments, the
system (110), via the reinforcement learning module (210), generates the expression
suggestion vector. The expression suggestion vector is generated, when a
corresponding policy function that determines behavior of the RL model, is updated.
[040] At step (310), the expression suggestion vector is sent to the respective RL
agent in each user equipment (104A, 104B and 104C) of each of the plurality of users
(102A, 102B and 102C). The RL agent in each user equipment (104A, 104B and
104C) generates a final state vector based on concatenation of the corresponding user
state vector with the expression suggestion vector. Further, the recommendation of
stickers for the plurality of users is generated based on the final state vector. The
stickers for the recommendations are referred from stickers stored in the sticker
recommendation module. The recommendation of stickers is displayed via each chat
application interface of the user equipment (104A-104C) (refer FIG. 4). When each
of the users (102A-102C) selects a sticker from the recommendation of stickers,
corresponding RL agent receives a reward. When the selected sticker is exchanged,
new user states of each of the users (102A-102C) is generated. Further, each of the
RL agents generates a recommendation of follow-up stickers for next response of the
users (102A-102C) in the group chat. The new user states that represent a new
sequence of recent exchanges of chat content (i.e. the stickers) are encoded by the RL
agents using a look-up table of a set of stickers in the sticker recommendation module
(212). The look-up table may correspond to a hash map that maps each sticker and a
word to an expression embedding space, using a similarity metric, e.g. a cosine
similarity metric. The set of stickers in the look-up table are assigned with weights by
the RL agents. Additionally, the RL agents generate next action of generating the
recommendation of follow-up stickers upon updating the policy function and the
20
value function. The policy function and the value function are updated after a predefined time interval, based on the chat trajectory from exchanges of stickers.
[041] The recommendation of stickers displayed to the users (102A-102C) in the
group chat are shown and described next with reference to FIG. 4.
[042] FIG. 4 illustrates an exemplary representation (400) of user-interfaces (UIs),
such as a UI (402A), a UI (402B) and a UI (402C) displaying recommendation of
stickers, such as a recommendation of stickers (404A), a recommendation of stickers
(404B) and a recommendation of stickers (404C) to the user (102A), the user (102B)
and the user (102C), respectively in a group chat, in accordance with an example
embodiment of the present invention. The UI (402A) corresponds to the user
equipment (104A), the UI (402B) corresponds to the user equipment (104B) and the
UI (402C) corresponds to the user equipment (104C). In the UI (402A), the
recommendation of stickers (404A) is displayed to the user (102A). The user (102A)
selects a sticker (406) from the recommendation of stickers (404A). The other users,
i.e. the user (102B) and the user (102C) receives the sticker (406) sent by the user
(102A). As shown in FIG. 4, the UI (402B) displays the sticker (406) and generates
the recommendation of stickers (404B) to the user (102B). The user (102B) selects a
sticker (408) from the recommendation of stickers (404B). The user (102C) receives
the sticker (406) and the sticker (408), as shown in the UI (402C). The UI (402C)
displays the recommendation of stickers (404C) to the user (102C). The user (102C)
selects a sticker (410) to respond to chats of the user (102A) and the user (102C).
[043] In this way, the described embodiments of the present disclosure may be
utilized for generating the recommendation of stickers that enable users in a group
chat to respond to chats in an efficient and feasible manner. The federated learning
model disclosed herein enable encoding process to be locally performed in
corresponding user equipment of the users, while preventing sharing of personal data
to a remote server. Moreover, the users are precluded from the need of taking time in
responding the chats as relevant stickers are generated. Thus, the users provide
21
responses without the need of searching for the relevant stickers. The
recommendation of stickers corresponds to stickers that are relevant to context of the
group chat. This also increases active participation of the users to engage in the group
chat, and maximize the minimum participation or the average participation of user in
the group chat.
[044] Many modifications and other embodiments of the inventions set forth
herein will come to mind of one skilled in the art to which these inventions pertain
having the benefit of the teachings presented in the foregoing descriptions and the
associated drawing. Therefore, it is to be understood that the inventions are not to be
limited to the specific embodiments disclosed and that modifications and other
embodiments are intended to be included within the scope of the present disclosure.
Moreover, although the foregoing descriptions and the associated drawing describe
example embodiments in the context of certain example combinations of elements
and/or functions, it should be appreciated that different combinations of elements
and/or functions may be provided by alternative embodiments without departing from
the scope of the present disclosure. In this regard, for example, different combinations
of elements and/or functions than those explicitly described above are also
contemplated as may be set forth in some of the present disclosure. Although specific
terms are employed herein, they are used in a generic and descriptive sense only and
not for purposes of limitation.

We Claim:

1. A method for recommending stickers in a group chat, the method comprising:
receiving (304), using a federated learning model, user state vectors
associated with a plurality of user states of a plurality of users
(102A, 102B and 102C) in a group chat, wherein the plurality of
user states correspond to a sequence of stickers exchanged in the
group chat;
determining (306) a group level context and a conversational direction for
future chat of the group chat based on the user state vectors using a
reinforcement learning (RL) model;
generating (308) an expression suggestion vector based on the group level
context and the conversational direction for future chat of the group
chat; and
sending (310) the expression suggestion vector to respective RL agent in
each user equipment (104A, 104B and 104C) of each of the
plurality of users (102A, 102B and 102C).
2. The method as claimed in claim 1, wherein each of the encoded user states are
locally encoded by the respective RL agents based on the federated learning
model.
3. The method as claimed in claim 1, wherein the RL agent in each user equipment
(104A, 104B and 104C) generates a final state vector based on concatenation of
the corresponding user state vector with the expression suggestion vector, and
wherein the recommendation of stickers for the plurality of users is generated
based on the final state vector.
4. The method as claimed in claim 2, wherein each of the user state vectors further
comprises historical information related to at least one of an action, a state or a
reward of the respective RL agent.
23
5. The method as claimed in claim 4, wherein the expression suggestion vector is
generated based on an updated policy function of the RL model,
wherein the policy function is updated based on the user state vectors, and
wherein the policy function provides values for actions of the RL model.
6. The method as claimed in claim 1, further comprising:
receiving a selection of at least one sticker from the recommendation of
stickers from the plurality of users;
receiving a reward based on the selection of the at least one sticker; and
updating a policy function and a value function of the respective RL
agents based on the reward,
wherein the policy function provides policies corresponding to actions
of the RL agents, and
wherein the value function provides value of each action of the RL
agents.
7. A system (110) for recommending stickers in a group chat, the system
comprising:
a memory (202) configured to store a set of instructions to be executed by
a processor (204), the processor (204) configured to:
receive, via a communication interface (206), user state vectors
associated with a plurality of users states of a plurality of users
(102A, 102B and 102C) in a group chat using a federated learning
model, wherein the plurality of user states correspond to a sequence
of recent past exchange of a chat content comprising sticker;
24
determine a group level context and a conversational direction
for future chat of the group chat based on the user state vectors
using a reinforcement learning (RL) model;
generate an expression suggestion vector based on the group
level context and the conversational direction for future chat of the
group chat; and
send, via the communication interface (206), the expression
suggestion vector to respective RL agent in each user equipment
(104A, 104B and 104C) of each of the plurality of users (102A,
102B and 102C).
8. The system (110) as claimed in claim 7, wherein each of the encoded user states
are locally encoded by the respective RL agents based on the federated learning
model.
9. The system (110) as claimed in claim 7, wherein the RL agent in each user
equipment (104A, 104B and 104C) generates a final state vector based on
concatenation of the corresponding user state vector with the expression
suggestion vector, and wherein the recommendation of stickers for the plurality
of users is generated based on the final state vector.
10. The system (110) as claimed in claim 8, wherein each of the user state vectors
further comprises historical information related to at least one of an action, a
state or a reward of the respective RL agent