Claims:CLAIMS
We claim:

1. A computer-implemented method, comprising:
accessing, by a server system, payment transaction data associated with a cardholder for a particular time segment from a transaction database;
generating, by the server system, a plurality of input vectors associated with a plurality of transaction features based, at least in part, on the payment transaction data;
determining, by the server system, a probability vector by executing a set of consumer control models with the plurality of input vectors as inputs, each consumer control model associated with a transaction feature of the plurality of transaction features, and the probability vector having a length equal to a number of the set of consumer control models;
determining, by the server system, one or more consumer controls for the cardholder based, at least in part, on a deep reinforcement learning (DRL) model and the probability vector, the one or more consumer controls representing cardholder actions for setting up threshold limits for a set of transaction variables via user selectable interface elements provided on a user interface of an application accessible on a user device of the cardholder; and
transmitting, by the server system, a recommendation message to the cardholder, the recommendation message comprising the one or more consumer controls to be set by the cardholder on the application.

2. The computer-implemented method as claimed in claim 1, further comprising:
aggregating, by the server system, the payment transaction data associated with the cardholder based, at least in part, on the plurality of transaction features, the plurality of transaction features comprising transaction velocity features for automatic teller machine (ATM), Point of Sale (POS) and electronic commerce based payment transactions for the cardholder, spending patterns at merchant industries, and cross-border transaction patterns.

3. The computer-implemented method as claimed in claim 1, wherein each consumer control model of the set of consumer control models is a classification model.

4. The computer-implemented method as claimed in claim 1, further comprising:
determining, by the server system, model recommendation strategy based, at least in part, on the probability vector, the model recommendation strategy comprising information of at least one consumer control for recommending the cardholder.

5. The computer-implemented method as claimed in claim 1, wherein the DRL model comprises a deep neural network trained based, at least in part, on historical payment transaction data of a plurality of cardholders.

6. The computer-implemented method as claimed in claim 5, wherein determining the one or more consumer controls for the cardholder comprising:
defining a state of the DRL model based, at least in part, on the model recommendation strategy, recommendation adoption rate for the cardholder, and consumer controls that are already set by the cardholder;
setting selection of a candidate number of consumer controls for recommendation as actions of the DRL model;
calculating Q-values corresponding to state-action pairs formed by state and actions based, at least in part, on a reward function; and
determining the one or more consumer controls to be transmitted as the recommendation message to the cardholder based, at least in part, on the calculated Q-values satisfying a preset condition.

7. The computer-implemented method as claimed in claim 6, wherein the reward function is defined based on one or more factors: probability scores associated with the plurality of transaction features of the cardholder for a particular time, a probability of occurrence of fraudulent transaction for the cardholder in the particular time, and a recommendation adoption rate associated with the cardholder.

8. The computer-implemented method as claimed in claim 7, wherein the DRL model is trained by:
accessing the historical payment transaction data associated with the plurality of cardholders over a period of time,
defining an initial state of the DRL model, the initial state comprising transaction features associated with each cardholder, and consumer controls that are already set by each cardholder,
defining an action space of the DRL model, the action space comprising a plurality of consumer controls,
generating reward values corresponding to simulated actions based, at least in part, on a reward function; and
identifying optimal actions to train the DRL model that leads to a maximum reward value.

9. A server system configured to perform the computer-implemented method as claimed in any of the claims 1-8.
, Description:
FORM 2
THE PATENTS ACT 1970
(39 of 1970)
&
The Patent Rules 2003
COMPLETE SPECIFICATION
(refer section 10 & rule 13)

TITLE OF THE INVENTION:
ARTIFICIAL INTELLIGENCE BASED METHODS AND SYSTEMS FOR RECOMMENDING CONSUMER CONTROLS TO A CARDHOLDER

APPLICANT(S):

Name:

Nationality:

Address:

MASTERCARD INTERNATIONAL INCORPORATED

United States of America

2000 Purchase Street, Purchase, NY 10577, United States of America

PREAMBLE TO THE DESCRIPTION

The following specification particularly describes the invention and the manner in which it is to be performed.

DESCRIPTION
(See next page)

 
ARTIFICIAL INTELLIGENCE BASED METHODS AND SYSTEMS FOR RECOMMENDING CONSUMER CONTROLS TO A CARDHOLDER

TECHNICAL FIELD
The present disclosure relates to artificial intelligence processing systems and, more particularly to, electronic methods and complex processing systems for recommending consumer controls to a cardholder.

BACKGROUND
In recent times, payment cards such as credit cards, debit cards, and ATM cards are widely used by people. These payment cards may be used to perform payment transactions not just at an ATM terminal, but several channels such as transactions at POS machines, payments at e-commerce websites, and so on. Because of the increasing channels of usage of the payment cards, there has been an increase in fraudulent transactions associated with these payment cards that happen every single day. To protect cardholders from such fraudulent transactions, the payment cards are developed with various consumer controls that may be set by a cardholder using a computing device. The various consumer controls may include a control for transactions occurring above a certain threshold amount, a control for transactions occurring in a specific geo-location, a control for transactions occurring with a certain merchant category code, daily or hourly budget controls for transactions, and so on. These consumer controls available to the cardholder may avoid a lot of fraudulent transactions by sending an alert/decline transaction notification to the cardholder if any of the payment transactions occurring through an account of the cardholder breach the consumer controls.
However, the cardholders are not actively using these consumer controls. One reason might be that the cardholders are not completely aware of which consumer controls to be set for avoiding fraudulent transactions. Another reason might be that the cardholder is unaware of such consumer controls available for their account protection. As a result, non-usage of the available consumer controls options leads to a lot of fraudulent transactions.
Thus, there exists a need for a technical solution for recommending one or more consumer controls to a cardholder.

SUMMARY
Various embodiments of the present disclosure provide systems, methods and electronic devices for recommending consumer controls to a cardholder.
In an embodiment, a computer-implemented method is disclosed. The method includes accessing, by a server system, payment transaction data associated with a cardholder for a particular time segment from a transaction database. The method includes generating, by the server system, a plurality of input vectors associated with a plurality of transaction features based, at least in part, on the payment transaction data. The method includes determining, by the server system, a probability vector by executing a set of consumer control models with the plurality of input vectors as inputs. Each consumer control model associated with a transaction feature of the plurality of transaction features. The probability vector has a length equal to a number of the set of consumer control models. The method includes determining, by the server system, one or more consumer controls for the cardholder based, at least in part, on a deep reinforcement learning (DRL) model and the probability vector. The one or more consumer controls represent cardholder actions for setting up threshold limits for a set of transaction variables via user selectable interface elements provided on a user interface of an application accessible on a user device of the cardholder. The method further includes transmitting, by the server system, a recommendation message to the cardholder, the recommendation message including the one or more consumer controls to be set by the cardholder on the application.

BRIEF DESCRIPTION OF THE FIGURES
For a more complete understanding of example embodiments of the present technology, reference is now made to the following descriptions taken in connection with the accompanying drawings in which:
FIG. 1 illustrates an exemplary representation of an environment, related to at least some example embodiments of the present disclosure;
FIG. 2 is a simplified block diagram of a server system, in accordance with an example embodiment of the present disclosure;
FIG. 3 is an example representation of communication data flow in various modules of the server system for generating consumer control recommendations for a cardholder, in accordance with an embodiment of the present disclosure;
FIG. 4A is a block diagram representation of a deep reinforcement learning (DRL) model, in accordance with an embodiment of the present disclosure;
FIG. 4B is a block diagram representation of a deep neural network of the DRL model, in accordance with an embodiment of the present disclosure;
FIG. 5 shows exemplary user interface (UI) of an application installed on a user device for setting up one or more consumer controls, in accordance with an embodiment of the present disclosure;
FIG. 6 is a flow chart of a method for training the DRL model, in accordance with an embodiment of the present disclosure;
FIG. 7 is a flow chart of a method for execution of the DRL model, in accordance with an embodiment of the present disclosure;
FIG. 8 is a process flow chart of a computer-implemented method for recommending the one or more consumer controls to the cardholder, in accordance with embodiment of the present disclosure;
FIG. 9 is a simplified block diagram of an issuer server used, in accordance with an embodiment of the present disclosure; and
FIG. 10 shows a simplified block diagram of a device for example, the user device capable of implementing the various embodiments of the present disclosure.
The drawings referred to in this description are not to be understood as being drawn to scale except if specifically noted, and such drawings are only exemplary in nature.

DETAILED DESCRIPTION
In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. It will be apparent, however, to one skilled in the art that the present disclosure can be practiced without these specific details.
Reference in this specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. The appearance of the phrase “in an embodiment” in various places in the specification is not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Moreover, various features are described which may be exhibited by some embodiments and not by others. Similarly, various requirements are described which may be requirements for some embodiments but not for other embodiments.
Moreover, although the following description contains many specifics for the purposes of illustration, anyone skilled in the art will appreciate that many variations and/or alterations to said details are within the scope of the present disclosure. Similarly, although many of the features of the present disclosure are described in terms of each other, or in conjunction with each other, one skilled in the art will appreciate that many of these features can be provided independently of other features. Accordingly, this description of the present disclosure is set forth without any loss of generality to, and without imposing limitations upon, the present disclosure.
The term "payment account" used throughout the description refers to a financial account that is used to fund the financial transaction (interchangeably referred to as "payment transaction"). Examples of the payment account include, but are not limited to a savings account, a credit account, an e-wallet account, a checking account and a virtual payment account. The payment account may be associated with an entity such as an individual person, a family, a commercial entity, a company, a corporation, a governmental entity, a non-profit organization and the like. In some scenarios, a payment account may be a virtual or temporary payment account that can be mapped or linked to a primary payment account, such as those accounts managed by payment wallet service providers.
The term "payment card", used throughout the description, refers to a physical or virtual card linked with a financial or payment account that may be presented to a merchant or any such facility in order to fund a financial transaction via the associated payment account. Examples of the payment card include, but are not limited to, debit cards, credit cards, prepaid cards, virtual payment numbers, virtual card numbers, forex cards, charge cards, e-wallet cards, and stored-value cards. A payment card may be a physical card that may be presented to the merchant for funding the payment. Alternatively or additionally, the payment card may be embodied in form of data stored in a user device, where the data is associated with payment account such that the data can be used to process the financial transaction between the payment account and a merchant's financial account.
The term “payment network”, used throughout the description, refers to a network or collection of systems used for transfer of funds through use of cash-substitutes. Payment networks may use a variety of different protocols and procedures in order to process the transfer of money for various types of transactions. Transactions that may be performed via a payment network may include product or service purchases, credit purchases, debit transactions, fund transfers, account withdrawals, etc. Payment networks may be operated to perform transactions via cash-substitutes, which may include payment cards, letters of credit, checks, financial accounts, etc. One example of a payment network includes those operated by Mastercard.
The term "consumer controls", used throughout the description, refers to control settings provided to the cardholders to set one or more user authorization controls over future payment transactions. The control settings may be displayed on an application accessible on user devices of the cardholders. The one or more user authorization controls refers to setting threshold limits for one or more transaction variables of the future payment transactions. For example, setting an alert or transaction decline for payment transactions greater than $10k at a time by a cardholder can be considered as a consumer control action.
OVERVIEW
Various example embodiments of the present disclosure provide systems and methods for recommending one or more consumer controls to a cardholder. The consumer controls represent consumer actions to be set by the cardholder. In general, consumer controls are personalized transaction controls or alerts that provide greater control to cardholders on how and when their account or payment card is being used. In an example embodiment, the one or more consumer controls are real-time alerts or personalized notifications received on a user device as payment transactions occur from account of the cardholder. In another example embodiment, the consumer controls are real-time alerts delivered on the user device for payment transactions occurring over a certain amount. In yet another example embodiment, the one or more consumer controls are set by the cardholder to turn payment cards on/off easily.
The present disclosure describes a server system that is configured to access payment transaction data associated with a cardholder from a transaction database. The server system includes at least a processor and a memory. The server system is configured to learn behavior of the cardholder from the accessed payment transaction data associated with the cardholder. The server system is further configured to recommend the one or more consumer controls to the cardholder. In one example, if the server system predicts that the cardholder is not going to perform any cross-border transaction in the next quarter, then the server system recommends the cardholder to activate cross-border decline consumer control.
In one example, the server system is configured to execute a set of consumer control models. The set of consumer control models access a plurality of transaction features associated with the cardholder as an input. The plurality of transaction features is extracted from the payment transaction data associated with the cardholder over a period of time (e.g., 1 year, 2 years, 5 years etc.) and the like. In one example embodiment, each of the plurality of transaction features indicate spending pattern of the cardholder.
The server system is configured to aggregate the payment transaction data associated with the cardholder based, at least in part, on the plurality of transaction features. The plurality of transaction features includes transaction velocity features for automatic teller machine (ATM), Point of Sale (POS) and electronic commerce based payment transactions for the cardholder, spending patterns at merchant industries, and cross-border transaction patterns.
The server system is further configured to determine a probability vector by executing a set of consumer control models. The set of consumer control models generate the probability vector. Each of the set of consumer control models is a classifier model. In general, classifier model predicts categorical labels (classes) to classify training data into various classes/labels. In one example, each of the set of consumer control models includes decision tree, random forest, multilayer perceptron, logistic regression, and the like.
The server system is configured to train using a deep reinforcement learning (DRL) model for generating optimal consumer control recommendations. In general, reinforcement learning is a subset of machine learning in which an intelligent agent takes decisions in an environment in order to maximize its rewards. In an embodiment, the DRL model implements a deep Q learning neural network for training. In general, deep Q learning neural network utilizes a neural network to approximate Q-value function. In addition, Q-learning is a model-free reinforcement learning algorithm that learns value of an action in a particular state. The DRL model includes a deep neural network trained based, at least in part, on historical payment transaction data of a plurality of cardholders over a period of time.
The server system is configured to define an initial state for each of the plurality of cardholders. The initial state includes transaction features associated with each cardholder, and consumer controls that are already set by each cardholder. The server system is also configured to define an action space of the DRL model, where the action space includes the plurality of consumer controls (such as transaction amount, channel of transaction, cross border transaction, static geo-location of transaction, merchant category code, budget of transaction, travel-based transactions, all transactions, and the like). Thereafter, the server system is configured to generate Q-values with a maximum reward value corresponding to state-action pairs formed by the state and actions using the DRL model. The server system is configured to select an action based on the predicted Q-value with the maximum reward value.
In particular, the server system is configured to generate the reward value corresponding to the simulated action based, at least in part, on a reward function. The reward function is defined based on one or more factors: probability scores associated with the plurality of transaction features of the cardholder for a particular time, a probability of occurrence of fraudulent transaction for the cardholder in the particular time, and recommendation adoption rate associated with the cardholder. The server system is further configured to identify optimal actions to train the DRL model that leads to a maximum reward value.
During execution phase, the server system is configured to receive the probability vector as input to the deep reinforcement learning (DRL) model. The DRL model is an integration of a deep learning neural network and Q-learning. In addition, length of the probability vector is equal to a number of the set of consumer control models. Based on the probability vector and the execution of the DRL model, the server system is configured to recommend the one or more consumer controls to the cardholder. The one or more consumer controls include, but are not limited to, transaction amount, channel of transaction, cross border transaction, static geo-location of transaction, merchant category code, budget of transaction, travel-based transactions and all transactions.
The server system is configured to define a state of the DRL model based, at least in part, on the model recommendation strategy, recommendation adoption rate for the cardholder, and consumer controls that are already set by the cardholder. The server system is configured to set selection of a candidate number of consumer controls for recommendation as actions of the DRL model. The server system is further configured to calculate Q-values corresponding to state-action pairs formed by state and actions based, at least in part, on a reward function. Moreover, the server system is configured to determine the one or more consumer controls to be transmitted as the recommendation message to the cardholder based, at least in part, on the calculated Q-values satisfying a predefined condition.
The server system determines the model recommendation strategy based, at least in part, on the probability vector, the model recommendation strategy including information of at least one consumer control for recommending the cardholder. The server system is further configured to transmit a recommendation message to the cardholder. The recommendation message includes the one or more consumer controls to be set by the cardholder on the application.
Various embodiments of the present disclosure offer multiple advantages and technical effects. For instance, the present disclosure provides an automated system for recommending the one or more consumer controls to the cardholder. In addition, the present disclosure helps to achieve reduction in number of fraudulent transactions. The present disclosure allows an issuer server to decide whether to send the recommendations to cardholders who currently use the consumer controls or to the cardholders that are vulnerable to fraudulent transactions in the future. Additionally, the present disclosure improves accuracy on the recommendation of the one or more consumer controls by applying a deep Q learning based neural network architecture over the probability vector. Furthermore, the present disclosure provides a significantly more robust solutions because of handling simultaneous/concurrent processor execution (such as applying multiple consumer control models over the same input transaction features, simultaneously).
Various example embodiments of the present disclosure are described hereinafter with reference to FIGS. 1 to 10.
FIG. 1 illustrates an exemplary representation of an environment 100 related to at least some example embodiments of the present disclosure. Although the environment 100 is presented in one arrangement, other embodiments may include the parts of the environment 100 (or other parts) arranged otherwise depending on, for example, recommending one or more consumer controls to a cardholder. The environment 100 generally includes a plurality of entities, for example, a user device 104 associated with a cardholder 102, a server system 106, an issuer server 108, a payment network 116 including a payment server 114, and a transaction database 112, each coupled to, and in communication with (and/or with access to) a network 110. The network 110 may include, without limitation, a light fidelity (Li-Fi) network, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a satellite network, the Internet, a fiber optic network, a coaxial cable network, an infrared (IR) network, a radio frequency (RF) network, a virtual network, and/or another suitable public and/or private network capable of supporting communication among the entities illustrated in FIG. 1, or any combination thereof. Various entities in the environment 100 may connect to the network 110 in accordance with various wired and wireless communication protocols, such as Transmission Control Protocol and Internet Protocol (TCP/IP), User Datagram Protocol (UDP), 2nd Generation (2G), 3rd Generation (3G), 4th Generation (4G), 5th Generation (5G) communication protocols, Long Term Evolution (LTE) communication protocols, future generation communication protocols, or any combination thereof. For example, the network 110 may include multiple different networks, such as a private network made accessible by the server system 106, separately, and a public network (e.g., the Internet etc.).
The cardholder such as the cardholder 102 may be any individual, representative of a corporate entity, non-profit organization, or any other person. The cardholder 102 may have a payment account issued by corresponding issuing banks (associated with the issuer server 108) and may be provided a payment card with financial or other account information encoded onto the payment card such that the cardholder 102 may use the payment card to initiate and complete a transaction using a bank account at the issuing bank. Examples of the payment card may include, but are not limited to, a smartcard, a debit card, a credit card, etc.
In one embodiment, the issuer server 108 is a financial institution that manages accounts of multiple cardholders (e.g., cardholder 102). Account details of the accounts established with the issuer bank are stored in cardholder profiles of the cardholders in a memory of the issuer server 108 or on a cloud server associated with the issuer server 108. The terms “issuer server”, “issuer”, or “issuing bank” will be used interchangeably herein.
In one embodiment, the user device 104 associated with the cardholder 102 may be a smartphone, a tablet, a laptop, a computer system or any computing device. In an example embodiment, the user device 104 may include a portable device such as laptop, smartwatch, personal digital assistant (PDA), smartphone, and the like. In another example embodiment, the user device 104 may include a fixed device such as desktop, workstation, and the like.
The user device 104 may be installed with an application 118 for setting up various authorization controls for payment transactions initiated by the cardholder 102. The application 118 may include any mobile program, software, or other suitable executable code suitable to conduct a payment transaction. In some embodiments, the application 118 may be an issuer-specific banking application. In one embodiment, the application 118 may be a general-purpose application, such as a web browser.
In one embodiment, the server system 106 is configured to perform one or more of the operations described herein. In one example, the server system 106 coupled with the transaction database 112 is embodied in the payment network 116. In another example, the server system 106 is the issuer server 108. In general, the server system 106 is configured to provide recommendations for setting up various authorization controls to the cardholder 102 based on deep reinforcement learning model and consumer control models. In particular, the server system 106 is configured to learn user transaction behavior data and recommend the cardholder 102 to set consumer controls or user authorization controls using the application 118 on the user device 104. The server system 106 is a separate part of the environment 100, and may operate apart from (but still in communication with, for example, via the network 110) the issuer server 108, the payment server 114, and any third party external servers (to access data to perform the various operations described herein). However, in other embodiments, the server system 106 may actually be incorporated, in whole or in part, into one or more parts of the environment 100, for example, the payment server 114, or the issuer server 108. In addition, the server system 106 should be understood to be embodied in at least one computing device in communication with the network 110, which may be specifically configured, via executable instructions, to perform steps as described herein, and/or embodied in at least one non-transitory computer-readable media.
In one embodiment, the server system 106 is configured to access datasets of historical payment transaction data from multiple data sources including, for example, the transaction database 112, the issuer server 108 (for example, application server, web server, media server, etc.), and other suitable data sources not shown in FIG. 1. Alternatively, the server system 106 can be directly coupled to data sources, as opposed to, via the network 110.
In an example, the transaction database 112 may implement relational and non-relational databases, big data stations, file systems, and other suitable applications storing datasets with historical payment transaction data.
In an embodiment, the server system 106 is in communication with the transaction database 112. The transaction database 112 may store trained DRL models and algorithms required for the server system 106 to perform one or more of the operations described herein, for example, to recommend the one or more consumer controls to the cardholder 102. The transaction database 112 may be incorporated in the server system 106 or may be an individual entity connected to the server system 106 or may be a database stored in a cloud storage.
The payment network 116 may be used by the payment cards issuing authorities as a payment interchange network. Examples of payment interchange network include, but not limited to, Mastercard® payment system interchange network. The Mastercard® payment system interchange network is a proprietary communications standard promulgated by Mastercard International Incorporated® for the exchange of financial transaction data between financial institutions that are members of Mastercard International Incorporated®. (Mastercard is a registered trademark of Mastercard International Incorporated located in Purchase, N.Y.).
The number and arrangement of systems, devices, and/or networks shown in FIG. 1 are provided as an example. There may be additional systems, devices, and/or networks; fewer systems, devices, and/or networks; different systems, devices, and/or networks; and/or differently arranged systems, devices, and/or networks than those shown in FIG. 1. Furthermore, two or more systems or devices shown in FIG. 1 may be implemented within a single system or device, or a single system or device shown in FIG. 1 may be implemented as multiple, distributed systems or devices. Additionally, or alternatively, a set of systems (e.g., one or more systems) or a set of devices (e.g., one or more devices) of the environment 100 may perform one or more functions described as being performed by another set of systems or another set of devices of the environment 100.
FIG. 2 is a simplified block diagram of a server system 200, in accordance with an example embodiment of the present disclosure. The server system 200 is similar to the server system 106 or the issuer server 108 of FIG. 1. In some embodiments, the server system 200 is embodied as a cloud-based and/or SaaS-based (software as a service) architecture. In some embodiments, the server system 200 is embodied in the payment network 116.
The server system 200 includes a computer system 202 and a database 204. The computer system 202 includes at least one processor 206 for executing instructions, a memory 208, a communication interface 210, and a user interface 216 that communicate with each other via a bus 212.
In some embodiments, the database 204 is integrated within computer system 202. For example, the computer system 202 may include one or more hard disk drives as the database 204. A storage interface 214 is any component capable of providing the processor 206 with access to the database 204. The storage interface 214 may include, for example, an Advanced Technology Attachment (ATA) adapter, a Serial ATA (SATA) adapter, a Small Computer System Interface (SCSI) adapter, a RAID controller, a SAN adapter, a network adapter, and/or any component providing the processor 206 with access to the database 204. In one embodiment, the database 204 is configured to store a set of consumer control models 228 and a deep reinforcement learning (DRL) model 230.
Examples of the processor 206 include, but are not limited to, an application-specific integrated circuit (ASIC) processor, a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a field-programmable gate array (FPGA), graphical processing units (GPUs), and the like. The memory 208 includes suitable logic, circuitry, and/or interfaces to store a set of computer-readable instructions for performing operations. Examples of the memory 208 include a random-access memory (RAM), a read-only memory (ROM), a removable storage drive, a hard disk drive (HDD), and the like. It will be apparent to a person skilled in the art that the scope of the disclosure is not limited to realizing the memory 208 in the server system 200, as described herein. In another embodiment, the memory 208 may be realized in the form of a database server or a cloud storage working in conjunction with the server system 200, without departing from the scope of the present disclosure. The processor 206 is operatively coupled to the communication interface 210 such that the processor 206 is capable of communicating with a remote device 218 such as, the issuer server 108, or communicated with any entity connected to the network 110 (as shown in FIG. 1).
It is noted that the server system 200 as illustrated and hereinafter described is merely illustrative of an apparatus that could benefit from embodiments of the present disclosure and, therefore, should not be taken to limit the scope of the present disclosure. It is noted that the server system 200 may include fewer or more components than those depicted in FIG. 2.
In one embodiment, the processor 206 includes a data pre-processing engine 220, a consumer control engine 222, a DRL agent 224, and a recommendation engine 226. It should be noted that components, described herein, can be configured in a variety of ways, including electronic circuitries, digital arithmetic and logic blocks, and memory systems in combination with software, firmware, and embedded technologies.
The data pre-processing engine 220 includes suitable logic and/or interfaces for accessing historical payment transaction data associated with a plurality of cardholders associated with at least one issuer (e.g., “issuer server 108) from the transaction database 112 over a period of time (e.g., 3 years, 2 years, and the like). The data pre-processing engine 220 performs featurization process over the historical payment transaction data for extracting a plurality of transaction features associated with each cardholder (for example, cardholder 102). The plurality of transaction features includes transaction velocity features for automatic teller machine (ATM), Point of Sale (POS) and electronic commerce based payment transactions for the cardholder 102, spending patterns at merchant industries, and cross-border transaction patterns, location pattern data related to the payment transactions, etc. The plurality of transaction features can be defined using one or more transaction variables such as transaction velocity features (for example, transaction activeness, transaction frequency, etc.), total transaction counts and spending amounts in different payment transaction types (e.g., ATM, POS, e-commerce, etc.), total transactions counts and spending amounts (i.e., spending patterns) in different merchant industries with a particular time period, total transaction counts and spending amounts associated with payment transaction types (such as, cross-border transactions, location-specific transactions). The plurality of transaction features may also include card product types (such as, Standard, Platinum, etc.) and card industry transaction map (i.e., normalized histogram for all the industries, where the payment card is used).
In one embodiment, the data pre-processing engine 220 is configured to generate a plurality of input vectors associated with the plurality of transaction features for each cardholder (for example, cardholder 102). The plurality of input vectors is generated by aggregating the historical payment transaction data of each cardholder on a timely basis. In one embodiment, the frequency of aggregation can be defined as hourly, weekly, daily, monthly, quarterly, based on spend frequency associated with the cardholder 102 at various merchants.
In an example, the historical payment transaction data represents spend transactions performed by the plurality of cardholders across various merchant categories such as, grocery, airlines, and the like. In another example, the historical payment transaction data includes spend transactions performed by the cardholders across various merchant industries such as, retail clothing, hotel industry, and the like. In yet another example, the historical payment transaction data includes spend transactions performed by the cardholders across various locations where the spend transactions occurred, and payment transaction types such as, contactless, card-present, and the like. Further, the historical payment transaction data may denote payment transactions performed by the plurality of cardholders via different transaction channels in the past over the period of time. Each transaction record is augmented with any number of predictive variables and their values. Any previous period of time may be used to gather the historical payment transaction data from which the plurality of input vectors is extracted. In one embodiment, the period of time extends from the current time and date when the plurality of input vectors is extracted (or a practical equivalent such as the morning of the current day or the day before) to a previous date such as three to six months ago as mentioned. By including data up until the current date, a set of the consumer control models is trained, validated and tested using the most current transaction data.
The consumer control engine 222 includes suitable logic and/or interfaces for generating a probability vector based on the plurality of input vectors associated with the cardholder 102 and a set of consumer control models 228. The set of consumer control models 228 may include, but is not limited to, cross-border (CB) model, transaction amount model, Automatic Teller Machine (ATM) model, e-commerce model, Point of Sale (POS) model and merchant category code (MCC) model. Each consumer control model implements a classification model and corresponds to a particular transaction feature. In one example, the classification model is a CatBoost classifier model. The set of consumer control models 228 is trained based on the historical payment transaction data associated with the plurality of cardholders. In one embodiment, the set of consumer control models 228 may be defined by the issuer server 108 associated with the cardholder 102. Thus, the set of consumer control models 228 may vary based on transaction features available for cardholders of the issuer server 108.
In one example, the CB model is configured to determine a probability score of a future payment transaction being a cross-border payment transaction. The CB model is configured to take an input vector corresponding to a transaction feature indicating the transaction channel and generate the probability score of occurrences of cross-border payment transaction in the future (example, next quarter).
The transaction amount model is configured to determine a probability score based on amount of payment transaction exceeding a threshold amount of transaction associated with the payment card set by the cardholder 102. The ATM model may generate a probability score determining whether the payment card associated with the cardholder 102 is going to perform the ATM transaction in the future or not.
The e-commerce model may generate a probability score determining whether the payment card associated with the cardholder 102 is going to perform transaction on e-commerce websites in the future or not. The POS model may generate a probability score determining whether the payment card associated with the cardholder 102 is going to perform transaction on POS machine in the future or not. The MCC model may generate a probability score determining whether the payment card associated with the cardholder 102 is going to perform transaction for a certain merchant category code in the future or not.
In one embodiment, the consumer control engine 222 is configured to combine outputs (i.e., probability scores) of the set of consumer control models 228 to generate a single dimension probability vector. The length of probability vector is equal to a number of the set of consumer control models 228. In one embodiment, the processor 206 is configured to determine which consumer controls need to be recommended to the cardholder 102 for setting those consumer controls based on the probability vector. In other words, the processor 206 is configured to determine a model recommendation strategy for the cardholder 102 based on the probability vector. For example, if the probability score indicating a payment transaction with a transaction amount greater than $5k is greater than a threshold value, then, the processor 206 is configured to include transaction amount related consumer control in the model recommendation strategy for the cardholder 102.
The DRL agent 224 includes suitable logic and/or interfaces for predicting whether the cardholder 102 should be recommended a consumer control or not. The DRL agent 224 implements the deep reinforcement learning (DRL) model 230 with a deep neural network. The DRL agent 224 is configured to determine one or more consumer controls to be recommended to the cardholder 102 to set the one or more consumer controls on the application 118. The one or more consumer controls are determined based, at least in part, on the probability vector generated by the consumer control engine 222 and already set consumer controls by the cardholder 102. The main objective of the DRL model 230 is to recommend cardholders to set a plurality of consumer controls on the application 118 to reduce fraudulent transactions such that spam recommendations may not increase. In one example, it is assumed that a cardholder 'A' does not have any plans for a foreign trip in the next quarter due to global pandemic. Hence, the cardholder 'A' cannot perform any cross-border payment transaction within the next quarter. In such case, the DRL agent 224 may recommend a cross-border alert/decline consumer control for the cardholder 'A' to set on the application 118 so that any future cross-border payment transaction within the next quarter may get declined due to control setting.
In order to express the use of reinforcement learning in determining the one or more consumer controls to be recommended to the cardholder 102 for setting up on the applications, the present disclosure explains theoretical models of deep Q learning, the Markov Decision Process (MDP) with reference to FIGS. 4A and 4B in more detail. It would be apparent to those skilled in the art that several of deep reinforcement learning models may be applied to accomplish the spirit of the present disclosure. In general, the goal of the DRL model 230 is to learn a policy (control strategy) that maximizes the expected return (cumulative, discounted reward).
In one embodiment, the DRL agent 224 is configured to train the deep neural network in an off-line manner using the DRL model 230. In one example embodiment, the DRL agent 224 is trained using historical payment transaction data of cardholders who have already set some consumer controls in their user devices. In another example, the DRL agent 224 may also be trained with historical payment transaction data of cardholders who faced fraud transactions in the past.
The state of the DRL model 230 is defined based on the plurality of transaction features (e.g., transaction amount, transaction count, etc.) of a particular cardholder (e.g., “cardholder 102”), user features (e.g., user activeness parameter indicating recommendation adoption rate), consumer controls that are already set by the particular cardholder, and the model recommendation strategy for the particular cardholder determined by the consumer control models 228. The consumer controls already set by the particular cardholder can be represented using one dimensional vector, where each index represents control setting of a particular consumer control set by the cardholder 102.
The deep neural network of the DRL agent 224 is trained by repeating episodes of start-to-finish simulation. The DRL agent 224 simulates sample actions based on adoption rate of recommendations by the cardholder 102 or a probability of fraudulent transactions in the future. In general, the DRL agent 224 is configured to observe a state S(t) at time step t and transition to a new state S(t+1) by taking an action A(t) in the state S(t). The DRL agent 224 is configured to generate reward values corresponding to the sampled actions based, at least in part, on a reward function. The DRL agent 224 is further configured to identify optimal actions to train the deep reinforcement learning (DRL) model 230 that leads to a maximum reward value.
The DRL agent 224 determines the one or more consumer controls for the cardholder 102 based, at least in part, on the DRL model 230 and the probability vector. The one or more consumer controls represent consumer actions for setting up threshold limits for a set of transaction variables via user selectable interface elements provided on a user interface of the application 118 accessible on the user device 104 of the cardholder 102.
In one embodiment, the set of transaction variables may include variables to store information in the form of number or text. In an example, the set of transaction variables may include a variable to store transaction amount that is set by the cardholder 102 in the transaction amount control. In another example, the set of transaction variables may include variables to store various merchant category codes that are set by the cardholder 102 in the merchant-specific consumer control.
In an example, based on the probability vector and the execution of the DRL agent 224, the server system 200 recommends the cardholder 102 to turn off switch for cross border transactions through the application 118 installed in the user device 104 of the cardholder 102. In another example, based on the probability vector and the execution of the DRL agent 224, the server system 200 recommends the cardholder 102 to turn off switch for ATM transactions through the application 118 installed in the user device 104 of the cardholder 102. In yet another example, based on the probability vector and the execution of the DRL agent 224, the server system 200 recommends the cardholder 102 to turn off switch for POS machine-based transactions through the application 118 installed in the user device 104 of the cardholder 102.
The recommendation engine 226 includes suitable logic and/or interfaces for transmitting a notification including the one or more consumer controls to be set by the cardholder 102 on the application 118, to the cardholder 102. The recommendation engine 226 transmits the recommendation in the form of notifications (push notifications, application notifications, web notifications, and the like) in real-time to the user device 104 of the cardholder 102. The cardholder 102 may further activate/deactivate the recommended one or more consumer controls via the application 118 installed in the user device 104 of the cardholder 102. The server system 200 may further re-train the DRL agent 224 based on whether the cardholder 102 accepts the recommendation or not.
In an example, based on the historical payment transaction data of a cardholder A, the server system 200 determines that the cardholder A has never performed any e-commerce transaction since issuance of the payment card to the cardholder A. As a result, the server system 200 recommends the cardholder A to activate the e-commerce transaction control via the application 118 installed in the user device 104 of the cardholder A. In another example, based on the historical payment transaction data of a cardholder B, the server system 200 determines that the cardholder B has never performed any transaction on a POS device since last one year. As a result, the server system 200 recommends the cardholder B to activate POS control via the application 118 installed in the user device 104 of the cardholder B for the next one year.
FIG. 3 is an example representation 300 of communication data flow in various modules of the server system 200 for generating consumer control recommendations for a cardholder, in accordance with an embodiment of the present disclosure.
At first, the data pre-processing engine 220 is configured to receive payment transaction data associated with a cardholder 102 (e.g., Cardholder A) from the transaction database 112 for a particular time segment (for example, last three months) (see, 302). The transaction database 112 stores all previously recorded payment transactions of the cardholder 102. The data pre-processing engine 220 is configured to extract a plurality of transaction features and generate a plurality of input vectors corresponding to the plurality of transaction features. In particular, the data pre-processing engine 220 is configured to perform a featurization process over the payment transaction data to extract the plurality of transaction features. The plurality of transaction features is based on payment terminal types (such as, ATM, POS, e-commerce), payment transaction channels (such as, cross-border, domestic), merchant industry types, etc. For generating the plurality of input vectors, the payment transaction data is aggregated based on the merchant category codes, POS transaction type, ATM transaction type, etc. In one example, suppose, the cardholder 102 spends a total amount of $10, $20, $0, $40, $30 using a payment card C1 at a grocery merchant M1 on weekly basis in a month. The cardholder 102 also spends a total amount of $300, $600, $1000, $0, $0 in cross-border payment transactions using the payment card C1 on the weekly basis in the same month. Thus, the multivariate payment transaction sequence for the cardholder 102 can be represented in the following exemplary manner:
Week 1 Week 2 Week 3 Week 4 Week 5
Merchant M1 $40 $20 $0 $40 $30
Cross Border Transactions $300 $600 $1000 0 $0
Table 1
The data pre-processing engine 220 is configured to provide a plurality of input vectors corresponding to the plurality of transaction features to the consumer control engine 222 (see, 304). The consumer control engine 222 is configured to apply a set of consumer control models over the plurality of input vectors and generate a probability score corresponding to each of the set of consumer control models. The set of consumer control models includes, but is not limited to, the CB model, the transaction amount model, the ATM model, the e-commerce model, the POS model, and the MCC model.
In one example, each of the set of consumer control models analyzes an input vector corresponding to a particular transaction feature and classifies transaction behavior of the cardholder 102 into one category or class. In other words, each consumer control model generates a probability score based on the input vector associated with the particular transaction feature. In one example, the consumer control engine 222 predicts a probability whether next transaction of the cardholder 102 may exceed a threshold transaction amount or not based on the transaction amount model. In another example, the consumer control engine 222 may predict whether the cardholder 102 may perform the next transaction at an ATM machine or not based on the ATM model. In yet another example, output of the e-commerce model may predict whether the cardholder 102 may perform the next transaction at an e-commerce website or not and the output of the MCC model may predict whether the cardholder 102 may perform the next transaction at a certain merchant category code or not
Based on the probability scores from the set of consumer control models, the consumer control engine 222 is configured to determine model recommendation strategy for the cardholder 102. The model recommendation strategy includes information of consumer controls that need to be set by cardholder 102 in real-time. The consumer control engine 222 is configured to provide a probability vector representing the model recommendation strategy to the DRL agent 224 (see, 306).
In one embodiment, the data pre-processing engine 220 is configured to access past recommendation data sent to the cardholder 102 from the cardholder profile database 308 (see, 310) and calculate the user activeness data using survival analysis methods. The user activeness data represents recommendation adoption rate of the cardholder 102 after sending recommendations to the cardholder 102. The purpose of calculating recommendation adoption rate is to send consumer control recommendations to only potential cardholders and cardholders having similar transaction behavior data as the potential cardholders.
In one embodiment, the data pre-processing engine 220 is configured to calculate a hazard value using the following hazard function:
λ(t) = lim┬(dt→0)⁡〖(Pr{t≤T, where,
1) S is a State Space, which includes a set of environmental states that the agent 402 may perceive. Herein, at any time t, a state of the DRL model 400 is defined based on the plurality of transaction features of each cardholder, model recommendation strategy determined by the one or more consumer control models for each cardholder, and information of previously set consumer controls by each cardholder on the application.
2) A is an Action Space, which includes a set of actions that the agent 402 may take on each state of the environment. Herein, the set of actions includes the set of consumer controls that may be recommended to a cardholder.
3) R is a reward function and R (s, a, s’) represents a reward that the agent 402 obtains from the environment 404 when the action ‘a’ is performed on the state s and the state is changed to state s’. Herein, the reward function is defined based on one or more factors such as: probability scores associated with the plurality of transaction features of a particular cardholder for a particular time (for example, next quarter), a probability of occurrence of fraudulent transaction for the particular cardholder in the particular time, and recommendation adoption rate associated with the particular cardholder.
4) T is a state transition function and T (s, a, s’) may represent a probability of executing action ‘a’ on state ‘s’ and moving to state s'.
5) γ is a constant factor, based on which the reward is calculated.
Further, the MDP methods may also require following parameters to define:
6) Episode: One episode represents providing a recommendation of a particular consumer control to the cardholder.
7) Policy space: A stochastic policy outputs the possibility of an action for every time step t.
In the process of interaction between the agent 402 and the environment 404, the agent 402 senses that the environment state at time t is 'st'. Based on the environment state 'st', the agent 402 may select an action 'at' from the action space A to execute. After the environment 404 receives the action selected by the agent 402, it returns corresponding reward signal feedback Rt+1 to the agent 402 and transfers to new environment state st+1, and waits for the agent 402 to make a new decision. In the process of interacting with the environment 404, the goal of the agent 402 is to find the maximum Q-value based on the reward function by finding the threshold limits for the set of transaction variables.
The optimal action-value function may be calculated using the following Bellman equation:
Q*(s, a) = 𝔼s´~ε [r + γmax¦(a´) Q* (s´, a´)|s, a] …. Eqn. (3)
The above equation states that the Q*-value yielded from being at state s and performing action a is equal to the immediate reward r plus the highest Q*-value possible from the next state s', and γ is a discount factor which controls the contribution of rewards further in the future. In other words, the Q*(s, a) is a cumulative reward value of rewards generated in the subsequent learning optimization when the agent 402 executes the action 'a' in the state 's'.
The optimal action-value function as an iterative update may be calculated using the following equation:
Qi+1 (s, a) = 𝔼 [r + γ maxa´ Qi (s´, a´)|s, a] …. Eqn. (4)
However, calculation of the optimal action-value function as an iterative update may not be possible practically as it requires infinite samples for action-value function to converge. Thus, the DRL model utilizes an approximator function to calculate the action-value function. The action-value function may be calculated (using the approximator function) using the following equation:
Q (s, a; θ)≈Q^* (s,a) …. Eqn. (5)
Further, in the DRL model 400, a neural network architecture is utilized to approximate Q value-function. The state is given as the input and the Q-values of all possible actions are generated as the output.
The deep neural network may be trained by minimizing loss function at each iteration, where the loss function is defined based on the following equation :
Li (θ_i) = 𝔼 s, a~ρ(.) [(yi – Q (s, a; θ_i))2] …. Eqn. (6)
Based on the above DRL model 400, the server system 200 provided by the present disclosure determines the one or more consumer controls to be recommended to the cardholder 102. In addition, the server system 200 determines whether to recommend the one or more consumer controls to the cardholder 102 based, at least in part, on control adoption rate of the recommended consumer controls by the cardholder 102 in the past and a probability of occurrence of fraudulent payment transaction associated with the cardholder 102 in the future.
FIG. 4B is a block diagram representation 420 of a deep neural network 422 of the DRL model 400, in accordance with an example embodiment of the present disclosure.
As mentioned above, in deep reinforcement learning, in the process of interacting with the environment, the goal of the agent 402 is to find an optimal strategy such that it receives the maximum cumulative reward in any state s and any time step t. In some example embodiments, the above objective may be achieved using a Q-value function approximation algorithm. In other example embodiments, the foregoing objectives may also be implemented by using other DRL algorithms such as a strategy approximation algorithm, which is not limited herein.
In one embodiment, the DRL model 400 may include one or more neural networks. The deep neural network 422 receives a current state 424 as input. As shown in the FIG. 4B, the deep neural network 422 includes an input layer, multiple hidden layers, and an output layer.
More illustratively, during the training, the DRL agent 402 runs a plurality of simulations by selecting one or more consumer controls, randomly, and identifies reward values corresponding to the plurality of simulations using a reward function.
The deep neural network 422 is utilized to approximate the Q-value function. The MDP in the DRL model 400 includes a state space S and action space A. The current state is defined based on the model recommendation strategy determined by the set of consumer control models for a cardholder, previously set consumer controls by the cardholder, the plurality of transaction features and user features of the cardholder. The action space includes a plurality of actions of recommending a plurality of consumer control actions for setting up one or more consumer controls on a user interface of the application 118 installed on the user device.
Since the state space S can be infinite and the action space A is finite, the DRL model 400 utilizes the deep neural network 422 to predict Q-value of each of the actions. The Q-value with the maximum reward will be shared as the notification for the recommendation of the one or more consumer controls to the cardholder 102. The overall objective of the deep neural network 422 is to select the action in such a way that it maximizes the future rewards for the DRL model 400.
In one embodiment, the input to the deep neural network 422 is a current state determined based on the user features, the model recommendation strategy determined by the set of consumer models and previously set consumer controls by the cardholder 102. The current stage is changed when an action such as, adding a consumer control in consumer control recommendation for the cardholder 102 is performed. The reward value may be calculated based on an infinite horizon discounted model. In one embodiment, the reward value may be based on multiple different values and may be determined using an algorithm that weighs different values differently. The reward function may be calculated using the following equation:
Ri = ti + Pc …. Eqn. (7)
Where (a) ti depends upon a probability score for a particular transaction feature corresponding to the cardholder 102 received from a consumer control model and a probability score of fraud associated with the particular transaction feature for the cardholder 102 and (b) Pc is the probability of conversion of a transaction feature of the plurality of transaction features recommendation adoption rate). In other words, the reward function is calculated as the sum of the first term ti and the second term (i.e., recommendation adoption rate or conversion probability).
In one embodiment, sigmoid function may be applied on value of Ri. The sigmoid function is applied to determine a value in range of 0 to 1 as reward space may become very large. The DRL agent 224 recommends the one or more consumer controls whose reward chances are above a threshold value. In addition, reward chances are slightly nudged in the positive direction by value of Pc.
In addition, ti may be calculated using the following equation:
ti = {█(S_i,if S_(i )≥ k@〖-S〗_i,if S_i < k )┤ …. Eqn. (8)
Where ti is a step function that facilitates generation of positive rewards if value of Si (i.e., weighted sum of feature transaction chances and fraud chances) is high enough to be considered, otherwise, the value of ti facilitates generation of negative rewards. The threshold k is chosen by issuers to control number of recommendations. If k is set to a high value, then number of recommendations is low but high quality. So, k controls trade-off between number of recommendations and precision value. The Si is calculated using the below formula:
Si = α (1-Pi)+ βPfi …. Eqn. (9)
where, Pi is a probability of transaction feature in next quarter, Pfi is a probability of fraud for the transaction feature in the next quarter, and ⍺, β are the hyper-parameters.
In one embodiment, the issuer server 108 or the server system 200 may set value of α as low and value of β as high to only recommend the consumer controls to the cardholders having high probability of fraudulent transactions in the future. The Si will be high if the probability of transaction feature in next quarter is low and probability of fraud for the transaction feature in the next quarter is high and thus the cardholder 102 is recommended to set decline/alert for this feature. In one example, if the consumer control models predict that the cardholder will not do any ATM transactions and fraud chances are high then the cardholder 102 is recommended to set decline/alert for the transaction feature.
In one embodiment, the deep neural network 422 uses value-based reinforcement learning model as the action space A is finite. In one example embodiment, the deep neural network 422 is a model free algorithm as it solves reinforcement learning task directly using samples from the environment without explicitly estimating the environment.
The deep neural network 422 predicts Q-value of each of the actions and the goal of the deep neural network 422 is to select the high Q-values (Q-value with the high rewards) and recommend the one or more consumer controls associated with the high Q-values to the cardholder 102. In one embodiment, the deep neural network 422 selects the budget transaction control, the transaction amount control, and the cross-border transaction control as the one or more consumer controls to be recommended to the cardholder 102.
In an embodiment, after offline training of the DRL model, the DRL model learns from data of the cardholders in real-time using off-policy training. In addition, the DRL agent utilizes a prioritized experience replay to store experiences. The prioritized experience replay stores the samples that create highest surprise (highest loss) and re-trains the DRL agent based on these stored samples after every few iterations. In one example embodiment, the prioritized experience replay facilitates determination of samples used for training of the DRL model.
The DRL model 400 uses exploration to learn behavior of the cardholder 102 in real-time. In one example embodiment, the DRL model uses duel banding GD for performing exploration. The DRL model generates parameters W_hat by slightly varying current weights W and further generates Q-values based, at least in part, on the generated parameters W_hat. The DRL model further recommends the actions generated from W_hat with some probability value. If the cardholder 102 selects the recommended action, the DRL agent moves the DRL model 230 in the direction of W_hat. The value of W_hat may be calculated using the following equation:
∆W=α*rand(-1,1)*W …. Eqn. (10)
W´ = W + ηW ̃ …. Eqn. (11)
Based on exploration, the DRL agent sends random recommendation of any of the one or more consumer controls to the cardholder 102. If the cardholder 102 accepts the recommendation of the DRL agent and sets the recommended consumer control in the application, the DRL agent fine-tunes itself to learn user behavior of the cardholder 102.
The DRL agent provides learning upon observing the consequences of selected actions and alters its own behavior in response to rewards received. The DRL agent including the deep neural network 422 receives a state s(t) at timestep t and performs an action at state s(t). When the DRL agent selects the action, the state gets transitioned to a new state s(t + 1) based on the current state and the chosen action. The new state s(t + 1) is returned to the agent to await a further action. The state is a sufficient statistic of the simulation and includes information for the DRL agent to suggest an optimal action at that time.
The output of the deep neural network 422 represents predicted Q-values (i.e., Q value-action 1 426a, Q value-action 2 426b….Q value-action n 426n) for each state-action pair. The reward function may be the error of the predicted Q-value and the target Q-value. To the extent the predicted Q value from the deep neural network 422 differs from the target Q-value, various training techniques, (such as, back propagation, stochastic gradient descent, etc.,) may be employed to adjust various weights associated with the deep neural network 422 to reduce the loss values.
It is noted that performance metrics of some of the set of consumer control models are studied. Specifically, performance metrics for the ATM model, the electronic commerce model, the cross-border model, and the transaction amount model are studied. The above stated set of models have been implemented on the plurality of cardholders and information of number of recommended cardholders with particular consumer controls along with accuracy is illustrated in Table 2:
Consumer Control Model Total number of cards Total number of recommended cards Accuracy
Automated teller machine model 2.8M 238k 99%
Electronic commerce model 2.8M 271k 85%
Cross border model 2.8M 1.3M 80%
Transaction amount model 2.8M 566k 87%
Table 2. Performance metrics of some of the set of consumer control models
FIG. 5 shows exemplary user interface (UI) 500 of an application installed on a user device for setting up one or more consumer controls, in accordance with an embodiment of the present disclosure.
As mentioned earlier, the user device 104 associated with the cardholder 102 is installed with the application 118 for manually setting up one or more user authorization controls for payment transactions associated with the cardholder 102.
The server system 200 transmits a recommendation to the cardholder 102 for manually setting up one or more consumer controls. The cardholder 102 can launch the application 118 to set up one or more consumer controls for the payment transactions. The one or more consumer controls represent consumer actions for setting up threshold limits for a set of transaction variables via user selectable interface elements provided on a user interface of the application 118 accessible on a user device of the cardholder 102. The one or more consumer controls may limit the set of transaction variables of the payment transactions.
As shown in the FIG. 5, the UI 500 includes consumer control data fields (see, 502a, 502b, 502c, 502d, 502e, 502f, 502g, 502h) that can be set by the cardholder 102 for controlling the payment transactions.
Each consumer control data field is associated with a user selectable interface element (see, 504a, 504b, 504c, 504d, 504e, 504f, 504g, 504h) to set the corresponding consumer control. For example, the consumer control data field 502a associated with a transaction amount can set an alert via the user selectable interface element 504a when a threshold limit for the transaction amount for a payment transaction of the cardholder 102 is reached. The cardholder 102 may click/press/tap on the corresponding user selectable element to activate the particular consumer control of the set of consumer controls. The cardholder 102 may again click/press/tap on any of the buttons (activated consumer controls) to deactivate the particular consumer control of the set of consumer controls.
In one example, the cardholder 102 clicks/presses/taps on the user selectable element (see, 504c) to activate the cross-border transaction control. Suppose the payment card associated with the cardholder 102 is issued in United States of America and a transaction request associated with the same payment card originates in London. Since, the consumer control for the cross-border transaction control is activated and thus, the server system 200 may transmit a notification or decline the transaction request originating through the payment card of the cardholder 102 in London.
Based on the recommendations received from the server system 200 via push notifications, the cardholder 102 can set the recommended consumer controls using the UI 500.
In similar manner, the consumer data field (see, transaction channel control 502b) can select transaction channels through which payment transaction can be performed. In one example, the cardholder 102 can set an alert or decline for internet transactions. The cardholder 102 may set alerts for payment transactions performed at merchants associated with some specific merchant category codes (such as, restaurants) via the merchant category code data field 502d. The cardholder 102 may also set an alert and decline when spending exceeds a limited amount set of an hourly or daily time interval via the budgets consumer control data field 502f.
FIG. 6 is a flow chart of a method 600 for training the deep reinforcement learning (DRL) model 230, in accordance with an example embodiment of the present disclosure. The sequence of operations of the method 600 may not be necessarily executed in the same order as they are presented. Further, one or more operations may be grouped and performed in the form of a single step, or one operation may have several sub-steps that may be performed in parallel or in a sequential manner.
At 602, the server system 200 accesses historical payment transaction data associated with a plurality of cardholders from the transaction database 112. The historical payment transaction data may include weekly, monthly or yearly recorded data associated with various transactions performed by the plurality of cardholders in past. In one example embodiment, the historical payment transaction data associated with the plurality of cardholders is accessed to identify transaction behavior of each of the plurality of cardholders.
FIG. 6 is a flow chart of a method 600 for training the deep reinforcement learning (DRL) model 230, in accordance with an example embodiment of the present disclosure. The sequence of operations of the method 600 may not be necessarily executed in the same order as they are presented. Further, one or more operations may be grouped and performed in the form of a single step, or one operation may have several sub-steps that may be performed in parallel or in a sequential manner.
At 602, the server system 200 accesses historical payment transaction data associated with a plurality of cardholders from the transaction database 112. The historical payment transaction data may include weekly, monthly or yearly recorded data associated with various transactions performed by the plurality of cardholders in past. In one embodiment, the historical payment transaction data associated with the plurality of cardholders is accessed to identify transaction behavior of each of the plurality of cardholders.
At 604, the server system 200 generates the plurality of input vectors associated with the plurality of transaction features based, at least in part, on the historical payment transaction data of each cardholder. Each of the plurality of transaction features indicate spending pattern of each of the plurality of cardholders.
At 606, the server system 200 determines a probability vector by executing the set of consumer control models with the plurality of input vectors as input of each cardholder. In one embodiment, each of the set of consumer control models is a classifier model.
At 608, the server system 200 trains the deep reinforcement learning (DRL) model 230. In one embodiment, the DRL model 230 is fed with the probability vector as an input. The DRL model 230 is trained by performing steps 608a-608d in iterative manner.
At 608a, the server system 200 defines an initial state of the deep reinforcement learning (DRL) model 230. The initial state is defined based on the plurality of transaction features (e.g., transaction amount, transaction count, etc.) of a particular cardholder (e.g., “cardholder 102”), user features (e.g., user activeness parameter indicating recommendation adoption rate), consumer controls that are already set by the particular cardholder, and the model recommendation strategy for the particular cardholder determined by the consumer control models.
At 608b, the server system 200 defines an action space of the deep reinforcement learning (DRL) model 230. The action space includes the set of consumer controls.
At 608c, the server system 200 samples actions for simulations based on control adoption rate of the recommended consumer controls by the cardholder 102 in the past and a probability of occurrence of fraudulent payment transaction associated with the cardholder 102 in the future. (As explained above)
At 608d, the server system 200 generates reward values corresponding to the sampled actions based, at least in part, on a reward function. The reward function is defined based on one or more factors such as: probability scores associated with the plurality of transaction features of a particular cardholder for a particular time (for example, next quarter), a probability of occurrence of fraudulent transaction for the particular cardholder in the particular time, and control adoption rate associated with the particular cardholder.
At 608e, the server system 200 identifies optimal actions to train the deep reinforcement learning (DRL) model 230 that leads to a maximum reward value. The DRL model 230 is trained to further recommend the one or more consumer controls to the cardholder. In one embodiment, the DRL model 230 may be re-trained based on feedback received in real-time.
FIG. 7 is a flow chart of a method 700 for execution of the deep reinforcement learning (DRL) model 230, in accordance with an example embodiment of the present disclosure. The sequence of operations of the method 700 may not be necessarily executed in the same order as they are presented. Further, one or more operations may be grouped and performed in form of a single step, or one operation may have several sub-steps that may be performed in parallel or in a sequential manner.
At 702, the server system 200 accesses payment transaction data associated with the cardholder 102 from the transaction database 112. The payment transaction data may include weekly, monthly or yearly recorded data associated with various transactions performed by the cardholder 102 in past. In one example embodiment, the payment transaction data associated with the cardholder 102 is accessed to identify transaction behavior of the cardholder 102.
At 704, the server system 200 generates the plurality of input vectors associated with the plurality of transaction features based, at least in part, on the transaction data of the cardholder 102. In one example embodiment, each of the plurality of transaction features indicate spending pattern of the cardholder 102.
At 706, the server system 200 determines the probability vector by executing the set of consumer control models with the plurality of input vectors as input for the cardholder 102. In one example embodiment, each of the set of consumer control models is a classifier model. The set of consumer control models include the CB model, the transaction amount model, the ATM model, the e-commerce model, the POS model, and the MCC model.
At 708, the server system 200 determines a state in the deep reinforcement learning (DRL) model 230. The server system 200 sets selection of a candidate number of consumer controls for recommendation as actions of the DRL model.
At 710, the server system 200 calculates Q-values corresponding to state-action pairs formed by the state and the actions using a deep neural network.
At 712, the server system 200 selects an action i.e., recommend a consumer control to the cardholder 102.
At 714, the server system 200 determines a reward value corresponding to the selected action (i.e., the recommended consumer control) based on the reward function. After performing the selected action, the deep reinforcement learning model 400 goes to a new state. In one embodiment, the action with the reward value in the action space satisfies the preset condition is added in the recommendation strategy. The preset condition may include an action with the reward value greater than a threshold value or a preset number of actions with maximum Q-value. For example, the action space is the set of consumer controls. The Q-function estimation values are arranged in descending order and the first three or four values with the highest Q-function estimation values are presented as the recommended actions.
At 716, the server system 200 checks whether the reward value is greater than or equal to a threshold value.
At 718, when the reward value is greater than or equal to the threshold value, the server system 200 adds the one or more consumer controls into the recommendation.
At 720, when the reward value is not greater than the threshold value, the server system 200 selects another action (e.g., another consumer control) that has a maximum Q-value from all Q-values for all possible actions in the new state.
At 722, the server system 200 transmits the notification including the determined one or more consumer controls to be recommended to the cardholder 102, on the user device 104 of the cardholder 102. The server system 200 transmits the notification via web push notification, application push notification, and the like.
FIG. 8 is a process flow chart of a computer-implemented method 800 for recommending the one or more consumer controls to the cardholder 102, in accordance with an example embodiment of the present disclosure. The method 800 depicted in the flow chart may be executed by, for example, a computer system. The computer system is identical to the server system 200. Operations of the flow chart of the method 800, and combinations of operation in the flow chart of the method 800, may be implemented by, for example, hardware, firmware, a processor, circuitry, and/or a different device associated with the execution of software that includes one or more computer program instructions. It is noted that the operations of the method 800 can be described and/or practiced by using a system other than these computer systems. The method 800 starts at operation 802.
At operation 802, the method 800 includes accessing, by the server system 200, payment transaction data associated with the cardholder 102 for the particular time segment from the transaction database 112.
At operation 804, the method 800 includes generating, by the server system 200, the plurality of input vectors associated with the plurality of transaction features based, at least in part, on the payment transaction data.
At operation 806, the method 800 includes determining, by the server system 200, the probability vector by executing the set of consumer control models with the plurality of input vectors as inputs. Each consumer control model is associated with the transaction feature of the plurality of transaction features. The probability vector has a length equal to the number of the set of consumer control models.
At operation 808, the method 800 includes determining, by the server system 200, the one or more consumer controls for the cardholder 102 based, at least in part, on the DRL model 230 and the probability vector. The one or more consumer controls represents cardholder actions for setting up threshold limits for the set of transaction variables via user selectable interface elements provided on the user interface of the application 118 accessible on the user device 104 of the cardholder 102.
At operation 810, the method 800 includes transmitting, by the server system 200, a recommendation message to the cardholder 102. The recommendation message includes the one or more consumer controls to be set by the cardholder 102 on the application 118.
FIG. 9 is a simplified block diagram of an issuer server 900 used for sending recommendations of the one or more consumer controls to the cardholder 102, in accordance with an example embodiment of the present disclosure. The issuer server 900 is an example of the issuer server 108 of FIG. 1. The issuer server 900 is associated with an issuer bank/issuer, in which a cardholder (e.g., “the cardholder 102”) may have an account, which provides a payment card. The issuer server 900 includes a processing module 905 operatively coupled to a storage module 910 and a communication module 915. The components of the issuer server 900 provided herein may not be exhaustive and the issuer server 900 may include more or fewer components than those depicted in FIG. 9. Further, two or more components may be embodied in one single component, and/or one component may be configured using multiple sub-components to achieve the desired functionalities. Some components of the issuer server 900 may be configured using hardware elements, software elements, firmware elements and/or a combination thereof.
The storage module 910 is configured to store machine executable instructions to be accessed by the processing module 905. Additionally, the storage module 910 stores information related to, contact information of the cardholder 102, bank account number, availability of funds in the account, payment card details, transaction details and/or the like. Further, the storage module 910 is configured to store payment transactions.
In one embodiment, the issuer server 900 is configured to store profile data (e.g., an account balance, a credit line, details of the cardholder (i.e., “the cardholder 102”), account identification information, payment card number) in the transaction database 112. The details of the cardholder 102 may include, but not limited to, name, age, gender, physical attributes, location, registered contact number, family information, alternate contact number, registered e-mail address, or the like of the cardholder etc.
The processing module 905 is configured to communicate with one or more remote devices such as a remote device 920 using the communication module 915 over a network such as the network 110 of FIG. 1. The examples of the remote device 920 include the server system 200, the payment server 114, the transaction database 112 or other computing systems of the issuer server 900 and the network 110 and the like. The communication module 915 is capable of facilitating such operative communication with the remote devices and cloud servers using API (Application Program Interface) calls. The communication module 915 is configured to receive a payment transaction request performed by the cardholder (i.e., “the cardholder 102”) via the network 110. The processing module 905 receives a payment card information, a payment transaction amount, a customer information and merchant information from the remote device 920 (i.e., the user device 104 or the payment server 114). The issuer server 900 includes a transaction database 930 for storing transaction data. The transaction data may include, but not limited to, transaction attributes, such as transaction amount, source of funds such as bank or credit cards, transaction channel used for loading funds such as POS terminal or ATM machine, transaction velocity features such as count and transaction amount sent in the past x days to a particular user, transaction location information, external data sources and other internal data to evaluate each transaction. The issuer server 900 includes a user profile database 925 storing user profile associated with the plurality of cardholders.
In one embodiment, the issuer server 900 is also configured to store historical fraudulent chargeback activities associated with the plurality of cardholders in a fraud and chargeback database 935. The user profile data may include an account balance, a credit line, and details of the cardholder (i.e., “the cardholder 102”), account identification information, payment card number, or the like. The details of the cardholder (i.e., “the cardholder 102”) may include, but not limited to, name, age, gender, physical attributes, location, registered contact number, family information, alternate contact number, registered e-mail address, or the like of the cardholder (i.e., “the cardholder 102”).
FIG. 10 shows a simplified block diagram of a device 1000 for example, the user device 104 of FIG. 1 capable of implementing the various embodiments of the present disclosure. The device 1000 is depicted to include one or more applications 1006. The device 1000 is an example of the user device 104. It should be understood that the device 1000 as illustrated and hereinafter described is merely illustrative of one type of device and should not be taken to limit the scope of the embodiments. As such, it should be appreciated that at least some of the components described below in connection with that the device 1000 may be optional and thus in an example embodiment may include more, less or different components than those described in connection with the example embodiment of the FIG. 10. As such, among other examples, the device 1000 could be any of an electronic device, for example, cellular phones, tablet computers, laptops, mobile computers, personal digital assistants (PDAs), mobile televisions, mobile digital assistants, or any combination of the aforementioned, and other types of communication or multimedia devices.
The illustrated device 1000 includes a controller or a processor 1002 (e.g., a signal processor, microprocessor, ASIC, or other control and processing logic circuitry) for performing such tasks as signal coding, data processing, image processing, input/output processing, power control, and/or other functions. An operating system 1004 controls the allocation and usage of the components of the device 1000 and support for one or more applications programs (see, applications 1006), such as an application interface in a user device (e.g., the user device 104) of a user (e.g., the cardholder 102) or an application interface in a user device (e.g., the user device 104). The application interface in the user device 104 is used for turning on/off the one or more consumer controls, receiving notifications containing recommendations of the one or more consumer controls from the server system 106 of FIG. 1, and the like.
In addition to the application interface, the applications 1006 may include common mobile computing applications (e.g., telephony applications, email applications, calendars, contact managers, web browsers, messaging applications such as USSD messaging or SMS messaging or SIM Tool Kit (STK) application) or any other computing application.
The illustrated device 1000 includes one or more memory components, for example, a non-removable memory 1008 and/or removable memory 1010. The non-removable memory 1008 and/or removable memory 1010 may be collectively known as database in an embodiment. The non-removable memory 1008 can include RAM, ROM, flash memory, a hard disk, or other well-known memory storage technologies. The removable memory 1010 can include flash memory, smart cards, or a Subscriber Identity Module (SIM). The one or more memory components can be used for storing data and/or code for running the operating system 1004 and the applications 1006. The device 1000 may further include a user identity module (UIM) 1012. The UIM 1012 may be a memory device having a processor built in. The UIM 1012 may include, for example, a subscriber identity module (SIM), a universal integrated circuit card (UICC), a universal subscriber identity module (USIM), a removable user identity module (R-UIM), or any other smart card. The UIM 1012 typically stores information elements related to a mobile subscriber. The UIM 1012 in form of the SIM card is well known in Global System for Mobile Communications (GSM) communication systems, Code Division Multiple Access (CDMA) systems, or with third-generation (3G) wireless communication protocols such as Universal Mobile Telecommunications System (UMTS), CDMA7000, wideband CDMA (WCDMA) and time division-synchronous CDMA (TD-SCDMA), or with fourth-generation (4G) wireless communication protocols such as LTE (Long-Term Evolution).
The device 1000 can support one or more input devices 1020 and one or more output devices 1030. Examples of the input devices 1020 may include, but are not limited to, a touch screen / a screen 1022 (e.g., capable of capturing finger tap inputs, finger gesture inputs, multi-finger tap inputs, multi-finger gesture inputs, or keystroke inputs from a virtual keyboard or keypad), a microphone 1024 (e.g., capable of capturing voice input), a camera module 1026 (e.g., capable of capturing still picture images and/or video images) and a physical keyboard 1028. Examples of the output devices 1030 may include but are not limited to a speaker 1032 and a display 1034. Other possible output devices can include piezoelectric or other haptic output devices. Some devices can serve more than one input/output function. For example, the touch screen 1022 and the display 1034 can be combined into a single input/output device.
A wireless modem 1040 can be coupled to one or more antennas (not shown in the FIG. 10) and can support two-way communications between the processor 1002 and external devices, as is well understood in the art. The wireless modem 1040 is shown generically and can include, for example, a cellular modem 1042 for communicating at long range with the mobile communication network, a Wi-Fi compatible modem 1044 for communicating at short range with an external Bluetooth-equipped device or a local wireless data network or router, and/or a Bluetooth-compatible modem 1046. The wireless modem 1040 is typically configured for communication with one or more cellular networks, such as a GSM network for data and voice communications within a single cellular network, between cellular networks, or between the device 1000 and a public switched telephone network (PSTN).
The device 1000 can further include one or more input/output ports 1050 for establishing connection with peripheral devices including a power supply 1052, one or more sensors 1054 for example, an accelerometer, a gyroscope, a compass, or an infrared proximity sensor for detecting the orientation or motion of the device 1000 and biometric sensors for scanning biometric identity of an authorized user, a transceiver 1056 (for wirelessly transmitting analog or digital signals) and/or a physical connector 1060, which can be a USB port, IEEE 1294 (FireWire) port, and/or RS-232 port. The illustrated components are not required or all-inclusive, as any of the components shown can be deleted and other components can be added.
With the application (see, applications 1006) and/or other software or hardware components, the device 1000 can implement the technologies described herein. In one example embodiment, the processor 1002 can cause receiving of the notification including the one or more recommended consumer controls from the server system 106 of FIG. 1.
Without limiting the scope of the present disclosure, the one or more example embodiments disclosed herein provide methods and systems for recommending one or more consumer controls to the cardholder. The server system transmits the notification including the one or more consumer controls recommended to the cardholder. The server system facilitates reduction in number of fraudulent transactions associated with the cardholder. The server system decides whether to send the one or more consumer control recommendations to only the cardholders who have already set the consumer controls or to the cardholders that are vulnerable to fraudulent transactions in future.
The disclosed methods with reference to FIGS. 1 to 10, or one or more operations of the method 600, 700 and 800 may be implemented using software including computer-executable instructions stored on one or more computer-readable media (e.g., non-transitory computer-readable media, such as one or more optical media discs, volatile memory components (e.g., DRAM or SRAM), or nonvolatile memory or storage components (e.g., hard drives or solid-state nonvolatile memory components, such as Flash memory components) and executed on a computer (e.g., any suitable computer, such as a laptop computer, net book, Web book, tablet computing device, smart phone, or other mobile computing device). Such software may be executed, for example, on a single local computer or in a network environment (e.g., via the Internet, a wide-area network, a local-area network, a remote web-based server, a client-server network (such as a cloud computing network), or other such network) using one or more network computers. Additionally, any of the intermediate or final data created and used during implementation of the disclosed methods or systems may also be stored on one or more computer-readable media (e.g., non-transitory computer-readable media) and are considered to be within the scope of the disclosed technology. Furthermore, any of the software-based embodiments may be uploaded, downloaded, or remotely accessed through a suitable communication means. Such suitable communication means include, for example, the Internet, the World Wide Web, an intranet, software applications, cable (including fiber optic cable), magnetic communications, electromagnetic communications (including RF, microwave, and infrared communications), electronic communications, or other such communication means.
Although the disclosure has been described with reference to specific exemplary embodiments, it is noted that various modifications and changes may be made to these embodiments without departing from the broad spirit and scope of the disclosure. For example, the various operations, blocks, etc., described herein may be enabled and operated using hardware circuitry (for example, complementary metal oxide semiconductor (CMOS) based logic circuitry), firmware, software and/or any combination of hardware, firmware, and/or software (for example, embodied in a machine-readable medium). For example, the apparatuses and methods may be embodied using transistors, logic gates, and electrical circuits (for example, application specific integrated circuit (ASIC) circuitry and/or in Digital Signal Processor (DSP) circuitry).
Particularly, the server system 200 (e.g., the server system 106) and its various components such as the computer system 202 and the database 204 may be enabled using software and/or using transistors, logic gates, and electrical circuits (for example, integrated circuit circuitry such as ASIC circuitry). Various embodiments of the disclosure may include one or more computer programs stored or otherwise embodied on a computer-readable medium, wherein the computer programs are configured to cause a processor or computer to perform one or more operations. A computer-readable medium storing, embodying, or encoded with a computer program, or similar language, may be embodied as a tangible data storage device storing one or more software programs that are configured to cause a processor or computer to perform one or more operations. Such operations may be, for example, any of the steps or operations described herein. In some embodiments, the computer programs may be stored and provided to a computer using any type of non-transitory computer readable media. Non-transitory computer readable media include any type of tangible storage media. Examples of non-transitory computer readable media include magnetic storage media (such as floppy disks, magnetic tapes, hard disk drives, etc.), optical magnetic storage media (e.g., magneto-optical disks), CD-ROM (compact disc read only memory), CD-R (compact disc recordable), CD-R/W (compact disc rewritable), DVD (Digital Versatile Disc), BD (BLU-RAY® Disc), and semiconductor memories (such as mask ROM, PROM (programmable ROM), EPROM (erasable PROM), flash memory, RAM (random access memory), etc.). Additionally, a tangible data storage device may be embodied as one or more volatile memory devices, one or more non-volatile memory devices, and/or a combination of one or more volatile memory devices and non-volatile memory devices. In some embodiments, the computer programs may be provided to a computer using any type of transitory computer readable media. Examples of transitory computer readable media include electric signals, optical signals, and electromagnetic waves. Transitory computer readable media can provide the program to a computer via a wired communication line (e.g. electric wires, and optical fibers) or a wireless communication line.
Various embodiments of the disclosure, as discussed above, may be practiced with steps and/or operations in a different order, and/or with hardware elements in configurations, which are different than those which, are disclosed. Therefore, although the disclosure has been described based upon these exemplary embodiments, it is noted that certain modifications, variations, and alternative constructions may be apparent and well within the spirit and scope of the disclosure.
Although various exemplary embodiments of the disclosure are described herein in a language specific to structural features and/or methodological acts, the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as exemplary forms of implementing the claims.