Claims:A method for processing machine-readable optical codes, the method comprising:
receiving, by processing circuitry, an image of a machine-readable optical code, wherein the received image corresponds to a failed scanning-attempt of the machine-readable optical code;
applying, by the processing circuitry, a trained classifier to the received image, wherein a neural network based application of the trained classifier to the received image comprises:
comparing, by the trained classifier, the received image with a plurality of error categories, wherein the plurality of error categories are associated with a plurality of pre-defined errors in at least one of a display of the machine-readable optical code and a capturing of the image of the machine-readable optical code; and
generating, by the trained classifier, a classification output based on the comparison of the image with the plurality of error categories, wherein the classification output includes a plurality of confidence scores for the plurality of error categories, respectively;
classifying, by the processing circuitry, the received image into a first error category of the plurality of error categories based on the classification output; and
rendering, by the processing circuitry, a first user interface on a user device to present a pre-defined error associated with the first error category, wherein upon resolving the pre-defined error, the machine-readable optical code is processed successfully.

2. The method of claim 1, wherein the neural network based application of the trained classifier to the received image further comprises:
generating, by the trained classifier, a feature map based on the image, wherein the feature map is a numerical representation of one or more image features of the image, wherein the image is compared with the plurality of error categories based on the generated feature map, and wherein a confidence score of the plurality of confidence scores indicates a degree of similarity between the image and a corresponding error category of the plurality of error categories.

3. The method of claim 1, further comprising detecting, by the processing circuitry, a capture of the image at the user device based on at least one of an activation of an imaging device or a notification received from the user device, wherein the image is received based on the detection of the capture of the image.
4. The method of claim 1, further comprising:
selecting, by the processing circuitry, from a plurality of recommended actions, a recommended action to resolve the pre-defined error associated with the first error category; and
rendering, by the processing circuitry, the first user interface to further present the selected recommended action, wherein the pre-defined error is resolved by implementing the selected recommended action.

5. The method of any of claims 1 to 4, further comprising:
receiving, by the processing circuitry, a merchant identifier and location information of each of a plurality of merchant entities from corresponding plurality of merchant devices, wherein the plurality of merchant entities are associated with a plurality of machine-readable optical codes including the machine-readable optical code; and
storing, by the processing circuitry, in a memory, the merchant identifier and the location information of each of the plurality of merchant entities.

6. The method of claim 5, further comprising:
receiving, by the processing circuitry, real time location information of the user device in combination with the image;
selecting, by the processing circuitry, from the plurality of merchant entities, a merchant entity associated with the machine-readable optical code based on the received real time location information and the location information stored in the memory; and
rendering, by the processing circuitry, a second user interface on a merchant device of the selected merchant entity to present the pre-defined error associated with the first error category, wherein upon resolving the pre-defined error, the machine-readable optical code is processed successfully.

7. The method of any of claims 1 to 4, wherein a payment transaction between a user of the user device and a merchant entity is executed based on a successful processing of the machine-readable optical code.

8. The method of any of claims 1 to 4, wherein the machine-readable optical code is one of a barcode or a quick response (QR) code.

9. A system for processing machine-readable optical codes, the system comprising:
memory configured to store therein a trained classifier, wherein the trained classifier is a neural network; and
processing circuitry configured to:
receive an image of a machine-readable optical code, wherein the received image corresponds to a failed scanning-attempt of the machine-readable optical code;
apply the trained classifier stored in the memory to the received image, wherein the trained classifier is configured to:
compare the image with a plurality of error categories, wherein the plurality of error categories are associated with a plurality of pre-defined errors in at least one of a display of the machine-readable optical code and a capturing of the image of the machine-readable optical code; and
generate a classification output based on the comparison of the image with the plurality of error categories, wherein the classification output includes a plurality of confidence scores for the plurality of error categories, respectively;
classify the received image into a first error category of the plurality of error categories based on the classification output; and
render a first user interface on a user device to present a pre-defined error associated with the first error category, wherein upon resolving the pre-defined error, the machine-readable optical code is processed successfully.

10. A method for processing machine-readable optical codes, the method comprising:
receiving, by processing circuitry, a training dataset including a plurality of images of a plurality of machine-readable optical codes, wherein each image in the training dataset is associated with an error category of a plurality of error categories, and wherein the plurality of error categories are associated with a plurality of pre-defined errors in at least one of a display of a machine-readable optical code and a capturing of an image of the machine-readable optical code; and
training, by the processing circuitry, a neural network using the training dataset to learn a relationship between the plurality of images of the plurality of machine-readable optical codes and the plurality of error categories, wherein upon successful training, the neural network is applied to a first image of a first machine-readable optical code and the first image is classified into a first error category of the plurality of error categories based on the neural network.
, Description:BACKGROUND

FIELD OF THE DISCLOSURE
Various embodiments of the present disclosure relate generally to machine-readable optical codes. More particularly, various embodiments of the present disclosure relate to methods and systems for processing machine-readable optical codes.

DESCRIPTION OF THE RELATED ART
Advancement in technology and proliferation of the Internet has revolutionized the way consumers perform payment transactions. For example, individuals nowadays can conduct payment transactions using machine-readable optical codes, for example, Quick Response (QR) codes and barcodes. Such optical codes offer a seamless and contactless option to consumers and merchants for conducting payment transactions.
Though optical codes enable instantaneous payment transactions, many a times consumers face problems while scanning the optical codes. There could be a variety of reasons that can lead to failed scanning attempts of optical codes, for example, improper positioning of a scanning equipment (for example, a smartphone), damaged or distorted optical code, poor contrast ratio of the optical code, or the like. Most of the times, a consumer tries to identify problems in the scanning equipment; however, there is no way for the consumer to know the exact cause that led to a failed scanning attempt. Such problems degrade optical code-based transaction experience of the consumers, which is undesirable.
In light of the foregoing, there is a need for a technical solution that solves the above-mentioned problems and improves optical code-based transaction experience for both consumers and merchants.

SUMMARY

In an embodiment of the present disclosure, a method for processing machine-readable optical codes is provided. The method includes receiving, by processing circuitry, an image of a machine-readable optical code such that the received image corresponds to a failed scanning-attempt of the machine-readable optical code. A trained classifier is applied to the received image. A neural network based application of the trained classifier to the received image includes comparing, by the trained classifier, the received image with a plurality of error categories, and generating, by the trained classifier, a classification output based on the comparison of the image with the plurality of error categories. The plurality of error categories are associated with a plurality of pre-defined errors in at least one of a display of the machine-readable optical code and a capturing of the image of the machine-readable optical code. The classification output includes a plurality of confidence scores for the plurality of error categories, respectively. The method further includes classifying, by the processing circuitry, the image into a first error category of the plurality of error categories based on the classification output. A first user interface is rendered on a user device by the processing circuitry to present a pre-defined error associated with the first error category. The machine-readable optical code may be processed successfully upon resolving the pre-defined error.
In another embodiment of the present disclosure, a system for processing machine-readable optical codes is provided. The system includes a memory and processing circuitry. The memory is configured to store therein a trained classifier that is a neural network. The processing circuitry is configured to receive an image of a machine-readable optical code and apply the trained classifier stored in the memory to the received image. The received image corresponds to a failed scanning-attempt of the machine-readable optical code. The trained classifier is configured to compare the image with a plurality of error categories and generate a classification output based on the comparison of the image with the plurality of error categories. The plurality of error categories are associated with a plurality of pre-defined errors in at least one of a display of the machine-readable optical code and a capturing of the image of the machine-readable optical code. The classification output includes a plurality of confidence scores for the plurality of error categories, respectively. The processing circuitry is further configured to classify the image into a first error category of the plurality of error categories based on the classification output. The processing circuitry is further configured to render a first user interface on a user device to present a pre-defined error associated with the first error category. Upon resolving the pre-defined error, the machine-readable optical code is processed successfully.
In another embodiment of the present disclosure, a method for processing machine-readable optical codes is provided. The method includes receiving, by processing circuitry, a training dataset including a plurality of images of a plurality of machine-readable optical codes. Each image in the image dataset is associated with an error category of a plurality of error categories. The plurality of error categories are associated with a plurality of pre-defined errors in at least one of a display of a machine-readable optical code and a capturing of an image of the machine-readable optical code. A neural network is trained by the processing circuitry using the training dataset to learn a relationship between the plurality of images of the plurality of machine-readable optical codes and the plurality of error categories. Upon successful training, the neural network is applied to a first image of a first machine-readable optical code and the first image is classified into a first error category of the plurality of error categories based on the neural network.

BRIEF DESCRIPTION OF DRAWINGS
The accompanying drawings illustrate the various embodiments of systems, methods, and other aspects of the disclosure. It will be apparent to a person skilled in the art that the illustrated element boundaries (e.g., boxes, groups of boxes, or other shapes) in the figures represent one example of the boundaries. In some examples, one element may be designed as multiple elements, or multiple elements may be designed as one element. In some examples, an element shown as an internal component of one element may be implemented as an external component in another and vice versa.
Various embodiments of the present disclosure are illustrated by way of example, and not limited by the appended figures, in which like references indicate similar elements:
FIG. 1 is a block diagram that illustrates an exemplary environment for processing machine-readable optical codes, in accordance with an exemplary embodiment of the present disclosure;
FIG. 2A is a block diagram that illustrates a merchant device of FIG. 1, in accordance with an exemplary embodiment of the present disclosure;
FIG. 2B is a block diagram that illustrates a user device of FIG. 1, in accordance with an exemplary embodiment of the present disclosure;
FIG. 3A is a block diagram that illustrates an application server of FIG. 1, in accordance with an exemplary embodiment of the present disclosure;
FIG. 3B is a block diagram that illustrates training of a classifier, in accordance with an embodiment of the present disclosure;
FIG. 4 is a process flow diagram that illustrates registration of a merchant entity with an application server of FIG. 1, in accordance with an exemplary embodiment of the present disclosure;
FIG. 5A is a diagram that illustrates an exemplary scenario for processing a machine-readable optical code by an application server of FIG. 1, in accordance with an exemplary embodiment of the present disclosure;
FIG. 5B is a diagram that illustrates an exemplary scenario for processing an optical code-based payment transaction, in accordance with an exemplary embodiment of the present disclosure;
FIG. 6 is a block diagram that illustrates a user device of FIG. 1, in accordance with another embodiment of the present disclosure;
FIG. 7 is a block diagram that illustrates a system architecture of a computer system, in accordance with an exemplary embodiment of the present disclosure;
FIG. 8 is a high-level flow chart that illustrates a method for training a classifier to facilitate processing of machine-readable-optical codes, in accordance with an exemplary embodiment of the present disclosure;
FIGS. 9A and 9B, collectively, represent a high-level flow chart that illustrates a method for processing machine-readable-optical codes, in accordance with an exemplary embodiment of the present disclosure;
FIG. 10 is a high-level flow chart that illustrates a method creating a look-up table for registered merchant entities, in accordance with an exemplary embodiment of the present disclosure; and
FIGS. 11A and 11B, collectively, represent a flow chart that illustrates a method for processing machine-readable-optical codes, in accordance with an exemplary embodiment of the present disclosure.
Further areas of applicability of the present disclosure will become apparent from the detailed description provided hereinafter. It should be understood that the detailed description of exemplary embodiments is intended for illustration purposes only and is, therefore, not intended to necessarily limit the scope of the present disclosure.

DETAILED DESCRIPTION OF EMBODIMENTS

The present disclosure is best understood with reference to the detailed figures and description set forth herein. Various embodiments are discussed below with reference to the figures. However, those skilled in the art will readily appreciate that the detailed descriptions given herein with respect to the figures are simply for explanatory purposes as the methods and systems may extend beyond the described embodiments. In one example, the teachings presented and the needs of a particular application may yield multiple alternate and suitable approaches to implement the functionality of any detail described herein. Therefore, any approach may extend beyond the particular implementation choices in the following embodiments that are described and shown.
References to “an embodiment”, “another embodiment”, “yet another embodiment”, “one example”, “another example”, “yet another example”, “for example”, and so on, indicate that the embodiment(s) or example(s) so described may include a particular feature, structure, characteristic, property, element, or limitation, but that not every embodiment or example necessarily includes that particular feature, structure, characteristic, property, element or limitation. Furthermore, repeated use of the phrase “in an embodiment” does not necessarily refer to the same embodiment.
OVERVIEW
Conventionally, a machine-readable optical code-based transaction executed between a user and a merchant may fail due to a failed scanning-attempt of the machine-readable optical code. Besides, there is no way for the user or the merchant to know the exact cause that led to the failed scanning-attempt.
Various embodiments of the present disclosure provide a method and a system that solve the above-mentioned problem by identifying an error at user device’s end and/or at merchant’s end that may have led to a failed scanning-attempt of a machine-readable optical code. The method is implemented by a system (e.g., an application server) including processing circuitry. The processing circuitry may train a classifier (for example, a neural network) using an image dataset including multiple images of incorrectly scanned machine-readable optical codes. Each image in the image dataset may be labelled or tagged with a pre-defined error related to the corresponding machine-readable optical code. For example, an image in the image dataset may be labelled as “dead zone violation” indicating that the machine-readable optical code in the image has dead zone violation error. Similarly, another image in the image dataset may be labelled as “incorrect scanning position” indicating that a position of a scanning equipment used for scanning the machine-readable optical code was incorrect. In other words, the images in the image dataset are associated with a plurality of error categories indicated by the labels, where the plurality of error categories are associated with a plurality of pre-defined errors. A pre-defined error may be one of an error in a display of a machine-readable optical code or an error in capturing of an image of the machine-readable optical code. During training, the classifier may learn a relationship between the images of the machine-readable optical codes in the image dataset and the plurality of error categories. In an implementation phase, the processing circuitry may receive an image of a machine-readable optical code, where the image is captured during a failed scanning-attempt of the machine-readable optical code. The processing circuitry may apply the trained classifier to the received image. The trained classifier may compare the received image with the plurality of error categories and generate a classification output based on the comparison of the image with the plurality of error categories. The classification output of the trained classifier may include a plurality of confidence scores for the plurality of error categories, respectively. Each confidence score indicates a strength of association or a degree of similarity between the received image and the corresponding error category. The processing circuitry may further classify the received image into a first error category of the plurality of error categories based on the classification output. For example, the processing circuitry may classify the image into the first error category for which the confidence score in the classification output is highest. When the first error category into which the received image is classified is associated with a pre-defined error, the processing circuitry establishes that the received image is associated with the pre-defined error which led to the erroneous scanning of the machine-readable optical code. The pre-defined error associated with the first error category may be an error in a display of the machine-readable optical code or an error in capturing of the image of the machine-readable optical code. The processing circuitry may further render a user interface on a user device and/or a merchant device to present the pre-defined error associated with the first error category and a recommended action to resolve the pre-defined error, such that the machine-readable optical code may be processed successfully, upon resolving the pre-defined error.
Thus, at times when a machine-readable optical code is not getting scanned properly, the method and system of the present disclosure enable identification of a potential error that hindered the scanning process. The system of the present disclosure performs image classification using a neural network to predict the potential error. The system not only predicts the potential error, but may also recommend a suitable or a most appropriate action to resolve the potential error, thereby improving optical code-based transaction experience of consumers and merchants.
TERM DESCRIPTION (in addition to plain and dictionary meaning)
Machine-readable optical code may be an optical code or label that includes encrypted information. The encrypted information may be a unique identifier, a website link, or any other information. Examples of the machine-readable optical code may include a quick response (QR) code, a universal product codes (UPC), a barcode, or the like. The machine-readable optical code may be utilized by merchants for facilitating various activities in a contactless manner. Examples of such activities may include identifying items, people, documents, objects, facilitating digital payment transactions, displaying digital menus, ticketing links, or the like. The machine-readable optical code may be displayed on a display screen or printed on a surface to be scanned and decoded by a scanning equipment, for example a smartphone, a scanner, an imaging device, or the like. For example, a user may scan a QR code using a smartphone for performing a payment transaction for a purchase at a merchant. Scanning of the machine-readable optical code may sometimes fail due to an error in a position of the scanning equipment capturing the machine-readable optical code or an error in a display of the machine-readable optical code.
User device may be an electronic communication device that enables a user to scan a machine-readable optical code and perform an online payment transaction or any other activity facilitated by the machine-readable optical code. Examples of the user device may include a mobile phone, a laptop, a smartphone, a tablet, a phablet, a wearable device such as a digital watch, or the like.
Plurality of error categories may refer to different categories of error associated with optical code scanning. Each error category corresponds to a specific error that may potentially occur while scanning a machine-readable optical code, for example, a QR code. Examples of the plurality of error categories may include, but are not limited to, an incorrect positioning category, a dead zone violation category, a damage or distorted category, an inverted color category, improper size category, a poor contrast ratio category, an overcrowded optical code category, a quiet zone violation category, a blurred category, or the like
Pre-defined error may refer to an error in capturing of a machine-readable optical code or an error in a display of the machine-readable optical code due to which a scanning attempt of the machine-readable optical code may fail. Error in the capturing of the machine-readable optical code may refer to any error associated with a scanning equipment used for scanning the machine-readable optical code. Examples of errors in the capturing of the machine-readable optical code may include an incorrect position, such as an angle with respect to the machine-readable optical code and a distance between the scanning equipment and the machine-readable optical code, of the scanning equipment. Examples of errors in the display of the machine-readable optical code may include a dead zone violation in the optical code, inverted colors in the optical code, an improper size (e.g., smaller or larger than required readable size) of the optical code, a poor contrast ratio between a background of the optical code and the optical code, an overcrowded optical code, a quiet zone violation in the optical code, a blurred optical code, a damager optical code, a distorted optical code, or the like.
Trained classifier may refer to a trained neural network for image classification. The classifier may be trained based on a plurality of images that are labelled with various pre-defined errors in optical code scanning. In other words, the classifier may be trained based on the plurality of images that are associated with a plurality of error categories where the plurality of error categories correspond to a plurality of pre-defined errors in optical code scanning. When inputted with a target image, the trained classifier generates a feature map for the target image and compares the received image with the plurality of error categories based on the generated feature map. The trained classifier further generates a classification output indicating confidence scores for the error categories. A confidence score indicates a strength of association or a degree of similarity between the inputted target image and an error category. For example, if the classification output has first and second confidence scores for first and second error categories, respectively, and the first confidence score is greater than the second confidence score, the inputted target image has higher similarity or correlation with the first error category as compared to the second error category. Examples of the trained classifier may include, but are not limited to, a convolutional neural network (CNN), a recurrent neural network (RNN), and a radial basis functional neural network.
User interface may refer to a human machine interface using which a user interacts with a computer or a using which the computer interacts with the user. In an example, the user interface may be a graphical user interface including selectable icons, input boxes, and information display area.
A merchant identifier may refer to a unique identifier assigned to a merchant to uniquely identity the merchant for various payment related activities. Examples of the merchant identifier may include a contact number of the merchant, an email id of the merchant, a unified payment interface (UPI) identifier of the merchant, or the like.
An application server may refer to a computing server that facilitates error-fre processing of machine-readable optical codes. The application server may be physical or cloud data processing system on which a server program runs. The application server may be implemented in hardware or software, or a combination thereof. In one embodiment, the application server may be implemented in computer programs executed on programmable computers, such as personal computers, laptops, or a network of computer systems. The application server may be implemented as a third-part server, or as a part of an acquirer server, a payment network server, or an issuer server.
FIG. 1 is a block diagram that illustrates an exemplary environment for processing machine-readable optical codes, in accordance with an exemplary embodiment of the present disclosure. With reference to FIG. 1, an exemplary environment 100 is shown that includes a plurality of merchant devices, such as a first merchant device 102a and a second merchant device 102b, an acquirer server 104, a user device 106, an issuer server 108, a payment network server 110, and an application server 112. The first merchant device 102a, the second merchant device 102b, the acquirer server 104, the user device 106, the issuer server 108, the payment network server 110, and the application server 112 may communicate with each other by way of a communication network 114 or through separate communication channels therebetween.
The first merchant device 102a and the second merchant device 102b may refer to computing devices deployed at a plurality of merchant premises, such as a first merchant premise and a second merchant premise, respectively. Examples of the first merchant premise and the second merchant premise may include, but are not limited to, restaurants, retail shops, or ticketing counters. Examples of the first and second merchant devices 102a and 102b may include mobile phones, laptops, smartphones, desktops, tablets, phablets, touch-based interactive display screens, or the like. The first and second merchant devices 102a and 102b may be handled or operated by a plurality of operators (for example, a first operator 116a and a second operator 116b) at the first merchant premise and the second merchant premise, respectively. Further, each of the first and second merchant premises may be associated with a unique geographical location indicated by one or more geographical coordinates, such as longitude, latitude, elevation, and altitude coordinates. For example, the first merchant premise may be established at a first geographical location represented by first geographical coordinates and the second merchant premise may be established at a second geographical location that is different from the first geographical location and represented by second geographical coordinates. In an example, the first merchant premise may be a retail shop on a 2nd floor of a shopping complex represented by a unique combination of longitude, latitude, and altitude coordinates, and the second merchant premise may be another retail shop on a 1st floor of the shopping complex represented by another unique combination of longitude, latitude, and altitude coordinates. Different geographical locations in multistorey buildings may be represented more accurately by including altitude coordinates along with longitude and latitude coordinates.
The first and second merchant devices 102a and 102b may include suitable logic, circuitry, interfaces, and/or code, executable by the circuitry, for managing various operations associated with the first and second merchant premises, respectively. For example, the first merchant device 102a may be configured to maintain a log of various products and/services available for sale at the first merchant premise, historical purchase data of the first merchant premise, or the like. The first and second merchant devices 102a and 102b may further include one or more location capturing sensors, for example, global positioning system (GPS) sensors (shown in FIG. 2A), to capture a current location of the first and second merchant devices 102a and 102b. Since the first and second merchant devices 102a and 102b are deployed at the first and second merchant premises, the current locations of the first and second merchant devices 102a and 102b are same as the respective first and second geographical locations of the first and second merchant premises. Various components of the first merchant device 102a are described in detail in FIG. 2A.
Each of the first and second merchant premises may be further associated with a unique machine-readable optical code (such as machine-readable optical codes 118a and 118b, respectively) to facilitate one or more payment activities (for example, payment transactions) by consumers. Examples of the machine-readable optical codes 118a and 118b may include Quick response (QR) codes and barcodes. Hereinafter, the terms “machine-readable optical codes”, “optical codes”, “QR codes”, and “barcodes” are used interchangeably.
In an embodiment, the optical codes 118a and 118b may be displayed using the first and second merchant devices 102a and 102b, respectively. In another embodiment, the optical codes 118a and 118b may be printed or affixed on cardboard, plastic, paper, or wall displays at the respective first and second merchant premises. Users visiting the first and second merchant premises may scan the respective displayed optical codes 118a and 118b using their scanning devices (for example, smartphones) to carry out payment transactions in a contactless manner. Each optical code 118a and 118b may include encrypted information that uniquely identifies a corresponding merchant entity and facilitates instantaneous payment transactions. For example, the optical code 118a may include an encrypted unique identifier linked to a payment account of a merchant entity of the first merchant premise. Examples of the unique identifier may include a contact number linked to a payment account, a unified payment interface (UPI) identifier linked to the payment account, or the like.
In order to accept payments from users, merchant entities of the first and second merchant premises may have established first and second merchant accounts, respectively, with a financial institution, such as an acquirer, that is a part of a financial payment system. Thus, the optical codes 118a and 118b include encrypted identifiers linked to the first and second merchant accounts of the respective merchant entities. The acquirer server 104 is associated with the acquirer.
The acquirer server 104 may be a server arrangement which includes suitable logic, circuitry, interface, and/or code, executable by the circuitry, for processing various payment transactions. For example, the acquirer server 104 may be configured to credit and debit the first and second merchant accounts based on payment transactions made to and from the first and second merchant accounts. The acquirer server 104 may be configured to communicate one or more notifications to the first and second merchant devices 102a and 102b to indicate various payment activities (for example, payment credit, payment debit, or the like) related to the respective first and second merchant accounts. Examples of the first and second merchant accounts may include a bank account, a digital wallet, or the like. The acquirer server 104 may communicate with the first and second merchant devices 102a and 102b via the communication network 114.
The user device 106 may refer to a portable computing device of a user, for example, a user 120. Examples of the user device 106 may include a mobile phone, a laptop, a smartphone, a tablet, a phablet, a wearable device such as a digital watch, or the like. The user device 106 may be used by the user 120 to perform one or more payment related activities, such as performing optical code-based payment transactions. It will be apparent to a person of ordinary skill in the art that the user device 106 may also be used by the user 120 to perform various non-payment related activities along with the payment related activities.
The user device 106 may include one or more location capturing sensors, for example, GPS sensors, to capture a current location of the user device 106. Since the user device 106 is in the possession of the user 120, the current location of the user 120 is same as the current location of the user device 106. The user device 106 may further include one or more imaging devices (shown in FIG. 2B) that enable the user 120 to capture, scan, and decrypt various optical codes (such as the optical codes 118a and 118b) displayed at various merchant premises to perform instantaneous payment transactions. Various components of the user device 106 are described in detail in FIG. 2B.
In order to perform digital payment transactions, the user 120 may have established a payment account with a financial institution, such as an issuer, that is a part of the financial payment system. The issuer server 108 is associated with the issuer that maintains the payment account (hereinafter referred to as “the user account”) of the user 120. The issuer server 108 may be a server arrangement which includes suitable logic, circuitry, interface, and/or code, executable by the circuitry, for processing various payment transactions. For example, the issuer server 108 may be configured to credit and debit the user account based on payment transactions made to and from the user account. The issuer server 108 may be configured to communicate one or more notifications to the user device 106 to indicate various payment activities (for example, payment credit, payment debit, or the like) related to the user account. Examples of the user account may include a bank account, a digital wallet, or the like. The issuer server 108 may communicate with the user device 106 via the communication network 114.
The payment network server 110 may be a computing server that is operated by a payment network. The payment network may be an intermediate entity between acquirers (for example, the acquirer associated with the acquirer server 104) and issuers (e.g., an issuer associated with the issuer server 108) for processing transactions. In other words, the acquirer server 104 may communicate with the issuer server 108 through the payment network server 110 to determine whether a user account is in good standing and has sufficient funds and balance to cover a transaction amount of a payment transaction. Authorization of the payment transaction is declined or accepted based on such determinations performed between the issuer server 108 and the acquirer server 104 via the payment network server 110.
Examples of the acquirer server 104, the issuer server 108, and the payment network server 110 may include, but are not limited to, computers, laptops, mini-computers, mainframe computers, any non-transient and tangible machines that may execute a machine-readable code, cloud-based servers, distributed server networks, a network of computer systems, or a combination thereof.
The user device 106 may be further configured to run (or execute) one or more computer executable applications for performing various operations. In an example, the user device 106 may have various mobile applications (for example, optical code scanning applications) installed thereon for recognizing, scanning, and decrypting optical codes. Such mobile applications typically utilize the imaging device of the user device 106 to capture an image of a displayed optical code (for example, the optical codes 118a and 118b) and then decrypt the information included in the scanned optical code. The decrypted information may be utilized by a payment service application running on the user device 106 to initiate a payment transaction between the user account and the other payment account linked to the scanned optical code. However, such mobile applications are usually incapable of decrypting the optical code if the optical code is not scanned or captured properly. Besides no conventional optical code scanning application provides the user 120 with a solution that would help in correct scanning of the optical code in case of a failed scanning-attempt of the optical code. Thus, the user 120 may not be left with any choice other than to keep performing scanning attempts or pay via cash or a physical payment card.
The application server 112 may be configured to offer a solution for the abovementioned problem arising during scanning of optical codes (e.g., the optical codes 118a and 118b) by identifying and reporting an error that may have led to the failed scanning-attempt of the optical code at the user device 106. The application server 112 may include suitable logic, circuitry, interface, and/or code, executable by the circuitry, that perform various operations required for identifying and reporting errors associated with optical code scanning. In other words, the application server 112 may identify the error that led to the failed scanning attempt of an optical code (e.g., the optical code 118a or 118b) at the user device 106 and may report the identified error to the user device 106 and/or a corresponding merchant device (e.g., the first merchant device 102a or the second merchant device 102b) along with a recommended action to resolve the error. The application server 112 may be configured to utilize one or more artificial intelligence models (for example, convolutional neural network model, recurrent neural network model, radial basis functional neural network model, or the like) to identify the error that led to the failed scanning attempt of the optical code and recommend the most appropriate action that when performed by the user 120 and/or the corresponding merchant entity will resolve the identified error. Various components and operation of the application server 112 are described in detail in FIGS. 3A and 3B.
Examples of the application server 112 may include, but are not limited to, computers, laptops, mini-computers, mainframe computers, any non-transient and tangible machines that may execute a machine-readable code, cloud-based servers, distributed server networks, a network of computer systems, or a combination thereof. Although the application server 112 is shown to be a standalone entity in FIG. 1, the scope of the disclosure is not limited to it. In another embodiment, the application server 112 may be implemented as a part of the acquirer server 104, the issuer server 108, or the payment network server 110 without deviating from the scope of the disclosure.
The communication network 114 may be a medium through which content and messages are transmitted between the first merchant device 102a, the second merchant device 102b, the user device 106, the acquirer server 104, the issuer server 108, the payment network server 110, and the application server 112, and other entities that are pursuant to one or more standards for the interchange of transaction requests, such as the ISO8583 standard. Examples of the communication network 114 may include, but are not limited to, a Wi-Fi network, a light fidelity (Li-Fi) network, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a satellite network, the Internet, a fiber optic network, a coaxial cable network, an infrared (IR) network, a radio frequency (RF) network, and combinations thereof. Various entities in the environment 100 may connect to the communication network 114 in accordance with various wired and wireless communication protocols, such as Transmission Control Protocol and Internet Protocol (TCP/IP), User Datagram Protocol (UDP), Long Term Evolution (LTE) communication protocols, or any combination thereof.
In operation, the application server 112 may be configured to host a service application (shown in FIGS. 2A and 2B) executable on various devices (for example, the first and second merchant devices 102a and 102b, and the user device 106). The service application, when executed on the first and second merchant devices 102a and 102b, and the user device 106, may function as a gateway between the application server 112 and the first and second merchant devices 102a and 102b, and the user device 106. The application server 112 may be configured to execute a registration process (for example, a sign-up process) when the merchant entities and the user 120 request access to the service application for the first time.
During the registration process, unique login credentials (for example, a login identifier (id) and a password) are created for an entity (e.g., the merchant entities and the user 120) requesting access to the service application. In addition, during registration, basic profile information, such as name, geographical location, valid contact information, or the like, is procured from the entity. Upon successful validation of the profile information, the entity is registered and is able login to the service application using the login credentials. For various merchant entities that have registered with the application server 112 for accessing the service application, the application server 112 may be configured to maintain a look-up table (shown in FIG. 3A) that includes various records, each record having registered contact information and geographical location information of a respective merchant entity. The look-up table may enable the application server 112 to identify merchant entities based on their geographical locations and establish contact using the registered contact information, when required.
The user device 106 may have various payment applications, optical code scanning applications, and the service application running thereon. The user device 106 be utilized by the user 120 to scan the optical code 118a to make a payment transaction for a purchase made at the first merchant premise. In a scenario, where an optical code scanning application on the user device 106 is unable to correctly recognize the optical code 118a based on an image of the optical code 118a captured by the user device 106, the service application may receive the image of the improperly scanned optical code 118a and transmit the received image to the application server 112 via the communication network 114. The application server 112 may be configured to receive the image of the optical code 118a and apply a trained neural network to the received image. The application server 112 may be further configured to classify the received image into one of a plurality of error categories based on a classification output of the trained neural network. Here, the plurality of error categories may be associated with a plurality of pre-defined errors in either a display of an optical code or a capturing of an image of the optical code. In other words, based on the classification output of the trained neural network, the application server 112 may identify a cause (or error) that led to the improper scanning of the optical code 118a. The cause may be an improper way of using the user device 106 to capture the optical code 118a or some inherent fault with the display of the optical code 118a. Examples of various pre-defined errors that may lead to a failed scanning attempt of an optical code include dead zone violation (e.g., characters printed in a dead-zone of optical code), damage or distortion in the optical code, an improper reading position of the user device 106 while scanning the optical code, inverted colors in the optical code, improper size (e.g., smaller than required readable size) of the optical code, poor contrast ratio between a background of the optical code and the optical code, overcrowding of the optical code, or the like. The abovementioned examples of the pre-defined errors are well known in the domain of optical codes. Hereinafter, the term “pre-defined error” and “error” are used interchangeably.
Once the error that led to the improper scanning of the optical code 118a is identified based on the classification of the image, the application server 112 may be configured to control the service application running on the user device 106 or the first merchant device 102a associated with the optical code 118a to render a user interface to present the identified error. Based on the display of the identified error, the user 120 or the merchant entity associated with the optical code 118a may rectify the error, resulting in a successful processing of the optical code 118a in a subsequent scanning attempt, and in turn a successful processing of the payment transaction by the issuer server 108, the payment network server 110, and the acquirer server 104.
In an embodiment, the application server 112 may be further configured to select, from a plurality of recommended actions, a suitable recommended action that may resolve the identified error. The selection of the suitable recommended action may vary depending upon the identified error. For example, when the error corresponds to an improper reading position of the user device 106, the selected recommended action may be a suggestion to achieve a proper reading position (for example, a required angle of the user device 106 with respect to a plane of the optical code 118a) for the user device 106. In another example, when the identified error corresponds to a distortion in the optical code 118a, the selected recommended action may be a suggestion to the merchant entity of the optical code 118a to reprint the optical code 118a.
In an embodiment, the application server 112 may be configured to implement a local version of the artificial intelligence model on the user device 106 by way of the service application. In such a scenario, the operations performed by the application server 112 for identifying the error may be performed locally on the user device 106 by way of the service application. An operation of the user device 106 based on the local version of the artificial intelligence model is described in FIG. 6.
FIG. 2A is a block diagram that illustrates a merchant device of FIG. 1, in accordance with an exemplary embodiment of the present disclosure. FIG. 2A is described in conjunction with elements of FIG. 1. With reference to FIG. 2A, a block diagram 200A is shown that illustrates various components of the first merchant device 102a. Though FIG. 2A only illustrates the components of the first merchant device 102a, it will be understood by a person of ordinary skill in the art that the second merchant device 102b may be structurally and functionally similar to the first merchant device 102a, without deviating from the scope of the disclosure.
The first merchant device 102a may include first processing circuitry 202, a first memory 204, and a first network interface 206. The first processing circuitry 202 may include one or more processors (for example, a first processor 208), one or more input/output (I/O) interfaces (for example, a first I/O interface 210), and one or more sensors (for example, a first location sensor 212) that enable the first merchant device 102a to execute various operations.
The first processor 208 may include suitable logic, circuitry, interface, and/or code, executable by the circuitry, to perform one or more operations required for the operation of the first merchant device 102a. Examples of the first processor 208 may include, but are not limited to, an application-specific integrated circuit (ASIC) processor, a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a field-programmable gate array (FPGA), a central processing unit (CPU), or a microprocessor.
The first I/O interface 210 may include suitable logic, circuitry, interface, and/or code, executable by the circuitry, to perform one or more input and output operations required for the operation of the first merchant device 102a. The first I/O interface 210 may include, but are not limited to, a touch screen, a keyboard, a mouse, a joystick, a projector, an audio output interface, a microphone, an image-capture device, a liquid crystal display (LCD) screen, a light emitting diode (LED) screen, and/or a speaker.
The first location sensor 212 may include suitable logic, circuitry, interface, and/or code, executable by the circuitry, to perform one or more location sensing operations required for the operation of the first merchant device 102a. The first location sensor 212 may be a GPS sensor. It will be apparent to a person of ordinary skill in the art that the first location sensor 212 can be any other location sensor that is different from a GPS sensor without deviating from the scope of the disclosure.
The first memory 204 may include suitable logic, circuitry, and/or interfaces to store various instructions or code which when executed by the first processor 208 controls the first processor 208 to perform the corresponding operations. For example, the first memory 204 may store one or more instructions, related to a service application 214 hosted by the application server 112, which when executed may cause the first processor 208 to run the service application 214. Examples of the first memory 204 may include a random-access memory (RAM), a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), a removable storage drive, a hard disk drive (HDD), a flash memory, a solid-state memory, or the like.
The second network interface 214 may include suitable logic, circuitry, interfaces, and/or code, executable by the circuitry, to transmit and receive data over the communication network 114 using one or more communication network protocols. The second network interface 214 may transmit requests, messages, and notifications to and receive requests, messages, and notifications from the acquirer server 104 and the application server 112. Examples of the second network interface 214 may include, but are not limited to, an antenna, a radio frequency transceiver, a wireless transceiver, a Bluetooth transceiver, an ethernet port, a USB port, or any other device configured to transmit and receive data.
In operation, the service application 214 may be downloaded and installed on the first merchant device 102a by the merchant entity or the first operator 116a. In other words, one or more instructions that may cause the first processor 208 to execute the installed service application 214 are stored in the first memory 204. Based on the instructions, the first processor 208 may execute the service application 214, and the first I/O interface 210 may render a user interface (UI) of the executed service application 214 on a display of the first merchant device 102a. When the service application 214 is accessed for the first time on the first merchant device 102a, the registration or sign-up process may be executed. The UI may prompt the merchant entity or the first operator 116a to input a unique login-id and password (e.g., login credentials), and basic profile information such as a name, a geographical location, valid contact information, or the like for the sign-up. In an embodiment, the geographical location of the first merchant device 102a may be automatically inputted in the UI by the first processor 208 based on a current location captured by the first location sensor 212. However, if the merchant entity or the the first operator 116a wants to input an alternate geographical location, an address may be inputted manually using the first I/O interface 210.
The first network interface 206 may be configured to communicate the information inputted through the displayed UI to the application server 112 via the communication network 114. Based on a successful validation, the merchant entity is registered with the application server 112 for the use of the service application 214. For subsequent use of the service application 214 on the first merchant device 102a, the UI may prompt the merchant entity to input the login credentials for sign-in.
When a payment transaction is successfully performed using the optical code 118a, the first network interface 206 may be configured to receive a credit notification from the acquirer server 104 indicting a credit in the first merchant account. The first I/O interface 210 may be configured to render a UI to present the credit notification.
In an embodiment, when the application server 112 identifies an error in the display of the optical code 118a based on an image of the optical code 118a captured by the user device 106 or any other user device registered with the application server 112, the first network interface 206 may be configured to receive a notification (for example, an in-app notification of the service application 214) from the application server 112 that may control the first I/O interface 210 to render a UI for presenting the identified error in the optical code 118a to the first operator 116a. The UI may further present a recommended action to resolve the identified error.
In an embodiment, the first I/O interface 210 (for example, a display screen) may be utilized to display the optical code 118a for facilitating one or more payment related activities at the first merchant premise. In such an embodiment, the optical code 118a may be generated by the first processor 208 under the control of a payment application running on the first merchant device 102a.
FIG. 2B is a block diagram that illustrates a user device of FIG. 1, in accordance with an exemplary embodiment of the present disclosure. FIG. 2B is described in conjunction with elements of FIG. 1. With reference to FIG. 2B, a block diagram 200B is shown that illustrates various components of the user device 106.
The user device 106 may include second processing circuitry 216, a second memory 218, and a second network interface 220. The second processing circuitry 216 may include one or more processors (for example, a second processor 222), one or more auxiliary devices (for example, an imaging device 224), one or more I/O interfaces (for example, a second I/O interface 226), and one or more sensors (for example, a second location sensor 228) that enable the user device 106 to execute various operations.
The second processor 222 may include suitable logic, circuitry, interface, and/or code, executable by the circuitry, to perform one or more operations required for the operation of the user device 106. Examples of the second processor 222 may include, but are not limited to, an ASIC processor, a RISC processor, a CISC processor, an FPGA, a CPU, or a microprocessor.
The imaging device 224 may include suitable logic, circuitry, interface, and/or code, executable by the circuitry, to perform one or more image capturing operations. When activated, the imaging device 224 may be configured to capture a digital image of a front view or rear view of the user device 106. For example, the imaging device 224 may be activated to capture a digital image of an optical code (for example, the optical codes 118a and 118b) for scanning the optical code. Examples of the imaging device 224 may include a digital camera, a solid-state imaging device, or the like.
The second I/O interface 226 may include suitable logic, circuitry, interface, and/or code, executable by the circuitry, to perform one or more input and output operations required for the operation of the user device 106. The second I/O interface 226 may include, but are not limited to, a touch screen, a keyboard, a mouse, a joystick, a projector, an audio output interface, a microphone, an image-capture device, an LCD screen, an LED screen, and/or a speaker.
The second location sensor 228 may include suitable logic, circuitry, interface, and/or code, executable by the circuitry, to perform one or more location sensing operations required for the operation of the user device 106. The second location sensor 228 may be a GPS sensor. It will be apparent to a person of ordinary skill in the art that the second location sensor 228 can be any other location sensor that is different from a GPS sensor without deviating from the scope of the disclosure.
The second memory 218 may include suitable logic, circuitry, and/or interfaces to store various instructions or code which when executed by the second processor 222 causes the second processor 222 to perform corresponding operations. For example, the second memory 218 may store one or more instructions related to the service application 214, hosted by the application server 112, which when executed may cause the second processor 222 to run the service application 214. The second memory 218 may be further configured to store one or more digital images captured by the imaging device 224. The second memory 218 may be accessible by the service application 214 based on a consent of the user 120. Examples of the second memory 218 may include a RAM, a ROM, a PROM, an EPROM, a removable storage drive, an HDD, a flash memory, a solid-state memory, or the like.
The second network interface 220 may include suitable logic, circuitry, interfaces, and/or code, executable by the circuitry, to transmit and receive data over the communication network 114 using one or more communication network protocols. The second network interface 220 may transmit requests, messages, and notifications to and receive requests, messages, and notifications from the issuer server 108 and the application server 112. Examples of the second network interface 220 may include, but are not limited to, an antenna, a radio frequency transceiver, a wireless transceiver, a Bluetooth transceiver, an ethernet port, a USB port, or any other device configured to transmit and receive data.
In operation, the service application 214 may be downloaded and installed on the user device 106 by the user 120. In other words, one or more instructions that may cause the second processor 222 to execute the installed service application 214 are stored in the second memory 218. Based on the instructions, the second processor 222 may execute the service application 214, and the second I/O interface 226 may render a UI of the executed service application 214 on a display. When the service application 214 is accessed for the first time on the user device 106, the registration or sign-up process may be executed. The UI may prompt the user 120 to input a unique login-id and password (e.g., login credentials), and basic profile information such as a name, a geographical location, valid contact information, or the like for the sign-up. In an embodiment, the geographical location of the user device 106 may be automatically inputted in the UI by the second processor 222 based on a current location captured by the second location sensor 228. However, if the user 120 wants to input an alternate geographical location, an address may be inputted manually using the second I/O interface 226.
The second network interface 220 may be configured to communicate the information inputted through the displayed UI to the application server 112 via the communication network 114. Based on a successful validation, the user 120 is registered with the application server 112 for the use of the service application 214. For subsequent use of the service application 214 on the user device 106, the UI may prompt the user 120 to input the login credentials for sign-in.
The user device 106 may be utilized by the user 120 for performing an optical code-based payment transaction. For example, the user 120 may activate the imaging device 224 to capture an image of an optical code (for example, the optical codes 118a and 118b). In an embodiment, the user device 106 may have installed thereon a mobile application capable of recognizing and decrypting an optical code based on an image captured by the imaging device 224. In another embodiment, the imaging device 224 may be capable of recognizing and decrypting optical codes based on captured images.
In a scenario, where the optical code is successfully processed by the mobile application, the second processor 222 may be configured to generate a transaction request including transaction details, such as a transaction amount, payer and payee identifiers, or the like. When the payment transaction is successfully processed by the issuer server 108 based on the transaction request, the second network interface 220 may be configured to receive a debit notification from the issuer server 108 indicting a debit from the user account. The second I/O interface 226 may be configured to render another UI to present the debit notification.
In an alternate scenario, where the mobile application or the imaging device 224 is unable to process the optical code within a defined time duration of activation of the imaging device 224, the digital image of the optical code captured during the failed scanning-attempt may be retrieved by the service application 214 from the second memory 218 and communicated to the application server 112 via the second network interface 220. When the application server 112 identifies an error that caused the failed scanning-attempt of the optical code based on the received image, the second network interface 220 may be configured to receive a notification (for example, an in-app notification of the service application 214) from the application server 112 that may control the second I/O interface 226 to render another UI for presenting the identified error to the user 120. The UI may further present a recommended action to resolve the identified error. Upon resolving the error, a subsequent scanning attempt may be performed using the user device 106.
It will be apparent to a person of ordinary skill in the art that the scope of the user device 106 is not limited to scanning the optical code for payment transactions. The user device 106 may be utilized by the user 120 to scan optical codes which might not be related to any payment activity. For example, the user device 106 may be utilized by the user 120 to scan an optical code displayed for accessing a digital menu at a restaurant. In such a scenario, instead of the payment identifier, the optical code may include an encrypted website link where the digital menu is hosted. Upon successful processing of the optical code, the UI of the user device 106 may be redirected to the website hosting the digital menu. In another example, the user device 106 may be utilized by the user 120 to scan an optical code displayed for accessing a service menu of a merchant entity. In another example, the user device 106 may be utilized by the user 120 to scan an optical code displayed on a boarding pass, such as a boarding pass of an airplane ticket. In a scenario, when any of the optical code scanning attempts related to non-payment activity fail, the service application 214 in conjunction with the application server 112 may render the UI on the second I/O interface 226 to present the error that caused the failed scanning attempt.
FIG. 3A is a block diagram that illustrates an application server of FIG. 1, in accordance with an exemplary embodiment of the present disclosure. FIG. 3A is described in conjunction with elements of FIG. 1. With reference to FIG. 3A, a block diagram 300A is shown that illustrates the application server 112.
The application server 112 may include third processing circuitry 302, a third memory 304, and a third network interface 306. The third processing circuitry 302 may include one or more processors and hosts (such as a third processor 308, an application host 310, an image processor 312, and a machine learning processor 314) that enable the application server 112 to execute various operations related to processing of optical codes.
The third processor 308 may include suitable logic, circuitry, interface, and/or code, executable by the circuitry, to perform one or more operations required for the operation of the application server 112. For example, the third processor 308 may be configured to receive an image dataset, including a plurality of digital images of various optical codes, from a database, for example, an open source database. Each image of the plurality of images may display a corresponding optical code captured erroneously. The error could be due to an incorrect positioning of a user device while capturing the image of the optical code or due to an error in a display of the optical code. The third processor 308 may be further configured to store the retrieved image dataset in the third memory 304. Examples of the third processor 308 may include, but are not limited to, an ASIC processor, a RISC processor, a CISC processor, an FPGA, a CPU, or a microprocessor.
The application host 310 may include suitable logic, circuitry, interface, and/or code, executable by the circuitry, to perform one or more application hosting operations required for the operation of the application server 112. The application host 310 may be further configured to maintain and store in the third memory 304, a look-up table 316 including a plurality of records, each record having registered contact information and geographical location information of a respective merchant entity. Examples of the application host 310 may include, but are not limited to, an ASIC processor, a RISC processor, a CISC processor, an FPGA, a CPU, or a microprocessor.
The image processor 312 may include suitable logic, circuitry, interface, and/or code, executable by the circuitry, to perform one or more image processing operations required for the operation of the application server 112. For example, the image processor 312 may be configured to pre-process the plurality of images in the received image dataset prior to storing the image dataset in the third memory 304. Examples of the image processor 312 may include, but are not limited to, an ASIC processor, a RISC processor, a CISC processor, an FPGA, a CPU, a microprocessor, or a digital image processor.
The machine learning processor 314 may include suitable logic, circuitry, interface, and/or code, executable by the circuitry, to perform one or more machine learning operations required for the operation of the application server 112. For example, the machine learning processor 314 may be configured to train, validate, and test an image classifier 318 using the image dataset stored in the third memory 304. The image classifier 318 may be a neural network, for example, a convolutional neural network (CNN), a recurrent neural network (RNN), a radial basis functional neural network, or the like. After successful training, the image classifier 318, when inputted with an image of an erroneously captured optical code (payment related or non-payment related), may be configured to generate a classification output that indicates an error associated with the inputted image. Specifically, based on the classification output, the third processor 308 may be configured to classify the image of the optical code in one of the plurality of error categories where each error category corresponds to a different pre-defined error that may have caused a scanning attempt of the optical code to fail. Examples of the machine learning processor 314 may include, but are not limited to, an ASIC processor, a RISC processor, a CISC processor, an FPGA, a CPU, or a microprocessor.
The third memory 304 may include suitable logic, circuitry, and/or interfaces to store various instructions or code which when executed by the third processor 308, the application host 310, the image processor 312, and the machine learning processor 314 causes the third processor 308, the application host 310, the image processor 312, and the machine learning processor 314 to perform their corresponding operations. The third memory 304 may be further configured to store the image dataset, the look-up table 316, and a plurality of recommended actions 320. The plurality of recommended actions 320 may include various actions that when performed may resolve an error in the scanning of the optical code. Each recommended action of the plurality of recommended actions 320 may target a specific pre-defined error. In other words, each recommended action of the plurality of recommended actions 320 may be associated with a specific pre-defined error by way of labels or tags. For example, a recommended action of reprinting the optical code on a white background may target a physical error in the optical code, such as a poor contrast ratio between the background of the optical code and the optical code. In such a scenario, reprinting the optical code action may have a label “poor contrast” assigned thereto. Similarly, a recommended action to tilt the user device 106 by 10-15 degree to make it parallel to the plane of the optical code may target an error in positioning of the user device 106 while capturing the optical code. One recommended action of the plurality of recommended actions 320 may be linked or associated with multiple pre-defined errors, without limiting the scope of the disclosure. Examples of the third memory 304 may include a RAM, a ROM, a PROM, an EPROM, a removable storage drive, an HDD, a flash memory, a solid-state memory, or the like. It will be apparent to a person skilled in the art that the scope of the disclosure is not limited to realizing the third memory 304 in the application server 112, as described herein. In another embodiment, the third memory 304 may be realized in form of a database server or a cloud storage working in conjunction with the application server 112, without departing from the scope of the disclosure.
The third network interface 306 may include suitable logic, circuitry, interfaces, and/or code, executable by the circuitry, to transmit and receive data over the communication network 114 using one or more communication network protocols. The third network interface 306 may be configured to transmit requests and messages to and receive requests and messages from the first and second merchant devices 102a and 102b, and the user device 106. Examples of the third network interface 306 may include, but are not limited to, an antenna, a radio frequency transceiver, a wireless transceiver, a Bluetooth transceiver, an ethernet port, a USB port, or any other device configured to transmit and receive data.
In operation, the third processor 308 may be configured to receive the image dataset, including the plurality of digital images of various optical codes, from the database, for example, an open source database. Each image of the plurality of images may display a corresponding optical code captured erroneously. In other words, each image of the plurality of images is associated with a pre-defined error that may potentially cause a scanning attempt of the corresponding optical code to fail. The pre-defined error may be an incorrect positioning of a user device while capturing the image of the optical code or a display error in the optical code. Examples of various errors may include a dead zone violation, damage or distortion in the optical code, an improper reading position of the user device 106 while scanning the optical code, inverted colors in the optical code, improper size (e.g., smaller or larger than required readable size) of the optical code, poor contrast ratio between a background of the optical code and the optical code, overcrowded optical code, quiet zone violation (e.g., quiet zone too small or absent), blurred optical code, or the like. It will be apparent to a person of ordinary skill in the art that the scope of the disclosure is not limited to the above-mentioned pre-defined errors in optical codes. The image dataset may include images that indicate errors other than the pre-defined errors mentioned above. In addition, the image dataset may include multiple images pertaining to the same error. For example, the image dataset may include multiple images in which the corresponding optical code has a dead zone violation error.
Each image of the plurality of images may have a label or a tag assigned thereto. Here, the tag or the label may indicate a pre-defined error that may have led to a failed scanning-attempt. For example, an image in the plurality of images may have a label “Dead Zone Violation” indicating that the image has a dead zone violation error which led to failed scanning-attempt of a corresponding optical code. The third processor 308 may be configured to determine a plurality of error categories associated with the plurality of images based on the labels assigned to the plurality of images. In an embodiment, a count of error categories in the plurality of error categories may be equal to a count of different labels that are assigned to the plurality of images. For example, the plurality of images may include 1000 images having 400 images labeled as “Dead Zone Violation”, 200 images labeled as “incorrect capturing position”, 150 images labeled as “Poor Contrast Ratio”, and 250 images labeled as “Distorted”. In such a scenario, since there are four different types of labels assigned to 1000 images, the third processor 308 may be configured to determine four error categories such as a dead zone violation category, an incorrect capturing position category, a poor contrast ratio category, and a distorted category. Here, the plurality of error categories may be associated with a plurality of pre-defined errors indicated by the labels or tags. For example, each error category of the plurality of error categories may be associated with a different pre-defined error indicated by a label or a tag. Thus, the third processor 308 may determine the plurality of error categories based on a variety of different labels that were assigned to the plurality of images of the image dataset.
In an embodiment, the third processor 308 may be configured to provide the image dataset to the image processor 312 for pre-processing for data augmentation. For example, the image processor 312 may be configured to resize the plurality of images having different and inconsistent sizes to a default height and width. The image processor 312 may be configured to maintain an aspect ratio of each image while resizing the image, for example, small sized images may be padded with, for example, black, white, or any other colored edge. The image processor 312 may be further configured to pre-process the plurality of images to maintain a uniform color scheme, for example, RGB color scheme, Grayscale, or the like. Other examples of pre-processing performed by the image processor 312 may include contrast stabilization, brightness adjustment, image normalization, noise removal, or the like. The aim of pre-processing is to achieve uniformity in format and quality of the plurality of images.
The third processor 308 may be further configured to shuffle the pre-processed image dataset randomly, and split the pre-processed image dataset into three groups, for example, a training dataset 322, a validation dataset 324, and a testing dataset 326, such that each group has one or more images pertaining to each of the plurality of error categories. In an embodiment, the training dataset 322, the validation dataset 324, and the testing dataset 326 may include same number of images, for example, uniform split of the image dataset by the third processor 308. In another embodiment, the number of images in the training dataset 322, the validation dataset 324, and the testing dataset 326 may not be the same, for example, non-uniform split of the image dataset by the third processor 308. The third processor 308 may be further configured to store the training dataset 322, the validation dataset 324, and the testing dataset 326 in the third memory 304.
The machine learning processor 314 may be configured to receive the training dataset 322 from the third memory 304 and train the image classifier 318 (for example, a CNN, a deep CNN, an RNN model, or the like) using the training dataset 322 for image classification. The machine learning processor 314 may train the image classifier 318 to learn a relationship between the images included in the training dataset 322 and the plurality of error categories. In other words, the image classifier 318 may learn a mapping between different image features of the images included in the training dataset 322 and the plurality of error categories.
Referring now to FIG. 3B, a block diagram that illustrates training of a classifier, in accordance with an embodiment of the present disclosure, is shown. With reference to FIG. 3B, a block diagram 300B that illustrates training of the image classifier 318 by the application server 112 is shown. For the sake of brevity, the image classifier 318 is shown to be a CNN in FIG. 3B. However, in an actual implementation, the image classifier 318 may be any neural network that may execute image classification upon training without deviating from the scope of disclosure.
The image classifier 318 may include one or more convolution and pooling layers 328, one or more fully connected layers 330, and an output layer 332. The one or more convolution and pooling layers 328 and the one or more fully connected layers 330 may represent hidden layers of a neural network that learn weights and biases for various image features when an image is iteratively passed through the one or more convolution and pooling layers 328 and the one or more fully connected layers 330.
The machine learning processor 314 may be configured to input tagged or labelled images included in the training dataset 322 to the one or more convolution and pooling layers 328. For illustration purposes, the images included in the training dataset 322 are shown to be associated with exemplary codes such as “Blur”, “Too Small”, “Too Big”, “Damaged”, and “Improper Position”. The one or more convolution and pooling layers 328 may apply one or more filters to the inputted images of the training dataset 322 to generate feature maps that define image features of the inputted images. A feature map refers to a numerical representation of one or more image features of an image. For example, a convolution layer may apply a corresponding filter to each pixel in an image and generate a feature map that numerically represents the image features of the image. The output of the convolution layer is passed to a subsequent pooling layer which in turn applies a corresponding filter to the feature map. Typically, a pooling layer is used between two convolution layers to reduce a dimension of a feature map generated by a preceding convolution layer. The one or more convolution layers and pooling layers of the one or more convolution and pooling layers 328 may be repeated multiple times to ensure that visual information (e.g., numerical representation of image features) is extracted at multiple levels. Output of the last convolution layer of the one or more convolution and pooling layers 328 may be concatenated and mapped to the one or more fully connected layers 330. The one or more fully connected layers 330 may be configured to map visual information extracted from the one or more convolution and pooling layers 328 to the labels associated with the input images. For example, if the training dataset 322 had ten distinct labels for ten distinct error categories, the one or more fully connected layers 330 ensures that visual information (e.g., a feature map) extracted from the one or more convolution and pooling layers 328 is mapped to the ten labels. After the images of the training dataset 322 are passed through the one or more convolution and pooling layers 328 and the one or more fully connected layers 330 threshold feature maps may be generated and mapped to the labels associated with the input images. Thus, each error category may be numerically represented by a threshold feature map that numerically represents the corresponding error category.
The output layer 332, for example, may be a softmax classifier, that is configured to generate a classification output 334 that includes a confidence score for each label. In other words, the classification output 334 may be a vector including confidence scores, where each confidence score corresponds to a different error category, and a value of the confidence score may indicate a strength of association or correlation between the image and the corresponding error category. In other words, the value of the confidence score may indicate a degree of similarity between the image and the corresponding error category.
The machine learning processor 314 may be further configured to validate the trained image classifier 318 (e.g., the trained classifier 318) using the validation dataset 324. During validation, the image classifier 318 may be back-propagated to minimize cross-entropy between the classification output 334 and ground truth labels. During back-propagation, weights and biases of the threshold feature maps may be updated to increase classification accuracy of the trained image classifier 318. The machine learning processor 314 may be configured to repeat the process of validation, until validation loss stops decreasing.
The machine learning processor 314 may be further configured to test the performance of the trained image classifier 318 using the testing dataset 326. During testing, the machine learning processor 314 may input images of the testing dataset 326 that are already tagged with labels to the trained image classifier 318 and compare classification outputs of the trained image classifier 318 with labels of the inputted images to determine classification accuracy (e.g., performance) of the trained image classifier 318.
Upon successful training, the trained image classifier 318 may capable of generating a classification output for an inputted image such that based on the classification output the third processor 308 may be configured to classify the inputted image into one of the plurality of error categories, thus successfully identifying the pre-defined error that may have led to a failed scanning-attempt of the optical code. The machine learning processor 314 may be configured to store the trained image classifier 318 in the third memory 304.
Referring back to FIG. 3A, the application host 310 may be configured to host the service application 214 that is executable on various devices such as the first and second merchant devices 102a and 102b, and the user device 106. In an example, the application host 310 may be a software as a service (SaaS) hosting solution that allows the service application 214 to be available for access on various user equipment (UE), such as the first and second merchant devices 102a and 102b and the user device 106, globally through the Internet, for example, the communication network 114. The application host 310 may be further configured to control the registration and sign-in of various merchants and users attempting to access the service application 214 on their devices (as described in the foregoing description of FIG. 1). For example, during registration of a plurality of merchant entities, the application host 310 may be configured to receive a merchant identifier (for example, a contact number, an email id, or the like) and geographical location information of each of the plurality of merchant entities from corresponding merchant devices such that each merchant entity may be associated with a corresponding optical code. The application host 310 may be further configured to maintain and store in the third memory 304, the look-up table 316 including the plurality of records, each having the registered contact information and the geographical location information of the respective merchant entity.
The application host 310 may be further configured to monitor various optical code scanning activities on the user device 106 based on the service application 214 running on the user device 106. The application host 310 may be configured to detect a capture of an image at the user device 106. In one embodiment, the application host 310 may detect the capture of the image at the user device 106 based on an indication from the service application 214 running on the user device 106 that the imaging device 224 was activated for image capturing. In another embodiment, the application host 310 may detect the capture of the image at the user device 106 based on notification from the user device 106, for example, an application interface (API) call from the user device 106. The application host 310 may be configured to receive the indication or the notification from the user device 106, when a time duration of a current optical code scanning session on the user device 106 exceeds a threshold value. The threshold value may be a configurable parameter set by the application server 112 or the user 120. For example, the threshold value may be set to one second, two second, or the like.
Based on the notification, the application host 310 may be configured to control the service application 214 running on the user device 106 to receive a first image of the incorrectly captured optical code, for example, the optical code 118a. Thus, the application host 310 may receive the first image of the incorrectly captured optical code from the user device 106 over the communication network 114 based on the detection of the capture of the image at the user device 106.
The application host 310 may be configured to provide the received first image to the third processor 308, which in turn may apply the trained image classifier 318 to the received first image. In other words, the third processor 308 may provide the received first image as an input to the trained image classifier 318. In an embodiment, prior to providing the received first image to the trained image classifier 318, the image processor 312 may pre-process the received first image for image augmentation.
When the trained image classifier 318 is applied to the received first image, the received first image is passed iteratively through the one or more convolution and pooling layers 328 and the one or more fully connected layers 330 of the trained image classifier 318. The one or more convolution and pooling layers 328 may then apply the one or more filters to a plurality of pixels of the first image and generate a feature map that numerically represents one or more image features of the first image. The one or more fully connected layers 330 may then compare the generated feature map with the threshold feature maps of the plurality of error categories. In other words, the trained image classifier 318 compares the first image with the plurality of error categories based on the generated feature map of the first image. Based on the comparison of the first image with the plurality of error categories, the output layer 332 may generate a classification output (e.g., the classification output 334) for the first image. The classification output generated by the trained image classifier 318 may include a plurality of confidence scores, each representing a degree of similarity between the generated feature map and the threshold feature map of the corresponding error category of the plurality of error categories. For example, if there are ten error categories each having a corresponding threshold feature map learnt during training of the image classifier 318, the classification output may include ten confidence scores (e.g., numerical values), each representing a degree of similarity between the inputted first image and a corresponding error category. In other words, a value of confidence score for a first error category is higher than a value of a confidence score for a second error category when the generated feature map has a higher degree of similarity with the threshold feature map of the first error category as compared to the second error category.
The third processor 308 may be configured to classify the first image into one of the plurality of error categories based on the classification output. For example, the third processor 308 may be configured to classify the first image into that error category for which the confidence score is highest. Thus, if the confidence score for the dead zone violation category is highest among the confidence scores, the third processor 308 may classify the first image into the dead zone error category.
Based on the classification of the first image into one of the plurality of error categories, the application host 310 may be configured to control the service application 214 running on the user device 106 to render a UI on the user device 106 to present the pre-defined error associated with the error category into which the first image is classified. Thus, when the first image is classified into the dead zone category, the application host 310 may control the service application 214 running on the user device 106 to render the UI on the user device 106 to present the dead zone violation error to the user 120. Upon resolving the error, the optical code may be processed successfully in a subsequent scanning attempt.
In an embodiment, the application host 310 may be configured to receive a real time location information of the user device 106 in combination with the first image. The real time location information may be received by the service application 214 from the second location sensor 228 and provided to the application host 310. Based on the received location information, the third processor 308 may be configured to look up in the look-up table 316 for selecting, from the plurality of merchant entities registered with the application server 112, a merchant entity that has the same registered geographical location as the real time location information of the user device 106. Upon selection of the merchant entity, the application host 310 may be configured to render a UI on a merchant device (e.g., the first merchant device 102a) of the selected merchant entity to present the identified error to the merchant entity. In another embodiment, the application host 310 may utilize the registered contact information of the selected merchant entity to contact the merchant entity for presenting the identified error to the merchant entity. For example, the application host 310 may communicate a message on the registered contact number or an email on a registered email id indicating the identified error to the merchant entity.
In an embodiment, the third processor 308 may be further configured to select from the plurality of recommended actions 320, a most appropriate recommended action to resolve the error associated with the classified error category of the first image. For example, when the classified error category is dead zone violation category, the third processor 308 may select from the plurality of recommended actions 320, a most appropriate recommended action to resolve the dead zone violation error. The recommended action is presented to the user 120 and/or the selected merchant entity. In a scenario, where the error is associated with the capturing of the image of the optical code, the recommended action may be implemented by the user 120. In another scenario, where the error is associated with the display of the optical code, the recommended action may be implemented by the merchant entity associated with the optical code. For example, the merchant entity may reprint the optical code or re-display the optical on a corresponding merchant device.
In an embodiment, after the pre-defined error is presented on the user device 106, the third processor 308 may be further configured to monitor a subsequent attempt of optical code scanning at the user device 106. In a scenario, where the pre-defined error still persists in the subsequent scanning attempt after the implementation of the recommended action, the third processor 308 may be configured to recommend an alternate action to resolve the pre-defined error. In addition, the third processor 308 may be configured to reduce a weight associated with the previous recommended action for the pre-defined error. In other words, the application server 112 implements active machine learning to increase accuracy in error identification and action recommendation.
FIG. 4 is a process flow diagram that illustrates registration of a merchant entity with an application server of FIG. 1, in accordance with an embodiment of the present disclosure. With reference to FIG. 4, a process flow diagram 400 is shown to describe the registration of the merchant entity of the first merchant device 102a with the application server 112.
The first merchant device 102a may be utilized by the corresponding merchant entity or the first operator 116a to access the service application 214 for the first time (as shown by arrow 402). The application server 112 may be configured to communicate a sign-up request to the first merchant device 102a via the UI of the service application 214 running on the first merchant device 102a (as shown by arrow 404). The merchant entity inputs the profile details, including a merchant identifier (such as contact information) and geographical location information. The first merchant device 102a then communicates the profile details to the application server 112 (as shown by arrow 406). The application server 112 validates the profile details and upon successful validation registers the merchant entity (as shown by arrow 408). The application server 112 further creates a new record in the look-up table 316 for the newly registered merchant entity, where the new record stores the merchant identifier and the geographical location information of the merchant entity (as shown by arrow 410). The updated look-up table 316 is stored in the third memory 304 of the application server 112. The application server 112 further communicates a registration successful message to the first merchant device 102a (as shown by arrow 412). Similarly, the application server 112 registers other merchant entities that want access to the service application 214.
FIG. 5A is a diagram that illustrates an exemplary scenario for processing a machine-readable optical code by an application server of FIG. 1, in accordance with an exemplary embodiment of the present disclosure. With reference to FIG. 5A, an exemplary scenario 500A is shown that describes various operations performed by the application server 112 to assist users and merchants in optical code scanning. FIG. 5A has been described in conjunction with elements from FIG. 1, 2A, 2B, 3A, and 3B.
The optical code 118a (e.g., a QR code 118a) is displayed at the first merchant premise (as shown by dotted box 502). The user 120 visiting the first merchant premise may use the user device 106 to scan the QR code 118a for payment or non-payment related activities. However, the scanning attempt may fail and an image IM1 may be captured by the imaging device 224 (of FIG. 2B) of the user device 106 during the failed QR code scanning attempt (as shown by dotted box 504). The third processing circuitry 302 of the application server 112 may detect the activation of the imaging device 224 on the user device 106 by way of the service application 214 running on the user device 106 and may detect a failed or stuck scanning session based on the activity of the imaging device 224. Thus, the third processing circuitry 302 may control the service application 214 on the user device 106 to receive the image IM1 captured during the failed scanning-attempt and transmit the image IM1 to the application server 112 (as shown by arrow 506). In an embodiment, the service application 214 may receive the image IM1 from the second memory 218. The third processing circuitry 302 may further control the service application 214 to transmit a real-time geographical location of the user device 106 along with the image IM1 (as shown by arrow 506). The third processing circuitry 302 may apply the trained image classifier 318 to the image IM1 (as shown by dotted box 508) and classify the image IM1 into a first error category of the plurality of error categories based on the classification output of the trained image classifier 318 for the image IM1 (as shown by dotted box 510). The first error category may be associated with a pre-defined error in at least one of a display of the QR code 118a and a capturing of the image IM1. For the sake of illustration, it is assumed that the first error category into which the image IM1¬ is classified corresponds to the pre-defined error “distortion”, which is an error in a display of the QR code 118a. The third processing circuitry 302 may further select, from the plurality of merchant entities registered with the application server 112, the merchant entity associated with the QR code 118a based on a match between the real time geographical location information of the user device 106 and one of the location information stored in the look-up table 316 in the third memory 304 (as shown by dotted box 512).
The third processing circuitry 302 may further select, from the plurality of recommended actions 320 stored in the third memory 304, one or more recommended actions to resolve the pre-defined error “distortion” associated with the first error category (as shown by dotted box 513). In an embodiment, the recommended action selected for the user 120 may be different from the recommended action selected for the merchant entity. For example, in a scenario, where the error is in the display of the QR code 118a, the recommended action selected for the user 120 may be to use another QR code; however, the recommended action selected for the concerned merchant entity may be to reprint the QR code 118a for seamless interaction.
The third processing circuitry 302 may communicate the identified error and the selected recommended action to the user device 106 and the first merchant device 102a of the selected merchant entity (as shown by arrows 514a and 514b). By communicating the identified error and the selected recommended action to the user device 106, the third processing circuitry 302 renders a first UI 516 on the user device 106 to present the pre-defined error associated with the first error category and the recommended action. By communicating the identified error and the selected recommended action to the first merchant device 102a, the third processing circuitry 302 renders a second UI 518 on the first merchant device 102a to present the pre-defined error associated with the first error category and the recommended action.
In an embodiment, the merchant entity may not be registered with the application server 112. In such a scenario, the third processing circuitry 302 may not communicate the the identified error and the selected recommended action to the first merchant device 102a.
FIG. 5B is a diagram that illustrates an exemplary scenario for processing an optical code-based payment transaction, in accordance with an exemplary embodiment of the present disclosure. With reference to FIG. 5B, an exemplary scenario 500B is shown that describes various operations performed by the issuer server 108, the payment network server 110, and the acquirer server 104 for processing a transaction. FIG. 5B has been described in conjunction with elements from FIG. 1, 2A, 2B, 3A, 3B, and 5A.
Based on the recommended action as described in FIG. 5A, the merchant entity may reprint and display a new QR code 520 (as shown by dotted box 522). The user 120 may again use the user device 106 to scan the QR code 520 for a payment related activity. As the error in the QR code is now resolved, the QR scanning attempt at the user device 106 is successful (as shown by dotted box 524). Based on the successful scanning of the QR code 520, a transaction request is generated at the user device 106 and communicated to the issuer server 108 (as shown by arrow 526). The issuer server 108 may authorize the transaction and debit the user account, when the user account has sufficient funds to cover a transaction amount of the transaction (as shown by dotted boxes 528 and 530a). The issuer server 108 may be further configured to communicate a debit notification to the user device 106 to notify the transaction status (as shown by arrow 530b). The issuer server 108 may further communicate the transaction request to the payment network server 110 (as shown by arrow 532). Based on the transaction request, the payment network server 110 may identify a respective acquirer server 104 and communicate the transaction request to the acquirer server (as shown by arrow 534). The acquirer server 104 may receive the transaction request and credit the merchant account with the transaction amount (as shown by dotted box 536a). The acquirer server 104 may further communicate a credit notification to the first merchant device 102a and the payment network server 110 (as shown by arrows 536b and 538). Upon receiving the credit notification from the acquirer server 104, the payment network server 110 may communicate a notification to the issuer server 108 regrading a successful completion of the transaction (as shown by arrow 540).
In one embodiment, the application server 112 may be implemented by the payment network server 110. In such a scenario, the payment network server 110 may execute the operations of the application server 112 as described in FIGS. 1, 3A-3B, 4, and 5A along with payment processing operations.
FIG. 6 is a block diagram that illustrates a user device of FIG. 1, in accordance with another embodiment of the present disclosure. With reference to FIG. 6, the user device 106 is shown. The user device 106 shown in FIG. 6 is similar to the user device 106 shown in FIG. 2B, except the user device 106 of FIG. 6 has stored in the second memory 218, a local version of the trained image classifier 318 received from the application server 112.
In such an embodiment, the service application 214 running on the user device 106 may be configured to receive the image captured by the imaging device 224 during a failed scanning-attempt of an optical code from the second memory 218. As the trained image classifier 318 is stored in the second memory 218, the second processor 222 under the control of the service application 214 may apply the trained image classifier 318 to the image and classify the image into one of the plurality of error categories based on the classification output of the trained image classifier 318. Generation of the classification output has been described din the foregoing description of FIGS. 3A and 3B. Upon successful classification, the second processor 222 may render a UI on the second I/O interface 226 to present the error that led to the failure of the scanning attempt.
In an embodiment, the service application 214 and the trained image classifier 318 may be implemented in QR code or bar code scanning kiosks to assist users in identifying reasons for failed QR code or bar code scanning attempts. Such implementation may improve a QR code scanning experience of users. For example, a ticket printing kiosk at an airport may have the trained image classifier 318 stored thereon, such that when the kiosk detects a failed scanning attempt of a barcode or a QR code, the kiosk may utilize the trained image classifier 318 to identify an error that led to the failed scanning attempt and present the identified error to a user using the kiosk for scanning.
FIG. 7 is a block diagram that illustrates a system architecture of a computer system 700, in accordance with an exemplary embodiment of the present disclosure. An embodiment of disclosure, or portions thereof, may be implemented as computer readable code on the computer system 700. In one example, the acquirer server 104, the issuer server 108, and the payment network server 110 may be implemented as the computer system 700. Hardware, software, or any combination thereof may embody modules and components used to implement methods of FIGS. 8A and 8B, and 9. The computer system 700 may include a processor 702, a communication infrastructure 704, a main memory 706, a secondary memory 708, an I/O interface 710, and a communication interface 712.
The processor 702 may be a special-purpose or a general-purpose processing device. The processor 702 may be a single processor, multiple processors, or combinations thereof. Further, the processor 702 may be connected to the communication infrastructure 704, such as a bus, message queue, multi-core message-passing scheme, and the like. Examples of the main memory 706 may include a RAM, a ROM, and the like. The secondary memory 708 may include an HDD or a removable storage drive, such as a floppy disk drive, a magnetic tape drive, a compact disc, an optical disk drive, a flash memory, and the like.
The I/O interface 710 includes various input and output devices that are configured to communicate with the processor 702. Examples of the input devices may include a keyboard, a mouse, a joystick, a touchscreen, a microphone, and the like. Examples of the output devices may include a display screen, a speaker, headphones, and the like. The communication interface 712 may be configured to allow data to be transferred between the computer system 700 and various devices that are communicatively coupled to the computer system 700. Examples of the communication interface 712 may include a modem, a network interface, i.e., an Ethernet card, a communication port, and the like. Data transferred via the communication interface 712 may correspond to signals, such as electronic, electromagnetic, optical, or other signals as will be apparent to a person skilled in the art.
FIG. 8 is a high-level flow chart 800 that illustrates a method for training a classifier (e.g., the image classifier 318) to facilitate processing of machine-readable-optical codes, in accordance with an exemplary embodiment of the present disclosure.
At step 802, a training dataset, including a plurality of images of a plurality of machine-readable optical codes is obtained such that each image in the training dataset is associated with an error category of a plurality of error categories. Each error category of the plurality of error categories corresponds to a pre-defined error in at least one of the display of a machine-readable optical code and a capturing of an image of the machine-readable optical code. For example, the third processing circuitry 302 may be configured to obtain the training data from a database.
At step 804, a classifier is trained using the training dataset to learn a relationship between the plurality of images of the plurality of machine-readable optical codes and the plurality of error categories. For example, the third processing circuitry 302 may be configured to train the image classifier 318 using the training dataset 322 to relationship between the plurality of images of the plurality of machine-readable optical codes and the plurality of error categories. Upon successful training, the trained image classifier 318 (e.g., the trained classifier) when applied to a first image (e.g., the image IM1) of a first machine-readable optical code, generates a classification output (e.g., the classification output 334) for the first image. The first image is classified into a first error category of the plurality of error categories based on the classification output of the trained image classifier 318 for the first image.
FIGS. 9A and 9B, collectively, represent a high-level flow chart 900 that illustrates a method for processing machine-readable-optical codes, in accordance with an exemplary embodiment of the present disclosure.
At step 902, an image of a machine-readable optical code is received. For example, the third processing circuitry 302 may be configured to receive the image of the machine-readable optical code from the user device 106. At step 904, a trained classifier is applied to the received image. For example, the third processing circuitry 302 may be configured to apply the trained image classifier 318 to the received image. The neural network based application of the trained image classifier 318 to the received image is described in conjunction with FIG. 9B.
Referring now to FIG. 9B, at step 904a, a feature map is generated based on the image. The trained image classifier 318 may be configured to generate the feature map based on the image. The feature map is a numerical representation of one or more image features of the image. At step 904b, the received image is compared with a plurality of error categories. The trained image classifier 318 may be configured to compare the received image with the plurality of error categories. The plurality of error categories may be associated with a plurality of pre-defined errors in at least one of a display of the machine-readable optical code and a capturing of the image of the machine-readable optical code. The image is compared with the plurality of error categories based on the generated feature map.
At step 904c, a classification output is generated based on the comparison of the image with the plurality of error categories. The trained image classifier 318 may be configured to generate the classification output based on the comparison of the image with the plurality of error categories. The classification output may include a plurality of confidence scores for the plurality of error categories, respectively. A confidence score of the plurality of confidence scores may indicate a degree of similarity between the image and a corresponding error category of the plurality of error categories. The process may then return to step 906.
Referring now to FIG. 9A, at step 906, the image is classified into a first error category of the plurality of error categories based on the classification output. For example, the third processing circuitry 302 may be configured to classify the image into the first error category of the plurality of error categories based on the classification output of the trained image classifier 318.
At step 908, a first UI is rendered on a user device to present the pre-defined error associated with the first error category. Upon resolving the pre-defined error, the machine-readable optical code may be processed successfully. For example, the third processing circuitry 302 may be configured to render the first UI (for example, the UI 516 of FIG. 5A) on the user device 106 to present the pre-defined error associated with the first error category.
FIG. 10 is a high-level flow chart 1000 that illustrates a method creating a look-up table for registered merchant entities, in accordance with an exemplary embodiment of the present disclosure.
At step 1002, merchant identifier and location information of each of a plurality of merchant entities is received from corresponding plurality of merchant devices. The plurality of merchant entities are associated with a plurality of machine-readable optical codes. For example, the third processing circuitry 302 may be configured to receive the merchant identifier and location information of each of the plurality of merchant entities from corresponding plurality of merchant devices (e.g., the first and second merchant devices 102a and 102b.
At step 1004, the merchant identifier and the location information of each of the plurality of merchant entities are stored in a memory. For example, the third processing circuitry 302 may be configured to store in the third memory 304 the merchant identifier and the location information of each of the plurality of merchant entities.
FIGS. 11A and 11B, collectively, represent a flow chart 1100 that illustrates a method for processing machine-readable-optical codes, in accordance with an exemplary embodiment of the present disclosure. For the sake of brevity, the flow chart 1100 is explained with respect to the application server 112. In another embodiment, operations of the flow chart 1100 may be performed the second processing circuitry 216 of the user device 106 when the trained image classifier 318 is locally stored in the second memory 218 of the user device 106, without deviating from the scope of the disclosure.
With reference to FIG. 11A, at step 1102, a capture of an image at a user device is detected. For example, the third processing circuitry 302 may be configured to detect the capture of the image at the user device 106 based on at least one of an activation of the imaging device 224 at the user device 106 or a notification received from the user device 106. At step 1104, the image of a machine-readable optical code is received. For example, the third processing circuitry 302 may be configured to receive the image of the machine-readable optical code stored in the second memory 218 of the user device 106 where the received image is associated with a failed scanning-attempt of the machine-readable optical code. At step 1106, a real time location of the user device is also received in combination with the image. For example, the third processing circuitry 302 may be configured to receive the real time location of the user device 106 in combination with the image.
At step 1108, a trained classifier is applied to the received image. For example, the third processing circuitry 302 may be configured to apply the trained image classifier 318 to the received image. The operation of the trained image classifier 318 on the received image is described in the foregoing description of FIGS. 3A, 3B, and 9B. At step 1110, the image is classified into a first error category of a plurality of error categories based on a classification output of the trained classifier 318 for the image. Each of the plurality of error categories is associated with a pre-defined error in at least one of a display of a machine-readable optical code and a capturing of the image of the machine-readable optical code. For example, the third processing circuitry 302 may be configured to classify the image into the first error category of the plurality of error categories based on the classification output of the trained image classifier 318. The classification output may include a plurality of confidence scores for the plurality of error categories, respectively. A confidence score of the plurality of confidence scores may indicate a degree of similarity between the image and a corresponding error category of the plurality of error categories. At step 1112, from a plurality of recommended actions, a recommended action to resolve the pre-defined error associated with the first error category is selected. For example, the third processing circuitry 302 may be configured to select at least one of the plurality of recommended actions 320 to resolve the pre-defined error associated with the first error category.
With reference to FIG. 11B, at step 1114, from the plurality of merchant entities, a merchant entity associated with the machine-readable optical code is selected based on the received real time location information and the location information stored in the memory. For example, third processing circuitry 302 may be configured to select, from the plurality of merchant entities registered with the application server 112, a merchant entity associated with the machine-readable optical code based on the received real time location information and the location information stored in the third memory 304 in the form of the look-up table 316. At step 1116, a first UI is rendered on a user device to present the pre-defined error associated with the first error category. Upon resolving the pre-defined error, the machine-readable optical code may be processed successfully. For example, the third processing circuitry 302 may be configured to render the first UI (for example, the UI 516 of FIG. 5A) on the user device 106 to present the pre-defined error associated with the first error category. At step 1118, the first UI is rendered to further present the selected recommended action. For example, the third processing circuitry 302 may be configured to render the first UI (for example, the UI 516 of FIG. 5A) to further present the selected recommended action to resolve the pre-defined error.
At step 1120, a second UI is rendered on a merchant device of the selected merchant entity to present the pre-defined error associated with the first error category. For example, the third processing circuitry 302 may be configured to render the second UI (for example, the UI 518 of FIG. 5A) on the first merchant device 102a to present the pre-defined error associated with the first error category. At step 1122, the second UI is rendered to further present the selected recommended action. For example, the third processing circuitry 302 may be configured to render the second UI (for example, the UI 518 of FIG. 5A) to further present the selected recommended action to resolve the pre-defined error and then the process ends.
The method and system (for example, the application server 112) of the present disclosure offers various advantages. For example, the application server 112 is capable of not only identifying a pre-defined error that may have caused a scanning of a machine-readable optical code to fail but also recommend a suitable action to resolve the pre-defined error. The application server 112 is capable of identifying any type of error, for example, an error in capturing of an image of the machine-readable optical code or an error in a display of the machine-readable optical code, thereby offering a fool-proof solution. Technical improvements offered by the application server 112 makes the scanning of optical codes a seamless activity. For example, now the users and the merchants may exactly know a reason that caused a failed scanning attempt, which in turn improves transaction experience of the users and the merchants for optical code-based transactions. The service offered by the application server 112 may be easily accessible to both users and merchants by subscribing or requesting access to the service application 214. In addition, the trained image classifier 318 may be implemented locally on user devices to solve optical code scanning errors occurring while the user devices are in limited network connectivity areas.
Techniques consistent with the present disclosure provide, among other features, systems, and methods for facilitating processing of machine-readable codes. While various exemplary embodiments of the disclosed system and method have been described above it should be understood that they have been presented for purposes of example only, not limitations. It is not exhaustive and does not limit the disclosure to the precise form disclosed. Modifications and variations are possible in light of the above teachings or may be acquired from practicing the disclosure, without departing from the breadth or scope.
In the claims, the words ‘comprising’, ‘including’, and ‘having’ does not exclude the presence of other elements or steps than those listed in a claim. The terms “a” or “an,” as used herein, are defined as one or more than one. Unless stated otherwise, terms such as “first” and “second” are used to arbitrarily distinguish between the elements such terms describe. Thus, these terms are not necessarily intended to indicate temporal or other prioritization of such elements. The fact that certain measures are recited in mutually different claims does not indicate that a combination of these measures cannot be used to advantage.
While various embodiments of the present disclosure have been illustrated and described, it will be clear that the present disclosure is not limited to these embodiments only. Numerous modifications, changes, variations, substitutions, and equivalents will be apparent to those skilled in the art, without departing from the spirit and scope of the present disclosure, as described in the claims.