Claims: 1. A method for detecting one or more operational issues in visual merchandising, the method comprising:
capturing first set of images of the one or more operational issues in a commercial establishment in real time using a plurality of installed media acquisition modules;
extracting one or more features from the first set of images using a digital image processing module;
detecting the one or more operational issues based on the extracted one or more features from the first set of images using a digital image processing module;
classifying the detected one or more operational issues into dynamic operational issues and non-dynamic operational issues based on frequency of occurrence of the one or more operational issues using a media content analysis module; and
warranting the application of one or more corrective actions on each of the detected one or more operational issues using a corrective action warranting module, wherein the dynamic operational issues warrant a real time corrective action and the non-dynamic operational issues warrant corrective actions at a predefined periodic interval.

2. The method claimed in claim 1, further comprising:
monitoring the one or more corrective actions warranted for each of the detected one or more operational issues using a monitoring module, wherein monitoring comprises:
capturing a second set of images comprising at least one change occurred as a result of the applied one or more corrective actions upon each of the one or more operational issues;
comparing the captured second set of images to the corresponding first set of images stored in a database to determine one or more changes occurred as a result of the applied one or more corrective actions using the media content analysis module; and
analyzing the resolution of the one or more operational issues based on the determined one or more changes using video content analysis module.

3. The method claimed in claim 2, wherein the at least one changes occurred as a result of the applied one or more corrective actions comprises at least one of:
number of people visiting locations of the commercial establishment;
illumination level at locations of the commercial establishment;
depletion rate of items of the shelves and emptiness of shelf;
condition of the artefacts of the establishment; and
usable space of the commercial establishment.

4. The method claimed in claim 1, wherein the one or more operational issues include at least one of a poor lighting, empty shelf space, empty establishment area, wrong placement, outdated promotion or low footfall.

5. The method claimed in claim 1, wherein the non-dynamic operational issues can resolve a dynamic operational issue by correlating one or more non-dynamic operational issues and one or more dynamic issues in a cause effect relationship.

6. A system for detecting a one or more operational issues in visual merchandising, the system comprising:
a processor,
a memory communicatively coupled to the processor and the memory contains instructions that are readable by the processor;
a plurality of media acquisition modules configured to capture first set of images of the one or more operational issues in a commercial establishment in real time;
a digital image processing module configured to extract one or more features from the captured first set of images and to detect the one or more operational issues based on the extracted one or more features from the one or more images;
a media content analysis module configured to classify the detected one or more operational issues into dynamic operational issues and non-dynamic operational issues based on frequency of occurrence of the one or more operational issues; and
a corrective action warranting module configured to warrant the application of one or more corrective actions on each of the detected one or more operational issues, wherein the dynamic operational issues warrant a real time corrective action and the non-dynamic operational issues warrant corrective actions at a predefined periodic interval.

7. The system claimed in claim 6, further comprising:
a monitoring module configured to monitor the one or more corrective actions warranted foe each of the detected one or more operational issues, wherein the monitoring comprises:
capturing a second set of images comprising changes occurred as a result of the applied one or more corrective actions upon each of the one or more operational issues;
comparing the captured second set of images to the corresponding first set of images stored in a database to determine one or more changes occurred as a result of the applied one or more corrective actions using the media content analysis module; and
analyzing the resolution of the one or more operational issues based on the determined one or more changes using video content analysis module.

8. The system claimed in claim 7, wherein the monitoring module is configured to count the number of people visiting one or more locations of the commercial establishment on a predefined periodic interval basis;

9. The system claimed in claim 7, wherein the monitoring module is configured to measure the illumination level at one or more locations of the commercial establishment;

10. The system claimed in claim 7, wherein the monitoring module is configured to monitor one or more shelves for depletion rate of items of the shelves and emptiness of shelf.

11. The system claimed in claim 7, wherein the monitoring module is configured to detect one or more artefacts of the establishment and to report the condition of the one or more artefacts;

12. The system claimed in claim 7, wherein the monitoring module is configured to track usable space of the commercial establishment on a predefined periodic interval basis.
, Description:FORM 2

THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENT RULES, 2003

COMPLETE SPECIFICATION
(See Section 10 and Rule 13)

Title of invention:
SYSTEM AND METHOD FOR DETECTION AND MONITORING OF OPERATIONAL ISSUES IN VISUAL MERCHANDISING

Applicant:
Tata Consultancy Services Limited
A company Incorporated in India under the Companies Act, 1956
Having address:
Nirmal Building, 9th floor,
Nariman point, Mumbai 400021,
Maharashtra, India

The following specification particularly describes the embodiments and the manner in which it is to be performed.
TECHNICAL FIELD
[0001] The embodiments herein generally relate to image processing for operational issues in retail marketing, and more particularly, providing a holistic solution based on image processing for operational issues in visual merchandising.

BACKGROUND
[0002] Visual Merchandising (VM) is a key element in Retail Marketing for attracting and inducing potential customers. Usually most stores use innovative VM techniques to plan how the various items are to be placed while optimizing the store space and shelf space available. These techniques make effective use of colors (contrast between products and background, between adjacent products), Categories and sub-categories (Groceries, Furniture, Clothes, etc.), Signage’s (to display brand names, promotional schemes, point to particular sections, etc.), Lighting (to highlight certain products, to improve visibility, etc.), Fixtures like Spinners, Hooks (to optimize display of products), and so on.

[0003] But it is a well-known fact that most orderly scheme of things can get disordered and messy over time and it is extremely important to track and monitor the store arrangements. Besides, the retail industry is very dynamic with constant inflow of new products and brands, changing customer preferences, etc. what this implies is that the actual situation in the store is very different from what was intended or assumed at the time of planning.

[0004] The issues associated with visual merchandising are poor lighting, empty shelves, wrong product placement, outdated promotional material, unused areas, low footfall areas, cluttered areas, improper placing of fixtures, etc. These impact significantly on retail store performance but are not addressed in a holistic, structured manner with well-defined processes and technology.

[0005] These issues continue to be handled in a manual, unstructured manner and viewed more as administrative issues rather than a marketing opportunity. While most retailers are anxious to have higher footfalls, these issues, if unattended can cause greater harm with increasing footfall and hence their resolution is more crucial than increasing footfalls.

SUMMARY
[0006] The following presents a simplified summary of some embodiments of the disclosure in order to provide a basic understanding of the embodiments. This summary is not an extensive overview of the embodiments. It is not intended to identify key/critical elements of the embodiments or to delineate the scope of the embodiments. Its sole purpose is to present some embodiments in a simplified form as a prelude to the more detailed description that is presented below.

[0007] In view of the foregoing, an embodiment herein provides a system and method for identifying and monitoring operational issues visual merchandising. It provides a holistic solution based on image processing for detection, identification, classification and monitoring of operation issues in visual merchandising (VM). It is an automated process and also aimed at to eliminate both errors and subjectivity in the entire process of detection, identification, classification and monitoring of operational issues related to visual merchandising.

[0008] In one aspect, a method for detecting one or more operational issues in visual merchandising, the method comprising capturing first set of images of the one or more operational issues in a commercial establishment in real time using a plurality of installed media acquisition modules, extracting one or more features from the first set of images using a digital image processing module, detecting the one or more operational issues based on the extracted one or more features from the one or more images using a digital image processing module, classifying the detected one or more operational issues into dynamic operational issues and non-dynamic operational issues based on frequency of occurrence of the one or more operational issues using a media content analysis module and warranting the application of one or more corrective actions on each of the detected one or more operational issues using a corrective action warranting module, wherein the dynamic operational issues warrant a real time corrective action and the non-dynamic operational issues warrant corrective actions at a predefined periodic interval.

[0009] In another aspect, a method for monitoring the one or more corrective actions warranted for each of the detected one or more operational issues using a monitoring module. The monitoring comprises capturing a second set of images comprising at least one change occurred as a result of the applied one or more corrective actions upon each of the one or more operational issues, comparing the captured second set of images to the corresponding first set of images stored in a database to determine one or more changes occurred as a result of the applied one or more corrective actions using the media content analysis module, and analyzing the resolution of the one or more operational issues based on the determined one or more changes using video content analysis module.

[0010] In another aspect, a system for detecting a one or more operational issues in visual merchandising, the system comprising a processor, a memory communicatively coupled to the processor and the memory contains instructions that are readable by the processor, a plurality of media acquisition modules configured to capture first set of images of the one or more operational issues in a commercial establishment in real time, a digital image processing module configured to extract one or more features from the captured first set of images and to detect the one or more operational issues based on the extracted one or more features from the one or more images, a media content analysis module configured to classify the detected one or more operational issues into dynamic operational issues and non-dynamic operational issues based on frequency of occurrence of the one or more operational issues and a corrective action warranting module configured to warrant the application of one or more corrective actions on each of the detected one or more operational issues, wherein the dynamic operational issues warrant a real time corrective action and the non-dynamic operational issues warrant corrective actions at a predefined periodic interval. The system further comprising a monitoring module, wherein monitoring module configured to monitor the one or more corrective actions warranted for each of the detected one or more operational issues.

[0011] It should be appreciated by those skilled in the art that any block diagram herein represent conceptual views of illustrative systems embodying the principles of the present subject matter. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, pseudo code, and the like represent various processes which may be substantially represented in computer readable medium and so executed by a computing device or processor, whether or not such computing device or processor is explicitly shown.

BRIEF DESCRIPTION OF THE DRAWINGS
[0012] The embodiments herein will be better understood from the following detailed description with reference to the drawings, in which:

[0013] Figure. 1 illustrates a system for detecting one or more operational issues in visual merchandizing and monitoring one or more corrective actions according to an embodiment of the present disclosure;

[0014] Figure. 2 illustrates a flow diagram for detecting one or more operational issues and monitoring one or more corrective actions warranted for each of the detected operational issues in visual merchandizing according to an embodiment of the present disclosure; and

[0015] Figure. 3 illustrates a method for detecting the one or more operational issues in visual merchandising and monitoring one or more corrective actions according to an embodiment of the present disclosure.

DETAILED DESCRIPTION OF EMBODIMENTS
[0016] The embodiments herein and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein may be practiced and to further enable those of skill in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.

[0017] Referring now to fig. 1, a system 100 for detecting and monitoring one or more operational issues in visual merchandizing. The system 100 comprising a processor 102, a memory 104 communicatively coupled to the processor 102 and the memory 104 contains instructions that are readable by the processor. Further, the system 100 comprising a plurality of media acquisition modules 106, a digital image processing module 108, a media content analysis module 110, a corrective action warranting module 112 and a monitoring module 114.

[0018] In the preferred embodiment, the media acquisition module 106 is configured to capture first set of images of the one or more operational issues in a commercial establishment in real time. It would be appreciated that the media acquisition module 106 includes closed circuit television (CCTV) systems or any other video surveillance systems, installed in such a manner to cover a predefined space. The media acquisition module 106 capture the images of various sections of the establishment in the real time. The captured images are stored at a central server and displayed on monitors given to supervisory staff, VM planners, Sales & Marketing and senior management. The one or more operational issues include at least one of a poor lighting, empty shelf space, empty establishment space, wrong placement, outdated promotion or low footfall.

[0019] In the preferred embodiment, the digital image processing module 108 extract one or more features from the captured first set of images and to detect the one or more operational issues based on the extracted one or more features from the one or more images. Firstly, the digital image processing module 108 classify the operational issues into the sub-categories based on the training set of data containing predefined observations or instances and whose category membership is also known. Secondly, the digital image processing module 108 to extract one or more features of the one or more operation issues as classified into their sub-categories. In addition to this, the digital image processing module 108 invokes the optical character recognition to recognize one or more promotional material for validity and correctness.

[0020] In the preferred embodiment, the media content analysis module 110 is configured to further classify the detected one or more operational issues into dynamic operational issues and non-dynamic operational issues based on frequency of occurrence of the one or more operational issues. The media content analysis module 110 can recognize one or more changes occur in the environment of the commercial environment. It can identify and compare objects in the database using size, speed, and sometimes color. In addition to this, the media content analysis module 110 can also issue an alarm if the objects of the establishment has moved from original location. For example, if a painting is missing from the wall of the establishment, the video content analysis module 110 will identify the missing painting based on field view and will also issue an alarm of missing painting.

[0021] In the preferred embodiment, the corrective action warranting module 112 configured to warrant the application of one or more corrective actions on each of the detected one or more operational issues, wherein the dynamic operational issues warrant a real time corrective action and the non-dynamic operational issues warrant corrective actions at a predefined periodic interval. It would also be appreciated that a dynamic operational issue can be resolved by correlating one or more non-dynamic operational issues and one or more dynamic issues in a cause effect relationship.

[0022] In the preferred embodiment, the monitoring module 114 configured to monitor the one or more corrective actions warranted for each of the detected one or more operational issues. The monitoring module 114 captures a second set of images comprising changes occurred as a result of the applied one or more corrective actions upon each of the one or more operational issues. The captured second set of images are compared with the first set of images stored in a database and determines one or more changes occurred as a result of the applied one or more corrective actions. And finally the monitoring module 114 analyze the resolution of the one or more operational issues based on the determined one or more changes. The one or more change occurred as a result of the applied one or more corrective actions such as number of people visiting locations of the commercial establishment, illumination level at locations of the commercial establishment, depletion rate of items of the shelves and emptiness of shelf, condition of the artefacts of the establishment and usable space of the commercial establishment.

[0023] In the preferred embodiment, the one or more operational issues include at least one of a poor lighting, empty shelf space, empty establishment space, wrong placement, outdated promotion or low footfall. There are few operational issues in the merchandising where there is no actual changes has been noticed since beginning. It may be due to planning oversight or deliberate as at that time these operational issues might not have any impact. Therefore, inclusion of these operational issues as a non-dynamic will have a great significance in visual merchandising.

[0024] Referring fig. 3, a method 300 for detecting one or more operational issues in visual merchandising.

[0025] In the preferred embodiment, at step 302, where a first set of images of the one or more operational issues in a commercial establishment are captured in real time using a plurality of installed media acquisition module 106. The captured images are stored at a central server and displayed on monitors given to supervisory staff, security and senior management. In addition to this, the media acquisition module 106 come with in-built software that to identify and track the operation issues of the establishment.
[0026] In the preferred embodiment, at step 304, where the method 300 extracts one or more features from the first set of images using a digital image processing module 108. The digital image processing module 108 starts feature extraction from an initial set of measured data and builds derived features intended to be informative and non-redundant. In addition to this, the optical character recognition that allows recognition of characters and numerals from a given images and it also converts the recognized characters and numerals to text.

[0027] In the preferred embodiment, at step 306, where the method 300 detects the one or more operational issues based on the extracted one or more features from the first set of images using a digital image processing module 108. The one or more operational issues include at least one of a poor lighting, empty shelf space, empty establishment space, wrong placement, outdated promotion or low footfall.

[0028] In the preferred embodiment, at step 308, where the method 300 classifies the detected one or more operational issues into dynamic operational issues and non-dynamic operational issues based on frequency of occurrence of the one or more operational issues using a media content analysis module 110. The dynamic operational issues such as fast moving objects getting depleted off the shelf or movement of the people crowding at a few specific place within the establishment. The dynamic issues often warranted corrective actions in real time. While the non-dynamic operational issues such as empty walls, poor lighting etc. often go undetected. But these operational issues can be corrected in batches.

[0029] In the preferred embodiment, at step 310, where the method 300 warrants the application of one or more corrective actions on each of the detected one or more operational issues using a corrective action warranting module 112. The dynamic operational issues warrant a real time corrective action and the non-dynamic operational issues warrant corrective actions at a predefined periodic interval.

[0030] In the preferred embodiment, at step 312, where the method 300 monitors the one or more corrective actions warranted for each of the detected one or more operational issues using a monitoring module 114. The monitoring comprises capturing a second set of images comprising at least one change occurred as a result of the applied one or more corrective actions upon each of the one or more operational issues, comparing the captured second set of images to the corresponding first set of images stored in a database to determine one or more changes occurred as a result of the applied one or more corrective actions using the media content analysis module and analyzing the resolution of the one or more operational issues based on the determined one or more changes using video content analysis module.

[0031] In the preferred embodiment, the one or more changes occurred as a result of the applied one or more corrective actions comprises number of people visiting locations of the commercial establishment, illumination level at locations of the commercial establishment, depletion rate of items of the shelves and emptiness of shelf, condition of the artefacts of the establishment and usable space of the commercial establishment.

[0032] The written description describes the subject matter herein to enable any person skilled in the art to make and use the embodiments. The scope of the subject matter embodiments is defined by the claims and may include other modifications that occur to those skilled in the art. Such other modifications are intended to be within the scope of the claims if they have similar elements that do not differ from the literal language of the claims or if they include equivalent elements with insubstantial differences from the literal language of the claims.

[0033] The embodiments of present disclosure herein addresses unresolved problem of one or more operational issues in visual merchandising. The unresolved problem of one or more operation issues includes poor lighting, empty shelves, wrong product placement, outdated promotional material, unused areas, low footfall areas, cluttered areas, improper placing of fixtures etc. The embodiment, thus provides a system and method for detecting and monitoring these one or more operational issues in visual merchandising. Moreover, the embodiments herein further provides a holistic solution based on image processing for operational issues in visual merchandising. In one embodiment, the system captures one or more images of one or more operational issues in a commercial establishment with a plurality of installed media acquisition module. Further, the system categorize these one or more images based on the field view of the one or more operational issues using a video content module. Furthermore, the system classifies the categorized one or more operational issues into dynamic and non-dynamic operation issues based on the frequency of change of objects or movement of people in real time. The digital image processing extracts one or more features of the classified images and warrants corrective actions based on the features of the operational issues. In addition to this, the system tracks and analyze impact of the corrective actions on operational issues of visual merchandizing.

[0034] It is, however to be understood that the scope of the protection is extended to such a program and in addition to a computer-readable means having a message therein; such computer-readable storage means contain program-code means for implementation of one or more steps of the method, when the program runs on a server or mobile device or any suitable programmable device. The hardware device can be any kind of device which can be programmed including e.g. any kind of computer like a server or a personal computer, or the like, or any combination thereof. The device may also include means which could be e.g. hardware means like e.g. an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination of hardware and software means, e.g. an ASIC and an FPGA, or at least one microprocessor and at least one memory with software modules located therein. Thus, the means can include both hardware means and software means. The method embodiments described herein could be implemented in hardware and software. The device may also include software means. Alternatively, the embodiments may be implemented on different hardware devices, e.g. using a plurality of CPUs.

[0035] The embodiments herein can comprise hardware and software elements. The embodiments that are implemented in software include but are not limited to, firmware, resident software, microcode, etc. The functions performed by various modules described herein may be implemented in other modules or combinations of other modules. For the purposes of this description, a computer-usable or computer readable medium can be any apparatus that can comprise, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

[0036] The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. Examples of a computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk. Current examples of optical disks include compact disk-read only memory (CD-ROM), compact disk-read/write (CD-R/W) and DVD.

[0037] A data processing system suitable for storing and/or executing program code will include at least one processor coupled directly or indirectly to memory elements through a system bus. The memory elements can include local memory employed during actual execution of the program code, bulk storage, and cache memories which provide temporary storage of at least some program code in order to reduce the number of times code must be retrieved from bulk storage during execution.

[0038] Input/output (I/O) devices (including but not limited to keyboards, displays, pointing devices, etc.) can be coupled to the system either directly or through intervening I/O controllers. Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modem and Ethernet cards are just a few of the currently available types of network adapters.

[0039] A representative hardware environment for practicing the embodiments may include a hardware configuration of an information handling/computer system in accordance with the embodiments herein. The system herein comprises at least one processor or central processing unit (CPU). The CPUs are interconnected via system bus to various devices such as a random access memory (RAM), read-only memory (ROM), and an input/output (I/O) adapter. The I/O adapter can connect to peripheral devices, such as disk units and tape drives, or other program storage devices that are readable by the system. The system can read the inventive instructions on the program storage devices and follow these instructions to execute the methodology of the embodiments herein.

[0040] The system further includes a user interface adapter that connects a keyboard, mouse, speaker, microphone, and/or other user interface devices such as a touch screen device (not shown) to the bus to gather user input. Additionally, a communication adapter connects the bus to a data processing network, and a display adapter connects the bus to a display device which may be embodied as an output device such as a monitor, printer, or transmitter, for example.

[0041] The preceding description has been presented with reference to various embodiments. Persons having ordinary skill in the art and technology to which this application pertains will appreciate that alterations and changes in the described structures and methods of operation can be practiced without meaningfully departing from the principle, spirit and scope.