Claims:
1. A method (200) for context driven processing of multimedia content on an edge device, the method comprising:
(a) receiving (204) input multimedia content (402);
(b) generating (212) context characteristics based on user characteristics, edge device characteristics, multimedia characteristics, or combinations thereof;
(c) retrieving (214) at least one model (324, 412) based on the context characteristics;
(d) identifying (216) one or more processing steps based on the at least one model (324, 412), the context characteristics, or both the at least one model (324, 412) and the context characteristics, wherein the one or more processing steps are used to perform context driven processing of the input multimedia content (402) on the edge device (104);
(e) selecting (218), based on the at least one model (324, 412), the context characteristics, or both the at least one model (324, 412) and the context characteristics, one or more target processing units (418, 420, 422, 424, 426) to perform the one or more processing steps;
(f) executing (220) the one or more processing steps on the selected one or more target processing units (418, 420, 422, 424, 426) to generate improved output multimedia content, wherein the improved output multimedia content comprises enhanced visual quality, enhanced aural quality, enhanced information content, or combinations thereof; and
(g) providing (222) the improved output multimedia content.

2. The method (200) of claim 1, wherein steps (a)-(g) are performed in real-time on the edge device (104).

3. The method (200) of claim 1, wherein generating the context characteristics comprises:
receiving (208) user characteristics corresponding to a user (102) of the edge device (104);
receiving (206) edge device characteristics corresponding to the edge device (104);
extracting (210) multimedia characteristics from the input multimedia content (402);
extracting context-based features, contextual information, content information, or combinations thereof and corresponding relationships there between from one or more of the user characteristics, the edge device characteristics, and the multimedia characteristics; and
creating (212) the context characteristics using the extracted context-based features, the contextual information, the content information, and the corresponding relationships,
wherein the context characteristics are created in real-time on the edge device (104), and wherein the context characteristics are employed to enhance one or more of visual quality, aural quality, and information content of the input multimedia content (402) in real-time.

4. The method (200) of claim 1, wherein retrieving at least one model (324, 412) comprises dynamically identifying a model (324, 412) that is optimized for processing the input multimedia content (402) based on the context characteristics.

5. The method (200) of claim 4, wherein dynamically identifying the model comprises:
performing content aware extraction of the multimedia characteristics from the context characteristics;
performing edge device aware extraction of the edge device characteristics from the context characteristics; and
selecting a model (324, 412) that is most suited to perform a task to process the input multimedia content (402) based at least on content aware extraction of the multimedia characteristics and the edge device aware extraction of the edge device characteristics.

6. The method (200) of claim 1, further comprising generating one or more models (324, 412), and wherein the one or more models (324, 412) are tuned for performing one or more tasks.

7. The method (200) of claim 6, wherein generating the one or more models (324, 412) comprises:
receiving (302) an input corresponding to the one or more tasks to be performed;
obtaining (304) a plurality of multimedia datasets of known visual quality, known aural quality, known information content, or combinations thereof;
obtaining (306) a plurality of multimedia datasets of known higher visual quality, known higher aural quality, known higher information content, or combinations thereof;
generating (308) one or more training multimedia dataset pairs, wherein each training multimedia dataset pair comprises an input multimedia dataset and a corresponding output multimedia dataset; and
receiving (310) a plurality of visual metrics based on the one or more training multimedia dataset pairs and the one or more tasks to be performed.

8. The method (200) of claim 7, further comprising:
receiving (312) edge device characteristics corresponding to one or more edge devices (104);
receiving (314) user characteristics corresponding to one or more users (102);
extracting (316) multimedia characteristics from the one or more training multimedia dataset pairs; and
generating (318) context characteristics based on the edge device characteristics corresponding to one or more edge devices (104), the user characteristics corresponding to one or more users (102), the multimedia characteristics corresponding to the one or more training multimedia dataset pairs, or combinations thereof.

9. The method (200) of claim 8, further comprising selecting (320) one or more training processes based on the context characteristics, the one or more training multimedia dataset pairs, and the plurality of visual metrics.

10. The method (200) of claim 9, further comprising training (322) a neural network using the one or more training processes to generate a model (324, 412), model metadata, or a combination thereof, wherein the model (324, 412) and the model metadata are configured to perform the one or more tasks.

11. The method (200) of claim 1, wherein identifying (216) the one or more processing steps comprises:
performing content aware extraction of the multimedia characteristics from the context characteristics, and
selecting one or more processing steps to process the input multimedia content (402) based at least on the content aware extraction of the multimedia characteristics.

12. The method (200) of claim 1, wherein selecting (218) the one or more target processing units comprises:
performing edge device aware extraction of the edge device characteristics from the context characteristics; and
for each processing step, identifying one or target processing units (418, 420, 422, 424, 426) that are optimized to perform that processing step based at least on the edge device aware extraction of the edge device characteristics.
, Description:BACKGROUND
[1] Embodiments of the present specification relate generally to processing of multimedia content, and more particularly to systems and methods for context driven processing of multimedia content on an edge device.
[2] Rapid advances in broadband internet, cloud computing, networking, and the like have led to an exponential increase in the demand for the real-time streaming of multimedia content. More recently, there has been an increased demand for streaming multimedia content on an edge device such as a smart phone, a television, a router, and the like. However, the huge footprint of the multimedia content disadvantageously results in substantially high costs associated with storage and/or transmission of the multimedia content. Additionally, insufficient bandwidth adversely impacts the streaming of the multimedia content, thereby resulting in buffering, long load times, pixellation, poor quality of viewer experience, and the like. Consequently, quality of an end user’s viewing experience may be undesirably impacted as the limited bandwidth and/or the high streaming costs restrict the amount of information that can be streamed to the end user. Furthermore, some edge devices may not be capable of streaming certain types of multimedia content.
[3] Certain presently available techniques for addressing the issues with variations in the availability of the bandwidth entail compressing the multimedia content for both storage and streaming. Also, some other video streaming techniques tackle the challenges of bandwidth variability by creating and storing multiple versions of the same video at varying resolutions and bitrates and/or by transcoding a video in real-time at varying bitrates depending on the bandwidth available. However, these techniques disadvantageously result in higher transcoding, storage, and/or transmission costs.
[4] Moreover, in recent times, there have been attempts to use machine learning techniques to address issues with video streaming. However, these techniques are restricted to larger cloud-based and/or desktop-class compute resources and hence fail to achieve real-time video streaming performance on resource-constrained edge devices. Additionally, performance metrics associated with the currently available techniques are generally targeted to sets of well-researched, limited, competition-focused data. Hence, these techniques fail to scale to real-world video streaming datasets, which have larger range of features, parameters, and/or deviations.
[5] Furthermore, use of contextual information in the content streamed to any particular user to enhance the quality of a user’s experience has not been explored heretofore, thereby limiting streaming service providers’ quality of offerings as well as end users’ quality of experience all, while increasing operational costs.

BRIEF DESCRIPTION
[6] In accordance with aspects of the present specification, a system for context driven processing of multimedia content on an edge device is presented. The system includes an acquisition subsystem configured to obtain input multimedia content. Furthermore, the system includes a processing subsystem in operative association with the acquisition subsystem and including a context aware artificial intelligence platform, where the context aware artificial intelligence platform is, on the edge device, configured to generate context characteristics based on user characteristics, edge device characteristics, multimedia characteristics, or combinations thereof, retrieve at least one model based on the context characteristics, identify one or more processing steps based on the at least one model, the context characteristics, or both the at least one model and the context characteristics, where the one or more processing steps are used to perform context driven processing of the input multimedia content on the edge device, select, based on the at least one model, the context characteristics, or both the at least one model and the context characteristics, one or more target processing units to perform the one or more processing steps, and execute the one or more processing steps on the selected one or more target processing units to generate improved output multimedia content, where the improved output multimedia content comprises enhanced visual quality, enhanced aural quality, enhanced information content, or combinations thereof. In addition, the system includes an interface unit configured to provide, on the edge device, the improved output multimedia content.
[7] In accordance with another aspect of the present specification, a method for context driven processing of multimedia content on an edge device is presented. The method includes (a) receiving multimedia content, (b) generating context characteristics based on user characteristics, edge device characteristics, multimedia characteristics, or combinations thereof, (c) retrieving at least one model based on the context characteristics, (d) identifying one or more processing steps based on the at least one model, the context characteristics, or both the at least one model and the context characteristics, wherein the one or more processing steps are used to perform context driven processing of the input multimedia content on the edge device, (e) selecting, based on the at least one model, the context characteristics, or both the at least one model and the context characteristics, one or more target processing units to perform the one or more processing steps, (f) executing the one or more processing steps on the selected one or more target processing units to generate improved output multimedia content, wherein the improved output multimedia content comprises enhanced visual quality, enhanced aural quality, enhanced information content, or combinations thereof, and (g) providing the improved output multimedia content.
[8] In accordance with yet another aspect of the present specification, a processing system for context driven processing of multimedia content on an edge device is presented. The processing system includes a context aware artificial intelligence platform, wherein the context aware artificial intelligence platform is, in real-time on the edge device, configured to generate context characteristics based on user characteristics, edge device characteristics, multimedia characteristics, or combinations thereof, retrieve at least one model based on the context characteristics, identify one or more processing steps based on the at least one model, the context characteristics, or both the at least one model and the context characteristics, wherein the one or more processing steps are used to perform context driven processing of the input multimedia content on the edge device, select, based on the at least one model, the context characteristics, or both the at least one model and the context characteristics, one or more target processing units to perform the one or more processing steps, execute the one or more processing steps on the selected one or more target processing units to generate improved output multimedia content, wherein the improved output multimedia content comprises enhanced visual quality, enhanced aural quality, enhanced information content, or combinations thereof, and provide the improved output multimedia content, wherein the context aware processing of the multimedia content in performed real-time, on the edge device.

DRAWINGS
[9] These and other features and aspects of embodiments of the present specification will become better understood when the following detailed description in read with reference to the accompanying drawings in which like characters represent like parts throughout the drawings, wherein:
[10] FIG. 1 is a schematic representation of an exemplary system for context driven processing of multimedia content on an edge device, in accordance with aspects of the present specification;
[11] FIG. 2 is a flow chart illustrating a method for context driven processing of multimedia content on an edge device, in accordance with aspects of the present specification;
[12] FIG. 3 is a flow chart illustrating a method for generating one or more models for use in the method for context driven processing of multimedia content on an edge device of FIG. 2, in accordance with aspects of the present specification;
[13] FIG. 4 is a schematic representation of one embodiment of a context driven multimedia processing system for use in the system of FIG. 1, in accordance with aspects of the present specification; and
[14] FIG. 5 is a schematic representation of one embodiment of a digital processing system implementing a context driven multimedia processing system for use in the system of FIG. 1, in accordance with aspects of the present specification.
DETAILED DESCRIPTION
[15] The following description presents exemplary systems and methods for context driven processing of multimedia content on an edge device. In certain embodiments, the exemplary systems and methods facilitate context driven processing of multimedia content in real-time on an edge device. Embodiments described hereinafter present exemplary systems and methods that facilitate enhanced quality of experience for a user of an edge device and/or a viewer of streamed multimedia content on an edge device in real-time and independent of network bandwidths constraints. Moreover, these systems and methods provide a solution that is agnostic of the currently available infrastructure and aids in reducing streaming costs. Use of the present systems and methods presents advantages in reliably providing significant enhancement in the quality of experience for end consumers and reducing transcoding, storage, and/or transmission costs for streaming service providers, thereby overcoming the drawbacks of currently available methods of enhancing quality of streamed multimedia content.
[16] For ease of understanding, the exemplary embodiments of the present systems and methods are described in the context of streaming multimedia. However, use of the exemplary embodiments illustrated hereinafter in other systems and applications such as multimedia storage, transmission, consumption, and multiplexed communication is also contemplated. An exemplary environment that is suitable for practising various implementations of the present systems and methods is discussed in the following sections with reference to FIG. 1.
[17] As used herein, the term “user” refers to a person using an edge device or the system of FIG. 1 for streaming multimedia content. For example, the user uses an edge device for viewing the streaming multimedia content. The terms “user,” “viewer,” “consumer,” “end user,” and “end consumer” may be used interchangeably.
[18] Also, as used herein, the term “edge device” refers to a device that is a part of a distributed computing topology in which information processing is performed close to where things and/or people produce or consume information. Some non-limiting examples of the edge device include a mobile phone, a tablet, a laptop, a smart television (TV), and the like. Additionally, the term “edge device” may also be used to encompass a device that is operatively coupled to an edge device noted hereinabove. Some non-limiting examples of such a device include a streaming media player that is connected to a viewing device such as a TV and allows a user to stream video and/or music, a gaming device/console, and the like. Other examples of the edge device also include networking devices such as a router, a modem, and the like.
[19] Further, as used herein, the term “multimedia” or “multimedia data” or “multimedia content” encompasses one or more types of data such as, but not limited to, video data, audio data, movies, games, TV shows, images, text, graphic objects, animation sequences, and the like. It may be noted that the terms “multimedia,” “multimedia data,” and “multimedia content” may be used interchangeably.
[20] Also, as used herein, the term “context” or “contextual information” refers to edge device characteristics, user characteristics, multimedia characteristics, or combinations thereof. Furthermore, as used herein, the term “edge device characteristics” refers to characteristics associated with an edge device. Some non-limiting examples of the edge device characteristics include processors of the edge device, processing speed of the processors, processing power, capability to handle specific types of operations, accuracy of output, efficiency of specific operations, and the like. In a similar fashion, the term “user characteristics” refers to characteristics associated with a user of an edge device. Some non-limiting examples of the user characteristics include user preferences, user’s usage statistics, user location, user network capabilities, current user environment, and the like. Moreover, as used herein, the term “multimedia characteristics” refers to characteristics associated with the multimedia content. Some non-limiting examples of the multimedia characteristics include a type of the multimedia content such as a video, a genre of the multimedia content, temporal variability, spatial variability, special features, presence of specific subjects such as humans, animals, buildings, and the like.