«N AUTOMATED SYSTEM FOR MULTIMEDIA DATA OBJECTS AAANAGEMENT AND
RETRIEVAL
Field of invention: -
The present embodiments of the invention relate to an automated system and method for multimedia data objects management and retrieval for identification, association of object information for multimedia data object and retrieval of a multimedia data object on the basis of the object information provided by a user.
Background of the invention: -
Multimedia has become extremely popular over the years thereby increasing the use of multimedia devices such as digital cameras, camcorders, cellular phones etc. The popularity of such devices has increased as the user has migrated from using the conventional analog techniques to the digital form of capturing data as the devices enabled to capture data digitally are easier to use and the memory of these devices are reusable unlike the conventional options wherein a silver halide coated polyester film roll is used in case of images however, in case of capturing video and audio objects magnetic tape is used. Also, the digital techniques provide us with an option for immediate viewing of the captured data and the ease of getting the data printed if the data is an image. Further if the data captured is an image, audio or video; these can be instantly and easily shared with others over a digital network or otherwise.
A widespread application of the digital capture of data is in using cellular phones wherein the phone is enabled to capture multimedia data such as images, audio files as well as video images using the camera available. The camera is embedded in the cellular phone itself and is easy to use; making the
-1-

capture of memorable moments easy. Moreover, as against the early years of camera-enabled cellular phones where said camera was of low resolution and limited functionality, the new generation camera phones are matching the capabilities and features of high-end digital cameras now. Thus, the sale of such multimedia capture enabled phones has increased as these handheld devices also offer storage of the captured data wherein the memories run into several gigabytes, similar to digital cameras. This enables the user to capture large amount of images/data without having to worry about the memory storage, cost and maintenance of the captured data such as images and videos.
With the availability of large amounts of memory in such devices, retrieval of captured multimedia data has becomes cumbersome, as the users, over a period of time, tend to accumulate a huge collection of data. Therefore retrieval of desired data such as an image and viewing of the same becomes difficult wherein the user needs to filter through the entire collection of multimedia data available on said device. As a solution the user may organize the data and categorize the captured data. However, it is not effective in case there are a huge number of images or audio files in the categories as well.
A user might remember the date and time of capture of the data or the folder where it was stored. However, in a huge collection of such data it becomes cumbersome and time-consuming to locate a desired captured data such as a photograph or a video image. Also, when the cellular phones capture images or video images they are by default saved on the basis of their date or time of capture or image number. When these images are loaded onto a computer, the computer reassigns the data with the date and time they are stored into said computer thereby changing the date and time the user may remember and hence the retrieval of the images or video images becomes difficult.
To overcome the above-mentioned problem, there are several conventional methods, which offer the retrieval of captured data using information

associated with the data. This information known as the metadata Identifies various attributes of captured data such as the image resolution, image size, camera exposure and aperture settings, date and time when the data was captured, etc. Metadata information is used to sort the data as per user preferences. There are various methods as described below which help simplify the organizing, browsing, searching and retrieval operations of the captured data.
The term "metadata" has been used interchangeably with "object information" in the description that follows.
A category of systems and methods describe the transmission of the personal data and metadata among computing devices. US patent application number 20060221190 discloses a camera enabled computing device for capturing media files such as video, audio, image data comprising a first component for synchronizing with metadata information stored on at least computing device which hosts personal data services; a second component which displays the metadata information and receives the input from a user of the camera enabled computing device that associates a subset of the metadata information with the media file and a third component for uploading the media file and the associated subset of metadata information to at least one computing device that hosts the personal data services. It tags the captured data using predefined tags stored on the user's personal web server when the capture device is coupled to the web server of the user. Using only predefined tags is a limitation towards effective usage by regular or low-skilled users of such devices. Users prefer storing metadata information without being constrained by predefined tags and are not able to store certain information as metadata, which is not covered by predefined tags. This system inherently uses tags stored on the Web server of the user thus, making internet an essential part of such a system.

us 6,833,865 discloses another category of systems and methods that offers a metadata engine embedded in the digital capture device. Said device has an embedded real time content-based analysis function to extract the metadata from the digital signals. The metadata is formatted and stored separately from the content and the same may be formatted and combined with digital format such as MPEG-7, Quick time or Flash Pix. This application focuses only in the extraction of content-based metadata information in real time from the digital content and uses image feature vectors for extraction of such metadata from the captured data. The content based information is in the form of closed captioned text (CC), teletext, time/date, and media properties such as frame rates, bit rates, annotations. Speech to text and keyword spotting speaker identification (ID), Optical character recognition etc. However the use of such formats for cellular phones is difficult and may be used only in prohibitively expensive, high-end cellular phones and focuses on container formats.
Another aspect of systems and methods available as described in US 20070127833 discloses the capture of exact environment during the acquisition of the multimedia data such as pressure, temperature, humidity, sunshine etc wherein the means for capture includes a sensor to measure these conditions. Thus each user needs to remember the exact environment when the data was captured, which is practically not possible. Also the proposed solution is not cost-effective and is very complex to implement as it requires integration of extra hardware in the capture device. However, these devices are not enabled to retrieve the data at a later stage on the basis of the identification of the captured exact environment.
The US patent US 7162053 describes another aspect of such systems where once the images are received from an image-capturing device, the metadata from the said device is also received. However, the said metadata is first used to generate a new metadata value through analyzers and filters and then stored. The new metadata values corresponding to the acquired image data are

also integrated v^ith the image information. The system is dependent on metadata already captured by the capturing device and is focused on rendering the captured metadata.
US 6629104 deals v/ith a method for adding personal image metadata to a collection of images. A database of pre-assigned metadata labels is configured according to a set of queries as answered by the user which is useful is classification of the images. The labels consist of a label identifier relating to emotional or aesthetic judgment of the user. The labels are then assigned to an image and stored in association with the same. The user may categorize the images according to his priority and may through an application of the invention display the labels according to choice and hence, create a personal database referencing to appropriate images. However, such a system is completely dependent on user's response for configuring the database of pre-assigned database labels and thus we require an automated system with an option for user intervention without the metadata labels being pre-assigned.
In the light of above it is seen that there is a need for a comprehensive, captured multimedia data management system which is fully automated and has in built provisions for user interface for efficient retrieval of the same as well.
Summary of the invention:
To achieve an automated, cost effective and efficient management and retrieval of multimedia data objects, embodiments of the present invention are disclosed, which deal with an automated system for multimedia data objects management and retrieval comprising acquisition means to acquire multimedia data object, object information manager communicatively coupled to the acquisition means to identify relevant object information and associate the same with acquired multimedia data object and means for storage and

retrieval communicatively coupled to the object information manager to store multimedia data object with associated object information and retrieving a multimedia data object based on object information specified by a user.
Another automated method for multimedia data objects management and retrieval comprises acquiring a multimedia object by means of a multimedia data acquisition means, identifying and associating relevant object information by means of an object information manager in respect of a multimedia data object and storing the acquired multimedia data object associated with object information and retrieving a multimedia data object on the basis of object information specified by the user by means for storage and retrieval. The identification of object information as disclosed in the method above further comprises analyzing acquired multimedia data object for unique attributes such as mode, texture etc. by means of an object analysis unit, identifying location of acquisition of said multimedia data object by means of a location identifier, controlling object information in said acquisition means and the network and identifying relevant object information specific to said acquired multimedia data object by means of an information control unit and controlling input user defined object information in said multimedia data acquisition means and identifying relevant object information by means of an user control unit.
Brief Description of Drawings
The detailed description is described with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the drawings to reference like features and components.
Figure 1 illustrates an exemplary automated system for multimedia data

object management and retrieval according to the present invention.
Figure 2 illustrates an exemplary block diagram of an object information identifier according to the present invention.
Figure 3 illustrates schematic representation of an exemplary method for automated multimedia data object management and retrieval according to the present invention.
Figure 4 illustrates a flow diagram of an exemplary method for object information identification according to the present invention.
Detailed description of the invention:
Embodiments of the instant invention have been described. The systems and methods are not intended to be restricted to any particular form or arrangement, or any specific embodiment, or any specific use, disclosed herein, since the same may be modified in various particulars or relations without departing from the spirit or scope of the claimed invention hereinabove shown and described of which the apparatus or method shown is intended only for illustration and disclosure of an operative embodiment and not to show all of the various forms or modifications in which this invention might be embodied or operated.
Figure 1 refers to an embodiment of the present invention wherein an automated system for multimedia data objects management and retrieval of the same is disclosed. Acquisition means 1.1 helps acquire a multimedia data object from an environment and uses an object information manager 1.2 to identify various object data information. As an example, a digital camera enabled cellular phone is used as an acquisition means that captures images as well as video files. These images are then processed through an object

information manager which identifies and associates relevant information in respect of a multimedia data object. The identified information is metadata which is unique to the acquired image such as date, time, mode of camera, texture, location of capture of the photograph etc and is associated to the multimedia data object.
The object information manager is further communicatively coupled to a means for storage and retrieval 1.3, which stores the multimedia data object with associated object information. Said means are also enabled to retrieve a multimedia data object on the basis of object information specified by the user. Said means filters through the stored data objects using the object information specified and retrieves the desired data object. Further to the example as described above, the system stores the image with the associated metadata. When the user requests for a desired data object and specifies object information i.e. metadata specific to said image such as the date, time, location or texture or mode of the digital camera etc, the system then filters through the stored images identifies the desired image and gives it as an output to the user.
Figure 2 illustrates an object information manager which comprises of object analysis unit 2.1, location identifier 2.2, information control unit 2.3 and the user control unit 2.4. The object analysis unit 2.1 analyses the acquired multimedia data object to determine relevant object information such as the mode of the acquisition means and texture of the acquired data if the data object is vision based. The location identifier 2.2 is used to identify the location of the acquisition of the multimedia data object using one or more location techniques available such as the GPS or tower related information obtained. The information control unit 2.3 is used to control the object information in the acquisition means and on the network and determines the relevant object information to be associated with the acquired multimedia data object. A user control unit 2.4 is used to control the user defined object

information input by a user and identifies the relevant object information specific to said multimedia data object. The user inputs labels that convey the details regarding the specific data object. As another embodiment of the invention, it is assumed that the acquisition means 1.1 is a digital camera enabled cellular phone and the acquired multimedia data object is an image. The image is processed through the object analysis unit 2.1 which analyses the image to identify the relevant metadata in form of the mode of the cellular phone during acquisition of said image and texture of the acquired image. The location identifier 2.2 determines the location of the acquisition of the image using GPS, if the cellular phone is GPS enabled, however, incase the phone is not GPS enabled, it is possible to relate the tower related information regarding location to said image and identifies the location information as relevant metadata. The information control unit 2.3 controls the data available in the cellular phone itself such as the phone book, messages, calendar events, reminders etc and identifies the relevant information as metadata specific to said image. Also, the information control unit 2.3 controls the information available on the network when the same is connected and identifies relevant information that may be related to the image such as important events in the country, a national holiday or a religious event etc. The user control unit 2.4 controls the user defined information input by the user. The information defined by the user is in form of labels and the user control unit identifies the relevant labels specific to an image. For example, an image is 2.4 captured in the evening using night mode of capture in a cellular phone on 4^^ July. The object information manager 1.2 processes the image and by means of the object analysis unit 2.1, location identifier 2.2, object information control unit 2.3 and user control unit 2.4 identifies the mode of the image to be night mode, location as of the location where captured, and identification of an important event on 4**^ July as the Independence Day of America when the control unit 2.3 is connected to the network and the identification of the relevant label input by the user. Therefore, the identified object information is associated to said image and is stored in the means for association and storage.

The user may retrieve the image using the object information as American Independence Day and retrieve the desired data object using the retrieval logic unit 1.4.
Exemplary methods for automated multimedia data object management and retrieval have been described in Figures 3 and 4. The methods are illustrated as a collection of blocks in a logical flow graph, which represents a sequence of operations that can be implemented in hardware, software, or a combination thereof. The order in which the process is described is not intended to be construed as a limitation, and any number of the described blocks can be combined in any order to implement the process, or an alternate process. Additionally, individual blocks may be deleted from the process without departing from the spirit and scope of the subject matter described herein.
Figure 3 illustrates a schematic representation of an exemplary method for automated multimedia data object management and retrieval comprising the steps of acquiring a multimedia data object 3.1 from an environment by an acquisition means and identifying and associating relevant object information by means of an object information manager in respect of a multimedia data object 3.2 wherein the object information identifier identifies relevant information i.e. the metadata specific to the acquired data object. The data object may be an image, video file or an audio and the object information identifier determines the relevant metadata such as the date, time, location, mode and texture of the data object etc. The identified object information is then associated with said data object acquired by the acquisition means and the data object associated with the multimedia is stored 3.3 by means of means for storage and retrieval. Retrieving of a desired data object based on the object information specified by a user 3.4 by means of storage and retrieval is accomplished after filtering through the stored data objects in the means for storage and retrieval.

Figure 4 illustrates a flow diagram of a method for object information identification wherein the object information is identified after analyzing the acquired multimedia data object for unique attributes 4.1 by means of an object analysis unit. The unique attributes comprises of the mode of the acquisition means during acquiring of the data object and texture of the data object acquired. Identifying the location of acquisition of said multimedia data object 4.2 by means of a location identifier helps ascertain the location of acquisition of the data object. Controlling object information in said acquisition means and the network 4.3 and identifying relevant object information specific to said acquired multimedia data object 4.4 by means of an information control unit helps identify relevant object information available in the acquisition means and on the network such as the person in the phone book, calendar events, reminders and important events available on the network. Controlling input user defined object information 4.5 and identifying relevant object information 4.6 by means of a user control unit. The user enters information for pre-defined or new labels, which convey the details regarding the data object captured.
As an example of the exemplary embodiments of systems and methods mentioned above it is assumed that the multimedia data object acquisition means is a digital camera enabled cellular phone.
By means of an object analysis unit 2.1, various camera modes of the acquisition means are identified as metadata related to the image. Different camera modes are used for capturing different types of images, as the particular mode has the most-optimized settings for capturing certain type of photographs. For example. Portrait mode is best suited for capturing photographs of people at a short distance from camera. In Portrait mode, the camera automatically sets the camera for wide aperture setting, minimizing depth of field and tries to minimize the red-eye effect by adjusting flash settings. Similarly, Landscape mode is best suited to capture photographs of

distant panoramic objects such as mountains, fields, rivers, etc. Flash does not have much use for photographs in the Portrait mode and is therefore used minimally. In yet another example, the Macro mode is used to capture extreme close-ups of objects, such as flowers, faces, etc. This mode allows good clarity for close-by objects, which otherwise appear blurred if other mode or Auto mode is on. Apart from Landscape, Portrait and Macro modes, other common modes available on digital and camera phones are Night mode. Beach Mode and Snow Mode. The type of mode can largely define the type of picture being taken and is used identify object information i.e. metadata.
As an exemplary embodiment of the present invention, camera modes (Landscape, Portrait, Macro, Night, Beach, Snow, Sports, Flash on/off, etc.) are used to identify the subject of the photograph. Associated with each mode are certain keywords, which are the most likely candidates for describing object, captured in the photograph. For example, if a photograph is clicked in Landscape mode, it is most likely to contain objects such as mountains, river, field, sea, lake, house, desert, highway, etc. The example uses camera-mode metadata in addition to other techniques such as Texture Finding (described next) to provide good results in determining and describing the objects in the photograph.
The present embodiment of the instant invention also includes a very important feature, known as the Texture Finding by means of object analysis unit 2.1. Texture-finding techniques are used to detect common elements such as sky, clouds, water, roads, snow, wood, etc., in images. Information about the presence of one or more of the common textures in the image is associated by means of an object information identifier and is stored as image metadata in means for storage and retrieval 1.3.
When using a cellular phone for capturing images, location information can be easily captured using a location identifier 2.2 using the information continuously beamed to the phone from the transmitting tower for cellular

phones that are not GPS enabled. GPS data, though highly accurate, is still very much a feature of top-end cell phones, and out of reach of many. The use of tov^^er related location information is easier and cost-effective. Also, location data, which is usually in the form of city name/area name/transmitting tower location, is good-enough information for a user to intuitively guess the location information when searching for images.
All the cellular phones invariably have a built-in calendar/organizer for storing events, birthdays, meeting, etc. and putting a reminder for these. This calendar information, stored in the user's cellular phone, is used as metadata by means of an information control unit 2.3 and is exploited as a critical criterion on which user retrieves images. For particular events saved in the calendar, images captured on that date are automatically associated with those events.
In the present embodiment of the invention, Internet or Web connection is not an integral but an optional part. It requires an Internet connection only for specific features. When connected to the Internet, based on the dates, on which the images were captured, important events/public holidays specific to the images are identified by means of information control unit 2.3. The system is intelligent enough to perform localized search as well as generalized search/retrieval. Localized search/ retrieval seeks information relating to the festivals/events/public holidays for the geography/area in which the photograph has been clicked. The generalized search/retrieval allows metadata for well-known festivals/events/public holidays to be associated with the photograph. In this manner, the system is able to capture most relevant metadata for the photograph.
The embodiment of the invention enables a user to save audio data also as part of the metadata. The user can record brief information describing the photograph, and later use the recorded information for retrieval of the image.

The embodiments described above and illustrated in the figures are presented by way of example only and are not intended as a limitation upon the concepts and principles of the present invention. Elements and components described herein may be further divided into additional components or joined together to form fewer components for performing the same functions. As such, it will be appreciated by one having ordinary skill in the art that various changes in the elements and their configuration and arrangement are possible without departing from the spirit and scope of the present invention as set forth in the appended claims.
We claim: -
1. An automated system for multimedia data objects management and retrieval comprising: -

acquisition means to acquire multimedia data object;
object information manager communicatively coupled to the acquisition means to identify relevant object information and associate the same with acquired multimedia data object; and
means for storage and retrieval communicatively coupled to the object information manager to store multimedia data object with associated object information and retrieving a multimedia data object based on object information specified by a user.
2. The system as claimed in claim 1 wherein the multimedia data object includes image and audio data objects.
3. The system as claimed in claim 1 wherein said object information manager comprises : -
object analysis unit; location identifier; information control unit; and user control unit.
4. The system as claimed in claim 1 or 3 wherein the object analysis unit communicatively coupled to the system analyses the acquired multimedia data object for one or more unique attributes such as mode of said acquisition means and texture of said multimedia data object.
5. The system as claimed in claim 1 or 3 wherein the location identifier unit communicatively coupled to the system identifies the location of the system by one or more location identification techniques as GPS and/or tower related information.

6. The system as claimed in claim 1 or 3 wherein the information control unit communicatively coupled to the system identifies relevant object information specific to the said multimedia data object,
7. The system as claimed in claim 1 or 3 wherein the user control unit communicatively coupled to the system enables user defined object information to be input for said multimedia data object.
8. The system as claimed in claim 7 wherein the object information input by the user comprises labels specific to said multimedia data object.
9. An automated method for multimedia data objects management and retrieval comprising the following steps: -
acquiring a multimedia object by means of a multimedia data acquisition means;
identifying and associating relevant object information by means of an object information manager in respect of a multimedia data object; and
storing the acquired multimedia data object associated with object information and retrieving a multimediahdata object on the basis of object information specified by the user by means for storage and retrieval;
10. The method as claimed in claim wherein said object information is
identified and associated in respect of the acquired multimedia data
object by means of an object information manager comprising: -
analyzing acquired multimedia data object for one or more unique attributes such as mode, texture etc. by means of an object analysis unit;

I
identifying location of the system by means of a location identiffer;
controlling object information in said acquisition means and the network and identifying relevant object information specific to said acquired multimedia data object by means of an information control unit; and
controlling input user defined object information in said multimedia data acquisition means and identifying relevant object information by means of an user control unit.
11. The method as claimed in claim v^herein said acquisition means are enabled to acquire image and audio objects.
14. The method as claimed in claim v/herein location of acquired
multimedia data object is identified by said location identifier by
means of one or more location depicting techniques as GPS and tower
related information.
15. The method as claimed in claim wherein the user defined object information input by the user comprise of labels specific to said acquired multimedia data object.
16. A computer product for automated multimedia data object management and retrieval comprising one or more computer readable means configured to perform the method as claimed in claim 9-15.