DESC:F O R M 2
THE PATENTS ACT, 1970
(39 of 1970)
&
The Patents Rules, 2003 5
COMPLETE SPECIFICATION
(See section 10; rule 13)

1. Title of the invention -
SYSTEMS AND METHODS TO GENERATE 3-D AVATAR

2. Applicant(s)
(a) NAME : Mayaa-Verse Creations Private limited

(b) NATIONALITY : An Indian Company

(c) ADDRESS : FC-19, sector 16A, Noida, Noida Gautham Budhha Nagar UP, 201301, India

3. PREAMBLE TO THE DESCRIPTION

The following specification particularly describes the invention and the manner in which it is to be performed:

FIELD OF INVENTION

The present disclosure relates generally to generation of automatic and accurate 3-D digital avatar. The present disclosure further relates to the system and process to generate more accurate and realistic 3-D avatar of user or object and applications of same. The present disclosure further provides a system that utilizes visual and infrared data and applies advanced machine learning techniques to generate realistic and high fidelity 3-D avatar.

BACKGROUND

Operating in a virtual environment is not new thing. Virtually everyone on this planet now is part of virtual environment by one means or other. However, to make the experience more realistic everyone not just wants to see or view the cartoon character but also wants to be part of this Virtual Universe or Metaverse. The thirst of this personal experience in the metaverse led to evolution of technology that can create Avatar which can resemble the appearance of the user.

We live in the age of the avatar. Creating a digital persona is no longer confined to the walls of a video game. From Snapchat’s Bitmoji to Apple’s Memoji and more, give users the ability to create their own, customizable avatar. Even without the support of platforms, the users have been creating and uploading their own avatars. Even the application of Avatars goes beyond video gaming and social networking websites. Avatar can be used to enhance the shopping experience of user in clothing, make up, hair dressing and other activities. Virtual avatars have also been making the remote meetings, calls more interactive. Even this technology is enabling support in the medical world also.

As discussed above avatars enable the much-needed presence of tangible human connection to the virtual world. One can say that Avatars are a digital reflection of self, except we control our reflection.

Since Avatars are digital reflection of self, thus avatars that distantly resemble the user is not sufficient. Further the experience in video games, shopping is also not satisfactory. Thus, there is a dire need to solve the issue of generating the avatars that reflects the most accurate form of the user or object. Though there are plenty of devices and service providers that allow one to create their own avatar but there is no solution that is simple, more efficient (cost wise and time wise) and still develop a avatars that reflects the most accurate form of the user or object.

The present disclosure provides a system, device and a process that generates avatars that reflects the most accurate form of the user or object. Further the present disclosure also provides a cost and time efficient device and process for generation of avatars that that reflects the most accurate form of the user or object. The present disclosure provides a method and system to generate 3-D avatar with minimal user intervention and commands.

BRIEF DESCRIPTION OF DRAWINGS

Some embodiments are illustrated by way of example and not limitation in the figures of the accompanying drawings in which:

Figure 1 - is a block diagram exemplifying computing environment suitable for using embodiments of the present invention
Figure 2 - is a flow chart exemplifying method for generating 3-D avatar reflecting a user’s or object’s appearance in accordance with the disclosure of present specification.

Figure 3 – is a illustration of the system of generating 3-D avatar

Figure 4 – is the image of Image Capturing Device

Figure 5 – is the 3-D generated Avatar of an object

Figure 6 (a-c) – is the 3-D generated avatar of a person or user

SUMMARY OF THE INVENTION

The present disclosure complements the challenge and problem described above. The present disclosure provides systems, methods, and instructions to generate Avatars that are ‘as-is’ or at least resembles near to the most accurate form of the user’s or object’s appearance.

Embodiments of the present disclosure relate to systems, methods, and computer media for generating an avatar reflecting a user’s or object(s)’ most accurate appearance. Data describing user’s or object’s current appearance is received with the help of image receiving unit comprising of at least two cameras. The data includes a visible spectrum image of the user or the object and at least an IR image of the user or the object. The visible spectrum image data is received using RGB camera and IR data of the user is obtained using an IR camera. The IR data is used to further supplement the RGB data of the image and helps to generate the most accurate avatar of the user or object. Once the data is received, the data is sent to a processor where sequential machine learning models are applied to obtain the most accurate avatar of the user. A 3D avatar resembling the user or object is generated by combining the captured RGB and IR image and applying sequential machine learning models and is displayed on the out-put unit.

Embodiments of the present disclosure provides an avatar that may be as simple as a picture of the user/object or may be a graphical object that may represent the user's actions, beliefs, interests, appearance, identity, personality, and the like. An avatar may further be animated or customized. In addition to representing the characteristics recited above, a user may opt to utilize an avatar for number of applications including but not limited to video games, VR games, metaverse experience, cosmetics consulting, clothing consulting, virtual meetings, personal holograms, enhanced shopping experience and like.

BRIEF DESCRIPTION OF THE INVENTION

It is an aspect of present disclosure to provide computer implemented method for generating a 3D avatar of the user, wherein such method comprises following steps:

1. Placing user or object in front of image receiving device;
2. Image receiving device, receiving the image data both RGB and Infrared;
3. processing the image and transferring the data to the processing unit;
4. processing unit applies sequential machine learning models to stitch the most accurate 3D avatar of the user or object;
5. displaying 3D avatar to the user via out-put unit;
6. optionally, further processing the 3D avatar to change its clothes, hair color and other cosmetic looks as user deems fit;
7. further optionally using and exporting the 3D avatar obtained for various applications like makeup testing, clothing, gaming and like.

In yet another aspect of the disclosure that object or user in step 1 is placed at a distance of 0.5-2 meter, 1-2 meters and preferably 1 metre or less from the image receiving device.

In another aspect of the disclosure the image receiving device of step 1 comprises at least two cameras, one RGB camera and second an infrared camera.

In another aspect of the disclosure the processing unit as described above applies at-least 3 machine learning models in following sequence:

a. High fidelity face texture extraction: This model provides face texture by mapping the texture information obtained from the images. High image grey scale map enables creating photorealistic representation of the user or object. Once face texture is obtained following machine learning model is applied;
b. Generating Head and Stitching Face Texture: This uses a head generating model and stitches the face to the head. In this step facial components like ears, neck hair are also added and stitched to the face;
c. Generating full body and Stitching Head generated: Model is applied in this to obtain full body that resembles the body appearance of the user or object. Following this model 3D avatar is obtained it is exported and displayed on the out-put device or unit.

It is another aspect of the disclosure to provide a simple computing system for generating a 3-D avatar reflecting or resembling a user’s or object’s most accurate current appearance; where in such system comprises:
a. the image capturing device (A);
b. computing device (B) and;
c. out-put component or unit (C).

It is yet another aspect of the disclosure that the image capturing device (A) comprises at least two cameras, one RGB camera and second an infrared camera.

It is yet another aspect of the disclosure that out-put unit allows the user to view, export and further customise the 3-D avatar thus obtained.

It is yet another aspect of the disclosure to provide cost-efficient method and system to prepare 3-D avatar of user or object.

It is another aspect of the disclosure to generate a 3-D avatar of the user or object with minimal human intervention.

It is another aspect of the disclosure to provide process to generate a 3-D avatar of the user or object with minimal human intervention, wherein such process comprises:
a. placing user or object in front of an image capturing device;
b. allowing the image capturing device to capture the image of the object;
c. using sequential machine learning models for generating 3-D avatar;
d. displaying 3-D avatar thus obtained;
e. optionally store the 3-D avatar thus obtained; and
f. optionally exporting or customising the 3-D avatar thus obtained.

It is another aspect of the present disclosure that steps b-d are automatic in nature.

DETAILED DESCRIPTION

Embodiments of the present disclosure are described with specificity herein to meet statutory requirements. However, the description itself is not intended to limit the scope of this specification. Rather, the inventors have contemplated that the claimed subject matter might also be embodied in other ways, to include different steps or combinations of steps similar to the ones described in this document, in conjunction with other present or future technologies. Moreover, although the terms “step” and/or “block” or “module” etc. might be used herein to connote different components of methods or systems employed, the terms should not be interpreted as implying any particular order among or between various steps herein disclosed unless and except when the order of individual steps is explicitly described.

Embodiments of the present disclosure may be described in the general context of computer code or machine-useable instructions, including computer-executable instructions such as program modules, being executed by a computer or other machine, such as a personal data assistant or other handheld device. Generally, program modules including routines, programs, objects, components, data structures, etc., refer to code that perform particular tasks or implement particular abstract data types. Embodiments of the present disclosure may be practiced in a variety of system configurations, including hand-held devices, consumer electronics, general-purpose computers, more specialty computing devices, etc. Embodiments of the present disclosure may also be practiced in distributed computing environments where tasks are performed by remote-processing devices that are linked through a communications network.

Embodiments of the present disclosure relate to systems, methods, and computer media for generating an avatar reflecting or resembling a user’s most accurate current appearance. Various aspects of the user’s appearance are captured based on the data and combined with predetermined avatar features to generate a 3D avatar resembling the user.

In an embodiment of the present disclosure, an image capture device is provided that captures the data of the user.

In a preferred embodiment the present disclosure provides an image capturing device that comprise at least two image capturing cameras, wherein one camera is RGB camera to captures colours or visible spectrum details while the other is IR (infrared camera) to capture the non-visible parameters or images of the user.

In an embodiment the image capture device captures the data of user’s face and optionally of user’s body or body parts, clothes, hair, skin and other relevant details.

In yet another embodiment the image capturing device (A) is linked to data processing unit or computing device (B). The computing device processes the data received and uses a Sequential Machine learning algorithms or models to stiches the real like 3D avatar of the user.

In yet another embodiment the computing device is also able to store the computer readable media.

By way of example, and not limitation, computer-readable media may comprise computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computing device .

In yet another embodiment user means any person who wishes to use the system and process of the instant invention to make his or her own avatar, or avatar of any other person or an object,
Communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave. The term “modulated data signal” refers to a propagated signal that has one or more of its characteristics set or changed to encode information in the signal. By way of example, and not limitation, communication media includes wired media, such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, radio, microwave, spread-spectrum, and other wireless media. Combinations of the above are included within the scope of computer-readable media.

Memory includes computer storage media in the form of volatile and/or nonvolatile memory. The memory may be removable, nonremovable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical-disc drives, etc. Computing device includes one or more processors that read data from various entities such as memory or I/O components. Presentation component(s) present data indications to a user or other device. Exemplary presentation components include a display device, speaker, printing component, vibrating component, etc.

I/O ports allow computing device to be logically coupled to other devices including I/O components, some of which may be built in. Illustrative components include a microphone, joystick, game pad, satellite dish, scanner, printer, wireless device, etc.

In another embodiment the present disclosure provides an out-put component or unit (C), that allows the user to view the 3-D generated avatar as per the present disclosure. Further out-out unit also allows the user to download, export and further customise the 3-D avatar as per personal choice.

In another embodiments, data from user’s current appearance is only received for the front of the user. Data for the sides and back of the user is inferred using various machine learning algorithms. In other embodiments, images of the user from multiple angles or sides are received, and feature recognition is performed for each side of a user 's head and body.

In another embodiment once a 3D avatar is generated, user can further customise it as per his or her choice using various computing commands.

In yet another embodiment in the present disclosure the user or object for which 3D avatar is to be generated is placed within the distance of 0.5-2 meter, 1-2 meters and preferably 1 metre or less from the image capturing device.

In yet another embodiment the present disclosure provides computer implemented method for generating a 3D avatar of the user, wherein such method comprises following steps:

a) Placing user or object in front of image receiving device;
b) receiving the image data using image receiving device;
c) processing the image and transferring or communicating the data to the processing unit;
d) processing unit applies sequential machine learning models to stitch the most accurate 3D avatar of the user or object; the processing unit communicates this to out-put unit;
e) displaying 3D avatar to the user via out-put unit;
f) optionally, further processing the 3D avatar to change its clothes, hair color and other cosmetic looks as user deems fit;
g) optionally storing 3-D avatar on computer storage media;
h) further optionally using the 3D avatar obtained for various applications like makeup testing, clothing, gaming and like.

In another embodiment the object or user in step 1 is placed at a distance of 0.5-2 meter, 1-2 meters and preferably 1 metre or less from the camera.

In another embodiment the image receiving device of step 1 comprise atleast two cameras, one RGB camera and second an infrared camera.

In yet another embodiment the processing unit applies at-least 3 machine learning models in following sequence:

d. High fidelity face texture extraction: This model provides face texture by mapping the texture information obtained from the images. High image grey scale map enables creating photorealistic representation of the user or object. Once face texture is obtained following machine learning model is applied;
e. Generating Head and Stitching Face Texture: This uses a head generating model and stitches the face to the head. In this step facial components like ears, neck hair are also added and stitched to the face;
f. Generating full body and Stitching Head generated: Model is applied in this to obtain full body that resembles the body appearance of the user or object. Following this model 3D avatar is obtained it is exported and displayed on the out-put device or unit.

It is yet another embodiment of the disclosure to provide cost-efficient method and system to prepare 3-D avatar of user or object.

It is another embodiment of the disclosure to generate a 3-D avatar of the user or object with minimal human intervention.

It is another embodiment of the disclosure to provide process to generate a 3-D avatar of the user or object with minimal human intervention, wherein such process comprises:
a. placing user or object in front of the image capturing device;
b. allowing the image capturing device to capture the image of the object;
c. using sequential machine learning models for generating 3-D avatar;
d. displaying 3-D avatar thus obtained; and
e. optionally exporting or customising the 3-D avatar thus obtained.

It is another aspect of the present disclosure that steps b-d are automatic in nature.

It yet another embodiment the present disclosure provides a method to enhance user experience in virtual reality by generating the 3-D avatar following the methods and systems of the instant invention.

In yet another embodiment the out-put unit further comprises options for the user to customise the generated 3-D avatar as per the instant inveiton.

In yet another aspect the present disclosure provides the application of 3-D avatar generated in multiple industries including cosmetics, fashion, video-gaming, virtual reality and virtual meetings.

The present disclosure has been described in relation to particular embodiments, which are intended in all respects to be illustrative rather than restrictive. Alternative embodiments will become apparent to those of ordinary skill in the art to which the present disclosure pertains without departing from its scope.

Examples:
The present disclosure will be described with reference to examples and the like, but the present disclosure is not limited to these examples.

Example 1: The 3-D avatar of a person’s face:

The system involves placing a person in front of the image capturing device. The device, equipped with RGB and IR cameras, captures detailed visual and infrared images. These images are processed by the processing unit using machine learning models to extract facial textures, generate a 3-D head, and finally create a full-body avatar. This avatar is displayed on the output unit, where users can interact and customise their digital representation.
The process followed is illustrated in Figure 2. The output obtained is produced in Figure 6 (a-c)

Example 2: The 3-D avatar of an object

The process and system to obtain the avatar is followed as per the instant invention. The output obtained is produced in Figure 5.

Conclusion:
From the above examples it can be observed that the 3-D Avatars obtained as per the instant invention are of high quality and closely resemble the user or object, thus providing user satisfaction and multi-dimensional application of the avatar.

The user friendly system of the instant invention, also enable a low-skilled user also to use the system and have a satisfactory output. The invention also has proven to be highly efficient and more effective, thus saving time and labour cost.
,CLAIMS:We Claim,
1. A computer implemented method for generating a 3D avatar of a user or an object, wherein such method comprises following steps:

i) Placing user or object in front of image receiving device;
j) receiving the image data using image receiving device;
k) processing the image and transferring or communicating the data to the processing unit;
l) processing unit applies sequential machine learning models to stitch the most accurate 3D avatar of the user or object; the processing unit communicates this to out-put unit;
m) displaying 3D avatar to the user via out-put unit;
n) optionally, further processing the 3D avatar to change its clothes, hair color and other cosmetic looks as user deems fit;
o) optionally storing 3-D avatar on computer storage media;

2. The object or user as claimed in claim 1 is placed at a distance of 0.5-2 meter, 1-2 meters and preferably 1 metre or less from the camera.

3. The camera as claimed in claim 1 comprises at least two cameras, one RGB camera and second an infrared camera.

4. The processing unit as claimed in claim 1 further applies at-least 3 machine learning models in following sequence:

g. High fidelity face texture extraction: This model provides face texture by mapping the texture information obtained from the images. High image grey scale map enables creating photorealistic representation of the user or object. Once face texture is obtained following machine learning model is applied;
h. Generating Head and Stitching Face Texture: This uses a head generating model and stitches the face to the head. In this step facial components like ears, neck hair are also added and stitched to the face;
i. Generating full body and Stitching Head generated: Model is applied in this to obtain full body that resembles the body appearance of the user or object. Following this model 3D avatar is obtained it is exported and displayed on the out-put device or unit.

5. A process to generate a 3-D avatar of the user or object with minimal human intervention, wherein such process comprises:
f. placing user or object in front of the image capturing device;
g. allowing the image capturing device to capture the image of the object;
h. using sequential machine learning models for generating 3-D avatar;
i. displaying 3-D avatar thus obtained; and
j. optionally exporting or customising the 3-D avatar thus obtained.

wherein the steps b-d are automatic in nature.

6. A computing system for generating a 3-D avatar of a user or object where in such system comprises:
d. the image capturing device (A);
e. computing device (B) and;
f. out-put component or unit (C).

7. The image capturing device (A) as claimed in claim 6 comprises at least two cameras, one RGB camera and second an infrared camera.

8. The out-put unit as claimed in claim 6 further comprises options for the user to customise the generated 3-D avatar.

9. A process for generating a 3-D avatar of a user or object using computing system as claimed in claim 6 comprises:
a. Placing user or object in front of image receiving device;
b. receiving the image data using image receiving device;
c. processing the image and transferring or communicating the data to the processing unit;
d. processing unit applies sequential machine learning models to stitch the most accurate 3D avatar of the user or object; the processing unit communicates this to out-put unit;
e. displaying 3D avatar to the user via out-put unit;
f. optionally, further processing the 3D avatar to change its clothes, hair color and other cosmetic looks as user deems fit;
g. optionally storing 3-D avatar on computer storage media;

10. The object or user as claimed in claim 6-9 is placed at a distance of 0.5-2 meter, 1-2 meters and preferably 1 metre or less from the camera.

11. A method to enhance a user experience in virtual reality by following the steps as claimed in claim 1 and claim 5.

12. A method to enhance a user experience in virtual reality by using the system as claimed in claim 6.