Description:BACKGROUND
Technical Field
[0001] The embodiments herein generally relate to three-dimensional (3D) video calling, more particularly to a system and a method for generating a three-dimensional (3D) image from a two-dimensional (2D) image for enabling the3D video calling in an electronic device using a machine learning model.
Description of the Related Art
[0002] Existing video calling typically happens only in two-dimensional (2D) format. The 2D formatvideo calling may feel less dynamic than three-dimensional (3D) video calling. The quality experience of existing 2D video calling is very low especially if the users who are having the 2D video calling are at long distances. There is noexisting solution available that enables3D video calling for virtual online classrooms and interviewsas the 3D video calling mechanisms may require a separate physical environment.
[0003] Some existing solutions employedholographic mechanisms for video calling.Holography is a photographic principle that is used to construct three-dimensional images that are closer to reality.In a holographic mechanism,the wave is recorded and later reconstructed to form a 3D image.
[0004] Further, holographic video callingrequires a physical environment for recording and reconstruction of the 3D image. Practical implementation feasibility of holographic video calling is very low as holographic video calling is less compatible with current existing devices.Even, Google’s Project Starlinealso requires an expensive physical device,which is currently in development. The implementation of Google’s Project Starline is also expensive and less compatible with current existing devices. Another existing solution incorporates haptic technology in the 3D video calling that helps in enabling a sense of touch and feel in 3D video calling,however,the implementation of the haptic technology is more expensive.
[0005] Accordingly, there is a need for a system for generating a three-dimensional (3D) image from a two-dimensional (2D) image that overcomes the aforementioned drawback and disadvantages of the existing systems.
SUMMARY
[0006] In view of the foregoing, an embodiment herein provides a system forgenerating a three-dimensional (3D) image from a two-dimensional (2D) image in an electronic device. The system is communicatively connected with the electronic device for generating the 3D image. The system includes a sensor and a processor. The sensor is configured to receive atwo-dimensional (2D) image from a sender deviceand generate a pixel matrix for the 2D image. The processor is communicatively connected to the sensor. The processor implements a machine learning model that (i) identifies object image componets and background image components and spatial boundaries from a pixel matrix associated with the 2D image by detecting object boundaries using an edge detection technique; (ii) identifies a pixel color of the spatial boundaries; (iii) determines a spatial depth of the spatial boundaries; (iv) generatesthe 2D image with transparent background by separatingthe object image components from the background image components using a background subtraction method; (v) incorporates the spatial depth information intothe pixel matrix of the 2D image with the transparent background by matching a first color of the spatial depth with a second color of the spatial boundaries; and (vi) implements a digital image processing technique that generates a three-dimension (3D) image from the 2D image with the transparent background for enabling a 3D video calling in the electronic device.
[0007] In some embodiments, the system includes a rotary filterthat is communicatively connected to the processor. The rotary filter generates one or more RGB pixels by passing thelight associated with the 2D image through the rotary filter.When light strikes the rotary filter, the rotary filter enables the one or moreRGB pixels to pass through, if the third color of the one or more RGB pixelsmatches a fourthcolor of the rotary filter.
[0008] In some embodiments, the system includes a microscopic mirrorthat generates pixelsfor the 3D image by (i) merging the one or moreRGB pixels and (ii) moving the microscopic mirroraccording to the RGB components of the 2D image.The movement of the microscopic mirroris dynamically tuned according to the RGB components of the 2D image.
[0009] In some embodiments,theobject boundaries are detected by (i) comparing values of each pixel from the pixel matrix of the 2D image with its surrounding pixels and (ii) determining one or more pixels that lie on an edge if there is a significant change in pixel values.
[0010] In some embodiments, the processor determinesthe spatial depth of the spatial boundaries by (i) capturing, using at least two cameras, a stereo image pair including a left and a right stereo images of the 2D image; (ii) identifying a correspondence between pixels in the left and the right stereo images using a block matching technique or a feature matching technique after pre-processing the left and the right stereo images; (iii) determining a spatial depth of each pixel in the 2D image by identifying disparities in the pixel positions between the correspondences in the left and the rightstereo images, where the spatial depth of each pixel is inversely proportional to the disparity; and (iv) refining the spatial depth using any one of hole filling, smoothing, or surface fitting.
[0011] In some embodiments, the processor is configured to incorporatethe spatial depth information intothe pixel matrix of the 2D imagewith the transparent background by any one of (i) generating a depth map image by encoding the spatial depth information for each pixel in the 2D image with the transparent background, wherein the intensity of each pixel of the 2D image with the transparent background represents its corresponding spatial depth information; (ii)encoding the spatial depth information into color channels of the 2D image with the transparent background. (iii) encoding the spatial depth information into the 2D image with transparent background as a floating-point value for each pixel; or (iv) generating a depth map image including the spatial depth information as an auxiliary image and combining the depth map image with the 2D imageusing compositing techniques.
[0012] In some embodiments, the sensor includes a segregator and a pocket sorter. The segregator separates audio packets and image packets from the 2Dimage that is decoded by a decoder of the sender device. The pocket sorter receives the image packets and processes the image packets to generate the pixel matrix for the 2Dimage that is decoded.
[0013] In one aspect, a method of generating a three-dimensional (3D) image from a two-dimensional image (2D) image in an electronic deviceusing a system is provided. The system is communicatively connected with the electronic device for generating the 3D image. The method includes (i) receiving, using a sensor of the system, atwo-dimensional (2D) image from a sender device; (ii) generating, using the sensor of the system, a pixel matrix for the 2D image for generating a 3D image; and (iii)implementing, using a processor of the system, a machine learning model that (a) identifies object image componentsand background image components and spatial boundaries from the pixel matrix associated with the 2D image by detecting object boundaries using an edge detection technique; (b) identifies a pixel color of the spatial boundaries; (c) determines a spatial depth of the spatial boundaries; (d) generates a 2D image with transparent background by separatingthe object image components from the background image components using a background subtraction method; (e) incorporates the spatial depth information intothe pixel matrix of the 2D image with the transparent background by matching a second color of the spatial depth with a third color of the spatial boundaries; and(f) implements a digital image processing technique that generates a three-dimension (3D) image from the 2D image with the transparent background for enabling a 3D video calling in the electronic device.
[0014] In some embodiments, the method includes generating, using a rotary filterof the system,one or moreRGB pixels by passing thelight associated with the 2D image through the rotary filter, wherein when the light strikes the rotary filter, the rotary filterenables the one or moreRGB pixels to pass through if athird color of the one or moreRGB pixelsmatches a fourthcolor of the rotating filter.
[0015] In some embodiments, the method includes generating, using a microscopic mirrorof the system,pixelsfor the 3D image by (i) merging the one or moreRGB pixels and (ii) moving the microscopic mirror according to RGB components of the 2D image.The movement of the microscopic mirroris dynamically tuned according to the RGB components of the 2D image.
[0016] The system enhances the communication experienceof users during the 3D video calling and does not require a separate physical environment for performing the 3D video calling. The system is compatible with existing smartphones and practically feasible to implement in the existing smartphones for performing the 3D video calling. The system is cost-effective as it does not require a separate physical environment for performing the 3D video calling.
[0017] These and other aspects of the embodiments herein will be better appreciated and understood when considered in conjunction with the following description and the accompanying drawings.It should be understood, however, that the following descriptions, while indicating preferred embodiments and numerous specific details thereof, are given by way of illustration and not of limitation.Many changes and modifications may be made within the scope of the embodiments herein without departing from the spirit thereof, and the embodiments herein include all such modifications.
BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The embodiments herein will be better understood from the following detailed description with reference to the drawings, in which:
[0019] FIG. 1illustrates a block diagram of a system for generating a three-dimensional (3D) image from a two-dimensional (2D) image in an electronic deviceaccording to an embodiment herein;
[0020] FIG. 2illustrates a block diagram of a system including a rotary filter and a microscopic mirror for generating a three-dimensional (3D) image from a two-dimensional (2D) image in an electronic deviceaccording to an embodiment herein;
[0021] FIG. 3illustrates a process flowof a system for generating a three-dimensional (3D) image from a two-dimensional (2D) image in an electronic deviceaccording to an embodiment herein;
[0022] FIG. 4illustrates a block diagram of a sensor of a system for generating a pixel matrix fora two-dimensional (2D) image according to an embodiment herein;
[0023] FIGS. 5A-5Billustrate a visual representation of a process for generating a three-dimensional (3D) image from a two-dimensional (2D) image using a system according to an embodiment herein;
[0024] FIG. 6is an exemplary illustration of an edge detection techniquethat detects object boundaries of a two-dimensional (2D) image according to an embodiment herein;
[0025] FIG. 7is an exemplary illustration of a background subtraction method that generatesa two-dimensional (2D) image with transparent background according to an embodiment herein;
[0026] FIG. 8isa flow diagram illustrating a method of generating a three-dimensional (3D) image from a two-dimensional image (2D) image in an electronic deviceusing a system according to an embodiment herein; and
[0027] FIG. 9 is a block diagram of a schematic diagram of a device used (e.g. system) in accordance with embodiments herein.
DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
[0028] The embodiments herein and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known components and processing techniques are omitted so as to not unnecessarily obscure the embodiments herein.The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein may be practiced and to further enable those of skill in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.
[0029] As mentioned, there remains a need for a system and a method for generating a three-dimensional (3D) image from a two-dimensional (2D) image in an electronic device.Various embodiments disclosed herein providea system for generating a three-dimensional (3D) image from a two-dimensional (2D) image in an electronic device. Referring now to the drawings, and more particularly throughFIG. 1 to FIG. 9, where similar reference characters denote corresponding features consistently throughout the figures, preferred embodiments are shown.
[0030] FIG. 1illustrates a block diagram of a system 100 for generating a three-dimensional (3D) image from a two-dimensional (2D) image in an electronic device 108 according to an embodiment herein.The system 100 is communicatively connected with the electronic device108 for generating the 3D image. The system 100 includes a sensor102 and a processor 104. The sensor 102 is configured to receive atwo-dimensional (2D) image from a sender device110 and generate a pixel matrix for the 2D image. The processor 104 is communicatively connected to the sensor 102. The processor 104 implements a machine learning model 106 that (i) identifies object image components and background image components and spatial boundaries from a pixel matrix associated with the 2D image by detecting object boundaries using an edge detection technique; (ii) identifies a pixel color of the spatial boundaries; (iii) determines a spatial depth of the spatial boundaries; (iv) generatesthe 2D image with transparent background by separatingthe object image components from the background image components using a background subtraction method; (v) incorporates the spatial depth information intothe pixel matrix of the 2D image with the transparent background by matching a first color of the spatial depth with a second color of the spatial boundaries; and (vi) implements a digital image processing technique that generates a three-dimension (3D) image from the 2D image with the transparent background for enabling a 3D video calling in the electronic device 108. In some embodiments, the system is a plug-and-play device.
[0031] In some embodiments, theobject boundaries are detected by (i) comparing values of each pixel from the pixel matrix of the 2D image with its surrounding pixels and (ii) determining one or more pixels that lie on an edge if there is a significant change in the pixel values.
[0032] In some embodiments, the processor 104 determinesthe spatial depth of the spatial boundaries by (i) capturing, using at least two cameras, a stereo image pair including a left and a right stereo images of the 2D image; (ii) identifying a correspondence between pixels in the left and the right stereo images using a block matching technique or a feature matching technique after pre-processing the left and the right stereo images; (iii) determining a spatial depth of each pixel in the 2D image by identifying disparities in the pixel positions between the correspondences in the left and the rightstereo images, where the spatial depth of each pixel is inversely proportional to the disparity; and (iv) refining the spatial depth using any one of hole filling, smoothing, or surface fitting.
[0033] In some embodiments, the processor 104 identifies thepixel color of the spatial boundaries using any one of (i) RGB values, (ii) hue, saturation, lightness values, (iii) color histograms, (iv) K-means clustering, or (v) mean shift.
[0034] In some embodiments, the RGB valuesuse the RGB (red, green, blue) values of each pixel. Each pixel in an image (e.g., 2D image) is made up of three 8-bit color channels (red, green, and blue), which allows for a total of 16,777,216 possible colors. The RGB values of a pixel can be determined by reading the values of the red, green, and blue channels at that specific location.In some embodiments, the hue, saturation, lightness (HSL) valuesuse the HSV (hue, saturation, value) or HSL (hue, saturation, lightness) values of each pixel. These color models are similar to RGB, and the color models represent colors differently. In HSV, hue represents the dominant wavelength of light, saturation represents the purity of the color, and the value represents the brightness of the color. In HSL, lightness is a measure of whether a color is light or dark.
[0035] In some embodiments, thecolor histogramsarethe graphical representation of the distribution of colors in an image (e.g., 2D image). The color histograms plot a number of pixels in the 2D image with a certain color value. The color histograms are used to identify the dominant colors in the 2D image and can be useful for tasks such as object recognition and image retrieval.
[0036] In some embodiments, the K-means clustering is a method of vector quantization that aims to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean. The K-means clustering method is a way to classify a given data set into a fixed number of clusters (e.g., k clusters) in which each observation belongs to the cluster with the nearest mean. In some embodiments, the mean shift is a non-parametric method for density estimation, which is used to find the modes of a density function given discrete data samples. The mean shift is used to track the centroid of a dense region of points.
[0037] In some embodiments, the processor 104 incorporates the spatial depth information intothe pixel matrix of the 2D imagewith transparent background by any one of (i) generating a depth map image by encoding the spatial depth information for each pixel in the 2D image with transparent background, wherethe intensity of each pixel of the 2D image with transparent background represents its corresponding spatial depth information; (ii)encoding the spatial depth information into color channels of the 2D image with transparent background; (iii) encoding the spatial depth information into the 2D image with transparent background as a floating-point value for each pixel; or (iv) generating a depth map image including the spatial depth information as an auxiliary image and combining the depth map image with the 2D imageusing compositing techniques.
[0038] In some embodiments, the sender device 110 includes a decoder that decodes the 2D image before sending it to the sensor 102. The image may include a frame of a video.In some embodiments, the sensor 102 includes a segregator and a pocket sorter. The segregator separates audio packets and image packets from the 2D image that is decoded by the decoder of the sender device 110. The pocket sorter receives the image packets and processes the image packets to generate the pixel matrix for the 2D image that is decoded.
[0039] The digital image processing technique is a technique that is used to sharpen object boundaries in the image (e.g., 2D image) by enhancing the edges and increasing the overall contrast. One common method of achieving this is through the use of unsharp masking. The unsharp masking works by subtracting a blurred version of the 2D image from the original 2D image. The blurred version of the 2D image is typically created by convolving the original 2D image with a Gaussian kernel. By subtracting the blurred 2D image from the original 2D image, the edges and fine details are emphasized and the overall image appears sharper. The unsharp masking may improve the visibility of fine details in the 2D image by an average of 15%. Further, the unsharp masking may increase the accuracy of object detection in a video surveillance system by 5%. In some embodiments, a Laplacian of Gaussian (LoG) technique is used to sharpen object boundaries in the 2D image. This technique is based on convolving the image with a kernel that represents the second derivative of a Gaussian function. It is commonly used for edge detection. Both the unsharp masking and the Laplacian of Gaussian techniques are widely used techniques and effectively sharpen object boundaries in images. The specific results may vary depending on the specific application and the quality of the original 2D image.
[0040] The system can be used for one-on-one video conference/virtual meetings where one can see another exactly standing/sitting in front of one like real communication. The system can be used for one to many video conference/virtual meetings. One-to-many video conference/virtual meetings are similar to broadcasting. If one user is talking, all other users are listeners. Here,the broadcaster can be seen as real in front of the listeners. The system can be used for many to many video conference/virtual meetings. In this, the one who is currently communicating is focused and can be seen real as like a classroom. The system can be used for interviews (i.e., many-to-one video conference/virtual meetings). Interviews are where many ask questions to a single one, in which the interviewee can see a real interview board in front of him/her with the help of the system and can communicate accordingly, while the interviewers can see only the interviewee. The system can be used for many-to-one video conference/virtual meetings (where there is no communication needed between many).This many-to-one video conference/virtual meetings, it's similar to an interview but all the interviewers cannot connect, can only see the interviewee and the interviewee can see all the interviewers. In this, there is no communication between the interviewers or they cannot see each other here. The system can be also used for virtual classrooms. This is a similar use case to one (e.g., teacher) to many (e.g.,students)video conference/virtual meetings,but here, the listeners can ask questions and let the teacher answer them. Hence, the communication between teacher-student and student-teacher is only allowed andthe student-student communication is not allowed. When one student asks doubts, all the other students can listen to it.
[0041] FIG. 2illustrates a block diagram of a system 100 including a rotary filter 202 and a microscopic mirror 204 for generating a three-dimensional (3D) image from a two-dimensional (2D) image in an electronic device 108 according to an embodiment herein.The system 100 is communicatively connected with the electronic device 108 for generating the 3D image. The system 100 includes a sensor 102 and a processor 104. The sensor 102 is configured to receive atwo-dimensional (2D) image from a sender device 110 and generate a pixel matrix for the 2D image. The processor 104 is communicatively connected to the sensor 102. The processor 104 implements a machine learning model 106 that (i) identifies object and background image components and spatial boundaries from the pixel matrix associated with the 2D image by detecting object boundaries using an edge detection technique; (ii) identifies a pixel color of the spatial boundaries; (iii) determines a spatial depth of the spatial boundaries; (iv) generatesthe 2D image with transparent background by separatingthe object image components from the background image components using a background subtraction method; (v) incorporates the spatial depth information intothe pixel matrix of the 2D image with transparent background by matching a first color of the spatial depth with a second color of the spatial boundaries; and (vi) implements a digital image processing technique that generates a three-dimensional (3D) image from the 2D image with the transparent background for enabling a 3D video calling in the electronic device 108.
[0042] The system 100 includes a rotary filter 202 and a microscopic mirror 204. The rotary filter 202 is communicatively connected to the processor 104. The rotary filter 202 generates one or moreRGB pixels by passing the light associated with the 2D image through the rotary filter 202. When the light strikes the rotary filter 202, the rotary filter 202 enables the one or moreRGB pixels to pass through if the third color of the one or moreRGB pixels matches the fourth color of the rotary filter 202. The microscopic mirror 204 generates pixels for the 3D image by (i) merging the one or moreRGB pixels and (ii) moving the microscopic mirror 204 according to the RGB components of the 2D image. The movement of the microscopic mirror 204 is dynamically tuned according to the RGB components of the 2D image.
[0043] In some embodiments, the rotating filter 202 includesthreecolors (i.e., red, green, and blue). The rotation of therotating filter 202 is based on the received pixel matrix of the image. When the light hits therotating filter 202, and if the thirdcolor of the RGB pixel matches the fourthcolor of the rotating filter 202, thepixel color passes through the rotating filter 202 and is blocked if otherwise.
[0044] In some embodiments, the microscopic mirror 204 is a conceptual mirror thatincludes severalhundreds or thousands of tiny electromechanical tiltable mirrors arranged in agrid. The tilting of the tiny mirror controls the formation of image pixels, i.e., thelight passing through the rotating filter 202 is converted into a pixel if the light getsenabled by the microscopic mirror 204. The tilting of the specific tiny mirror(s) is basedon the received pixel matrix of the image. These relatively small-sized mirrors are cost-effective and help to reduce the size and weight of the system without compromising on the quality of the rendered final output(i.e., the 3D image).
[0045] FIG. 3illustrates a process flowof a system 100 for generating a three-dimensional (3D) image from a two-dimensional (2D) image in an electronic device 108 according to an embodiment herein. At a step 302, a sender device 110 sends a two-dimensional (2D) image to a decoder 320. The sender device 110 may includethe decoder 320.At a step 304, the decoder 320 decodes the 2D image and sendsthe decoded image to a sensor 102.At a step 306, thesensor 102 generates a pixel matrix for the 2D image. At a step 308, a processor104 of the system 100 implements a machine learning model 106 that (i) identifies object/foreground and background image components and spatial boundaries from the pixel matrix associated with the 2D image by detecting object boundaries using an edge detection technique, (ii) identifies a pixel color of the spatial boundaries,and (iii) determines a spatial depth of the spatial boundaries.The processor 104generatesthe 2D image with transparent background by separatingthe object image components from the background image components using a background subtraction method. At a step 310, the processor 104 incorporates the spatial depth information intothe pixel matrix of the 2D image with transparent background by matching a first color of the spatial depth with a second color of the spatial boundaries to form a foundation of a 3D image. At a step 312, the processor 104 sends the projected2Dimage to a rotary filter 202. The rotary filter 202 is communicatively connected to the processor 104. At a step 314,the rotary filter 202 generates one or moreRGB pixels by passing the LED light associated with the 2D image through the rotary filter 202. When the light strikes the rotary filter 202, the rotary filter 202 enables the one or moreRGB pixels to pass through if the third color of the one or moreRGB pixels matches the fourth color of the rotary filter 202. At a step 316,the system 100 includes a microscopic mirror 204 that generates pixels for the 3D image by (i) merging the one or moreRGB pixels and (ii) moving the microscopic mirror 204 according to RGB components of the 2D image. The movement of the microscopic mirror 204 is dynamically tuned according to the RGB components of the 2D image. At a step 318, the system 100implements a digital image processing technique that generates a three-dimensional (3D) image from the 2D image with transparent background for enabling a 3D video calling in the electronic device 108.
[0046] In some embodiments, theobject boundaries are detected by (i) comparing values of each pixel from the pixel matrix of the 2D image with its surrounding pixels and (ii) determining one or more pixels that lie on an edge if there is a significant change in the pixel values.In some embodiments, the processor 104 determinesthe spatial depth of the spatial boundaries by (i) capturing, using at least two cameras, a stereo image pair including a left and a right stereo images of the 2D image; (ii) identifying a correspondence between pixels in the left and right stereo images using a block matching technique or a feature matching technique after pre-processing the left and the right stereo images; (iii) determining a spatial depth of each pixel in the 2D image by identifying disparities in the pixel positions between the correspondences in the left and the rightstereo images, where the spatial depth of each pixel is inversely proportional to the disparity; and (iv) refining the spatial depth using any one of hole filling, smoothing, or surface fitting.
[0047] In some embodiments, the processor 104 identifies thepixel color of the spatial boundaries using any one of (i) RGB values, (ii) hue, saturation, lightness values, (iii) color histograms, (iv) K-means clustering, or (v) mean shift.In some embodiments, the processor 104 incorporates the spatial depth information intothe pixel matrix of the 2D imagewith transparent background by any one of (i) generating a depth map image by encoding the spatial depth information for each pixel in the 2D image with transparent background, where the intensity of each pixel of the 2D image with transparent background represents its corresponding spatial depth information; (ii)encoding the spatial depth information into color channels of the 2D image with transparent background; (iii) encoding the spatial depth information into the 2D image with transparent background as a floating-point value for each pixel; or (iv) generating a depth map image including the spatial depth information as an auxiliary image and combining the depth map image with the 2D image using compositing techniques.
[0048] FIG. 4illustrates a block diagram of a sensor 102 of the system 100 for generating a pixel matrix fora two-dimensional (2D) image according to an embodiment herein.The sensor 102 includes a segregator 402 and a pocket sorter 404. The segregator 402 separates audio packets and image packets from the 2D image that is decoded by the decoder 320 of the sender device 110. The pocket sorter 404 receives the image packets and processes the image packets to generate the pixel matrix for the 2D image that is decoded.
[0049] In some embodiments, the sensor 102 includes a lens and an array of photosensitive elements. When the sensor 102 creates the pixel matrix by (i) converting, using the array of photosensitive elements, the light that is captured through the lens into an electrical signal by generating an electrical charge when the light strikes the photosensitive elements, and (ii) converting the electrical signal generated by each photosensitive element into digital data which is then organized into a grid of pixels to form the pixel matrix for the image. The amount of the electrical charge generated by each photosensitive element is proportional tothe amount of light that strikes that photosensitive element.
[0050] FIGS. 5A-5Billustrate a visual representation of a process for generating a three-dimensional (3D) image from a two-dimensional (2D) image using the system 100according to an embodiment herein.At a step 502, anobject image component/foreground pixel matrix, spatial boundaries/spatial boundary matrix, and a background image component/background pixel matrix areidentified from apixel matrix associated with atwo-dimensional (2D) image (i.e., 2D decoded image)by detecting object boundaries using an edge detection technique.The system is communicatively connected to a sender device 110 that sends the 2Dimage to a decoder 320. The sender device 110 may includethe decoder 320.The decoder 320 decodes the 2D image and sends the 2D decoded image to a sensor 102 of the system 100.The sensor 102 generatesthe pixel matrix for the 2D image.In some embodiments, theobject boundaries are detected by (i) comparing values of each pixel from the pixel matrix of the 2D image with its surrounding pixels and (ii) determining one or more pixels that lie on an edge if there is a significant change in the pixel values.
[0051] At a step 504,a pixel color of the spatial boundaries/spatial boundary matrixis identifiedusing a pixel color identification algorithm 512.In some embodiments, thepixel color of the spatial boundaries is identified using any one of (i) RGB values, (ii) hue, saturation, lightness values, (iii) color histograms, (iv) K-means clustering, or (v) mean shift.At a step 506,a 2D image with transparent background is generatedby separatingthe object image component/foreground pixel matrix from the background image component/background pixel matrix using a background subtraction method.At a step 508,a spatial depth of the spatial boundaries/spatial boundary matrix is determined using a spatial depth adding algorithm. The spatial depth information is incorporated intothe3D pixel matrix of the 2D image with transparent background by matching a first color of the spatial depth with a secondcolor of the spatial boundaries. The process employs a digital image processing technique that is implemented togenerate a three-dimension (3D) image from the 2D image with transparent background for enabling a 3D video calling in anelectronic device 108.
[0052] In some embodiments, the spatial depth of the spatial boundaries is determined by (i) capturing, using at least two cameras, a stereo image pair including a left and a right stereo images of the 2D image; (ii) identifying a correspondence between pixels in the left and right stereo images using a block matching technique or a feature matching technique after pre-processing the left and the right stereo images; (iii) determining a spatial depth of each pixel in the 2D image by identifying disparities in the pixel positions between the correspondences in the left and the rightstereo images, where the spatial depth of each pixel is inversely proportional to the disparity; and (iv) refining the spatial depth using any one of hole filling, smoothing, or surface fitting.
[0053] In some embodiments,the spatial depth information is incorporated intothe pixel matrix of the 2D imagewith the transparent background by any one of (i) generating a depth map image by encoding the spatial depth information for each pixel in the 2D image with transparent background, where the intensity of each pixel of the 2D image with the transparent background represents its corresponding spatial depth information; (ii)encoding the spatial depth information into color channels of the 2D image with the transparent background; (iii) encoding the spatial depth information into the 2D image with the transparent background as a floating-point value for each pixel; or (iv) generating a depth map image including the spatial depth information as an auxiliary image and combining the depth map image with the 2D imageusing compositing techniques.
[0054] FIG. 6is an exemplary illustration of an edge detection techniquethat detects object boundaries of a two-dimensional (2D) image according to an embodiment herein. The edge detection techniqueis an image processing technique for finding the boundaries of an object in the 2D image. The 2D image, asshown in FIG. 6,represents a digit 8 being represented in a pixelformat using the edge detection technique.The object boundaries/edges of the image (e.g., the digit 8) are detected/extracted by (i) comparing values of each pixel from the pixel matrix of the image with its surrounding pixels, and (ii) determining one or more pixels that lie on an edge if there is a significant change in the pixel values.The pixel does not lie on the edge if there is no change in the pixel values.
[0055] FIG. 7is an exemplary illustration of a background subtraction method that generatesa two-dimensional (2D) image with transparent background according to an embodiment herein.The background subtraction method separates thebackground image component from the foreground/object image component ofthe 2D image to generate the 2Dimage with transparent background/without a background.A reference 2D image is provided to the background subtraction method inorder toimprove thequality of the 2Dimage with transparent background.The background subtraction methodis performedby comparing moving parts of a video/image (e.g., 2D image) to a background image and a foreground image.
[0056] FIG. 8isa flow diagram illustrating a method of generating a three-dimensional (3D) image from a two-dimensional image (2D) image in an electronic device 108 using a system 100 according to an embodiment herein.At a step 802, a sensor of the systemreceivesatwo-dimensional (2D) image from a sender device. At a step 804, the sensor of the systemgenerates a pixel matrix for the 2D image for generating a 3D image. At a step 806, a processor of the systemimplements a machine learning model that (a) identifies object and background image components and spatial boundaries from the pixel matrix associated with the 2D image by detecting object boundaries using an edge detection technique; (b) identifies a pixel color of the spatial boundaries; (c) determines a spatial depth of the spatial boundaries; (d) generates a 2D image with transparent background by separatingthe objectimage components from the background image components using a background subtraction method; (e) incorporates the spatial depth information intothe pixel matrix of the 2D image with transparent background by matching a third color of the spatial depth with a fourth color of the spatial boundaries; and (f) implements a digital image processing technique that generates a three-dimension (3D) image from the 2D image with transparent background for enabling a 3D video calling in the electronic device.
[0057] In some embodiments, the method includes generating, using a rotary filter of the system,one or moreRGB pixels by passing thelight associated with the 2D image through the rotary filter. When the light strikes the rotary filter, the rotary filter enables the one or moreRGB pixels to pass through if the third color of the one or moreRGB pixelsmatches the fourth color of the rotating filter.
[0058] In some embodiments, the method includes generating, using a microscopic mirror of the system,pixelsfor the 3D image by (i) merging the one or moreRGB pixels and (ii) moving the microscopic mirror according to RGB components of the 2D image. The movement of the microscopic mirroris dynamically tuned according to the RGB components of the 2D image.
[0059] A representative hardware environment for practicing the embodiments herein is depicted in FIG. 9, with reference to FIGS. 1 through 8. This schematic drawing illustrates a hardware configuration of a server/computer system/ user device in accordance with the embodiments herein. The user device includes at least one processing device 10 and a cryptographic processor 11. The special-purpose CPU 10 and the cryptographic processor (CP) 11 may be interconnected via system bus 14 to various devices such as a random-access memory (RAM) 15, read-only memory (ROM) 16, and an input/output (I/O) adapter 17. The I/O adapter 17 can connect to peripheral devices, such as disk units 12 and tape drives 13, or other program storage devices that are readable by the system. The user device can read the inventive instructions on the program storage devices and follow these instructions to execute the methodology of the embodiments herein. The user device further includes a user interface adapter 20 that connects a keyboard 18, mouse 19, speaker 25, microphone 23, and/or other user interface devices such as a touch screen device (not shown) to the bus 14 to gather user input. Additionally, a communication adapter 21 connects the bus 14 to a data processing network 26, and a display adapter 22 connects the bus 14 to a display device 24, which provides a graphical user interface (GUI) 30 of the output data in accordance with the embodiments herein, or which may be embodied as an output device such as a monitor, printer, or transmitter, for example. Further, a transceiver 27, a signal comparator 28, and a signal converter 29 may be connected with the bus 14 for processing, transmission, receipt, comparison, and conversion of electric or electronic signals.
[0060] The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and/or adapt for various applications such specific embodiments without departing from the generic concept, and, therefore, such adaptations and modifications should and are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation.Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modification within the scope of appended claims.
, Claims:I/We claim:
1.A system (100) for generating a three-dimensional (3D) image from a two-dimensional (2D) image in an electronic device (108), wherein the system (100) is communicatively connected with the electronic device (108) for generating the 3D image, wherein the system (100) comprises:
a sensor (102) that is configured to receive atwo-dimensional (2D) image from a sender device (110) and generate a pixel matrix for the 2D image; and
a processor (104) that is communicatively connected to the sensor (102), wherein the processor (104) implements a machine-learning model(106) that
identifies object image components, background image components, and spatial boundaries from a pixel matrix associated with the 2D image by detecting object boundaries using an edge detection technique;
identifies a pixel color of the spatial boundaries;
determines a spatial depth of the spatial boundaries;
generatesthe 2D image with transparent background by separatingthe object image components from the background image components using a background subtraction method;
characterized in that, incorporates the spatial depth information intothe pixel matrix of the 2D image with the transparent background by matching a first color of the spatial depth with a second color of the spatial boundaries; and
implements a digital image processing technique that generates a three-dimension (3D) image from the 2D image with the transparent background for enabling a 3D video calling in the electronic device (108).

2.The system (100) as claimed in claim 1, wherein the system (100) comprises
a rotary filter (202) that is communicatively connected to the processor (104), wherein the rotary filter (202) generates a plurality of RGB pixels by passing light associated with the 2D image through the rotary filter (202), wherein when the light strikes the rotary filter(202), the rotary filter (202) enables the plurality of RGB pixels to pass through if a thirdcolor of the plurality of RGB pixelsmatches a fourthcolor of the rotary filter (202).

3.The system (100) as claimed in claim 1, wherein the system (100) comprises
a microscopic mirror (204) that generates pixelsfor the 3D image by (i) merging the plurality of RGB pixels and (ii) moving the microscopic mirror (204) according to the RGB components of the 2D image, wherein amovement of the microscopic mirror (204) is dynamically tuned according to the RGB components of the 2D image.

4.The system (100) as claimed in claim 1, wherein theobject boundaries are detected by (i) comparing values of each pixel from the pixel matrix of the 2D image with its surrounding pixels, and (ii) determining one or more pixels that lie on an edge if there is a significant change in pixel values.

5.The system(100) as claimed in claim 1, wherein the processor (104) determinesthe spatial depth of the spatial boundaries by
capturing, using at least two cameras, a stereo image pair comprising a left and a right stereo images of the 2D image;
identifying a correspondence between pixels in the left and the right stereo images using a block matching technique or a feature matching technique after pre-processing the left and the right stereo images;
determining a spatial depth of each pixel in the 2D image by identifying disparities in the pixel positions between the correspondences in the left and the rightstereo images, where the spatial depth of each pixel is inversely proportional to the disparity; and
refining the spatial depth using any one of hole filling, smoothing, or surface fitting.

6.The system (100) as claimed in claim 1, wherein the processor (104) incorporates the spatial depth information intothe pixel matrix of the 2D imagewith the transparent background by any one of
generating a depth map image by encoding the spatial depth information for each pixel in the 2D image with the transparent background, wherein the intensity of each pixel of the 2D image with the transparent background represents its corresponding spatial depth information;
encoding the spatial depth information into color channels of the 2D image with the transparent background;
encoding the spatial depth information into the 2D image with the transparent background as a floating-point value for each pixel; or
generating a depth map image comprising the spatial depth information as an auxiliary image and combining the depth map image with the 2D imageusing compositing techniques.

7.The system (100) as claimed in claim 1, wherein the sensor (102) comprises
a segregator (402) that separates audio packets and image packets from the 2D image that is decoded by a decoder (320) of the sender device (110); and
a pocket sorter (404) that receives the image packets and processes the image packets to generate the pixel matrix for the 2D image that is decoded.

8.A method of generating a three-dimensional (3D) image from a two-dimensional image (2D) image in an electronic device (108) using a system (100), wherein the system (100) is communicatively connected with the electronic device (108) for generating the 3D image, wherein the method comprises:
receiving, using a sensor (102) of the system (100), atwo-dimensional (2D) image from a sender device (110);
generating, using the sensor (102) of the system (100), a pixel matrix for the 2D image for generatingthe3D image; and
implementing, using a processor (104) of the system (100), a machine learning model (106) that
identifies object image components,background image components, and spatial boundaries from the pixel matrix associated with the 2D image by detecting object boundaries using an edge detection technique;
identifies a pixel color of the spatial boundaries;
determines a spatial depth of the spatial boundaries;
generates a 2D image with transparent background by separatingthe object image components from the background image components using a background subtraction method;
characterized in that, incorporates the spatial depth information intothe pixel matrix of the 2D image with the transparent background by matching a first color of the spatial depth with a second color of the spatial boundaries; and
implements a digital image processing technique that generates a three-dimensional (3D) image from the 2D image with the transparent background for enabling a 3D video calling in the electronic device (108).

9.The method as claimed in claim 8, wherein the method comprises
generating, using a rotary filter (202) of the system (100), a plurality of RGB pixels by passing light associated with the 2D image through the rotary filter (202), wherein when the light strikes the rotary filter (202), the rotary filter (202) enables the plurality of RGB pixels to pass through if a thirdcolor of the plurality of RGB pixelsmatches a fourthcolor of the rotary filter (202).

10.The method as claimed in claim 8, wherein the method comprises
generating, using a microscopic mirror (204) of the system (100),pixelsfor the 3D image by (i) merging the plurality of RGB pixels and (ii) moving the microscopic mirror (204) according to RGB components of the 2D image, wherein the movement of the microscopic mirror (204) is dynamically tuned according to the RGB components of the 2D image.

Dated this June 26th 2023

Arjun Karthik Bala,

(IN/PA 1021)
Agent for Applicant