DESC:FIELD OF THE INVENTION
Embodiments of the present invention relate generally to technologies in augmented reality, mixed reality and virtual reality environments and more particularly to an optical see-through augmented reality display arrangement for a Head Mounted Device (HMD).
BACKGROUND OF THE INVENTION
Augmented Reality (AR) and Mixed Reality (MR) technologies are one of the most rapidly growing technology fields and have been attracting a lot of attention across the globe. Augmented reality is the blending of interactive digital elements – like dazzling visual overlays, buzzy haptic feedback, or other sensory projections – into our real-world environments. Engineers and software developers have been able to find new applications of AR and MR such as in gaming, education, medical science, engineering, designing etc. So, the people around the world have eagerly been waiting to get their hands on such AR/MR devices.
In the field of Augmented Reality, small form factor displays with a high field of view and angular resolution are becoming increasingly desirable. However, creating optics that allow for high resolution, a large field of view, a bright background image, and lightweight form factor are difficult to design. To date, a number of different display designs have been introduced into the market, but very few solve all of these problems effectively.
But there are a few major issues that are preventing such devices from reaching to the public at large. Firstly, there are very few players in the global market such as Hololens, Magic Leap, Google, Meta Space Glasses etc. that have been able to achieve a desired standard of display quality in their AR/MR devices. In particular, the combination of small form factor and wide field of view are difficult to achieve. As field of view increases, the size of optical elements also increases. This problem is also related to the fact that optics have traditionally been placed away from the eye, in positions above the eyebrow (LetinAR, Minolta, Meta 2), to the side of the face (Google Glass, Pinlight Displays), and/or in combination with a waveguide system (Hololens, Magic Leap). While these designs have resulted in lighter and smaller form-factors, they still have not been able to approach the weight, field of view, or form factor of a typical pair of glasses.
Therefore, there is a need in the art for an optical see-through augmented reality display arrangement for a Head Mounted Device (HMD), that can realize a wide field of view with lightweight, thin optics and a compact form-factor.
OBJECT OF THE INVENTION
An object of the present invention is to provide an optical see-through augmented reality display arrangement for a Head Mounted device.
Another object of the present invention is to provide an augmented reality display arrangement offering compact, lightweight and thin optics.
Yet another object of the present invention is to provide an optical arrangement design leading to a wider field of view of the head mounted device without making the HMD bulky.
SUMMARY OF THE INVENTION
The present invention is described hereinafter by various embodiments. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiment set forth herein.
According to a first aspect of the present invention, there is provided an optical see-through augmented reality display arrangement for a Head Mounted Device (HMD), the HMD having a wearable means and a visor each eye of a user. The display arrangement comprises a micro display provided on each side of the HMD for providing a light projection or laser projection of one or more images to be displayed; a first prism in front of micro display to magnify the one or more images; a micro mirror positioned in an orientation pointing away from the eye of the user and towards the visor; a second prism positioned in front and parallel to the micro mirror; and a curved beam splitter positioned in front of the micro mirror and the second prism. Further, the micro mirror is adapted to receive the magnified light projection or laser projection of one or more images and reflect it towards the curved beam splitter. Then, the second prism is adapted to expand the reflected magnified light projection or laser projection of one or more images and the curved beam splitter is adapted to reflect the expanded light projection or laser projection of one or more images onto the eye of the user, thereby providing one or more virtual images with a wider field of view.
In accordance with an embodiment of the present invention, the display arrangement further comprises an eye tracking camera provided proximal to the micro mirror and angled towards the eye of the user.
In accordance with an embodiment of the present invention, the display arrangement further comprises one or more InfraRed (IR) based LEDs provided at a back of the micro mirror forming a predetermined shape and configured to illuminate the eye to assist in computation of the eye and virtual image position.
In accordance with an embodiment of the present invention, the HMD further comprises a processing module. Further, the processing module is configured to track the eye of the user by determining the position of the display relative to the eye using the eye tracking camera and the one or more IR based LEDs.
In accordance with an embodiment of the present invention, the processing module is configured to undistort the one or more images projected from the micro display and through the micromirror, before displaying to the eye of the user.
In accordance with an embodiment of the present invention, the micro mirror is selected from, but not limited to, a concave mirror or a plane mirror arranged at an angle away from the eye of the user and towards the curved beam splitter.
In accordance with an embodiment of the present invention, the first prism is a magnifier lens, and the second prism is an expander lens.
In accordance with an embodiment of the present invention, the wider field of view ranges from 50 – 70 degrees.
BRIEF DESCRIPTION OF THE DRAWINGS
So that the manner in which the above recited features of the present invention can be understood in detail, a more particular to the description of the invention, briefly summarized above, may be had by reference to embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of this invention and are therefore not to be considered limiting of its scope, the invention may admit to other equally effective embodiments.
These and other features, benefits and advantages of the present invention will become apparent by reference to the following text figure, with like reference numbers referring to like structures across the views, wherein:
Fig. 1A illustrates an optical see-through augmented reality display arrangement for a Head Mounted Device (HMD), in accordance with an embodiment of the present invention.
Fig. 1B illustrates the optical see-through augmented reality display arrangement for a Head Mounted Device (HMD), in accordance with another embodiment of the present invention.
Fig. 1C illustrates a front view of the display arrangement of Fig. 1A for the HMD worn by a user, in accordance with an embodiment of the present invention.
Fig. 2A-2C illustrates a micro mirror having one or more IR based LEDs at the back, in accordance with an embodiment of the present invention; and
Fig. 3 illustrates an implementation of the display arrangement of the Fig. 1A-1B in the HMD from a user’s point of view, in accordance with an embodiment of the present invention.
DETAILED DESCRIPTION OF DRAWINGS
While the present invention is described herein by way of example using embodiments and illustrative drawings, those skilled in the art will recognize that the invention is not limited to the embodiments of drawing or drawings described and are not intended to represent the scale of the various components. Further, some components that may form a part of the invention may not be illustrated in certain figures, for ease of illustration, and such omissions do not limit the embodiments outlined in any way. It should be understood that the drawings and detailed description thereto are not intended to limit the invention to the particular form disclosed, but on the contrary, the invention is to cover all modifications, equivalents, and alternatives falling within the scope of the present invention as defined by the appended claims. As used throughout this description, the word "may" is used in a permissive sense (i.e. meaning having the potential to), rather than the mandatory sense, (i.e. meaning must). Further, the words "a" or "an" mean "at least one” and the word “plurality” means “one or more” unless otherwise mentioned. Furthermore, the terminology and phraseology used herein is solely used for descriptive purposes and should not be construed as limiting in scope. Language such as "including," "comprising," "having," "containing," or "involving," and variations thereof, is intended to be broad and encompass the subject matter listed thereafter, equivalents, and additional subject matter not recited, and is not intended to exclude other additives, components, integers or steps. Likewise, the term "comprising" is considered synonymous with the terms "including" or "containing" for applicable legal purposes. Any discussion of documents, acts, materials, devices, articles and the like are included in the specification solely for the purpose of providing a context for the present invention. It is not suggested or represented that any or all of these matters form part of the prior art base or were common general knowledge in the field relevant to the present invention.
In this disclosure, whenever a composition or an element or a group of elements is preceded with the transitional phrase “comprising”, it is understood that we also contemplate the same composition, element or group of elements with transitional phrases “consisting of”, “consisting”, “selected from the group of consisting of, “including”, or “is” preceding the recitation of the composition, element or group of elements and vice versa.
The present invention is described hereinafter by various embodiments with reference to the accompanying drawings, wherein reference numerals used in the accompanying drawing correspond to the like elements throughout the description. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiment set forth herein. Rather, the embodiment is provided so that this disclosure will be thorough and complete and will fully convey the scope of the invention to those skilled in the art. In the following detailed description, numeric values and ranges are provided for various aspects of the implementations described. These values and ranges are to be treated as examples only and are not intended to limit the scope of the claims. In addition, a number of materials are identified as suitable for various facets of the implementations. These materials are to be treated as exemplary and are not intended to limit the scope of the invention.
The present invention provides a new optical design for a small form-factor, optical see-through Augmented Reality display arrangement. The display arrangement uses a series of magnifiers, lenses, and curved beam splitters that can realize a wide field of view with lightweight and thin optics. The key difference between this display and other designs like the curved beam splitters is that the design first reflects beams of light towards a small micro-mirror placed close to the user’s eye. This micro-mirror is in a reversed orientation or position that is angled away from the user’s eye. Because of the acute angle of curvature of the beam splitter (114) and the proximity of the expander to the eye, a larger field of view is possible. This multiple reflection design allows for a much more compact design that can still realize a wide field of view with a focal distance of greater than 1 meter.
The present invention is now described with reference to the following drawings:
Figure 1A illustrates an optical see-through augmented reality display arrangement (100) for a Head Mounted Device (HMD (102)), in accordance with an embodiment of the present invention. The optical see-through augmented reality display arrangement (100) (hereinafter referred to as “the display arrangement (100)”) is envisaged to be provided in an Augmented Reality (AR) and/or a Mixed Reality (MR) and/or a Virtual Reality device. Apart from the display arrangement (100), the HMD (102) is envisaged to include one or more of, but not limited to, a wearing means, a visor, eyes pieces/boxes for each eye, a processing module (having microprocessor, GPU etc.), one or more sensors, a user interface, an audio input and/or output module and one or more ports. The wearable means may include a plastic/polycarbonate casing (1022), one or more straps, temples (if HMD (102) is in a form of spectacle).
As shown in figure 1A, the display arrangement (100) comprises, but not limited to, a curved beam splitter (114), a first prism (108), a micro mirror (110), a micro display (104) and a second prism (112), all arranged in the casing (1022). The casing (1022) is envisaged to protect the components enclosed therein from physical damage. It will be appreciated by skilled addressee that the display arrangement (100) and the components thereof, are for one eye (120). Therefore, two such display arrangements (100) will be provided in the HMD (102), one in each eye box for the respective eye (120), without departing from the scope of the present invention.
Herein, the micro display (104) may be provided on either side of the HMD (102). The micro display (104) is, but not limited to, a Liquid Crystal on Silicon (LCoS) display. The LCoS display is capable of providing a high-resolution display of 720p at a very high frame rate of 120 FPS. The micro display (104) is configured to provide a light projection or laser projection (106) of one or more images to be displayed. The one or more images may have a width in a range of, but not limited to, 8mm - 12 mm. Additionally, the first prism (108) is disposed directly in front of micro display (104) and is adapted to optically magnify the one or more images. In that sense, the first prism (108) may be, but not limited to, a magnifying lens.
Further, the micro mirror (110) is provided on the other side of user’s eye (120) (in the same box itself) as shown in figure 1A, positioned in an orientation pointing away from the eye (120) of the user and towards the visor or the curved beam splitter (114). The micro mirror (110) is selected from, but not limited to, a concave mirror or a plane mirror arranged at an angle away from the eye (120) of the user and towards the curved beam splitter (114). The micro mirror (110) receives the magnified light projection or laser projection (106) from the micro display (104) and changes the direction towards the curved beam splitter (114). Moreover, the second prism (112) is positioned in front and parallel to the micro mirror (110). The second prism (112) is adapted to expands an angle of the reflected magnified light projection or laser projection (106) of one or more images from the micro mirror (110) towards the curved beam splitter (114).
Also, the curved beam splitter (114) positioned in front of the micro mirror (110) and the second prism (112). The curved beam splitter (114) is adapted to reflect the expanded light projection or laser projection (118) of one or more images onto the eye (120) of the user, which enables user’s eye (120) to see the one or more virtual images with a wider field of view.
In accordance with another embodiment of the present invention shown in figure 1B, it can be seen that as the micro mirror (110) is small, the micro mirror (110) and the second prism (112) can sit just in front of the user’s pupil (1202) without being perceived as a hindrance. The micro mirror (110) then reflects light outward towards a beam splitter. Outgoing light is then distributed along the larger, curved beam splitter (114), which is then reflected back to the user’s eye (120) to produce the resulting virtual image. This configuration takes advantage of pupil-division, which refers to a class of displays situated immediately in front of a user’s pupil. The small width of the display allows light to pass into the user’s eye (120) from around the edges of the display, allowing the user to maintain a view of the real world.
Figure 1C illustrates a front view of the display arrangement of Fig. 1A for the HMD worn by a user (128), in accordance with another embodiment of the present invention. As previously mentioned, the figure 3 shows the two display arrangements (100) arranged in the HMD (102), one for each eye (120). So, the practical application of the present invention may look similar to the figure 1C. Herein the HMD’s outer components have not been shown in order to clearly illustrate how the components of the display arrangement would be positioned, once these are provided in the HMD and the HMD is worn by the user (128). It can be seen that the micro display (104) and the first prism (108) are positioned on the outside of the user’s face while the micro mirror (110) and the second prism (expander lens) (112) are placed proximal to user’s nose. Further, a nose rest (130), along with each curved beam spitter (114) may also be provided for additional support/stability to the HMD after wearing. Similarly, the display arrangement (100) shown in figure 1B can also be provided in the HMD in the same manner.
The method of working of the present invention:
Referring to figures 1A-1B, the process starts at the micro display (104), where one or more images are generated from a small display approximately, but not limited to, one centimetre in width. The light or laser projection (106) from this image then passes through the first prism (108) (magnifying lens) that optically magnifies the one or more image. The magnified light or laser projections (106) then travels to the curved (or non-curved) micro mirror (110), which reflects the light outwards and towards the curved beam splitter (114). The light or the laser projections then passes through the second prism (112) (expander lens) that changes (expands) the angle of the outgoing light or laser projections (118). In different embodiments, the micro mirror (110) and the second prism (112) (expander lens) are situated immediately in front of the user’s pupil (1202). Outgoing light (118) is then distributed along a larger, curved beam splitter (114), which is then reflected back to the user’s eye (120) to produce the resulting one or more virtual image with a wider field of view. Because of the acute angle of curvature of the curved beam splitter (114) and its proximity to the eye (120), a larger field of view is possible. As previously explained, the above-mentioned process happens in the respective display arrangements (100) in both eye boxes of the HMD (102), separately but simultaneously.
The present invention with its multiple reflection design is capable of providing a wider field of view in a range of, but not limited to, 50 – 70 degrees with a focal distance of greater than 1 meter for the virtual image. In accordance with an embodiment of the present invention, due to multiple reflections of the one or more images within the display arrangement (100), the final one or more virtual images being displayed to the user’s eye (120) may be distorted. In order to tackle this problem, the processing module is configured to undistort the one or more images using, but not limited to, an algorithm or the like. For example: the processing module may take in to consideration measurements of the all the mirrors, lenses, distances therebetween and those travelled by the light, angle of reflections and expansions, to pre-calculate possible distortions and then apply the algorithms at the start of the process itself so that final one or more virtual images do not come out as distorted.
In accordance with an embodiment of the present invention, a pupil (1202) of the eye (120) is tracked via an eye tracking camera (116). In one implementation, the position of this eye tracking camera (116) is directly behind the micro mirror (110), preventing additional clutter of the user’s field of view. In other implementations, the eye tracking camera (116) is located at the bottom of the casing (1022) to allow for more accurate eye tracking. The eye tracking camera (116) is angled towards the eye (120) of the user. The same has been shown in figure 1A and 1B. The eye tracking camera (116) is configured to enable eye tracking in the HMD (102).
Further, the micro mirror (110) is also provided with one or more Infrared (IR) based LEDs (124) on the back (1102), forming a predetermined shape. For example, in figure 2A, the micro mirror (110) is can be seen to form a constellation of LEDs at the back. In figure 2B and 2C, there is an LED (124) forming a unique shape, e.g. a plus or circle/ring symbol, respectively. These one or more IR based LEDs (124) have two functions, including the illumination of the eye (120) and provision of a constellation of lights for use in an eye tracking algorithm. Unlike other LED orientations that are used for eye tracking and often mounted to a beam splitter several centimetres away from the eye (120), these near-eye IR based LEDs (124) provide a unique way to track the eye (120) via eye camera due to the reflections generated from the cornea.
The processing module of the HMD (102) is configured to track the eye (120) of the user by determining the position of the display relative to the eye (120) using the eye tracking camera (116) and the one or more IR based LEDs (124). The unique predetermined shapes allow the processing module to implement the eye tracking algorithm to use object or feature tracking to find the LED constellation. This LED constellation has several functions, but the primary function is to help compute the position of content such that it is correctly transformed to match the real world. For this purpose, the processing module implements an algorithm that computes the LED constellation’s position relative to the user’s eye (120) via the eye tracking camera image. The eye tracking camera image contains the reflection of the LED constellations or shapes, which in turn allows the algorithm to compute its position relative to the eye (120). The position of the curved beam splitter (114) relative to the one or more IR based LEDs (124) is known, so the tracking algorithm can also solve for the optimal position at which to display the virtual image.
Moreover, the changes in the shape of this LED (124) allow the eye tracking algorithm to more efficiently compute eye position at different eyeball rotations as well as the curvature of the eye (120) itself. Knowing eye (120) curvature can also be used to compute accommodation or focal distance without a refractometer.
Furthermore, figure 3 illustrates an implementation of the display arrangement (100) of the Fig. 1A-1B in the HMD (102) from a user’s point of view. As shown in figure 3, a virtual image (204) of a wall of a room can be seen originating from the micro display (104). It can be observed that the original image (202) from the micro display (104) is first magnified by the first prism (108), then is reflected by the micro mirror (110). The second prism (112) (expander lens) further expands the magnified image before letting the light or laser projections (106) reach the beam splitter (114). The beam splitter (114) then reflects the much-expanded virtual image (204) to user’s eye (120), that sees the same in a wide field of view. The difference between the size of original projected image (202) and the final virtual image (204) is clearly visible in figure 3. This multiple reflection design allows for a much more compact design that can still realize a wide field of view with a focal distance of greater than 1 meter for the virtual image (204).
The key difference between the display arrangement (100) of the present invention and other designs like the curved beam splitters of the existing HMDs or the wave-guide optics based design is that the display arrangement (100) of the present invention, first reflects beams of light towards the small micro mirror (110) placed close to the user’s eye (120). This micro mirror (110) is in a reversed orientation or position that is angled away from the user’s eye (120). Because the micro mirror (110) is small, it can sit just in front of the user’s pupil (1202) without being perceived as a hindrance due to the pupil division principle. The micro mirror (110) then reflects light outward towards the curved beam splitter (114) and back to the eye (120). Existing designs reflect light through a prism located away from the eye and directly towards a beam splitter without being angled towards the eye first, which is a key difference from the present invention.
The present invention offers a number of advantages:
1. Works for AR, MR and VR devices.
2. Wide Field of View (FOV).
3. A small form-factor.
4. High-See-through accuracy.
5. Lightweight.
6. Thin Optics.
7. Large eye box.
In various embodiments, the processing module referred herein includes a microprocessor. The microprocessor may be a multipurpose, clock driven, register based, digital integrated circuit that accepts binary data as input, processes it according to instructions stored in its memory and provides results as output. The microprocessor may contain both combinational logic and sequential digital logic. For example: the microprocessor may be selected from one of, but not limited to, a ARM based or Intel based processor (1044) in the form of field-programmable gate array (FPGA), a general-purpose processor and an application specific integrated circuit (ASIC).
In general, the word “module,” as used herein, refers to logic embodied in hardware or firmware, or to a collection of software instructions, written in a programming language, such as, for example, Java, C, or assembly. One or more software instructions in the modules may be embedded in firmware, such as an EPROM. It will be appreciated that modules may comprised connected logic units, such as gates and flip-flops, and may comprise programmable units, such as programmable gate arrays or processors. The modules described herein may be implemented as either software and/or hardware modules and may be stored in any type of computer-readable medium or other computer storage device.
Further, while one or more operations have been described as being performed by or otherwise related to certain modules, devices or entities, the operations may be performed by or otherwise related to any module, device or entity. As such, any function or operation that has been described as being performed by a module could alternatively be performed by a different server, by the cloud computing platform, or a combination thereof. It should be understood that the techniques of the present disclosure might be implemented using a variety of technologies. For example, the methods described herein may be implemented by a series of computer executable instructions residing on a suitable computer readable medium. Suitable computer readable media may include volatile (e.g. RAM) and/or non-volatile (e.g. ROM, disk) memory, carrier waves and transmission media. Exemplary carrier waves may take the form of electrical, electromagnetic or optical signals conveying digital data steams along a local network or a publicly accessible network such as the Internet.
Various modifications to these embodiments are apparent to those skilled in the art from the description and the accompanying drawings. The principles associated with the various embodiments described herein may be applied to other embodiments. Therefore, the description is not intended to be limited to the embodiments shown along with the accompanying drawings but is to be providing broadest scope of consistent with the principles and the novel and inventive features disclosed or suggested herein. Accordingly, the invention is anticipated to hold on to all other such alternatives, modifications, and variations that fall within the scope of the present invention.
,CLAIMS:1. An optical see-through augmented reality display arrangement (100) for a Head Mounted Device (HMD (102)), the HMD (102) having a wearable means and a visor, the display arrangement (100) comprising:
a micro display (104) provided on each side of the HMD (102) for providing a light projection or laser projection (106) of one or more images to be displayed to each eye (120) of a user;
a first prism (108) in front of micro display (104) to magnify the one or more images;
a micro mirror (110) positioned in an orientation pointing away from the eye (120) and towards the visor;
a second prism (112) positioned in front and parallel to the micro mirror (110); and
a curved beam splitter (114) positioned in front of the micro mirror (110) and the second prism (112);
wherein the micro mirror (110) is adapted to receive the magnified light projection or laser projection of one or more images and reflect it towards the curved beam splitter (114);
wherein the second prism (112) is adapted to expand the reflected magnified light projection or laser projection of one or more images; and
wherein the curved beam splitter (114) is adapted to reflect the expanded light projection or laser projection (118) of one or more images onto the eye (120) of the user, thereby providing one or more virtual images with a wider field of view.
2. The display arrangement (100) as claimed in claim 1, further comprising an eye tracking camera (116) provided proximal to the micro mirror (110) and angled towards the eye (120) of the user.
3. The display arrangement (100) as claimed in claim 2, further comprising one or more InfraRed (IR) based LEDs (124) provided at a back (1102) of the micro mirror (110) forming a predetermined shape and configured to illuminate the eye (120) to assist in computation of the eye (120) and display position.
4. The display arrangement (100) as claimed in claim 3, wherein the HMD (102) further comprises a processing module;
wherein the processing module is configured to track the eye (120) of the user by determining the position of the virtual image relative to the eye (120) using the eye tracking camera (116) and the one or more IR based LEDs (124).
5. The display arrangement (100) as claimed in claim 4, wherein the processing module is configured to undistort the one or more images projected from the micro display (104) and through the micro mirror (110), before displaying to the eye (120) of the user.
6. The display arrangement (100) as claimed in claim 1, wherein the micro mirror (110) is selected from a concave mirror or a plane mirror arranged at an angle away from the eye (120) of the user and towards the curved beam splitter (114).
7. The display arrangement (100) as claimed in claim 1, wherein the first prism (108) is a magnifier lens and the second prism (112) is an expander lens.
8. The display arrangement (100) as claimed in claim 1, wherein the wider field of view ranges from 50 – 70 degrees.