DESC:FIELD OF THE INVENTION
Embodiment of the present invention relates to light weight, compact, low power consuming, and easily wearable haptic devices for experiencing virtual objects, and more particularly to untethered and safe haptic device capable of providing a sensory richness of life with a realistic touch of virtual objects in a virtual or augmented world.
BACKGROUND OF THE INVENTION
With widespread adoption of virtual/augmented reality (VR/AR) head mounted devices, increased emphasis is laid on uplifting performance of VR/AR paraphernalia for achieving enhanced immersive experience. Today, while interacting with virtual objects in virtual world, due to in part limitations of currently existing haptic devices, sense of realism is not achieved as sense of touch falls flat.
Human sense of touch is dispersed and is experienced throughout the human body. To simulate touch in any simulated environment is thus a herculean task, which requires determination of both material (surface texture and weight) and geometric (curvature, orientation, size) properties of the (virtual) object to be manipulated. Wearability and portability are other characteristic and elemental features that define prompt acceptance and widespread adoptability of tactile devices that can create a true perception of touch for virtual environments.
Tactile sensation has been attempted using vibrational feedback devices that are light weight and portable and create sense of touch via vibrational patterns. However, the experience has not been very rich and met with limited realism. Besides, force feedback based haptic devices makes use of miniaturized motors to ignite feel factor; however they lack providing rich texture information and contact orientation, much needed to experience different surfaces- smooth, rough, spiky, silken and the like virtual objects in artificial set ups. Additionally, counter reactive force exerted by these devices further distorts the entire tactile experience.
Other approaches that have been proposed and implemented include kinesthetic, thermal, electrotactile, and skin stretch actuation based on use of fingertip pin arrays, piezoelectric actuators, electric motors, electromagnetic solenoids, and fluidic actuation. Most of them require high density actuators concentrated in small regions to achieve high tactile fidelity; however it makes the overall adoption costly and unscalable. Further, pneumatic control of small inflatable glove-worn actuators is attempted, yet these are overall large and also not fully untethered.
By applying a localised mechanical force or vibration on the skin, mechanical actuators can elicit stable and continuous tactile sensations, but since they are bulky with limited spatial resolution, their application and adoption remains limited in realistic scenarios. So, mechanical stimulation alone does not qualify. Electrotactile stimulators, in contrast, evoke touch sensations in the skin by passing a local electric current through the skin. Though light and flexible, and capable of offering higher resolution and a faster response, they rely on high voltage direct-current (DC) pulses (up to hundreds of volts) to penetrate the outermost layer of the skin and stimulate the receptors and nerves, which poses a safety concern.
Furthermore, fluid based haptic devices have been attempted to achieve high fidelity touch; however, the response elicited from fluid filled pockets that is modulated by an additional electrical circuitry is very insubstantial and feather-like. This necessitates use of amplifiers to amplify the magnitude of electrical signal in order to make a solid virtual object to be felt as rigid and robust as in real life. Electrical signals of higher magnitude may raise safety concerns and will require high power, which limits its application to sensing of only light weighted virtual objects.
Thus, none of the above discussed configurations and combinations of various tactile devices is capable of presenting a rich, realistic information with fine tactile details that can provide detailed texture, geometrical information along with contact point orientation in a manner that is comfortable and maximally useful to the user. Further, there is little known ability to achieve high resolution, highly sensitive, vibrational feedback, orientation, fast actuation, minimum lag, minimum counter reactive force, low power consumption without sacrificing form factor, sense of freedom, wearability and compactness of such devices.
In the background of foregoing limitations, there exists a need for light weight and high-performance haptic device that is not posed with aforementioned challenges and offers ease of wearability, scalability and affordability in order to contribute to immersive experiences of virtual world. In order to provide solution to the above issues of compactness, safety, light weight, untethered and high power consumption in prevailing haptic devices, and to achieve sensing of different tactile perceptions with high spatial resolution and in high fidelity, a thin and flexible haptic device designed with a unique combination of constituting elements has been attempted in present disclosure, and that may address one or more of the challenges or needs mentioned herein, as well as provide other benefits and advantages.
The above information disclosed in this Background section is only for enhancement of understanding of the background of the invention, and therefore, it may contain information that does not form prior art.
OBJECT OF THE INVENTION
An object of the present invention is to provide a low-cost, high-performance and lightweight haptic device that simulate the sense of touch with unprecedented levels of visual and fingertip level realism.
Another object of the present invention is to provide a thin and flexible haptic device that offers a high-resolution tactile sensation with rapid response rate and no perceivable latency.
Yet another object of the present invention is to provide a light weight, compact and easy to manufacture haptic device capable of eliciting stable and continuous tactile sensations when worn over entire hand.
Yet another object of the present invention is to provide a true contact haptics that can display different tactile sensations, such as pressure, vibration, and texture roughness in high fidelity.
Yet another object of the present invention is to provide safe, untethered and low power consuming haptic device capable of sensing broad range of virtual objects having different shape, size, texture, orientation with high sense of realism and efficacy.
In yet another embodiment, easy to use and ergonomically designed haptic device that can transmit fine tactile details with supreme precision in modulating the signals to perceive wide range of objects more realistically in virtual world.
BRIEF DESCRIPTION OF THE DRAWINGS
So that the manner in which the above recited features of the present invention can be understood in detail, a more particular to the description of the invention, briefly summarized above, may be had by reference to embodiments, some of which are illustrated in the appended drawings. It is to be noted, however, that the appended drawings illustrate only typical embodiments of this invention and are therefore not to be considered limiting of its scope, the invention may admit to other equally effective embodiments.
These and other features, benefits and advantages of the present invention will become apparent by reference to the following text figure, with like reference numbers referring to like structures across the views, wherein:
Fig. 1 illustrates unique configuration of haptic device, in accordance with an embodiment of the present invention.
Fig. 2 illustrates block diagram of overall system architecture, in accordance with an embodiment of the present invention.
DETAILED DESCRIPTION
While the present invention is described herein by way of example using embodiments and illustrative drawings, those skilled in the art will recognize that the invention is not limited to the embodiments of drawing or drawings described and are not intended to represent the scale of the various components. Further, some components that may form a part of the invention may not be illustrated in certain figures, for ease of illustration, and such omissions do not limit the embodiments outlined in any way. It should be understood that the drawings and detailed description thereto are not intended to limit the invention to the particular form disclosed, but on the contrary, the invention is to cover all modifications, equivalents, and alternatives falling within the scope of the present invention as defined by the appended claims. As used throughout this description, the word "may" be used in a permissive sense (i.e., meaning having the potential to), rather than the mandatory sense, (i.e., meaning must). Further, the words "a" or "an" mean "at least one” and the word “plurality” means “one or more” unless otherwise mentioned. Furthermore, the terminology and phraseology used herein is solely used for descriptive purposes and should not be construed as limiting in scope. Language such as "including," "comprising," "having," "containing," or "involving," and variations thereof, is intended to be broad and encompass the subject matter listed thereafter, equivalents, and additional subject matter not recited, and is not intended to exclude other additives, components, integers or steps. Likewise, the term "comprising" is considered synonymous with the terms "including" or "containing" for applicable legal purposes. Any discussion of documents, acts, materials, devices, articles, and the like are included in the specification solely for the purpose of providing a context for the present invention. It is not suggested or represented that any or all of these matters form part of the prior art base or were common general knowledge in the field relevant to the present invention.
Reference will now be made to embodiments, examples of which are illustrated in the accompanying drawings. In the following description, numerous specific details are set forth in order to provide an understanding of the various described embodiments. However, it will be apparent to one of ordinary skill in the art that the various described embodiments may be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail so as not to unnecessarily obscure aspects of the embodiments. In this disclosure, whenever a composition or an element or a group of elements is preceded with the transitional phrase “comprising”, it is understood that we also contemplate the same composition, element or group of elements with transitional phrases “consisting of”, “consisting”, “selected from the group of consisting of, “including”, or “is” preceding the recitation of the composition, element or group of elements and vice versa.
The present invention is described hereinafter by various embodiments with reference to the accompanying drawings, wherein reference numerals used in the accompanying drawing correspond to the like elements throughout the description.
This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiment set forth herein. Rather, the embodiment is provided so that this disclosure will be thorough and complete and will fully convey the scope of the invention to those skilled in the art. In the following detailed description, numeric values and ranges are provided for various aspects of the implementations described. These values and ranges are to be treated as examples only and are not intended to limit the scope of the claims. In addition, a number of materials are identified as suitable for various facets of the implementations. These materials are to be treated as exemplary and are not intended to limit the scope of the invention.
The present disclosure devises a haptic device 500 comprising of a unique combination of selective elements that interplay with each other to achieve sub millimetre level precision and unprecedented level of realism while interacting with virtual objects in a simulated world. Tactile sensation generally refers to feeling of touch in roughness, texture, pressure, vibration, etc. as experienced or perceived by the user. The haptic device 500 of present disclosure advantageously makes use of a combination of ferromagnetic and magnetorheological materials along with electrotactile rendering arrangement to obtain dynamic level of different tactile sensations with no sacrifice of realism and efficacy. Also, a rich tactile feedback is achieved without limiting the user by external cables or wires. The tactile feedback may take the form of various kinds of tactile sensations, such as vibration feedback, temperature feedback, pressure feedback, etc.
In accordance with one general embodiment of present disclosure, the haptic device 500 comprises of (a) a user interactive upper layer 100 constituted of a magnetorheological fluid (MRF) 150; (b) a middle layer 200 beneath the upper user interactive layer 100 constituted of a ferro-fluid (FF) 250; (c) a bottom layer 300 comprising of electrotactile rendering component 350 contributing to enhanced levels of tactile perception; and (d) an electrical circuitry 400. The unique and novel layout of various types of ferromagnetic materials coupled with an electrotactile rendering component achieves high haptic fidelity with no perceivable latency.
Briefly, the magnetorheological fluid (MRF) 150 may be used that can also function with a changed rheological behaviour in presence of magnetic field. Here, the magnetic particles are of micrometre or micron scale and suspended in a liquid matrix such as synthetic oil. The fluid tends to change to near solid in presence of magnetic field and return to its liquid state as soon as the magnetic field is removed within few milliseconds. With the application of the field, the particles in the fluid align together forming a chain, thus resulting in an increase in yield stress.
On the contrary, Ferro-fluids 250 are colloidal mixture of surfactant coated nanometre scale magnetic particles suspended in a liquid carrier medium, which when subjected to magnetic field above a predetermined threshold, coordinate under the magnetic influence to display an exquisite pattern that varies with strength of applied magnetic field. While the particles remain suspended due to Brownian motion, the surfactant coating prevents their agglomeration. Thus, when a magnetic field is applied to ferro-fluid 250, the ferromagnetic particles align themselves with the lines of magnetic flux forming a localized raised region on the surface of the ferro-fluid. If the magnetic field is strong enough, a pattern of regular corrugations (in the form of spiky bumps) are formed on the surface of the ferro-fluid that contributes in achieving a sense of touch during interaction with any virtual object 50.
As understood, magnetorheological fluids 150 act differently to ferro-fluids 250 in the presence of a magnetic field. Rather than causing a raised surface region (as is the case with ferrofluids) which generally contributes to a feather like tenuous sensation, the presence of a magnetic field causes an increase in viscosity of the magnetorheological fluid 150 at the location of the magnetic field. This increase in viscosity can be detected by the touch of a user and consequently very firm, solid and rigid virtual objects are experienced with same level of stiffness.
Finally, using the electrotactile rendering component 350 transmits fine tactile details using light electrodes as sensor array that captures the pressure distribution and generates pressure information to elicit real time vibrotactile, kinesthetic or electrotactile stimulus. The electrotactile rendering component 350 electro-stimulates the user's skin with pulsed voltage signals that creates a haptic sensation at the user end.
The combination of magnetorheological fluid (MRF) 150, ferro-fluid (FF) 250 and electrotactile component (ET) 350 based haptic device 500 of present disclosure makes use of vivid manipulation at various levels that exert some force on palm area along with the fingertips to feel myriad of virtual objects (50). The haptic device 500 encompassing the MRF 150, FF 250 and electrotactile (ET) component 350 generates magnetically actuated vibrational cues, shape and texture information and other expressive capabilities without mechanical linkages. The untethered, self-contained configuration of present haptic device 500 with enhanced technical performance promotes its easy wearability.
Fig. 1 illustrates a haptic device 500 comprising an upper layer 100 that interacts with virtual object 50 within the virtual/augmented environment. The upper layer 100 is composed of magnetorheological fluid (MRF) 150 that has magnetic properties and fluidity, and may change shape depending on the strength of applied magnetic field. MRF 150 is enclosed within a flexible and deformable casing 160, which when comes in contact with virtual object 50 make the user feel the same as the shape is changed by the magnetic field.
The casing 160 comprises of an electromagnet 170 configured to generate a magnetic field that causes MRF 150 to change shape with created magnetic field. Activation of a portion of the electromagnet 170 is configured to attract a portion of MRF 150, the portion of MRF 150 being positioned on a region of the haptic device 500 interfacing upper layer 100 and contacting the virtual object 50. The portion of MRF 150 that corresponds to a region that does not contact the virtual object 50 involves deactivation of portion of electromagnet 170. Here, the strength of activation is correlated to a level of pressure exerted on the upper layer 100 that contacts the virtual object 50. In one exemplary embodiment, a couple of strong, compact electromagnets 170 may be embedded at the fingertips to control MRF 150 viscosity effectively, at joints (knuckles and finger joints) to simulate bending resistance and stiffness, at palm and base of fingers to provide distributed force feedback for larger object handling.
Primarily, the upper layer 100 is configured to displace the MRF 150 when a virtual object 50 having a medium range spike needs to be experienced. Since the virtual object 50 should feel stiff and yet smooth e.g. in case of curves and rounded geometries (like rubber articles, spring, elastic based materials and the like), the MRF 150 filled upper layer 100 is self-activated with created magnetic field to generate corresponding mid- range spike for simulating the exactness of feel for such articles within virtual world. In one example embodiment, the MR fluid 150 may be encapsulated in larger chambers or reservoirs 160, and covers broader areas of hand, including back of hand, joints, and major surfaces of palm and fingers. Its unique positioning helps to deliver resistance during interactions like gripping or pressing. As will be explained in greater detail below, it is selectively activated when larger force feedback is needed e.g. holding or pushing objects 50.
Next, the middle layer 200 comprises of a ferro-fluid 250 enclosed within a deformable casing 260 just beneath the upper layer 100. The middle layer 200 is coupled with small miniaturized electromagnets 270 to enable localized control for simulating surface roughness, edges or vibrations. It is noteworthy that unlike the conventional haptic wearables where amplifiers are introduced to cause sufficient displacement of ferro fluid in order to experience the sensation of touch (which requires higher voltage and high power), the present middle layer 200 is capable of simulating textures using smaller electromagnets 270. Thence, the present middle layer 200 is configured to be triggered in event of interaction with virtual objects 50 that are soft, smooth and requires to be just slightly felt when actuated by minimal stimulation. For example, imagine touching something as light as a feather or a hair strand or the like, where the stimulation is of very low spike range. The ferro-fluid 250 having a capability of exhibiting a degree of hysteresis enable such tickle or caress with selective activation.
In one exemplary working embodiment, the small electromagnets 270 may be placed at fingertips for experiencing detailed tactile perception, or at finger pads for simulating textures like sliding or stroking virtual objects, at palm for simulating larger, rough textures or vibrations across the palm. Here, medium-sized electromagnets 270 may be distributed evenly across the palm area. Overall, a grid of small electromagnets 270 may be utilized for precise, localized control that enable individual or group activation to simulate continuous textures or patterns. Thus, by carefully placing and configuring electromagnets for both the upper layer 100 and the middle layer 200, the haptic glove 500 can achieve seamless, multi-modal feedback for an immersive and realistic virtual interaction experience.
In one exemplary embodiment, the ferrofluid 250 containing middle layer 200 may comprise of ferrofluid encapsulated in microchannels or small chambers made from stretchable, non-reactive materials like thermoplastic polyurethane (TPU). This layer 200 is distributed below the MR fluid 150 containing layer 100 in thin, flexible channels on the fingertips, palm, and edges of fingers. This enables creating detailed tactile sensations (e.g. surface roughness, small ridges etc.). As will be explained later, the layer 200 may be actuated by real-time feedback for precise response to virtual textures and may be selectively actuated based on type of virtual object 50 that is being manipulated.
Finally, the electrotactile component 350 containing bottom layer 300 that runs beneath the middle layer 200 comprises of an array of electrodes 360 for electro-stimulating the defined portion on user hand which is interacting with the virtual object 50. In a way, the electrotactile component 350 is the bottom most layer 300 that touches the skin of user and provide most vital feedback with respect to holding and grasping of stiff, rigid objects that require a force feedback, pressure, vibration, and texture roughness in high fidelity for exact experience in virtual world. In accordance with one example embodiment, the electrotactile component 350 containing bottom layer 300 is thin, flexible, and formed of skin-safe conductive materials, say for example, silver-coated fabric or conductive silicone or the like. It can also be controlled to adjust intensity, frequency, and pattern based on virtual object interaction.
In one significant embodiment, the present haptic device 500 displays a complex interplay of MRF 150, ferro-fluid 250 and the electrotactile component 350. Generally, in any given electrotactile based haptic configuration, the conductors/electrodes 360 are densely located in a given section of palm/finger/other regions of hand to experience the desired sensation with the same intensity as in the real world. In scenarios where the user intends firmly grasping a solid virtual object, as he draws his fingers inwards to form a fist, the electrodes are drawn closer to each other causing them to overlap with each other.
This, undesirably, conducts current across almost all the electrodes placed in close proximity to each other, thereby giving a vibrational feedback or electrical stimulation across the entire hand, and not exactly at location where the object (virtual) is held. This spread of stimulation across the entire haptic device 500 defeats the very purpose of selective stimulation, much necessitated to realize touching of virtual objects 50 in a manner one does in real world setting. To overcome this limitation, the modulation of electrotactile component 350 in bottom layer 300 is correlated with flow of MRF 150 and ferro-fluid 250 of upper layer 100 and middle layer 200 respectively. Apropos, the selective activation of various components of haptic device 500 has been possible with careful examination of different properties of MRF 150, ferromagnetic fluids 250 along with electro tactile sensation generated from array of electrodes 360.
Next, the haptic device 500 is configured with electric circuitry 400 that includes components like microcontrollers, electromagnet drivers, power supply units, and sensors for actuating electromagnets 170, 270. In one exemplary embodiment, the wrist portion in haptic device 500 is a convenient location to house the main circuit board and power supply without adding significant weight or restricting hand movement. The miniaturised circuitry with flexible PCBs is used for individual electromagnet control for upper 100 and middle layer 200 of haptic device 500 and seamless integration.
In accordance with one example embodiment, the combination of three layers comprising an upper user interactive layer 100, middle layer 200 and an electrotactile component containing bottom layer 300 is distributed across the hand of user to stimulate different portions or sections of user hand based on the kind of virtual object 50 the user is interacting with. The selective excitation of different layers and their controlled modulation based on size, shape, texture, geometric and other notable parameters of the virtual object 50 is extremely critical for generating a desired sensation of touching a specific virtual object 50.
This unique layout of these components in different layers to constitute a complete haptic device 500 makes it wearable like a haptic glove and also interact with user worn head mounted device 600 via a selected network. In one example embodiment, the haptic device 500 may receive input from the user worn head mounted device 600 to identify and categorize the type and configuration of virtual object 50 the user wishes to interact with, which accordingly selects the layer or combination of layers within the haptic device 500 to get actuated.
The middle layer 200 of ferrofluid 250, being close to skin, provides fine tactile sensations like surface texture, micro-vibrations, and subtle pressure changes. The MR fluid 150 layer 100 positioned above the ferrofluid layer 200 handles broader feedback like resistance weight, and rigidity when “holding” or interacting with virtual objects 50. To begin with, the hand glove may be provided with sensors such as inertial measurement units (IMUs) or touch sensors that can track hand movements in virtual environment 50. Independent, localized electromagnets 170, 270 that have optimally tuned field strengths and shielding are used for different layers 100, 200 to prevent cross-interference i.e. to prevent magnetic field from MR fluid 150 control or effect ferrofluid behaviour. Further, for power optimization and optimized running of electromagnetic controls, adaptive activation of both layers 100, 200 is proposed, which depends on kind of haptic experience to be reproduced based on type of virtual object 50 being manipulated.
Next, the upper 100, middle 200 and electrotactile component containing bottom layer 300 may not be required to be fully operational under all situations within the virtual world, as it not only consumes power, but may also unreasonable interfere in smooth operations. Thus, in order to achieve selective actuation of different layers of the haptic device 500 during interaction with the virtual object 50, there are few parameters around the virtual object, which should be considered. These properties determine how the different layers of the glove respond, ensuring a realistic and immersive experience.
At first, the object texture, material properties such as rigidity, elasticity, object weight, object shape and size, deformability, sliding and friction, force and pressure exerted by the object, and other dynamic and environmental factors like object state change, impact forces, environmental factors, object interaction mode (active/passive interaction), etc. needs to be factored while drawing activation strategy for different layers of haptic glove 500.
To begin with, object texture that includes object roughness, smoothness, patterns (ridges, bumps, grooves) is most experienced with fine tactile feedback, which require modulation of MRF 150, ferromagnetic fluid 250 containing spikes to replicate fine textures. This may require use of small, rapid high-frequency and low-intensity magnetic field changes for experiencing subtle micro-vibrations across dynamic textures like sandpaper, fabric, paper etc. On the other hand, for experiencing hard objects like rock, metal or plastic, the MR fluid 150 containing upper layer 100 is actuated, which in turn, increase the MR fluid’s viscosity for high resistance and apply static magnetic fields to simulate solidity. Likewise, for rubbery spongy materials, the viscosity of MR fluid 150 may be dynamically modulated as the user squeezes or manipulates the virtual object 50. One may allow partial flow of fluid to mimic such elastic deformation.
For simulating experience of feeling object weight, the MR fluid 150 may be stiffened across the haptic glove 500, especially in areas of palm and fingers, while for sensing irregular shapes, smooth spheres, sharp edges, the property of localized resistance of MR fluid 150 may be utilized for simulating specific shapes. However, for experiencing dynamic interactions, both the layers 100, 200 may have to be simultaneously actuated. For example, for activities like gripping, pushing or pulling, the actuation of MR fluid 150 containing layer will stiffen the resistance proportionally to simulate the applied force, while the actuation of ferrofluid 250 containing layer 200 shall mimic minor deformations or surface adjustments based on applied pressure.
Similarly, to experience frictional force and sliding interactions like rubbing hand across a surface, the ferrofluid 250 containing layer 200 can be adjusted for texture feedback, while the MR fluid 150 containing layer 100 may simulate friction by increasing viscosity in specific areas. Object deformability, on the other hand, during activities such as modelling clay or holding soft toys may be experienced by gradual viscosity adjustments of MR fluid 150 containing layer to replicate squishiness or compression, while the ferrofluid 250 containing layer 200 actuation may add tactile detail to indicate minor surface imperfections.
Interestingly, for understanding object state change(s) due to dynamic and environmental properties, such as object melting, breaking or morphing, the activation of ferrofluid layer 250 shall simulate changes in surface texture (such as liquid dripping sensation), while the MR fluid 150 activation may reduce or increase resistance to reflect state transitions (e.g. solid to liquid or vice versa). Likewise, for experiencing impact forces like striking or bumping into objects or simulating feel of environmental factors such as wind, water, vibrations etc., brief alteration of spikes in ferrofluid layer 200 can replicate sudden tactile feedback and can generate patterns or motions that mimic external forces. On the other hand, the viscosity of MR fluid layer 100 can be rapidly increased to simulate impact resistance or dynamic forces to simulate drag or currents.
In addition to MR fluid 150 containing layer 100 and ferrofluid 250 containing middle layer 200, the electrotactile layer 300 can be utilized to deliver low-intensity, high-frequency electrical pulses effectively. It can be effective for achieving localized stimulation and mimicking sensations like pinpricks, buzzing or brushing. Combining the actuation of above three layers 100, 200 and 300 for a heavy textured object, one example scenario may require activating ferrofluid layer 200 for texture, MR fluid layer 100 for weight, and electrotactile layer 300 for sharp details like static and subtle dynamic effects. This configuration creates a comprehensive multi-modal haptic glove, offering a realistic and immersive interaction experience.
In another working embodiment, the present disclosure proposes use of machine learning for classifying virtual objects 50 and predicting the upper 100, middle 200 or lower-layer 300 activation based on classified virtual object. To enable object identification and actuation, the system 1000, as shown in Fig. 2, comprising of a haptic device 500 and head mounted device 600 relies on input regarding virtual environment data that includes information of virtual environment, object properties and object state changes. For the same, metadata such as texture, rigidity, and weight and object state changes such as melting or deforming are relayed in real time to the processing engine 700 hosted on the head mounted device 600.
In one exemplary embodiment, user interaction history with the virtual object 50 may be recorded that includes past interactions and patterns, which are analysed to understand user’s ways and techniques of handling virtual objects. This helps to generate customization data to help user experience specific sensations. In parallel, the haptic glove 500 may be configured with:
(a) force sensors that can be used to detect pressure or grip strength applied to virtual object;
(b) Motion sensors (IMU) that can track hand and finger movements, such as position, velocity and acceleration;
(c) Touch sensors that can detect contact points and gestures (such as sliding or tapping)
Likewise, user environment data and virtual object environment data is captured using sensors hosted on the head mounted device 600. Here, detailed discussion of sensor disposition on the head mounted device 600 is eliminated for reasons of brevity as extensive literature exists on the same in the art.
Once the data is received from sensors disposed on the head mounted device 600 and the haptic device 500, it is extracted for key features (e.g. pressure peaks, velocity changes, contact duration and the like), which are synced in real time with virtual object properties. Next, based on adopted machine learning model, one may use labelled datasets of virtual objects with known properties and corresponding actuation settings. In one exemplary embodiment, support vector machine is utilized for establishing clear distinctions between rigid, soft and textured objects, while neural networks may be opted for more complex classifications involving mixed properties of the object (e.g. elastic and textured).
Next, the processing engine 700 of the system 1000 utilizes reinforcement learning or deep learning to determine how to activate each layer for optimal feedback. Briefly, the model is fed with classified object properties (texture, rigidity, hardness, elasticity, friction etc.), object shape and size (surface contours and object dimensions), object weight and density (perceived mass and resistance), user interaction data (force applied, grip strength, contact pressure, spatial distribution, hand movement and gestures), environment state (e.g. dynamic object changes), and other such contextual information such as temperature, humidity, or virtual environment dynamics that influence haptic feedback.
The processing engine 700 identifies the object 50 for its material properties and maps the object to its pre-learned actuation profile. Reinforcement learning may be used to adjust actuation intensities based on user feedback or interaction history. In one preferable embodiment, machine learning model e.g. deep neural network or transformers is utilized to classify the virtual object 50 based on metadata and user interaction data. Once the key information is extracted of the virtual object 50, the identified properties are mapped to predefined actuation profiles stored in a database. For example, mapping of hard metal object requires high MR fluid 150 viscosity, minimal ferrofluid 250actuation and activation of high-frequency electrotactile pulses 350. Likewise, for soft fabric object, MR fluid 150 is actuated to less extent, compared to moderate ferrofluid 250 texture actuation and no electrotactile component 350 feedback.
The above profile is dynamically adjusted based on environment data and interaction forces. For example, wet surfaces may be simulated by altering ferrofluid 250 dynamics, while the MR fluid 150 resistance may be increased if the user applies more force. In this manner, the electromagnet currents may be adjusted for viscosity control of MR fluid 150, fine grained magnetic fields may be activated for controlling FF 250 movement for texture simulation, and pulse-width-modulated signals maybe generated for electrotactile component 350 to receive electrical feedback.
In accordance with one preferable embodiment, Recurrent Neural Network (RNN) architecture is adopted since much of virtual object 50 interaction involves temporal dependencies such as:
(a) Time-Dependent Object properties, where the virtual object dynamically changes its texture, stiffness or other properties over time (e.g. a ball being squeezed or a vibrating object and the like);
(b) User interaction patterns, where continuous gestures, force variations, or touch interactions evolve over time; and
(c) Dynamic environment feedback, where environmental factors such as temperature or vibrations change with time.
As well understood, RNNs are better designed to process sequential data by maintaining a hidden state to capture information about previous time steps, the present disclosure explains using them for learning patterns in collected data. The recurrent connections allow the model to dynamically adjust outputs (e.g. MR fluid actuation) based on past inputs (such as pressure changes). In one exemplary embodiment, LSTM (Long Short-Term Memory) is preferred as they are more stable and efficient for longer sequences, and solve the vanishing gradient problem that helps them aptly capture long-term dependencies.
In one exemplary embodiment, the architecture of LSTM is explained, wherein the LSTM model processes the sequential data to predict actuation parameters for MR fluid 150, FF 250 and Electrotactile component 350. Accordingly, the first layer of LSTM captures low-level temporal patterns in object properties, and outputs hidden state for each time step. This detect changes in hardness or texture over time. Next, the second layer of LSTM captures higher-level temporal dependencies and long-term relationships. It receives hidden state from first LSTM layer, and outputs final hidden state summarizing the sequence. This helps to understand how texture changes influence overall object perception. Finally, the dense layer maps the summarized temporal features to actuation parameters.
Here, the temporal features from last LSTM layer is received as an input and outputs predicted actuation values for MR fluid viscosity, FF magnetic actuation, and electrotactile intensity. The output layer determines the viscosity to simulate stiffness of MR fluid 150, magnetic patterns to simulate texture via FF 250, and electrical intensity for actuating electrotactile component 350. Thus, LSTM comprising of memory cells have gates that control the flow of information, and allows the model to remember important features like sudden texture changes while forgetting irrelevant ones. The model is also capable of handling long-term dependencies where the model can associate a gradual increase in hardness with a rise in MR fluid 150 viscosity over multiple time steps.
The model thus dynamically updates predictions for each layer based on sequential changes in input features. It therefore provides a rich understanding of temporal patterns, enabling realistic and adaptive feedback for immersive virtual environments. In an embodiment, the handheld haptic device 500 may be used to generate haptic effects for VR applications that are experienced via head mounted device 600, and may be in the form of a wearable device. The wearable haptic device 500 may be configured to be worn on user hand. The handheld haptic device 500 requires processing on processing engine 700 that may be either hosted on the head mounted device 600 or may be an independent processing unit external to both the head mounted device 600 and wearable haptic device 500. The processing engine 700 may be microprocessors, controllers or any other suitable type of processors for processing computing executable instructions to control the operation of the haptic device 500.
In accordance with an embodiment, the head mounted device 600 comprises a memory unit configured to store machine-readable instructions. The machine-readable instructions may be loaded into the memory unit from a non-transitory machine-readable medium, such as, but not limited to, CD-ROMs, DVD-ROMs and Flash Drives. Alternately, the machine-readable instructions may be loaded in a form of a computer software program into the memory unit. The memory unit in that manner may be selected from a group comprising EPROM, EEPROM and Flash memory. Further, a processor is operably connected with the memory unit. In various embodiments, the processor is one of, but not limited to, a general-purpose processor, an application specific integrated circuit (ASIC) and a field-programmable gate array (FPGA).
In general, the word “module,” as used herein, refers to logic embodied in hardware or firmware, or to a collection of software instructions, written in a programming language, such as, for example, Java, C, or assembly. One or more software instructions in the modules may be embedded in firmware, such as an EPROM. It will be appreciated that modules may comprised connected logic units, such as gates and flip-flops, and may comprise programmable units, such as programmable gate arrays or processors. The modules described herein may be implemented as either software and/or hardware modules and may be stored in any type of computer-readable medium or other computer storage device.
Further, while one or more operations have been described as being performed by or otherwise related to certain modules, devices or entities, the operations may be performed by or otherwise related to any module, device or entity. As such, any function or operation that has been described as being performed by a module could alternatively be performed by a different server, by the cloud computing platform, or a combination thereof. It should be understood that the techniques of the present disclosure might be implemented using a variety of technologies. For example, the methods described herein may be implemented by a series of computer executable instructions residing on a suitable computer readable medium. Suitable computer readable media may include volatile (e.g., RAM) and/or non-volatile (e.g., ROM, disk) memory, carrier waves and transmission media. Exemplary carrier waves may take the form of electrical, electromagnetic or optical signals conveying digital data steams along a local network or a publicly accessible network such as the Internet. It should also be understood that, unless specifically stated otherwise as apparent from the following discussion, it is appreciated that throughout the description, discussions utilizing terms such as "controlling" or "obtaining" or "computing" or "storing" or "receiving" or "determining" or the like, refer to the action and processes of a computer system, or similar electronic computing device, that processes and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices. Various modifications to these embodiments are apparent to those skilled in the art from the description and the accompanying drawings. The principles associated with the various embodiments described herein may be applied to other embodiments. Therefore, the description is not intended to be limited to the embodiments shown along with the accompanying drawings but is to be providing broadest scope of consistent with the principles and the novel and inventive features disclosed or suggested herein. Accordingly, the invention is anticipated to hold on to all other such alternatives, modifications, and variations that fall within the scope of the present invention.
,CLAIMS:We Claim:
1) A system (1000) for experiencing one or more virtual objects (50) within simulated environment, wherein the system (1000) comprises of:
a head mounted device (600) configured to capture contextual information of the virtual object (50);
a haptic device (500) configured to capture one or more properties of the virtual object (50), and comprising:
an upper layer (100) constituted of magnetorheological fluid (150) configured to change in viscosity with applied first magnetic field;
a middle layer (200) constituted of ferrofluid (250), wherein ferrofluid particles align in response to applied second magnetic field; and
a bottom layer (300) constituted of electrotactile component (350) configured to experience tactile details of the virtual object (50);
wherein the upper layer (100), the middle layer (200) and the bottom layer (300) of the haptic device (500) is selectively actuated based on the captured contextual information of the virtual object (50) and the one or more properties of the virtual object (50).

2) The system (1000), as claimed in claim 1, wherein the upper layer (100) comprises of a flexible casing (160) to contain the magnetorheological fluid (150) and a plurality of electromagnets (170) to generate the first magnetic field and cause change in the viscosity of the magnetorheological fluid (150) to experience the virtual object (50).

3) The system (1000), as claimed in claim 2, wherein the plurality of electromagnets (170) are embedded at fingertips, joints, palm and base of fingers to simulate bending resistance, stiffness and distributed feedback for generating the first magnetic field and stimulating medium range spike to experience stiffness, rigidity and resistive features of the virtual object (50).

4) The system (1000), as claimed in claim 1, wherein the middle layer (200) comprises of a casing (260) to contain the ferrofluid fluid (250) and a plurality of electromagnets (270) to generate the second magnetic field and enable localized control to experience softness and smoothness of the virtual object (50).

5) The system (1000), as claimed in claim 4, wherein the plurality of electromagnets (270) are embedded at fingertips, finger pads, and palm to simulate the localized control for experiencing detailed tactile experience of the virtual object (50).

6) The system (1000), as claimed in claim 1, wherein the bottom layer (300) comprises of an array of electrodes (360) for electro-stimulating portion of user hand to interact with the virtual object (50) and provide force feedback, pressure, or vibrational experience.

7) The system (1000), as claimed in claim 1, wherein the one or more properties of the virtual object (50) comprises of object size, shape, weight, texture, geometric properties, deformability, sliding and friction, object state changes, force and pressure exerted by the virtual object (50).

8) The system (1000), as claimed in claim 1, wherein the contextual information of the virtual object (50) comprises of environmental factors of the virtual object (50) and object interaction mode.

9) The system (1000), as claimed in claim 1, wherein the system (1000) further comprises of a processing engine (700) configured to selectively actuate the upper layer (100), the middle layer (200), and the bottom layer (300) in steps of:
extract key features from data relating to the one or more properties of the virtual object (50) and the contextual information of the virtual object (50);
map the extracted key features to actuation profile of the virtual object (50); and
actuate the upper layer (100) to simulate stiffness of the magnetorheological fluid (150); the middle layer (200) to simulate texture via the ferrofluid (250); and the bottom layer (300) to stimulate electrical intensity via the electrotactile component (350).

10) The system (1000), as claimed in claim 9, wherein the processing engine (700) utilizes Recurrent Neural Network (RNN) along with Long Short-Term Memory (LSTM) for the selective actuation of the upper (100), middle (200) and the bottom layer (300).

11) A haptic device (500) for experiencing one or more virtual objects (50) within simulated environment, wherein the haptic device (500) comprises of:
an upper layer (100) constituted of magnetorheological fluid (150) configured to change in viscosity with applied first magnetic field;
a middle layer (200) constituted of ferrofluid (250), wherein ferrofluid particles align in response to applied second magnetic field; and
a bottom layer (300) constituted of electrotactile component (350) configured to experience tactile details of the virtual object (50);
wherein the upper layer (100), the middle layer (200) and the bottom layer (300) of the haptic device (500) is selectively actuated based on contextual information of the virtual object (50) and one or more properties of the virtual object (50).

12) The haptic device (500), as claimed in claim 11, wherein the contextual information of the virtual object (50) and one or more properties of the virtual object (50) are obtained from one or more sensors hosted on a head mounted device (600) and the haptic device (500).

13) The haptic device (500), as claimed in claim 11, wherein the upper layer (100) comprises of a flexible casing (160) to contain the magnetorheological fluid (150) and a plurality of electromagnets (170) to generate the first magnetic field and cause change in the viscosity of the magnetorheological fluid (150) to experience the virtual object (50).

14) The haptic device (500), as claimed in claim 13, wherein the plurality of electromagnets (170) are embedded at fingertips, joints, palm and base of fingers to simulate bending resistance, stiffness and distributed feedback for generating the first magnetic field and stimulating medium range spike to experience stiffness, rigidity and resistive features of the virtual object (50).

15) The haptic device (500), as claimed in claim 11, wherein the middle layer (200) comprises of a casing (260) to contain the ferrofluid fluid (250) and a plurality of electromagnets (270) to generate the second magnetic field and enable localized control to experience softness and smoothness of the virtual object (50).

16) The haptic device (500), as claimed in claim 15, wherein the plurality of electromagnets (270) are embedded at fingertips, finger pads, and palm to simulate the localized control for experiencing detailed tactile experience of the virtual object (50).

17) The haptic device (500), as claimed in claim 11, wherein the bottom layer (300) comprises of an array of electrodes (360) for electro-stimulating portion of user hand to interact with the virtual object (50) and provide force feedback, pressure, or vibrational experience.

18) The haptic device (500), as claimed in claim 11, wherein the one or more properties of the virtual object (50) comprises of object size, shape, weight, texture, geometric properties, deformability, sliding and friction, object state changes, force and pressure exerted by the virtual object (50).

19) The haptic device (500), as claimed in claim 11, wherein the contextual information of the virtual object (50) comprises of environmental factors of the virtual object (50) and object interaction mode.

20) The haptic device (500), as claimed in claim 11, wherein the haptic device (500) is communicatively coupled with a processing engine (700) to selectively actuate the upper layer (100), the middle layer (200), and the bottom layer (300) in steps of:
extract key features from data relating to the one or more properties of the virtual object (50) and the contextual information of the virtual object (50);
map the extracted key features to actuation profile of the virtual object (50); and
actuate the upper layer (100) to simulate stiffness of the magnetorheological fluid (150); the middle layer (200) to simulate texture via the ferrofluid (250); and the bottom layer (300) to stimulate electrical intensity via the electrotactile component (350).

21) The haptic device (500), as claimed in claim 20, wherein the processing engine (700) utilizes Recurrent Neural Network (RNN) along with Long Short-Term Memory (LSTM) for the selective actuation of the upper (100), middle (200) and the bottom layer (300).