WO2025046267A1

WO2025046267A1 - Method and wearable device for providing a virtual reality image to a user

Info

Publication number: WO2025046267A1
Application number: PCT/IB2023/058573
Authority: WO
Inventors: Andrey Mikhailovich DEMCHINSKY; Nikolai Vladimirovich SYROV; Lev Vladimirovich YAKOVLEV
Original assignee: Global Target Ventures Llc
Priority date: 2023-08-30
Filing date: 2023-08-30
Publication date: 2025-03-06

Abstract

A device and method for providing a virtual reality image to a user involving the application of transcranial magnetic stimulation (IMS) to the user. The method comprises a) adjusting a navigational TMS unit to the user's individual characteristics including the features of the user's anatomy and visual perception; b) receiving information about the at least one virtual reality artefact to be provided by the TMS unit; c) controlling the, TMS unit to affect the user's brain regions in order to provide the at least one virtual reality artefact according to the received information about said virtual reality artefact, wherein said adjusting (a) includes determining of the TMS intensity, the TMS applicator positions, and TMS frequency providing the at least one virtual reality artefact. A wearable device for providing a virtual reality image to a user involving the application of transcranial magnetic stimulation (TMS) to the user.

Description

METHOD AND WEARABLE DEVICE FOR PROVIDING A VIRTUAL REALITY IMAGE TO A USER

FIELD OF THE INVENTION

The following generally relates to improving a virtual reality user experience via the application of transcranial magnetic stimulation (TMS) to the user.

BACKGROUND OF THE INVENTION

Throughout the history of medical study, more and more functionality and possibilities of the brain have been discovered. The complexity of the cerebral functions and the control that the brain has over the human body and consciousness has inspired many researchers to apply external stimuli to the brain in order to achieve some particular effects on the human condition and perception. With the recent development in virtual and augmented reality activities and their growing availability, the possibilities of overcoming some of the drawbacks and limitations of the currently existing techniques are given closer scrutiny.

While modem VR devices are continuously improving, the approach of combining VR with brain stimulation is not thoroughly studied to the present moment but is promising in terms of overcoming the physical limitations of VR devices. In this context, document CN214971164U gives an inspiring example of a device for transcranial electrostimulation combined with VR. The device comprises an electrostimulation module and the disclosed method involves stimulation of the brain with a weak electric current in order to enhance the comprehensive user experience by providing visual, auditory, physical, and other sensory simulation experiences. However, the type of stimulation used in CN214971164U leads to the occurrence of slow-developing effects: it requires continuous stimulation within several minutes. Then the cumulated effect occurs in the form of a general change in the excitability of the stimulated section of the cerebral cortex. Thus, the disclosed method demonstrates a modulating effect without explicit instantaneous (live) effects. Such stimulation may not produce the appearance of a short, live visual effects augmenting and supplementing the VR content. In addition, the disclosed electrostimulation method has poor spatial resolution, the electrodes typically have an area of several cm², and the area of the brain cortex subjected to the impact can reach up to 10 cm². Thus, the approach disclosed in CN214971164U has a modulating effect and may be considered as a therapy for various cognitive disorders but does not offer much for the entertainment industry. In terms of improving the current VR applications, the concept of additional cerebral stimulation and non-invasiveness are promising features of the approach disclosed in CN214971164U, however the disclosed techniques cannot propose any significant live effect in the user’s vision or sense.

However, in order to achieve a complete immersion effect when having a virtual reality session, it is essential to pursue the ability to use the sensory organs just like in reality to feel touch, pain, and odors, and also to use the whole scope of visual capabilities. Summing up the above, there is still needed a technique to augment and improve the live performance of the existing VR applications and overcome the physical limitations thereof.

SUMMARY OF THE INVENTION

Transcranial magnetic stimulation (TMS) is a method that allows non-invasive stimulation of the cerebral cortex using short magnetic pulses. The TMS does not require any painful procedures, yet can provide impressive effects, depending on the zone of the cortex being subject to the stimulation. For example, magnetic stimulation of the motor zone of the cerebral cortex causes a contraction of specific peripheral muscles according to their topographic representation in the cortex. Motor responses caused by the TMS (motor evoked potentials (MEPs)) can be registered using the electromyography method. In medicine, the MEP registration is usually used to measure the central conduction time of motor conducting paths and study the patient’s corticospinal excitability, which allows more efficient research of the patient’s memory, central vision, movements, mapping of motor and speech centers, etc.

As the aspects of the invention described herein address the above-referenced problem of providing the means to improve the user experience while using VR, the procedures relating to the specific implementation of VR and the embodiments of the apparatus itself are not covered in detail by the present description.

According to one aspect of the present invention, provided is a method for giving a virtual reality image to a user involving the application of transcranial magnetic stimulation (TMS) to the user, the method comprising a) adjusting a navigational TMS unit to the user’s individual characteristics including the features of the user’s anatomy (such as based on MRI or another method for in vivo brain anatomy data) and visual perception; b) receiving information about the at least two virtual reality artefacts to be provided by the TMS unit; c) controlling the TMS unit to affect the user’s brain regions in order to provide the at least two virtual reality artefacts according to the received information about said virtual reality artefact, wherein said adjusting (a) includes determining of the TMS intensity, the TMS applicator positions, and TMS frequency providing the at least two virtual reality artefacts. According to some of the embodiments, stage a) further includes the following: obtaining a magnetic resonance imaging (MRI) scan of the user’s brain by performing an MRI thereof or downloading the results of such imaging; performing a visual cortex mapping based on the brain MRI scan in order to obtain the information about at least two active visual cortex points the stimulation of which provides stable parameters of a visual artefact; and developing an individual stimulation strategy for the user based on the brain MRI scan and the visual cortex mapping; and stage c) is performed based on said individual stimulation strategy developed for the user.

According to further embodiment of the claimed method, said adjusting includes determining the TMS frequency providing the required stability and noticeability of the at least two virtual reality artefacts.

According to yet another embodiment of the claimed method, the parameters of the visual artefact include at least one of its brightness and its location in the user’s visual field.

According to another aspect of the invention, a wearable device for providing a virtual reality image to a user is provided, the device involving the application of transcranial magnetic stimulation (TMS) to the user according to the above method. The device comprises a controller system in communication with a virtual reality image-generating source and a virtual reality image-displaying unit configured for producing controlling signals based on the information about the virtual reality image to be generated, the virtual reality image displaying unit state, and the user information; a navigational transcranial magnetic stimulation unit (wherein the navigation is achieved by shifting the coils or switching between several stationary coils or another possible method, and there is the possibility of monitoring the coils’ positions relative to the particular user’s brain and the stimulation points positions in real-time mode using the user’s brain MRI results) arranged in close proximity to the user’s head so that it can provide the TMS for the user, the unit configured to provide said TMS according to the controlling signals received from the controller system. The controlling signals include the TMS intensity controlling signals, the TMS applicator positions controlling signals and the TMS frequency controlling signals.

According to some of the embodiments, the controller system is further in communication with a virtual reality image-displaying unit.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention may take form in various components and arrangements of components, and in various steps and arrangements of steps. The drawings are only for purposes of explaining the essential features of the provided method and system and illustrating their preferred embodiments and are not to be construed as limiting the invention.

For a more complete understanding of the invention, reference is made to the following description and accompanying drawings, in which:

FIG. 1 gives an illustration of the TMS operational concept.

FIG. 2 provides a flowchart of the claimed method according to one embodiment.

DETAILED DESCRIPTION OF EMBODIMENTS

According to one exemplary embodiment, the provided method includes but is not limited to, the following important steps to improve the quality and rate of tuning when more modern technologies appear.

The functional illustration of the transcranial magnetic stimulator unit is shown on Fig. 1. The key elements of the TMS unit (1) are electrically connected capacitor and electromagnet, in particular, an induction coil (2). The capacitor discharge results in a high current (several thousand amperes) passing through the coil in a short period of time (about 1 ms), which results in the generation of an alternating magnetic field in the coil. The shape and the direction of the magnetic field arising in the induction coil (inductor) (2) depends on its type. According to an embodiment of the invention, ring-shaped inductors and 8-shaped inductors may be used. Annular coils are characterized by a circular magnetic field distribution with a minimum in the center of the ring. The 8-shaped coils are two annular coils (of smaller diameter) with oppositely directed currents. This causes a more precise (focal) distribution of the magnetic field and allows for direct stimulation of individual areas of the brain cortex. The most commonly used stimulators and stimulation coils to the moment are MagTstim 200 + 8-shaped coil 70-mm and MagVEncapture MagPro R30 + 8 - shaped coil Cool-B65 -RO». It is possible to simultaneously use two 40 mm MagTstim coils with a reliable call of phosphene in both hemispheres. The researchers working in the field of art hone in on minimizing TMS units dimensions, which also means that the TMS unit may be implemented as a part of a VR device.

The magnetic field penetrates through the scalp (a), the cranial bones (b), and the brain membranes (c), reaching the neural tissue of the brain, in which the induction of membrane currents occurs (via the electromagnetic induction mechanism, the magnetic field induces changes in the electric fields of the nerve tissue) and the membrane potential of the neurons’ changes. The design of the coil affects not only the focus of the effect but also the depth of the stimulation, while these parameters have an inverse relationship: the deeper the effect of the field generated by the inductor, the less localized it is (due to the scattering effect).

Fig. 2 shows the flowchart of an illustrative embodiment of the method applied to a VR game running on a controller system (5) (for example, a computer). Some VR content is run on a controller system (5), and a VR unit (4) (for example, a VR helmet or a headset) has a wire or wireless connection (6) thereto. The TMS unit (1,) with a TMS coil (2) may be implemented in various ways, but for the initial cortex mapping it is essential to use a TMS navigation system. The TMS unit is first used for visual cortex mapping in order to determine the cortex areas to be further engaged, and then the TMS unit is involved in the VR performance process. The TMS coil (2) is part of the TMS unit that directly applies stimulation to the proper area of the cortex (visual cortex (3) for the discussed application) for neuron activation.

According to an embodiment, an MRI of the user’s brain is made (100) to have data on the anatomy of the user’s brain (other diagnostic methods can also be used to obtain in vivo brain anatomy data if they are used in a particular navigational TMS). Alternatively, the results of brain MRI can be downloaded. Then, the user’s anatomical (brain MRI scan) is loaded into the navigational TMS unit to prepare for visual cortex mapping (101). Visual cortex mapping is a process that allows the determination of the boundaries of the user’s visual cortex and the phosphene properties that may be generated over the entire visual cortex. The visual cortex mapping provides information about the points of the visual cortex that are active, and what visual effects are available for the user, what dynamical, geometrical, shape, and other parameters the induced visual artefacts have for this particular user.

After obtaining the user’s visual cortex phosphene map, an individual stimulation strategy is built on stage (102) in accordance with the points of stimulation on the cortex and visual properties of what the user sees when stimulated (including but not limited to dimensions, intensity, shape, and chromaticity of at least one phosphene to be induced in the user’s visual field, which are given more detailed attention below). While the effects of the TMS depend on the points of the brain that are exposed to stimulation and the parameters of the stimulation, particular points’ coordinates and effects may vary from user to user, which makes the stages (101) of the visual cortex mapping and (102) of developing individual stimulation strategy important to account for the individual features of a particular user. Individual stimulation strategy is built via a computer program based on the mapping results and the features of the visual artefacts to be induced by the TMS unit. For visual effects, such a program determines the dependency of the visual artefacts parameters and the stimulation parameters. Today, there are no programs available on the market for creating individual stimulation strategies for TMS, but the feasibility is proven by similar solutions in the field of retinal visual prosthetics (direct stimulation of the neurons of the visual system), for example, Argus II.

According to one embodiment, the program for building individual stimulation strategy first identifies the TMS intensity that provides stable visual effects (phosphenes). After that, several different TMS coil positions evoking stable and well noticeable for the user phosphenes are to be determined. This is performed by shifting the TMS coil along the occipital head area and applying stimulation thereto, then fixing the positions of the TMS coil on which the user consistently sees bright phosphenes.

After creating the stimulation strategy, the information thereabout is synchronized with the VR application scenario in the virtual reality time axis. Synchronization is usually achieved through specialized software utilities or other software solutions that generates the controlling instructions for the time characteristics of the stimulation for providing the needed visual artefacts (phosphenes) based on the analysis of the individual stimulation strategy and the features of the visual artefacts to be induced. In order to properly perform this step it is desired that the navigational TMS unit enables exporting of the study data.

Finally, the stimulation is provided for the user according to the VR application scenario and the developed individual stimulation strategy: TMS unit is activated and induces a magnetic field onto certain points of the visual cortex that were determined during cortex mapping and that were assigned a particular property according to the built individual stimulation strategy.

According to some embodiments, the method can also include a user monitoring unit that traces signals indicative of the user’s condition. In general, TMS stimulation is considered safe, but it is undesirable to use it, if the user has brain diseases in the anamnesis, has metal plates in the skull, implants, or pacemakers. The use of transcranial magnetic stimulation is not recommended for patients who underwent main cerebral vessel transplantation and have large aneurysms of the brain vessels. At the same time, the risk of convulsive seizures using high- frequency and intense stimulation protocols, or when using psychoactive substances increases. Seizures are described mainly during stimulation via electrodes and are due to the involvement of large groups of neurons. According to the studies, the first marker of the occurrence of adverse consequences of excessive stimulation of TMS may be the occurrence of visual sensations that go beyond the time frames of stimulation. Monitoring of the occurrence of such effects may avoid complications. It is also important to note that the majority of the commercially available navigational transcranial magnetic stimulators are designed to operate in the range of frequencies and intensities that are potentially safe, and the preliminary adjustment to the individual user’s perception is meant to determine the particular parameters ranges that are most efficient and comfortable for the user based on the user’s feedback during mapping and developing individual stimulation strategy.

Speaking of the technique of invoking visual artefacts (which are also referred to herein as phosphenes) via TMS is well known in the art, while the parameters of the stimulation may vary from user to user. According to studies, there is no value of the intensity that could be elaborated for most users as a threshold for providing a reproducible phosphene.

According to experimental stimulation sessions, in a vast majority of instances, the phosphenes are white or grey. The occurrence of color phosphene is possible, easier to reproduce those in weak shades of red and green. The possibilities of controlling the colour of the TMS-induced phosphenes are elaborated in the commonly available studies,

Without a specific stimulation type, the form of phosphenes usually corresponds to "clouds", "bubbles", or lines with distinct contours. Localization of the occurrence of phosphene depends on the stimulated region in the somatotopic map. According to studies, there are more points for which stimulation provides phosphene at the bottom of the field of a user’s view than those that produce phosphene at the top of the view field. It also can be noted that once determined, the size and shape of phosphene are usually constant within one singlepoint stimulation session.

In addition to color, size and shape, the phosphenes may differ in the direction of their motion. Moving phosphenes are caused by TMS stimulation of the V5/MT region (the midtemporal region of the extrastriate visual cortex), which is believed to play an important role in motion perception. According to studies, in order to find the stimulation region to call moving phosphene, the TMC coil needs to be offset by 2.5 -3 cm above and 5 cm lateral to the occipital tuber, while the single-pulse V5/MT stimulation of the left hemisphere region is more likely to appear as a moving phosphene. Also, for the highest reliability of bright moving phosphene, a three-pulse paradigm of stimulation is possible when a series of three consecutive stimuli with an interval of 25 ms and a stimulation force of 130% of the threshold of the occurrence of the phosphene is applied for invoking one phosphene.

Allowing for the above mentioned principles of inducing one or other properties of the artefact, the stage of adjusting the TMS for a particular user’s individual characteristics, which according to some of the embodiments includes performing a visual cortex mapping and developing an individual stimulation strategy for the user based on the visual cortex mapping, is intended to focus on ensuring that the stimulation will produce the desired parameters of the artefact(s), such as stability, noticeability, brightness, its location in the user’s visual field and the like, or any combination thereof. According to different embodiments, the information about the desired parameters of the artefact can be provided before said adjusting stage, during the adjusting or after that.

Further development of the disclosed concept may also allow a reduction of the load on VR graphics processors and expand the visual fields. While performing a video or VR application, rendering and drawing of each frame require the greater resources the better the graphics has to be. With the use of the proposed TMS approach, part of the processing related to picture production can be reallocated on the TMS as described herein. Thereby, part of the picture can be directly induced in the visual cortex. Speaking of the view field, a conventional VR display is not exactly hemisphere- shaped and therefore cannot provide the 180°vision, especially peripheral vision, and with the use of the present invention, it becomes possible to expand the view field with the help of TMC. Such peripheral virtual view areas may be produced by the proposed device contributing to immersion effect of the application. As may be understood by a person skilled in the art, implementation of such feature is associated with developing corresponding stimulation strategy according to the previously obtained mapping results. According to studies available at the moment, there are visual cortex areas responsive for the peripheral vision.

In order to effectively implement some of the features of the claimed invention, the controller system is also to account for the physiological features of the user’s perception mechanisms and cortex functions, some examples of which are mentioned above (regarding the colour and the motion of the phosphenes) and below. However, it is understood by a person skilled in the art that many other features described in the prior art are applicable here too, if considered necessary for particular embodiments.

While the prior art does not offer ready-made solutions for creating specific objects from phosphenes, it provides numerous studies that explore and illustrate the possibilities of achieving such a result using artificial stimulation of the visual cortex. For example, one of such works (Michael S. Beauchamp, Denise Oswalt, Ping Sun, Brett L. Foster, John F. Magnotti, Soroush Niketeghad, Nader Pouratian, William H. Bosking, Daniel Yoshor, Dynamic Stimulation of Visual Cortex Produces Form Vision in Sighted and Blind Humans, Cell, Volume 181, Issue 4, 2020, Pages 774-783. e5, ISSN 0092-8674, https://doi.Org/10.1016/j.cell.2020.04.033) uses direct stimulation of the visual cortex using a matrix of electrodes, which was placed on the surface of the brain and via which a controlled stimulation was applied that allowed the patient to see letters. A similar result is achievable with any other method that activates brain neurons. Depending on the exposure area of each method (the number of neurons simultaneously involved in a single stimulation), the properties of phosphenes may differ, i.e. the effect will differ from method to method. Below are some physiological features of the visual cortex related to phosphenes.

In order for the artefact details to become noticeable, their brightness differences relative to the background must reach a threshold for perceptual contrast. This level of perception is primarily described by brightness characteristics rather than spatial characteristics, so the later has higher priority. For the periphery of the user’s visual field, the response (brain reaction to stimulus) is brighter if the outlines of visual artifacts are thicker (or the artifacts themselves are larger) and this dependence is linear. At the same time, flickering allows amplification of the perception of the object or its parts, if necessary. In addition to the above, is necessary to take into account the general concept of visual perception, namely, the "stages of shape vision", since they form the basis for attention control. In order of increasing complexity, the stages are the ability to detect the presence of an object (minimum visible); the ability to see the structure of an object in detail (minimum separable or resolvable); the ability to recognize and identify an object (minimum cognoscible); and evaluation of the spatial arrangement of objects (spatial minimum discriminable) and their shapes along the contour (minimum deformable).

It should be noted that the above-mentioned embodiments illustrate rather than limit the invention and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. A controller system and navigational transcranial magnetic stimulation unit may be implemented by means of hardware comprising several distinct elements, or by means of a one-body device, while the functionality according to the claimed invention may be provided by means of a suitably programmed processor or processors.

In the claims, the word "comprising" does not exclude the presence of elements or steps other than those listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. In a device claim enumerating several means, all or some of these means may be embodied by one and the same item of hardware. Measures recited in mutually different dependent claims may advantageously be used in combination.

Claims

CLAIMS:

1. A method for providing a virtual reality image to a user involving the application of transcranial magnetic stimulation (TMS) to the user, the method comprising: a) adjusting a navigational TMS unit to the user’s individual characteristics including the user’s anatomy and visual perception; b) providing information about at least one virtual reality artefact to the navigational TMS unit; c) controlling the navigational TMS unit to affect the user’s brain regions in order to provide the at least one virtual reality artefact to the user according to the provided information about at least one virtual reality artefact, wherein said adjusting (a) includes determination of the TMS intensity, the TMS applicator positions, and TMS frequency providing the at least one virtual reality artefact.

2. A method for providing a virtual reality image to a user according to claim 1, wherein stage a) further includes the following: obtaining a magnetic resonance imaging (MRI) scan of the user’s brain by performing a MRI thereof or downloading the results of such scan; performing a visual cortex mapping based on the brain MRI scan in order to obtain the information about at least two active visual cortex points the stimulation of which provides stable parameters of a visual artefact; developing an individual stimulation strategy for the user based on visual cortex mapping; and stage c) is performed based on said individual stimulation strategy developed for the user.

3. A method for providing a virtual reality image to a user according to any one of claims 1-2, wherein said adjusting includes determining the TMS frequency providing the required stability and noticeability of the at least one virtual reality artefact.

4. A method for providing a virtual reality image to a user according to any one of claims 1-3, wherein the parameters of the visual artefact include at least one of its brightness and its location in the user’s visual field.

5. A wearable device for providing a virtual reality image to a user involving the application of transcranial magnetic stimulation (TMS) to the user according to the method of claim 1, the device comprising: a controller system in communication with a virtual reality artefact-generating source configured for producing controlling signals based on the information about the virtual reality artefact to be generated and the user information; a navigational transcranial magnetic stimulation unit arranged in close proximity to the user’s head so that it can provide the TMS for the user, the unit configured to provide said TMS according to the controlling signals received from the controller system; wherein the controlling signal includes the TMS intensity controlling signals, the TMS applicator positions controlling signals and TMS frequency controlling signals.

6. A wearable device for providing a virtual reality image to a user according to claim

5, wherein the controller system is further in communication with a virtual reality image displaying unit.

7. A wearable device for providing a virtual reality image to a user according to claim

6, wherein the one or more virtual reality artefacts are to be produced in the periphery of the user’s view field.