WO2021119766A1

WO2021119766A1 - Mixed reality system for treating or supplementing treatment of a subject with medical, mental or developmental conditions

Info

Publication number: WO2021119766A1
Application number: PCT/AU2020/051409
Authority: WO
Inventors: John William Down; Gregory Watson
Original assignee: John William Down; Gregory Watson
Priority date: 2019-12-19
Filing date: 2020-12-21
Publication date: 2021-06-24

Abstract

A mixed reality system for treating or supplementing treatment of a subject with mental or developmental conditions, comprising a database comprising indications of a selected treatment for an identified condition of a subject, stimulatory devices that provide visual, auditory, olfactory and/or haptic stimulation, a processor to process information related to the indications, a controller controlling operation of the stimulatory devices to provide a first set of stimulatory content to the subject, sensors being configured to generate sensory data in response to the first set wherein the sensory data is indicative of a current state of the subject and is used to assess the current state in relation to a state threshold. In response to the current state being assessed as greater than a state threshold, the stimulatory devices to provide a second set of content for modifying the current state to be equal to or below the state threshold.

Description

Mixed Reality System for treating or supplementing treatment of a subject with medical, mental or developmental conditions

TECHNICAL FIELD

[001] The present invention relates to a mixed reality system or method of providing therapy particularly to patients with mental or developmental conditions such as but not limited to autism. The invention may be useful in medical, psychiatry, psychotherapy, education and may be implemented using computer hardware and computer software.

BACKGROUND

[002] Any references to methods, apparatus or documents of the prior art are not to be taken as constituting any evidence or admission that they formed, or form part of the common general knowledge.

[003] Autism is known to be a complex neurodevelopmental disorder in which affected subjects commonly show difficulties engaging in social interactions and communication and repetitive behaviours. Although there is no known cure, autism therapy can ranges across a myriad of specialties including neurology, psychiatry, physical therapy, occupational therapy, behavioural therapy, and speech language pathology. Such therapeutic methods can help autistic individuals gain social and communication skills.

[004] The use of psycho-pharmacological medications dominate psychiatric practice today. However, pharmacological interventions provide symptom management and patients report some relief from emotional distress. However, such interventions can also lead to day-to-day life restricting side-effects of the medications. For example, psychiatric medications frequently leave patients with dry mouth, constipation, reduced or suppressed sexual interest, weight gain, bloating, sedation, benzodiazepine dependence and withdrawal, frustration with treatment failure, and dependence on the psychiatrist. Therefore, it is desirable to provide at least an alternative to known pharmacological intervention methods for supplementing treatment of a patient with mental or developmental conditions.

[005] Virtual or augmented reality is the coordinated experience of a user in an environment manufactured by and housed in computer processors and memory, where the environment may be coordinated with the user's motion and use of the senses such as sight, sound, and touch. Virtual reality systems are often used to create simulations so that a user may learn or train while being exposed to little or no physical risk.

[006] Virtual reality based applications and solutions have been developed for art, business, entertainment, flight simulators, medicine, and military battlefield operations. Until 1993, medical applications included computed-aided surgery, building designs for handicapped persons, wheelchair equipped with a virtual reality system, rehabilitation, repetitive strain injury, surgical workstation, and teaching aids for surgeons. However, none of the known method of using virtual reality for medical applications address the issue of assisting or supplementing treatment of patients with mental or developmental conditions.

SUMMARY OF INVENTION

[007] In an aspect, the invention provides a mixed reality system for treating or supplementing treatment of a subject with medical mental and/or developmental conditions, comprising: a database comprising an account of the subject comprising one or more indications of a selected treatment related to the subject for treating an identified medical, mental and/or developmental condition of the subject, one or more stimulatory devices that provide visual, auditory, olfactory and/or haptic stimulation; a processor in communication with the database and the one or more stimulatory devices to process information related to the one of more indications related to the subject in accordance with one or more rules saved on a non-volatile memory device in communication with the processor; a controller in communication with the processor, the controller being arranged to receive control signals from the processor for providing visual, auditory, olfactory and/or haptic stimulation to the subject by controlling operation of the one or more stimulatory devices in accordance with the indications of the selected treatment saved on the account of the subject on the database to provide a first set of content to the subject thereby providing visual, auditory, olfactory and/or haptic stimulation to the subject; one or more sensors in communication with the processor, the one or more sensors being configured to generate sensory data of the subject in response to the first set of content, wherein the sensory data is indicative of a current state of the subject and is used to assess the current state of the subject in relation to a state threshold, wherein, in response to the current state of the subject being assessed as greater than a state threshold, the controller is further arranged to control operation of the one or more stimulatory devices to provide a second set of content to the subject for modifying the current state of the subject to be equal to or below the state threshold thereby calming the subject.

[008] In an embodiment, the system further comprises one or more virtual or augmented reality devices for providing the first set of content and the second set of content to the subject.

[009] In an embodiment, the sensors are in communication with the controller for progressively controlling operation of one or more virtual or augmented reality devices. [010] In an embodiment, the system comprises a three dimensional space for positioning the one or more stimulatory devices and allowing the subject to be provided with the first set of content and the second set of content in the three dimensional space.

[011] In an embodiment, the system further comprises: an identification processing module for identifying the subject and processing identity parameters received on a user interface, the identification module being in communication with the processor and the controller for verifying identity of the subject and in response to identifying the subject retrieving information in relation to the indication of the selected treatment related to the identified subject and processing said information to generate control signals for providing visual, auditory, olfactory or haptic stimulation to the identified subject thereby providing the selected treatment to the identified subject.

[012] In an embodiment, the user interface is arranged to be accessible by a parent, therapist, carer or peer associated with the subject.

[013] In an embodiment, the database and the processor are located at a remote location relative to the one or more virtual or augmented reality devices. [014] In an embodiment, the system further comprises an evaluation module in communication with the sensors for evaluating an experience of the subject to determine progress of the subject towards one or more therapeutic goals.

[015] Preferably, the one or more stimulatory devices comprise one or more visual stimulatory devices. Preferably, the one or more visual stimulatory devices comprise at least one of a television, a projector and a tablet computer.

[016] Preferably, the one or more stimulatory devices comprise one or more auditory stimulatory devices. Preferably, the one or more auditory stimulatory devices comprise at least one of a speaker and a siren.

[017] Preferably, the one or more stimulatory devices comprise one or more olfactory stimulatory devices. Preferably, the one or more olfactory stimulatory devices comprise at least one of a mister and a vaporizer.

[018] Preferably, the one or more stimulatory devices comprise one or more haptic stimulatory devices. Preferably, the one or more haptic stimulatory devices comprise at least one of a vibratory device and a force feedback device.

[019] Preferably, the one or more stimulatory devices comprises one or more of the visual stimulatory devices, the auditory stimulatory devices, the olfactory stimulatory devices and the haptic stimulatory devices.

[020] In another aspect, the invention provides a method of using mixed reality for treating or supplementing treatment of a subject with medical, mental and/or developmental conditions, comprising the steps of: compiling a database with an account of the subject, the account comprising one or more indications of a selected treatment related to the subject for treating an identified medical, mental and/or developmental condition of the subject, placing the subject in a three dimensional space with one or more stimulatory devices that provide visual, auditory, olfactory and/or haptic stimulation; processing, by using a processor in communication with the database, information related to the one of more indications related to the subject in accordance with one or more rules saved on a non-volatile memory device in communication with the processor; and controlling, by using a controller, operation of the one or more stimulatory devices in accordance with the indications of the selected treatment saved on the account of the subject on the database to provide a first set of content to the subject thereby providing visual, auditory, olfactory and/or haptic stimulation to the subject; generating, by using one or more sensors, sensory data of the subject in response to the first set of content, wherein the sensory data is indicative of a current state of the subject; assessing the current state of the subject in relation to a state threshold; and in response to the current state of the subject being assessed as greater than the state threshold, controlling, by using the controller, operation of the one or more stimulatory devices to provide a second set of content to the subject for modifying the current state of the subject to be equal to or below the state threshold thereby calming the subject.

[021] In an embodiment, the method further comprises the step of progressively controlling the operation of the one or more virtual or augmented reality devices in response to acquisition of the sensory data.

[022] In an embodiment, the method further comprises the step of: identifying, using an identifying module, the subject and processing identity parameters received on a user interface, the identification module being in communication with the processor and the controller; verifying identity of the subject and, in response to identifying and verifying the subject, retrieving information in relation to the indication of the selected treatment related to the identified subject and processing said information to generate control signals for providing visual, auditory, olfactory and/or haptic stimulation to the identified subject thereby providing the selected treatment to the identified subject. [023] In an embodiment, the method further comprises the step of an experience of the subject, using an evaluation module in communication with the sensors, to determine progress of the subject towards one or more therapeutic goals.

[024] In an embodiment, the first set of content provided by the stimulatory devices is interactive, such that the subject can interact with the first set of content.

[025] In an embodiment, the first set of content is responsive and adapts to interaction by the subject.

[026] In another aspect, the invention provides a system for treating or supplementing treatment of a subject with mental or developmental conditions, comprising: a database comprising a subject’s account comprising one or more indications of a selected treatment related to the subject, one or more virtual or augmented reality devices that provide visual, auditory, olfactory or haptic stimulation; a processor in communication with the database and said virtual or augmented reality devices to process information related to the one of more indications related to the subject in accordance with one or more rules saved on a non-volatile memory device in communication with the processor; and a controller in communication with the processor, the controller being arranged to receive control signals from the processor for providing visual, auditory, olfactory or haptic stimulation to the subject by controlling operation of the one or more virtual or augmented reality devices in accordance with the indications of the selected treatment saved on the subject’s account on the database.

[027] In an embodiment, the system further comprises one or more sensors for generating sensory data of the subject in response to the visual, auditory, olfactory or haptic stimulation, wherein the sensory data is indicative of the subject's progress toward one or more therapeutic goals.

[028] In an embodiment, the sensors are in communication with the controller for progressively controlling the operation of the one or more virtual or augmented reality devices.

[029] In an embodiment, the system comprises a three dimensional space for positioning said virtual or augmented reality devices and allowing the subject to be provided with said visual, auditory, olfactory or haptic stimulation in the three dimensional space.

[030] In an embodiment, the system further comprises: an identification processing module for identifying the subject and processing identity parameters received on a user interface, the identification module being in communication with the processor and the controller for verifying identity of the subject and in response to identifying the subject retrieving information in relation to the indication of the selected treatment related to the identified subject and processing said information to generate control signals for providing visual, auditory, olfactory or haptic stimulation to the identified subject thereby providing the selected treatment to the identified subject.

[031] In an embodiment, the user interface is arranged to be accessible by a parent, therapist, carer or peer associated with the subject. [032] In an embodiment, the database and the processor are located at a remote location relative to the one or more virtual or augmented reality devices.

[033] In an embodiment, the system further comprises an evaluation module in communication with the sensors for evaluating the subject’s virtual or augmented reality experience to determine the subject’s progress towards the one or more therapeutic goals.

[034] In another aspect, the invention provides a method of treating or supplementing treatment of a subject with mental or developmental conditions, comprising the steps of: compiling a database with the subject’s account, the account comprising one or more indications of a selected treatment related to the subject, placing the subject in a three dimensional space with one or more virtual or augmented reality devices that provide visual, auditory, olfactory or haptic stimulation; processing, by using a processor in communication with the database, information related to the one of more indications related to the subject in accordance with one or more rules saved on a non-volatile memory device in communication with the processor; and controlling, by using a controller, operation of the one or more virtual or augmented reality devices in accordance with the indications of the selected treatment saved on the subject’s account on the database to provide visual, auditory, olfactory or haptic stimulation to the subject.

[035] In an embodiment, the method further comprises the step of generating, by using one or more sensors, sensory data of the subject in response to the visual, auditory, olfactory or haptic stimulation, wherein the sensory data is indicative of the subject's progress toward one or more therapeutic goals.

[036] In an embodiment, the method further comprises the step of progressively controlling the operation of the one or more virtual or augmented reality devices in response to acquisition of the sensory data.

[037] In an embodiment, the method further comprises the step of: identifying, using an identifying module, the subject and processing identity parameters received on a user interface, the identification module being in communication with the processor and the controller; verifying identity of the subject and, in response to identifying and verifying the subject, retrieving information in relation to the indication of the selected treatment related to the identified subject and processing said information to generate control signals for providing visual, auditory, olfactory or haptic stimulation to the identified subject thereby providing the selected treatment to the identified subject. [038] In an embodiment, the method further comprises the step of evaluating the subject’s virtual or augmented reality experience, using an evaluation module in communication with the sensors, to determine the subject’s progress towards the one or more therapeutic goals.

BRIEF DESCRIPTION OF THE DRAWINGS

[039] Preferred features, embodiments and variations of the invention may be discerned from the following Detailed Description which provides sufficient information for those skilled in the art to perform the invention. The Detailed Description is not to be regarded as limiting the scope of the preceding Summary of the Invention in any way. The Detailed Description will make reference to a number of drawings as follows:

Figure 1 is a box diagram for a system 100 for treating or supplementing treatment of a subject for a mental or developmental disorder in accordance with an embodiment of the present invention.

Figure 2 is box diagram for a computer system 300 in communication with the system 100.

Figure 3 is a box diagram for a mixed reality system 400 for treating or supplementing treatment of a subject for a mental or developmental disorder in accordance with an embodiment of the present invention.

Figure 4 is box diagram for a computer system 300 in communication with the system 400.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS [040] In one or more preferred embodiments, systems and methods are provided for treating or supplementing treatment of a subject for a medical, mental and/or developmental disorder, such as, for example, autism, attention deficit hyperactivity disorder (ADHD), and social anxiety disorder. [041] The method and system requires compilation of a database 110 comprising the subject’s account or user profile with important information on indication of treatment related to the subject. In some instances, the method may involve undertaking several data compilation steps for creating the user profile and acquiring sufficient information in relation to the subject’s condition before commencing with VR or AR therapy. By way of example, the initial database compilation step may involve or require the subject to interact with a therapist, parent, or peer to help the subject progress toward the one or more therapeutic goals. For example, the subject 255 may be paired with the therapist, parent, or peer 235, or a plurality of therapists, parents, or peers, or any combination of the above. Once the information has been acquired or received into the database through an input interface 115, the information may be saved securely on a secure server. It is important to note that the secure server may be located remotely relative to the location of providing the VR or AR therapy and known security protocols may be used for ensuring that the data associated with the subject’s treatment is not released from the server without appropriate verification as will be discussed in the foregoing sections.

[042] A treatment chamber 200 comprising a three dimensional space with one or more of the virtual or augmented reality devices 120 may be provided in order to provide visual, auditory, olfactory or haptic stimulation to a subject for treatment or supplementing treatment of a mental or developmental disorder. This step may comprise using a mixed reality system comprising a virtual or augmented reality system with virtual or augmented reality devices 120 to place the subject in a virtual world, where the subject can be presented with visual, auditory, and/or haptic stimulation in accordance with the subject’s user profile as saved on the database. In at least some embodiments, the augmented reality devices 120 may include spatial augmented reality devices that provide projections to map a virtual interactive space onto a physical object or a space thereby turning objects or surfaces that are often irregular in shape into a display surface for video projection.

[043] Specifically, a processor 130 may be arranged to communicate with the database 110 and the virtual and/or augmented reality devices 120 to process information related to the one of more indications related to the subject as saved on the database 110. Upon receiving information from the database 110 about the user profile of the subject, the information may be processed in accordance with one or more rules saved on a non-volatile memory device 135 which is in communication with the processor 130. Upon processing the information, one or more control signals may be sent to a controller 140 arranged in communication with the processor 130 and the virtual and/or augmented reality devices 120 for providing visual, auditory, olfactory or haptic stimulation to the subject by controlling operation of the one or more virtual or augmented reality devices in accordance with the indications of the selected treatment saved on the subject’s user profile on the database.

[044] The systems and methods as described herein can be used for using virtual and augmented reality devices 120 for treating the subject for a given mental or developmental disorder, such as autism. The systems and methods can be used for treating a subject suffering a deficiency in a given social skill (e.g., ability to retain attention). The virtual world, and the one or more stimulations (e.g., visual, auditory, haptic) presented in the virtual world, may be designed to treat the given mental developmental disorder and/or improve the given social skill by processing information related to the subject, the information being indicative of a selected treatment that is likely to be suitable for the subject. For example, the virtual world can be tailored for a given mental or developmental disorder. Alternatively or in addition, the virtual world can be tailored for a given social skill. Alternatively or in addition, the virtual world can be tailored for a subject. One or more parameters and/or stimulations in a virtual world can be provided or adjusted in real-time, such as by another user (e.g., therapist, operator, supervisor) monitoring the subject in the virtual world, for example, based on the subject's interactions, reactions, or responses (or lack thereof) as saved on the user’s profile.

[045] Visual stimulation may also be provided by presenting one or more images or video on a display screen. Auditory stimulation may be presented through one or more headphones or speakers of the virtual or augmented reality system. The one or more headphones or speakers may be synchronized with images or video provided by the display screen to the subject. Haptic stimulation may be presented through the one or more controllers 140 attached (or otherwise coupled) to the body and/or limb of the subject. The monitoring may also permit another individual, such as a supervisor, to monitor treatment. The supervisor may be a therapist, an operator, a teacher, or a parent. The monitoring may or may not be remote. The monitoring may also be carried out over one or more networks.

[046] It is also preferable to provide the virtual and augmented reality devices 120 in a secure three dimensional space such as but not limited to a treatment room 200. In the preferred embodiment, entry into the three dimensional space may be restricted to authorised subjects. In one embodiment, an identification processing module 170 may be used to identifying and authorising subjects by receiving one or more input parameters on a user input interface from either the subject or a helper, carer, therapist or medical professional assisting the subject. The identification module 170 is arranged to be in communication with the processor 130 and the controllers 140 for initially verifying identity of the subject and in response to identifying the subject allowing the subject to access the three dimensional space of the treatment room 200 for receiving the visual reality and augmented reality treatment. The identification module 170 is also arranged to retrieve information in relation to the indication of the selected treatment related to the identified subject from the database 110 and the processor 430 and process the information to generate subject specific control signals which may then be communicated back to the controllers 140 for providing customised or specialised visual, auditory, olfactory or haptic stimulation to the identified subject thereby providing the selected treatment to the identified subject.

[047] The method and system of providing stimulation through the use of virtual and augmented reality devices 120 may also involve monitoring the subject's interaction with the virtual and/or augmented reality devices and measuring the subject's progress toward one or more therapeutic goals. The subject's interactions, reactions, and/or responses to the stimulations presented in the virtual world may be quantified based at least in part on sensory data measured for the subject, such as a reaction time, gaze accuracy, gaze stability, response volume, and/or other forms or units of outputs by the subject. In some cases, the subject's progress toward one or more therapeutic goals may also be quantified based at least in part on such quantifications. The subject's progress toward one or more therapeutic goals may also be measured based on qualitative observations made by another person monitoring the subject's interactions, reactions, and responses in the virtual world, such as a therapist, operator, or supervisor. For example, the subject's progress toward one or more therapeutic goals may be measured based on subjective and/or objective feedback from the monitoring user.

[048] In at least some embodiments, the subject may interact with the virtual world or the augmented world in the treatment room 200 using eye movement, head movement, or one or more trackers 230 attached to the body and/or the limb of the subject. To track the subject's movement, the one or more sensors 240 may be employed. In some embodiments, these sensors 240 may include one or more cameras. The system may also employ other optical sensors, auditory sensors (e.g., microphones), touchpads, touchscreens, motion sensors, heat sensors, inertial sensors, touch sensors, or other sensors. The one or more sensors 240 may be capable of measuring sensory data indicative of an output by a subject (e.g., eye movement, head movement, facial expressions, speech, etc.). The sensory data may be arranged in communication with the controllers 140 and the processor 130 for progressively controlling the operation of the virtual or augmented reality devices 120 in the treatment room 200 based on the sensory data generated by the sensors 240. [049] As outlined earlier, the subject may interact with an authorised therapist, supervisor (e.g., parent), or peer to progress toward the one or more therapeutic goals. The therapist, supervisor, or peer may also be located remotely from the subject. In some embodiments, the sensory data may be received by the therapist or supervisor or a peer associated with the subject. In response to receiving the sensory data, the therapist or the supervisor or the peer may provide additional input on a user input interface to progressively manage the operation of the virtual and augmented reality devices 120. Once again, an authorisation module 270 may be provided to ensure that the identity of the therapist, supervisor or peer is authenticated and correctly matched with the user profile of the subject before any sensory data is received by the [050] The systems and methods may allow a therapist to prescribe one or more therapy sessions to achieve one or more therapeutic goals to a subject. The subject may perform the prescribed therapy sessions, without direct supervision from the therapist prescribing the therapy sessions. In some instances, the therapy sessions are capable of being reviewed at a later time and place. Alternatively or in addition, the therapy sessions are capable of being monitored directly or indirectly (e.g., remotely) in real time. In some instances, the therapy sessions can be monitored by another individual, such as a parent or guardian in real time or at a later time and place.

[051] The virtual or augmented reality system may be a virtual reality system in which the subject is presented with content in an environment that may be separate from the surrounding of the subject. Alternatively, the virtual or augmented reality system may be an augmented reality system in which the subject is presented with content that may be overlaid or at least partially integrated with the environment of the subject. Such system can comprise a display for presenting the subject with content. Such display can be provided to the subject through a user device such as but not limited to a VR headset or a mobile phone that can be docked into a VR headset. The user device may or may not be portable. Such system may further comprise one or more headphones, earphones, or speakers for presenting the subject with audio. The one or more headphones, earphones, or speakers may be synchronized with images or video provided by the display screen to the subject. The subject may access the virtual or augmented reality system with the use of a supplemental headgear (e.g., Google® Daydream/Cardboard, Oculus® Gear/Rift, and HTC® Vive). These systems may work in conjunction with the one or more controllers 140 for presenting the subject with visual, auditory or haptic stimulation. Similarly, olfactory stimuli may also be provided in conjunction with the controllers 140 for providing olfactory stimulation. In addition, the controllers 140 may also present the subject with haptic output. The controllers 140 can be configured to, for example, vibrate. The controllers 140 may also comprise an actuator.

[052] Using the virtual or augmented reality systems may advantageously provide various therapeutic values to a subject (or user) receiving treatment. The subject having or suspected of having a neurodevelopmental disorder, such as autism, may be treated by receiving VR or AR content through the system.

[053] It is believed by the inventors that building up an initial database 110 comprising the subject’s user profile with data comprising one or more indication of a selected treatment and utilising the database for controlling the operation of the virtual and augmented reality devices 120 allows the VR or AR to be effectively tailored to the subject’s needs. It has been found by the inventors that providing individualised virtual and augmented reality experiences for subjects is far more advantageous. Moreover, developing a tailored plan by accessing the subject’s information from a secure database helps in making the VR and AR treatment to be more effective.

[054] The present disclosure provides may use one or more computers to implement the method and system of using virtual reality and augmented reality for treatment of the subject with mental or developmental conditions such as autism. Figure 2 denotes a computer system 300 that is programmed or otherwise configured to operate and display a virtual reality or augmented reality environment with or without the use of supplemental headsets or gear, execute algorithms within the virtual reality or augmented reality platform or a companion application to the virtual reality or augmented reality platform, track, save, and analyze a user's behaviours and responses in a VR or AR environment, and execute and pair a companion application that may allow the computer system 300 to communicate with a remotely located database 110 to retrieve information from the subject’s user profile and in response control in real-time a user's virtual reality or augmented reality experience in the treatment room 200. The computer system 300 can be an electronic device of a user or a computer system that is remotely located with respect to the electronic device. The electronic device can be a mobile electronic device. The computer system 300 may be used to treat a neurodegenerative disorder, such as, for example, autism. The computer system 300 may communicate with the one or more controllers 140. In some instances, the control system 140 may include the computer system 300.

[055] The computer system 300 includes a central processing unit (CPU, also “processor” and “computer processor” herein) 305, which can be a single core or multi core processor, or a plurality of processors for parallel processing. The computer system 300 also includes memory or memory location 310 (e.g., random-access memory, read-only memory, flash memory), electronic storage unit 315 (e.g., hard disk), communication interface 320 (e.g., network adapter) for communicating with one or more other systems, and peripheral devices 325, such as cache, other memory, data storage and/or electronic display adapters. The memory 310, storage unit 315, interface 320 and peripheral devices 325 may be in communication with the CPU 305 through a communication bus (solid lines), such as a motherboard. The storage unit 315 can be a data storage unit (or data repository) for storing data. The computer system 300 can be operatively coupled to a computer network (“network”) 330 with the aid of the communication interface 320. The network 330 can be the Internet, an internet and/or extranet, or an intranet and/or extranet that is in communication with the Internet. The network 330 in some cases may be a telecommunication and/or data network. The network 330 may include one or more computer servers, which may enable distributed computing, such as cloud computing.

[056] The CPU 305 may execute a sequence of machine-readable instructions, which can be embodied in a program or software. The instructions may be stored in a memory location, such as the memory 310.

[057] The CPU 305 can be part of a circuit, such as an integrated circuit. One or more other components of the system 300 can be included in the circuit. In some cases, the circuit is an application specific integrated circuit (ASIC). The storage unit 315 can store files, such as drivers, libraries and saved programs. The storage unit 315 can store user data, e.g., user preferences and user programs. The computer system 300 in some cases may include one or more additional data storage units that are external to the computer system 300, such as located on a remote server that is in communication with the computer system 300 through an intranet or the Internet. [058] The computer system 300 can communicate with one or more remote computer systems through the network 330. For instance, the computer system 300 may communicate with a remote computer system of a user (e.g., VR or AR user, supervisor, therapist). Examples of remote computer systems include personal computers (e.g., portable PC), slate or tablet PC's (e.g., Apple® iPad, Samsung® Galaxy Tab), telephones, Smart phones (e.g., Apple® iPhone, Android-enabled device, Blackberry®), or personal digital assistants. The user can access the computer system 300 via the network 330.

[059] Methods as described herein may be implemented by way of machine (e.g., computer processor) executable code stored on an electronic storage location of the computer system 300, such as, for example, on the memory 310 or electronic storage unit 315. The machine executable or machine readable code can be provided in the form of software. During use, the code can be executed by the processor 805. In some cases, the code can be retrieved from the storage unit 315 and stored on the memory 310 for ready access by the processor 305. In some situations, the electronic storage unit 815 can be precluded, and machine-executable instructions are stored on memory 310.

[060] Aspects of the systems and methods provided herein, such as the computer system 300, can be embodied in programming. Various aspects of the technology may be thought of as “products” or “articles of manufacture” typically in the form of machine (or processor) executable code and/or associated data that is carried on or embodied in a type of machine readable medium. Machine-executable code can be stored on an electronic storage unit, such as memory (e.g., read-only memory, random-access memory, flash memory) or a hard disk. “Storage” type media can include any or all of the tangible memory of the computers, processors or the like, or associated modules thereof, such as various semiconductor memories, tape drives, disk drives and the like, which may provide non-transitory storage at any time for the software programming. All or portions of the software may at times be communicated through the Internet or various other telecommunication networks. Such communications, for example, may enable loading of the software from one computer or processor into another, for example, from a management server or host computer into the computer platform of an application server. Thus, another type of media that may bear the software elements includes optical, electrical and electromagnetic waves, such as used across physical interfaces between local devices, through wired and optical landline networks and over various air-links. The physical elements that carry such waves, such as wired or wireless links, optical links or the like, also may be considered as media bearing the software. As used herein, unless restricted to non-transitory, tangible “storage” media, terms such as computer or machine “readable medium” refer to any medium that participates in providing instructions to a processor for execution. [061] A machine readable medium, such as computer-executable code, may take many forms, including but not limited to, a tangible storage medium, a carrier wave medium or physical transmission medium. Non-volatile storage media include, for example, optical or magnetic disks, such as any of the storage devices in any computer(s) or the like, such as may be used to implement the databases, etc. shown in the drawings. Volatile storage media include dynamic memory, such as main memory of such a computer platform. Tangible transmission media include coaxial cables; copper wire and fiber optics, including the wires that comprise a bus within a computer system. Carrier-wave transmission media may take the form of electric or electromagnetic signals, or acoustic or light waves such as those generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer-readable media therefore include for example: a floppy disk, a flexible disk, hard disk, magnetic tape, any other magnetic medium, a CD-ROM, DVD or DVD- ROM, any other optical medium, punch cards paper tape, any other physical storage medium with patterns of holes, a RAM, a ROM, a PROM and EPROM, a FLASH- EPROM, any other memory chip or cartridge, a carrier wave transporting data or instructions, cables or links transporting such a carrier wave, or any other medium from which a computer may read programming code and/or data. Many of these forms of computer readable media may be involved in carrying one or more sequences of one or more instructions to a processor for execution.

[062] Methods and systems of the present disclosure can be implemented by way of one or more algorithms. An algorithm can be implemented by way of software upon execution by the central processing unit 305. The algorithm can, for example, track a user's activity, including voice, motion, and gaze within the virtual reality or augmented reality environment, run analytics on metrics collected from a user's experience, provide therapeutic learning by iterating training principles with varying levels of difficulty, and translate commands from a paired control device.

[063] In one or more further preferred embodiments, systems and methods are provided for using mixed reality for treating or supplementing treatment of a subject for a mental or developmental disorder, such as, for example, autism, post-traumatic stress disorder (PTSD), attention deficit hyperactivity disorder (ADHD), and social anxiety disorder.

[064] In the context of the present disclosure, mixed reality refers to a combination of real-world and computer-generated elements.

[065] The method and system 400, shown in Figures 3 and 4, require compilation of a database 410 comprising an account of the subject 555 or user profile with important information on indication of treatment related to the subject for treating an identified medical, mental and/or development condition of the subject.

[066] In some instances, the method may involve undertaking several data compilation steps for creating the user profile and acquiring sufficient information in relation to the condition of the subject 555 before commencing with mixed reality exposure therapy which involves the use of a combination of immersion or exposure type therapy (by way of visual and/or auditory simulation of a triggering event or activity that is traumatic or anxiety inducing for a patient) alternating with sensory stimulation for calming and relaxation through a combination of real-world and computer- generated elements provided to the subject to ultimately de-sensitise or condition the subject not to react adversely to the triggering event.

[067] In some embodiments, the mixed reality therapy is utilised in the form of story based therapy, which combines individualised treatment with therapy elements, such as cognitive behavioural therapy (CBT) or dialectical behaviour therapy (DBT), for example, to maximise immersion and improve the therapeutic benefit to the subject. [068] By way of example, the initial database compilation step may involve or require the subject 555 to interact with a therapist, parent, or peer 535 to help the subject progress toward the one or more therapeutic goals. For example, the subject may be paired with the therapist, parent, or peer, or a plurality of therapists, parents, or peers, or any combination of the above.

[069] The initial interactions may be used to compile data relating to the types of stimulation (e.g. visual, auditory, olfactory and haptic/touch) that the subject finds most comforting, soothing and/or familiar and which is most likely to return the subject to a relaxed or calm state should the subject become anxious or aroused to unacceptable levels.

[070] In some embodiments, this process includes identifying memories or experiences of the subject 555 that the subject 555 finds pleasing. For example, the subject 555 may have visited the Eiffel Tower in Paris and finds the memory of this experience soothing and one that brings joy. Identifying these important memories and experiences can be useful in preparing immersive experiences for the subject 555 that can be used in the systems and methods described herein to help the subject 555 overcome their identified condition.

[071] Once the information has been acquired or received into the database 410 through an input interface 415, the information may be saved securely on a secure server 405. It is important to note that the secure server 405 may be located remotely relative to the location of providing the mixed reality therapy and known security protocols may be used for ensuring that the data associated with the subject’s treatment is not released from the server 405 without appropriate verification as will be discussed in the latter sections of this disclosure.

[072] A treatment chamber 500 comprising a three dimensional space with one or more stimulatory devices 420 may be provided in order to provide visual, auditory, olfactory and/or haptic stimulation to a subject 555 for treatment or supplementing treatment of a mental or developmental disorder. This step may comprise using stimulatory devices 420 including a television in combination with speakers and misters for visual, auditory and olfactory stimulation.

[073] In at least some embodiments, the stimulatory devices 420 may include interactive projection mapping which utilises spatial augmented reality devices that provide projections to map a virtual interactive space onto a physical object or a space thereby turning objects or surfaces that are often irregular in shape into a display surface for video projection. Thus,

[074] Specifically, a processor 430 may be arranged to communicate with the database 410 and the stimulatory devices 420 to process information related to the one of more indications related to the subject 555 as saved on the database 410. [075] Upon receiving information from the database 410 about the user profile of the subject 555, the information may be processed in accordance with one or more rules saved on a non-volatile memory device 435 which is in communication with the processor 430.

[076] Upon processing the information, one or more control signals may be sent to a controller 440 arranged in communication with the processor 430 and the stimulatory devices 420 (via processor 430) for providing a first set of content 450 (stored in non- volatile memory device 450) to the subject 555 and thereby providing visual, auditory, olfactory and/or haptic stimulation to the subject by controlling operation of the stimulatory devices 420 in accordance with the indications of the selected treatment saved on the user profile of the subject 555 on the database 410. This typically occurs through initialisation of the program by a clinician through an input/output interface 475, for example, but may also be automated.

[077] Subsequent to providing the first set of content 450 to the subject 555, one or more parameters and/or stimulations can be provided or adjusted in real-time, such as by another user (e.g., therapist, operator, supervisor) monitoring the subject 555 via sensors 540 either monitoring the subject 555 remotely (e.g. through cameras) or attached to the subject directly through trackers 530 (e.g. a heart rate monitor or a blood pressure monitor).

[078] In an embodiment, the first set of content provided by the stimulatory devices is interactive, such that the subject can interact with the first set of content and directly influence the story or narrative being presented. In such an embodiment, the first set of content may be responsive to and adapts to interaction by the subject so that the subject feels properly immersed in the narrative.

[079] These sensors 540 and trackers 530 generate sensory data of the subject 555 in response to the first set of content 450. The sensory data is indicative of a current state of the subject 555. The current state informs or assists the therapist, operator, etc. in assessing whether the subject 555 is anxious or stressed.

[080] Thus, using the sensory data, the current state of the subject 555 is assessed in relation to a state threshold.

[081] If the subject is presently or currently equal to or below the state threshold this indicates that the subject 555 is not dangerously or unacceptably anxious, aroused or stressed, and no action is currently necessary.

[082] In this situation, monitoring of the subject 555 via the sensors 540 and trackers 530 continues. [083] Alternatively, if the subject 555 is presently or currently above the state threshold this indicates that the subject 555 is dangerously or unacceptably anxious, aroused or stressed and action must be taken.

[084] In response to the current state of the subject 555 being assessed as greater than or above the state threshold, the therapist, parent, or peer 535 uses input/output interface 475 to instruct the controller 440 to control operation of the one or more stimulatory devices 420 to provide a second set of content 451 to the subject 555 for modifying the current state of the subject 555 to be equal to or below the state threshold thereby calming the subject 555. The second set of content 451 replaces the first set of content 450 being presented and is designed, either using research- based or evidence-based techniques or subject-specification techniques known to the therapist, etc. through previous observation, interaction or data provided to them, to return the subject 555 to a less anxious state.

[085] While the second set of content 451 is provided, monitoring via the sensors 540 and trackers 530 continues and the current state of the subject 555 is updated and re- assessed periodically

[086] Once the current state of the subject 555 is equal to or below the state threshold, the first set of content 450 can be provided again to the subject 555. Once again, monitoring of the subject 555 continues via the sensors 540 and trackers 530 and the current state of the subject 555 is updated and re-assessed periodically.

[087] The system 400 and methods as described herein can be used for using stimulatory devices 420 and techniques for treating the subject for a given mental or developmental disorder, such as autism. The systems and methods can be used for treating a subject suffering a deficiency in a given social skill (e.g., ability to retain attention). The one or more stimulations (e.g., visual, auditory, haptic) presented to the subject, may be designed to treat the identified mental developmental disorder and/or improve the given social skill by processing information related to the subject 555, the information being indicative of a selected treatment that is likely to be suitable for the subject 555. For example, the stimulatory content can be tailored for a given mental or developmental disorder. Alternatively or in addition, the stimulatory content can be tailored for a given social skill. Alternatively or in addition, the stimulatory content can be tailored for a subject.

[088] Visual stimulation may also be provided by presenting one or more images or video on a display screen. Auditory stimulation may be presented through one or more headphones or speakers of the stimulatory devices. The one or more headphones or speakers may be synchronized with images or video provided by the display screen to the subject. Haptic stimulation may be presented through the one or more controllers 440 attached (or otherwise coupled) to the body and/or limb of the subject. The monitoring may also permit another individual, such as a supervisor, to monitor treatment. The supervisor may be a therapist, an operator, a teacher, or a parent. The monitoring may or may not be remote. The monitoring may also be carried out over one or more networks.

[089] It is also preferable to provide the stimulatory devices 420 in a secure three dimensional space such as but not limited to a treatment room 500. In the preferred embodiment, entry into the three dimensional space may be restricted to authorised subjects.

[090] In one embodiment, an identification processing module 470 may be used to identifying and authorising subjects by receiving one or more input parameters on a user input interface from either the subject or a helper, carer, therapist or medical professional assisting the subject. The identification module 470 is arranged to be in communication with the processor 430 and the controllers 440 for initially verifying identity of the subject and in response to identifying the subject allowing the subject to access the three dimensional space of the treatment room 500 for receiving the visual reality and augmented reality treatment. The identification module 470 is also arranged to retrieve information in relation to the indication of the selected treatment related to the identified subject555 from the database 410 and the processor 430 and process the information to generate subject specific control signals which may then be communicated back to the controllers 440 for providing customised or specialised visual, auditory, olfactory or haptic stimulation to the identified subject 555 thereby providing the selected treatment to the identified subject 555.

[091] The method and system of providing stimulation through the use of stimulatory devices 420 may also involve monitoring the subject’s 555 interaction with the stimulatory devices 420 and measuring the progress of the subject 555 toward one or more therapeutic goals.

[092] The subject’s 555 interactions, reactions, and/or responses to the stimulations presented in the stimulatory content may be quantified based at least in part on sensory data obtained from sensors 540 and trackers 530 measured for the subject 555, such as a reaction time, gaze accuracy, gaze stability, response volume, and/or other forms or units of outputs by the subject 555.

[093] In some cases, the subject’s progress toward one or more therapeutic goals may also be quantified based at least in part on such quantifications. The subject’s progress toward one or more therapeutic goals may also be measured based on qualitative observations made by another person monitoring the subject’s interactions, reactions, and responses to the stimulatory content, such as a therapist, peer, or supervisor 535. For example, the subject’s progress toward one or more therapeutic goals may be measured based on subjective and/or objective feedback from the monitoring user.

[094] In at least some embodiments, the subject 555 may interact with the stimulatory content and stimulatory devices 420 in the treatment room 500 using eye movement, head movement, or one or more trackers 530 attached to the body and/or the limb of the subject 555. To track the movement of the subject 555, the one or more sensors 540 are employed. In some embodiments, these sensors 540 may include one or more cameras. The system 400 may also employ other optical sensors, auditory sensors (e.g., microphones), touchpads, touchscreens, motion sensors, heat sensors, inertial sensors, touch sensors, or other sensors.

[095] The one or more sensors 540 may be capable of measuring sensory data indicative of an output by a subject 555 (e.g., eye movement, head movement, facial expressions, speech, etc.). The sensory data may be arranged in communication with the controllers 440 and the processor 430 for progressively controlling the operation of the stimulatory devices 420 in the treatment room 500 based on the sensory data generated by the sensors 540 and/or trackers 530.

[096] As outlined earlier, the subject 555 may interact with an authorised therapist, supervisor (e.g., parent), or peer 525 to progress toward the one or more therapeutic goals. The therapist, supervisor, or peer 535 may also be located remotely from the subject. In some embodiments, the sensory data may be received by the therapist or supervisor or a peer associated with the subject.

[097] In response to receiving the sensory data, the therapist or the supervisor or the peer may provide additional input on a user input interface to progressively manage the operation of the stimulatory devices 420.

[098] Once again, an authorisation module 570 may be provided to ensure that the identity of the therapist, supervisor or peer is authenticated and correctly matched with the user profile of the subject before any sensory data is received by the therapist, supervisor or peer.

[099] The system 400 and methods may allow a therapist to prescribe one or more therapy sessions to achieve one or more therapeutic goals to a subject 555.

[100] The subject 555 may perform the prescribed therapy sessions, without direct supervision from the therapist prescribing the therapy sessions. In some instances, the therapy sessions are capable of being reviewed at a later time and place. Alternatively or in addition, the therapy sessions are capable of being monitored directly or indirectly (e.g., remotely) in real time. In some instances, the therapy sessions can be monitored by another individual, such as a parent or guardian in real time or at a later time and place.

[101] The stimulatory system and stimulatory devices 420 may be a system or environment (e.g. treatment room 500) in which the subject is presented with stimulatory content in an environment that may be separate from the surrounding of the subject 555. Alternatively, the stimulatory system and stimulatory devices 420 may provide the subject with content that may be overlaid or at least partially integrated with the environment of the subject 555.

[102] Such systems can comprise a display for presenting the subject with content. Such display can be provided to the subject through a user device such as but not limited to a VR headset or a mobile phone that can be docked into a VR headset. The user device may or may not be portable. Such systems may further comprise one or more headphones, earphones, or speakers for presenting the subject with audio. The one or more headphones, earphones, or speakers may be synchronized with images or video provided by the display screen to the subject. The subject may access the virtual or augmented reality system with the use of a supplemental headgear (e.g., Google® Daydream/Cardboard, Oculus® Gear/Rift, and HTC® Vive). These systems may work in conjunction with the one or more controllers 440 for presenting the subject 555with visual, auditory or haptic stimulation. Similarly, olfactory stimuli may also be provided in conjunction with the controllers 440 for providing olfactory stimulation. In addition, the controllers 440 may also present the subject with haptic output. The controllers 440 can be configured to, for example, vibrate. The controllers 440 may also comprise an actuator. [103] In an embodiment, the sensory data may be collected and stored in the database 410 and subsequently analysed by a clinician or doctor for diagnostic purposes.

[104] Alternatively, or additionally, the sensory data may be provided to a diagnostic tool that is trained to analyse the sensory data and provide analytical and diagnostic outputs to assist a clinician in diagnosing a medical, mental and/or developmental condition of a subject.

[105] In a first example, a child with severe autism is required to attend a tertiary hospital for surgery. Hospitals are considered high stimulation environments and very anxiety provoking for children with autism.

[106] The child may be provided with p re- medication prior to attending the hospital to minimize initial stimulation and anxiety.

[107] Once the child is in the hospital, the child is transported to an enclosed room with a stimulatory device.

[108] In one example, prior to the child attending the hospital, there was a discussion between the child’s medical team and parents about the specific types of visual, auditory and/or olfactory stimulation the child finds soothing.

[109] In an alternative example, the child has a personal sensory stimulation device which the parents have used to determine settings of the device which are most beneficial for the child.

[110] After the types of stimulation have been determined, the child sits or stands in the enclosed room with the stimulatory device and a range of settings and first set of content are used and provided to reduce the child’s anxiety to an acceptable level.

[111] To assess the anxiety of the child, there is feedback about the child’s state of arousal provided to a trained clinician using sensors which provide quantifiable measurements, such as a blood pressure monitor for monitoring blood pressure and a heart rate monitor for monitoring the heart rate of the child. This allows the clinician to assess the anxiety of the child and make determinations as to when the child is in a de-escalated (non-anxious) or escalated (anxious) statement.

[112] When the child is calm or their anxiety levels are deemed to be at an acceptable level, the stimulatory device is adjusted to provide a second set of content in the form of a narrative about what the child should expect during their hospital visit. This could include a visual representation of the corridor they would walk down to get to the surgical unit (on a television screen, for example), the face of the child’s surgeon for that day and a brief description of the anesthetist giving the child sedation to allow the operation to proceed.

[113] During this narrative, the child’s level of arousal and anxiety would continue to be monitored by monitoring the physical characteristics of the child through the sensors.

[114] If the child’s anxiety/arousal increases to unacceptable levels, then the stimulatory device is again adjusted to provide the calming first set of content to reduce the child’s anxiety. At all times the sensors are providing feedback to the clinician so that an assessment of the current state of anxiety of the child can be made.

[115] Once the child’s anxiety levels lower to an acceptable level, the second set of content can be re-introduced.

[116] This switching between the first and second sets of content may occur numerous times until the child is able to tolerate the anxiety induced by the narrative aspect in the second set of content. The child is then ready to exit the room and proceed to surgery.

[117] In a second example, a patient suffering from post-traumatic stress disorder (PTSD) is placed in an enclosed room with a stimulatory device which projects the first set of content (in the case, images pertaining to the past trauma) onto a wall of the enclosed room.

[118] It is expected that the presentation of the content will evoke strong anxiety in the patent who is then encouraged to use eye movement de-sensitization therapy (EMDR) by shifting their gaze back and forth between two alternatively flashing points that are also projected onto the wall or a ceiling of the enclosed room by the stimulatory device.

[119] During this process, the patient is monitored by a trained clinician. To assess the anxiety of the patient, there is feedback about the patient’s state of arousal, anxiety or distress provided to the trained clinician using sensors which provide quantifiable measurements, such as a blood pressure monitor for monitoring blood pressure and a heart rate monitor for monitoring the heart rate of the patient. This allows the clinician to assess the anxiety of the patient and make determinations as to when the patient is in a de-escalated (non-anxious) or escalated (anxious) statement.

[120] In the event that the patient becomes overwhelmed, the clinician assisting with the treatment of the patient adjusts the stimulatory device to a low sensory stimulation environment which calms the patient. At all times the sensors are providing feedback to the clinician so that an assessment of the current state of anxiety of the patient can be made.

[121] When the patient’s anxiety levels are again manageable or at an acceptable level as determined by the clinician, the stimulatory device is adjusted to again project images related to their past trauma with a simultaneous overlay of EM DR therapy.

[122] Once again multimodal therapy (EM DR, exposure therapy) is combined with sensory based soothing to allow the patient to effectively change the neural pathways associated with the previous trauma and thus improve and/or alleviate their symptoms.

[123] Using the mixed reality systems may advantageously provide various therapeutic values to a subject (or user) receiving treatment. The subject having or suspected of having a neurodevelopmental disorder, such as autism, may be treated by receiving stimulatory content through the system.

[124] It is believed by the inventors that building up an initial database 410 comprising the user profile of the subject 555 with data comprising one or more indication of a selected treatment and utilising the database for controlling the operation of the virtual and augmented reality devices 420 allows the content to be effectively tailored to the subject’s needs. It has been found by the inventors that providing individualised virtual and augmented reality experiences for subjects is far more advantageous. Moreover, developing a tailored plan by accessing the subject’s information from a secure database helps in making the mixed reality treatment to be more effective.

[125] Embodiments of the present invention may use one or more computers to implement the method and system of using mixed reality reality for treatment of the subject with mental and/or developmental conditions such as autism. Figure 4 denotes a computer system 300 (the same as computer system 300 described above) that is programmed or otherwise configured to operate and display the first and second sets of content described above.

[126] The first and second sets of content may be provided with or without the use of supplemental displays, displays headsets or gear, execute algorithms within the platform or a companion application to the platform, track, save, and analyze a user's behaviours and responses to the first and second sets of content, and execute and pair a companion application that may allow the computer system 300 to communicate with a remotely located database 410 to retrieve information from the subject’s user profile and in response control in real-time a user's experience in the treatment room 500.

[127] The computer system 300 can be an electronic device of a user or a computer system that is remotely located with respect to the electronic device. The electronic device can be a mobile electronic device. The computer system 300 may be used to treat a neurodegenerative disorder, such as, for example, autism. The computer system 300 may communicate with the one or more controllers 440. In some instances, the control system 440 may include the computer system 300.

[128] Methods and systems of the present disclosure can be implemented by way of one or more algorithms. An algorithm can be implemented by way of software upon execution by the central processing unit 305. The algorithm can, for example, track a user's activity, including voice, motion, and gaze within the mixed reality environment, run analytics on metrics collected from a user's experience, provide therapeutic learning by iterating training principles with varying levels of difficulty, and translate commands from a paired control device.

[129] Embodiments of the invention described herein can be used to design treatment and therapy for a subject utilising a range of technologies to produce better engagement and therapeutic benefit to the subject.

[130] Embodiments of the invention can also advantageously be modified for varying cultural backgrounds to be highly adaptable.

[131] Importantly, embodiments of the invention provide mental health prophylaxis thereby preventing the development of mental health disorders.

[132] It has also been found that multimodal medical benefits can be derived from sensory modulation which is effective in treating and managing chronic pain, dementia and delirium, for example.

[133] In some embodiments, the systems and methods described herein can be effective in treating acute pain, depression, anxiety and various phobias. Furthermore, the systems and methods described herein may be used in palliation, functional neurology and rehabilitation.

[134] In compliance with the statute, the invention has been described in language more or less specific to structural or methodical features. The term “comprises” and its variations, such as “comprising” and “comprised of” is used throughout in an inclusive sense and not to the exclusion of any additional features. [135] It is to be understood that the invention is not limited to specific features shown or described since the means herein described comprises preferred forms of putting the invention into effect.

[136] The invention is, therefore, claimed in any of its forms or modifications within the proper scope of the appended claims appropriately interpreted by those skilled in the art.

Claims

1. A mixed reality system for treating or supplementing treatment of a subject with medical, mental and/or developmental conditions, comprising: a database comprising an account of the subject comprising one or more indications of a selected treatment related to the subject for treating an identified medical, mental and/or developmental condition of the subject, one or more stimulatory devices that provide visual, auditory, olfactory and/or haptic stimulation; a processor in communication with the database and the one or more stimulatory devices to process information related to the one of more indications related to the subject in accordance with one or more rules saved on a non-volatile memory device in communication with the processor; a controller in communication with the processor, the controller being arranged to receive control signals from the processor for providing visual, auditory, olfactory and/or haptic stimulation to the subject by controlling operation of the one or more stimulatory devices in accordance with the indications of the selected treatment saved on the account of the subject on the database to provide a first set of content to the subject thereby providing visual, auditory, olfactory and/or haptic stimulation to the subject; one or more sensors in communication with the processor, the one or more sensors being configured to generate sensory data of the subject in response to the first set of content, wherein the sensory data is indicative of a current state of the subject and is used to assess the current state of the subject in relation to a state threshold, wherein, in response to the current state of the subject being assessed as greater than a state threshold, the controller is further arranged to control operation of the one or more stimulatory devices to provide a second set of content to the subject for modifying the current state of the subject to be equal to or below the state threshold thereby calming the subject.

2. A mixed reality system in accordance with claim 1, wherein the one or more stimulatory devices comprise one or more virtual or augmented reality devices for providing the first set of content and the second set of content to the subject.

3. A mixed reality system in accordance with claim 1 or claim 2 wherein the sensors are in communication with the controller for progressively controlling the operation of the one or more stimulatory devices.

4. A mixed reality system in accordance with any one of the preceding claims wherein the system comprises a three dimensional space for positioning the one or more stimulatory devices and allowing the subject to be provided with the first set of content and the second set of content in the three dimensional space.

5. A mixed reality system in accordance with any one of the preceding claims wherein the system further comprises: an identification processing module for identifying the subject and processing identity parameters received on a user interface, the identification module being in communication with the processor and the controller for verifying identity of the subject and in response to identifying the subject retrieving information in relation to the indication of the selected treatment related to the identified subject and processing said information to generate control signals for providing visual, auditory, olfactory or haptic stimulation to the identified subject via the one or more stimulatory devices thereby providing the selected treatment to the identified subject.

6. A mixed reality system in accordance with claim 5 wherein the user interface is arranged to be accessible by a parent, therapist, carer or peer associated with the subject.

7. A mixed reality system in accordance with claim 5 or 6 wherein the database and the processor are located at a remote location relative to the one or more stimulatory devices.

8. A mixed reality system in accordance with claim 2 further comprising an evaluation module in communication with the sensors for evaluating an experience of the subject in response to the first set of content to determine progress of the subject towards one or more therapeutic goals.

9. A mixed reality system in accordance with any one of the preceding claims, wherein the one or more stimulatory devices comprises at least one of: a visual stimulatory device; an auditory stimulatory device; an olfactory device; and a haptic stimulatory device.

10. A method of using mixed reality for treating or supplementing treatment of a subject with medical, mental and/or developmental conditions, comprising the steps of: compiling a database with an account of the subject, the account comprising one or more indications of a selected treatment related to the subject for treating an identified medical, mental and/or developmental condition of the subject, placing the subject in a three dimensional space with one or more stimulatory devices that provide visual, auditory, olfactory and/or haptic stimulation; processing, by using a processor in communication with the database, information related to the one of more indications related to the subject in accordance with one or more rules saved on a non-volatile memory device in communication with the processor; and controlling, by using a controller, operation of the one or more stimulatory devices in accordance with the indications of the selected treatment saved on the account of the subject on the database to provide a first set of content to the subject thereby providing visual, auditory, olfactory and/or haptic stimulation to the subject; generating, by using one or more sensors, sensory data of the subject in response to the first set of content, wherein the sensory data is indicative of a current state of the subject; assessing the current state of the subject in relation to a state threshold; and in response to the current state of the subject being assessed as greater than the state threshold, controlling, by using the controller, operation of the one or more stimulatory devices to provide a second set of content to the subject for modifying the current state of the subject to be equal to or below the state threshold thereby calming the subject.

11. A method in accordance with claim 10, wherein the one or more stimulatory devices comprise one or more virtual or augmented reality devices for providing the first set of content and the second set of content to the subject.

12. A method in accordance with claim 11 further comprising the step of progressively controlling the operation of the one or more stimulatory devices in response to acquisition of the sensory data.

13. A method in accordance with any one of claims 10 to 12 further comprising the step of: identifying, using an identifying module, the subject and processing identity parameters received on a user interface, the identification module being in communication with the processor and the controller; verifying identity of the subject and, in response to identifying and verifying the subject, retrieving information in relation to the indication of the selected treatment related to the identified subject and processing said information to generate control signals for providing the first set of content and the second set of content to the identified subject thereby providing the selected treatment to the identified subject.

14. A method in accordance with any one of claims 10 to 13 evaluating an experience of the subject in response to the first set of content, using an evaluation module in communication with the sensors, to determine the subject’s progress towards the one or more therapeutic goals.

15. A method in accordance with any one of claims 10 to 14, wherein the one or more stimulatory devices comprises at least one of: a visual stimulatory device; an auditory stimulatory device; an olfactory device; and a haptic stimulatory device.

16. A method in accordance with any one of claims 10 to 15, wherein the first set of content provided by the stimulatory devices is interactive, such that the subject can interact with the first set of content.

17. A method in accordance with claim 16, wherein the first set of content is responsive and adapts to interaction by the subject.

18. A mixed reality system in accordance with any one of claims 1 to 9, wherein the first set of content provided by the stimulatory devices is interactive, such that the subject can interact with the first set of content.

19. A mixed reality system in accordance with claim 18, wherein the first set of content is responsive and adapts to interaction by the subject.