GB2598609A - Computer-implemented method and system for content delivery - Google Patents

Computer-implemented method and system for content delivery Download PDF

Info

Publication number
GB2598609A
GB2598609A GB2013942.4A GB202013942A GB2598609A GB 2598609 A GB2598609 A GB 2598609A GB 202013942 A GB202013942 A GB 202013942A GB 2598609 A GB2598609 A GB 2598609A
Authority
GB
United Kingdom
Prior art keywords
response
phrase
patient
training
medical
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
GB2013942.4A
Other versions
GB202013942D0 (en
Inventor
Nonny Nze
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Primetime Connect Group Ltd
Original Assignee
Primetime Connect Group Ltd
Primetime Connect Group Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Primetime Connect Group Ltd, Primetime Connect Group Ltd filed Critical Primetime Connect Group Ltd
Priority to GB2013942.4A priority Critical patent/GB2598609A/en
Publication of GB202013942D0 publication Critical patent/GB202013942D0/en
Priority to EP21759365.6A priority patent/EP4208880A1/en
Priority to PCT/GB2021/052097 priority patent/WO2022049364A1/en
Publication of GB2598609A publication Critical patent/GB2598609A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H80/00ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B5/00Electrically-operated educational appliances
    • G09B5/04Electrically-operated educational appliances with audible presentation of the material to be studied
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B23/00Models for scientific, medical, or mathematical purposes, e.g. full-sized devices for demonstration purposes
    • G09B23/28Models for scientific, medical, or mathematical purposes, e.g. full-sized devices for demonstration purposes for medicine
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B5/00Electrically-operated educational appliances
    • G09B5/06Electrically-operated educational appliances with both visual and audible presentation of the material to be studied

Abstract

A computer implemented method of delivering content is provided. The method provides a virtual reality environment where a response can be generated based on the identification of a condition in an input using a response generation model. The response generation model may be a trained data model wherein the response content stored in the response generation model comprises a plurality of training files and determining that the characteristic in the input content comprises determining that the characteristic corresponds to a label in training input corresponding to a feature of interest. The output content which makes up the response content may comprise an audio component in the form of a phrase identified in a patient parlance and a corresponding video component which shows a virtual patient speaking the words of the phrase. The generation may be based on an artificial neural network.

Description

COMPUTER-IMPLEMENTED METHOD AND SYSTEM FOR CONTENT DELIVERY
FIELD
The invention relates to a method. Particularly, but not exclusively, the invention relates to a computer implemented method. Further particularly, but not exclusively, the invention relates to a computer-implemented method and system for content delivery.
BACKGROUND
The training of medical practitioners is a fundamental part of any society. Medical practitioners such as doctors and nurses are often trusted to treat people when they are ill and so good training is essential to making sure patients are given the correct treatment and in the correct way.
Training of medical practitioners is time-consuming, material intensive, people-intensive and expensive. Training of medical practitioners when they are most required can also be inherently difficult. For example, during times of pandemic, people are often advised to stay away from one another and therefore it can be technically difficult to gather people in the same room due to the requirement for those individuals to interact whilst maintaining boundaries and distance between them.
Medical practitioners, such as nurses and doctors, are often assessed based on practical skills such as people skills and particularly their interaction with patients. It is not possible to examine these skills in writing and requires the colocation of three parties to perform such training. Those three parties being the trainee practitioner, the patient and the assessor. There is also a high degree of risk involved. If the assessed procedure goes wrong then it can be dangerous for the trainee practitioner and the patient.
Traditional in-person meeting substitutes, such as web-based meeting and video -conferences between such are not suitable for delivering the hands on training and assessment, especially for testing a trainee's reaction to a hypothetical medical situation involving life or death.
Aspects and embodiments were conceived with the foregoing in mind.
SUMMARY
Aspects and embodiments may relate to the training of medical practitioners such as medical doctors and nurses. The training may be as part of the objective structure clinical examination (OSCE) process.
Viewed from a first aspect, there is a computer implemented method of delivering content to a virtual reality environment, the method comprising instantiating a virtual reality environment configured to render a scene representative of a medical scenario, providing input content from the virtual reality environment to a response generation model, detecting at least one characteristic in the input content corresponding to a feature of interest in response content stored in the response generation model, determining that the at least one characteristic in the input content corresponds to a label in the training input corresponding to the said detected feature of interest, identifying that there is a condition in or pertaining to the input content which corresponds to the condition associated with the label, and generating a response pertaining to the identified condition in the input content, wherein the response is based on the label.
A virtual reality environment is an environment generated by a piece of virtual reality apparatus, such as, for example, a virtual reality headset. The virtual reality environment is configured to render a scene representative of a medical scenario such as, for example, an injection, a mental health examination, a removal of a Foley Catheter or any other medical procedure. The data used to render the virtual reality environment is constructed from the 360-degree views of day-to-day operations of a healthcare environment. The data is stored in suitable storage.
The input content may be a phrase uttered by a healthcare practitioner. The input content is detected and processed to detect a characteristic in the input content such as, for example, the presence of specific words. The method then determines it corresponds to a label in the training input and it is then identified that the presence of those words corresponds to the condition, i.e. the presence of specific words. Generating a response may be retrieving an output phrase based on the label.
A method in accordance with the first aspect is a method which enables a virtual reality environment to be provided where a response can be generated based on the identification of a condition in an input using a response generation model. This may be implemented in a training environment for a trainee doctor or trainee nurse where the trainee nurse or doctor speaks in a virtual environment and is met with a generated response.
The response generation model may be a trained data model wherein the response content stored in the response generation model comprises a plurality of training files and determining that the at least one characteristic in the input content comprises determining that the at least one characteristic corresponds to a label in training input corresponding to a feature of interest.
The output content which makes up the response content may comprise an audio component in the form of a phrase identified in a patient parlance and a corresponding video component which shows a virtual patient speaking the words of the phrase.
The effect of this is that response content is generated by a trained data model which can automatically generate a response comprising trained content such as a phrase and/or a video clip which contains a response which would likely come from a patient.
Generating a response pertaining to the identified condition in the input content may generate a response based on training output associated with the label.
That is to say, the method enables a response to be generated without a human being required to generate a response which would be expected of a patient, for example.
The method may further comprise training the response generation model to detect a characteristic associated with a feature of interest, the feature of interest pertaining to the input content, the training including the steps of providing a plurality of training files, each training file depicting a condition among a plurality of scenarios which relate to a medical scenario among a plurality of medical scenarios; and for each given training file among said plurality, providing a training input including a label for the feature of interest associated with a specific condition for a specific medical scenario; and providing a training output identifying a specific response that is associated with the feature of interest pertaining to the label.
The feature of interest may comprise a specific phrase or selection of words which may be a phrase stored in a healthcare practitioner parlance.
The training files may comprise correct and incorrect responses corresponding to a specific medical scenario.
The response generation may implement an artificial neural networks (ANN), otherwise known as connectionist systems are computing systems vaguely inspired by the biological neural networks. Such systems "learn" tasks by considering examples, generally without task-specific programming. They do this without any a prior knowledge about the task or tasks, and instead, they evolve their own set of relevant characteristics from the learning/training material that they process. ANNs are considered nonlinear statistical data modeling tools where the complex relationships between inputs and outputs are modeled or patterns are found.
ANNs can be hardware -(neurons are represented by physical components) or software-based (computer models) and can use a variety of topologies and learning algorithms.
ANNs usually have three layers that are interconnected. The first layer consists of input neurons. Those neurons send data on to the second layer, referred to a hidden layer which implements a function and which in turn sends the output neurons to the third layer. There may be a plurality of hidden layers in the ANN. With respect to the number of neurons in the input layer, this parameter is based on training data.
The second or hidden layer in a neural network implements one or more functions. For example, the function or functions may each compute a linear transformation or a classification of the previous layer or compute logical functions. For instance, considering that the input vector can be represented x, the hidden layer functions as h and the output as y, then the ANN may be understood as implementing a function f using the second or hidden layer that maps from x to h and another function g that maps from h to y. So the hidden layer's activation is f(x) and the output of the network is g(f(x)) The first layer may be an input node which receives an input phrase which may be uttered by a healthcare practitioner such as a trainee nurse or a doctor. The hidden layer may provide input to a dedson tree which implements functions based on contextual information such as a particular scenario, the age of a patient, or an underlying health condition. The output may be a phrase retrieved from a patient parlance. The output may be a phrase combined with a video clip which is identified as corresponding to the phrase during the training process.
The training content may comprise video content which is captured from a real healthcare environment during the day-to-day workings of a healthcare practitioner. The training content may comprise phrases which are uttered by a healthcare practitioner and/or a patient during the day-to-day operations.
The label may indicate that the uttered phrase is correct or appropriate for the scenario or that it would be inappropriate or wrong for another scenario.
The training output identifies a specific response associated with the feature of interest pertaining to the label. It may say that it is right or wrong. It may be an assignment of an output phrase in a healthcare practitioner parlance.
A medical practitioner can be any individual who works in the healthcare environment such as, for example, a doctor, a nurse, a consultant.
A virtual environment may be an environment provided through a piece of apparatus such as a virtual reality headset. A virtual environment is a computer-generated simulation of an environment which can be interacted within a seemingly real or physical way by a person using a virtual reality headset.
A virtual patient may be a computer generated image of a patient. It is provided with human like features such as eyes, a nose a mouth, ears and hair. It may also be provided with specific skin tones, eye colours, clothing and other aesthetic features which can differentiate a patient relative to others.
A medical scenario may be an event which would require attention by a healthcare practitioner. This may be the insertion and/or removal of a Foley catheter, a blood test and/or subsequent monitoring of blood sugar levels, obtaining a sample of blood for monitoring for blood sugar levels, a urine test and/or obtaining a sample for that purpose, providing an IV infusion, an insulin injection, the assessment of a respiratory condition using, for example, an inhaler or a peak flow device, the assessment and/or diagnosis of a physical or mental health condition, performing a procedure using the Aseptic non-touch technique and providing an intramuscular injection or a subcutaneous injection.
The collection of stored responses may be stored in the cloud or some other suitable means of storage.
A computer-implemented method in accordance with the first aspect provides a virtual environment which can generate automatic and human like interaction between a healthcare practitioner and a virtual patient.
The virtual environment may be initialised responsive to the healthcare practitioner donning a virtual reality headset. The effect of this is that the method can be implemented anywhere provided there is an internet connection.
The virtual patient may be overlaid onto a manikin. The effect of this is that a fixed frame of reference can be used to ensure the virtual patient is correctly located inside the virtual environment. The position of the manikin can be fed into an appropriate piece of hardware which is feeding the data representative of the virtual environment to a virtual reality headset and then used to change the orientation of the virtual patient inside the virtual environment. The manikin may be part of the physical environment used to implement the method.
The method may further comprise feeding sound effects into the virtual environment. The sound effects may be fed into the virtual environment through a virtual reality headset. The sound effects provide the effect of making the virtual training environment more immersive and realistic. The sound effects may be fed into the environment through speakers mounted on a virtual reality headset or speakers which are situated in the physical location in which the method is being implemented.
The sound effects may be fed into the virtual environment at randomly generated intervals. The effect of this is that sounds will be fed into the virtual environment at unexpected times which makes the virtual environment more redolent of a real working environment. Random generation of the intervals means that there is no fixed pattern between the feed of the sound effects into the virtual environment. That is to say, for example, a first sound effect could be fed in at 5 seconds, a second sound effect could be fed in at 5 minutes, but a third sound effect could be fed in at 5 minutes 10 seconds and so on.
The sound effects may represent a healthcare environment. For the sound effects to represent a healthcare environment, the sounds generated by the sound effects are sounds one would expect to hear in a hospital or a doctor's surgery. They include for example, doors closing, doors slamming, sirens, a baby crying, electronic alarms, IV Pump alarms, the sound of blood pressure monitoring machines, telephone sounds, the sound of healthcare-related conversations nearby, people walking by.
In one embodiment, the response to the phrase from the medical practitioner may be generated using a trained neural network. The effect of this is that the response can be provided quickly and efficiently without the requirement for any contemporaneous human input. That is to say, the method can be allowed to run without the need for human attention to provide a human like response. This makes the virtual training environment more realistic as the response can be automatically generated in much the same way a real patient would generate a response.
The training of the neural network may be based on supervised learning to impart a correspondence between a healthcare practitioner parlance and a patient parlance.
Imparting a correspondence between a healthcare practitioner parlance and a patient parlance may mean that phrases in the healthcare parlance are assigned to a phrase in the patient parlance during the training process. This means that the response to a specific phrase in the healthcare practitioner parlance would generate the assigned phrase in the patient parlance.
A healthcare practitioner parlance is a stored collection of all of the phrases which would likely be uttered during the day to day operations of a medical practitioner such as a doctor or a nurse. They include smaller simpler phrases such as, for example, "How are you?" and "How do you feel?" and other phrases that convey the same message, to more complex phrases like "lam prescribing a course of anti-depressants which you must take twice a day for the next year." Each phrase in the healthcare practitioner parlance may be stored in a database which contains a field for each of the words in the phrase.
A patient parlance is a stored collection of all of the phrases which would likely be uttered by a patient in response to a phrase uttered by a medical practitioner. They include simple phrases such as, for example, "I am fine", "Yes" to more complex phrases such as, "I am struggling to sleep at the moment".
Each phrase in the patient parlance may be stored in a database which contains a field for each of the words in the phrase.
The effect of using a healthcare practitioner parlance and a patient parlance is that there is a smaller pool of phrases to select from which will increase the speed at which the method can find a phrase and reduce the storage demands of any system implementing the method.
The supervised learning may be based on expert input from healthcare practitioners. Expert input may in the form of advice on the likely responses of patients to specific phrases in the healthcare parlance. Expert input may be in the form of a set of data regarding a specific healthcare condition and the difference it may make to the likelihood of specific responses. The expert input may be used to insert a case into a decision tree which is implemented by the method to generate a response from an input from the medical practitioner. Healthcare practitioners may be doctors, surgeons, senior nurses, consultants and other senior healthcare professionals.
In another embodiment, the response may be selected from a look up table of stored responses to the phrase received from the medical practitioner.
The effect of this is that the response does not incur the computational cost of using a neural network or other form of machine learning. It is also simple to add responses to the system and it is also simple to add phrases to the system.
In a further embodiment, the response may be selected using a weighted random number generator. The effect of this is that more likely responses can be used to bias the weighted random number generator. This means that some responses can be made more likely than others, without removing the possibility for other responses.
The virtual patient may be provided with features indicative of at least one of an age profile, an ethnic profile or a health profile.
An age profile may mean aesthetic features indicative of an older or younger person such as for example facial shape, skin texture, dress style or any other feature which would demarcate a person as looking like they within a specific age profile.
An ethnic profile may mean aesthetic features indicative of a specific ethnic background such as, for example, hair colour, skin colour or specific outfit.
A health profile may mean aesthetic features which suggest a patient is suffering from one or more health conditions. One example may be yellowing of the skin if the patient has hepatitis. Another example may be blood shot eyes which may indicate conjunctivitis.
A method according to any preceding claim, wherein the generated response is generated based on an age profile, an ethnic profile or a health profile.
A generated response based on an age profile may be modified by the neural network to be shorter if the person is younger or it may be longer if the person is older.
A generated response based on an ethnic profile may be modified by the neural network to have the words changed around or to have an accent added to their responses.
A response based on health profile may mean that a modification is made to a generated response in that "I am fine" is changed to "I am generally fine but I frequently suffer from headaches" Aspects may also provide a system which can implement a method in accordance with the first aspect. DESCRIPTION An embodiment in accordance with the first and second aspects will now be described by way of example only and with reference to the following drawings in which: Figure la is a schematic of a system which can be used to train medical practitioners in accordance with the embodiment; Figure lb is a schematic of a response generator module used in the embodiment; Figures 2a to 2d is a flow chart illustrating the use of the system during training of a nurse in accordance with the embodiment; Figure 2e is an example look up table containing responses which may be generated by a system in accordance with the embodiment; Figures 3a to 3d is a flow chart illustrating the use of the system during training of a doctor in accordance with the embodiment; Figure 3e is an example look up table containing responses which may be generated by a system in accordance with the embodiment; Figure 4a is schematic of a node which may be trained to generate responses in accordance with the embodiment; Figure 4b is a schematic of a node which may receive part of an input phrase in accordance with the embodiment; Figure 4c is a schematic of a further node which may receive part of an input phrase in accordance with the embodiment; and Figure 4d is a schematic of a further node which may receive part of an input phrase in accordance with the embodiment.
A system in accordance with the embodiment enables a medical practitioner to be trained in a safe environment and without the colocation of others This is without compromising on the need for realistic training.
The system uses a medical practitioner interface module to generate a virtual reality representation of a medical scenario which would be used to test the skills of a practitioner. Such a medical scenario could be the removal of a Foley catheter, the administration of a blood test, an intramuscular injection, a respiratory assessment using an inhaler or peak flow meter or a mental health assessment.
The skills involved with the assessment of a medical practitioner's skills set require a practical scenario in which those skills are displayed. For example, for an assessment of a headache and a mental health problem, the assessment may be based on positive descriptors such as the taking of a headache history, the exploration of triggers for the headache, the enquiry as to suicide risk, the enquiry as to psychosocial problems such as debt and substance abuse and the establishment of what the patient has already tried. For a patient with a mental health problem, this level of enquiry can be the trigger for a violent reaction which can compromise the safety of the trainee practitioner.
The use of a virtual reality representation of a medical scenario with a virtual patient alleviates this danger.
An assessment module in accordance with the embodiment provides a means by which the practitioner can be assessed remotely and accurately without the assessor needing to be collocated with the trainee practitioner.
We will now describe with reference to Figure 1 a system 100 which can be used to train a medical practitioner such as a doctor or a nurse. System 100 comprises a medical practitioner interface module 102, a medical scenario generator module 104, an assessment module 106, a scenario data store 108, an assessment data store 110 and a response generator module 112. The response generator module 112 may be connected to the other components by any suitable communication means. It may be remotely located relative to the other components and the communication with the response generator module 112 may utilise the world wide web or any other suitable telecommunications network.
The medical practitioner interface module 102 is configured to receive data from the medical scenario generator module 104. The data is retrieved by the medical scenario generator module 104 from the scenario data store 108 and is representative of a virtual representation of a medical scenario.
The data used to generate the virtual representation is captured using traditional VR content capture methods using devices such as 360 degree video and photo capture. Different scenes are constructed such as, for example, a room in a hospital, a consultant's office or any other environment where a nurse or doctor may receive training. The data is then stored in the scenario data store 108.
The medical practitioner interface module 102 and the assessment module 106 are configured to communicate with each other using any suitable communication means. They may be remotely located relative to each other and the communication means may utilise the world-wide web or any other suitable telecommunications network. Remotely located means they may be in different places such as different towns, cities, countries or continents.
The medical practitioner interface module 102 and the medical scenario generator module 104 are configured to communicate with each other using any suitable communication means. Again, they may be remotely located relatively to each other and the communication means may utilise the world wide web or any other suitable telecommunications network. Like with the medical practitioner interface module 102 and the assessment module 106, remotely located means they, i.e. the medical practitioner interface module 102 and the medical scenario generator module 104, may be in different places such as different towns, cities, countries or continents.
The medical scenario generator module 104 is configured to communicate with the scenario data store 108 using any suitable communication means. The medical scenario generator module 104 and the scenario data store 108 may be remotely located relative to one another, i.e. in a different city, country or even continent.
The medical scenario generator module 104 may be located in the cloud using a service like Amazon Web Services and run using a cloud computing service, e.g. Amazon Lambda, and similarly the scenario data store 108 may also be cloud located using something like the Amazon Simple Storage Service 53 or the Amazon Elastic File System EFS. Alternatively, the scenario data store 108 may be located in a data centre or in physical storage in a physical location.
The assessment data store 110 and the assessment module 106 are also configured to communicate with each other using any suitable communication means. Again, they may be remotely located relative to one another. The assessment data store 110 may be located in the cloud using a service such as the Amazon Simple Storage Service 53 or the Amazon Elastic File System EFS. Alternatively, the assessment data store 110 may be located in a data centre or in physical storage in a physical location.
The medical practitioner interface module 102 may comprise a virtual reality headset such as the Pico Neo Eye 2 which comprises a 4K display which provides 1600x1440 pixels per eye, a 6 degree-offreedom headset, two 6 degree-of-freedom controllers, an 845 Snapdragon Processor (from Qualcomm), 128 MB of Storage and 6 MB of RAM. The medical practitioner interface module 102 may also comprise virtual reality gloves. One example of such a pair of gloves are those developed by HaptX. The medical practitioner interface module also comprises speakers so that sounds and input from examiner's can be communicated to the respective trainee and a microphone to detect phrases uttered by the trainee.
The response generator module 112 is configured to generate the responses provided by the virtual patient during a training session which is provided by the system 100.
The response generator module 112 is trained using an artificial neural network to generate responses to phrases which are uttered by a trainee nurse or doctor. The generation of a response using the response generator module 112 will be described with reference to various examples described with reference to Figures 2a, 2b, 2c, 2d, 2e, 3a, 3b, Sc, 3d and 3e below but is also now described with reference to Figures lb and lc.
Artificial neural networks (ANN), otherwise known as connectionist systems are computing systems vaguely inspired by the biological neural networks. Such systems "learn" tasks by considering examples, generally without task-specific programming. They do this without any a prior knowledge about the task or tasks, and instead, they evolve their own set of relevant characteristics from the learning/training material that they process. ANNs are considered nonlinear statistical data modeling tools where the complex relationships between inputs and outputs are modeled or patterns are found.
ANNs can be hardware-(neurons are represented by physical components) or software-based (computer models) and can use a variety of topologies and learning algorithms.
ANNs usually have three layers that are interconnected. The first layer consists of input neurons. Those neurons send data on to the second layer, referred to a hidden layer which implements a function and which in turn sends the output neurons to the third layer. There may be a plurality of hidden layers in the ANN. With respect to the number of neurons in the input layer, this parameter is based on training data.
The second or hidden layer in a neural network implements one or more functions. For example, the Function or functions may each compute a:inear transformation or a classification of the previous layer or compute logical functions. For instance, considering that the input vector can be represented as x, the hidden layer functions as h and the output as y,then the ANN may be understood as implementing a function f using the second or hidden layer that maps from x to h and another function g that maps from h toy. So, the hidden layer's activation is f(x) and the output of the network is g(f(x)) The response generator module 112 contains a healthcare practitioner parlance 114 and a patient parlance 118.
The healthcare practitioner parlance 114 contains a list of phrases which have been input into the system during the training of the response generator module 112. The list of phrases contains the phrases which are most commonly uttered by a practitioner as suggested by established practitioners such as registered doctors, registered nurses, surgeons, registrars, anaesthetists and other healthcare practitioners. The phrases are stored in word order in that the first word in the phrases is stored in a first word field, the second word in a phrase is stored in a second word field and so on and so forth. Each word is assigned a score which is stored in a dictionary 116 of all of the words which are in the healthcare practitioner parlance alongside the corresponding word. The score is determined uniquely for each word by a weight assigned to each letter and a score assigned to each letter.
For example, the word "are" can be given a score using the following method. The letter "a" is assigned a score of 1 as it the first letter in the alphabet, the letter "r" is assigned a score of 18 as it is the eighteenth letter in the alphabet and "e" is assigned a score of 5 as it is the fifth letter in the alphabet.
As "are" is a three letter word, the first letter is assigned a weight of 1, the second letter a weight of 2/3 and the third letter a score of 1/3.
The word "are" is therefore given a score of (1x1)+(2/3*18)+(1/3*5)=14.67 In another example, the word "hello" can be similarly given a score. The letter "h" is given a score of 8 as it the eighth letter of the alphabet. The letter "e" is given a score of 5 as it is the fifth letter of the alphabet. The letters "I" are each given the score 12 as they are the twelfth letter of the alphabet. The letter "o" is given the score 15 as it is the fifteenth letter of the alphabet.
The letter h is assigned a weight of 1. The letter e is assigned a weight of 4/5. The first letter I is assigned a weight of 3/5. The second letter I is assigned a weight of 2/5. The letter o is assigned a weight of 1/5.
The word "hello" is therefore given a score of (8*1)+(5*4/5)÷(12*3/5)÷(12*2/5)÷(15*1/5) = 27 The practitioner parlance 118 contains a list of all of the phrases which have been input into the system during the training of the response generator module 112. The list of phrases contains the phrases which are most commonly uttered by a patient during sessions between a practitioner and a patient. All of the words contained in the phrases are stored in the dictionary 120 alongside a score assigned to each word in the same manner as the method used for scoring words in the dictionary 116 in the healthcare practitioner parlance 114.
The phrases in both the practitioner parlance 118 and the healthcare practitioner parlance 114 are stored as described in Table 1 below in that the first word in the phrase is stored in the first word field, the second word in the phrase is stored in second word field and so on and so forth. Table 1 illustrates the storage of the phrase "How are you?" and sirtilar phrases.
Phrase ID First Word Second Word Third Word Fourth Word 1 How Are You 2 How Are You Feeling 3 How Are You Today 4 How Are You Doing Table 1: Storage of example phrases in both the practitioner parlance and the healthcare practitioner parlance The healthcare practitioner parlance 114 and the practitioner parlance 118 are also used to store similar phrases to those thought of by the healthcare practitioner and also phrases where the ordering of words has changed but the meaning is very similar such as, in the above example, "how you are feeling".
The response generator module 112 comprises input nodes 400, 402, 404 and 406 which are each configured to receive a phrase sent to the response generator module 112 as an audio file with a request for a generated response.
Input node 400 is configured to determine the first three words of the phrase. Input node 404 is configured to determine the second three words of the phrase. Input node 406 is configured to determine the third three words of the phrase. Input node 402 is configured to receive a phrase output from each of nodes 400, 404 and 406 The phrase output from each of nodes 400, 404 and 406 is used to estimate which phrase is being uttered by scoring the received portions of the phrase and then comparing to the stored phrases in the healthcare practitioner parlance 114. The phrase output from each of nodes 400,404 and 406 is scored as follows.
The phrase "How are you feeling today?" would be received at node 400 as "How are you" and at node 404 as "today" and not received at all at node 406.
Node 400 would provide "How are you?" to node 402. Node 402 would assign a score to "How are you?". The score is determined as the sum of a score for "How" (that is the score for "how" in the dictionary) multiplied by a weight assigned to the first term in the phrase, a score for "are" multiplied by a weight assigned to the second term in a phrase and a score for "you" multiplied by the weight assigned to the third term in a phrase.
The node 402 would then determine an error term by subtracting the score assigned to the phrase "How are you?" by node 400 from the score assigned to the first three terms in every phrase stored in the healthcare practitioner parlance 114 and squaring the result to remove negative outcomes. The smallest error term would then be taken.
If the input from node 400 was "How are you" then the smallest error term would be generated by each of the phrases in Table 1.
During training of the response generator module 112, each of the phrases in the healthcare practitioner parlance 114 is assigned a corresponding phrase in the patient parlance 118. That is to say, an identified input is assigned to an identified output. The assignment of the phrase in the patient parlance 118 is based on input from healthcare practitioners who guide the selection of the phrase which would be expected to come out of the patient's mouth in response to hearing the corresponding phrase in the healthcare practitioner parlance 114. That is to say, during training appropriate responses are assigned to phrases uttered by the trainee doctor or nurse. The input from the healthcare practitioners also determines what would be inappropriate output. For example, it is not likely a patient would response with "I have a broken leg" when asked how they are during a mental health examination.
In other words, during training, phrases in the healthcare practitioner parlance are labelled in accordance with specific context based characteristics. The labels include the identified output in the patient parlance 118. The identified output also has a corresponding video file which is a capture of a patient saying the phrase in the patient parlance 118. The video file is transmitted with the identified output to the virtual reality apparatus, such as a headset so that it looks to the trainee as if the patient is saying the response. The labels may also indicate that some phrases are not appropriate for the selected medical scenario. That is to say, the selected medical scenario becomes an input to the node 402 so that any flags indicating inappropriate responses can be identified. For example, if the selected scenario is the examination of a mental health condition, flags can be used as labels on the input to indicate that certain phrases are indicated as not being likely. One phrase may be "How is your broken leg" which is not likely to be discussed in the examination of a mental health condition. This can be used to flag that this phrase is not likely to be an input to the response generator module 112. This means that output phrases assigned as a response in the patient parlance 118 to the phrase "How is your broken leg" are also excluded as they are also not likely to be uttered by a patient in a mental health condition.
Each of the phrases in Table 1 has been assigned the same phrase in the patient parlance 118 and so the node 402 selects that phrase in the patient parlance as the output and does not utilise the input from nodes 404 and 406. The assigned phrase in the patient parlance 118 can be modified by an examiner at assessment module 106. The assessment module 106 is configured to enable a user, e.g. an Examiner, to provide contextual input such as an underlying health condition, the age of a patient, previous problems such as suicidal thoughts etc. This input is mapped to a modified assignment of a phrase in the patient parlance 118. For example, if the Examiner provides contextual input indicating the patient is a younger person then the assigned phrase may be changed to something shorter, i.e. changed from "I am well" to "Good".
A different example would be the utterance of phrase "You need to get more sleep" by the practitioner. This would be mapped by the input node 402 to a large number of phrases in the healthcare practitioner parlance 114 as the words "You need to" would generate a minimal error term with a large number of different phrases and many of them would be mapped to a different phrase in the practitioner parlance 118 during the training of the response generator module 112. The node 402 would then retrieve the input from node 404 which would be the phrase "get more sleep".
The node 402 is then configured to compare the first six terms of "You need to get more sleep", i.e. input from both nodes 400 and 404 to generate an error term as before by comparing the scores from "You need to get more sleep" with each phrase in the healthcare practitioner parlance 114. That is, by determining the score for "You need to get more sleep" by summing a weighted score for "You", a weighted score for "need", a weighted score for "to", a weighted score for "get", a weighted score for "more" and a weighted score for "sleep" where the weights are assigned as before dependent on the position of the term in the phrase. This comparison would yield a significantly smaller list of different phrases from the healthcare practitioner parlance 114 as it would exclude phrases such as "You need to get more exercise", "You need to avoid caffeine before bed" and other such phrases which start with "You need to" but do not contain "get more sleep" as the next three terms.
The list of phrases corresponding to "You need to get more sleep" in their first six terms would look like Table 2.
Phrase ID First Word Second Word Third Word Fourth Word Fifth Word Sixth Word 101 You need to get more sleep 102 You need to get more Sleep 103 You Need to get more Sleep 104 You need to get More Sleep Table 2: Storage of phrases which have the first six terms "You need to get more sleep" In Table 2, four phrases are stored which are Phrase ID: 101 "You need to get more sleep"; Phrase ID: 102 "You need to get more sleep, try to go for a walk before bed"; Phrase ID: 103 "You need to get more sleep, try to avoid caffeine before bed"; and Phrase ID: 104 "You need to get more sleep, try to go to bed earlier". During training of the response generator module 112, each of phrases 101, 102, 103 and 104 can be mapped to the same response in the patient parlance 118 such that when the node 402 determines that one of these phrases has been uttered, as they generate the same error term from "You need to get more sleep", node 402 selects the phrase "Thank you Doctor, I will try" from practitioner parlance 118 as the generated response as each of these phrases has been assigned that response in the patient parlance 118.
Response generator module 112 is also configured to enable new phrases to be added to the healthcare practitioner parlance 114 and they will be scored accordingly and they too can be assigned to responses in the patient parlance 118.
Response generator module 112 is configured to communicate with the medical scenario generator module 104 and the assessment module 106 using any suitable means. The assessment module 106 may provide contextual input to the node 402 which may modify the output selected by node 402. The medical scenario generator module 104 is configured to issue requests for responses to be generated by the response generator module 112 by transmitting phrases which are then processed using nodes 400, 402, 404 and 406 and generate an output as a result of the supervised learning which matched input phrases to phrases in the patient parlance 118 as described above. The output is then transmitted back to the medical scenario generator module 104 as a response to the request with a corresponding audio file for that response.
That is to say, nodes 400,404 and 406 may be described as a first layer and node 402 may be described as a hidden layer.
In another embodiment, the medical scenario generator module 104 may not utilise the response generator module 112 to generate requests. Scenario data store 108 comprises look up tables which comprise lists of stored responses to each of the phrases in the healthcare practitioner parlance 114. The medical scenario generator module 104 is configured to deploy a weighted random number generator which selects a response from a look up table corresponding to a received phrase. The weighted random number generator may be weighted toward a response which is more likely in practice but the weighted random number generator will infrequently generate a less likely response which will provide a more realistic training scenario for the trainee doctor or nurse. The medical scenario generator module 104, on selecting a response, also retrieves an audio file corresponding to the selected response.
The decision tree implemented by the node 402 may be configured to determine that an input does not correspond to a phrase in the healthcare practitioner parlance 114 by determining the score of the input phrase and identifying only phrases with a score lower than a specific threshold. The node 402 is then configured to determine that the phrase is incorrect and the node 402 can then correct the input. For example, the phrase "how are you" can be commonly uttered by those with English as a second language as "how you are", which would cause problems as it would not be assigned to any specific output and the node can retrieve the inputs from node 400 to determine if specific combinations of words are in different order and the decision tree implemented by node 402 is configured to re-assign the ordering of the words to generate a meaningful output. Other combinations of words which also be re-ordered and these combinations of words can be fed into the response generator module 112 during the training phase.
Additionally, if a trainee doctor or nurse has poor intonation then some words may not be uttered clearly which is likely to lead to a situation where an output cannot be generated. The dictionary can then be used to generate an error term for the individual words to see if words can be retrieved from the dictionary in the healthcare practitioner parlance 114. For example, if the trainee intends to say the words "headache" and the nodes only pick up "headach", the score for "headach" can be compared to the dictionary and it is likely that "headache" would generate the lowest error and it would therefore be used to correct the input and the node 402 can continue accordingly.
On receiving the response from the response generator module 112 or selecting a response from a lookup table, medical scenario generator module 104 will transmit the audio file to the medical practitioner interface module 102. The medical scenario generator module 104 also transmits the video content to the medical practitioner interface module 102 which provide the virtual reality imagery which provides the virtual reality representation of the medical scenario in which the training is taking place. The video content is captured during the training of the response generator module 112 and is assigned to an output phrase as it contains that output phrase. The transmission of the video is simultaneous with the transmission of the audio file containing the response.
The system 100 may be used in a physical location where a physical manikin is positioned and the location coordinates of the physical manikin are fed into the system by the Examiner at the assessment module 108. The location coordinates can then be used, with the position-finding capabilities of the medical practitioner interface module 102, to overlay the virtual patient onto the physical manikin. The medical practitioner interface module 102 is also configured to feed position and orientation information to the medical scenario generator module 104 which ensures the virtual patient is rendered at the correct orientation and position relative to the trainee in the virtual environment generated by the medical practitioner interface module 102.
In generating the data used to implement the virtual patient through the medical practitioner interface module 102, the medical scenario generator module 104 may utilise context information from the assessment module 108 which indicates the patient is intended to be a younger or older patient. The context information may also indicate the patient is of a specific ethnicity. This contextual information may be used to alter the visual appearance of the virtual patient when it is rendered by the medical practitioner interface module 102.
The assessment module 108 may be manned by a supervisor who supervises the sessions using the system 100. The assessment module 108 receives a feed from the medical practitioner interface module 102 to show what is happening in the generated virtual environment. They can also hear the phrases uttered by the trainee and the virtual patient. The assessment module 108 is also configured to establish a path of communication with the medical practitioner interface module 102 to ensure the supervisor can pass messages to the trainee if they need correction or encouragement. Additionally, the assessment module 108 may switch off the response generation functionality of the medical scenario generator module 104 and the response generator module 112 and allow the examiner to provide the responses of the virtual patient.
The communication between the respective modules is optimally implemented using the fifth generation telecommunications network but it does not have to be We will now describe with reference to Figures 2a, 2b, 2c and 2d how the system 100 can be used to implement training of a doctor who is being trained in patient interaction. The example scenario is the investigation into a headache and mental health problem.
Prior to the practitioner training session using the system 100, the example scenario, i.e. the investigation into a headache and mental health problem, is selected either at the assessment module 106 or at the medical practitioner interface module 102.
At a step 5200, the trainee doctor dons the medical practitioner interface module 102 in the form of a virtual reality (VR) headset in the form of the Pico Neo Eye 2 and presses a button to initialise the session in the system 100.
Responsive to the initialisation of the session in the system 100, in a step 5202, the medical practitioner interface module 102 requests the data required to implement the session from the medical scenario generator module 104. The medical scenario generator module 104, in a step 5204, retrieves the data for the scenario from the scenario data store 108.
The data is captured using a standard VR capture method such as using a 3D camera or 3D video camera. The capture takes a 3D image of a consultant's room in a hospital where a doctor may be expecting to work or be called into action. By using this imagery, the system 100 provides a more realistic immersive environment when a training session takes place.
The data for the scenario is then transmitted to the VR headset 102 worn by the doctor which, in a step S206, generates a virtual reality environment which represents the selected scenario. The data is fed to the VR headset 102 at a rate of at least 90 frames per second. This is the minimum frame rate which needs to be implemented to avoid trainees becoming motion sick when they are using the system. lithe VR headset 102 detects that the frame rate has dropped below 90 frames per second then it will generate an alarm to warn the user's that the system cannot be used.
In this example, the selected scenario is the investigation into a headache and mental health problem. This is represented as a virtual reality environment in which a consultant's room is generated using stored data where a virtual patient is sat in a chair.
The trainee doctor then needs to gather information about the patient which could be relevant to a headache or a mental health problem. The information includes facts related to family, social and occupational factors.
In a step 5208, the doctor begins their assessment of the situation by asking the virtual patient "how are you feeling?", the VR headset 102 comprises a microphone which detects this question to the virtual patient. The VR headset 102 converts the question to an audio file suitable for transmission, and the audio file is transmitted to the medical scenario generator module 104 in a step S210.
The text data is fed to the response generator module 112 which is trained to generate a response. The arrangement of node 402 is illustrated in Figure 4a. It forms a neural network trained to generate a response. The inputs to the neural network are the phrase output, i.e. derived from the uttered phrase "how are you feeling?", the age of the patient and a health condition. These are fed into node 402.
The age of the patient can be fed into the node 402 by the medical scenario generator 104 or the assessment module 108. It can be made constant by the system in that it can simply be set to an average age adult who is not likely to provide any unusual health problems to take care of for the trainee doctor. That is to say, the age may be fixed to 35 or a similar age. However, if the Examiner at the assessment module 106 wants to test the skill of the trainee doctor at dealing with younger or older patients, they may change the age prior to step S200. If the Examiner did want to change the age to a younger or older person then the medical scenario generator module 104 will generate the virtual patient accordingly. That is to say, the medical scenario generator module 104 will generate a virtual patient with features which resemble a child or an elderly person if the Examiner provides an instruction that a respective younger or older patient is to be the virtual patient. Features which represent a younger person may be smoother skin, clearer eyes and thicker hair. Features which represent an older person may be wrinkled skin and thinner hair. In this example, the Examiner has not selected an older or younger person.
The health condition may be set to a null state by the Examiner or, if the Examiner wanted to vary the task a little, the Examiner may set the health condition to something more severe such as "heart condition" or "influenza". The health condition may also be set to a mental health condition or tendency such as "previously suicidal". In this example, the Examiner has set the health condition to "previously suicidal" to provide a test of how well the trainee doctor deals with potential mental health problems.
The phrase output is fed by a preceding node 400 which is illustrated in Figure 4b. The inputs to node 400 are the first second and third words of the phrase "how are you?". Node 400 then provides the phrase output as "how are you?" to node 402.
The decision tree in node 402 compares the phrase "how are you?" to the first three terms of phrases in the healthcare practitioner parlance 114 to determine the error term which indicates how far "how are you?" is from each of the phrases in the healthcare practitioner parlance 114. The phrase with the smallest error term is selected as the basis for the output response generated by the node 402.
The smallest error term is generated by the phrases "how are you?"; "how are you doing?" and "how are you feeling?" in the healthcare practitioner parlance 114. This is a small list of phrases, i.e. smaller than a selection threshold where further information from nodes 404 and 406 is not required. Each of these phrases is assigned the same output in the patient parlance 118.
However, the health condition "previously suicidal" is also fed into node 402 by the examiner at the assessment module 108. Node 402 is then trained to determine the response is a concatenation of a standard response phrase such as "I am fine" and a qualifying phrase such "but I have contemplated suicide in the past". The first component "I am fine" is determined by assignment of the output in the patient parlance 118 but the second component "but I have contemplated suicide in the past" is determined by the decision tree inside node 402 as a qualifying phrase to use where a patient has the indicated health condition "previously suicidal".
The output from node 402 is therefore "I am fine, but I have contemplated suicide in the past" and this is the response selected by the response generator module 112.
The node 402 is also configured to discard some candidates for the response phrase. Whilst "How are you", "how are you doing?" and "how are you feeling?" are assigned to the same output in the patient parlance 118, i.e. I am fine, the node 402 acts as a hidden layer in that it is configured to receive the scenario as an input which can be used to dismiss some response phrases such as "I have a broken leg", "I have a runny nose" or another phrase which would not be appropriate for the scenario.
Alternatively, the audio file may be converted to text data in a step 5212. The text data generated in step 5212 is used by the medical scenario generator module 104 to open a look-up table which contains responses to the question "how are you feeling?" which is stored in the scenario data store 108 in a step 5214. In a step 5216, the medical scenario generator module 104 selects a response from the look up table. The selection may be at random in that the index in the look up table may be randomly generated and the randomly generated index used to identify the response in the look up table. The response selected by the medical scenario generator module 104 is "I feel fine now, but have contemplated suicide in the past." is then selected and the data corresponding to this response, either selected by the look up table or generated using nodes 400 and 402, is transmitted to the VR headset 102 in a step 5218.
The response, whether it is generated using the look up table or the response generator module 112, is transmitted with video content which corresponds to footage of the patient uttering the response phrase.
In a step 5220, the VR headset 102 receives the data corresponding to the response and generates the response in the virtual environment in which the trainee doctor is immersed by relaying the phrase "I feel fine now, but I have contemplated suicide in the past" through the speakers in the VR headset 102 and the video content makes it look like the mouth of the virtual patient is moving at the same time to replicate the scenario where they are receiving this news from the patient.
At this stage, having received this response, the trainee doctor would be expected to provide a response that addresses the patient's needs and adequately demonstrates understanding of the response as well as empathy for the patient represented by the virtual patient. This may involve taking an appropriate headache history and explore symptoms of stress and depression with the patient, whilst excluding suicide risk. This will involve further interaction with the virtual patient.
This continued interaction would be implemented by repeated questions to the virtual patient, which are then transmitted to the medical scenario generator module 104 in a similar way as described in steps 5210 and 5212, and they are followed by responses which are generated in the same way as described in steps 5214 to 5220.
Throughout this interaction, the trainee doctor needs to assess the situation by taking an appropriate headache history and explaining the red flags of headaches, as well as exploring symptoms of stress and depression, excluding suicide risk. Once the doctor has identified any psychological problems, such as an occupation that could affect the situation, the doctor needs to acquire information from the patient about what, if anything, they have already tried.
On establishing the information, the trainee doctor will need to credibly reassure the patient using phrases which show empathy for the patient. In step 5222, the trainee doctor may utter the phrase "This is not a permanent condition and you are not alone". This phrase is detected by the microphone on the VR headset 102 and transmitted to the medical scenario generator module 104 as an audio file in a step 5224.
The phrase is fed into the response generator module 112 where the node 400 receives the first three words as an input. That is to say, the first word is "This", the second word is "is" and the third word is "not". The first three terms are concatenated by node 400 and sent to node 402. The node 402 applies a decision tree to these three words in order to determine an output which is in the output set of words and responses. The decision tree determines that a large number of phrases, i.e. larger than a threshold value, in the healthcare practitioner parlance 114 has a minimal error when compared to "This is not". This indicates that more information is needed by node 402 to determine which phrase in the healthcare practitioner parlance 114 is being uttered and which phrase in the patient parlance 118 should be selected as an output.
That is to say, the non-descriptive and non-specific nature of this set of three words, when fed to the decision tree in node 402, will generate a set of possible outputs which is too large as during the training of the node 402, multiple phrases beginning with "This is not..." where used to train the neural network made up by the nodes 400, 402, 404 and 406. In other words, the decision tree provides identifies phrases in the healthcare practitioner parlance 114 ranging from "This is not me." to "This is not an appropriate way to speak" and to "This is not the correct medication for this condition". Each of them is assigned different response phrases in the patient parlance 118.
The node 402 is configured to then request input from a node 404 which receives the second three words of the phrase as an input. The node 404 therefore received the words "a", "permanent", "condition" as they are the second three words in the phrase uttered in the phrase "This is not a permanent condition and you are not alone".
The decision tree in the node 402 then concatenates the first three words, i.e. those received from node 400, with the second three words, i.e. those received from node 404, to infer that the first six words of the phrase are "This is not a permanent condition". The node uses this term to generate a smaller set of phrases in the healthcare practitioner parlance 114 as the comparison is based on the first six terms of the stored phrases rather than the first three. The number of identified phrases in the healthcare practitioner parlance 114 is below a threshold for determining the necessity for more information.
The identified phrases are "This is not a permanent condition"; "This is not a permanent condition, it will pass" and "This is not a permanent condition, you are not alone" and each are assigned to the same phrase in the patient parlance 118 as they are identified during training of node 402 as meaning substantially the same thing.
The node 402 then generates the response "Thank you Doctor, that is very reassuring" based on the assignment between the healthcare practitioner parlance 114 and the patient parlance 118 and this is selected by the response generator module 112 as the response to the phrase "This is not a permanent condition, you are not alone" uttered by the trainee doctor.
Alternatively, the medical scenario generator module 104 is configured to generate a response by opening a look up table corresponding to the phrase "This is not a permanent condition and you are not alone" in a step 5226 and selecting a response in a step 5228. The response may be randomly selected or it may be fixed by the medical scenario generator module 104.
An example lookup table is illustrated in Figure 2e. This lookup table would be stored in the scenario data store 108. The index can be randomly selected by the medical scenario generator module 104. For example, if the medical scenario generator module selects index 1, the response "Thank you, that is very reassuring" is selected in step 5228.
The response selected in step 5228 (or the response generated by the response generator module 112) is then used to retrieve data which is transmitted to the VR headset in a step 5230. The transmitted data is used by the VR headset 102 to relay the response to the trainee doctor through the speakers in the VR headset 102 in a step 5232. The transmitted data also includes captured video content which shows a patient uttering the phrase. This video content is also transmitted if the response is selected by the response generator module 112.
The trainee doctor then explains to the patient what stress and tension headaches are in a step 5234. The trainee doctor then needs to know how to explain the management of this going forward.
In a step 5236, the trainee doctor utters the phrase "Try to get more sleep, ensure you keep yourself clean and consider taking up some relaxation techniques and I will refer you to a course on stress management". The phrase is detected by the microphone on the VR headset and converted to an audio file before being transmitted to the medical scenario generator module 104 in a step 5238.
The medical scenario generator module 104 transmits the audio file to response generator module 112 and requests a response. The response generator module 112 feeds the first three words as inputs to node 400. That is to say, the inputs to node 400 are "Try", "to" and "get". The node 400 feeds a phrase output "Try to get" to node 402 which is configured to apply a decision tree to determine which phrase in the healthcare practitioner parlance 114 is being uttered by determining an error term comparing "Try to get" with each term in the healthcare practitioner parlance 114. The decision tree compares the number of terms which show a zero error term (or even the smallest error term) with a threshold to determine whether too many phrases are likely to have been generated. In this instance, the decision tree generates a list of phrases from the healthcare practitioner parlance 114 which contains many phrases ranging "Try to get some sleep", "Try to get some food inside you", "Try to get more rest", "Try to get up later" and all of them are assigned to different responses inside the patient parlance 118.
The node 402 therefore requests the next three words from node 404 and they are provided to node 402 as "more sleep ensure".
The decision tree at node 402 generates an error term based on the six words "Try to get more sleep ensure" and the phrases stored in the healthcare practitioner parlance 114. On identifying the terms with minimal or zero error term, the list is still larger than the threshold value for too many identified phrases. This is because "Try to get more sleep ensure" could be the first six terms of "Try to get more sleep ensure you take your medication", "Try to get more sleep ensure you avoid caffeine", "Try to get more sleep ensure you avoid horror movies before bed", "Try to get more sleep ensure you turn your telephone off" and many others. They are each identified during the training phase as having a different response in the patient parlance 118 and do not enable the decision tree at node 402 to deliver an output with sufficient confidence. This means that node 402 needs the third three words of the phrase as an input.
Node 406 provides the third three terms concatenated as "you keep yourself" and the node 402 then determines the error term for the phrase "Try to get more sleep ensure you keep yourself" by comparison to each phrase in the healthcare practitioner parlance 114 and the number of phrases identified is below the threshold for a confident identification and they are each assigned the same response in the patient parlance 118 as each of the candidate phrases means substantially the same thing.
The assigned response is "Thank you doctor, is there anything else you suggest" and this is fed back to medical scenario generator module 104.
Alternatively, the medical scenario generator module 104 processes the audio file in a step 5240 to convert it to a text file in a step 5240. The text file is used to identify the look up table corresponding to the possible responses to the phrase uttered by the doctor in step 5236. A response is then selected in step 5242 from the look up table. In this example, the response is "Thank you doctor, is there anything else you can suggest".
The medical scenario generator module 104 retrieves the audio data corresponding to the response and transmits the data to the VR headset 102 in a step S244. The VR headset 102 is configured to convert an audio component in the data to an audio file which is played through the speakers on the VR headset 102 so that the trainee doctor hears "Thank you doctor, is there anything else you can suggest". The data received at the VR headset 102 also includes video content which shows the patient saying "Thank you doctor, is there anything else you can suggest" In determining the response, the response generator module 112 discards response phrases which are inappropriate. For example, the phrase "Thank you doctor, I completely disagree" is not likely to be uttered by the patient in this scenario and the training of the response generator module 112 labels it as such.
The doctor would then be expected to utter a phrase like "Consider taking time off work, increase your exercise and consider taking a mild sedative such as antihistamine/amitriptyline before bed" in a step 5246.
Responsive to detecting this phrase through the microphone, the VR headset 102 transmits an audio file representing this phrase to the medical scenario generator module 104 in a step 5248.
Using the techniques described previously, the response generator module 112 generates a response "Thank you doctor, you have been a big help" from the patient parlance 118. This is fed to the medical scenario generator module 104 with captured video content of a patient saying those words.
Alternatively, the audio file is converted into a text file in a step 5250. The text of the phrase is then used to retrieve a look-up table from scenario data store 108 containing responses corresponding to the phrase "Consider taking time off work, increase your exercise and consider taking a mild sedative such as antihistamine/amitriptyline before bed" in a step 5252.
A response is selected from the look up table in a step 5254. The response may be selected randomly by randomly generating an index and choosing the response corresponding to that index in the lookup table. In this example, the selected response is "Thank you doctor, you have been a big help" and the medical scenario generator module 104 retrieves the data corresponding to that response in a step 5256. The data is then transmitted to the VR headset 102 in a step 5258 The VR headset 102 is configured to relay the audio component of the data to the user through the speakers in the form of the phrase "Thank you doctor, you have been a big help" and also to relay the captured video content in the virtual reality environment in a step 5260..
Outside of the steps S200 to S260 described above other forms of interaction also occur between the doctor and the virtual patient, and they lead to the generation of responses from the medical scenario generator module 104 in much the same way, i.e. by request from the response generator module 112 which is trained to find a suitable response using supervised learning or by reference to a look up table which are reflected through the VR headset 102 and a change in the generated virtual reality representation of the medical scenario, i.e. such as movement in the mouth of the virtual patient. The other forms of interaction may be discussion about the particular suggestions the doctor makes. After this the session ends. This may include the doctor clearing safety netting which may also be generated as part of the virtual reality environment or may be located in the physical world as a boundary on the physical environment surrounding the trainee doctor.
The interaction throughout the steps S200 to S260 is transmitted in real-time to the assessment module 106 as both an audio and visual component. This means that Examiner's at the assessment module 106 can also see the virtual reality representation of the medical scenario and can see and hear the actions of the trainee doctor. The trainee doctor may be represented as an avatar in a perspective view or alternatively, the view of the virtual representation of the medical scenario which is provided to the trainee doctor may be relayed by the VR headset 102 to the assessment module so the Examiner's at the assessment module can see the scenario as the trainee doctor sees it and also listen to the interaction between the trainee doctor and the virtual patient. This will enable the Examiner's to make an assessment of the performance of the trainee doctor.
This will also enable the Examiner's to send live feedback to the trainee doctor about what they are doing and potentially provide pointers and/or reminders about what is the correct step to take.
The Examiner will assess the trainee doctor using positive and negative descriptors such as those set out in the table below: Assessment Domain: 1. Data-gathering, technical and assessment skills Positive descriptors: Negative descriptors: * Takes an appropriate headache history * Inappropriate concentration on headache.
using SOCRATES (Site; Onset; Character; . Does not explore possible stress or depression.
Radiation; Associations; Time courses; * Does not take neurological history.
Exacerbating/relieving factors; Severity) * Fails to explore psychosocial problems.
* Explores potential triggers eg depression, stress etc. * Excludes suicide risk.
* Explores psychosocial problems eg. debt, substance abuse * Establishes what the patient has already tried (if anything) by questioning the patient and reacting to responses.
Throughout the session during steps 5200 to 5260, sound effects will also be played through the VR headset to improve the immersive experience. These sound effects may be a crying baby, a door slamming, low-level chatter, an ambulance siren and machine sounds sound as those which would typically be made by blood pressure monitors or IV alarms. The sound effects may also be generated by the response generator module 112. The training phase of the response generator module 112 may record sound effects which are familiar in hospital environments and feed them into the virtual reality environment rendered by headset 102 at randomly generated times.
We will now describe with reference to Figures 3a to 3d, how the system 100 can be used to train a nurse with reference to an example scenario in which a nurse would be required to provide an intramuscular injection.
At a step 900, the trainee nurse dons the medical practitioner interface module 102 in the form of a virtual reality (VR) headset in the form of the Pico Neo Eye 2 and presses a button to initialise the session in the system 100.
Responsive to the initialisation of the session in the system 100, in a step 5302, the medical practitioner interface module 102 requests the data required to implement the session from the medical scenario generator module 104. The medical scenario generator module 104, in a step 5304, retrieves the data for the scenario from the scenario data store 108.
The data for the scenario is then transmitted to the VR headset 102 worn by the trainee nurse which, in a step 5306, generates a virtual reality environment which represents the selected scenario. Example scenarios for a trainee nurse would be the administration of injections, administration of an inhaler, the assessment of peak flow exhalation and the removal of a Foley Catheter. In this example, the nurse is expected to administer an injection.
The virtual representation of the scenario is a room with a virtual patient in a seated position. The virtual patient is overlaid on a manikin. The trainee nurse would be expected to move around in the virtual environment to carry out checks on the environment and the needs of the patient, i.e. whether they have any communication needs which they are missing such as a hearing aid or glasses. The physical environment which is used for the implementation of the system may contain a physical manikin and the coordinates of the physical manikin may be fed to the medical scenario generator module 104. The medical scenario generator module 104 may then generate a virtual patient which overlays the manikin using those coordinates. The medical scenario generator module 104 can render the virtual patient with more human like features such as a face with facial features, clothing and physical shape such as a different body shape or particular facial features.
The nurse will then, in a step 908, introduce themselves to the patient and tells the patient the purpose of their visit by uttering the phrase "I'm here to administer your injection today".
The VR headset 102 detects this phrase and converts it into an audio file which is transmitted to the medical scenario generator module 104 in a step 910.
The medical scenario generator module 104 is configured, responsive to receiving the audio file, to issue a request to the response generator module 112 for a response to the uttered phrase.
Node 400 receives the first three words of the phrase and concatenates them as input to node 402 where the decision tree compares the score which would be assigned to the first three terms in the phrase with the score assigned to the first three terms in each of the phrases in the healthcare practitioner parlance 114 to determine the error term and identifies the terms with minimal (or zero) error term.
The number of identified terms exceeds the threshold for the number of terms for which the node 402 will not need any further information. That is to say, the identified terms include "I'm here to tell you your diagnosis", "I'm here to provide your medication" and "I'm here to discharge you" which all have different meanings and are assigned to different phrases in the patient parlance 118. The node 402 then requests the second three terms from node 404 and it receives "administer your injection". Node 404 is illustrated in Figure 4c. Node 406 is illustrated in Figure 4d.
Node 402 then determines the score for the six terms "I'm here to administer your injection" and compares the score with a score for the first six terms in each of the other phrases in the healthcare practitioner parlance 114 to determine the error term. The phrases with minimal error term are taken and a smaller number of phrases are identified, and they are all assigned to the same response as a result of the supervised learning which maps phrases in the healthcare practitioner parlance 114 to phrases in the patient parlance 118. That phrase being "I understand, thank you for the clarification" which is selected by node 402 as the response. The response is then transmitted back to the medical scenario generator module 104 as an audio component to be relayed through the VR headset 102 and video footage of the patient saying "I understand, thank you for the clarification" to provide the effect that it looks like the patient is speaking back to the trainee nurse The node 402 is configured to also discard phrases which would not be appropriate for the situation such as "I understand, I have not been well this week" based on the selected medical scenario.
Alternatively, the medical scenario generator module 104 generates a response by retrieving a look up table containing responses corresponding to that phrase in a step S312.
The look up table contains responses to the phrase "I'm here to administer your injection today". They are selected using a weighted random number generator which selects the response according to the likelihood a human would respond with that response. That is to say, in this situation, a patient is not likely to do anything other than be compliant, so the weighted random number generator is weighted to be biased toward generating a number which corresponds to the most likely response.
This can be more easily illustrated by reference to Figure 3e. Using a weighted random number generator, the selection of index 1 can be biased so that the corresponding response, i.e. "I understand, thank you for the clarification", is selected 85% of the time and the indices 2 and 3 are selected respectively 8% and 7% of the time as those responses are considered to be less likely.
In a step 5314, the response corresponding to index 1 is selected, i.e. the selected response is "I understand, thank you for the clarification". Data corresponding to the response is retrieved by the medical scenario generator module 104 which will enable the VR headset 102 to render the response in the virtual environment both audio and visually in a step 5316. The data is transmitted to the VR headset 102 as an audio and visual component in a step 5318. The video component is rendered by the VR headset 102 to make it looks like the virtual patient's mouth is moving as the response is being relayed through the VR headset 102 and the audio component, i.e. the response "I understand, thank you for the clarification" is relayed to the trainee nurse through the speakers in a step 5320.
The trainee nurse then needs to demonstrate hand hygiene by washing their hands in accordance with the recommended 7-steps as set out by the WHO. The 7-steps are set and pre-programmed to include: palm to palm; back of the hands; fingers interlaced; fingers interlocked; base of the thumb; rotational rubbing of the wrists and pat dry using a paper towel. The trainee nurse will verbalise each step at least once during the scenario.
The handwashing technique is strictly observed by the Examiner as the full virtual reality scenario is displayed to the Examiner through the VR headset 102 by transmitting the images to the assessment module 108. ID checks may then be conducted.
To conduct the allergy checks, the trainee nurse utters the phrase "Do you have any allergies or reactions?" and the VR headset 102 detects this phrase and it is converted to an audio file and transmitted to the medical scenario generator module 104 in a step S322.
The medical scenario generator module 104 issues a request for a response to the response generator module 112. On receiving the request, the node 400 receives the first three terms and concatenates them to form "Do you have" which, upon generation of the error term, generates a long list of identified candidate phrases in the healthcare practitioner parlance 114. This is because many phrases in the healthcare practitioner parlance start with "Do you have".
The node 402 then retrieves the second three terms from node 404 and uses those to determine the error term for the first six words in the phrase, i.e. "Do you have any allergies or" which leads to a substantially smaller set of identified terms and they are all assigned to the same response phrase in the patient parlance 118 and that is "No allergies".
The Examiner may feed context information to node 402 through the assessment module 108 which indicates the patient has a particular allergy. This will modify the assigned response so that the response describes the allergy. That is to say, the node 402 forms a hidden layer in the artificial neural network formed by the response generator module 112 in that it forms contextual information into the decision tree which influences the output.
The selected response, i.e. "No allergies" is transmitted to the medical scenario generator module 104 in response to the request for a response.
Alternatively, the medical scenario generator module 104 retrieves the look up table corresponding to the transmitted phrase in a step S324 from the scenario data store 108. The look up table contains responses to the phrase "Do you have any allergies?" and, if the patient responds in the affirmative, a follow up question of, "what happens when you come into contact with this allergen" and a response is selected using a weighted random number generator which is weighted based on the commonality of the allergies present in the population at the time.
That is to say, the look up table may, for example, contain 5 common allergies indexed as "1. Paracetamol", "2. Peanut", "3. Soy", "4. Milk", "5. Sesame", 6. Penicillin, 7. Ibuprofen and an eight possibility "8. No allergies" and the weighted random number generator is weighted to index 8, i.e. the random number generator selects index 8 55% of the time, as it is thought that most people do have no allergies and most people would answer that way. However, the 5 allergies may be weighted such that index 1 is generated 10% of the time, index 2 is generated 9% of the time, index 3 is generated 15% of the time, index 4 is generated 6% of the time, index 5 is generated 3% of the time, index 6 is generated 1% of the time and index 7 is generated 1% of the time.
The weighted random number generator is used to select a response in a step 5326. In this example, the selected response is "8. No allergies" and, upon generating this response, the medical scenario generator module 104 retrieves a corresponding audio file, i.e. a sound recording, of the response and a video component which is a video capture of a patient saying the words "No allergies" . The audio file and the video component are transmitted to the VR headset 102 in a step S328. The audio file is played to the trainee nurse and the video component is played to make it look like the patient is saying the response. This is step S330. The video component is transmitted even if the response generator module 112 is used as opposed to the look up tables to generate a response.
It is also possible in step 5322 that the request for the patient to state allergies is unclear and the medical scenario generator module 104 may not be able to obtain it from the transmitted signal. In this instance, the medical scenario generator module 104 transmits a request to the trainee in the form of a sound recording which asks the trainee nurse to repeat the question. Steps 5322 to S330 will then be re-started until a clear question regarding the allergies can be determined.
The trainee will then ask the patients consent to provide the injection in a step 5332 with a question "Do you consent to this injection?" The VR headset 102 is configured to detect that phrase through its microphone and transmit the corresponding audio file to the medical scenario generator module 104 in a step 5334.
The medical scenario generator module 104 transmits a request to response generator module for a response with the corresponding audio file. The node 400 transmits the concatenated phrase, i.e. concatenated from "Do", "you" and "consent", to the node 402 as "Do you consent". The node 402 is configured to utilise the decision tree to determine a score for the three words "Do you consent" and compare that to the first three terms of each phrase in the healthcare practitioner parlance 114 to determine an error term for each phrase compared to "Do you consent". This generates a large number of identified phrases as many phrases in the healthcare practitioner parlance 114 start with "Do you consent", i.e. "Do you consent to this procedure", "Do you consent to this consultation?" and multiple others but they are all assigned during training of the nodes 400, 402, 404 and 406 to the same phrase in the patient parlance, i.e. "Yes, I consent" and this is therefore selected as the response. This is generated and transmitted back to the medical scenario generator module 104 with a video component of a patient saying those words. The video component being captured during the training of the response generator module 112.
Alternatively, the audio file is converted to text and the text is used to retrieve a look up table in the scenario data store 108 corresponding to the phrase "Do you consent to this injection?" in a step 5336.
The look up table is indexed with three possible responses "1. Yes, I consent", "2. No, I do not consent" and "3. Please, I don't want to be here". A response is selected using a weighted random number generator in a step 938 which is configured to generate an index from the look up table.
The weighted random number generator is configured to select index 1 85% of the time. This is to represent the expectation that most people will consent to their injection and the system 100 is therefore likely to generate a realistic training scenario. The weighted random number generator is configured to generate index 2, i.e. "No, I do not consent", 10% of the time and index 3, i.e. "Please, I don't want to be here" 5% of the time.
In this example, index 1 is selected and the sound file corresponding to the phrase "Yes, I consent" is selected and the corresponding video component are transmitted to the VR headset in a step 5340.
The response "Yes, I consent" is played through the speakers in the VR headset 102 and the video component is played to make it look like the virtual patient is saying "Yes, I consent". This is step 5342.
In the event another response is generated, the trainee nurse would be expected to respond to the lack of consent with further phrases.
The trainee nurse, in a step 5344, will use the VR controllers in the generated virtual reality environment to pick up the prescription which is provided inside the virtual environment. The trainee nurse then reads and verifies the information in the prescription in all sections and then checks with the patient which medications are due with the injection with the phrase "You are here for an insulin injection?" in a step 5346.
The VR headset 102 detects the phrase and it is converted to a sound file which is transmitted to the medical scenario generator module 104 in a step 948.
The medical scenario generator module 104 requests a response to be generated by response generator module 112. The node 400 detects the words "You are here" as they are first, second and third terms of the uttered phrase and they are fed to node 402.
Node 402 determines the score for "You are here" and calculates the error term compared to each phrase in the healthcare practitioner parlance 114. The terms with the smallest (or zero) error term are identified and that produces a list which is too large, i.e. larger than a threshold value and with multiple different assignments in the patient parlance 118.
The node 402 then retrieves the second three terms from the node 404, i.e. for an insulin, and feeds them to the node 402. Node 402 then calculates the error term based on the first six terms in the phrase and the first six terms in the phrases in the healthcare practitioner parlance 114. This produces a much smaller list and they are all assigned during training to the same response phrase in the patient parlance 118, i.e. "Yes, I am here for an insulin injection". The selected response is therefore "Yes, I am here for an insulin injection" and this is transmitted back to the medical scenario generator module 104 with a corresponding video component of a virtual patient saying the words "Yes, I am here for an insulin injection".
In generating this response, the training of the artificial neural network enables inappropriate responses to be discarded. An example would be "No, I am here to have an examination of my thyroid". This would be discarded as an inappropriate response.
Alternatively, the medical scenario generator module 104 generates a response to the phrase in a step 5350 using a look up table to select a response. The look up table corresponds to the phrase "You are here for an insulin injection". The responses in the lookup table are indexed using numbers which can be generated by a weighted random number generator. The weighted random number generator can generate one of 1, 2 or 3 and the look up table contains response 1 "Yes, I am here for an insulin injection."; "2. No, I am in the wrong place" and "3. Can you clarify the purpose of this injection?". The weighted random number generator is biased to generate the number 190% of the time, number 2 7% of the time and number 33% of the time. In this example, the selected response is "Yes, I am here for an insulin injection" The selected response is then transmitted to the VR headset 102 as a sound file with a corresponding video component. This is step 5352.
The response is then played through the speakers in the VR headset 102, synchronised with the video component to make it look as though the virtual patient is speaking. This is step 5354. That is to say, the trainee nurse hears the words "Yes, I am here for an insulin injection".
The trainee nurse may also verbally check blood sugar levels with the listening examiner, who can feedback whether they agree or not with the indicated level. If the patient indicated a lack of understanding of the medication, the trainee nurse would be expected to provide this information either from memory or from a virtual British National Formulary (BNF) which is implemented in the virtual environment through the VR headset 102 and the medical scenario generator module 104.
The nurse would then, in the virtual environment, be expected to wash their hands before preparing the materials in the virtual environment. The nurse would then be expected to use the VR controllers to pick up the medication and check that all packaging is intact and that all the expiry dates are appropriate and express this to the virtual patient.
The expression of this is declared by the uttered phrase "The packaging is intact and the expiry date is 31 September 2020. The batch number is 00022890" in a step S356.
The VR headset 102 detects this phrase and converts it into an audio file which is transmitted to the medical scenario generator module 104 in a step 958.
The medical scenario generator module 104 is configured to convert the audio file into text. The medical scenario generator module 104 is then configured to extract the expiry date and the batch number from the text and compare it to corresponding numbers in the scenario data store 108 in a step 5360. The medical scenario generator module 104 may also be configured to recognise speech from the audio file and will therefore not need to convert the audio file into a text file. That is to say, the batch number and the expiry number are determined using voice recognition.
If the comparison yields a positive result, i.e. the batch number and the expiry date uttered by the trainee nurse match what is stored in the scenario data store 108, then the medical scenario generator module 104 will generate a response which says "correct". If the comparison yields a negative result, i.e. the batch number and the expiry date uttered by the trainee nurse do not match what is stored in the scenario data store 108, then the medical scenario generator module 104 will generate a response which says "incorrect". This is in step 5362.
The response is then transmitted to the VR headset 102. This is step 5364. The response is then played to the trainee nurse. If the response is "incorrect" the trainee is expected to repeat the step until the "correct" response is achieved.
If the trainee nurse is not familiar with the drug, which can only be assessed by listening to the information they provide about it, they need to read the medication leaflet or check the BNF (in the virtual environment).
The nurse needs to show that they are maintaining hygiene in the virtual environment by applying hand gel and putting gloves on. This can be virtually implemented using the VR controllers or even VR gloves. The nurse then prepares the injection using the aseptic non-touch technique and using a filter needle or gauge 13 from a glass ampoule. This is all placed into the virtual environment as it is generated. That is to say, it is provided upon initialisation by the VR headset 102 having received the data from the medical scenario generator module 104. The trainee nurse needs to use the controllers to clean the ampoule with an alcohol swab. The nurse needs to be seen to snap an end of the ampoule using gauze-sharps and draw up the medication. This, again, is in the virtual environment. The VR controllers provide the feeling of touch, force and weight which helps to simulate the feeling of picking up and manipulating objects.
The nurse then needs to approach the virtual patient and select the correct injection site. This is again visually assessed by the Examiner as they can see what the trainee is seeing through the VR headset 102. This is in step S366. The explanation of the injection site needs to be verbalised to the patient in a step 5368.
After this, with the trainee nurse's non-dominant hand they need to be seen to stretch the skin and hold the gauze between their fingers in preparation for use after, as well as holding the syringe in the dominant hand and the needle at a 90-degree angle. The trainee nurse needs to pull back the plunger to check for blood. Common mistakes here include not pulling the plunger back to check for blood; not holding the gauze; inserting the needle at the wrong angle. This is visually assessed by the examiner who may fail a candidate for incorrect technique.
After the injection, the trainee nurse needs to wait 10 seconds for absorption, then withdraw the needle and apply gentle pressure with gauze, ensuring that they do not massage the site (as they would do and verbalise if the injection were a subcutaneous injection), then proceed to apply a plaster. Once the syringe is dropped into the sharps bin, the trainee nurse needs to remind the patient of any side effects and apply hand gel. After this, the trainee nurse needs sign a virtual chart with the expiry date and batch number by picking up a pen in the virtual environment. They need to leave the patient in a professional manner, ensuring safety, comfort and giving any relevant phone numbers that the patient may need. Finally, the trainee nurse needs to ensure hand hygiene is maintained by washing their hands according to the 7-steps of the WHO. Adequate reflection also needs to take place, in case any vital steps may have been missed.
The needle then needs to be recapped using a single scoop technique then placed into a virtual sharps bin, i.e. a sharps bin in the virtual environment. The needle then needs to be changed to a safety needle and the ampoule to be left as well as the drug in the virtual administration tray. This can be assessed by the Examiner who is watching the images that are displayed to the trainee nurse at the assessment module 106. That is to say, the procedure is visually examined by the Examiner through the VR headset 102.
Upon completion of the procedure, the trainee nurse needs to remove their gloves and dispose of them in the virtual waste bin in the virtual environment, as well as apply hand gel.
Throughout the session during steps 5300 to 5366, sound effects will also be played through the VR headset 102 to improve the immersive experience. These sound effects may be a crying baby, a door slamming, low-level chatter, an ambulance siren and machine sounds sound as those which would typically be made by blood pressure monitors and IV machines. The sound effects may be generated using the response generator module 112 which may be trained to provide the sound effects.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be capable of designing many alternative embodiments without departing from the scope of the invention as defined by the appended claims. In the claims, any reference signs placed in parentheses shall not be construed as limiting the claims. The word "comprising" and "comprises", and the like, does not exclude the presence of elements or steps other than those listed in any claim or the specification as a whole. In the present specification, "comprises" means "includes or consists of" and "comprising" means "including or consisting of". The singular reference of an element does not exclude the plural reference of such elements and vice-versa. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In a device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims (21)

  1. CLAIMS1. A computer implemented method of delivering content to a virtual reality environment, the method comprising: instantiating a virtual reality environment configured to render a scene representative of a medical scenario; providing input content from the virtual reality environment to a response generation model; detecting at least one characteristic in the input content corresponding to a feature of interest in response content stored in the response generation model; determining that the at least one characteristic in the input content corresponds to a label in the training input corresponding to the said detected feature of interest; identifying that there is a condition in or pertaining to the input content which corresponds to the condition associated with the label; and generating a response pertaining to the identified condition in the input content, wherein the response is based on the label.
  2. 2. The method according to Claim 1, wherein the response generation model is a trained data model wherein the response content stored in the response generation model comprises a plurality of training files and determining that the at least one characteristic in the input content comprises determining that the at least one characteristic corresponds to a label in training input corresponding to a feature of interest.
  3. 3. A method according to Claim 1 or Claim 2, wherein generating a response pertaining to the identified condition in the input content generates a response based on training output associated with the label.
  4. 4. A method according to any preceding claim, the method further comprising: 38 training the response generation model to detect a characteristic associated with a feature of interest, the feature of interest pertaining to the input content, the training including the steps of: providing a plurality of training files, each training file depicting a condition among a plurality of scenarios which relate to a medical scenario among a plurality of medical scenarios; and for each given training file among said plurality, providing a training input including a label for the feature of interest associated with a specific condition for a specific medical scenario; and providing a training output identifying a specific response that is associated with the feature of interest pertaining to the label.
  5. 5. A method according to Claim 4, wherein the feature of interest comprises a specific phrase.
  6. 6. A method according to Claim 5, wherein the specific phrase is a phrase in a healthcare practitioner parlance.
  7. 7. A method according to any of Claims 4 to 6, wherein the training files comprise correct and incorrect responses corresponding to a specific medical scenario
  8. 8. A method according to any preceding claim, wherein a virtual reality environment is initialised responsive to a medical practitioner donning a virtual reality headset.
  9. 9. A method according to any preceding claim, wherein the virtual reality environment comprises a virtual patient
  10. 10. A method according to any preceding claim, wherein the virtual patient is overlaid onto a manikin.
  11. 11. A method according to any preceding claim, the method further comprising feeding sound effects into the virtual reality environment.
  12. 12. A method according to Claim 11, wherein the sound effects are fed into the virtual reality environment at randomly generated intervals.
  13. 13. A method according to Claim 11 or Claim 12 wherein the sound effects represent a healthcare environment.
  14. 14. A method according to any preceding claim, wherein the response to the phrase from the medical practitioner is generated using a trained artificial neural network.
  15. 15. A method according to Claim 14, wherein the training is based on supervised learning to impart a correspondence between a healthcare practitioner parlance and a patient parlance.
  16. 16. A method according to Claim 15, wherein the supervised learning is based on expert input from healthcare practitioners.
  17. 17. A method according to any preceding claim, wherein the response is selected from a look up table of stored responses to the phrase received from the medical practitioner.
  18. 18. A method according to any preceding claim wherein the response is selected using a weighted random number generator.
  19. 19. A method according to any preceding claim, wherein the virtual patient is provided with features indicative of at least one of an age profile, an ethnic profile or a health profile.
  20. 20. A method according to any preceding claim, wherein the generated response is generated based on an age profile, an ethnic profile or a health profile.
  21. 21. A system configured to implement the method of any of Claims 1 to 20.
GB2013942.4A 2020-09-04 2020-09-04 Computer-implemented method and system for content delivery Pending GB2598609A (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
GB2013942.4A GB2598609A (en) 2020-09-04 2020-09-04 Computer-implemented method and system for content delivery
EP21759365.6A EP4208880A1 (en) 2020-09-04 2021-08-12 Computer-implemented method and system for content delivery
PCT/GB2021/052097 WO2022049364A1 (en) 2020-09-04 2021-08-12 Computer-implemented method and system for content delivery

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
GB2013942.4A GB2598609A (en) 2020-09-04 2020-09-04 Computer-implemented method and system for content delivery

Publications (2)

Publication Number Publication Date
GB202013942D0 GB202013942D0 (en) 2020-10-21
GB2598609A true GB2598609A (en) 2022-03-09

Family

ID=72841410

Family Applications (1)

Application Number Title Priority Date Filing Date
GB2013942.4A Pending GB2598609A (en) 2020-09-04 2020-09-04 Computer-implemented method and system for content delivery

Country Status (3)

Country Link
EP (1) EP4208880A1 (en)
GB (1) GB2598609A (en)
WO (1) WO2022049364A1 (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6692258B1 (en) * 2000-06-26 2004-02-17 Medical Learning Company, Inc. Patient simulator
US20120139828A1 (en) * 2009-02-13 2012-06-07 Georgia Health Sciences University Communication And Skills Training Using Interactive Virtual Humans

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6692258B1 (en) * 2000-06-26 2004-02-17 Medical Learning Company, Inc. Patient simulator
US20120139828A1 (en) * 2009-02-13 2012-06-07 Georgia Health Sciences University Communication And Skills Training Using Interactive Virtual Humans

Also Published As

Publication number Publication date
GB202013942D0 (en) 2020-10-21
WO2022049364A1 (en) 2022-03-10
EP4208880A1 (en) 2023-07-12

Similar Documents

Publication Publication Date Title
US11961197B1 (en) XR health platform, system and method
Carel Phenomenology of illness
Lancioni et al. Assistive technology: Interventions for individuals with severe/profound and multiple disabilities
Crepeau Achieving intersubjective understanding: Examples from an occupational therapy treatment session
Won et al. Automatic detection of nonverbal behavior predicts learning in dyadic interactions
Forster et al. The nature of affect attunement used by disability support workers interacting with adults with profound intellectual and multiple disabilities
Lisetti et al. Building an on-demand avatar-based health intervention for behavior change
Razmus Body representation in patients after vascular brain injuries
RU2581707C1 (en) Method for evaluating the efficiency treatment and rehabilitation in patients with disorders of higher mental functions in focal brain lesions
Sumioka et al. Technical challenges for smooth interaction with seniors with dementia: Lessons from Humanitude™
Do et al. Clinical screening interview using a social robot for geriatric care
Degramont Language and the Distortion of Meaning
Mast et al. The role of nonverbal communication in medical interactions: Empirical results, theoretical bases, and methodological issues
Stephenson A textbook of general practice
CN110021203A (en) A kind of Oral healthy education experiencing system, method and medical education device
Burns Clinical management of agnosia
GB2598609A (en) Computer-implemented method and system for content delivery
Yang et al. Prototyping virtual reality interactions in medical simulation employing speech recognition
Jadhav et al. Use of Assistive Techniques for the Visually Impaired People
Mutlu Designing gaze behavior for humanlike robots
Gallagher et al. Neuropsychological assessment of adults with visual impairment
Jönsson et al. Situated research and design for everyday life
Amini Learning data-driven models of non-verbal behaviors for building rapport using an intelligent virtual agent
Brenneis Observations on psychoanalytic listening
Braad The Readability of Language: Observing the Non-Linguistic and Linguistic Features in the Communication of a Child with Congenital Deafblindness

Legal Events

Date Code Title Description
COOA Change in applicant's name or ownership of the application

Owner name: PRIMETIME CONNECT (GROUP) LTD

Free format text: FORMER OWNER: IELTS MEDICAL LTD