JP7450183B2 - Phantom limb pain care system, patient device, and program - Google Patents

Description

The present invention relates to a phantom limb pain care system.

Phantom limb pain is a phenomenon in which, after an arm or leg is amputated, the limb that was supposed to be lost is felt to exist, and pain is perceived in the space where the limb used to exist. Phantom limb pain is said to occur in more than 80% of patients with limb amputation.

A method of treating phantom limb pain is to show the existing limb that is paired with the missing limb in a mirror and move the existing limb to provide visual feedback so that the missing limb is moving normally, thereby relieving the pain. There are known things that can relieve it.

Furthermore, in Patent Document 1, a position sensor detects the posture of a patient, and based on this, a phantom image corresponding to the movement of the patient performing a predetermined task is presented in a virtual space to reduce pain. A treatment support device is disclosed.

Japanese Patent Application Publication No. 2004-298430

It is estimated that there are 4.5,000 patients with upper limb amputations and brachial plexus injuries, and the number of patients who suffer from brain damage such as stroke or thalamic pain resulting in pain in their limbs is 500 who require palliative care through exercise therapy. It is said that there are 10,000 people. Phantom limb pain varies from person to person and can vary from day to day. For this reason, treatment requires appropriate knowledge and experience, and there are few people who can properly treat it. Although experienced doctors are relatively concentrated in urban areas, patients with phantom limb pain are not only found in urban areas. For this reason, there was a problem in that some patients were unable to attend treatment.

Therefore, an object of the present invention is to provide a technique that makes it possible to care for a patient with phantom limb pain who is located away from the practitioner.

In order to solve the above problems, the phantom limb pain care system of the present invention,
A patient device that provides an image of a virtual space to the patient at a first point where the patient is present, and is operated by a practitioner located at a second point away from the first point. Equipped with a practitioner device,
The subject device is
a position detection unit that detects a position of a predetermined limb of the patient as a healthy limb;
Generating a three-dimensional model of the healthy limb based on the position of the healthy limb detected by the position detection unit, and generating a three-dimensional model of a phantom limb to be paired with the healthy limb based on the position of the healthy limb. a three-dimensional model generation unit,
an image generation unit that generates an image of the virtual space including a three-dimensional model of the healthy limb and the phantom limb;
a head-mounted display that is attached to the subject's head and displays an image of the virtual space on a display screen;
a situation information transmitting unit that transmits patient situation information indicating the condition of the patient including at least the position of the healthy limb to the practitioner device;
an instruction receiving unit that receives instruction information indicating instructions from the practitioner from the practitioner device;
an instruction output unit that outputs the instruction information to the patient as at least one of a part of the video and audio of the virtual space;
The practitioner device includes:
a situation information receiving unit that receives the patient situation information from the patient device;
a display control unit that displays the condition of the patient on a display based on the patient situation information;
and an instruction information transmitting section that transmits the instruction information to the patient's device.

The phantom limb pain care system includes:
The practitioner device includes:
a position detection unit that detects the position of a predetermined one of the subject's limbs as a healthy limb;
Generating a three-dimensional model of the healthy limb based on the position of the healthy limb detected by the position detection unit, and generating a three-dimensional model of a phantom limb to be paired with the healthy limb based on the position of the healthy limb. a three-dimensional model generation unit,
The instruction information transmitting unit includes a three-dimensional model based on the healthy limb of the practitioner and a three-dimensional model of the phantom limb in instruction information and transmits the instruction information to the patient device,
The image generation unit of the patient device synthesizes and displays the three-dimensional model of the healthy limb and phantom limb of the practitioner with the image of the virtual space based on the instruction information including the three-dimensional model. Good too.

In the phantom limb pain care system, the image generation unit generates an image in which a three-dimensional model of the patient and a three-dimensional model of the practitioner face each other, or a three-dimensional model of the patient and a three-dimensional model of the practitioner. An image may be generated in which the three-dimensional model of the practitioner is positioned on the back side of the three-dimensional model of the subject, with the three-dimensional model of the practitioner facing the same direction.

In the phantom limb pain care system, the image generation unit generates an image in which the three-dimensional model of the patient and the three-dimensional model of the practitioner face each other and an image in which they face the same direction from the practitioner's device. The switching may be based on the instruction information.

In the phantom limb pain care system, one of the practitioner's devices may connect to a plurality of the patient's devices and transmit the instruction information to the plurality of patient's devices.

In order to solve the above problems, the patient device of the present invention includes:
Providing an image of a virtual space to the patient at a first point where the patient is present, and communicating with a practitioner device operated by a practitioner located at a second point away from the first point. A patient device that performs
a position detection unit that detects the position of a predetermined one of the subject's limbs as a healthy limb;
Generating a three-dimensional model of the healthy limb based on the position of the healthy limb detected by the position detection unit, and generating a three-dimensional model of a phantom limb to be paired with the healthy limb based on the position of the healthy limb. a three-dimensional model generation unit,
an image generation unit that generates an image of the virtual space including a three-dimensional model of the healthy limb and the phantom limb;
a head-mounted display that is attached to the subject's head and displays an image of the virtual space on a display screen;
a situation information transmitting unit that transmits patient situation information indicating the condition of the patient including at least the position of the healthy limb to the practitioner device;
an instruction receiving unit that receives instruction information indicating instructions from the practitioner from the practitioner device;
an instruction output unit that outputs the instruction information to the subject as at least one of a portion of the video of the virtual space and audio;
Equipped with.

In order to solve the above problems, the phantom limb pain care program of the present invention includes:
Providing an image of a virtual space to the patient at a first point where the patient is present, and communicating with a practitioner device operated by a practitioner located at a second point away from the first point. A program to be executed by a patient's device to perform the operation,
A step of determining a predetermined one of the subject's limbs as a healthy limb, and detecting the position of the healthy limb;
generating a three-dimensional model of the healthy limb based on the detected position of the healthy limb, and generating a three-dimensional model of a phantom limb paired with the healthy limb based on the position of the healthy limb;
generating an image of the virtual space including a three-dimensional model of the healthy limb and the phantom limb;
Displaying an image of the virtual space on a display screen of a head-mounted display attached to the subject's head;
transmitting patient status information indicating the condition of the patient including at least the position of the healthy limb to the practitioner device;
receiving instruction information indicating the practitioner's instructions from the practitioner device;
outputting the instruction information to the patient as at least one of a portion of the video of the virtual space and audio;
the patient's device executes.

The present invention can provide a technique that enables care for a patient with phantom limb pain who is located away from the practitioner.

1 is a diagram showing the configuration of a phantom limb pain care system. FIG. 2 is a diagram showing the configuration of a head-mounted display. It is a figure showing the composition of the control part in a patient's device. It is a figure showing the composition of the control part in a practitioner's device. FIG. 3 is a diagram illustrating a procedure for phantom limb pain care that is executed by the patient's device based on a program. FIG. 3 is a diagram illustrating a procedure for phantom limb pain care that is executed by the practitioner's device based on a program. An example of a healthy limb visible to the patient in real space is shown when the left upper limb is designated as the patient's healthy limb. FIG. 6 is a diagram showing an example of video information when a three-dimensional model of a healthy limb is generated and placed in a virtual space. FIG. 6 is a diagram showing an example of video information when a three-dimensional model of a healthy limb and a phantom limb is generated and placed in a virtual space. FIG. 6 is a diagram showing an example of video information when a three-dimensional model of the patient and a three-dimensional model of the practitioner are placed in the same virtual space. 11 is a diagram showing the positional relationship between the three-dimensional model of the patient and the three-dimensional model of the practitioner in FIG. 10. FIG. FIG. 6 is a diagram showing an example of video information when a three-dimensional model of a patient and a three-dimensional model of a practitioner are placed close to each other in the same virtual space. 13 is a diagram showing the positional relationship between the three-dimensional model of the patient and the three-dimensional model of the practitioner in FIG. 12. FIG. FIG. 6 is a diagram illustrating an example of video information when a three-dimensional model of a patient and a three-dimensional model of a practitioner are placed at substantially the same position in virtual space and facing the same direction. 15 is a diagram showing the positional relationship between the three-dimensional model of the patient and the three-dimensional model of the practitioner in FIG. 14. FIG. FIG. 3 is a diagram showing an example in which three-dimensional models of a plurality of subjects are displayed in one virtual space.

<First embodiment>
"System configuration"
An embodiment of the present invention will be described below with reference to the drawings. In this embodiment, a system for supporting treatment of phantom limb pain will be described. The phantom limb pain care system 1 of this embodiment is a system that supports care for phantom limb pain. Phantom limb pain care refers to actions to eliminate or alleviate phantom limb pain, such as treatment and rehabilitation. In this embodiment, care for phantom limb pain is not limited to therapeutic actions permitted by the Medical Practitioners Act, but may include actions such as training and relaxation.

FIG. 1 is a diagram showing the configuration of a phantom limb pain care system 1. The phantom limb pain care system 1 includes a patient device 10 that provides an image of a virtual space to a patient at a first location where the patient is present, and a second patient device located away from the first location. The apparatus includes a practitioner device 20 operated by a practitioner present at the location.

《Subject device》
As shown in FIG. 1, the patient apparatus 10 includes position detection sections 11 and 12, a control section 13, and a head mounted display (HMD) 14.

The position detection units 11 and 12 define one predetermined limb of the subject as a healthy limb, and detect at least the position of the healthy limb. The position detection units 11 and 12, for example, periodically determine the positions of the limbs and detect changes in the positions over time as movements. This embodiment includes two position detectors: a position detector 11 that detects movements of fingers, and a position detector 12 that detects movements of the body including healthy limbs.

The position detection units 11 and 12 include, for example, two cameras and an infrared ray irradiation unit, and the infrared ray irradiation unit irradiates the body of the detection target (subject) with an infrared pattern, and the position detection units 11 and 12 are arranged at a predetermined interval (baseline length). The body is photographed using two cameras placed apart. The position detection units 11 and 12 identify the area in which the body is captured from the photographed image, and perform image processing to identify joints and parts connected to them (target body parts). Then, the position detection units 11 and 12 determine the position of each target part in the photographed image, the size and inclination of the infrared pattern irradiated to each target part, and the difference between these cameras. Based on this, the position and orientation of each target region on the three-dimensional coordinates are determined.

The position detecting units 11 and 12 are not limited to the configuration in which the position on three-dimensional coordinates is detected based on the difference between images taken by two cameras as described above. For example, the position detection units 11 and 12 include a camera and a ToF sensor, and use the camera to photograph the body of the detection target, and use the ToF sensor to determine the distance to the object in the photographed image. The position detection units 11 and 12 identify the area where the body is shown from the photographed image in the same manner as described above, and recognize the target body part of the person to be detected through image processing. Then, the position detection units 11 and 12 determine the three-dimensional coordinate position of each target part based on the position in the photographed image where the target part appears and the distance to each target part. Furthermore, since the ToF sensor can detect the distance to each target part pixel by pixel, the position detection units 11 and 12 recognize each target part as a point group, and from the shape formed by this point group, the orientation of each part is determined. Detect etc.

In this embodiment, the position detection unit 11 detects the position of the healthy limb in detail, and the position detection unit 12 detects the position of the healthy limb and the body around the healthy limb. For example, when the upper limb is set as a healthy limb, the position detection unit 11 detects the positions of the hand, wrist, forearm, each finger of the hand, and the first to third joints of each finger as target parts. do. Note that when a lower limb is set as a healthy limb, only the position detection unit 12 may be used if there is no need to detect fine movements such as a hand.

The HMD 14 is attached to the patient's head and displays an image of the virtual space on a display screen. FIG. 2 is a diagram showing the configuration of the HMD 14.

The HMD 14 includes a display section 141, earphones 142, a microphone 143, a detection section 144, a lens 145, and a control section 146. The lens 145 is an eyepiece system that guides the light flux emitted from the display surface of the display unit 141 to the wearer's eye 149 and forms an image on the retina. This allows the wearer to observe the image as an enlarged virtual image.

The detection unit 144 is, for example, an angular velocity sensor or a geomagnetic sensor, and detects the direction of the HMD 14, that is, the direction of the head of the wearer wearing the HMD 14, that is, the direction of the line of sight. The control unit 146 causes the display unit 141 to display the video generated by the video generation unit 313.

FIG. 3 is a diagram showing the configuration of the control section 13. The control unit 13 includes, for example, a processor 132, a memory 133, an input/output IF (interface) 134, and a communication IF 135, which are interconnected by a connection bus 131. The processor 132 controls the entire device by processing input information and outputting processing results. The processor 132 is also called a CPU (Central Processing Unit), an MPU (Micro-processing unit), or a controller. The processor 132 is not limited to a single processor, but may have a multiprocessor configuration. Alternatively, a multi-core configuration may be used in which a single chip has a plurality of cores connected by a single socket.

Memory 133 includes a main storage device and an auxiliary storage device. The main storage device is used as a work area for the processor 132, a storage area for temporarily storing information processed by the processor 132, and a buffer area for communication data. The main storage device is a storage medium used by the processor 132 to cache programs and data, and to develop a work area. The main storage device includes, for example, RAM (Random Access Memory), ROM (Read Only Memory), and flash memory. The auxiliary storage device is a storage medium that stores programs executed by the processor 132, data used for information processing, operation setting information, and the like. Examples of the auxiliary storage device include a HDD (Hard-disk Drive), an SSD (Solid State Drive), an EPROM (Erasable Programmable ROM), a flash memory, a USB memory, and a memory card.

The input/output IF 134 is an interface that inputs and outputs data with a device connected to the control unit 13. The input/output IF 134 inputs and outputs data to and from devices such as operation units such as a mouse and keyboard, cameras, scanners, speakers, and display devices. In addition, the input/output IF 134 inputs and outputs data to and from devices such as a disk drive that reads data from storage media such as CDs and DVDs, card readers/writers, amplifiers, displays, and speakers. The operation unit is an input unit, such as an input button, a dial, or a touch panel, through which information to the control unit 13 is input by a user's operation. The speaker and display unit are output units that output information to the user.

The communication IF 135 is an interface (communication module) that communicates with other devices via a communication line, and is also referred to as a CCU (Communication Control Unit). The communication IF 135 is, for example, WiMAX (Worldwide Interoperability for Microwave Access) or LT.
Communication is performed using wireless communication methods such as E (Long Term Evolution), WiFi, and Bluetooth (registered trademark). Note that a plurality of the components of the control unit 13 shown in FIG. 3 may be provided, or some components may not be provided.

In the control unit 13, the processor 132 functions as each processing unit such as a three-dimensional model generation unit 312, a video generation unit 313, a situation information transmission unit 331, an instruction reception unit 332, and an instruction output unit 333 by executing an application program. do. In the processor 132, one processor 132 or one core may be used as multiple processing units. For example, the processor 132 functions as a plurality of processing units using techniques such as multithreading or multitasking. In addition, some or all of the plurality of processing units may include a dedicated LSI (large scale integration) such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), a logic circuit, It may also be formed from hardware such as other digital circuits. Further, at least a portion of each of the processing units may include an analog circuit.

The three-dimensional model generating section 312 generates a three-dimensional model of the body including healthy limbs based on the position of the body detected by the position detecting section. A virtual space corresponding to the real space where the detection target person exists is set, and the three-dimensional model generation unit 312 generates a three-dimensional model created at a position in the virtual space that corresponds to the position where the detection target person exists in the real space. Set the position of the 3D model to place the model. Furthermore, the three-dimensional model generation unit 312 generates a three-dimensional model of a phantom limb that is paired with the healthy limb. In this embodiment, a symmetrical model is generated with respect to the three-dimensional model of a healthy limb, and the phantom limb is placed in the desired position.

The image generation unit 313 generates an image of the virtual space including three-dimensional models of the healthy limb and the phantom limb. Here, when the detection target moves a healthy limb, the 3D model created by the 3D model generation unit also moves in the same way in the virtual space. Generate a video of the model. In addition, the video generation unit 313 of this embodiment generates an image (left-eye video) when looking into the virtual space from the position of the left eye of the detection target, and an image when looking inside the virtual space from the position of the right eye of the detection target. The image for the right eye (image for the right eye) is generated. That is, the left-eye image and the right-eye image are images when viewing the same space, but have a difference (parallax) in viewing position by the distance between the left and right eyes (eye width). Further, the video generation unit 313 may generate the video by determining the direction in which the virtual space is viewed, depending on the orientation of the head of the detection target wearing the HMD 14.

The situation information transmitting unit 331 transmits patient situation information indicating the condition of the patient including at least the position of the healthy limb to the practitioner device. The subject situation information is, for example, information indicating a three-dimensional model generated by the three-dimensional model generation section 312 or video information generated by the image generation section 313. Further, the situation information transmitting unit 331 may include the patient's voice information acquired by the microphone 143 in the situation information and transmit it.

The instruction receiving unit 332 receives instruction information indicating instructions from the practitioner from the practitioner's device 20. The instruction information is, for example, at least one of information indicating a three-dimensional model of the practitioner generated by the practitioner device 20, video information, and audio information.

The instruction output unit outputs at least one of video and audio in the virtual space to the patient based on the instruction information. For example, if the instruction information is information indicating a three-dimensional model of the practitioner, this information is sent to the image generation unit 313, and the three-dimensional model of the patient and the three-dimensional model of the practitioner are created in the virtual space. Generate video information in the arranged state. Further, when the instruction information is video information of the practitioner, this information is sent to the video generation unit 313, and the video information of the practitioner is combined with the video information of the patient. If the instruction information is the practitioner's voice information, this information is output as voice from the earphones 142.

《Practitioner device》
As shown in FIG. 1, the practitioner device 20 includes position detection units 21 and 22, a control unit 23, and an HMD 24. In the practitioner's device 20, the position detecting sections 21, 22 and HMD 24 have the same configuration as the position detecting sections 21, 22 and HMD 14 in the patient's device 10 described above, so a repeated explanation will be omitted.

FIG. 4 is a diagram showing the configuration of the control section 23. As shown in FIG. The control unit 23 includes, for example, a processor 232, a memory 233, an input/output IF (interface) 234, and a communication IF 235, which are interconnected by a connection bus 231. The control unit 23 in the practitioner device 20 has a different function realized by the processor 232 than the control unit 13 in the patient device 10 described above. In addition, the hardware configuration of the connection bus 231, processor 232, memory 233, input/output IF 234, and communication IF 235 in the control unit 23 is as follows: Since the configuration is the same as that of the output IF 134 and the communication IF 135, repeated explanation will be omitted.

In the control unit 23, the processor 232 functions as each processing unit such as a three-dimensional model generation unit 412, a situation information reception unit 431, a situation information output unit 432, and an instruction information transmission unit 433 by executing an application program.

The three-dimensional model generation unit 412 generates a three-dimensional model of the body including healthy limbs based on the body positions detected by the position detection units 21 and 22. A virtual space corresponding to the real space where the detection target person exists is set, and the three-dimensional model generation unit 412 generates a three-dimensional model created at a position in the virtual space that corresponds to the position where the detection target person exists in the real space. Set the position of the 3D model to place the model. Furthermore, the three-dimensional model generation unit 412 generates a three-dimensional model of a phantom limb that is paired with the healthy limb. In this embodiment, a symmetrical model is generated with respect to the three-dimensional model of a healthy limb, and the phantom limb is placed in the desired position.

The image generation unit 413 generates an image of the virtual space including three-dimensional models of the healthy limb and the phantom limb. Here, when the detection target moves a healthy limb, the 3D model created by the 3D model generation unit also moves in the same way in the virtual space. Generate a video of the model. In addition, the video generation unit 413 of this embodiment generates an image (left-eye video) when looking into the virtual space from the position of the left eye of the detection target, and an image when looking inside the virtual space from the position of the right eye of the detection target. The image for the right eye (image for the right eye) is generated. That is, the left-eye image and the right-eye image are images when viewing the same space, but have a difference (parallax) in viewing position by the distance between the left and right eyes (eye width). Further, the video generation unit 413 may generate the video by determining the direction in which the virtual space is viewed, depending on the direction of the head of the detection target wearing the HMD 24.

The situation information receiving unit 431 receives patient situation information from the patient device 10. The situation information output unit 432 causes the patient's situation to be displayed on the display based on the patient's situation information. In this embodiment, the situation information output section 432 is one form of a display control section. Further, the situation information output unit 432 causes the audio information received from the patient's device 10 to be output as audio from the earphone of the HMD 24 .

The instruction information transmission section 433 transmits, for example, the three-dimensional model of the practitioner generated by the three-dimensional model generation section 412 and the video information generated by the video generation section 413 to the patient's device 10 as instruction information. Further, the instruction information transmitting unit 433 transmits the practitioner's voice information acquired by the microphone of the HMD 24 to the patient device 10 as instruction information. Further, the instruction information transmitter 433 may include position designation information that designates a position in the virtual space where the three-dimensional model of the practitioner is to be placed.

《Phantom limb pain care method》
FIG. 5 is a diagram showing a processing procedure for phantom limb pain care that is executed by the patient's device 10 based on a program. The patient's device 10 starts the process shown in FIG. 5 when it is activated or receives an instruction to start the process, and repeatedly executes the process until it is instructed to stop the process.

In step S10, the patient's device 10 detects the position of the patient's body, including the healthy limbs.

In step S20, the patient's device 10 receives instruction information from the practitioner's device 20.

In step S30, the patient device 10 generates a three-dimensional model of the patient based on the position detected in step S10.

In step S40, the patient device 10 arranges the three-dimensional model of the practitioner received as the instruction information in step S20 and the three-dimensional model of the patient generated in step S30 in the virtual space. Image information is generated when the virtual space is viewed from the positions of the subject's left and right eyes in the virtual space. Note that if audio information is received as the instruction information in step S20, this audio information is added to the video information.

In step S50, the patient's device 10 causes the display unit of the HMD 14 to display an image of the virtual space based on the image information, and causes the earphones 142 to output audio instructions from the practitioner.

FIG. 6 is a diagram showing the processing procedure of phantom limb pain care that the practitioner device 20 executes based on the program. The practitioner device 20 starts the process shown in FIG. 6 when activated or receives an instruction to start the process, and repeatedly executes the process until it is instructed to stop the process.

In step S110, the practitioner's device 20 detects the position of the practitioner's body, including the healthy limbs.

In step S120, the practitioner device 20 receives patient status information from the patient device 10.

In step S130, the practitioner device 20 generates a three-dimensional model of the practitioner based on the position detected in step S110.

In step S140, the practitioner device 20 places the three-dimensional model of the patient received as the patient situation information in step S120 and the three-dimensional model of the practitioner generated in step S130 in the virtual space. , generates left-eye image information and right-eye image information when looking into the virtual space from the positions of the practitioner's left and right eyes in this virtual space. Note that if audio information is received as the subject situation information in step S120, this audio information is added to the video information.

In step S150, the practitioner device 20 causes the display unit 141 of the HMD 24 to display an image of the virtual space based on the image information, and outputs the patient's voice from the earphones of the HMD 24.

"Concrete example"
FIG. 7 shows an example of the healthy limb 5 seen by the patient in real space when the left upper limb is designated as the patient's healthy limb.

FIG. 8 is a diagram showing an example of video information when a three-dimensional model 5M of a healthy limb 5 is generated and placed in a virtual space. In addition, in FIG. 8, elements other than the three-dimensional model 5M of the healthy limb 5 are omitted for explanation. As shown in FIG. 8, the three-dimensional model 5M displayed on the HMD 14 is displayed in a position overlapping the actual healthy limb 5 (FIG. 7).

The patient device 10 also generates a three-dimensional model 5L of the phantom limb symmetrically with the three-dimensional model 5M of the healthy limb 5, and as shown in FIG. Place and display model 5L. The images of these models 5M and 5L are generated in real time at a predetermined cycle, such as 30 frames/second, so the subject can move the models 5M and 5L in accordance with the healthy limb 5 that he or she is moving, so that the patient can see his or her own images. It can be seen as an upper limb. In this case, since the three-dimensional model 5L of the phantom limb is displayed within the field of view, the subject can feel as if the phantom limb exists.

Furthermore, as shown in FIG. 10, the patient device 10 displays a three-dimensional model 50 of the practitioner in the same virtual space as the three-dimensional models 5M and 5L of the patient. This allows the practitioner to show a video sample of the operation and give concise instructions such as ``Twist your arm like this'' or ``Please point your hand this way.'' In this embodiment, as shown in FIG. 10, the three-dimensional model of the practitioner is also displayed in the subject's field of view, but the configuration is not necessarily limited to this, and the three-dimensional model of the practitioner is also displayed. The configuration may be such that instructions are given only by voice without being displayed. In this case, the position detection units 21 and 22, the three-dimensional model generation unit 412, and the image generation unit 413 of the practitioner device 20 may be omitted.

In addition, in the virtual space, the positional relationship between the position where the three-dimensional model 50 of the practitioner is placed and the position of the patient may be arbitrarily selected by the practitioner. For example, when a practitioner inputs information specifying this positional relationship (layout information) into the practitioner device 20, the practitioner device 20 adds this placement information to instruction information and transmits it to the patient device 10. . The patient device 10 arranges the three-dimensional model 50 based on the received arrangement information.

FIG. 11 is a plan view showing the positional relationship between the three-dimensional model 51 of the patient and the three-dimensional model 50 of the practitioner in FIG. 10. As shown in FIG. 11, FIG. 10 is an arrangement in which the distance LA between the three-dimensional model 51 of the subject and the three-dimensional model 50 of the practitioner is relatively large (distal mode). In FIG. 12, as shown in FIG. 13, the distance LB between the three-dimensional model 51 of the patient and the three-dimensional model 50 of the practitioner is set smaller than in FIG. 10 (proximal mode). Furthermore, in FIG. 13, as shown in FIG. 14, the three-dimensional model 51 of the patient and the three-dimensional model 50 of the practitioner are placed in a position where they substantially overlap, and the three-dimensional models 50, 51 of the patient and practitioner are arranged in the same direction (superimposed mode). In this way, the distal mode allows you to see the entire practitioner, the proximal mode allows you to see the practitioner's limbs in close detail, and the superimposed mode allows you to see the practitioner's limbs side by side with the patient's limbs. I'm making it so you can see it. Note that FIGS. 10 and 12 show examples of images seen from the patient, but at this time, the practitioner device 20 is configured such that the three-dimensional model of the patient is facing displayed in the position you want.

Alternatively, one practitioner device 20 may be connected to a plurality of patient devices 10 and transmit instruction information to the plurality of patient devices 10 simultaneously. In this case, one practitioner device 20 receives patient situation information from a plurality of patient devices 10 and displays the conditions of the plurality of patients. FIG. 15 shows an example in which three-dimensional models 51 of a plurality of subjects are displayed in one virtual space.

In this way, according to the phantom limb pain care system of the present embodiment, the practitioner can give instructions to the patient's device 10 via the communication line N, so that a person (patient) in a remote location can receive instructions. It can be operated on. That is, even a person (patient) in a remote location can receive care for phantom limb pain.

In addition, in the phantom limb pain care system of this embodiment, by displaying three-dimensional models of the patient and the practitioner in the same virtual space, the practitioner can demonstrate the movement and provide easy-to-understand instructions. It can be performed.

In the phantom limb pain care system of this embodiment, by changing the positional relationship of the three-dimensional models of the patient and practitioner and displaying them, movements can be shown in a position that is easy for the patient to see, and instructions can be given in an easy-to-understand manner. It can be carried out.

In the phantom limb pain care system of this embodiment, by simultaneously transmitting instruction information from one practitioner device 20 to multiple patient devices 10, one practitioner can treat multiple patients at the same time. It can be operated on.

1: Phantom limb pain care system 5: Healthy limb 5M: 3D model of healthy limb 5L: 3D model of phantom limb 10: Subject device 11, 12: Position detection unit 13: Control unit 14: Head mounted display ( HMD)
20: Practitioner device 21: Position detection unit 22: Position detection unit 23: Control unit 24: HMD
N: Communication line

Claims (7)

A patient device that provides an image of a virtual space to the patient at a first point where the patient is present, and is operated by a practitioner located at a second point away from the first point. Equipped with a practitioner device,
The subject device is
a position detection unit that detects the position of a predetermined one of the subject's limbs as a healthy limb;
Generating a three-dimensional model of the healthy limb based on the position of the healthy limb detected by the position detection unit, and generating a three-dimensional model of a phantom limb to be paired with the healthy limb based on the position of the healthy limb. a three-dimensional model generation unit,
an image generation unit that generates an image of the virtual space including a three-dimensional model of the healthy limb and the phantom limb;
a head-mounted display that is attached to the subject's head and displays an image of the virtual space on a display screen;
a situation information transmitting unit that transmits patient situation information indicating the three-dimensional model to the practitioner device;
an instruction receiving unit that receives instruction information indicating instructions of the practitioner from the practitioner device;
an instruction output unit that outputs the instruction information to the patient as at least one of a part of the video and audio of the virtual space;
The practitioner device includes:
a situation information receiving unit that receives the patient situation information from the patient device;
a display control unit that displays the condition of the patient on a display based on the patient situation information;
A phantom limb pain care system, comprising: an instruction information transmitter that transmits the instruction information to the patient's device. The practitioner device includes:
a position detection unit that detects the position of a predetermined one of the limbs of the practitioner as a healthy limb, and the position of the healthy limb;
Generating a three-dimensional model of the healthy limb based on the position of the healthy limb detected by the position detection unit, and generating a three-dimensional model of a phantom limb to be paired with the healthy limb based on the position of the healthy limb. a three-dimensional model generation unit,
The instruction information transmitting unit includes a three-dimensional model based on the healthy limb of the practitioner and a three-dimensional model of the phantom limb in instruction information and transmits the instruction information to the patient device,
The image generation unit of the patient device synthesizes and displays the three-dimensional models of the healthy limb and phantom limb of the practitioner with the image of the virtual space based on the instruction information including the three-dimensional model. The phantom limb pain care system according to item 1. The image generation unit generates an image in which the three-dimensional model of the subject and the three-dimensional model of the practitioner face each other, or the three-dimensional model of the subject and the three-dimensional model of the practitioner 3. The phantom limb pain care system according to claim 2, wherein the phantom limb pain care system generates an image in which the three-dimensional model of the practitioner is positioned on the back side of the three-dimensional model of the subject, with the three-dimensional models facing the same direction. The image generation unit generates an image in which the three-dimensional model of the patient and the three-dimensional model of the practitioner face each other, and an image in which the three-dimensional model of the practitioner faces the same direction, based on the instruction information from the practitioner's device. The phantom limb pain care system according to claim 3, wherein the phantom limb pain care system is switched. The phantom limb pain care according to any one of claims 1 to 4, wherein one of the practitioner's devices connects to a plurality of the patient's devices and transmits the instruction information to the plurality of patient's devices. system. Providing an image of a virtual space to the patient at a first point where the patient is present, and communicating with a practitioner device operated by a practitioner located at a second point away from the first point. A patient device that performs
a position detection unit that detects the position of a predetermined one of the subject's limbs as a healthy limb;
Generating a three-dimensional model of the healthy limb based on the position of the healthy limb detected by the position detection unit, and generating a three-dimensional model of a phantom limb to be paired with the healthy limb based on the position of the healthy limb. a three-dimensional model generation unit,
an image generation unit that generates an image of the virtual space including a three-dimensional model of the healthy limb and the phantom limb;
a head-mounted display that is attached to the subject's head and displays an image of the virtual space on a display screen;
a situation information transmitting unit that transmits patient situation information indicating the three-dimensional model to the practitioner device;
an instruction receiving unit that receives instruction information indicating instructions of the practitioner from the practitioner device;
an instruction output unit that outputs the instruction information to the subject as at least one of a portion of the video of the virtual space and audio;
A patient device comprising: Providing an image of a virtual space to the patient at a first point where the patient is present, and communicating with a practitioner device operated by a practitioner located at a second point away from the first point. A program to be executed by a patient's device to perform the operation,
A step of determining a predetermined one of the subject's limbs as a healthy limb, and detecting the position of the healthy limb;
generating a three-dimensional model of the healthy limb based on the detected position of the healthy limb, and generating a three-dimensional model of a phantom limb paired with the healthy limb based on the position of the healthy limb;
generating an image of the virtual space including a three-dimensional model of the healthy limb and the phantom limb;
Displaying an image of the virtual space on a display screen of a head-mounted display attached to the subject's head;
transmitting patient situation information indicating the three-dimensional model to the practitioner device;
receiving instruction information indicating the practitioner's instructions from the practitioner's device;
outputting the instruction information to the patient as at least one of a portion of the video of the virtual space and audio;
A program for causing the patient's device to execute.

Images