JP2024003313A - Information processing device, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To build a reliability relation between a patient and a doctor on the occasion of online medical care and then transmit a high quality image needed for diagnosis of a diseased part to the doctor.

SOLUTION: A server 1 assists an online medical care between a doctor and a patient, and comprises a user image transmission unit 101, a user permission acceptance unit 102, and a specific region image transmission unit 103. The user image transmission unit 101 transmits an image to an expert terminal 3 in accordance with the acceptance of a request for transmission of the image from a user terminal 2 that includes the face of a patient. The user permission acceptance unit 102 accepts from the user terminal 2 an indication for permission to transmit an 8K video of a patient's diseased part (e.g., an eye) to the expert terminal 3. The specific region image transmission unit 103 switches the transmission quality to a high image quality in response to the acceptance of the indication for permission and transmits the 8K video of the patient's diseased part (e.g., an eye) to the expert terminal 3.

SELECTED DRAWING: Figure 9

COPYRIGHT: (C)2024,JPO&INPIT

Description

The present invention relates to an information processing device, an information processing method, and a program.

BACKGROUND ART Conventionally, in an image processing device for a dialogue device, there is a system that corrects the line of sight of the dialogue person to match, simultaneously solves the problem of "occlusion", and also performs facial beautification (see Patent Document 1).

Patent No. 6329121

However, no mechanism has been proposed for building a relationship of trust between a user and an expert and then transmitting an image that requires the expert's judgment to the expert.

The present invention was made in view of this situation, and it is possible to build a relationship of trust between a user and an expert and then send an image that requires the expert's judgment to the expert. The purpose is to

To achieve the above object, an information processing device according to one embodiment of the present invention includes:
A dialogue support device that supports dialogue between a user and an expert via a communication network,
User image transmitting means for transmitting the image to the expert's terminal in response to receiving the transmission of the image including the user's face from the user's terminal;
Expert image transmitting means for transmitting the image to the user's terminal in response to receiving the transmission of the image including the expert's face from the expert's terminal;
user permission receiving means for receiving permission from the user's terminal to transmit an image of a specific part of the user's body with a predetermined quality to the expert's terminal;
specific part image transmitting means for transmitting an image of the specific part to the terminal of the expert in response to receiving the permission;
Equipped with.
An information processing method and program corresponding to the above information processing apparatus according to one embodiment of the present invention are also provided as an information processing method and program according to one embodiment of the present invention.

According to the present invention, it is possible to build a relationship of trust between a user and an expert, and then transmit an image that requires the expert's judgment to the expert.

1 is a diagram illustrating an overview of this service that can be realized by an information processing system including a server according to an embodiment of an information processing apparatus of the present invention. It is a diagram for explaining remote medical diagnosis using remote medical equipment and medical diagnosis AI. FIG. 2 is a diagram showing an example of a communication business model for online medical treatment to which the present service of FIG. 1 is applied. FIG. 2 is a diagram showing a future image of telemedicine diagnosis innovation applying the service shown in FIG. 1. FIG. 2 is a diagram showing an IoT auscultation system as an example of an information processing system to which the present service of FIG. 1 is applied. 6 is a diagram showing the operation of each device of the IoT auscultation system of FIG. 5. FIG. 1 is a diagram illustrating an example of the configuration of an information processing system including a server according to an embodiment of an information processing apparatus of the present invention. 8 is a block diagram showing an example of the hardware configuration of a server in the information processing system shown in FIG. 7. FIG. 9 is a block diagram illustrating an example of a functional configuration for executing processing related to online medical treatment among the functional configurations of a server having the hardware configuration of FIG. 8. FIG. 10 is a diagram for explaining detailed functions of an online telemedicine function section in the functional configuration of the server in FIG. 9. FIG. FIG. 3 is a diagram showing an example of a display screen displayed by the conversation content display function according to the embodiment of the present invention. FIG. 3 is a diagram showing an example of a display screen displayed by the keyword specifying function according to the embodiment of the present invention. FIG. 3 is a diagram showing an example of a display screen displayed by the medical record description generation function according to the embodiment of the present invention. FIG. 3 is a diagram showing an example of a display screen displayed by the reference information presentation function according to the embodiment of the present invention. It is a figure which shows an example of the display screen displayed by the misrecognized word error correction function by embodiment of this invention. 10 is a flowchart showing the operation of executing processing related to online medical treatment in the server of FIGS. 8 and 9. FIG.

Embodiments of the present invention will be described below with reference to the drawings.
First, with reference to FIGS. 1 to 4 as appropriate, a service to which an information processing system (see FIG. 5 described later) to which an information processing apparatus according to an embodiment of the present invention is applied (hereinafter referred to as "this service") ”) will be explained below.

FIG. 1 is a diagram showing an overview of this service that can be realized by an information processing system including a server according to an embodiment of the information processing apparatus of the present invention.

Traditionally, patients and doctors have connected their respective terminals to a network to conduct online medical consultations, but as shown in Figure 1(a), it is difficult to understand what the doctor is saying even face-to-face. The patient's condition is unclear, and medical staff must be present at the patient's side to take the patient's pulse.
Specifically, the patient operates the stethoscope during auscultation, but it is difficult for the doctor to instruct the patient to operate the stethoscope. I can't understand what I want to say to the patient during an interview. It is difficult to understand the patient's condition during visual examination.

Therefore, as shown in Figure 1(b), this service provides remote medical equipment such as high-quality imaging equipment (cameras, microphones, etc.) and easy-to-operate remote stethoscopes (Shaberu, Hikaru), as well as 8K image quality. Rich biological information is collected by creating a remote environment with video (8K video), high resolution such as 5G communication, and high-speed communication, and AI (Artificial Intelligence) for medical diagnosis monitors conversations between patients and doctors. and present diagnostic support information to the doctor.
As diagnostic support information using AI for medical diagnosis, a message such as "You suspect that you have XX disease, please check for the presence of diastolic murmur" is presented, for example.
In addition, the remote medical device can observe the patient with 8K eyes (equivalent to 4.27 visual acuity), so the patient's facial expressions and the condition of the affected area can be clearly seen.
As a result, according to this service, it is possible to perform auscultation, visual inspection, and questioning in online medical treatment at a level equivalent to or better than that in face-to-face medical treatment.

Here, specific telemedicine diagnosis will be explained with reference to FIG. 2.
FIG. 2 is a diagram for explaining remote medical diagnosis using remote medical equipment and medical diagnosis AI.

As shown in FIG. 2, when a patient makes an utterance such as ``I've had chest pain since last week'' or ``I'm taking antihypertensive medication,'' the content of the interview is collected using a camera and microphone. Furthermore, as the patient speaks, the patient's biological information is collected by the remote medical device.
Here, the patient's biological information includes, for example, 8K video, heartbeat sound, blood flow sound, pulse wave, blood pressure, body temperature, blood oxygen concentration, etc.

Note that 8K video refers to video with a horizontal x vertical resolution of 7680 x 4320 pixels. Count 1000 as 1 kilometer (K). 8K video has a horizontal resolution of about 8000 pixels. 4K is 3840 x 2160, which is 4K, and full high-definition (full HD) is 1920 x 1080, which is 2K. Comparing the number of vertical and horizontal pixels, 8K is four times as many as 4K and 16 times as many as 2K. With the same pixel density, 8K can display an image 16 times larger than 2K. Conversely, if the area is the same, it can be expressed 16 times more precisely.

Medical diagnosis AI diagnoses, for example, "aortic stenosis" based on the collected interview contents (symptoms, etc.), biological information (numerical data), and patient information (pre-existing conditions, living conditions, etc.) stored in the database in advance. , aortic regurgitation disease, mitral stenosis, etc. are suspected.'' is posted on the doctor's terminal.
Specifically, the medical diagnosis AI automatically recognizes medical treatment keywords during a conversation, infers a disease (aortic stenosis) from the medical treatment keyword and biological information, and comments on the estimated disease (suspicion of aortic stenosis). , the reason (chest pain, taking antihypertensive medication, heart attack, etc.), and the recommended next action (e.g., ``Please check for humeral bone delay,'' etc.) are displayed on the screen of the doctor's terminal. Ru.

As a result, doctors can prevent diseases that they did not notice, such as "I did not notice aortic reflux disease" or overlooking rare diseases (such as Osler's nodules, Janeway lesions, etc.).
In addition, it is possible to efficiently narrow down suspected diseases.
In this way, online medical treatment using this service can provide richer diagnostic information than face-to-face diagnosis.

Here, a communication business model for online medical treatment will be explained with reference to FIG.
FIG. 3 is a diagram showing an example of an online medical treatment communication business model to which the present service of FIG. 1 is applied.
As shown in Figure 3, the communication business model for online medical treatment in this service includes doctor-side equipment (stethoscope, vital meter, short-range wireless communication network such as Bluetooth), AI server center, 4G/5G/IP, etc. Consists of a wide area wireless communication network, a watchful light bulb, a unique lighting type remote diagnostic device, a router for severe disasters, and patient-side devices (short-range wireless communication networks such as Bluetooth, remote medical devices, smart watches, IoT medical devices), etc. . Bluetooth is a registered trademark.
This telecommunications business model connects patients and doctors through high-quality, high-speed, large-capacity communications, making it possible to provide services related to online medical treatment.

Here, the future image of telemedicine diagnosis innovation will be explained with reference to FIG.
FIG. 4 is a diagram showing the future image of telemedicine diagnosis innovation applying the service shown in FIG. 1.

As shown in Figure 4, in the future, telemedicine diagnosis innovation will collect patient information from remote communication devices and vital information devices such as smartphones and smart watches, and AI will centrally manage the collected information in a life log bank. At the same time, the AI detects abnormalities in the patient from the collected information and reports them to medical institutions appropriate for the patient's abnormalities (family doctor, advanced medical institution, emergency medical institution, etc.).

For example, a 24-hour hearing aid device attached to the patient records data on the health status of the patient (Mr. A), a conversation between a communication robot and the patient (Mr. A), and monitoring of these devices using a monitoring light bulb or monitoring device. Monitor and Lifelog Bank collects data through its monitoring devices.
Additionally, in the event of a disaster, the smart watch records data on the victim's (Mr. B) condition and sends it to the lifelog bank.
Furthermore, while an emergency medical institution (ambulance team, etc.) goes to the scene and treats the patient (Mr. C), the life log bank acquires information from, for example, an AED via a smartphone.

The AI in the future image will analyze the data of each patient (Mr. A, Mr. B, and Mr. C) collected or acquired in the life log bank, and will make a report according to each patient's situation. Specifically, the AI inputs the data of each patient (Mr. A, Mr. B, and Mr. C) collected or acquired in the life log bank, and outputs a report according to the situation of each patient.
For example, in the case of a patient (Mr. A), the AI will contact the patient with a message such as "There are signs of mild heart disease. I will notify your family doctor."
Further, for example, in the case of the patient (Mr. B), the AI informs the emergency medical team, saying, ``He has decreased blood pressure and is in shock. Immediate treatment is required.''
Further, for example, in the case of the patient (Mr. C), the AI informs the advanced medical institution, ``Currently, heart rate is 40, blood pressure is 50-90, and cardiopulmonary bypass is required.''.

In this way, according to the future vision of telemedicine diagnosis innovation, wearable medical devices record patients' biometric information anytime and anywhere, AI analyzes the information recorded on the wearable medical devices, and biometric data and AI analysis results are generated. By providing this to medical institutions, it is possible to contribute to the realization of early treatment, improved diagnostic accuracy, and faster treatment.

Here, the IoT auscultation system will be explained with reference to FIG.
FIG. 5 is a diagram showing an IoT auscultation system as an example of an information processing system to which the present service of FIG. 1 is applied.
As shown in FIG. 5, the IoT auscultation system includes an auscultation graph terminal 51, auscultation doctor's equipment (auscultation headset (with microphone) 52 and joystick device 53), and remote patient-side equipment (stethoscope with microphone 54). ), a communication network (4G/5G/IP network) that connects these auscultation doctor side devices and remote patient side devices, and a wireless router 55 (near field wireless communication/wide area wireless communication router).

The auscultation graph terminal 51 can analyze an auscultation graph by operating in cooperation with an application program installed on a smartphone, a personal computer, or the like.

A remote stethoscope is configured by an auscultation doctor side device (auscultation headset (with microphone) 52 and joystick device 53) and a remote patient side device (stethoscope 54 with a microphone).

The joystick device 53 of the auscultation doctor's equipment has a circular portion when viewed from above, a joystick is placed in the center of the circular portion, and triangular LEDs are placed in eight directions (radially) around the joystick. This is what I did. A USB-C standard connector is arranged on the side surface of the joystick device 53 body. The connector is connected to a doctor's PC or the like.

In the joystick device 53 of the auscultation doctor's equipment, when the doctor operates the joystick, all the LEDs blink, indicating that an operation instruction has been received. Then, when the doctor tilts the joystick, the LED in the direction in which the doctor tilts the joystick blinks.

When the doctor tilts the joystick in any of the eight directions, the movement direction indication LED of the joystick device of the corresponding remote patient-side device lights up, allowing him to instruct the patient in the movement direction. When the doctor rotates the joystick, the volume is adjusted.
By pushing the joystick, the doctor can switch between low-frequency sounds (such as heart sounds) and high-frequency sounds (such as lung breathing sounds).

The stethoscope 54 with a microphone, which is a remote patient-side device, has a circular part when viewed from the top, a speaker is placed in the center of the circular part, and triangular parts are arranged in eight directions (radially) around the speaker. It is equipped with LEDs. A response microphone is arranged on the side of the stethoscope 54 body.

Normally, when a remote stethoscope is used for telemedicine, the patient is asked to operate the stethoscope based on conversational instructions from the doctor, but the conversational instructions do not accurately support the doctor's intended auscultation position. The problem was that it was difficult to perform auscultation diagnosis smoothly.

The functions of this IoT auscultation system include the following functions "1" to "8".
Specifically, ``1'' real-time vital waveform data transmission function, ``2'' motion sensor that detects movement of the device, ``3'' arrow LED that can be remotely operated to indicate direction, and ``4'' voice of doctor's instructions, etc. Output speaker, response microphone, ``5'' function to connect to smartphones and routers using short-range wireless communication, ``6'' IP connection function for short-range wireless communication network and wide area wireless communication to connect two joystick devices over long distances. ``7'' Charging function with a non-contact charger (hygienic as you don't have to touch the device); ``8'' Auscultation assistance and data recording function using vital waveform analysis software. .

In the IoT auscultation system of this embodiment, the joystick of the joystick device 53 of the auscultation doctor's device is used to instruct the direction of movement of the patient's stethoscope.
For example, the doctor tilts the joystick in the direction of movement of the part he wants to auscultate. Then, the LED of the stethoscope 54 placed on the patient side informs the direction.
The patient moves the stethoscope in the direction in which the LED lights up.
Doctors can use the conversation function to give detailed instructions such as "push harder" or "a little more to the right."
Further, by pushing the joystick device 53, the doctor can switch between a low frequency range (such as heart sounds) and a high frequency range (such as lung breathing sounds). This is the same as auscultation using the front and back sides of a regular stethoscope.
The physician can adjust the volume of the stethoscope by rotating the joystick to rotate the joystick axis of the joystick device 53.

Here, the operation of each device of the IoT auscultation system will be explained with reference to FIG.
FIG. 6 is a diagram showing the operation of each device of the IoT auscultation system of FIG. 5.
In the IoT auscultation system, as shown in FIG. 6, a stethoscope 54 with a microphone, which is a remote patient-side device, is connected to a smartphone 61 via Bluetooth communication (for example, BLE, etc.).
The auscultation sound is transmitted from the stethoscope 54 to the smartphone 61 using Bluetooth Audio. The smartphone 61 can analyze waveforms using an installed recording and waveform analysis application (general purpose).

As described above, according to the IoT auscultation system of the embodiment, in online medical treatment, the joystick device 53 is placed on the doctor's side, and the stethoscope 54 with a microphone that follows the operation of the joystick device 53 is placed on the patient's side. Since the doctor's instruction to move the stethoscope 54 can be accurately transmitted to the patient, remote auscultation can be realized.

Next, with reference to FIG. 7, the configuration of an information processing system that realizes the provision of the above-described service, that is, an information processing system including a server according to an embodiment of the information processing apparatus of the present invention will be described.
FIG. 7 is a diagram illustrating an example of the configuration of an information processing system including a server according to an embodiment of the information processing apparatus of the present invention.

The information processing system shown in FIG. 7 is configured to include a server 1, a user terminal 2, and an expert terminal 3.
The server 1, user terminal 2, and expert terminal 3 are each connected to be able to communicate with each other via a predetermined network N such as the Internet.

The server 1 is an information processing device managed by a service provider G. The server 1 executes various processes for realizing this service while appropriately communicating with each of the user terminal 2 and the expert terminal 3.

The user terminal 2 is an information processing device operated by the patient 1. The user terminal 2 is composed of, for example, a personal computer, a smartphone, a tablet, or the like.

The expert terminal 3 is an information processing device operated by a doctor. The expert terminal 3 is composed of, for example, a personal computer, a smartphone, a tablet, or the like.

Dedicated application software for users of this service (hereinafter referred to as "dedicated application") can be installed on each of the user terminal 2 and the expert terminal 3. Thereby, each of the patient and the doctor can use this service using each of the user terminal 2 and the expert terminal 3 in which the dedicated application is installed.
Furthermore, each of the patient and the doctor can also use this service by accessing a dedicated site displayed by the browser function of the user terminal 2 and the expert terminal 3, respectively.

In addition, unless otherwise specified, when the expression "the patient operates the user terminal 2" is used, it means any of the following. In other words, this means that the patient launches a dedicated application installed on the user terminal 2 and performs various operations, or accesses a dedicated site displayed by the browser function of the user terminal 2 and uses this service. ing.
Furthermore, when the phrase "a doctor operates the expert terminal 3" is used, it means one of the following. In other words, doctors are not allowed to use this service by launching a dedicated application installed on the expert terminal 3 and performing various operations, or by accessing a dedicated site displayed by the browser function of the expert terminal 3. It means.

FIG. 8 is a block diagram showing an example of the hardware configuration of a server in the information processing system shown in FIG. 7.

The server 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input/output interface 15, an input section 16, and an output section 17. , a storage section 18, a communication section 19, and a drive 20.

The CPU 11 executes various processes according to programs recorded in the ROM 12 or programs loaded into the RAM 13 from the storage section 18 .
The RAM 13 also appropriately stores data necessary for the CPU 11 to execute various processes.

The CPU 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input/output interface 15 is also connected to this bus 14 . An input section 16 , an output section 17 , a storage section 18 , a communication section 19 , and a drive 20 are connected to the input/output interface 15 .

The input unit 16 includes, for example, a keyboard, and inputs various information.
The output unit 17 includes a display such as a liquid crystal display, a speaker, and the like, and outputs various information as images and sounds.
The storage unit 18 is composed of a DRAM (Dynamic Random Access Memory) or the like, and stores various data.
The communication unit 19 communicates with other devices (for example, the user terminal 2 and the expert terminal 3 in FIG. 4) via a network N including the Internet.

A removable medium 30 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately installed in the drive 20. The program read from the removable medium 30 by the drive 20 is installed in the storage unit 18 as necessary.
Further, the removable medium 30 can also store various data stored in the storage section 18 in the same manner as the storage section 18.

Although not illustrated, the user terminal 2 and expert terminal 3 in FIG. 7 can also have basically the same hardware configuration as the hardware configuration shown in FIG. 8. Therefore, description of the hardware configurations of the user terminal 2 and the expert terminal 3 will be omitted.
It is assumed that the user terminal 2 and the expert terminal 3 are equipped with hardware and functions for conducting a remote conference.
Specifically, the user terminal 2 is equipped with a camera (imaging device) and a microphone that support functions to improve the quality of captured images, such as a switching function from full HD to 8K image quality, a zoom function, etc. or externally connected. It is assumed that the expert terminal 3 is equipped with or externally connected to a camera (imaging device) of general image quality and a microphone.

Through cooperation between the various hardware of the server 1 shown in FIG. 8 and the various software installed on the server 1, it becomes possible for the server 1 to execute various processes including processes related to online medical treatment. As a result, the service provider G can provide the above-described service to doctors and patients.
Here, the processing related to online medical treatment is a process executed to provide the above-mentioned service, and refers to a series of processes in which, for example, a doctor performs online medical treatment for a patient.

Hereinafter, with reference to FIG. 9, a functional configuration for executing processing related to online medical treatment, which is executed in the server 1 according to the present embodiment, will be described.
FIG. 9 is a block diagram illustrating an example of a functional configuration of an information processing system including the server of FIG. 7 for executing processing related to online medical treatment.

As shown in FIG. 9, in the CPU 11 of the server 1, when processing related to online medical treatment is executed, a user image transmitting section 101, a user permission receiving section 102, a specific region image transmitting section 103, and a vital information transmitting section 104, an expert image transmission section 105, a dialogue content display section 106, an expert document generation section 107, a speech support section 108, a judgment support section 109, and an online medical treatment function section 110.

Furthermore, a dialogue DB 181 is provided in one area of the storage unit 18 of the server 1 .
The dialogue DB 181 includes user information, expert information, patient medical record information (medical information associated with patient ID), and text information extracted from the audio of videos obtained from each terminal during online medical treatment. a list of system actions corresponding to patient and doctor gestures; a dictionary of medical terms for keywords; a speech-to-text conversion table for converting speech to text; , XX hours ago, YY days ago, etc.), a dictionary that stores information about the content of utterances that the doctor should say to the patient during the interview, and dialogue information between the doctor and the patient (for example, utterances and image information from both the doctor and the patient). etc.), patient vital information obtained from the user terminal 2 (heart rate, heart sounds, breathing sounds, eye movement, etc. obtained from the stethoscope 54), and expert documents (medical records of each patient, etc.) are stored. It can be read out as needed.

The user information is, for example, information about a patient, and is information in which the patient's name, address, contact information, etc. are associated with patient-specific identification information (patient ID).
The expert information is, for example, information about a doctor, and is information in which the name of the doctor is associated with identification information unique to the doctor (doctor ID).

The list of system actions includes text and gestures that allow the patient to transmit 8K video in response to an image switching request (inquiry) from the doctor, and system actions that allow the patient to transmit 8K video (or The operation of switching from 4K) to 8K is set as user permission information.

For example, in the list of system actions, for a gesture that raises the right hand, the system action is to allow the previous action, and for a gesture that lowers the right hand, the system action is to cancel the previous action. etc. are registered.
For example, if it has been prepared in advance to switch the patient's image to a high-quality 8K video and transmit it, the gesture of raising the right hand indicates that transmission of the 8K video is permitted, and the specific region image transmitting unit 103 8K video is transmitted.

In addition, in the list of system operations, the patient's affected area (image of the pupil of the eye) is displayed in response to meeting certain conditions (words such as transmitting a trajectory while switching 8K video, gesture of turning the right hand sideways, etc.). The system operation is set to track and transmit the trajectory (image) of the movement of that part (line of sight). In this case, when the doctor utters the above-mentioned words or makes the above-mentioned gesture, the locus (image) of the movement of the line of sight is transmitted to the expert terminal 3.

The user image transmitting unit 101 transmits the image to the expert terminal 3 in response to receiving a request from the user terminal 2 to transmit an image including a patient's face (for example, a bust image, etc.).
Specifically, when the patient and the doctor display the online medical treatment screen on their respective terminals and log in to start the online medical treatment, the user image transmission unit 101 transmits the patient's image transmitted from the user terminal 2. and transmits the received image to the destination expert terminal 3.

The user permission reception unit 102 receives from the user terminal 2 an instruction for permission to transmit 8K video of a patient's affected area (for example, the eye) to the expert terminal 3.
Specifically, when an 8K image transmission permission button (8K button, etc.) on the screen of the expert terminal 3 is operated by the patient, the button operation is accepted as an instruction indicating permission for higher image quality.

The specific region image transmitting unit 103 transmits the 8K video of the patient's eyes to the expert terminal 3 in response to receiving the permission instruction.
Specifically, the specific region image transmitting unit 103 transmits an image of a specific region, such as an 8K video of the patient's eyes, in response to a predetermined condition being met, such as receiving an 8K button operation. do. Satisfying the predetermined condition also includes the occurrence of a trigger.
Other permission instructions include, for example, receiving a predetermined utterance from the expert terminal 3, detecting a predetermined gesture, and accepting an operation of a predetermined button (such as an 8K button).
In addition, the specific region image transmitting unit 103 transmits an image related to the locus of the patient's line of sight movement in response to the satisfaction of a predetermined condition or the occurrence of a trigger. The predetermined condition here is, for example, that the eye tracking button on the screen of the user terminal 2 has been operated by the patient.

The vital information transmitting unit 104 transmits the patient's vital information (for example, the patient's heart rate, breathing rate, body temperature, etc.) to the expert terminal 3 in response to, for example, satisfying a predetermined condition.
Here, the predetermined conditions include, for example, receiving a predetermined utterance from the expert terminal 3 (e.g., "Is it okay if I switch to 8K video because I am going to measure your breathing rate?"), and receiving a predetermined gesture (e.g., handing a hand). This is any one or more of the following: detecting a button (such as raising the button), or receiving a predetermined button operation (for example, an 8K button operation).

The expert image transmitting unit 105 transmits the image to the user terminal 2 in response to receiving a request from the expert terminal 3 to transmit an image including the doctor's face.
Specifically, when the patient and the doctor display the online medical treatment screen on their respective terminals and log in to start the online medical treatment, the expert image transmitting unit 105 transmits the doctor's image transmitted from the expert terminal 3. and transmits the received image to the destination user terminal 2.

In response to receiving the utterance information from the patient and the doctor, the dialogue content display unit 106 outputs the utterance information as text information to the patient's user terminal 2 and the expert terminal 3.
Specifically, the dialogue content display unit 106 refers to the audio-text conversion table in the dialogue DB 181, converts the received audio data into corresponding text data, and sends the data to the patient's user terminal 2 and the doctor's expert terminal 3. The text is output in chat format (text is displayed separately on the left and right sides of the screen (see Figure 11)).
Further, the dialogue content display unit 106 extracts a predetermined keyword (for example, the back of the eye hurts) from the text information and outputs it on the screen of the expert terminal 3, or displays text information with the keyword portion emphasized. Output to the screen of home terminal 3.
As a result, the doctor can proceed with the interview based on the content of the patient's utterances, and consider and suggest treatment for the affected area, administration of medicine, etc., without missing a predetermined keyword such as the patient's pain in the back of the eye.

The expert document generation unit 107 generates a predetermined expert document such as a medical record based on a predetermined keyword (for example, eyes, stomach, chest, pain, XX hours ago, YY days ago, etc.) extracted by the dialogue content display unit 106. generate.
Specifically, the expert document generation unit 107 uses predetermined keywords (for example, eyes, stomach, chest, pain, XX hours ago, YY days ago, etc.) extracted by the dialogue content display unit 106 and the user of the patient under examination. Based on the ID, the dialog DB 181 is searched, the patient's examination information is read out, and a predetermined expert document such as the patient's medical record is generated. If the patient is seeing the patient for the first time, a new patient chart is created from the patient information obtained through dialogue with the patient.
In this way, based on the extracted predetermined keywords such as eyes, stomach, chest, pain, XX hours ago, YY days ago, etc., the doctor can proceed with the dialogue while presenting the patient's medical record.

The speech support unit 108 sends information regarding the content of speech that the doctor should utter (such as a message for interviewing) to the expert terminal 3 based on dialogue information of the patient or doctor (for example, content of speech, image information, vital information, etc.). Send.
Specifically, the speech support unit 108 extracts keywords (chest, pain, etc.) from patient or doctor interaction information (for example, speech content, image information, vital information, etc.) and identifies diseases, diseases, etc. that correspond to the keywords. (For example, messages for interview (such as fixed phrases such as checking the heartbeat and heart sounds with a stethoscope) are read from the medical dictionary of the dialogue DB 181 and sent to the expert terminal 3 as information regarding the content of speech that the doctor should utter. As a result, a message for an interview (for example, "Suspected: Aortic valve stenosis, etc.", see FIG. 2) is displayed on the screen of the expert terminal 3.
As a result, inexperienced doctors who have difficulty gaining a relationship of trust from patients can conduct online telemedicine without showing their inexperience by referring to messages for interviewing, etc. Telemedicine can be conducted efficiently.

The judgment support unit 109 transmits judgment information regarding the patient, such as information specifying the affected area, to the expert terminal 3 based on the dialogue information of the patient or the doctor.
Specifically, the judgment support unit 109 generates, for example, information identifying the affected area (heart image, etc.), information related to the patient, based on dialogue information of the patient or doctor (for example, utterance content, image information, vital information, etc.). Judgment information (for example, diagnosis information specifying the name of the disease and the affected area ("suspected of the following, aortic stenosis, etc.") is transmitted to the expert terminal 3.
This allows an inexperienced doctor to more efficiently perform remote online telemedicine by referring to information (heart images, etc.) that identifies the patient's affected area.

The online medical treatment function unit 110 executes processing of various functions related to online medical treatment.
Note that details of various functions of the online medical treatment function section 110 will be explained in detail with reference to FIG. 10.
Specifically, the online medical treatment function unit 110 uses the remote conference function provided by the remote conference providing means of the server 1 to collect moving images (videos) of the doctor or patient obtained by the user terminal 2 and the expert terminal 3, respectively. This provides an environment where doctors and patients can have smooth conversations by exchanging audio and voice data in real time.

Here, with reference to FIG. 10, detailed functions of the online medical treatment function section 110 of the server 1 shown in FIG. 9 will be described.
FIG. 10 is a diagram for explaining detailed functions of the online medical treatment function section 110 shown in FIG. 9.
As shown in FIG. 10, the online medical treatment function unit 110 includes a remote conference function, a keyword identification function, a conversation content display function, a keyword identification function, an error correction function for misrecognized words, a reference information presentation function, and the like.

The content of the conversation between the doctor and the patient during online medical treatment in a remote conference is converted into text by the voice recognition means (not shown) on the server 1 using the conversation content display function, and is sent to the user terminal 2 and the expert as text information. displayed on each screen of the terminal 3.

When converting into text on server 1, the keyword identification function checks whether the keyword is included in the text of the conversation, and if the keyword is included, the conversation is displayed with the keyword highlighted. After changing any of the display specifications such as display color, font, character size, etc., the information is displayed on the screens of the user terminal 2 and the expert terminal 3 respectively.

If an error in text is found in the content of the conversation displayed on the screen, the text will be corrected by the misrecognized phrase error correction function based on the doctor's or patient's operations. In addition, it is checked whether or not technical terms are included in the content of the text-based conversation, and if technical terms are included, the reference information display function displays item information with a link destination where the technical term can be searched. Since it is displayed on the respective screens of the user terminal 2 and the expert terminal 3, the doctor or patient can search by clicking on the item information to which this link is attached, and check the explanation of the technical term. I can do it.

If the doctor or patient contains a keyword necessary for creating a medical record in the content of the conversation that has been converted into text, by dragging and dropping the keyword onto the medical record description generation screen, the medical record description generation function will generate the medical record description necessary for creating a medical record. is generated. The generated medical record description is transferred to the medical record input screen using the medical record creation function, which saves doctors the time and effort required to create medical records, allowing them to focus more on conversation with the patient and extract the information necessary for diagnosis. .

FIG. 11 is a diagram showing an example of a display screen displayed by the conversation content display function according to the embodiment of the present invention.
Referring to FIG. 11, the display contents displayed on the conversation contents display screens of the user terminal 2 and the expert terminal 3 are shown, and the contents of the conversation exchanged between the doctor and the patient during the interview are displayed by the speaker. will be transcribed and displayed in an easy-to-understand manner. Since the speakers are displayed as separate icons on both sides of the conversation, it is possible to distinguish between the doctor's statement and the patient's statement at a glance.

The content of the conversation is sequentially converted into text by the server 1 and additionally displayed on the conversation content display screen. New conversation content is added to the bottom of the screen, and when the conversation content display screen is filled with conversation content, the old conversations at the top disappear as new conversation content is added to the bottom row. It will be displayed as follows. The conversation content display screen is displayed so that the contents of old conversations that have disappeared can be checked again by scrolling upward.
Both the doctor and the patient can check the content of the conversation on the conversation content display screen. This allows you to say or listen to things that you didn't say or hear enough, making the interview more fulfilling.

FIG. 12 is a diagram showing an example of a display screen displayed by the keyword specifying function according to the embodiment of the present invention.
Referring to FIG. 12, the display contents displayed on the conversation content display screens of the user terminal 2 and the expert terminal 3 are the same as those shown in FIG. 11, but the example shown in FIG. , the keywords are highlighted and displayed, for example, by shading, boldface, color change, etc.
The highlighted phrases such as "stomach,""pain," and "sharp pain" are important expressions expressing the patient's symptoms, and are information necessary to determine the cause of the disease.

If a particularly important keyword is not included in the conversation, the screen will be displayed in plain language without any highlighted keywords, as shown in the example diagram shown in Figure 12, but if the keyword is included, the diagram will be displayed. When displayed on the screen as in 12, the keywords are displayed prominently.

Keywords in their statements are highlighted and displayed, allowing patients to understand what information is important to convey to the doctor, and also to find out additional information related to the keyword. It can be an opportunity to speak out.

FIG. 13 is a diagram showing an example of a display screen displayed by the medical record description generation function according to the embodiment of the present invention.
Referring to FIG. 13, the conversation content display screen, medical record description generation screen, and medical record input screen of the user terminal 2 and the expert terminal 3 are divided and displayed. Note that the conversation content display screen is shown in a reduced size in the figure, but this is shown in a reduced size to explain the operation. Displayed in an easy-to-read size.

Keywords such as "stomach" and "pain" are highlighted on the conversation content display screen, similar to the example of FIG. 12. The doctor selects the keywords necessary for chart creation from the keywords, drags and drops the selected keywords into the corresponding item field in the chart description generation screen, and the keywords are reflected in the chart description generation screen. .

The medical record description generation screen is a screen that supports the generation of items to be entered into the medical record, and includes major items such as "symptoms," "family history," "travel history," and "drug history." Small items are prepared within each large item. For example, in the "Symptoms" column, there are sub-items such as "Symptoms, location", "How it starts, nature, time of onset, severity", etc.

In the example in Figure 13, the keywords ``stomach'' and ``pain'' in the conversation content display screen are added to the ``symptoms, site'' column of the ``symptoms'' column, and the keywords ``six hours ago'' and ``sharp pain'' are By dragging and dropping the required keywords into the ``How to begin, nature, time of onset, severity'' item column in the ``Symptoms'' column, the necessary keywords will be reflected on the medical record description generation screen. The keywords reflected on the medical record description generation screen are not limited to one symptom.

For this reason, the medical record creation function of the server 1 organizes the keywords based on the order and time of the keywords reflected on the medical record description generation screen, and generates a medical record description. The generated medical record description is transferred to the medical record input screen. The keywords reflected on the medical record description generation screen do not necessarily result in correct sentence descriptions just by arranging them in order.

Therefore, the medical record creation function may be provided with an AI function to generate correct sentence descriptions that include keywords. At this time, a term conversion function may be provided to replace "stomach" with "upper abdomen."

FIG. 14 is a diagram showing an example of a display screen displayed by the reference information presentation function according to the embodiment of the present invention.
Referring to FIG. 14, part of the display content displayed on the conversation content display screens of the user terminal 2 and the expert terminal 3 is shown, and a doctor's message ``The cause of gastric ulcers is Helicobacter pylori rather than stress.'' In response to the statement "There are many", the item information "gastric ulcer" and "H. pylori" is displayed below the doctor's statement in response to the reference terms "gastric ulcer" and "H. pylori", such as technical terms included in the statement. has been done. Item information such as ``gastric ulcer'' and ``Helicobacter pylori'' has links that allow you to search for explanations of each.

For example, if the explanation for "stomach ulcer" is saved as internal content, when a doctor or patient clicks on the item information "stomach ulcer", the saved explanation for "stomach ulcer" will be displayed on the screen. .
For example, if "H. pylori" is not saved as internal content, a searchable link is provided as external content, and when a doctor or patient clicks on the item information "H. pylori", the linked " An explanatory statement about "H. pylori" is displayed on the screen.

As a result, even if the patient cannot fully understand the information from the doctor's explanation alone, the patient can refer to the explanation and acquire correct knowledge. Furthermore, the doctor can appropriately refer to the explanatory notes and explain the information in an easy-to-understand manner to the patient.

FIG. 15 is a diagram showing an example of a display screen displayed by the misrecognized word error correction function according to the embodiment of the present invention.
Referring to FIG. 15, part of the display contents displayed on the conversation content display screens of the user terminal 2 and the expert terminal 3 are shown, and the doctor's question ``How did the pain start?'' is shown. An example is shown in which when a patient answers a question, ``It's getting more and more painful,'' the voice recognition is misrecognized and the answer is ``My husband's getting more and more painful.'' Originally, the expression ``It's getting more and more painful'' is a keyword that represents a symptom, but if ``gradually'' is mistakenly interpreted as ``husband,'' it is not recognized as a keyword.

Therefore, when the doctor or patient finds a misrecognized word and selects the misrecognized word with the mouse and pronounces it again as "dandan," the misrecognized word error correction function corrects "husband" to "dandan." . As a result, the part "It's getting more and more painful" is highlighted as a keyword.
Note that the misrecognized phrase error correction function corrects not only keywords, but also if the content of the conversation includes a misrecognized phrase that is different from the utterance, it will be selected and corrected in the same way. Since it can be corrected later in this way, it can ultimately be saved as correct progress information data of the medical interview.

Next, the operation of the server when online medical treatment is performed will be described with reference to FIG. 16.
FIG. 16 is a flowchart showing the operation of executing processing related to online medical treatment in the servers of FIGS. 8 and 9.

In this case, in the server 1, in step S11 of FIG. Send.

In step S12, the expert image transmitting unit 105 transmits the image to the user terminal 2 in response to receiving a request from the expert terminal 3 to transmit an image including the doctor's face.

In step S13, the user permission reception unit 102 receives an instruction for permission to transmit the 8K video of the patient's eyes to the expert terminal 3 from the user terminal 2.

In step S14, the specific region image transmitting unit 103 transmits the 8K video of the patient's eyes to the expert terminal 3 in response to receiving the permission instruction.

As a result, while patients and doctors perform interactive medical treatment via Network N, the patient can transmit images of the patient's eyes in 8K quality for eye treatment by giving permission to the doctor based on his/her own judgment. do. By viewing images of the eyes in 8K quality, doctors can visually diagnose the patient's eyes as if they were right in front of them, so they can build a relationship of trust between patient and doctor. Images necessary for diagnosing the affected area can be sent to a doctor and the doctor can make a decision.

Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and modifications, improvements, etc. within the range that can achieve the purpose of the present invention are included in the present invention. regarded as.

For example, in the above embodiment, an example of 8K video was explained, but other than this, for example, 16K video, 32K video, etc. may be used, and if the camera has a zoom function, zoom operation etc. may be combined. Good too.

In the above-described embodiment, the user image transmitting unit 101 transmits the image to the expert terminal 3 in response to receiving the transmission of the image including the patient's face from the user terminal 2. Often, it is sufficient to transmit the image to the expert's terminal in response to receiving the transmission of the image including the user's face from the user's terminal.

Furthermore, in the embodiment described above, the user permission receiving unit 102 receives permission from the user terminal 2 to transmit an 8K video with 8K image quality, which is an image of the patient's eyes, to the expert terminal 3. It is sufficient to receive permission from the user's terminal to transmit an image of a specific part of the user's body with a predetermined quality to the expert's terminal.

Further, in the above-described embodiment, the specific region image transmitting unit 103 transmits the 8K video of the patient's affected region (eyes, etc.) to the expert terminal 3 in response to receiving permission. It is sufficient to send an image of a specific region with a predetermined quality to the expert's terminal in response to acceptance of permission.

Further, in the above-described embodiment, the expert image transmitting unit 105 transmits the image to the user terminal 2 in response to receiving the transmission of the image including the doctor's face from the expert terminal 3. It is sufficient that the image is transmitted to the user's terminal in response to receiving the transmission of the image including the expert's face from the expert's terminal.

In addition, in the above-described embodiment, a service of online medical treatment (remote interview by a doctor to a patient) was described, but services other than this may also be used, such as psychological analysis using the patient's (user's) voice, or using images of pupil size. The present invention may be applied to services that perform psychological analysis of users, such as psychological analysis and psychological analysis based on skin temperature, and it is possible to provide services that combine these with line-of-sight analysis and polygraph analysis.

Further, the system configuration shown in FIG. 7 and the hardware configuration of the server 1 shown in FIG. 8 are merely examples for achieving the object of the present invention, and are not particularly limited.

Further, the functional block diagram shown in FIG. 9 is merely an example and is not particularly limited. In other words, it is sufficient that the information processing system is equipped with a function that can execute the above-mentioned series of processes as a whole, and what kind of functional blocks and databases are used to realize this function can be determined by referring to the example in FIG. 9 in particular. Not limited.

Further, the locations of the functional blocks and the database are not limited to those shown in FIG. 9, and may be arbitrary.
For example, in the example of FIG. 9, the above-mentioned processing related to online medical treatment is performed on the server 1 side, but the configuration is not limited to this, and at least part of the processing is performed on the user terminal 2 or the expert terminal 3 side. May be done.
That is, the functional blocks and database necessary for executing the process are configured to be included in an information processing device such as the server 1, but this is merely an example. An information processing device such as the user terminal 2 or the expert terminal 3 may include at least a portion of the functional blocks and database located on the server 1 side.

Furthermore, the series of processes described above can be executed by hardware or by software.
Further, one functional block may be configured by a single piece of hardware, a single piece of software, or a combination thereof.

When a series of processes is executed by software, a program constituting the software is installed on a computer or the like from a network or a recording medium.
The computer may be a computer built into dedicated hardware.
Further, the computer may be a computer that can execute various functions by installing various programs, such as a server, a general-purpose smartphone, or a personal computer.

Recording media containing such programs not only consist of removable media (not shown) that is distributed separately from the device itself in order to provide the program to advertisers, but also include a recording medium that is pre-installed in the device body and is used to store advertisements. It mainly consists of provided recording media, etc.

Note that in this specification, the step of writing a program to be recorded on a recording medium is not only a process that is performed chronologically in accordance with the order, but also a process that is not necessarily performed chronologically but in parallel or individually. It also includes the processing to be executed.
Furthermore, in this specification, the term system refers to an overall device composed of a plurality of devices, a plurality of means, and the like.

In summary, the information processing apparatus to which the present invention is applied only needs to have the following configuration, and can take various embodiments.
That is, an information processing device (for example, server 1 in FIG. 9) to which the present invention is applied,
An information processing device (e.g., server 1 in FIG. 9) that supports dialogue (e.g., online medical treatment, etc.) between a user (e.g., patient, etc.) and an expert (e.g., doctor, etc.) via a communication network,
In response to receiving the transmission of an image including the face of the user (e.g., patient, etc.) from the user's terminal (e.g., user terminal 2 in FIG. 9), the expert's terminal (e.g., the expert terminal in FIG. 9) a user image transmitting means (for example, the user image transmitting unit 101 in FIG. 9),
In response to receiving the transmission of an image including the face of the expert (for example, a doctor, etc.) from the expert's terminal (for example, the expert terminal 3 in FIG. 9), the user's terminal (for example, the user in FIG. 9) an expert image transmitting means (for example, the expert image transmitting unit 105 in FIG. 9) that transmits the image to the terminal 2, etc.;
An image (for example, 8K video) that is an image of a specific part of the body of the user (for example, a patient, etc.) (for example, an affected part such as an eye) with a predetermined quality (for example, a resolution of 7680 pixels horizontally x 4320 pixels vertically: 8K image quality, etc.) etc.) to the expert's terminal (for example, the expert terminal 3 in FIG. 9) from the user's terminal (for example, the user terminal 2 in FIG. 9). (user permission reception unit 102, etc.);
A specific part that transmits an image (for example, 8K video, etc.) of the specific part (for example, an affected part such as an eye) to the terminal of the expert (for example, the expert terminal 3 in FIG. 9) in response to receiving the permission. An image transmitting means (for example, the specific part image transmitting unit 103 in FIG. 9),
Equipped with.

According to such a configuration, while a user (e.g., patient, etc.) and an expert (e.g., doctor, etc.) interact via a communication network, the user (e.g., patient, etc.) By granting permission to an expert (e.g. doctor, etc.), an image of a specific part of the user's (e.g. patient's) body (e.g. an affected area such as an eye) can be transmitted at a predetermined quality (e.g. 8K image quality, etc.) etc.) allows an expert (e.g. doctor, etc.) to view an image of a specific body part (e.g., an affected part of the eye, etc.) at a predetermined quality (e.g., 8K video, etc.). Since it is possible to make judgments (e.g., diagnosis of the affected part) regarding the affected part of the eye, etc., it is possible to make judgments (e.g., diagnosis of the affected part, etc.) after building a relationship of trust between the user (e.g., patient, etc.) and the expert (e.g., doctor, etc.). Images necessary for judgment by a doctor, etc. can be sent to an expert (for example, a doctor, etc.), and the expert (for example, a doctor, etc.) can make a judgment.
In other words, a user (e.g., a patient, etc.) sends an image of a specific part of the user's body (e.g., an affected area such as an eye) by giving permission to an expert based on his or her own judgment. It is possible to support face-to-face consultation (for example, online telemedicine, etc.) based on a relationship of trust between the patient and a specialist (for example, a doctor).
As a result, it is possible to build a relationship of trust between users (e.g., patients, etc.) and experts (e.g., doctors, etc.) and then realize remote face-to-face consultations (e.g., online telemedicine, etc.) in a practical manner. can.

The information processing device (for example, server 1 in FIG. 9) is
The specific part image transmitting means (for example, the specific part image transmitting unit 103 in FIG. 9) is configured to meet predetermined conditions (for example, receiving a predetermined utterance from the expert terminal 3, detecting a predetermined gesture, and pressing a predetermined button). The image of the specific region (for example, 8K video of the patient's eyes, etc.) is transmitted in response to the condition (such as acceptance of an operation) being satisfied.
As a result, when a doctor feels it necessary, he or she can learn about the user's condition, such as eye movements, which should not be overlooked clinically, simply by making a predetermined utterance or operating a button.

The specific part image transmitting means (for example, the specific part image transmitting unit 103 in FIG. 9, etc.) transmits a message when a predetermined condition is satisfied or a trigger occurs (for example, a predetermined utterance is received from an expert terminal, a predetermined gesture is received, etc.). (detection, reception of a predetermined button operation, etc.), an image related to the locus of the movement of the user's line of sight is transmitted.
This allows experts (e.g., doctors, etc.) to monitor user (e.g., patients') conditions that should not be overlooked clinically, such as eye movements, simply by making predetermined utterances or button operations when they feel it is necessary. This information can be determined based on the image of the line of sight movement trajectory, allowing for more accurate diagnosis.

The information processing device (for example, server 1 in FIG. 9) is
The user (for example, a patient, etc.) in response to satisfying a predetermined condition (for example, receiving a predetermined utterance from the expert terminal 3, detecting a predetermined gesture, accepting a predetermined button operation, etc.) Vital information transmitting means (for example, the vital information transmitting unit 104 in FIG. 9) for transmitting vital information (for example, the patient's heart rate, breathing rate, body temperature, etc.);
Furthermore, it is equipped with.
This allows doctors to obtain vital information about users (e.g., patients) that should not be overlooked clinically, and make more accurate diagnoses, simply by making predetermined utterances or button operations when they feel it is necessary. can.

The information processing device (for example, server 1 in FIG. 9) is
In response to receiving utterance information from the user (e.g., patient, etc.) and the expert (e.g., doctor, etc.), the utterance information is converted into text information (by converting audio data into text data), and the user (e.g., patient, etc.) and a dialog content display means (for example, the dialog content display unit 106 in FIG. 9, etc.) outputted to the terminals of the experts (for example, the user terminal 2, the expert terminal 3, etc.);
Furthermore, it is equipped with.
As a result, the content of the dialogue (e.g., medical interview, etc.) that an expert (e.g., a doctor, etc.) has with a user (e.g., a patient, etc.) is output as text information to each terminal, so that the user (e.g., including specialized content) can be outputted as text information to each terminal. For example, the dialogue can be continued while confirming the patient's understanding.

The dialogue content display means (for example, the dialogue content display unit 106 in FIG. 9) extracts and emphasizes predetermined keywords (for example, eyes, stomach, chest, pain, XX hours ago, YY days ago, etc.) from the text information. and output it.
This allows experts (e.g., doctors) to proceed with dialogue (e.g., interviewing) based on the content of the utterance without overlooking predetermined keywords (e.g., pain in the back of the eye), and to come up with solutions to the consultation (e.g., how to treat the affected area). and the administration of drugs, etc.) and present them.

Based on the predetermined keywords (for example, eyes, stomach, chest, pain, XX hours ago, YY days ago, etc.) extracted by the dialog content display means (for example, the dialog content display section 106 in FIG. 9), a predetermined specialty is displayed. expert document generation means (for example, the expert document generation unit 107 in FIG. 9) that generates professional documents (for example, medical records, etc.);
Furthermore, it is equipped with.
In this way, based on the extracted predetermined keywords (e.g., eyes, stomach, chest, pain, XX hours ago, YY days ago, etc.), experts (e.g., doctors, etc.) can provide information to users (e.g., patients, etc.). Dialogue can proceed while presenting expert documents (for example, medical records, etc.).

The information processing device (for example, server 1 in FIG. 9) is
Information regarding the content of utterances that should be uttered by the expert (e.g., doctor, etc.) based on dialogue information (e.g., utterance content, image information, vital information, etc.) of the user (e.g., patient, etc.) or the expert (e.g., doctor, etc.) utterance support means (for example, the utterance support unit 108 in FIG. 9) that transmits (a message for medical interview, etc.) to the terminal of the expert (for example, the expert terminal 3 in FIG. 9);
Furthermore, it is equipped with.
This allows inexperienced professionals (e.g. doctors, etc.) to consult face-to-face consultations based on information about the content of utterances (messages for questioning, etc.) when it is difficult to gain a relationship of trust from users (e.g. patients, etc.). By doing (for example, online telemedicine, etc.), face-to-face consultations (for example, online telemedicine, etc.) can be efficiently conducted without showing the lack of experience.

In the information processing device (for example, server 1 in FIG. 9),
Judgment information regarding the user (for example, information identifying the affected area) based on dialogue information (for example, utterance content, image information, vital information, etc.) of the user (for example, patient, etc.) or the expert (for example, doctor, etc.) etc.) to the terminal of the expert (for example, the expert terminal 3 in FIG. 9) (for example, the decision support unit 109 in FIG. 9);
Furthermore, it is equipped with.
This allows inexperienced experts (e.g., doctors) to refer to judgment information about the user (e.g., information identifying the affected area) to make remote face-to-face consultations (e.g., online telemedicine, etc.) more efficient. can be done.

An information processing method executed by an information processing device (e.g., server 1 in FIG. 9) that supports dialogue (e.g., online medical treatment, etc.) between a user (e.g., patient, etc.) and an expert (e.g., doctor, etc.) via a communication network. There it is,
In response to receiving the transmission of an image including the user's face from the user's terminal (for example, user terminal 2 in FIG. 9), the expert's terminal (for example, expert terminal 3 in FIG. 9) transmits the image. a user image transmission step of transmitting an image (for example, step S11 in FIG. 16, etc.);
In response to receiving the transmission of an image including the face of the expert (for example, a doctor, etc.) from the expert's terminal (for example, the expert terminal 3 in FIG. 9), the user's terminal (for example, the user in FIG. 9) an expert image transmission step (for example, step S12 in FIG. 16) of transmitting the image to a terminal 2, etc.);
An image (e.g., 8K video, etc.) of a specific part of the body of the user (e.g., patient, etc.) (e.g., an affected area such as an eye) with a predetermined quality (e.g., resolution of 7680 pixels horizontally x 4320 pixels vertically, etc.) A user permission receiving step (for example, step S13 in FIG. 16) of accepting permission from the user terminal (for example, user terminal 2 in FIG. 9) to transmit data to the expert's terminal (for example, expert terminal 3 in FIG. 9); etc.) and
A specific part that transmits an image (for example, 8K video, etc.) of the specific part (for example, an affected part such as an eye) to the terminal of the expert (for example, the expert terminal 3 in FIG. 9) in response to receiving the permission. an image transmission step (for example, step S14 in FIG. 16),
including.
In this way, a user (e.g., a patient) sends an image of a specific part of the user's body (e.g., an affected area such as an eye) by giving permission to an expert based on his or her own judgment. It is possible to support face-to-face consultation (for example, online telemedicine, etc.) based on a relationship of trust between a patient, etc.) and an expert (for example, a doctor, etc.).
As a result, it is possible to build a relationship of trust between users (e.g., patients, etc.) and experts (e.g., doctors, etc.) and then realize remote face-to-face consultations (e.g., online telemedicine, etc.) in a practical manner. can.

A computer (see FIG. 9 CPU11 etc.),
In response to receiving the transmission of an image including the user's face from the user's terminal (for example, user terminal 2 in FIG. 9), the expert's terminal (for example, expert terminal 3 in FIG. 9) transmits the image. a user image transmission step of transmitting an image (for example, step S11 in FIG. 16, etc.);
In response to receiving the transmission of an image including the face of the expert (for example, a doctor, etc.) from the expert's terminal (for example, the expert terminal 3 in FIG. 9), the user's terminal (for example, the user in FIG. 9) an expert image transmission step (for example, step S12 in FIG. 16) of transmitting the image to a terminal 2, etc.);
An image (e.g., 8K video, etc.) of a specific part of the body of the user (e.g., patient, etc.) (e.g., an affected area such as an eye) with a predetermined quality (e.g., resolution of 7680 pixels horizontally x 4320 pixels vertically, etc.) A user permission receiving step (for example, step S13 in FIG. 16) of accepting permission from the user terminal (for example, user terminal 2 in FIG. 9) to transmit data to the expert's terminal (for example, expert terminal 3 in FIG. 9); etc.) and
A specific part that transmits an image (for example, 8K video, etc.) of the specific part (for example, an affected part such as an eye) to the terminal of the expert (for example, the expert terminal 3 in FIG. 9) in response to receiving the permission. an image transmission step (for example, step S14 in FIG. 16),
Execute control processing including
In this way, a user (e.g., a patient) sends an image of a specific part of the user's body (e.g., an affected area such as an eye) by giving permission to an expert based on his or her own judgment. It is possible to support face-to-face consultation (for example, online telemedicine, etc.) based on a relationship of trust between a patient, etc.) and an expert (for example, a doctor, etc.).
As a result, it is possible to build a relationship of trust between users (e.g., patients, etc.) and experts (e.g., doctors, etc.) and then realize remote face-to-face consultations (e.g., online telemedicine, etc.) in a practical manner. can.

1... Server, 2... User terminal, 3... Expert terminal, 11... CPU, 12... ROM, 13... RAM, 14... Bus, 15... Input Output interface, 16... Input section, 17... Output section, 18... Storage section, 19... Communication section, 20... Drive, 30... Removable media, 101... User image Transmission unit, 102... User permission reception unit, 103... Specific region image transmission unit, 104... Vital information transmission unit, 105... Expert image transmission unit, 106... Dialogue content display unit, 107... Expert document generation unit, 108... Speech support unit, 109... Judgment support unit, 110... Online medical treatment function unit, 181... Dialogue DB, G... Service provider, B...buyer, C...supplier, N...network

Claims (11)

An information processing device that supports dialogue between a user and an expert via a communication network,
User image transmitting means for transmitting the image to the expert's terminal in response to receiving the transmission of the image including the user's face from the user's terminal;
Expert image transmitting means for transmitting the image to the user's terminal in response to receiving the transmission of the image including the expert's face from the expert's terminal;
user permission receiving means for receiving permission from the user's terminal to transmit an image of a specific part of the user's body with a predetermined quality to the expert's terminal;
specific part image transmitting means for transmitting an image of the specific part to the terminal of the expert in response to receiving the permission;
An information processing device comprising: The specific part image transmitting means transmits an image of the specific part of the user in response to satisfying a predetermined condition.
The information processing device according to claim 1. The specific part image transmitting means transmits an image related to the trajectory of the user's line of sight movement in response to satisfying a predetermined condition.
The information processing device according to claim 1. vital information transmitting means for transmitting the user's vital information in response to satisfying a predetermined condition;
The information processing device according to claim 1, further comprising: Dialogue content display means for outputting the utterance information as text information to the terminals of the user and the expert in response to receiving the utterance information of the user and the expert;
The information processing device according to claim 1, further comprising: The dialogue content display means extracts a predetermined keyword from the text information, emphasizes it, and outputs it.
The information processing device according to claim 5. expert document generation means for generating a predetermined expert document (for example, a medical record, etc.) based on the predetermined keyword extracted by the dialogue content display means;
The information processing device according to claim 6, further comprising: speech support means for transmitting information regarding the content of speech to be uttered by the expert to the terminal of the expert based on dialogue information of the user or the expert;
The information processing device according to claim 1, further comprising: Judgment support means for transmitting judgment information regarding the user to the terminal of the expert based on interaction information of the user or the expert;
The information processing device according to claim 1, further comprising: An information processing method executed by an information processing device that supports dialogue between a user and an expert via a communication network, the method comprising:
a user image transmitting step of transmitting the image to the expert's terminal in response to receiving the transmission of the image including the user's face from the user's terminal;
an expert image transmitting step of transmitting the image to the user's terminal in response to receiving the transmission of the image including the expert's face from the expert's terminal;
a user permission receiving step of receiving permission from the user's terminal to transmit an image of a specific part of the user's body with a predetermined quality to the expert's terminal;
a specific part image transmitting step of transmitting an image of the specific part to the terminal of the expert in response to receiving the permission;
Information processing methods including. A computer that controls an information processing device that supports dialogue between users and experts via a communication network,
a user image transmitting step of transmitting the image to the expert's terminal in response to receiving the transmission of the image including the user's face from the user's terminal;
an expert image transmitting step of transmitting the image to the user's terminal in response to receiving the transmission of the image including the expert's face from the expert's terminal;
a user permission receiving step of receiving permission from the user's terminal to transmit an image of a specific part of the user's body with a predetermined quality to the expert's terminal;
a specific part image transmitting step of transmitting an image of the specific part to the terminal of the expert in response to receiving the permission;
A program that executes control processing including.

Images