JP2018195164A - Analysis device, analysis program, and analysis method - Google Patents

Abstract

To provide an analysis device, an analysis program, and an analysis method for appropriately presenting an introduction destination satisfying a user.SOLUTION: An analysis device is configured to, when a first user (for example, a doctor) presents an introduction destination to a second user (for example, a patient), acquire image information acquired by photographing expressions or gestures of the second user for items (topics) presented by the first user or voice information issued by the second user, and to associate the items presented by the first user with a feeling level determined on the basis of the image information or voice information of the second user, and to analyze a satisfaction level of the second user for each item (specify the items satisfying or dissatisfying the second user) on the basis of a change in the feeling level of the second user of each item, and to extract candidates of the introduction destination having a high satisfaction level to be presented to the second user.SELECTED DRAWING: Figure 7

Description

  The present invention relates to an analysis apparatus, an analysis program, and an analysis method, and in particular, an analysis apparatus that appropriately presents an introduction destination that a user is satisfied based on the user's image information and audio information, an analysis program that operates on the analysis apparatus, and The present invention relates to an analysis method in an analysis apparatus.

  In recent years, there has been a tendency for the number of doctors engaged in local clinics to become scarce, and since local patients have become concentrated in core hospitals, the burden on doctors at core hospitals has increased and doctors have quit. A vicious cycle of going on has occurred. In response to this problem, in addition to securing doctors in the region, it has become important to realize the treatment of patients in an integrated manner by appropriately sharing the roles of the local clinics and the core hospitals.

  For example, depending on the disease and its severity, the local clinic, etc. should be used so that patients in the recovery phase and patients with mild illness are examined at a local clinic, etc. It is important to divide medical roles between the hospital and the core hospital, and in order to perform such role sharing, a proposal of a method for introducing appropriate medical facilities to patients is required.

  As a technique related to such medical facility introduction, for example, the following Patent Document 1 describes the reason for introduction when a medical institution introduces a patient to another medical institution, the institution of introduction and introduction destination, the date of introduction of the letter of introduction. And a storage unit that stores and stores patient information as one record, a counting unit that counts the number of records in the storage unit for each medical institution to which the referee is introduced, and the medical unit that receives the referral based on the counting result In the medical institution evaluation apparatus including an evaluation unit that assigns an evaluation point indicating whether the introduction is suitable for the introduction reason for each institution, the counting unit calculates the number of records with the same introduction reason as the introduction destination medical institution. The number of referrals counting means as the number of referrals, and a record in which the referral medical institution of the record counted as the number of referrals is the referral source medical institution, The number of records that coincides with the patient information of the counted records and that falls on the referral letter creation date within the first predetermined period from the referral letter creation date counted as the number of referrals is counted as a reintroduction correction value. There is disclosed a configuration having re-introduction correction value counting means, wherein the evaluation unit subtracts and corrects the number of introductions by the re-introduction correction value to give an evaluation point.

  Patent Document 2 below describes a function for creating an introduction letter that introduces a patient to another medical institution based on patient information for identifying the patient, the name of the patient's injury and illness, and self-information for identifying the own medical institution. And a medical institution that can be set as a referral destination, and a wound / disease / department storage unit that stores at least one name of a department that is suitable for the treatment of a wound Introduce medical institution information storage means that stores medical department names installed in the hospital in association with each medical institution, and medical institutions in which clinical departments corresponding to the names of wounds and illnesses for patients who are issued referral letters are installed A configuration having candidate display means for extracting and displaying as a previous candidate is disclosed.

  Moreover, although it is not a technique regarding introduction of a medical facility, regarding the technique for analyzing user satisfaction from the pronunciation of a conversation, Patent Document 3 below discloses a plurality of words extracted from the speech of a target conversation participant in the target conversation. A data acquisition unit that acquires data and a plurality of pronunciation time data indicating the utterance time of each word in the target conversation participant, and a polite expression or a non-poor expression from the plurality of word data acquired by the data acquisition unit A target conversation participation in a target conversation based on a plurality of specific word data extracted by the extraction unit and a plurality of utterance time data related to the plurality of specific word data Based on the detection result of the change point by the change detection unit and the change detection unit that detects the change point from the polite expression of the person to the non-poor expression. Dissatisfaction conversation determination device is disclosed having a determining dissatisfaction judging section that judges whether the conversation.

JP 2008-112390 A JP 2003-167955 A WO2014 / 069095

  When a clinic in the area introduces a patient at a clinic to a core hospital (clinic → core hospital) or because it is not possible to conduct a close examination or treatment for a disease, or during a recovery period at a core hospital When introducing a patient to a clinic (core hospital → clinic), in either case, the attending physician needs to select a medical facility to be introduced to the patient. In selecting the introduction destination, in Patent Document 1, the introduction destination is selected based on the past introduction results. Further, in Patent Document 2, a medical facility having a medical department capable of treating a patient's disease is extracted, and further notified as an introduction destination candidate in the order closer to the patient's living place (home, company).

  Here, when introducing a patient in recovery phase at a core hospital to a clinic (core hospital → clinic), the main purpose is to follow-up. The method of selecting a referral destination based on the above or selecting a facility that can be treated close to the patient's place of life does not cause significant problems.

  However, when a patient is referred to a core hospital because it cannot be examined or treated at the clinic (clinic office → core hospital), multiple core hospitals that can treat the target disease are treated by each hospital ( There is a difference in the quality of examination technology and treatment technology, and patient values vary widely (therapeutic results, evaluation from external institutions, gender of referral physicians, ease of attendance), and what viewpoint is emphasized Since the referral destination desired by the patient varies greatly depending on whether or not the user is satisfied, there is a problem that it is difficult to appropriately introduce the referral destination that the user is satisfied with.

  The present invention has been made in view of the above problems, and a main object of the present invention is to provide an analysis apparatus, an analysis program, and an analysis method capable of appropriately presenting an introduction destination satisfied by a user. .

  One aspect of the present invention is an analysis apparatus in which a first user presents an introduction destination to a second user, and a video input unit and / or audio input provided in the analysis apparatus or an apparatus capable of communicating with the analysis apparatus Information for acquiring image information of the face or upper body of the second user and / or voice information of the conversation of the second user, which represents a response of the second user to an item presented by the first user An acquisition unit, an item presented by the first user, and an emotion level determined based on the image information and / or audio information of the second user are associated with each other, and based on a change in the emotion level for each item An analysis unit that analyzes the satisfaction level of the second user for each of the items, and an output unit that extracts introduction destination candidates based on the satisfaction level of the second user and presents the candidates to the second user. Specially prepared To.

  One aspect of the present invention is an analysis program that operates on an analysis device in which a first user presents an introduction destination to a second user, and is provided in the analysis device or in a device that can communicate with the analysis device. Image information of the second user's face or upper body representing the reaction of the second user to the item presented by the first user from the video input unit and / or audio input unit, and / or the second Information acquisition processing for acquiring voice information of a user's conversation, an item presented by the first user and an emotion level determined based on the image information and / or voice information of the second user, Analysis processing for analyzing the satisfaction level of the second user with respect to each of the items based on the change in the emotion level, extracting introduction destination candidates based on the satisfaction level of the second user, and extracting the second user. Output process of presenting to The, characterized in that to the execution.

  One aspect of the present invention is an analysis method in a system including an analysis device in which a first user presents an introduction destination to a second user, and the analysis device is a device that can communicate with the analysis device or the analysis device. Image information of the face or upper body of the second user representing the reaction of the second user to an item presented by the first user from the provided video input unit and / or audio input unit, and / or the first An information acquisition process for acquiring voice information of a conversation between two users, an item presented by the first user, and an emotion level determined based on the image information and / or voice information of the second user, Based on the change in the emotion level for each item, an analysis process for analyzing the satisfaction level of the second user with respect to each of the items, and extracting introduction candidates based on the satisfaction level of the second user An output process of presenting the serial second user, characterized in that the run.

  According to the analysis apparatus, the analysis program, and the analysis method of the present invention, it is possible to appropriately present an introduction destination that the user is satisfied with.

  The reason is that the analysis device in which the first user presents the introduction destination to the second user obtains the item (topic) presented by the first user and the image information of the second user with respect to the item presented by the first user. And / or audio information is acquired, the item presented by the first user is associated with the emotion level determined based on the image information and / or audio information of the second user, and the emotion level is changed for each item. This is because the satisfaction level of the second user for each item is analyzed, and introduction destination candidates with high satisfaction levels are extracted and presented to the second user.

It is a schematic diagram which shows the structure of the introduction system which concerns on one Example of this invention. It is a schematic diagram which shows the other structure of the introduction system which concerns on one Example of this invention. It is a schematic diagram which shows the other structure of the introduction system which concerns on one Example of this invention. It is a schematic diagram which shows the other structure of the introduction system which concerns on one Example of this invention. It is a block diagram which shows the structure of the management apparatus which concerns on one Example of this invention. It is a block diagram which shows the structure of the analyzer which concerns on one Example of this invention. It is a schematic diagram which shows schematic operation | movement of the introduction system which concerns on one Example of this invention. It is a figure which shows an example of the output screen displayed on the display part of the analyzer which concerns on one Example of this invention. It is a flowchart figure which shows operation | movement of the analyzer which concerns on one Example of this invention. It is a flowchart figure which shows operation | movement of the analyzer which concerns on one Example of this invention. It is a flowchart figure which shows operation | movement of the management apparatus which concerns on one Example of this invention.

  As shown in the background art, it is important to divide the role of medical care between local clinics and core hospitals, and when introducing patients from the clinic to the core hospital (clinic office → core hospital) When introducing a patient from a core hospital to a clinic (core hospital → clinic), the attending physician needs to select a medical facility to introduce the patient.

  At the time of selection of this referral destination, methods of selecting a referral destination based on past referral results and selecting a facility that can be treated close to the patient's place of life have been proposed. When patients are referred to a core hospital because treatment cannot be performed (clinic office → core hospital), there are differences in the quality of medical care among multiple core hospitals that can treat the target disease. Since patient values vary, there is a problem that it is difficult to appropriately introduce a referral destination that the user is satisfied with.

  Such a problem is not limited to the case where a doctor introduces a medical facility to a patient. When a real estate agent introduces real estate to a customer, a securities agent introduces shares to a customer, a travel agent travels to the customer. In the case of introducing a plan, the same occurs when the first user introduces an arbitrary kind of object or information to the second user, such as when an insurance agent introduces insurance to a customer.

  Therefore, in one embodiment of the present invention, when a first user (for example, a doctor) presents an introduction destination to a second user (for example, a patient), a second item for an item (topic) presented by the first user. The image information obtained by photographing the user's facial expression and gesture and the sound information pronounced by the second user are acquired, and the items presented by the first user and the emotion level determined based on the image information and sound information of the second user are obtained. Based on the association and the change in the emotion level of the second user for each item, the satisfaction level of the second user for each item is analyzed (identified items that the second user is satisfied with or feels dissatisfied with) Then, introduction candidate candidates with high satisfaction are extracted and presented to the second user. Hereinafter, a case where a doctor introduces a medical facility to a patient will be described as an example.

  In order to describe the above-described embodiment of the present invention in more detail, an analysis apparatus, an analysis program, and an analysis method according to an embodiment of the present invention will be described with reference to FIGS. 1 to 4 are schematic diagrams showing the configuration of the introduction system of the present embodiment, FIG. 5 is a block diagram showing the configuration of the management device of the present embodiment, and FIG. 6 is the analysis device of the present embodiment. It is a block diagram which shows a structure. FIG. 7 is a schematic diagram illustrating a schematic operation of the introduction system according to the present embodiment. FIG. 8 is a diagram illustrating an example of an introduction screen displayed on the display unit of the analysis apparatus according to the present embodiment. FIGS. 9 and 10 are flowcharts illustrating the operation of the analysis apparatus according to the present embodiment. FIG. 11 is a flowchart illustrating the operation of the management apparatus according to the present embodiment.

  As shown in FIG. 1, the introduction system 10 of this embodiment refers to a DB (Data Base), acquires a management device 20 that determines a user's emotion level, and acquires and manages user image information and audio information. In addition to providing to the device 20, the emotion level for each item acquired from the management device 20 is analyzed, and the analysis device 30 or the like that introduces a medical facility that the user is satisfied with. These are connected via a communication network 40 such as a LAN (Local Area Network) or a WAN (Wide Area Network) defined by standards such as Ethernet (registered trademark), token ring, and FDDI (Fiber-Distributed Data Interface). .

  In FIG. 1, the introduction system 10 includes the management device 20 and the analysis device 30. However, as illustrated in FIG. 2, the introduction system 10 includes an input device 50 (video input described later) that can communicate with the analysis device 30. Short-distance wireless communication such as wired, wireless LAN, Bluetooth (registered trademark) or NFC (Near Field Communication), etc. It is good also as a structure which transmits to the analysis apparatus 30 using.

  In FIG. 1 and FIG. 2, the introduction system 10 includes the management device 20 including the DB. However, when the storage unit of the analysis device 30 includes the DB, as illustrated in FIG. 30 or the analysis device 30 and the input device 50.

  Further, in FIGS. 1 to 3, a first user (referred to as a doctor in this embodiment) and a second user (referred to as a patient in this embodiment) face each other and have a conversation in front of the analysis device 30. As shown in FIG. 4, a client device 51 used by the second user (patient) is provided separately from the analysis device 30 used by the first user (doctor), and the analysis device 30 and the client device 51 are provided. May be connected via the communication network 40 so that video and audio can be bidirectionally communicated in real time. Hereinafter, each device will be described in detail on the assumption of the configuration of FIG.

[Management device]
The management device 20 is a computer device or server that compares the information transmitted from the analysis device 30 with the information stored in the DB to determine the emotion level of the second user, as shown in FIG. The control unit 21, the storage unit 22, the network I / F unit 23, etc.

  The control unit 21 includes a CPU (Central Processing Unit) configured by a microprocessor and the like, and a memory such as a ROM (Read Only Memory) and a RAM (Random Access Memory), and the CPU is stored in the ROM or the storage unit 22. The operation of the entire management apparatus 20 is controlled by developing and executing the control program on the RAM.

  The storage unit 22 includes an HDD (Hard Disk Drive), an SSD (Solid State Drive), and the like, a program for the CPU to control each unit, and DB (representative image information of each emotion level (satisfaction to dissatisfaction)) Voice information) is stored.

  The network I / F unit 23 includes a NIC (Network Interface Card), a modem, and the like, connects the management device 20 to the communication network 40, receives image information and audio information from the analysis device 30, and also analyzes the analysis device 30. Send emotion level judgment results to

  Further, the control unit 21 functions as a level determination unit 21a and the like as shown in FIG. When the level determination unit 21a receives image information of the second user's face (upper body if a gesture is also used as a determination element) from the analysis device 30, the level determination unit 21a receives the received face or upper body image information and the DB of the storage unit 22 Compared with the representative image information of the face or upper body of each emotion level stored in, using known pattern recognition technology, for example, eyebrows and eye corners, the angle of the mouth, the degree of opening of the eyes and mouth, By calculating the matching rate such as the degree of facial depression, it is determined to which emotion level the received image information belongs, and the determination result (emotion level) is transmitted to the analysis device 30. Moreover, when the voice information pronounced by the second user is received from the analysis device 30, the received voice information is compared with representative voice information of each emotion level stored in the DB of the storage unit 22. Use pattern recognition technology to determine the emotion level to which the received voice information belongs by calculating the match rate of voice strength, height, inflection, conversation speed, conversation pause, etc. (Emotion level) is transmitted to the analysis device 30. The determination of the emotion level may be performed based on either image information or audio information, or may be performed based on both image information and audio information.

[Analyzer]
The analysis device 30 is a personal computer, a tablet terminal, or the like that the first user uses while facing the second user. As shown in FIG. 6A, the control unit 31, the storage unit 32, the network I / F unit 33, a video input unit 34, an audio input unit 35, a display unit 36, an operation unit 37, and the like.

  The control unit 31 includes a CPU constituted by a microprocessor or the like and a memory such as a ROM or a RAM, and the CPU analyzes the control program stored in the ROM or the storage unit 32 by developing the program in the RAM and executing it. The operation of the entire apparatus 30 is controlled.

  The storage unit 32 is configured by an HDD, an SSD, or the like, and includes a program for the CPU to control each unit, information on processing functions of the own device, image information acquired by the video input unit 34, and audio information acquired by the audio input unit 35. Information on medical facilities, DB (representative image information and voice information of each emotion level (satisfaction to dissatisfaction)) and the like are stored as necessary.

  The network I / F unit 33 is configured by a NIC, a modem, and the like, connects the analysis device 30 to the communication network 40, transmits image information and / or voice information of the second user to the management device 20, and the management device 20 Receives emotion level judgment results from. As shown in FIG. 2, when the input device 50 is separately provided, the network I / F unit 33 further performs short-range wireless communication such as wired, wireless LAN, Bluetooth (registered trademark), NFC, etc. The image information and / or audio information of the second user is received from the input device 50.

  The video input unit 34 is configured by a CCD (Charge Coupled Devices), a CMOS (Complementary Metal Oxide Semiconductor) camera, or the like, and the face or upper body of the second user (patient) at regular intervals (for example, every second) or continuously. To obtain image information (still image or moving image).

  The voice input unit 35 includes a microphone and records voice of the second user (patient) to acquire voice information. In addition, when specifying an item based on the voice of the first user (doctor), the voice information of the first user (doctor) and the second user (patient) is acquired. At that time, the voice of the first user and the voice of the second user may be discriminated based on, for example, the intensity or height of the voice, or two voice input sections 35 having directivity are provided, By installing the voice input unit 35 toward the first user and the second user, the voice of the first user and the voice of the second user may be discriminated.

  The display unit 36 is composed of a liquid crystal display (LCD), an organic EL (electroluminescence) display device, or the like, and displays an introduction screen for introducing a medical facility to the second user by the first user.

  The operation unit 37 includes a mouse, a keyboard, a touch sensor in which electrodes formed on the display unit 36 are arranged in a grid, and can perform various operations related to the analysis of the present embodiment and various operations on the introduction screen. To.

  Further, as shown in FIG. 6B, the control unit 31 also functions as a timing unit 31a, an information acquisition unit 31b, an analysis unit 31c, an output unit 31d, and the like.

  The timer 31a measures time based on the CPU clock and the like. Specifically, the time when the video input unit 34 acquires image information, the time when the audio input unit 35 acquires audio information, and the like are measured.

  The information acquisition unit 31b receives from the video input unit 34 and / or the audio input unit 35 the image information of the second user and / or the response of the second user (patient) to the items presented by the first user (doctor). Get audio information. Specifically, at predetermined time intervals (for example, every second), image information obtained by photographing the face or upper body of the second user (patient) is acquired from the video input unit 34 and the voice input unit 35 receives the first information. Audio information obtained by recording voices of two users (patients) for a certain period of time (and voice information obtained by recording voices of the first user (doctor) for a certain period of time if necessary) is acquired, and image information of the second user (patient) is obtained. And / or audio | voice information is transmitted to the management apparatus 20, and determination of an emotion level is instruct | indicated. At that time, the information acquisition unit 31b patterns the image information and / or the sound information using a known method as necessary so as to facilitate the determination of the emotion level in the management device 20. Image information (for example, image information that makes it possible to identify the eyebrows, the corners of the eyes, the angle of the mouth, the degree of opening of the eyes and the mouth, the state of depression of the face, etc.) and / or the patterned audio information (for example, the strength or weakness of the voice, Audio information that enables identification of inflection, conversation speed, conversation pause, etc.) may be transmitted to the management device 20.

  The analysis unit 31c receives the emotion level determination result from the management device 20, and associates the emotion level corresponding to each piece of image information and / or audio information with the time measured by the time measuring unit 31a. Further, using a known voice recognition technology, an item presented by the first user is identified from the voice information of the first user (doctor), and the identified item is associated with the time measured by the timer unit 31a. Then, based on the time measured by the time measuring unit 31a, the emotion level received from the management device 20 is associated with the item specified from the voice information of the first user (doctor), to which item the second user (patient ) The second user (patient) 's satisfaction with each item is analyzed based on how the emotion of () has changed, and the second user is satisfied (high emotion level) and the second user Identify items that are dissatisfied (low emotion level).

  The items presented by the first user are, for example, items related to the treatment results of the target disease of the second user, items related to the number of treatments or treatments of the target disease of the second user, items related to the gender of the referring doctor, and the like. The treatment results are, for example, the average number of days of hospitalization after operating the target disease of the second user, the success rate of treatment of the target disease of the second user, after a predetermined number of years after operating the target disease of the second user Such as the survival rate. Further, when the storage unit 32 of the analysis apparatus 30 includes a DB, the analysis unit 31c functions as a level determination unit, and image information and / or audio information acquired by the information acquisition unit 31b and each stored in the DB. The emotion level may be determined using a known pattern recognition technique by comparing typical image information and / or voice information of the emotion level. Further, the emotion level may be relatively determined based on temporal changes (changes from the initial state) of the image information and / or audio information acquired by the information acquisition unit 31b.

  Based on the analysis result of the analysis unit 31c, the output unit 31d extracts a medical facility candidate with a high degree of satisfaction of the second user (patient) from the medical facilities stored in the storage unit 32, and selects the extracted candidate An introductory screen to be enabled is displayed on the display unit 36. Specifically, based on items that the second user (patient) is satisfied with (high emotional level), medical facility candidates are extracted, or the second user (patient) feels dissatisfied ( Or the extracted candidates are excluded based on items with a low emotion level. Then, an introduction screen describing that one or a plurality of medical facility candidates can be selected is created and displayed on the display unit 36.

  The timing unit 31a, the information acquisition unit 31b, the analysis unit 31c, and the output unit 31d may be configured as hardware, and the control unit 31 may be configured as the timing unit 31a, the information acquisition unit 31b, the analysis unit 31c, and the output unit 31d. The analysis program may be configured to function, and the analysis program may be executed by the CPU of the control unit 31.

  1 to 6 are examples of the introduction system 10, the management device 20, and the analysis device 30 of the present embodiment, and the configuration and control thereof can be changed as appropriate. For example, in the above description, the introduction screen is displayed on the display unit 36 and the introduction destination is notified. However, the introduction destination may be notified by voice. In this case, the analysis apparatus 30 may be provided with a voice output unit such as a speaker. May be provided.

  The schematic operation of the introduction system configured as described above will be described below with reference to FIG. Here, the facial expression and gesture of the second user are photographed at regular intervals (t0, t1, t2,...) (Time t0: image 0, time t1: image 1, time t2: image 2). ...). And the change of an emotion level of a 2nd user is always measured in conversation.

  First, as shown in FIG. 7A, the image 0 (second user's facial expression) taken at time t0 is transmitted to the management device 20, and the emotion level to which the image 0 belongs is managed by the management device 20 (DB). Contact The DB compares the characteristics of the received image 0 (the second user's facial expression) with the characteristics of the image groups of each emotion level (level 2 to level-2) stored in advance, thereby receiving the received image. Which emotion level 0 (facial expression of the second user) belongs to is determined using a known pattern recognition technique, and the determination result is transmitted to the analysis device 30. Here, image 0 at time t0 is determined to be level 0.

  And as shown in FIG.7 (b), the said process is repeated for every fixed time, and by matching the emotion level received from the management apparatus 20 with the item specified from the audio | voice information of the 1st user (doctor), which conversation At this time, it is possible to read how the emotion of the second user (patient) has changed. Here, the second user (patient) judges that he / she is satisfied with the treatment result of the transfer destination because the emotional level has risen when talking about the treatment result in the hospital candidate of the transfer destination during the transfer. be able to. In addition, since the emotional level is lowered when talking about the hospital visit time in the hospital candidate of the transfer destination, it can be determined that the patient feels a little dissatisfied with the hospital visit time. Further, since the emotional level is the normal level when talking to the referral doctor in the hospital candidate of the transfer destination, it can be determined that the gender (male, female) viewpoint of the doctor is not emphasized.

  Note that the predetermined time can be adjusted on the setting screen (not shown) of the analysis device 30 or the input device 50. Here, the emotion level is determined based on the image information of the second user (patient), but the emotion level is determined based on the audio information instead of or in addition to the image information. Also good.

  FIG. 8 is an example of an introduction screen 60 showing the result of searching for a hospital based on the above analysis result. The analysis device 30 analyzes the second user's (patient) 's facial expression, gesture (image information), and conversation (voice information). As a result, the treatment results and the number of treatments for the target disease are important for selecting a hospital. The introduction screen 60 of the analysis device 30 has several viewpoints (here, two hospitals each) from the viewpoint of a hospital with good treatment results and a hospital with a large number of treatments for the target disease. Extract candidates and display them. Note that the number of each extraction can be arbitrarily set on the analysis device 30 side.

  As described above, the analysis device 30 transmits the image information and voice information of the second user (patient) to the management device 20 to instruct the determination of the emotion level, and the management device 20 makes the determination using the pattern recognition technology. It is possible to recognize which item is satisfied or dissatisfied by receiving the emotion level and associating the received emotion level with the item specified based on the voice information of the first user (doctor). The candidate (hospital) desired by the second user (patient) can be introduced appropriately.

  In the above, the image information and audio information of the second user (patient) acquired by the analysis device 30 are transmitted to the management device 20 as they are, but the configuration of the image information (which part of the second user is captured at which angle) If the image or the like changes, it may be difficult for the management device 20 to recognize the pattern. In such a case, the analysis device 30 may extract a feature part (for example, a face outline, eyes, eyebrows, lips, etc.) from the image information, and transmit the feature part information to the management device 20. Alternatively, patterned image information (for example, image information that can identify the eyebrows, the corners of the eyes, the angle of the mouth, the degree of opening of the eyes and the mouth, etc.) may be transmitted to the management device 20. In addition, if the configuration of the audio information (such as whether the voice of another user is mixed in the voice of the second user) changes, pattern recognition in the management device 20 may become difficult. In such a case, the analysis device 30 compares the second user's image information and audio information, extracts only the second user's audio by referring to the second user's image information, and extracts the extracted audio. The information may be transmitted to the management device 20, and in order to facilitate pattern recognition in the management device 20, it is possible to identify patterned voice information (for example, voice strength, level, inflection, etc.). Audio information) may be transmitted to the management apparatus 20.

  In addition, in extracting requests from the second user, it is possible to extract items that seem to be highly satisfactory in terms of important items that the second user himself does not know, rather than extracting based on the consciousness of the second user. May be.

  Hereinafter, the operation of the analysis device 30 will be described with reference to FIGS. 9 and 10. The CPU of the analysis device 30 executes the processing of each step shown in the flowcharts of FIGS. 9 and 10 by developing the analysis program stored in the ROM or the storage unit 32 in the RAM and executing it. In the following description, it is assumed that the analysis apparatus 30 includes the video input unit 34 and the audio input unit 35.

[Overall processing]
As shown in FIG. 9, first, the control unit 31 of the analysis device 30 determines whether or not the start of use of the analysis function is instructed (S101), and when the start of use of the analysis function is instructed (Yes in S101), the analysis is performed. Processing is executed (S102).

  FIG. 10 shows the details of this step. First, the controller 31 (timer 31a) acquires the current time (S201), and when a predetermined time has elapsed (Yes in S202), the controller 31 (information acquisition unit 31b) acquires image information of the second user (patient) from the video input unit 34 (S203), and voices of the first user (doctor) and the second user (patient) from the audio input unit 35. Information is acquired (S204).

  Next, the control unit 31 (information acquisition unit 31b) transmits the acquired image information and audio information of the second user (patient) to the management device 20, and the emotion of the second user (patient) to the management device 20. The level is inquired (S205).

  Next, if the control part 31 (analysis part 31c) receives the determination result of the emotion level from the management apparatus 20 (Yes of S206), the emotion level received from the management apparatus 20 and the present time which the time measuring part 31a measured will be shown. The associated information is stored in the storage unit 32 or the like (S207). Further, the control unit 31 (analysis unit 31c) identifies the item by analyzing the voice information of the first user, associates the identified item with the current time measured by the time measuring unit 31a, and stores the associated item in the storage unit 32 or the like. (S208). In addition, according to operation of the operation part 37 of a 1st user, you may make it display each item in order on the display part 36, In that case, voice information of a 1st user (doctor) is acquired by S204. There is no need, and in S208, the item and the operation time may be associated with each other.

  Then, the control unit 31 (analysis unit 31c) determines whether or not the analysis function end operation has been performed (S209). If the end operation has not been performed, the control unit 31 returns to S201 and repeats the same processing. On the other hand, when the end operation is performed (Yes in S209), the control unit 31 (analysis unit 31c) determines from the items identified from the voice information of the first user and the management device 20 based on the time measured by the time measuring unit 31a. The received emotion level is associated (S210), and the item that the second user is satisfied with and the item that is dissatisfied with are specified from the change in the emotion level for each item (S211).

  Returning to FIG. 9, the control unit 31 (output unit 31 d) determines whether or not the start of use of the introduction function is instructed (S <b> 103), and when the start of use of the introduction function is not instructed (No in S <b> 103), Returning to S102, the analysis process is continued.

  When the start of use of the introduction function is instructed (Yes in S103), the control unit 31 (output unit 31d) refers to the information on the medical facility stored in the storage unit 32 and satisfies the user's request at the current facility. Items that are not to be extracted are extracted (S104), and it is confirmed whether there is another facility that can satisfy the extracted items (S105).

  As a result of the confirmation, if there is another facility that can satisfy the extracted item (Yes in S106), the control unit 31 (output unit 31d) is displayed on the display unit 36 for the second user rather than the current facility. Information indicating that there is a good facility and excellent locations of the facility is displayed (S107). On the other hand, when there is no other facility that can satisfy the extracted item (No in S106), the control unit 31 (output unit 31d) has a better facility for the user than the current facility on the display unit 36. Information not to be displayed is displayed (S108).

  Next, the operation of the management device 20 (emotion level determination process) will be described with reference to FIG.

  The control unit 21 of the management device 20 stores the image information and audio information of the second user received from the analysis device 30 in the storage unit 22 (DB) (S301).

  Next, the control unit 21 (level determination unit 21a) compares the received image information and audio information of the second user with image information and audio information stored in advance in the DB, and uses a known pattern recognition technique. Then, it is determined to which level the emotion of the second user belongs (S302). Then, the determination result (emotion level) is transmitted to the analysis device 30 (S303).

  In addition, this invention is not limited to description of the said Example, The structure and control can be suitably changed unless it deviates from the meaning of this invention.

  For example, in the above embodiment, the case where the emotion level is determined by comparing the image information and audio information stored in the DB with the image information and audio information of the second user (patient) is described. The emotion level may be relatively determined based on changes in the image information and audio information of the patient) from the initial state.

  Moreover, in the said Example, although the case where image information and audio | voice information of a 2nd user (patient) were acquired was described, the information to acquire is the 2nd user (patient) with respect to the item which a 1st user (doctor) presents. For example, the analysis method of the present invention can be similarly applied to the case of acquiring biological information such as body temperature, pulse, and sweating.

  Moreover, in the said Example, although the audio | voice information of the 1st user (doctor) was analyzed and the item was specified, according to operation (for example, mouse click operation) of the operation part 37 of the 1st user (doctor), Each item may be displayed in order on the display unit 36. In that case, it is not necessary to acquire the voice information of the first user (doctor), and the item and the operation time may be associated with each other.

  Moreover, in the said Example, although the case where a doctor introduces a medical facility to a patient based on a patient's image information and audio | voice information was described, the 1st user gave arbitrary types of things and information to a 2nd user. The analysis method of the present invention can be similarly applied to the case of introducing.

  The present invention relates to an analysis device that presents an introduction destination that a user is satisfied based on image information and audio information of the user, an analysis program that operates in the analysis device, a recording medium that records the analysis program, and an analysis method in the analysis device Is available.

DESCRIPTION OF SYMBOLS 10 Introduction system 20 Management apparatus 21 Control part 21a Level determination part 22 Storage part 23 Network I / F part 30 Analysis apparatus 31 Control part 31a Timekeeping part 31b Information acquisition part 31c Analysis part 31d Output part 32 Storage part 33 Network I / F part 34 Video Input Unit 35 Audio Input Unit 36 Display Unit 37 Operation Unit 40 Communication Network 50 Input Device 51 Client Device 60 Introduction Screen

Claims (21)

An analysis device in which a first user presents an introduction destination to a second user,
The second user's response to the second user's response to an item presented by the first user from the video input unit and / or the voice input unit provided in the analysis device or a device communicable with the analysis device. An information acquisition unit for acquiring image information of the face or upper body and / or audio information of the conversation of the second user;
The item presented by the first user is associated with the emotion level determined based on the image information and / or audio information of the second user, and each of the items is determined based on the change in the emotion level for each item. An analysis unit for analyzing satisfaction of the second user with respect to an item;
An output unit that extracts introduction destination candidates based on the satisfaction degree of the second user and presents it to the second user;
An analysis device characterized by that. The information acquisition unit acquires voice information of the first user,
The analysis unit analyzes the first user's voice information to identify the item;
The analysis apparatus according to claim 1, wherein: The analysis unit associates the time when the image information and / or sound information of the second user is acquired with the emotion level determined based on the image information and / or sound information, and the first user selects the item. Associating the presented time with the item, and associating the item with the emotion level based on the time,
The analysis apparatus according to claim 1 or 2, characterized in that: The information acquisition unit transmits the acquired image information and / or audio information of the second user to an apparatus including a database that stores image information and / or audio information for each emotion level,
The analysis unit receives the emotion level determined by comparing the image information and / or voice information of the second user with the image information and / or voice information for each emotion level from the device.
The analysis device according to claim 1, wherein The analysis unit compares the image information and / or audio information of the second user with the image information and / or audio information for each emotion level stored in the storage unit in advance, or the image of the second user Determining the emotion level corresponding to the image information and / or audio information of the second user based on a change from the initial state of the information and / or audio information;
The analysis device according to claim 1, wherein The output unit displays on the display unit an introduction screen that enables selection of the extracted introduction destination.
The analysis apparatus according to claim 1, wherein The referral destination is a medical facility,
The item presented by the first user is selected from the items related to the treatment results of the target disease of the second user, the items related to the number of treatments or the number of treatments of the target disease of the second user, and the items related to the sex of the referring doctor Including one to be
The analysis apparatus according to claim 1, wherein An analysis program that operates on an analysis device in which a first user presents an introduction destination to a second user,
In the analysis device,
The second user's response to the second user's response to an item presented by the first user from the video input unit and / or the voice input unit provided in the analysis device or a device communicable with the analysis device. Information acquisition processing for acquiring image information of the face or upper body and / or voice information of the conversation of the second user;
The item presented by the first user is associated with the emotion level determined based on the image information and / or audio information of the second user, and each of the items is determined based on the change in the emotion level for each item. Analysis processing for analyzing satisfaction of the second user with respect to an item;
An output process for extracting introduction candidates and presenting them to the second user based on the satisfaction degree of the second user;
An analysis program characterized by that. In the information acquisition process, the voice information of the first user is acquired,
In the analysis process, the item is identified by analyzing the voice information of the first user.
The analysis program according to claim 8. In the analysis process, the time when the image information and / or audio information of the second user is acquired is associated with the emotion level determined based on the image information and / or audio information, and the item is selected by the first user. Associating the presented time with the item, and associating the item with the emotion level based on the time,
The analysis program according to claim 8 or 9, characterized in that. In the information acquisition process, the acquired image information and / or audio information of the second user is transmitted to an apparatus including a database that stores image information and / or audio information for each emotion level,
In the analysis process, the emotion level determined by comparing the image information and / or voice information of the second user with the image information and / or voice information for each emotion level is received from the device.
The analysis program according to any one of claims 8 to 10, wherein: In the analysis process, the image information and / or sound information of the second user is compared with the image information and / or sound information for each emotion level stored in the storage unit in advance, or the image of the second user Determining the emotion level corresponding to the image information and / or audio information of the second user based on a change from the initial state of the information and / or audio information;
The analysis program according to any one of claims 8 to 10, wherein: In the output process, an introduction screen that enables selection of the extracted introduction destination is displayed on the display unit.
The analysis program according to any one of claims 8 to 12. The referral destination is a medical facility,
The item presented by the first user is selected from the items related to the treatment results of the target disease of the second user, the items related to the number of treatments or the number of treatments of the target disease of the second user, and the items related to the sex of the referring doctor Including one to be
The analysis program according to any one of claims 8 to 13. An analysis method in a system including an analysis device in which a first user presents an introduction destination to a second user,
The analysis device includes:
The second user's response to the second user's response to an item presented by the first user from the video input unit and / or the voice input unit provided in the analysis device or a device communicable with the analysis device. Information acquisition processing for acquiring image information of the face or upper body and / or voice information of the conversation of the second user;
The item presented by the first user is associated with the emotion level determined based on the image information and / or audio information of the second user, and each of the items is determined based on the change in the emotion level for each item. An analysis process for analyzing the degree of satisfaction of the second user with respect to an item;
An output process of extracting introduction destination candidates based on the satisfaction degree of the second user and presenting the candidate to the second user;
An analysis method characterized by that. In the information acquisition process, the voice information of the first user is acquired,
In the analysis process, the item is identified by analyzing the voice information of the first user.
The analysis method according to claim 15. In the analysis process, the analysis process associates the time when the image information and / or sound information of the second user is acquired with the emotion level determined based on the image information and / or sound information, and Associating the time when the user presented the item with the item, and associating the item with the emotion level based on the time;
The analysis method according to claim 15 or 16, wherein: The system includes an apparatus including a database storing image information and / or audio information for each emotion level,
In the information acquisition process, the acquired image information and / or audio information of the second user is transmitted to the device,
In the analysis process, the emotion level determined by comparing the image information and / or voice information of the second user with the image information and / or voice information for each emotion level is received from the device.
The analysis method according to any one of claims 15 to 17, characterized in that: In the analysis process, the image information and / or sound information of the second user is compared with the image information and / or sound information for each emotion level stored in the storage unit in advance, or the image of the second user Determining the emotion level corresponding to the image information and / or audio information of the second user based on a change from the initial state of the information and / or audio information;
The analysis method according to any one of claims 15 to 17, characterized in that: In the output process, an introduction screen that enables selection of the extracted introduction destination is displayed on the display unit.
The analysis method according to any one of claims 15 to 19, wherein The referral destination is a medical facility,
The item presented by the first user is selected from the items related to the treatment results of the target disease of the second user, the items related to the number of treatments or the number of treatments of the target disease of the second user, and the items related to the sex of the referring doctor Including one to be
The analysis method according to any one of claims 15 to 20, wherein:

Images