JP2016202798A - Computer system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a computer system capable of meeting needs of users for grasping a state of their own biological part.SOLUTION: A computer system is configured to: search, when a user inquires about an intracorporeal part (YES in S66), for data on the intracorporeal part of an inquiry target (S67, S68) from a user DB; determine speech contents on the basis of the searched data (S69); and transmit speech data of the determined speech contents to a wearable computer of the user.SELECTED DRAWING: Figure 6

Description

  The present invention relates to a computer system for collecting and utilizing user biometric data acquired by a biometric sensor.

  With regard to this type of computer system, for example, the server collects the biological information of the user, determines an exercise program to be performed by the user based on the biological information, transmits it to the user terminal, and information on the exercise to be performed by the user Is presented to the user (see Patent Document 1).

JP2009-213528

  However, in such a computer system, an action such as exercise to be performed by the user is determined based on the collected biometric information of the user and is only fed back to the user. On the other hand, for users, there is a need for grasping the status of a living body part such as their heart and liver, and there is a drawback that it is impossible to meet the needs.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a computer system that can meet the needs of users who want to grasp the status of their own body parts.

  In the following, an example of an embodiment corresponding to each means for solving the problem is shown in parentheses.

The present invention collects user biometric data (for example, heart rate, body temperature, blood pressure, uric acid value, γGTP, acetone in body odor, etc.) acquired by a biosensor (for example, various biosensors 2, external sensor group 5). A computer system (for example, server 7, dialog DB 11, user DB 12, pattern DB 13, rule DB 14)
Biometric data storage means (for example, S23, user DB12) for storing the collected biometric data;
Communication means (for example, S39 to S46) for performing control for realizing communication between the living body part and the user based on the state of the living body part that can be discriminated by the biological data stored by the biological data storage means; With
The communication means is:
Natural language recognition means (for example, S50 to S54);
Natural languageization means (for example, S42, dialogue DB11) for expressing the state of the living body part in natural language;
Transmission means (for example, S43) for transmitting the natural language expressed by the natural languageization means to the user,
When there is an inquiry in a natural language from a user, the natural language recognition unit recognizes the inquiry, and the natural language expressing the state of the living body part corresponding to the recognized content is transmitted. The means communicates to the user to answer the question (for example, utterance content for the question is determined in S66 to S69, utterance data corresponding to the utterance content is generated in S42, and transmitted to the user in S43). system.

Also, action data storage means (for example, S64, S65, user DB12) for collecting and storing user action data that affects the living body part,
Based on the user behavior data stored by the behavior data storage means and the biological data stored by the biological data storage means, a correlation finding means for finding a correlation between the two (for example, The machine learning process of FIG. 8A may be further provided.

Further, the biometric data storage means and the behavior data storage means collect and store data from a plurality of users (for example, FIG. 1, storing action data and sensor detection data for each user ID),
The correlation finding means may find a correlation by data mining from a data set including the behavior data and the biological data for a plurality of users (for example, S82 and S86).

In addition, for calculating a user performing an action corresponding to the action data of the new correlation found by the correlation finding means from the action data of the user accumulated by the action data accumulating means Applicable user indexing means (for example, S93, user DB12),
You may further provide the message transmission means (for example, S94, S96) for transmitting a predetermined message with respect to the user indexed by the said user indexing means.

Further, the natural language recognition means includes voice recognition means (for example, S50 to S54) for recognizing the user's voice,
The transmission means includes voice transmission means (for example, S43) for transmitting the natural language expressed by the natural languageization means to the user by voice,
The communication means is a natural language in which the voice recognition means recognizes the question when the user asks a voice question, and the natural language means expresses the state of the living body part corresponding to the recognized content. Is transmitted to the user by voice so that the question is answered by voice (for example, utterance content for the question is determined in S66 to S69, utterance data corresponding to the utterance content is generated in S42, and S43 To the user).

  According to the present invention, the user communicates with the living body part based on the state of the living body part that can be discriminated by the living body data stored by the living body data storage means, thereby determining the state of the living body part. I can grasp it.

It is a functional block diagram which shows the whole health service system. It is a block diagram which shows the control circuit of a wearable computer and the various biological sensors connected to it, and an external sensor group. It is a block diagram which shows the control circuit of a server. (A) is a flowchart which shows the main routine of a wearable computer and a server, (b) is a flowchart which shows the subroutine program of a data transmission process and a data collection process. (A) is a flowchart which shows the subroutine program of the dialogue process of a wearable computer and a server, (b) is a flowchart which shows the subroutine program of a speech recognition process. (A) is a flowchart showing a subroutine program of corresponding processing according to the recognized result, (b) is a flowchart showing a subroutine program of message generation processing, and (c) is a flowchart showing a subroutine program of external sensor processing. (D) is a flowchart showing a subroutine program for imaging processing. It is a screen figure displayed on the display part of a wearable computer. (A) is a flowchart which shows the subroutine program of a machine learning process, (b) is a flowchart which shows the subroutine program of the dialogue process based on a machine learning result. (A) is a figure which shows the data memorize | stored in pattern DB, (b) is a figure which shows the data memorize | stored in rule DB. (A) is a flowchart showing a subroutine program for rule processing, (b) is a flowchart showing a subroutine program for effect verification processing, and (c) is a flowchart showing other corresponding subroutine programs.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. First, referring to the entire system of FIG. 1, a user 4 is wearing a wearable computer 3 including a smart glass and the like, and various biological sensors 2 for detecting the living body of the user 4 are connected to the wearable computer 3. ing. The detection data (biological data) of the various sensors 2 is transferred to the server 7 via the wearable computer 3 and the Internet 1, and the server 7 detects various sensors in a user database (hereinafter referred to as "DB") 12. Store the data. For example, when the user 4 approaches the external sensor group 5 including the urine sensor 59 and the weight sensor 60 provided in the toilet and the biometric data of the user 4 is detected by the external sensor group 5, the wearable computer 3 of the user 4 and the external The sensor group 5 communicates, and sensor data from the external sensor group 5 is transmitted to the server 7 via the wearable computer 3 and the Internet 1.

  When the user 4 sleeps in a bed or the like, the sleep sensor 6 installed in the bed or the like detects the user's sleep state, and the detection data is transmitted to the server 7 via the Internet 1. Furthermore, a health food manufacturer 8 such as a supplement (hereinafter referred to as “supplement”) is provided with a server 8a and a product development DB 8b, and the server 8a is connected to the Internet 1 and connected to the server 7 of the health service provider 10. It is configured to be able to communicate. The product development DB 8b stores product data such as developed supplements and data on products under development. In addition, the pharmaceutical manufacturer 9 is provided with a server 9a and a product development DB 9b. The server 9a is connected to the Internet 1 and configured to be able to communicate with the server 7 of the health service provider 10. In the product development DB 9b, product data of developed pharmaceuticals and the like and data of products under development are accumulated.

  The health service provider 10 is provided with the server 7 described above, and a dialogue DB 11, a user DB 12, a pattern DB 13, and a rule DB 14 are connected to the server 7. Also, actions that affect the living body of the user 4, for example, user action data such as meals, taking medicines, taking supplements, etc. are transmitted to the server 7 via the wearable computer 3 and the Internet 1. . For example, when the user eats a meal, the meal is captured by the camera input unit 29 of the wearable computer 3 and transmitted to the server 7. In the server 7, ingredients (for example, salad, beef, bread) eaten by the user 4 are determined based on the imaging data and stored in the user DB 12. Also, if the user 4 says “take cold medicine” when taking a medicine, the voice is transferred to the server 7 via the wearable computer 3 and the Internet 1, the voice is recognized by the server 7, and the “medicine” is stored in the user DB 12. : “Cold medicine”. Further, when the user 4 ingests the supplement, if the user says “turmeric intake”, the voice is transmitted to the server 7 via the wearable computer 3 and the Internet 1, and the server 7 recognizes the voice, and the user DB 12 receives “supple: turmeric”. "Ingestion" is memorized.

The health service provider 10 assigns a user ID to each user who is a member of the health service providing system, and the sensor detection data and action data transmitted from a large number of users 4 for each user ID. The data is classified and stored in the user DB 12. The sensor detection data and action data change with time t. For example, the sensor detection data and action data are stored every minute or every second. In FIG. 1, the date and time of the user ID: 051392704, April 9, 2015, 7:30:15,
(Action data)
Meals: Salad + Beef + Bread medicine: Cold medicine supplements: Turmeric intake (sensor detection data)
Heart rate sensor: 75
Body temperature sensor: 36.7
Blood pressure sensor: 140-75
Urine sensor: PH6, uric acid 8.2
Blood sensor: γGTP153, neutral fat 160, cholesterol 130, bilirubin 1.2
Body odor sensor: acetone and the like are stored. Such data is stored in the user DB 12 along the time axis t.

  The health service provider 10 provides various services for supporting the health of the user 4 based on the above-described various data stored in the user DB 12.

  Next, the hardware configuration of the control circuit of the wearable computer will be described with reference to FIG. The wearable computer 3 is typified by a smart glass or the like. A CPU (Central Processing Unit) 23 as a control center, a RAM (Random Access Memory) 25 as a work area of the CPU 23, and programs for executing various functions. ROM (Read Only Member) 24 and EEPROM (Electronically Erasable and Programmable Read Only Memory) 26 are connected by a bus 27.

  Various devices are connected to the bus 27 via an interface unit 28. For example, a digital video camera input unit 29 that captures images around the user 4 (images in the direction of the user's line of sight, etc.), a display unit 30 that displays various information to the user, and wireless communication with a base station via the Internet 1 A wireless communication processing unit 31 that performs data communication with a server, an input operation unit 32 for causing the CPU 23 of the wearable computer 3 to execute a desired function, and a voice output unit 33 and a voice input for a user to make a voice call Unit 34, position information acquisition unit 35 for acquiring the current position based on GPS information from the satellite, radio waves from the base station, and radio waves from the wireless LAN access point, Wi-Fi or Bluetooth (registered trademark) , Wi-Fi Direct, Zigbee (registered trademark), Zwave, Ant +, etc. to communicate with various sensors 2 and external sensor group 5, wireless communication interface unit 36, A line-of-sight position detection unit 37, various sensors 38, etc. for detecting the position of the line of sight of the user and identifying the position of the user's line of sight in the video captured by the digital video camera input unit 29 are connected to the interface unit 28. Yes.

  The input operation unit 32 receives not only a manual operation by the user but also an instruction by a user's gesture or a specific wink. The user's voice instruction is received by the voice input unit 34. The various sensors 38 are sensors such as temperature, humidity, illuminance, and ultraviolet light attached to the wearable computer 3.

  The various biosensors 2 are provided with a wireless communication interface unit 57. The wireless communication interface unit 57 wirelessly communicates with the wireless communication interface unit 36 of the wearable computer 3 to transmit sensor detection data to the wearable computer 3. . The types of various biosensors include a heart rate sensor 15 that detects the heart rate of the user 4, a body temperature sensor 16 that detects the body temperature of the user 4, a blood pressure sensor 17 that detects the blood pressure of the user 4, and the blood of the user 4 non-invasively. Non-invasive blood sensor 18 for detecting body odor, body odor sensor 19 for detecting body odor of user 4, step number sensor 50 for detecting the number of steps taken by user 4, tear sensor 51 for detecting a component of tears of user 4, AGEs sensor 52, etc. It is.

  As the blood pressure sensor 17, for example, a sensor that measures blood pressure with only a user's finger using a phase shift method (for example, using an optical sensor) can be considered. The non-invasive blood sensor 18 is a sensor that measures a blood component without collecting blood, and continuously measures the light transmitted through the living body by performing spectroscopic measurement of near-infrared light transmitted through the living body with high sensitivity and high speed. For example, the blood lipid level or blood glucose level is measured by detecting the fluctuation.

  The body odor sensor 19 uses a biosensor to detect a minute amount of an odor substance, and a disease name can be specified by the detected odor component. For example, if the detected odor component is a ketone body such as acetone, diabetes is suspected, if it is ammonia, kidney disease or liver disease is suspected, and if it is a sulfide compound such as methyl chloride, the gastrointestinal tract is suspected. Disease is suspected, and in the case of toluene, lung cancer and breast cancer are suspected.

  The tear sensor 51 is a biosensor provided in, for example, a contact lens, and measures glucose (blood sugar) in tears by fixing an element (glucose oxidase) that recognizes glucose on the contact lens. Measure the value.

  The AGEs sensor 52 is a sensor that can quantify the degree of accumulation of terminal glycation products (AGEs) formed by glycation of proteins in blood vessels. If you consume too much sugar in your daily life, the protein will saccharify and you will not be able to perform its original function. In addition to diabetes, this saccharification reaction is also involved in dementia, cancer, hypertension, arteriosclerosis and the like. These AGEs are substances that are said to cause all diseases, and are known to correlate with blood glucose levels, but unlike blood glucose levels, they have the advantage that measured values are less likely to vary depending on the content of meals before measurement. is there. By detecting these AGEs, the aging degree of blood vessels can be discriminated, and early detection of diabetes or the like is made.

  The various biological sensors described above are periodically connected to the CPU 55 via the interface unit 53. The CPU 55 is regularly connected to a ROM 54 that stores a control program and data, and is also connected to a RAM 56 that functions as a work area for the CPU 55. And the detection signal from the above-mentioned various biological sensors is transmitted to the wireless communication interface unit 36 of the wearable computer 3 through the wireless communication interface unit 57.

  The external sensor group 5 includes a toilet sensor 58 and 61 of other sensors. The toilet sensor 58 includes a urine sensor 59 that detects urine components and a weight sensor 60 that detects the weight of a user sitting in the toilet. The toilet sensor 58 and other sensors 61 communicate wirelessly with the wireless communication interface unit 36 of the wearable computer 3, and sensor detection data is transmitted to the wearable computer 3.

  The wearable computer 3 transmits sensor detection data (biological data) transmitted from the various biological sensors 2 and the external sensor group 5 from the wireless communication processing unit 31 to the server 7 via the Internet 1.

  Next, the hardware configuration of the control circuit of the server 7 will be described with reference to FIG. As described above, the CPU 40, the RAM 41, and the ROM 42 are connected by the bus 43. The interface unit 44 to which the bus 43 is connected is connected to a communication unit 45 with the Internet 1 and the like, a display unit 46 for displaying video and information to the operator, and an input operation unit 47 for receiving operations from the operator. .

  Next, the control operation of the main routine of the server 7 and the wearable computer 3 will be described with reference to FIG. The CPU 23 of the wearable computer 3 performs a process of transmitting the collected biological data to the server 7 in step S (hereinafter simply referred to as “S”) 10, then performs a dialogue process in S 11, and then performs an external sensor process in S 12. Next, the imaging process is performed in S13, and the control returns to S10.

  The server 7 performs data collection processing for collecting biometric data transmitted from the wearable computer 3 in S1, performs interactive processing with the wearable computer 3 in S2, performs machine learning processing in S3, and performs S4. Dialog processing based on the learning result is performed, rule processing is performed in S5, effect verification processing is performed in S6, pattern sales processing is performed in S7, and control returns to S1.

  The machine learning process of S3 performs, for example, data mining using a large number of user action data and sensor detection data (biological data) stored in the user DB 12 as a data set. The correspondence process (S4) based on the learning result is a process for feeding back the result of the machine learning process in S3 to each user. The rule forming process in S5 is a process for storing the result of the machine learning process in S3 in the rule DB 14 as a rule.

  Next, with reference to FIG. 4B, the control operation of the subroutine program of the data transmission process of S10 and the data collection process of S1 will be described. The CPU 23 of the wearable computer 3 determines whether or not there is detection data from the sensor (S18). If there is no detection data (biological data) from the biosensor, the control proceeds to S20, and a captured image of the meal It is determined whether or not there is data. If there is no data, the data transmission process returns and the control shifts to S11 described above.

  If there is detection data from the various biological sensors 2, a process of transmitting the user ID, the sensor identification ID, and the detection data to the server 7 is performed in S19. The sensor identification ID is an ID assigned to various biological sensors such as a heart rate sensor 15, a body temperature sensor 16, a blood pressure sensor 17.

  The CPU 40 of the server 7 determines whether or not the detection data has been received in S22. If not, the control proceeds to S24 to determine whether or not the captured image data has been received and received. If not, the data collection process returns and the process proceeds to S2. If detection data is sent from the wearable computer 3, a determination of YES is made in S22 and the control advances to S23, and the detection data of the type of sensor specified by the sensor identification ID in association with the received user ID is stored in the user DB 12. Is stored. As a result, the sensor detection data shown in FIG. 1 is stored in the user DB 12.

  If the user images a meal of food by the camera input unit 29 of the wearable computer 3 and transmits it to the server 7, a determination of YES is made in S24, and the control proceeds to S25, and the food is determined based on the captured image data. A process of determining the type and calorie and storing it in the user DB 12 is performed. As a result, the type of food such as “meal: salad + beef + bread” as shown in FIG. 1 is stored and calories (not shown) are stored in the user DB 12. The control for calculating the type of food and the calorie from the captured image of the dish is performed by comparing the database storing the images of many dishes and the received captured image, and calculating the appropriate dish from the dish image of the database. Calculate calories. Note that the type of food and the calorie may be artificially determined from the captured image of the dish.

  Next, with reference to FIG. 5 (a), the control operation of the subroutine program of the dialogue process by S11 of the wearable computer 3 and the dialogue process by S2 of the server 7 will be described. The CPU 23 of the wearable computer 3 determines whether or not there is a voice input in S29. If there is no voice input, the control advances to S33 to determine whether or not a message has been received from the server 7 and received. If not, the dialogue process returns and the control shifts to S3 described above. If the voice spoken by the user 4 is input from the voice input unit 34, a determination of YES is made in S29, and the input voice data is transmitted to the server 7 in S30.

  In the server 7, it is determined whether or not the CPU 40 has received a sound (S39). If not, the control proceeds to S44, a message generation process is performed, and it is determined whether or not there is a message in S45. If this is the case, the interactive process ends and returns, and the control shifts to S3.

  If the voice data is transmitted from the wearable computer 3 to the server 7 in S31, a determination of YES is made in S39, the control proceeds to S40, a voice recognition process is performed, and a corresponding process corresponding to the recognition is performed in S41. In step S42, utterance data generation processing based on the corresponding processing is performed. In step S43, processing for transmitting the utterance data and image data to the wearable computer 3 is performed. The wearable computer 3 receives the data in S31, and in S32, utters a voice from the voice output unit 33 based on the received utterance data, and displays an image on the display unit 30 based on the received image data. Perform the process.

  Here, the specific control operation of the speech recognition process in S40 and the specific control operation of the corresponding process according to the recognition result in S41 will be described.

  First, the control operation of the subroutine program for speech recognition processing in S40 will be described with reference to FIG. First, morphological analysis processing is performed in S50, syntax analysis processing is performed in S51, semantic analysis processing is performed in S52, context analysis processing is performed in S53, and intention analysis processing (intention interpretation processing) is performed in S54. These analysis processes are executed by a natural language processing engine.

  Morphological analysis is to divide a sentence into meaningful words and discriminate parts of speech and contents using a dictionary. Parsing is a process for determining the dependency relationship between clauses. Semantic analysis is a natural language processing technique that uses a vector space model, and analyzes the relationship between a group of documents and a group of terms contained therein by generating a set of related concepts. That is. Context analysis is to check the connection of multiple sentences. Intention analysis (intention interpretation) is a process of understanding the meaning of a sentence.

  Next, the control operation of the subroutine program of the corresponding process according to the recognition result in S41 will be described with reference to FIG. First, in S64, it is determined whether or not there is an action report from the user 4. For example, when the user speaks “sold cold medicine” by voice and the voice is recognized by the voice recognition process described above, it is determined by S64 that an action “take cold medicine” has been reported, and the control advances to S65. Then, processing for storing the reported action in the user DB 12 is performed. On the other hand, if it is determined that it is not an action report, the control advances to S66, where it is determined whether or not the dialogue is related to a body part (biological part). When the user 4 speaks, for example, “How is the tone specified by the liver?” And the voice is recognized by the voice recognition process described above, a determination of YES is made in S66 and the control advances to S67. Then, a process for determining the ID of the body part related to the recognition result is performed. Each body part (each biological part) such as the liver, stomach, and kidney is assigned an ID, and the process of determining the part ID is performed in S67. Next, in S68, a process for searching the data of the determined part ID from the user DB 12 is performed. For example, if the part ID determined in S67 is the liver ID, the data “γGTP153, bilirubin 1.2” in the blood sensor item in the sensor detection data stored in the user DB 12 is S68. Is searched. Next, in S69, the utterance content is determined based on the retrieved data. For example, if the numerical data of γGTP and bilirubin is stored in the user DB 12 along the time axis t and the data is improved with the passage of time, the content of the utterance “I have recovered a lot” Determined by S69. Such utterance contents are stored in the dialogue DB 11. The server 7 searches the dialogue DB 11 and determines the utterance content based on the template matched by template matching.

  On the other hand, if NO is determined in S66, the control proceeds to S70, and other corresponding processing is executed. This other handling process will be described later with reference to FIG.

  Returning to FIG. 5 (a), utterance data of the utterance content determined by the dialogue processing according to the recognition result of FIG. 6 (a) described above is generated in S42, and the utterance data together with the image data is a wearable computer in S43. 3 is transmitted. The image data transmitted in S43 includes a mark (icon) representing a body part (for example, liver) corresponding to the part ID determined in S68, a sentence representing speech content, and the like, for example, as shown in FIG. It is an image.

  The wearable computer 3 receives the utterance data and the image and utters it, and displays an image as shown in FIG. Next, the control operation of the subroutine program for the message generation process shown in S44 will be described with reference to FIG. By S75, a process of searching the user DB 12 and determining the ID of the deteriorated body part is performed. For example, when searching the user DB 12 in which the data shown in FIG. 1 is stored, the value of γGTP is 153, which greatly exceeds the reference value 55, so the liver in the body part has deteriorated. And the process of determining the liver ID is performed in S74. Next, it is determined whether or not notification is necessary in S76. Specifically, the determination is made based on whether or not the degree that the numerical value of the sensor detection data (biological data) exceeds the reference value exceeds a threshold value. For example, in the case of γGTP, since the reference value is 55, the threshold is set to 100. In the case of the user with the user ID: 051392704, the value of γGTP is 153 and exceeds the threshold value 100, so that a determination of YES is made in S76, the control proceeds to S77, and a notification message is generated. This notification message may be a notification that the body part corresponding to the ID determined in S75 has deteriorated, but is a message that takes into account the recent action data of the corresponding user. Also good. For example, as shown in FIG. 7, it may be “No more banquets at consecutive banquets. Help me. ΓGTP153”.

  Returning to FIG. 5A, if there is a message generated in this way, a determination of YES is made in S45, and the generated message is transmitted to the wearable computer 3 in S46. The wearable computer 3 that has received it controls to speak and display an image based on the received message in S34.

  Next, the control operation of the subroutine program for external sensor processing shown in S12 of FIG. 4A will be described with reference to FIG. This external sensor process is a process for transmitting detection data (biological data) of the user 4 by the external sensor group 5 shown in FIG. First, at S60, it is determined whether or not a signal has been received from an external sensor. If not, the process returns and proceeds to S13.

  When the user 4 approaches the external sensor group 5 and the biometric data is detected by the external sensor group 5, the sensor detection data (biometric data) becomes wearable by communication between the wireless communication interface unit 36 of the wearable computer 3 and the sensor. It is transmitted to the computer 3. The CPU 23 of the wearable computer 3 that has received the determination makes a determination of YES through S60, and performs a process of storing the sensor identification ID of the received sensor and the received detection data (biological data) in the EEPROM 26. Each sensor of the external sensor group 5 is also assigned a sensor identification ID for identifying the sensor processing in the same manner as the various biological sensors 2, and the sensor identification ID is transmitted to the wearable computer 3 together with the sensor detection data. Come. When the sensor identification ID and the detection data are stored in S61, it is determined that there is detection data from the sensor in S18 described above, the control proceeds to S19, and the stored sensor identification ID and detection data ( Is transmitted to the server 7 together with the user ID.

  Next, the control operation of the subroutine program for the imaging process shown in S13 of FIG. 4A will be described based on FIG. In S72, it is determined whether or not there has been an imaging operation. If not, the process returns to S10. If the user operates the input operation unit 32 of the wearable computer 3 and performs an imaging operation, the control proceeds to S73, and control is performed to capture an image of a subject (such as a food dish) by the camera input unit 29. In step S74, the captured image data is stored. By storing the captured image data in S74, a determination of YES is made in S20 described above, and processing for transmitting the captured image data to the server 7 is performed in S21.

  Next, an example of image display on the display unit 30 performed in S32 and S34 described above will be described with reference to FIG. The upper half image of FIG. 7 displays icons of each body part (biological part) such as the user's liver, stomach, and kidney, and displays the latest conversation characters and conversation date and time of each body part. The By tapping an icon corresponding to the body part that the user wants to newly talk among these body parts, the past conversation history with the tapped body part is displayed (see the lower half of FIG. 7), New communication with internal parts is possible. In the case of FIG. 7, a state where the user has tapped the liver icon is shown. As shown in the lower half image of FIG. 7, the conversation history with the liver is “No longer at banquets every day as a message from the liver at 22:25 on April 8,” γGTP153 Is displayed. This is because the value of γGTP in the detection data of the blood sensor with the user ID: 05139392704 is 153 (see FIG. 1), which is significantly higher than the reference value 55, so that the message generation in FIG. This is a result of generating a notification message by processing and displaying an image in S34. Such a message is displayed to the user by the display unit 30 and also notified to the user by voice from the voice output unit 33. The user who has heard (seen) it speaks by voice “I understand. I'll drink turmeric supplement”, and the voice is input from the voice input unit 34, and the voice recognition process (FIG. 5B). )), And a corresponding process is performed according to the recognized recognition result (see FIG. 6A), and image data based on the corresponding process is transmitted to the wearable computer 3 and the process proceeds to S32. An image is displayed. The result is "I understand. I'll drink turmeric supplement."

  If the user taps the liver icon at a later date and says "How is it?", The same processing as described above is performed, and the spoken word is displayed as an image, and YES is determined in S66 and S67- The content of the utterance is determined by the process of S69, and the utterance data and the image data are transmitted to the wearable computer 3 and uttered by S32 and displayed as an image. As a result, as shown in FIG. 7, the sound “Much recovered. ΓGTP70” is output from the sound output unit 33 and displayed on the display unit 30.

  Note that FIG. 7 shows that the user can make a new conversation by tapping the icon of the body part that the user wants to talk to, but instead of the tap, control is performed so that the user specifies the body organ that the user wants to talk by voice. May be. Next, the control operation of the subroutine program of the machine learning process S3 in FIG. 4A and the corresponding process S4 based on the learning result will be described based on FIGS. 8A and 8B.

  The server 7 is installed with a machine learning engine using artificial intelligence. This machine learning engine periodically performs machine learning. In S80, it is determined whether or not the machine learning time has come. If the machine learning time has not yet come, the process returns to S4. On the other hand, when the machine learning time comes, the control proceeds to S81, where only the data of the user ID whose condition has deteriorated is extracted from the user DB 12. Then, based on the extracted data, data mining is performed in S82 to perform a process of finding an NG pattern NGPi (i = 1, 2, 3,...). Specifically, an action that is a precursor to the deterioration of sensor detection data (biological data) is found by data mining from a data set including user action data and sensor detection data. Note that S86, which will be described later, is a process for finding an action that is a precursor to improved sensor detection data (biological data) by data mining. The server 7 uses a general Neumann computer, but may use a neural net processor (NNP). A large number of “artificial neurons” modeled on real neurons are mounted on the NNP chip, and each neuron cooperates in a network. Further, a quantum computer employing a “quantum annealing method” may be used. Thereby, the time required for the optimization calculation in machine learning can be greatly shortened.

  Next, in S83, it is determined whether NGPi is different from what has been found so far. NGPi (i = 1, 2, 3...) Found so far is stored in the pattern DB 13 (see FIG. 9A). The pattern DB 13 is searched to determine whether or not the pattern is already stored. If it is already stored, the control proceeds to S85, but if it is a new one that has not yet been stored, the control proceeds to S84, and the newly found NGPi is additionally stored in the pattern DB 13. Is called.

  Next, in S85, control is performed to search the user DB 12 and extract only the data of the user ID whose condition has improved. Next, in S86, control is performed to find a positive pattern (PG pattern) PGPi (i = 1, 2, 3,...) By performing data mining. Next, in S87, the pattern DB 13 is searched to determine whether or not the found PGPi is different from that found so far. If it is the same as the one that has already been found, the process returns. If it has been found, the control advances to S88, and control for additionally storing PGPi in the pattern DB 13 is performed.

  An example of NGPi (i = 1, 2, 3,...) And PGPi (i = 1, 2, 3,...) Stored in the pattern DB 13 by S84 and S88 is based on FIG. explain. The patterns found by S82 and S86 are not applicable to all users, and are often applied only to users belonging to a certain segment by segmenting the users based on the biological characteristics of the users. In the case of FIG. 9 (a), as the biometric attribute of the user, a pattern in which the body fat percentage is 25 to 35% and the man is limited to 35 to 55 years old is shown.

First, as an NG pattern (NGPi),
1 Increased risk of cerebral infarction by drinking after a large amount of sweating 2 Eating indigestion when eating a lot of lipids while drinking 3 If you eat a high-calorie meal early in the morning or late at night, you may become obese.
Moreover, as a PG pattern (PGPi),
1 Eating raw eggs before or during drinking can suppress the increase in γGTP 2 Ingesting sesame oil (ra oil) during drinking can suppress the increase in γGTP 3 High calorie meals become obese when taken around 2 pm It is hard to become.

  Next, based on FIG. 8B, the control operation of the subroutine program of the corresponding process based on the machine learning result will be described. Through S93, control is performed to search the user DB 12 for a user ID corresponding to NGPi newly added and stored in the pattern DB 13. Next, in S94, control is performed to notify the user of the searched user ID to NG. Next, in S95, control is performed to search the user DB 12 for a user ID corresponding to the PGPi newly added and stored in the pattern DB 13. Next, in S96, control is performed to notify the user of the searched user ID of PG (positive notification).

  In the notification control in S94 and S96, the server 7 notifies the wearable computer 3 of the corresponding user of the NG message and the action (behavior) that is a precursor thereof, and the PG message and the user action (the action that is a precursor thereof) ( Action).

  By this control, the NG precursor pattern and the PG precursor pattern of the user action (behavior) newly found as a result of machine learning are notified to the corresponding user at an early stage, and the user who knows this is NG promptly. There is an advantage that the precursor pattern is not performed and the PG precursor pattern is actively executed.

Next, the control operation of the subroutine program of the rule forming process of S5 in FIG. 4A will be described based on FIG. In S100, control is performed in which a rule including NGPi and PGPi additionally stored in the pattern DB is created and additionally stored in the rule DB. An example of this is shown in FIG. With reference to FIG.9 (b), about the user of the body attribute of 35-55 years old with the male of body fat percentage 25-35% as a user segment,
・ If you drink after a lot of sweating, rehydrate in advance.
・ When eating or drinking, eat raw eggs and sesame oil (ra oil) and refrain from lipids.
・ A high-calorie meal should be around 2pm, early in the morning and late at night.
Etc.

  Next, in S101, control for creating a message for notifying the user of the additionally stored rule is performed. As a result, YES is determined in S45 of FIG. 5A described above, and a message is transmitted to the wearable computer 3 in S46. The users to be transmitted are all users who are members of the service of the health service provider 10. As a result, there is an advantage that the newly additionally stored rule is thoroughly known to all users.

  Next, the control operation of the subroutine program for the effect verification process shown in S6 of FIG. 4A will be described with reference to FIG. In this effect verification process, pharmaceuticals made by the pharmaceutical manufacturer 9 or health foods (supplements, etc.) made by the health food manufacturer 8 are put on the market, and the biometric data of the user 4 who took them is sucked up to the server 7 side. It accumulates in the user DB 12 and verifies the effect of the pharmaceutical or health food based on the change over time of the biological data (change along the time axis).

  First, in S103, it is determined whether or not a pharmaceutical or health food effect verification request has been received. If not, the process returns to S7. On the other hand, if there is a request, control proceeds to S104, where the product information of the requested product (product information stored in the product development DB 8b or the product development DB 9b) is transmitted, and the requested product is taken. The user ID data is extracted from the user DB 12, and a change over time of the condition is determined based on the extracted data in S105, and an effect is determined based on the determined change in S106. Control is performed to calculate an evaluation value of the effect. In step S107, processing for returning the change data of the condition and the evaluation value of the effect to the client is performed. This change data of the condition is a time series change of the biometric data of the user taking the requested product, and is represented by a line graph or the like. The pharmaceutical manufacturer 9 or the health food manufacturer 8 that has received the change data of the condition and the evaluation value of the effect, when the evaluation value is good and the evaluation value is high, proves its high utility when advertising the requested product. There is an advantage that change data and evaluation values can be used as evidence. In addition, in order to make it easy to determine whether or not the user is taking the requested product, when the user 4 takes a pharmaceutical or health food (supplement etc.), the taken pharmaceutical or health food (supplement etc.) The product name is also transmitted from the wearable computer 3 to the server 7 by voice, and the product name is also stored in the user DB 12 as an action. For example, when the user 4 speaks “take cold medicine” by voice, the server 7 makes an inquiry “Please tell me the brand name of the powder” by the utterance control.

  Next, the control operation of the other corresponding subroutine program shown in S70 of FIG. 6A will be described with reference to FIG. Through S110, it is determined whether or not there has been a health consultation from the user. If not, it is determined in S112 whether or not there has been a request for acquiring health information of another person's organ, and if not, the process returns and control proceeds to S42. When the user 4 speaks a consultation such as a health question by voice, the voice is input by the voice input unit 34 and transmitted to the server 7, and the voice is analyzed by voice recognition processing (see FIG. 5B). A determination of YES is made in S110. As a result, control proceeds to S111, and control is performed to search the rule DB 14 and determine the utterance content to be answered based on the rule corresponding to the question (consultation). As a result, utterance data corresponding to the utterance content is generated in S42, and in S43, the utterance data and image data in which the utterance data is converted to characters are returned to the wearable computer 3, and uttered based on the utterance data. The image is displayed based on (S32).

On the other hand, when the user 4 requests the acquisition of the health information of another person's organ by voice, the analysis process by the voice recognition process is performed in the same manner as described above, and a determination of YES is made in S112, and the designation is made by S113. Control of anonymizing the health information of another person's organ and returning it to the wearable computer 3 is performed. The wearable computer 3 that has received the information displays the received health information of the other person's organ and informs it by voice.
In the above embodiment, machine learning results (NGPi, PGPi, rules, etc.) are fed back and notified to each user 4, and the user 4 takes action while taking communication with the body part referring to the machine learning results. (Action) can be taken. The result of the action (behavior) is reflected in the biometric data of each user 4, the biometric data is sucked up again to the server 7 side and used as machine learning data, and the machine learning result is fed back to each user 4 again. The By cycling through such a series of cycles, it is possible to accumulate changes over time in the biometric data of each user 4 reflecting the machine learning result in the user DB 12, and the machine learning result based on such data can be stored. The pattern is sold by the pattern sales process S7. This sales destination is, for example, a pharmaceutical manufacturer 9 or a health food manufacturer 8. Thereby, the pharmaceutical manufacturer 9 and the health food manufacturer 8 can obtain data useful for product development, and there is an advantage that research and development is promoted.

  As mentioned above, although embodiment of this invention was explained in full detail, this invention is not limited to these embodiment, A various change is possible in the range which does not deviate from the meaning of this invention.

1 Internet 2 Various biological sensors 3 Wearable computer 5 External sensor group 7 Server 10 Health service provider 1
11 Dialogue DB
12 User DB
13 Pattern DB
14 Rule DB

Claims (5)

A computer system for collecting and utilizing user biometric data acquired by a biometric sensor,
Biological data storage means for storing the collected biological data;
Communication means for performing control for realizing communication between the living body part and the user based on the state of the living body part that can be discriminated by the biological data stored by the biological data storage means;
The communication means is:
Natural language recognition means;
Natural languageizing means for expressing the state of the living body part in natural language;
Transmission means for transmitting the natural language expressed by the natural languageization means to the user,
When there is an inquiry in a natural language from a user, the natural language recognition unit recognizes the inquiry, and the natural language expressing the state of the living body part corresponding to the recognized content is transmitted. A computer system in which the means answers the question by communicating to the user. Behavior data accumulating means for collecting and accumulating user behavior data affecting the living body part;
A correlation finding means for finding a correlation between the two based on the user behavior data accumulated by the behavior data accumulation means and the biological data accumulated by the biological data accumulation means; The computer system according to claim 1, further comprising: The biometric data storage means and the behavior data storage means collect and store data from a plurality of users,
The computer system according to claim 1, wherein the correlation finding unit finds a correlation by data mining from a data set including the behavior data and the biological data for a plurality of users. Corresponding user for indexing the user performing the behavior corresponding to the behavior data of the new correlation found by the correlation discovery means from the behavior data of the user accumulated by the behavior data accumulation means Indexing means;
The computer system according to claim 3, further comprising message transmission means for transmitting a predetermined message to a user indexed by the corresponding user indexing means. The natural language recognition means includes voice recognition means for recognizing a user's voice,
The transmission means includes voice transmission means for transmitting the natural language expressed by the natural language conversion means to the user by voice,
The communication means is a natural language in which the voice recognition means recognizes the question when the user asks a voice question, and the natural language means expresses the state of the living body part corresponding to the recognized content. The computer system according to claim 1, wherein the voice transmission means answers the question by voice by transmitting the voice to the user.

Images