JP2023152840A - Information processing apparatus and information processing method - Google Patents

Abstract

To provide an information processing apparatus and an information processing method for determining a state of at least one user on the basis of biometric information of the at least one user.SOLUTION: An information processing system includes an information processing apparatus, and one or more biometric information measuring apparatus. The biometric information measuring apparatus includes various sensors to measure biometric information of a user and transmits the information to the information processing apparatus. A processor of the information processing apparatus acquires biometric information of at least one user, determines a state of the user on the basis of biometric information including brain waves of the user and historical information including time series of context information indicating contexts under which measurements of the biological information at respective points of time are made, and outputs the state of the user to the at least one user.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing device and an information processing method.

It is conceivable to operate an operation target using biological information such as brain waves.

Patent Document 1 describes a BMI device that uses steady visual stimulus evoked potentials (SSVEP).

Patent Document 2 describes a BMI exercise assisting device that detects an electroencephalogram biosignal from a user's brain waves, detects a surface myoelectric potential biosignal from the user's surface myoelectric potential, and calculates a control signal based on both biosignals. is listed.

In Patent Document 3, brain wave conditions, which are conditions for brain waves that occur when recalling a specific motion about the body, are stored in advance, brain wave information is acquired, and brain waves included in the acquired brain wave information are stored. An imaging control device is described that outputs imaging conditions, which are conditions under which the imaging device images a subject, to the imaging device when an electroencephalogram condition is satisfied.

Patent Document 4 describes a method of matching an electroencephalogram signal with a person's intention.

JP2014-071825A Japanese Patent Application Publication No. 2015-211705 Japanese Patent Application Publication No. 2015-102650 Japanese Patent Application Publication No. 2002-166050

An object of the present invention is to determine the condition of at least one user based on the biometric information of at least one user.

In a first embodiment, a processor acquires biometric information of at least one user, and indicates the biometric information including at least brain waves of the at least one user and the context at each time point in which the biometric information was measured. historical information including at least one time series of context information; It's a method.

In a second embodiment, in the first embodiment, the historical information is classified based on a plurality of time series, and the biological information and the historical information are analyzed for each of the plurality of time series. , at least one state of the at least one user is determined.

In a third embodiment, in the second embodiment, each of the plurality of time series is assigned a weight according to a context in which the user is placed, and the weight determined for the at least one user is At least one state is output with a priority determined by the weight, or a subset of the at least one state determined for the at least one user is output depending on the weight.

In a fourth embodiment, in the first embodiment, the state of the at least one user is determined based on a combination of the biometric information and the history information collected in a plurality of time series.

In the fifth embodiment, in the first embodiment, the state of the at least one user is output to at least one other user.

In a sixth embodiment, in the fifth embodiment, the proposed intervention means is determined based on the state or history information of the at least one user or the at least one other user, and the at least one intervention device for outputting the proposed intervention means to the at least one user, or the at least one other user; selected from environments with users.

In a seventh embodiment, in the first embodiment, the proposed intervention means is determined based on the state or history information of the at least one user, and the proposed intervention means is applied to the at least one user. further comprising outputting to the user.

An eighth embodiment in the first embodiment further includes determining the state of the at least one user based on the frequency spectrum of the biopotential measured from the at least one user.

In a ninth embodiment, in the seventh embodiment, the processor is included in a hearable device including a biological information measuring device, an audio input/output device, and a memory, and the method is implemented by the processor. , outputting the proposed intervention means to the at least one user via the audio input/output device, obtaining a response of the at least one user to the proposed intervention means, and transmitting the response to the at least one user. Store in memory.

A tenth embodiment provides, in the ninth embodiment, a first mode of obtaining the response from the at least one user before executing the proposed intervention, or the proposed intervention. The method further includes selecting one of the second modes in which the method is automatically executed by the artificial intelligence.

In the eleventh embodiment, in the seventh embodiment, transmitting at least one of light, image, video, voice, sound, music, vibration, temperature change, collision, stimulation, and odor is provided. further comprising implementing the intervention measures.

In the twelfth embodiment, a processor acquires biometric information of at least one user, determines the state of the at least one user based on the biometric information of the at least one user, and The information processing method supports communication of information by the at least one user based on the state of the user.

A thirteenth embodiment, in the twelfth embodiment, when the at least one user includes a plurality of users, and the processor supports communication of information from one user to another, After obtaining approval from the other user, the information is transmitted to the other user.

In a fourteenth embodiment, in the thirteenth embodiment, if the one user rejects the information to be sent to the other user, the information is stored in a memory associated with the one user. It further includes:

A fifteenth embodiment, in the thirteenth embodiment, includes at least one alternative action for the one user if the one user rejects the information to be sent to the other user. Further including suggesting options.

In the sixteenth embodiment, in the thirteenth embodiment, when the one user confirms transmission of the information to the other user, the information is transmitted in association with the determined state of the one user. The method further includes storing the information in a memory as learning data, and using the learning data for a next determination of the information based on the determined state of the one user.

In the seventeenth embodiment, in the thirteenth embodiment, if the one user approves the transmission of the information to the other user, it is determined whether the transmission of the information to the other user is successful or not. The method further includes storing data indicative of the information in the memory.

The eighteenth embodiment is the same as the twelfth embodiment, further including determining the state of the at least one user based on the frequency spectrum of the biopotential measured from the at least one user.

The nineteenth embodiment acquires biometric information of at least one user, and includes biometric information that includes at least a brain wave of the at least one user, and context information that indicates the context at each point in time when the biometric information was measured. historical information including at least one time series, and is configured to determine the state of the at least one user based on the at least one user's state, and output the state of the at least one user to the at least one user. It is an information processing device equipped with a processor.

The 20th embodiment acquires biometric information of at least one user, determines the state of the at least one user based on the biometric information of the at least one user, and determines the state of the at least one user. The information processing apparatus includes a processor configured to support communication of information by the at least one user based on a state.

According to the first to twentieth embodiments, the condition of at least one user can be determined based on the biometric information of at least one user.

FIG. 1 is a block diagram showing the configuration of an information processing system according to the present embodiment. FIG. 2 is a block diagram showing the hardware configuration of the information processing device. FIG. 2 is a block diagram showing the hardware configuration of a terminal device. It is a figure showing a management table. It is a figure showing a management table. It is a figure showing a management table. It is a figure showing a management table. FIG. 2 is a diagram schematically showing people and equipment present in a certain place. FIG. 2 is a diagram schematically showing people and equipment present in a certain place. It is a figure showing a screen. It is a figure showing a screen. It is a figure showing a screen. It is a figure showing a screen. It is a figure showing a screen. FIG. 2 is a diagram schematically showing people and equipment present in a certain place. It is a figure showing a screen. It is a figure showing a screen. FIG. 2 is a diagram schematically showing a tomographic image of the brain. It is a figure showing a screen. It is a figure showing a screen. FIG. 3 is a diagram showing a specific example of a graph. FIG. 3 is a diagram showing a specific example of a graph. It is a figure showing a setting screen. It is a figure showing a screen. It is a figure showing a screen. It is a figure showing a screen. 12 is a flowchart showing the flow of processing according to Example 20. 12 is a flowchart showing the flow of processing according to Example 20. FIG. 1 is a perspective view showing the entire hearable device. FIG. 2 is a diagram when the biological information measuring device is viewed from above. It is a diagram when the left earphone is viewed from the left. FIG. 3 is a perspective view showing the right earphone. It is a perspective view showing a left earphone. It is a diagram when the right earphone is viewed from above. FIG. 3 is an exploded perspective view showing the right earphone. FIG. 3 is a cross-sectional view of the right earphone. FIG. 3 is a perspective view showing the inside of the left earphone. FIG. 3 is a diagram showing brain wave data of a person who is in the psychological state of liking/disliking. FIG. 3 is a diagram showing brain wave data of a person in a surprised psychological state. FIG. 3 is a diagram showing brain wave data of a person viewing a VR image including a monster-like creature. FIG. 3 is a diagram showing brain wave data of a person performing a mental task to complete a white puzzle. FIG. 3 is a diagram showing brain wave data of a person receiving a head massage. FIG. 3 is a diagram showing brain wave data of a person who listened to music or took supplements. FIG. 4 is a diagram showing brain wave data of a person who has progressed to the stage of sleep when falling asleep and after falling asleep. FIG. 3 is a diagram showing changes in brain waves when a user feels itching.

An information processing system according to this embodiment will be described with reference to FIG. FIG. 1 shows an example of the configuration of an information processing system according to this embodiment.

The information processing system according to this embodiment includes an information processing device 10, one or more biological information measuring devices 12, one or more environmental information measuring devices 14, one or more devices 16, and one or more biological information measuring devices 12. It includes a terminal device 18. The number of each device shown in FIG. 1 is only an example, and the number of each device is not limited to the number of each device shown in FIG. 1. Further, the information processing system may include devices other than the devices shown in FIG. 1 (for example, external devices such as servers). If device 16 is not used, device 16 may not be included in the information handling system. If the terminal device 18 is not used, the terminal device 18 may not be included in the information processing system.

Each device included in the information handling system is configured to communicate with other devices. The communication may be wired communication using a cable or wireless communication. That is, each device may be physically connected to another device by a cable to send and receive information to each other, or may send and receive information to and from each other by wireless communication. As the wireless communication, for example, short-range wireless communication, Wi-Fi (registered trademark), etc. are used. Wireless communication standards other than these may also be used. Examples of short-range wireless communication include Bluetooth (registered trademark), RFID (Radio Frequency Identifier), and NFC. Each device may communicate with other devices via a communication path such as a LAN (Local Area Network) or the Internet.

The information processing device 10 is, for example, a personal computer (hereinafter referred to as “PC”), a tablet PC, a smartphone, a mobile phone, or other devices (for example, a server, a robot, or other equipment). The information processing device 10 may be a terminal device that can be carried by a person (for example, a tablet PC, a smartphone, a mobile phone, etc.), or a device that is installed on a table or the like. However, it may be a movable device (for example, a self-propelled device), a device that performs work, or a device that can interact with people. For example, the information processing device 10 may be a smart speaker having a communication function, a microphone, and a speaker, or may be a robot that interacts with and works with people. The information processing device 10 may be a device equipped with artificial intelligence (AI).

The biological information measuring device 12 is a device configured to measure biological information of living things. Living things are humans, non-human animals, or plants.

For example, the biological information measuring device 12 includes various sensors, electrodes, etc., and measures biological information of living things. When a plurality of biological information measuring devices 12 are used, each biological information measuring device 12 may measure different types of biological information. Some or all of the plurality of biological information measuring devices 12 may measure the same type of biological information. Furthermore, the biological information measuring device 12 may measure one type of biological information or multiple types of biological information.

Biological information may include various information emitted from living things. Biological information includes, for example, information indicating brain activity (e.g., brain waves, cerebral blood flow, brain magnetic field signals, etc.), information indicating pulse rate, myoelectric information such as myoelectric waveforms, and information regarding saliva (e.g., saliva volume). information indicating pulse wave, information indicating blood pressure, information indicating blood flow, information indicating pulse, information indicating heart rate, information indicating electrocardiogram waveform, information indicating eye movement, information indicating body temperature , information indicating the amount of perspiration, information indicating the line of sight, audio information, information indicating the movement of a person, or information obtained from body fluids (for example, blood, etc.). Furthermore, information specified by a biomarker may be used as biological information. Furthermore, the biological information may be information resulting from an electric potential detected from a living organism. For example, biological information includes electroencephalograms, which are the measurement results of minute currents generated by brain activity, electrocardiograms, which are the measurement results of minute currents generated by heart beats, and measurements of minute currents generated by muscle activity. It may be an electromyogram, which is the result, or a skin potential, which is the measurement result of microcurrent generated in the skin. These are only examples of biometric information, and biometric information other than these may be used.

By analyzing biological information, emotional information, mental information, psychological information, etc. may be obtained as information on a person's condition. For example, by analyzing a person's biological information, information indicating the person's emotions, information indicating the person's mental state, information indicating the person's psychological state, etc. may be obtained. Of course, information indicating the condition of non-human animals and plants may be obtained by analyzing biological information of non-human animals and plants. For example, information on a person's state includes the state of feeling the three major needs of appetite, desire for sleep, and sexual desire, as well as consciousness centered on feelings. Excitement, curiosity, sexual curiosity, calm, impatience, wonder (confusion), happiness, good fortune, relaxation, tension, honor, responsibility, respect, closeness, admiration, desire ( desire), fear, courage, pleasure, pleasure (regarding good deeds and virtues), regret, satisfaction, dissatisfaction, regret, disgust, shame, contempt, jealousy, guilt, murderous intent, schadenfreude, saudade, expectation, sense of superiority, inferiority It is information such as feeling, resentment, resentment, suffering, sadness, sadness, emotion, anger, worry (anguish, anguish, anguish), resignation (giving up), despair, hope, hatred (love/hate), love, and emptiness. Other symptoms include itching, pain, the urge to urinate, the urge to defecate, a feeling of hunger, and a feeling of fullness. Furthermore, as communication information transmitted from person to person, there is information regarding people's thoughts and intentions. For example, agreeing, agreeing, disagreeing, rejecting, being confused, confused, etc. In particular, human brain wave information has a high possibility of detecting more of the above information than other biological information, and is the main biological information.

Brain waves are generally classified as follows. Delta (δ) waves have a frequency range of 1 to 4 Hz, and correspond to a state of dreamless deep sleep or non-REM sleep, and decrease as sleep becomes lighter, corresponding to an unconscious state. Theta (θ) waves have a frequency range of 4 to 8 Hz and correspond to states of relaxation, slumber, light sleep, intuitive states, creative states, accessing memories, and states of inspiration. do. Low alpha (α) waves have a frequency range of 8-10 Hz and correspond to a relaxed state (calm, conscious but introverted state), appearing when the eyes are closed and decreasing when the eyes are open. High alpha (α) waves have a frequency range of 10 to 12 Hz and correspond to a relaxed state (a state of concentration and outward awareness), appearing when the eyes are closed and decreasing when the eyes are open. SMR (sensorimotor rhythm) has a frequency range of 12 to 15 Hz, and corresponds to a state of being in a zone, a state of balance between a state of relaxation and a state of concentration, and appears before moving or thinking. Low beta (β) waves have a frequency range of 15 to 20 Hz and correspond to times when we are thinking or solving problems. High beta (β) waves have a frequency range of 20-30 Hz and correspond to states such as alertness, agitation, tension, and anxiety. Low gamma (γ) waves have a frequency range of 31 to 40 Hz and correspond to memory formation, higher mental activities, higher cognitive activities, and states of well-being. Furthermore, the frequency range of medium gamma (γ) waves is 41 to 50 Hz, which corresponds to visual information processing and the like. Furthermore, the biological information measuring device 12 may measure some or all of the above-mentioned brain waves.

The biological information measuring device 12 may be a device (for example, a wearable device) that is attached to a living organism that is a subject to measure biological information, or a device that measures biological information without being attached to a living organism. good.

Wearable devices include, for example, hearable devices that are attached to an animal's ears, devices that are attached to an animal's head, and devices that are attached to an animal's wrist, arm, or finger (hereinafter referred to as "wrist, etc."). For example, wristwatch-type devices such as smart watches), devices worn around the animal's neck, devices worn on the animal's torso (such as the chest or abdomen), and lower limbs (such as human thighs, lower legs, knees, and feet). , ankle, etc.), a glasses-type device, a contact lens-type device, etc. The biological information measuring device 12 may be a wearable device that is attached to a part other than these. Furthermore, the biological information measuring device 12 may be attached to multiple locations.

The hearable device may be, for example, an earphone, a hearing aid, a piercing device, a clip-on device, or a device that includes a band or cable that is wrapped around the ear. The head-worn device may be, for example, a headset that includes a band, cable, or the like that is wrapped around the head. The device worn on the wrist or the like may be, for example, a device that includes a band, cable, or the like that is wrapped around the wrist or the like. Devices worn at other locations may also include bands, cables, and the like. In particular, when measuring brain wave information, a hearable device is preferable because it can measure brain waves with a natural appearance. Excellent. A natural appearance that makes it difficult for people to tell at first glance that they are measuring brain waves is an important element, and medical electroencephalographs that have electrodes on the head are clearly measuring brain waves. Because it is so visible, it can be a big hindrance when wearing it on a daily basis to take measurements, and many say they are hesitant to wear it. In addition, hearable devices can use methods that can input voice information to the human ear and output voice information from the mouth, so while detecting brain wave information, the results of the interpretation of that information can be confirmed by voice. It is excellent in that it allows you to correct the interpretation if there is a mistake in the interpretation. Furthermore, BMI/BCI, which mainly uses brain waves, is often used in situations where equipment is operated or communication is performed between people, but it is important to know how to operate the equipment with high precision. The challenge is whether you can correctly judge the content of the communication you want to convey. Since BMI/BCI using hearable devices is worn in the ear, it can be intentionally expressed by a person by using the audio input method mentioned earlier or by using voice information from the mouth, or by using other methods other than brain waves. Accuracy can be improved by utilizing biological information (for example, electromyographic information that appears when you intentionally move your jaw three times, or electrooculography that appears when you intentionally move your eyeballs strongly) It is possible to confirm whether the judgment is correctly made, and to use it for determining the implementation of operations and transmission of communication, etc.).

The biological information measuring device 12 may be a contact type device that measures the biological information of the living organism by contacting the living organism, or the biological information measuring device 12 may measure the biological information of the living organism without coming into contact with the living organism. It may also be a non-contact type device that measures information. The biological information measuring device 12 may be a device having both a contact type device and a non-contact type device. In other words, the biological information measurement device 12 may measure the biological information of the living organism by contacting the living organism, and may also measure the biological information of the living organism without contacting the living organism. The biological information measured by contacting a living organism and the biological information measured without contacting a living organism may be the same type of biological information or may be different types of biological information.

The biological information measuring device 12 includes electrodes for measuring biological information, sensors other than electrodes, and the like. The electrode may be in contact with a living body to detect an electric potential, which is an example of biological information, or may be in contact with a living body to detect an electric potential. Sensors other than electrodes may measure biological information by contacting a living body, or may measure biological information without contacting a living body. For example, electrodes may be brought into contact with an animal and detect electrical potentials indicative of the animal's brain waves. Further, the sensor may measure biological information indicating the body temperature of the animal without coming into contact with the animal. Of course, these are just examples, and other biological information may be measured.

For example, the biological information measuring device 12 includes one or more electrodes. For example, a plurality of electrodes are provided on a living organism, and electric potential is detected by the plurality of electrodes. The detected potential includes a biopotential that is an example of biological information of the living organism, and by processing the detected potential, the biopotential is extracted from the detected potential. For example, the detected potential may contain noise other than that derived from the biopotential, and by performing processing such as noise cancellation processing, the biopotential from which noise has been removed can be obtained. The noise includes, for example, an electric potential generated due to the movement of an organism, an electric potential transmitted from outside the organism, an electric potential generated from the earth's environment, an electric potential representing biological information other than the object to be measured, and the like. The noise may include a potential generated in a device such as a PC or a smartphone. The electrodes used for potential detection may be switched depending on the potential detection sensitivity, noise, and other measurement conditions. In addition, when potentials derived from various biopotentials are measured in combination, the potentials measured for each biopotential may be processed to be separated by frequency, or the composite portion may be separated during the measurement time. Potentials derived from various biopotentials may be estimated and separated by performing processing to separate the parts into weakly composite parts. For example, in the aforementioned hearable device, information indicating brain activity, information indicating pulse rate, myoelectric information derived from the movement of muscles, etc., information resulting from blood flow such as pulse waves and heart rate, etc. are combined. However, this information often differs in frequency and absolute value of output, and can be estimated separately based on frequency analysis of the measurement signal and differences in output value level. For example, brain waves are divided into α, β, θ, δ, and γ wave bands depending on their frequency. Another method is to measure each signal with a hearable device under the same state conditions, and then measure a single measurement signal multiple times individually with a commercially available electroencephalogram, electromyography measurement device, or blood flow measurement device. After analyzing the combined measurement signal information of the hearable device using frequency analysis methods such as Fourier transform and wavelet transform, we perform statistical processing to estimate which biological information is derived from. You may introduce means to do so.

The biological information measuring device 12 transmits the measured biological information to the information processing device 10. The biological information measuring device 12 may transmit biological information to the information processing device 10 every time it measures biological information, or may store the biological information and transmit the biological information to the information processing device 10 at predetermined time intervals. The biometric information may be transmitted to the information processing apparatus 10 at a timing specified by a person. In addition, the biological information measuring device 12 receives biological information measured by another biological information measuring device 12 from the other biological information measuring device 12, and compares the biological information measured by itself with the other biological information measuring device 12. The measured biological information may also be transmitted to the information processing device 10.

The biological information measuring device 12 may analyze biological information measured by itself or another biological information measuring device, and may transmit information indicating the result of the analysis to the information processing device 10. For example, the biological information measuring device 12 may include a processor, and the processor may analyze the biological information. Of course, the analysis may be performed by the information processing device 10 or other devices.

The biological information measuring device 12 includes a battery, and may be driven by power supplied from the battery, or may be driven by receiving power from another device (for example, the information processing device 10, etc.).

The information processing device 10 receives biological information from the biological information measuring device 12, analyzes the biological information, stores the biological information, outputs the biological information, stores information indicating the result of the biological information analysis, and stores the biological information. Outputs information indicating the results of the analysis. Of course, the analysis of biological information may be performed by the biological information measuring device 12 or other devices. Outputting biometric information includes, for example, displaying biometric information, outputting biometric information as voice information, and the like. Outputting information indicating the results of the analysis of biological information includes, for example, displaying information indicating the results of the analysis, outputting the results of the analysis as audio information, and the like. The information processing device 10 may transmit information indicating biological information and analysis results to other devices.

The information processing device 10 may include one or more biological information measuring devices 12. That is, one or more biological information measuring devices 12 may be incorporated into the information processing device 10 to configure one device. The entire information processing device 10 including one or more biological information measuring devices 12 may be attached to a living thing to measure biological information. In other words, the information processing device 10 may be a wearable device. For example, the information processing device 10 may be a device worn on a person's head (for example, forehead) or a hearable device (for example, earphones, headphones, etc.) worn on a person's ears. It may be a device that is worn on a person's arm, hand, wrist, finger, etc. (for example, a wristwatch-type device such as a smart watch), or it may be a device that is worn around a person's neck. It may be a device that is attached to a person's torso (e.g., abdomen or chest), or it may be a device that is attached to a person's lower limbs (e.g., thigh, lower leg, knee, foot, ankle, etc.). Good too. The information processing device 10 may be a health appliance or the like that is worn on a person's arm, hand, torso, or lower limb. The information processing device 10 may be attached to parts other than these.

The information processing device 10 and the biological information measuring device 12 may be separate devices. For example, the information processing device 10 may be a device such as a robot, a smart speaker, or a server, and the biological information measuring device 12 may be a wearable device attached to a living creature.

The environmental information measuring device 14 is configured to measure environmental information regarding living things and the environment around the environmental information measuring device 14 . For example, the environmental information measuring device 14 includes a camera that is a photographing device, a microphone that collects sound, a temperature sensor that measures temperature, a humidity sensor that measures humidity, an odor sensor that measures odor, an illuminance sensor that measures brightness, and an infrared sensor that measures brightness. A sensor, a pressure sensor such as an atmospheric pressure sensor, a position sensor that detects the movement of an object, a human sensor, or the like. One or more of these sensors may be included in the information processing system as the environmental information measuring device 14.

For example, the surroundings of the information processing device 10 and other places are photographed by a camera, and image data representing the surroundings and image data representing the other places are generated. The image data may be moving image data or still image data. Image data photographed by a camera corresponds to an example of environmental information regarding the environment included in the photographing range of the camera. Further, image data representing the living creature generated by photographing the living creature with a camera may be used as biological information of the living creature. For example, the movement of the living creature (for example, a human), the body shape of the living creature, etc. detected from the image data corresponds to an example of biological information of the living creature. In that sense, the camera corresponds to an example of the biological information measuring device 12 that measures biological information of the living creature.

Further, sounds around the microphone (for example, human voices and other sounds) are input to the microphone, and sound data is generated by the microphone. Sound data representing the sound input to the microphone corresponds to an example of environmental information regarding the environment around the microphone. Further, sound data representing a person's voice input into a microphone may be used as biological information of the person. In that sense, the microphone corresponds to an example of the biological information measuring device 12 that measures the biological information of the person. Further, sound data input into a microphone and representing the sounds of an animal other than a human may be used as biological information of the animal.

Further, data measured by a temperature sensor, humidity sensor, odor sensor, illuminance sensor, infrared sensor, pressure sensor, position sensor, human sensor, etc. corresponds to an example of environmental information. Further, data measured from living things by these sensors may be used as biological information of the living things. In that sense, these sensors correspond to an example of the biological information measuring device 12 that measures biological information of the living organism.

The environmental information includes information indicating the size of the room in which the living creature is located, information indicating the size of the room in which the device 16 is installed, information indicating the number of windows provided in the room, etc. You can. Furthermore, the environmental information is not limited to information measured by the environmental information measuring device 14, but may be predetermined information or information measured in advance.

Note that one or more environmental information measuring devices 14 may be included in the information processing device 10.

The device 16 is an example of an operation target. The devices 16 include, for example, a PC, a tablet PC, a smartphone, a mobile phone, a robot (for example, a humanoid robot, a non-human animal robot, a cleaning robot, and other robots), a projector, a display device such as a liquid crystal display, etc. devices, recording devices, playback devices, imaging devices such as cameras, refrigerators, rice cookers, microwave ovens, coffee makers, vacuum cleaners, washing machines, air conditioners, lighting equipment, clocks, surveillance cameras, automobiles, motorcycles, aircraft (e.g. unmanned Aircraft (e.g. drones)), game consoles, gas ranges, heated toilet seats, ventilation fans, doorbells, entrance monitors, elevators, doors, windows, or various sensing devices (e.g. temperature sensors, humidity sensors, voltage sensors, current sensors, etc.) ) etc. The concept of equipment 16 may include equipment in general. For example, information equipment, video equipment, audio equipment, and other equipment may also be included in the category of equipment 16 according to this embodiment.

The device 16 includes a communication device that is a communication interface, a memory that stores information, and a processor that controls the operation of the device 16. The device 16 may have a user interface including a display, an operating device, and the like. The device 16 may transmit device identification information for identifying the device 16 itself to the information processing device 10. The device identification information is, for example, the ID, name, model number, or address (eg, MAC address, IP address, etc.) of the device 16.

In this embodiment, an operation target is operated based on biometric information of one or more living things (for example, one or more people). For example, the information processing device 10 acquires biological information of a living thing from the biological information measuring device 12, and operates the operation target based on the acquired biological information. For example, the information processing device 10 acquires biological information of a living creature from the biological information measuring device 12, specifies the operation content of the operation target based on the acquired biological information, and operates the operation target according to the specified operation content. . The process of specifying the operation details may be performed by the biological information measuring device 12 or other devices (for example, the server, the device 16, the terminal device 18, etc.). A device other than the information processing device 10 (for example, a server, a terminal device 18, etc.) may operate the operation target. For example, the information processing device 10 may specify the operation content based on biometric information, and a device other than the information processing device 10 may operate the operation target according to the operation content specified by the information processing device 10.

The operation target may be hardware, software, or both hardware and software.

The hardware to be operated is not particularly limited, and may be, for example, the device 16 or a device other than the device 16 (for example, the information processing device 10, the biological information measuring device 12, the environmental information measuring device 14, or a server). be. A plurality of pieces of hardware may be the objects of operation.

The software to be operated is not particularly limited, and may be, for example, an application program or an OS (operating system). The software to be operated may be software installed in the information processing device 10, software installed in the device 16, or software installed in the terminal device 18. Alternatively, it may be software installed on other devices (for example, the biological information measuring device 12, the environmental information measuring device 14, a server, or other devices), or it may be provided via a communication path such as the Internet. It may be software that A plurality of pieces of software may be the target of operation. When multiple pieces of software are to be operated, each piece of software may be installed on the same hardware or on different pieces of hardware.

Hereinafter, the configuration of the information processing device 10 will be described in detail with reference to FIG. 2. FIG. 2 shows an example of the hardware configuration of the information processing device 10.

The information processing device 10 includes, for example, a communication device 20, a UI 22, a memory 24, and a processor 26. The information processing device 10 may include configurations other than these.

The communication device 20 is a communication interface having a communication chip, a communication circuit, etc., and has a function of transmitting information to other devices and a function of receiving information transmitted from other devices. The communication device 20 may have a wireless communication function or a wired communication function. The communication device 20 may communicate with other devices by using, for example, short-range wireless communication, or may communicate with other devices via a communication path such as a LAN or the Internet.

The UI 22 is a user interface and includes at least one of a display and an operating device. The display is a liquid crystal display, an EL display, or the like. The operating device is a keyboard, input keys, operation panel, or the like. The UI 20 may be a UI such as a touch panel that has both a display and an operation device. The UI 20 may include a microphone and a speaker. Note that the information processing device 10 does not need to include the UI 20.

The memory 24 is a device that constitutes one or more storage areas that store various types of information. The memory 24 is, for example, a hard disk drive, various types of memory (eg, RAM, DRAM, ROM, etc.), other storage devices (eg, optical disk, etc.), or a combination thereof. One or more memories 24 are included in the information processing device 10.

Management information is stored in the memory 24. The management information is information for specifying the operation content of the operation target based on biometric information.

The processor 26 is configured to control the operation of each part of the information processing device 10. Processor 26 may have memory.

For example, predetermined reference biological information and operation information indicating the operation contents of the operation target are linked in advance and registered in the above-mentioned management information. The reference biological information may be biological information that is assumed to be generated from an animal (e.g., a human or other animal) that performs the operation linked to the reference biological information, or may be biological information that is assumed to be generated from an animal (e.g., a human or other animal) that performs the operation linked to the reference biological information, or may request the execution of the operation. It may be biological information estimated to be generated from the desired animal, or it may be biological information estimated to be generated from a plant. The reference biometric information can also be said to be biometric information indicating the operation content of the operation target. The operation information indicating the operation content may include information indicating an operation target to be operated according to the operation content.

For each creature, the reference biological information and the operation information may be linked and registered in the management information. For example, reference biological information and operation information may be linked and registered in the management information for each type of living organism. For example, management information for humans, management information for animals other than humans, and management information for plants may be created and each management information may be stored in the memory 24. In the management information for humans, reference biological information that is assumed to be measured from humans and operation information are registered in a linked manner. The reference biological information and the operation information may be linked and registered in the management information for each person (for example, for each user). In the management information for animals other than humans, reference biological information estimated to be measured from the animal and operation information are registered in a linked manner. Management information may be created for each type of animal other than humans. Further, in the management information for plants, reference biological information estimated to be measured from plants and operation information are registered in a linked manner. Management information may be created for each type of plant.

The operation information may be information indicating the operation contents of the power supply of the device 16, information indicating the operation contents at the functional level of the device 16, or information indicating the operation contents of the power supply and the operation contents at the functional level. The information may indicate both.

The content of the operation for powering the device 16 is to turn on or turn off the power to the device 16. The operation information regarding the details of the power supply operation is information indicating an operation for turning on or off the power of the device 16. It can be said that the biometric information that is linked to the operation information regarding the operation content of the power source is the biometric information that corresponds to whether the power of the device 16 is turned on or off. For each type of living creature or for each individual living creature, the operation information regarding the operation content of the power supply of the device 16 and the reference biological information may be linked and registered in the management information.

The operation content for the functional level of the device 16 is an operation for setting the functional level of the device 16. The operation information regarding the operation content of the functional level is information indicating an operation for setting the functional level of the device 16. It can be said that the biometric information that is linked to the operation information regarding the operation content at the functional level is biometric information that corresponds to the functional level of the device 16. For each type of living organism or for each individual living organism, operation information regarding the functional level operation content of the device 16 and reference biological information may be linked and registered in the management information.

The functional level is, for example, a level related to the performance and output of the device 16. To explain with specific examples, the set temperature of the air conditioner, the air volume of the air conditioner, the direction of the air conditioner, whether or not the air conditioner has a dehumidifying function, the brightness of the display device, the brightness of the lighting device, the volume of the speaker, and the ability to run on its own. The moving speed of devices (e.g. robots, self-propelled vacuum cleaners, etc.), the setting values of devices such as photographing devices, recording devices, playback devices, etc., the setting values of devices such as refrigerators, rice cookers, microwave ovens, etc. Setting values of a sensing device, etc. correspond to an example of the device level. These are just examples, and setting values other than these may be used at the functional level.

Note that the management information may not be stored in the information processing device 10, or may be stored in the information processing device 10 and in a device other than the information processing device 10.

The processor 26 is configured to acquire biometric information of a living creature and operate the operation target based on the biometric information.

For example, when biometric information is measured from a living organism by the biometric information measuring device 12, the biometric information is transmitted from the biometric information measuring device 12 to the information processing device 10. The processor 26 receives the biometric information and specifies the operation content of the operation target based on the biometric information. The processor 26 operates the operation target according to the specified operation content. The processor 26 may specify the operation target and the operation content of the operation target based on the biological information of the living creature. As another example, the operation target may be determined in advance. For example, the operation target may be specified by a person. In this case, the processor 26 specifies the operation content of the determined operation target based on the biological information of the living creature.

When the operation target is the device 16, the processor 26 operates the device 16 by transmitting control information including operation information indicating the specified operation content to the device 16. The device 16 to be operated operates according to the control information. The same applies when hardware other than the device 16 is the object of operation.

If the operation target is software, the processor 26 operates the software according to the specified operation details. For example, the processor 26 starts software or performs processing using software. If software is installed on the information processing device 10, the processor 26 operates the software installed on the information processing device 10. If the software is installed on a device other than the information processing device 10, the processor 26 operates the software by transmitting control information including operation information indicating operation details to the device where the software is installed.

For example, the processor 26 compares the biometric information measured from the living organism with each reference biometric information registered in the above-mentioned management information, and selects the reference biometric information whose difference from the biometric information is within an allowable range. search for. The processor 26 identifies the operation details for the operation target that are linked to the retrieved reference biological information. Thereby, the operation target and the operation details of the operation target are specified. The permissible range is predetermined. The tolerance range may be changed by the user. The permissible range may be determined for each type of organism or for each individual organism.

When a plurality of reference biological information is retrieved that includes a difference from the biological information measured from a living organism within an allowable range, the processor 26 selects the biological information that has the smallest difference from the biological information among the plurality of reference biological information. It is also possible to specify the reference biological information that is the reference biological information and specify the operation details linked to the identified reference biological information, or to specify the operation details linked to each of the plurality of reference biological information. It's okay.

The reference biological information may be information indicating a characteristic component of the biological information. In this case, the processor 26 may extract a characteristic component from the biological information measured from the living organism, and search for reference biological information having a component whose difference from the extracted component is within an acceptable range. . For example, when brain waves are used as biological information, the processor 26 may extract characteristic components from the animal's brain waves and analyze the components to infer the operation content represented by the brain waves.

Reference biological information may be used as a threshold value. For example, when the reference biological information shows a waveform, the amplitude of the peak in the waveform is used as the threshold. In this case, the processor 26 searches for reference biological information having a waveform similar to or similar to the waveform shown by the biological information measured from the living organism, and for which the measured biological information is equal to or higher than a threshold value, and searches Specify the operation details linked to the reference biometric information. In other words, the waveform shown by the measured biological information is similar to or similar to the waveform shown by some reference biological information, and the amplitude of the peak of the waveform shown by the measured biological information is the amplitude of the peak of the waveform shown by the reference biological information. If the threshold value is greater than or equal to a certain threshold value, the processor 26 specifies the operation content linked to the reference biometric information.

When operating an object using information indicating the animal's brain activity, a brain-machine interface (BMI) is established by the biological information measuring device 12 that measures the brain activity and the information processing device 10. may be constructed. Generally, a brain-machine interface is a general term for devices that interface the brain with a computer, etc. by detecting brain waves or the like, or conversely by stimulating the brain. When the connection destination is a computer, it is also called a Brain-computer Interface (BCI). In this embodiment, we will discuss usage examples such as operating devices and communicating with people (mutual transmission of information about people's thoughts and intentions, status information, etc.) by utilizing biological information including brain waves. However, these are considered to be part of the brain-machine interface and will be described hereafter. The brain-machine interface method may be invasive or non-invasive. In this case, the processor 26 operates the operation target based on brain activity (for example, brain waves, etc.). In order to operate the operation target, the processor 26 may extract characteristic components from the brain waves and operate the operation target based on the extracted components. In order to extract characteristic components from brain waves, Fast Fourier Transform (FFT), Wavelet Transform (WT), Time Frequency Distribution (TFD), and Eigenvector (EM) are used. Methods), autoregressive model ( ARM (Auto Regressive Method) or the like may be used. In addition, the feature vector obtained by extracting the features can be used as a method of linking brain waves and the operation content of the operation target, or as a method of linking the communication target and the communication content in communication between people, for example. , Independent Component Analysis (ICA), k-means method, Support Vector Machine (SVM), convolutional neural network, etc. may be used.

Further, the processor 26 may receive identification information of the operation target transmitted from the operation target to identify the operation target. For example, the processor 26 transmits a request to obtain identification information to the operation target, and acquires the identification information transmitted from the operation target in response to the request. In addition, from hardware that is connected to the information processing device 10 and becomes able to communicate with the information processing device 10, identification information of the hardware and identification information of software installed on the hardware. may be transmitted to the information processing device 10. Processor 26 receives the identification information thus transmitted.

Note that the processing by the processor 26 may be executed by a device other than the information processing device 10 (for example, the biological information measuring device 12, the environmental information measuring device 14, the device 16, the terminal device 18, or other devices). Further, part of the process may be executed by the information processing apparatus 10, and another part of the process may be executed by a device other than the information processing apparatus 10. For example, the analysis of biological information (for example, the process of specifying the operation details) may be performed by a device other than the information processing device 10, and the operation of the operation target may be performed by the information processing device 10.

The configuration of the terminal device 18 will be described in detail below with reference to FIG. 3. FIG. 3 shows an example of the hardware configuration of the terminal device 18.

The terminal device 18 includes, for example, a communication device 28, a UI 30, a memory 32, and a processor 34.

The communication device 28 is a communication interface having a communication chip, a communication circuit, etc., and has a function of transmitting information to other devices and a function of receiving information transmitted from other devices. The communication device 28 may have a wireless communication function or a wired communication function. The communication device 28 may communicate with other devices by using short-range wireless communication, for example, or may communicate with other devices via a communication path such as a LAN or the Internet.

The UI 30 is a user interface and includes at least one of a display and an operating device. The display is a liquid crystal display, an EL display, or the like. The operating device is a keyboard, input keys, operation panel, or the like. The UI 30 may be a UI such as a touch panel that functions as both a display and an operation device. The UI 30 may include a microphone and a speaker.

The memory 32 is a device that constitutes one or more storage areas that store various types of information. The memory 32 is, for example, a hard disk drive, various types of memory (eg, RAM, DRAM, ROM, etc.), other storage devices (eg, optical disk, etc.), or a combination thereof. One or more memories 32 are included in terminal device 18 .

The processor 34 is configured to control the operation of each part of the terminal device 18. Processor 34 may include memory.

A specific example of the management information will be described below.

FIG. 4 shows an example of a management table that is an example of management information. The data of the management table is stored in the memory 24. The data of the management table may not be stored in the memory 24, but may be stored in a device other than the information processing device 10 (for example, a device such as a server).

In the management table, the ID, the reference electroencephalogram, and the operation information indicating the operation contents of the operation target are linked in advance.

The ID is information for managing information registered in the management table.

The reference electroencephalogram is an example of reference biological information. Further, the reference brain wave is a brain wave that is estimated to be measured from a person. Although brain waves are used here as an example of the reference biological information, biological information other than brain waves may be used as the reference biological information.

A reference brain wave is, for example, a brain wave that is determined by statistical processing and is estimated to be generated by a person performing the operation content linked to the reference brain wave, or a brain wave that is estimated to be generated from a person requesting the execution of the operation content. These are estimated brain waves.

The reference brain wave may be a brain wave in a specific frequency band, or may be a brain wave including brain waves in a plurality of frequency bands.

In the example shown in FIG. 4, the operation target is the device 16, which is an example of hardware. Of course, the object to be operated may be hardware other than the device 16, or may be software.

The operation information is information including device identification information for identifying the device 16 to be operated and information indicating the details of the operation performed on the device 16. For example, the operation content may be an operation to turn on or off the power of the device 16, an operation to set the function level of the device 16, or an operation to turn the power on or off and a function level. It may be both the setting operation and the setting operation.

For example, the reference brain wave with ID "1" is a brain wave representing an operation of turning on the cooling of an air conditioner. The reference brain wave with ID "2" is a brain wave representing the operation content of turning off the cooling of the air conditioner.

For example, when a brain wave whose difference from the reference brain wave with ID "1" is within an acceptable range is measured from a person, the processor 26 "turns on the cooling of the air conditioner" linked to the reference brain wave. Specify the operation content. Then, the processor 26 transmits control information including information indicating the operation content of "turning on the cooling of the air conditioner" to the air conditioner. The air conditioner operates according to the control information. This turns on the cooling of the air conditioner.

Further, the processor 26 may calculate the degree of similarity between the brain waves measured from the person and the reference brain waves, and determine whether the degree of similarity is equal to or greater than a threshold value. The threshold value is a value corresponding to an acceptable range. If the degree of similarity between the brain waves measured from the person and the reference brain waves is greater than or equal to the threshold, the processor 26 determines that the brain waves measured from the person and the reference brain waves are similar. In other words, the processor 26 determines that the difference between the brain waves measured from the person and the reference brain waves is within the permissible range. When a brain wave whose similarity with the reference brain wave of ID "1" is equal to or higher than a threshold is measured from a person, the processor 26 specifies the operation content of "turning on the cooling of the air conditioner".

In the example shown in FIG. 4, the reference brain wave, which is an example of biological information, is registered in the management table, but emotional information, mental information, psychological information, etc. obtained by analyzing the biological information are not included in the reference brain wave. Alternatively, it may be registered in the management table. For example, information indicating emotions such as hot or cold, mental state, or psychological state may be registered in the management table as reference information instead of reference brain waves. The processor 26 identifies the emotion, mental state, or psychological state of the person by analyzing the brain waves measured from the person, identifies reference information indicating the identified emotion, mental state, or psychological state, and stores the information in the management table. In this step, the operation details associated with the specified reference information are specified.

For example, as shown in FIG. 38, the difference in brain waves in the psychological state of liking/disliking can be detected based on the difference in the corresponding brain wave data. In Figure 38, when a person is writing something they like, their brain waves stay within a certain range (for example, "like-zone"), and when a person is writing something they don't like, some of their brain waves are different. area (eg, a "dis-like zone").

As another example, as shown in FIG. 39, a psychological state of surprise can be detected. For example, a user may encounter a near-accident situation while driving. In this case, the brakes can be automatically applied and evasive action can be automatically taken based on the driving situation detected by the sensor or camera and the detected brain state of the user.

To explain with a specific example, standard information indicating the emotion, mental state, and psychological state of ``hot'' is registered in a management table in association with operation information indicating the operation content of ``turn on the cooling of the air conditioner.'' has been done. The processor 26 identifies the person's emotions, mental state, and psychological state by analyzing biological information (for example, brain waves) measured from the person. If the emotion or the like is “hot”, the processor 26 specifies the operation content “turn on the cooling of the air conditioner” that is linked to the reference information “hot”.

In this way, the processor 26 may specify the operation content based on the biological information itself, or specify the operation content based on various types of information obtained from the biological information (e.g., emotional information, mental information, psychological information, etc.). may be specified. The same applies to each of the following embodiments.

For each creature, the reference biological information and the operation information may be linked and registered in the management table. For example, biometric information measured from an individual person may be registered in the management table as the reference biometric information of that person. Regarding animals and living things other than humans, biometric information measured from each individual animal or living creature may be registered in the management table as reference biometric information of the animal or living creature.

FIG. 5 shows an example of a management table in which specific reference biological information of each person is registered. In the management table shown in FIG. 5, an ID, a reference electroencephalogram that is an example of reference biological information, operation information, and user information are linked. The user information is information for identifying a human user (eg, user name, user ID, etc.).

The reference electroencephalogram linked to user information is the electroencephalogram measured from the user when the user indicated by the user information performs the operation linked to the reference electroencephalogram, or the electroencephalogram when the user performs the operation linked to the reference electroencephalogram. This is the brain wave measured from the user when requesting content. Each reference brain wave measured from each user is measured from each user in advance and registered in the management table.

For example, when user A manually turns on the cooling of the "air conditioner," the brain waves of user A are measured by the biological information measuring device 12, and the measured brain waves are reflected when user A "turns on the cooling of the air conditioner." '' is registered in the management table as a reference brain wave representing the operation content. In this case, the measured reference brain waves of user A, operation information indicating the operation content of "turning on the cooling of the air conditioner", and user information for identifying user A are linked and stored in the management table. be registered. The registration may be performed using the information processing device 10 or may be performed using another device. In the example shown in FIG. 5, these pieces of information are registered as information with ID "1". The same applies to other operations and other users.

Note that the registration process may be performed multiple times, and the average of the multiple brain waves measured thereby may be registered as the reference brain wave. For example, user A manually turns on the cooling of an "air conditioner" and the brain waves generated by user A at that time are measured by the biological information measuring device 12 multiple times. The average brain wave may be registered in the management table as user A's reference brain wave.

For example, if user A is logged in to the information processing device 10 and a brain wave whose difference from the reference brain wave with ID "1" is within an allowable range is measured from user A, the processor 26 By transmitting control information including the operation information of "1" to the "air conditioner", the cooling of the "air conditioner" is turned on. To explain in more detail, when a brain wave is measured by the biological information measuring device 12 while the user A is logged in to the information processing device 10, the processor 26 collects user information for identifying the logged-in user A. Search for the reference electroencephalogram that is linked to and registered in the management table. In the example shown in FIG. 5, the reference brain waves with ID "1" and the reference brain waves with ID "3" are registered in the management table as user A's reference brain waves, so these reference brain waves are searched. If the difference between the measured brain waves and the reference brain waves with ID "1" is within the allowable range, the processor 26 transmits control information including the operation information with ID "1" to the "air conditioner". Turn on the air conditioner. If the difference between the measured brain waves and the reference brain waves with ID "3" is within the allowable range, the processor 26 transmits control information including the operation information with ID "3" to the "air conditioner". Turn off the air conditioner.

As another example, in a state where the information processing device 10 is set that the user who operates the device 16 is "user A", user A informs us that the difference from the reference brain wave of ID "1" is within the allowable range. If the brain waves included in the "air conditioner" are measured, the processor 26 may turn on the cooling of the "air conditioner" by transmitting control information including the operation information of ID "1" to the "air conditioner". To explain in more detail, when a brain wave is measured by the biological information measuring device 12 in a state where the information processing device 10 is set that the user who operates the device 16 is “user A”, the processor 26 The reference brain wave registered in the management table is searched for in association with user information for identifying user A, who operates 16. If the difference between the measured brain waves and the reference brain waves with ID "1" is within the allowable range, the processor 26 transmits control information including the operation information with ID "1" to the "air conditioner". Turn on the air conditioner. The user who operates the device 16 is set in the information processing device 10 by the user, for example.

For users other than user A, each piece of information is registered in the management table in the same way as for user A. For example, each piece of information associated with ID "2" is information related to the operation when user B turns on the cooling of the "air conditioner." Each piece of information associated with ID "3" is information related to the operation performed when user A turns off the cooling of the "air conditioner."

In the management table shown in FIGS. 4 and 5, operation information indicating the operation of turning on or off the power of the device 16 is registered, but operation information indicating the functional level of the device 16 is registered in the management table. may be done.

FIG. 6 shows another example of the management table. The management table shown in FIG. 6 is a management table in which reference biological information and operation information are registered in association with each other for each user. In the example shown in FIG. 6, as an example, brain waves and body temperature are used as reference biological information. Therefore, in the management table, the ID, a reference electroencephalogram that is an example of reference biological information, a reference body temperature that is an example of reference biological information, operation information, and user information are linked.

For example, if a brain wave whose difference from a reference brain wave with ID "1" is within an allowable range is measured from user A, and a body temperature that is equal to or higher than a threshold is measured from user A, the processor 26 The operation content indicated by the operation information is specified as the operation content for the device 16. Then, the processor 26 turns on the cooling of the air conditioner by transmitting control information including operation information indicating the operation details of ID "1" to the air conditioner.

In the example shown in FIG. 6, the reference body temperature, reference brain waves, and operation information are linked for each person, but even if the reference body temperature, reference brain waves, and operation information are linked, not for each person. good.

FIG. 7 shows another example of the management table. The management table shown in FIG. 7 is a management table in which reference electroencephalograms, reference environment information, and operation information are registered in association with each other for each user. Although a reference electroencephalogram is used as an example of the reference biological information, other biological information may be used as the reference biological information.

The reference environment information is information indicating an environment in which the reference brain waves associated with the reference environment information are estimated to be measured. Although room temperature is used as the reference environment information, other environmental information may be used as the reference environment information.

For example, if a brain wave whose difference from the reference brain wave with ID "1" is within an allowable range is measured from user A, and the room temperature at the time the brain wave was measured is equal to or higher than the threshold, the processor 26 The operation content indicated by the operation information "1" is specified as the operation content for the device 16. In this way, the operation details for the device 16 are specified based on the brain waves measured from the user and the room temperature at the time the brain waves were measured.

Note that in the example shown in FIG. 7, the reference biological information, reference environment information, and operation information are linked for each person, but the reference biological information, reference environment information, and operation information are linked, not for each person. May be attached.

Each example will be described below.

<Example 1>
Example 1 will be described below. In the first embodiment, biological information measured from a plurality of living things located at a certain location in real space is used. Here, as an example, an example will be described in which the device 16 is operated based on biological information measured from a plurality of people gathered in a certain room.

FIG. 8 schematically shows the room 36. Inside the room 36 are users A, B, and C, who are humans. Moreover, the equipment 16 is installed in the room 36. Note that the number of users and devices 16 is an example, and a plurality of devices 16 may be installed in the room 36.

The biometric information of each user is measured by the biometric information measuring device 12 . For example, users A, B, and C are each equipped with a biometric information measuring device 12 that is a wearable device, and the biometric information of users A, B, and C is measured by the biometric information measuring device 12 that is attached to each user. be measured. Even if the biometric information measuring device 12 is attached to each user, or the biometric information measuring device 12 is not attached to each user, and the biometric information of each user is measured by the biometric information measuring device 12 that is not attached to the user. good. One or more types of biometric information may be measured from each user. Here, as an example, it is assumed that brain waves are measured from each of users A, B, and C. Of course, biological information other than brain waves may be measured.

For example, biometric information (for example, electroencephalogram signals) of users in the room 36 is transmitted from each user's biometric information measuring device 12 to the information processing device 10 . In the example shown in FIG. 8, biometric information (for example, electroencephalogram signals) of users A, B, and C is transmitted from each user's biometric information measurement device 12 to the information processing device 10.

The processor 26 of the information processing device 10 operates the device 16 based on the brain waves of each of the users A, B, and C. For example, the processor 26 calculates the average of the brain waves measured from each of the users A, B, and C, and specifies the operation details of the device 16 based on the average brain waves. For example, the processor 26 refers to the management table shown in FIG. 4, identifies a reference brain wave whose difference from the average brain wave is within an allowable range, and connects the reference brain wave to the identified reference brain wave. Specify the operation content. The processor 26 then operates the device 16 according to the specified operation details. For example, the processor 26 calculates the average amplitude of the brain waves, specifies a reference brain wave whose difference from the brain wave having the average amplitude is within an acceptable range, and specifies the operation content.

As another example, if the operation content of the device 16 is to set the functional level, the processor 26 may separately identify the functional level of the device 16 based on the brain waves of each of users A, B, and C. . The processor 26 then calculates the average of each functional level and operates the device 16 according to the average functional level. For example, when the device 16 is an air conditioner with a cooling function, the operation content specified based on user A's brain waves is "set the cooling temperature of the air conditioner to 26 degrees Celsius". , the operation content specified based on user B's brain waves is the operation content "setting the cooling temperature of the air conditioner to 27 degrees Celsius", and the operation content specified based on user C's brain waves is, If the operation content is "set the cooling temperature of the air conditioner to 28° C.", the processor 26 calculates the average of these temperature settings. Then, the processor 26 sets the cooling set temperature of the air conditioner to the average temperature.

If the operation content of the device 16 is to turn the device 16 on or off, the processor 26 may separately identify whether the device 16 is on or off based on the brain waves of each of users A, B, and C. . Then, the processor 26 determines whether the device 16 should be turned on or off according to the majority vote of the users A, B, and C, and turns the device 16 on or off according to that decision.

(Operation based on distance between device 16 and user)
The processor 26 of the information processing device 10 specifies the operation content of the device 16 based on the user's biological information (for example, an electroencephalogram signal) whose distance to the device 16 to be operated is equal to or less than a threshold value, and controls the device 16 according to the operation content. 16 may be operated. The location of the device 16 and each user is specified, for example, by using GPS (Global Positioning System) or other communication technology, and the location information of the device 16 and each user is transmitted to the information processing device 10. For example, when a user carries the biometric information measuring device 12 or the terminal device 18, the location of the biometric information measuring device 12 or the terminal device 18 is specified using GPS or the like, and the location information indicating the location is The information is transmitted to the information processing device 10 as location information indicating the location of the user.

For example, if the distance between users A and B and the device 16 is less than or equal to the threshold, and the distance between user C and the device 16 exceeds the threshold, the processor 26 of the information processing device 10 The average brain wave of B is calculated, the operation content of the device 16 is specified based on the average brain wave, and the device 16 is operated according to the specified operation content. Even if the user C's brain wave signal is being transmitted to the information processing device 10, the processor 26 identifies the operation content of the device 16 without using the user C's brain wave. By doing so, the device 16 can be operated based on the biometric information of the user whose distance to the device 16 is equal to or less than the threshold value.

(Operation based on distance between information processing device 10 and user)
The processor 26 of the information processing device 10 specifies the operation details of the device 16 based on the biometric information of the user whose distance to the information processing device 10 is equal to or less than a threshold value, and operates the device 16 according to the operation details. good. For example, when the information processing device 10 is a device such as a smart speaker and is installed in a room 36, the processor 26 uses the biometric information of the user whose distance to the information processing device 10 is less than or equal to a threshold value. The device 16 may also be operated by the operator.

(Weighting process)
The processor 26 of the information processing device 10 may perform weighting processing on each user's brain waves, calculate the average of the weighted brain waves, and specify the operation details of the device 16 based on the average brain waves.

For example, a weighting coefficient is determined for each user, and information indicating the weighting coefficient of each user is stored in the memory 24 of the information processing device 10. For example, each user may determine his or her own weighting coefficient, or a user selected from users A, B, and C may determine each user's weighting coefficient. Information indicating the weighting coefficient of each user may be transmitted to the information processing device 10 from the terminal device 18 of each user, or may be input to the information processing device 10 using the UI 22 of the information processing device 10, for example. .

The weighting coefficient for each user may be determined based on the user's attributes. The attributes are, for example, age, gender, height, or weight. A weighting coefficient for each attribute is determined in advance, and information indicating the weighting coefficient for each attribute is stored in advance in the memory 24 of the information processing device 10. For example, information indicating the attributes of each user is transmitted from the terminal device 18 of each user to the information processing apparatus 10, and the processor 26 of the information processing apparatus 10 determines the weight of each user based on the information indicating the attributes of each user. Determine the coefficients.

Further, the weighting coefficient may be determined based on the distance between the device 16 and the user. For example, the shorter the distance between the device 16 and the user, the larger the weighting coefficient, and the longer the distance, the smaller the weighting coefficient. Of course, the opposite may be true. The distance is calculated based on the respective position information of the device 16 and the user.

(If biological information changes)
When the brain waves of users A, B, and C change, the processor 26 of the information processing device 10 changes the operation details of the device 16 in accordance with the change. For example, the processor 26 calculates the average of the brain waves of users A, B, and C for each predetermined period, specifies the operation content for each period based on the average, and calculates the operation content for each period according to the specified operation content. The device 16 is operated each time. In this case as well, the weighting process described above may be performed.

In Example 1, the plurality of living things whose biological information is measured may be a combination of at least two of humans, non-human animals, and plants. For example, the device 16 may be operated based on biometric information of a person and a living thing. The same applies to the embodiments described below.

(Treatment when living organisms move)
Hereinafter, with reference to FIG. 9, a process when a creature moves will be described. In FIG. 9, a room 36 is schematically shown. Similar to the example shown in FIG. 8, there are users A, B, and C, who are humans, in the room 36. Moreover, the equipment 16 is installed in the room 36.

For example, as indicated by the arrow in FIG. 9, user A moves, the distance between user A and device 16 exceeds the threshold, and the distance between users B and C and device 16 becomes less than or equal to the threshold. In some cases, the processor 26 of the information processing device 10 specifies the operation content based on the brain waves of the users B and C, and operates the device 16 according to the specified operation content. As described above, the processor 26 may specify the operation content based on the average of the brain waves of the users B and C, and operate the device 16 according to the specified operation content, or may operate the device 16 based on the average of the brain waves of the users B and C. The functional level may be specified based on the function level, and the device 16 may be operated according to the average of the specified functional level, or the device 16 may be turned on or off based on the respective brain waves of users B and C, and the device 16 may be operated according to that determination. may be turned on or off.

The processor 26 may perform the weighting process described above. In this case, if the user A before moving is a user who influences the decision of the operation content (for example, if the weighting coefficient of user A is larger than the weighting coefficient of other users), the user A cannot move. Therefore, the operation details of the device 16 may change.

Note that even when biological information of an animal or living creature other than a human being is measured, if the non-human animal moves or the placement position of the living creature changes, the operation details of the device 16 may be changed accordingly. There is.

<Example 2>
Example 2 will be described below. In the second embodiment, biological information measured from a plurality of living things located at different locations is used. Here, as an example, an example will be described in which the device 16 is operated based on biological information measured from people in different locations (for example, locations that are not in the same room). The second embodiment is applied to, for example, a situation where a plurality of people have an online conversation or a meeting (for example, a remote meeting, a web meeting, etc.).

For example, assume that users A, B, C, and D are participating in an online conference. User A uses terminal device 18A, user B uses terminal device 18B, user C uses terminal device 18C, and user D uses terminal device 18D. Terminal devices 18A, 18B, 18C, and 18D have the same configuration as terminal device 18.

For example, the terminal devices 18A, 18B, 18C, and 18D receive services from a server that provides online conversation and conference services, and exchange information (e.g., images, sounds, character strings) with each other via the server. etc.) can be exchanged. As such a service, for example, a known service is used.

Embodiment 2 will be described in detail below with reference to FIG. FIG. 10 shows screen 38A. The screen 38A is a screen displayed on the display of the UI 30 of the user A's terminal device 18A.

Images of users A, B, C, and D participating in the online conference are displayed on the screen 38A. For example, screen 38A includes areas 40A, 40B, 40C, and 40D. An image of user A is displayed in area 40A, an image of user B is displayed in area 40B, an image of user C is displayed in area 40C, and an image of user D is displayed in area 40D. A screen having the same configuration as the screen 38A is also displayed on each display of the terminal devices 18B, 18C, and 18D.

For example, user A is photographed by a camera installed in user A's terminal device 18A or by a camera not installed in terminal device 18A, and the processor 34 of terminal device 18A stores the image generated by the photographing in an area. Display on 40A. Further, the data of the image of user A is transmitted from the terminal device 18A to the terminal devices 18B, 18C, and 18D via the server, and the image of user A is displayed on the respective displays of the terminal devices 18B, 18C, and 18D. Similarly, the same applies to each of the images of users B, C, and D.

As in the first embodiment, the biometric information of each user is measured by the biometric information measuring device 12. Here, as an example, it is assumed that the brain waves of each of users A, B, C, and D are measured similarly to the first embodiment. Of course, biological information other than brain waves may be measured. The biometric information (for example, electroencephalogram signals) of each of the users A, B, C, and D is transmitted from the biometric information measuring device 12 of each user to the information processing device 10 .

The processor 26 of the information processing device 10 operates the device 16 based on the brain waves of each of the users A, B, C, and D. The device 16 to be operated may be the terminal devices 18A, 18B, 18C, or 18D, or may be a device other than the terminal device 18.

Each terminal device 18 may have the function of the information processing device 10, and the processor 34 of each terminal device 18 may operate the device 16 to be operated. For example, each terminal device 18 receives biometric information from each user's biometric information measuring device 12 via a communication path and operates the device 16 .

For example, the processor 34 of each terminal device 18 controls the volume of the speaker provided in the terminal device 18 based on each user's brain waves. For example, if user B's brain waves are analyzed and it is determined that user B is excited, the processor 34 of each terminal device 18 sets the speaker of each terminal device 18 so that the volume of user B is reduced. control the volume. For example, the processor 34 of the terminal device 18A controls the volume of the speaker of the terminal device 18A so that the user B's volume becomes low. The same applies to other terminal devices 18. Note that when a speaker not provided in the terminal device 18 is used, the processor 34 controls the volume of the speaker.

Note that users participating in the online conference may be authenticated. Users who are successfully authenticated can participate in the online conference. For example, authentication techniques such as face authentication, voice authentication, retinal authentication, or ear acoustic authentication are used. Authentication may be performed by the information processing device 10 or by an authentication server.

In the second embodiment as well, the processor 26 of the information processing device 10 may operate the device 16 based on each user's brain waves (for example, based on the average of the brain waves). The device 16 to be operated may be a device used by at least one of the users A, B, C, and D, or may be another device. In this case, weighting processing may be performed.

(Treatment when living organisms move)
Hereinafter, with reference to FIG. 11, a process when a creature moves will be described. FIG. 11 shows the screen 38A.

For example, if user B who was participating in the online conference no longer participates in the online conference or if the image of user B is no longer displayed in the area 40B, the processor 26 of the information processing device 10 , D is operated based on the brain waves of each of them.

<Example 3>
Example 3 will be described below with reference to FIG. 12. Example 3 is a combination of Examples 1 and 2. FIG. 12 shows the screen 38A.

For example, users A1 and A2 are participating in an online conference using the terminal device 18A, users B1 and B2 are participating in the online conference using the terminal device 18B, and user C1 is using the terminal device 18C. Users D1, D2, and D3 are participating in the online conference using the terminal device 18D. For example, users A1 and A2 belong to group A, users B1 and B2 belong to group B, user C1 belongs to group C, and users D1, D2, and D3 belong to group D. Information regarding the group (for example, identification information of the group, identification information of each user belonging to the group, etc.) is stored in the information processing device 10, a server, or the like.

Images of users A1 and A2 are displayed in area 40A, images of users B1 and B2 are displayed in area 40B, images of user C1 are displayed in area 40C, and images of users D1, D2, and D3 are displayed in area 40D. be done.

As in the first embodiment, the biometric information of each user is measured by the biometric information measuring device 12. Note that the biometric information of one or more users participating in the online conference may not be measured. Here, as an example, it is assumed that brain waves are measured from each user similarly to the first embodiment. Of course, biological information other than brain waves may be measured. Each user's biological information (for example, an electroencephalogram signal) is transmitted from each user's biological information measurement device 12 to the information processing device 10.

Similar to the second embodiment, the processor 26 of the information processing device 10 operates the device 16 based on each user's brain waves. The device 16 to be operated may be the terminal devices 18A, 18B, 18C, or 18D, or may be a device other than the terminal device 18. Each terminal device 18 may have the function of the information processing device 10, and the processor 34 of each terminal device 18 may operate the device 16 to be operated.

For example, similar to the second embodiment, the processor 34 of each terminal device 18 controls the volume of the speaker provided in the terminal device 18 based on each user's brain waves. For example, if it is determined that the user B1 is excited by analyzing the brain waves of the user B1, the processor 34 of each terminal device 18 sets the speaker of each terminal device 18 so that the volume of the user B1 is reduced. control the volume.

As another example, the processor 34 of each terminal device 18 may calculate the average of the brain waves of one or more users belonging to the group, and determine the state of the group based on the average. For example, if it is determined that users B1 and B2 are excited by analyzing the average brain waves of users B1 and B2 belonging to group B, the processor 34 of each terminal device 18 controls the volume of the users B1 and B2. The volume of the speaker of each terminal device 18 is controlled so that the volume of the speaker becomes small.

In the third embodiment as well, the processor 26 of the information processing device 10 may operate the device 16 based on each user's brain waves (for example, based on the average of the brain waves). The processor 26 may also operate the device 16 based on the brain waves of one or more users belonging to the group (eg, based on the average of the brain waves of one or more users belonging to the group).

(Treatment when living organisms move)
Hereinafter, with reference to FIG. 13, the process when the living thing moves will be described. FIG. 13 shows a screen 38A.

For example, if the user C1 who was participating in the online conference no longer participates in the online conference, or if the image of the user C is no longer displayed in the area 40C, the processor 26 of the information processing device 10 , D, the device 16 is operated based on the brain waves of the users belonging to each of the groups.

<Example 4>
Example 4 will be described below. In the fourth embodiment, processing is performed to encourage people to move.

For example, the processor 26 of the information processing device 10 obtains emotional information, mental information, and psychological information of the person by analyzing the person's biological information. When a plurality of types of biometric information are measured, the processor 26 may obtain emotional information, mental information, or psychological information of the person by analyzing all or part of the plurality of biometric information. The processor 26 may perform weighting processing on each piece of biometric information based on the priority order of each piece of biometric information, and obtain emotional information, mental information, or psychological information of the person based on the result of the weighting process. Further, reference levels are determined for various types of biological information, and the processor 26 may determine that biological information having the same reference level indicates the same tendency in emotion, mental state, or psychological information.

The processor 26 prompts the person to move to an optimal location based on the results of the biometric information analysis. For example, if the processor 26 determines that the person is feeling uncomfortable based on the results of the analysis of the biometric information, the processor 26 prompts the person to move to a place where the person can feel comfortable. A place where you feel comfortable is an example of an optimal place.

The processor 26 identifies an optimal place (for example, a place where a person can feel comfortable) based on the biological information measured by the biological information measuring device 12 and the environmental information measured by the environmental information measuring device 14. .

To explain with a specific example, when the processor 26 determines that the person is feeling hot based on the person's biometric information, the processor 26 prompts the person to move to a cooler place. A cool place is an example of an optimal place. For example, the processor 26 analyzes the amount of perspiration and brain waves of the person, and determines that the person is feeling hot if the amount of perspiration is above a threshold and the brain waves indicate discomfort. For example, the environmental information measuring device 14 measures environmental information such as temperature, humidity, and air volume at each location, and the environmental information at each location is transmitted from the environmental information measuring device 14 to the information processing device 10 . The processor 26 identifies locations where the person may feel cool based on the environmental information of each location. For example, the processor 26 identifies a location where the temperature is lower than the temperature of the current location of the person, and prompts the person to move to the identified location.

Further, when the processor 26 determines that the person feels cold based on the person's biometric information, the processor 26 urges the person to move to a warm place. In this case, a warm place is an example of an optimal place.

For example, the processor 26 prompts each individual to move to an optimal location. For example, as shown in FIG. 8, when users A, B, and C are in the room 36, the processor 26 prompts each of the users A, B, and C to move to the optimal location for each user. The processor 26 may guide users whose biometric information has the same tendency to the same location. For example, if users A and B feel hot and user C feels cold, processor 26 guides users A and B to a cooler location and guides user C to a warmer location.

As another example, when a person is looking at an image or text displayed on a display, the processor 26 determines that the person feels that the image or text is difficult to see based on the person's biometric information. If determined, the person is prompted to approach the display.

As yet another example, when sound is being emitted from a speaker, if the processor 26 determines that the person feels that the sound is difficult to hear based on the person's biological information, it may cause the processor 26 to move closer to the speaker. Encourage the person.

For example, the processor 26 transmits information encouraging movement (for example, information indicating guidance or guidance) to the terminal device 18 of the person. The processor 34 of the terminal device 18 causes the information to be displayed on the display of the UI 30. FIG. 14 shows an example of the display. The screen 42 is a screen displayed on the display of the terminal device 18. For example, a message such as "Please move to XX (comfortable location)" and a map 44 showing the location are displayed on the screen 42. On the map 44, for example, a mark 46 representing the current position of the person, a mark 48 representing a comfortable place, and an arrow 50 guiding the person to the comfortable place are displayed. The person's current location and comfortable location are determined using technology such as GPS. In the example shown in FIG. 8, information prompting users to move is displayed on the display of each terminal device 18 of users A, B, and C.

FIG. 15 shows another display example. A room 36 is shown in FIG. For example, the processor 26 of the information processing device 10 may display guidance using light or the like on the floor of the room 36, as indicated by the reference numeral 52. For example, the processor 26 displays guidance by controlling lighting installed in the room 36. If the place where there is a person is a road or the like, the processor 26 displays a guide on the ground or the like. In addition, the processor 26 may display guidance on a wall or may guide a person by voice. The sound may be emitted from the terminal device 18 or may be emitted from another speaker. Furthermore, the processor 26 may display guidance on displays placed around the person.

The processor 26 may guide a plurality of people whose biometric information shows the same tendency to the same place, and guide a plurality of people whose biometric information shows different trends to different places. For example, processor 26 may guide people who like each other to the same location, and guide people who dislike each other to different locations.

<Example 5>
Example 5 will be described below. In Example 5, biological information of living things is displayed. Example 5 will be described in detail with reference to FIG. 16. FIG. 16 shows a screen 38A. Here, as an example, it is assumed that an online conference is being held by a plurality of users, similar to the third embodiment.

For example, each user's biometric information is transmitted to each user's terminal device 18, and the processor 34 of the terminal device 18 causes each user's biometric information to be displayed on a display. FIG. 16 shows a screen 38A of the terminal device 18A of user A, and the processor 34 of the terminal device 18A displays each user's biometric information on the screen 38A. In FIG. 16, brain waves, which are an example of biological information, are displayed. For example, the waveform of an electroencephalogram is displayed.

In the example shown in FIG. 16, user A1's brain waves 54, user B1's brain waves 56, user C1's brain waves 58, and user D3's brain waves 60 are displayed. For example, the processor 34 of the terminal device 18A displays the brain waves 54 of the user A1 in association with the image of the user A1 in the area 40A. The same goes for other users' brain waves.

The user can select whether to display or hide the brain waves. The processor of the terminal device 18A may, for example, display the brain waves of each user according to the instructions of the user of the terminal device 18A, or may not display the brain waves of each user, or may display the brain waves of each user according to the instructions of the user whose brain waves were measured. , the brain waves of the user may be displayed, or the brain waves of the user may not be displayed.

For example, when the user of the terminal device 18A instructs to display the brain waves of the users A1, B1, C1, and D3, the processor 34 of the terminal device 18A displays the brain waves of the users A1, B1, and D3, as shown in FIG. The brain waves of C1 and D3 are displayed.

As another example, the user A1 may set display or non-display of the user A1's own brain waves. When display is set, user A1's brain waves are displayed, and when non-display is set, user A1's brain waves are not displayed. The same applies to other users.

Further, the processor 34 of the terminal device 18 may display the waveform of the brain waves, and may also display information indicating emotions, mental state, and psychological state obtained by analyzing the brain waves. For example, the brain wave 58 represents a state of "awakening (excitement)," and the processor 34 displays a character string indicating the "awakening (excitement)" together with the brain wave 58 in the area 40C. The same applies to other brain waves.

The processor 26 of the information processing device 10 may convert the audio data during the online conference into text data, associate the text data with each user's biological information (for example, brain waves), and store the text data in the memory. Further, the processor 26 may store the audio data in the memory in association with each user's biometric information. Further, the processor 34 of the terminal device 18 may obtain the text data and display the text. For example, it may be possible to use biometric information as a search key to search for audio data or text data linked to the biometric information.

Further, the processor 34 of the terminal device 18 may display brain waves representing predetermined emotions, mental states, psychological states, and intention information. For example, if brain waves unsuitable for a meeting (for example, brain waves indicating drowsiness or sleep) are measured, the processor 34 displays the brain waves in association with the image of the user whose brain waves were measured.

An image representing emotion, mental state, psychological state, or intention information may be displayed. FIG. 17 shows an example of the display. For example, an image 62 representing concentration, an image 64 representing sleepiness, an image 66 representing relaxation, etc. are displayed. For example, the processor 34 of the terminal device 18 displays each image in association with the user, together with or instead of the brain waves.

The processor 34 of the terminal device 18 may change the number, color, emphasis, type of images, etc. of images depending on the emotion, mental state, psychological state, and degree of intention information.

The processor 34 may display a score according to the degree of emotion, mental state, psychological state, and intention information in association with the user.

The processor 34 may process and display the user's image itself (for example, an image of the user's face), or may process and display the user's background image, depending on the level of emotion, mental state, psychological state, and intention information. You may change it. For example, when the user's feelings are negative, the processor 34 displays the color of the user's face as white or pale, and when the user's feelings are positive, the processor 34 displays the skin color of the user's face as bright. The processor 34 may darken and display the background image when the user's feelings are negative, and may brighten the background image when the user's feelings are positive.

<Example 6>
Embodiment 6 will be described below with reference to FIG. 18. FIG. 18 schematically shows a tomographic image of a human brain.

For example, a person's brain waves are measured by the biological information measuring device 12, and the processor 26 of the information processing device 10 identifies the activated part of the person's brain based on the measured brain waves. For example, by providing electrodes at three or more locations on a person's head and measuring brain waves, the processor 26 identifies the location where the measured brain waves are occurring. The processor 26 may display a mark or the like pointing to the identified region on the tomographic image of the person's brain, or may fill out the identified region.

Note that the tomographic image may be an image taken by a CT (Computed Tomography) device or an MRI (Magnetic Resonance Imaging) device, or an image that schematically represents the brain (for example, an image created in advance). Good too.

<Example 7>
The screen on which biometric information is displayed will be described below. FIG. 19 shows an example of the home screen.

Home screen 100 is displayed on the display of terminal device 18, for example. The home screen 100 displays a "Scan" button, a "Record" button, a "Reset" button, a "File" button, and an "Edit" button. The screen 100 also includes display areas 102, 104, and 106. In the example shown in FIG. 19, the regions 102, 104, and 106 are arranged vertically in that order, but this arrangement is only an example. The arrangement may be changed by the user.

When the "Scan" button is pressed, a list of biological information measuring devices 12 that can communicate with the terminal device 18 (for example, can be paired with BLE, etc.) is displayed on the home screen 100. When the "Record" button is pressed, the biological information measured by the biological information measuring device 12 is stored in the memory. For example, the biological information measuring device 12 and the terminal device 18 communicate, and the biological information measured by the biological information measuring device 12 is transmitted from the biological information measuring device 12 to the terminal device 18 and stored in the memory 32 of the terminal device 18. be remembered. When the "Reset" button is pressed, the calibration settings are reset. When the "File" button is pressed, a screen for managing biometric information files is displayed. When the "Edit" button is pressed, a setting screen (see FIG. 23), which will be described later, is displayed.

In the area 102, the biological information itself measured by the biological information measuring device 12 is displayed, and in the area 104, information obtained by processing the biological information is displayed. Hereinafter, the unprocessed biological removal method will be referred to as "Raw data." For example, the area 102 displays a graph of raw data (hereinafter referred to as "graph 1"), and the area 104 displays a graph of processed biological information (hereinafter referred to as "graph 2"). Ru.
In the area 106, setting values of the graph displayed in the area 104, etc. are displayed.

The vertical axis (Y-axis) of graph 1 is a potential (for example, [μV]) representing biological information, and the horizontal axis (X-axis) is time. For example, the range of the vertical axis can be changed by pinching or zooming.

The vertical axis (Y-axis) of the graph 2 before switching is a potential (for example, [μV]) representing biological information, and the horizontal axis (X-axis) is time. For example, the range of the vertical axis can be changed by pinching or zooming.

The display contents of the areas 104 and 106 can be switched to other display contents. For example, when an operation to slide the areas 104, 106 is performed, such as a flick operation, the display contents are switched. Of course, the display contents may be switched by another operation.

FIG. 20 shows the display contents after switching.

The graph 2 before switching is, for example, a graph (for example, a real-time graph) generated by performing processing set on a setting screen to be described later on raw data. In the area 106 before switching, setting values and the like related to the graph are displayed. For example, an index that shows that biological information (e.g., brain waves) is being acquired, an index that shows that the person whose biological information is being measured is concentrated, an index that shows that the person whose biological information is being measured is relaxed. The index and the value of the 6-axis sensor installed in the biological information measuring device 12 (for example, the value indicating the amount of movement in the axial direction of the X, Y, and Z axes, the amount of rotation around the X, Y, and Z axes) etc.) are displayed in the area 106. Each value displayed in the area 106 is updated, for example, at every predetermined time (for example, every second). Furthermore, an image having a color corresponding to each value may be displayed.

The graph 2 after switching is, for example, a graph generated by performing FFT (Fast Fourier Transform) processing on Raw data (hereinafter referred to as an "FFT graph"). In the area 106 after switching, setting values regarding graph 1, graph 2, and FFT graph are displayed. For example, the item “Graph X”, the item “Graph Y”, the item “FFT Max X”, and the item “FFT Max Y” are displayed in the area 106. The item "Graph X" is an item for setting the scale value of the X axis common to the graph 1 and the FFT graph. The item "Graph Y" is an item for setting the scale values of the Y axis of graphs 1 and 2 separately. The item "Graph1" is an item for setting the value of the Y-axis scale of Graph 1, and the item "Graph2" is an item for setting the value of the Y-axis scale of Graph 2. The item "FFT Max X" is an item for setting the maximum value of the X axis of the FFT graph. The item "FFT Max Y" is an item for setting the maximum value of the Y axis of the FFT graph.

FIG. 21 shows specific examples of a raw data graph 1 and a graph 2 before switching. FIG. 22 shows a specific example of graph 2 after switching (graph after FFT processing).

FIG. 23 shows an example of a setting screen. When the "Edit" button is pressed on the home screen 100, a settings screen 110 shown in FIG. 23 is displayed.

The item "Micro Volts" is an item for setting the Y-axis display of graphs 1 and 2 to "μV".

The item "Cut Off Filter" is an item for setting setting values of a filter (for example, a high pass filter (HPF) and a low pass filter (LPS)) applied to biometric information. Filter processing is applied to the biometric information according to the setting values set here. For example, filter processing is applied to graphs 1 and 2. The filtering process may be performed by the biological information measuring device 12, the information processing device 10, or the terminal device 18. By performing filter processing, information having a specific frequency can be extracted from the measured biological information. For example, from biological information that includes brain waves and other myoelectric signals, extract only brain waves or extract other myoelectric signals (for example, signals caused by eye movements, facial movements, jaw movements, etc.) You can do it. The device 16 may be operated based on the extracted signal. For example, the processor 26 of the information processing device 10 may distinguish and extract brain waves and other myoelectric signals, and operate the same device 16 based on the brain waves and other myoelectric signals, or may operate different devices 16. may be operated separately.

The item "Peak Filter" is an item for setting values related to processing for peak values of Raw data. For example, an item for setting that it is impossible to measure biological information (e.g., brain waves) when the peak value of raw data is equal to or higher than a set value, and a setting value for measuring impulse noise are set. Items for this are included in the item "Peak Filter."

<Example 8>
Example 8 will be described below. In the eighth embodiment, the processor 26 of the information processing device 10 estimates the emotions, mental state, and psychological state of another person who has a specific relationship with the person based on the person's biological information. The processor 26 may infer another person's emotions, mental state, or psychological state based on the person's emotional information, mental information, or psychological information.

The specific relationship is, for example, a parent-child relationship or a blood relationship. For example, the processor 26 may estimate the child's emotions, mental state, or psychological state based on the biological information, emotional information, mental information, or psychological information of the parent, or may estimate the child's biological information, emotional information, or mental state. It is also possible to infer the emotions, mental state, and psychological state of a parent based on a person's biological, emotional, mental, and psychological information. You may also infer their mental or psychological state.

For example, if the biological information of the parent and the biological information of the child are measured by the biological information measuring device 12, and the processor 26 of the information processing device 10 detects that the biological information of the child is similar to the biological information of the parent (for example, If the child's brain waves have a waveform similar to the parent's brain waves), it is inferred that the child's emotions, mental state, and psychological state are the same as the parent's emotions, mental state, and psychological state. For example, in a case where the parent's biometric information expresses the emotion of "feeling good," and the child's biometric information is similar to the parent's biometric information, the processor 26 may cause the child to also express the emotion of "feeling good." It is assumed that it has. For example, if the child is an infant or young child who has difficulty controlling their emotions, mental state, or psychological state, the above estimation is performed.

As another example, if the parent is a person who has difficulty controlling their emotions, mental state, or psychological state (for example, a person suffering from dementia, etc.), the processor 26 may collect the child's biological information, emotional information, etc. Parents' emotions, mental state, and psychological state may be inferred based on mental information and psychological information.

As yet another example, the processor 26 may estimate the emotions, mental states, and psychological states of people who have similar lifestyle habits. For example, people who have lived together for a long time may have similar lifestyle habits, and in that case, the processor 26 may determine whether the person is Inferring the emotions, mental state, and psychological state of other people who have similar lifestyle habits.

<Example 9>
Example 9 will be described below. In the ninth embodiment, the processor 26 of the information processing device 10 informs the human subject of specific information that the subject cannot know. For example, the processor 26 transmits the specific information to the terminal device 18 of the subject.

The specific information to be informed to the subject includes, for example, visual information about a specific place, environmental information, information about the atmosphere, and the like. For example, information that cannot be known unless one is in the specific location is information that is made known to the target person. The location is specified, for example, by the administrator of the information processing device 10, the target person, or the like. Further, even if information indicating a danger that has occurred at a particular location or a danger that may occur at a particular location is sent to the terminal device 18 of the target person as specified by the administrator or the target person, etc. good.

The visual information is, for example, an image captured by a fixed point camera installed at a specific location, an image captured by a camera worn by a person at a specific location, or the like. The visual information may include sound information measured by a microphone installed at a specific location or a microphone worn by a person. Environmental information includes information measured by various sensors installed in a specific location (e.g., temperature, humidity, air chambers, weather, etc.) or by sensors worn by people in a specific location. information etc. Information regarding the atmosphere includes, for example, information indicating that a specific place is a comfortable or unpleasant place for a person, information indicating that a specific place is a place where a person can concentrate or a place that is distracting. For example, when a person in a specific location inputs information regarding the atmosphere of the specific location into his or her terminal device 18, the input information is transmitted from the terminal device 18 to the information processing device 10.

For example, when a target person uses the terminal device 18 to specify a specific location and requests the acquisition of specific information at that specific location, information indicating the request is sent from the terminal device 18 to the information processing device 10. Sent. In response to the request, the processor 26 of the information processing device 10 transmits specific information at the designated specific location to the terminal device 18. The processor 34 of the terminal device 18 causes the specific information to be displayed on a display or generated as audio from a speaker.

Further, the processor 26 of the information processing device 10 observes the situation at each location, and if the situation at a specific location designated by the subject corresponds to a predetermined condition, the processor 26 of the information processing device 10 provides specific information at the specific location. may be transmitted to the terminal device 18 of the subject. The situation of a place includes, for example, the environment-related situation obtained by analyzing environmental information (e.g., temperature, humidity, weather, etc.), the situation obtained from visual information (e.g., the congestion situation of the city, trains, etc.), , dangerous situations obtained from visual information, environmental information, etc. The situation of a place may be inferred or predicted from visual information, environmental information, or the like.

For example, if the weather at a specific place designated by the subject corresponds to predetermined weather, or if it is predicted that the weather will become predetermined, the processor 26 of the information processing device 10 Information regarding the weather at the location is transmitted to the target person's terminal device 18. Further, when the congestion situation at a specific place becomes a predetermined congestion situation or is predicted to become a predetermined congestion situation, the processor 26 of the information processing device 10 determines the congestion situation at the specific place. information regarding the subject is transmitted to the terminal device 18 of the subject.

Furthermore, the processor 26 of the information processing device 10 may select specific information to be transmitted to the terminal device 18 of the subject based on the subject's purpose (for example, avoiding a crisis). The purpose is specified by the subject, for example.

For example, when a car is running out of control, if a person uses the terminal device 18 to instruct the information processing device 10 to send information about the car that is running out of control, information about the car (for example, dangerous (information including information indicating) is transmitted from the terminal device 18 to the information processing device 10. The information regarding the vehicle includes location information of the terminal device 18. Further, when a car running out of control is photographed by a camera worn by a certain person, the information regarding the car may include the photographed image. When the processor 26 of the information processing device 10 receives the information regarding the vehicle, the processor 26 of the information processing device 10 converts images taken by fixed-point cameras installed around the location of the terminal device 18 (for example, images taken from a higher position) to the relevant information. The image is acquired from a fixed point camera and transmitted to the terminal device 18 of the subject. The image is displayed on the display of the target person's terminal device 18. The processor 26 may also transmit information indicating the danger to the subject's terminal device 18. Information indicating the danger is displayed on the display of the target person's terminal device 18. Further, when a plurality of fixed point cameras are installed, the processor 26 may transmit an image taken by the fixed point camera most effective for detecting danger to the terminal device 18 of the subject.

Further, the processor 26 selects information included in specific information or processes specific information depending on the attributes of the target person (for example, age, gender, medical history, etc.), and targets specific information. The information may also be transmitted to the terminal device 18 of the user. For example, because differences in visual function may occur depending on age (e.g., differences in visual function may occur due to cataracts, glaucoma, etc.), processor 26 may determine whether certain information is included depending on the age of the subject. The clarity of the image, etc. may be adjusted. Furthermore, since there may be differences in sense of crisis depending on gender and age, the processor 26 may select information included in the specific information depending on the gender and age of the subject. For example, the processor 26 may include only an image in the specific information, or may include a character string (for example, a character string indicating a crisis) in the specific information, depending on the gender and age of the subject. , specific information may include both images and text strings.

<Example 10>
Example 10 will be described below.

For example, biological information of a plant may be measured by the biological information measuring device 12, and the device 16 may be operated based on the measured biological information. The biological information to be measured is, for example, the potential of the plant (eg, potential including membrane potential), water content, etc. Of course, information other than these may be measured as biological information.

For example, when biometric information of a plant is being measured, if playing music near the plant improves the growth of the plant compared to when no music is played, the processor of the information processing device 10 26 operates the device 16 such as a music player so that music is played from the device 16 so that biological information (for example, electric potential) measured while playing music is continuously measured from the plant. . The processor 26 may also control how water and fertilizer are given to plants. For example, processor 26 may provide water or fertilizer to the plant so that biometric information is continuously measured from the plant.

Further, the processor 26 detects a change in the state of the plant based on biological information measured from the plant, and transmits information (for example, an image, a character string, etc.) corresponding to the change to a device such as the terminal device 18. It may also be displayed on a display. The processor 26 may anthropomorphize the plant and display information corresponding to the change on the display.

The processor 26 may determine whether or not the plant has a disease based on the biological information measured from the plant, and display the result of the determination on a display of a device such as the terminal device 18. .

Furthermore, the water content of living organisms changes over time, and this change is reflected in the biological potential. The biopotential of the living organism may be measured by the bioinformation measuring device 12, and the processor 26 of the information processing device 10 may detect the behavior or abnormality of the living creature based on the measured biopotential. For example, if an abnormality is detected, the processor 26 may transmit warning information indicating the abnormality to a device such as the terminal device 18. For example, when a person's biopotential is measured and an abnormality of the person is detected based on the biopotential, warning information is sent to the terminal device 18 of the person and the warning information is displayed on the display of the terminal device 18. be done.

In addition, in the morning, water is generally distributed evenly throughout a person, and during the day, when a person is active, water gradually increases in the lower part of the body. Changes in water content are reflected in biopotential. The processor 26 of the information processing device 10 may measure the biopotential of each part of the person and measure time based on changes in the biopotential. For example, changes in the biopotential may be used as a clock or timer.

The processor 26 of the information processing device 10 may also measure biological information (e.g., brain waves) of an animal other than a human and infer the state of the animal (e.g., illness) or the environment in which the animal is placed. good.

Further, the processor 26 of the information processing device 10 may detect the type of living creature wearing the biological information measuring device 12.

For example, humans and non-human animals have different skull sizes. When the biometric information measuring device 12 has a band for attaching the biometric information measuring device 12 to the head, and the biometric information measuring device 12 is attached to the head by wrapping the band around the head, The processor 26 may detect the length of the band using a sensor or the like, and based on the length of the band, detect the type of animal to which the biological information measuring device 12 is attached. To give a specific example, the skull sizes of humans and dogs are different, so the length of the band is also different accordingly. The processor 26 detects the length of the band and detects whether the creature to which the biological information measuring device 12 is attached is a human or a dog.

As another example, if the biological information measuring device 12 is a device worn on the ear of a living being, the processor 26 uses a sensor or the like to detect the length of the hair on the ear to which the biological information measuring device 12 is attached. The type of animal to which the biological information measuring device 12 is attached may be detected based on the length of its hair.

As yet another example, since the body temperature differs depending on the type of living thing, the processor 26 measures the body temperature of the living creature wearing the biological information measuring device 12, and identifies the type of living creature based on the body temperature. Good too.

As yet another example, the height and weight of living things differ depending on the type of living being, and the resistance value also differs accordingly, so the processor 26 determines the type of living creature wearing the biological information measuring device 12 based on the resistance value. May be specified.

The type of living organism (for example, human, dog, plant, etc.) may be linked to biological information measured from the living organism. The linking may be performed by the biological information measuring device 12 or the information processing device 10. By doing this, it is possible to specify what kind of organism a certain biological information was measured from. As described above, when the type of living organism whose biological information is measured is specified, information indicating the specified type may be linked to the biological information.

Animals may communicate with each other using biological information such as brain waves. For example, biological information may be used to communicate between people or between people and non-human animals (for example, dogs). When the type of living organism whose biological information is to be measured is specified as described above, the processor 26 may change the method of converting brain waves during communication.

<Example 11>
Example 11 will be described below. In Example 11, a stimulus is given to a living creature, and the processor 26 of the information processing device 10 links stimulus information indicating the content of the stimulus with biological information measured from the living creature. The linked stimulus information and biological information are stored in, for example, the memory 24 of the information processing device 10. A plurality of pieces of biometric information may be linked. For example, brain waves, pulse, blood pressure, complexion, body temperature, etc. may be linked to the stimulation information.

The stimulus is, for example, an image such as a still image or a moving image, an environment, a taste, or the like. For example, images that express joy, anger, sadness, or fear, images or environments that feel soothing, or the like are given as stimuli. The image is, for example, a virtual image such as VR (Virtual Reality). Of course, images representing real objects may also be used. The stimulation may be electrical stimulation provided by electrodes provided in the biological information measuring device 12.

For example, when a person is viewing an image, the biological information measuring device 12 measures biological information (for example, brain waves) from the person, and the image and the biological information are linked. Note that the brain waves to be measured are brain waves emitted from the left hemisphere, brain waves emitted from the right hemisphere, or brain waves emitted from both left and right brain waves.

Furthermore, the processor 26 may link the words uttered by the person (that is, voice information) when the stimulus is applied to the stimulus information and the biological information. Furthermore, the processor 26 may link information indicating the person's movements to the stimulus information and biological information.

A virtual image such as a VR image may be an image representing a background photographed in a person's daily life, or surrounding sounds may be played along with the VR image.

Further, as the VR image, an image corresponding to the experience of the person whose biological information is measured may be used. In advance, you may investigate what the person has experienced so far (for example, through a questionnaire).

When the biometric information measured from the person who viewed the VR image changes significantly, the processor 26 may link the biometric information of the changed part to the stimulus information as valid biometric information. For example, when the amount of change in the biological information exceeds a threshold, the processor 26 associates that part of the biological information with the stimulation information as valid biological information.

The order in which stimulus information is presented to a person may be controlled. For example, after giving the person visual information that the person does not like, auditory information, olfactory information, or sensory information that the person likes may be given to the person. The processor 26 may link the biological information measured at that time with each stimulus information.

The processor 26 may change the stimulation information given to the person according to changes in the measured biological information.

For example, FIG. 40 shows the brain waves of a person viewing a VR image containing a monster-like creature. As shown in FIG. 40, changes in brain waves are observed between the tense state before the monster-like creature appears and the surprised psychological state when the monster-like creature appears. Processor 26 may detect such changes and change the stimulus information accordingly.

Further, the processor 26 may change the playback speed of the VR image, change the viewpoint of the VR image, change the color tone of the VR image, or change the image quality or focus of the VR image. good.

The linked stimulus information and biometric information can be used for digital marketing (e.g. video production, attractions, products such as equipment, etc.) and digital healthcare (e.g. recording daily life to determine the rate of stress and refreshment. It may also be used for services that evaluate daily life, etc.).

<Example 12>
The biological information measuring device 12 may be shared by a plurality of people. For example, after person A uses the biological information measuring device 12, another person, person B, may use the biological information measuring device 12 that the person A was using.

For example, a contact-type biological information measuring device 12 (that is, a device that measures biological information by having an electrode or a sensor come into contact with a living thing such as a human being) is a biological information measuring device 12 that is attached to a living creature (for example, a human being). If the water has changed, information indicating the change or information prompting cleaning may be output. Outputting information means, for example, displaying the information on a display, emitting the information as audio from a speaker, or both.

For example, if the biological information measuring device 12 is provided with a microphone, the biological information measuring device 12 can recognize the person using the biological information measuring device 12 based on the person's voice input from the microphone. , it is determined whether the person using the biological information measuring device 12 has changed. The biological information measuring device 12 may be photographed by a camera provided in the terminal device 18 or the like, and the person using the biological information measuring device 12 may be recognized based on the image generated by the photographing. .

As another example, the account of the person using the biological information measuring device 12 is managed by the information processing device 10, the biological information measuring device 12, the terminal device 18, the server, etc., and the account is switched to another account. In this case, the biological information measuring device 12 may determine that the person using the biological information measuring device 12 has changed.

By outputting information indicating that the person using the biological information measuring device 12 has changed, the person using the biological information measuring device 12 can recognize that the biological information measuring device 12 has been used by another person. can do. Thereby, for example, a person who newly uses the biological information measuring device 12 can recognize that the biological information measuring device 12 needs to be cleaned. The same applies when information prompting cleaning is output.

Further, a sensor or the like that detects that the biological information measuring device 12 has been cleaned may be used. For example, a sensor (for example, a sensor that detects alcohol) that detects that an electrode or sensor for measuring biological information has been cleaned with alcohol is used, and the biological information measuring device 12 is used when the electrode or sensor is not cleaned with alcohol. When the biological information measuring device 12 is about to be used (for example, when the biological information measuring device 12 is turned on), the biological information measuring device 12 outputs information indicating that the biological information measuring device 12 has not been cleaned. Further, the biological information measuring device 12 does not need to turn on the power of the biological information measuring device 12. When the biological information measuring device 12 is about to be used with the electrodes and sensors cleaned with alcohol, the biological information measuring device 12 outputs information indicating that the biological information measuring device 12 has been cleaned. It is also possible to output such information.

<Example 13>
Example 13 will be described below with reference to FIG. 24. FIG. 24 shows a screen 38A. The screen 38A is the same screen as the screen according to the third embodiment (see FIG. 12).

In the thirteenth embodiment, information representing emotions, intentions, mental states, psychological states, etc. (for example, images representing bars, numerical values, etc.) is displayed. Emotions, intentions, mental states, and psychological states are estimated by the processor 26 based on the user's biological information. For example, a bar representing the user's concentration level, relaxation level, stress level, etc., or numerical values representing these values may be displayed. Bars and numerical values for all users may be displayed, or bars and numerical values for users specified by an administrator or the like may be displayed.

As an example, FIG. 41 shows the brain waves of a person who performs the mental task of completing a white puzzle in the morning, afternoon, and evening. It was found that the length of time that beta waves and gamma waves exist becomes shorter toward the evening, which characterizes the degree of fatigue caused by concentration.

As another example, FIG. 42 shows the brain waves of a person receiving a head massage. Looking at FIG. 42, an increase in the theta wave band is seen, and then, as the person loses consciousness and enters a slumbering state, an increase in the alpha wave band is seen, indicating the degree of relaxation effect.

As another example, FIG. 43 shows the brain waves of a person who listened to music or took supplements. EEG measurements were performed in the following conditions: (a) normal eyes open, (b) eyes open and only listening to music, (c) eyes open and taking only supplements, and (d) eyes open, listening to music and taking supplements. Each dose was administered for 4 minutes with the eyes open. The type of music used was 963Hz. The supplement is based on measurements taken 3 minutes after taking 5 drops (25mg) of CBD. In addition, fast Fourier transform was performed every second for the frequency range of each electroencephalogram range. In (b), (c), and (d), the FFT intensity in the alpha wave band increases, and the maximum increase is observed in (d), which is a 43% increase in the 9.67 Hz band compared to normal (a). did.

As another example, FIG. 44 shows the difference in brain waves between when a person falls asleep and when the sleep stage progresses after falling asleep. When sleep begins, theta waves begin to appear. As sleep stages progress, spindle waves begin to appear. As a result, the depth of sleep can be characterized by the type of brain waves, and the time elapsed since falling asleep can be estimated.

As another example, a user's physiological event, such as a desire to defecate or the sensation of itching, can be detected. FIG. 45 is a diagram showing changes in brain waves when the user feels itching.

Such brain wave characteristics may be converted into bars or values and displayed to the user. This allows the user's psychological state to be visualized.

In the example shown in FIG. 24, bars and numerical values representing the mental state and the like of the user C1 are displayed. Bar 120 is an image representing the degree of relaxation of user C1. The bar 122 is an image representing the degree of stress of the user C1. The degree of relaxation is "100" and the degree of stress is "30". These numerical values are calculated by the processor 26 based on the biometric information of the user C1.

Further, the processor 26 may identify an event that contributed to the user's state (for example, emotion, intention, mental state, psychological state, etc.) and calculate the extent to which the event contributed to the user's state. For example, the processor 26 may identify stress or an event that contributed to recovery from the stress, and display information indicating the extent of that contribution on the screen 38A. This makes it easy for the user to understand what kind of event has affected the user's condition and to what extent. For example, the processor 26 detects the occurrence of an event using various sensors, etc., and based on the user's biological information, the processor 26 detects a change in the state of the user when the event occurs, and the state of the user before and after the occurrence of the event. Estimate. Processor 26 estimates the extent to which the event contributed to the user's state, for example, based on the change in the user's state.

Further, the processor 26 indicates a history regarding the occurrence of an event, the user's biometric information before and after the event, changes in the user's biometric information, the user's state before and after the event, and changes in the user's state. History information may be stored in the memory 24. The processor 26 may reproduce past events based on the history indicated by the history information. For example, processor 26 may record events before and after the user relaxes and recreate the same situation at that time. For example, a target value for relaxation is specified, and processor 26 reproduces a situation in which the target value is obtained. The processor 26 may reproduce the situation by a combination of playing music, displaying images, etc., and giving action instructions to the person. The processor 26 may acquire the state of the user's brain waves, classify the state of the brain waves, and record the history in association with work hours (for example, time excluding sleeping time). Note that the processor 26 may display a numerical value or a bar indicating a state of relaxation or the like by inserting it into the image of the meeting using a virtual camera.

<Example 14>
Embodiment 14 will be described below with reference to FIGS. 25 and 26. Similar to FIG. 17, FIG. 25 shows images representing each of users A, B, and C. Furthermore, information indicating the state of each user (eg, brain waves, etc.) is displayed in association with each user. Here, as an example, assume that users A, B, and C are participating in the same online conference.

For example, if all users participating in the online conference or a predetermined number of users or more have the same state (for example, the same emotion, intention, mental state, or psychological state), The frame of the image of the user in the same state may be displayed in a specific color (for example, red, etc.) or may be decorated with a specific decoration. The user's condition is estimated based on the user's biometric information.

For example, if users A, B, and C have the same state (e.g., awake state), the processor 26 colors the entire frame surrounding the images of users A, B, and C in a specific color, as shown in FIG. Display or add a specific decoration (for example, a thick border). By doing so, it can be shown that users A, B, and C have the same status. For example, when all users participating in an online conference are in the awake state, such display or decoration indicates that the conference mode is hot-blooded mode.

As in the fourteenth embodiment, information that collectively represents the status of multiple users participating in the conference may be displayed.

<Example 15>
Example 15 will be described below.

The data transmitted from the biological information measurement device 12 to another device (for example, the information processing device 10) may be data generated by applying processing such as FFT instead of raw data of the biological information. By doing so, leakage of information that is included in the raw data but not included in the data after processing such as FFT is suppressed.

Additionally, information indicating items of measured biological information (for example, information indicating channels and parts) is encrypted in the biological information measuring device 12 and transferred from the biological information measuring device 12 to other devices (for example, information processing device 10). ) may be sent. By doing so, it becomes difficult to understand what type of biometric information the transmitted biometric information belongs to, thereby improving the security of the biometric information.

Further, when the measured biometric information or information obtained from the biometric information is rented or sold to a user, the processor 26 may restrict the use of the biometric information. The restrictions include, for example, the information will be deleted after a predetermined period has elapsed, or the information will be returned to the provider. Further, the location where the information is permitted to be used, the purpose of its use, etc. may be determined.

<Example 16>
Example 16 will be described below.

In the sixteenth embodiment, an AI assistant (that is, an artificial intelligence assistant) that emits voice is used. For example, an AI assistant executes processing or operates equipment according to the user's biometric information or user instructions. The AI assistant may be installed, for example, in a user's terminal device, in a device other than the terminal device, or in the information processing device 10, a server, or the like.

For example, the processor 26 may change the volume of the voice emitted from the AI assistant or the speed at which the AI assistant speaks depending on the user's biometric information.

Specifically, if the user's biometric information indicates that the user wants to relax, the processor 26 adjusts the voice emitted from the AI assistant to the user's preferred voice (e.g., high pitch, low pitch, speaking speed). etc.) or change it to the voice of the user's family.

Further, if the user's biometric information indicates that the user is in a hurry, the processor 26 increases the speed at which the AI assistant speaks or causes the AI assistant to quickly output voice.

Furthermore, when a user learns using sounds emitted by the AI assistant, the processor 26 may control the AI assistant to emit sounds at a conversation speed and volume that are highly effective for learning for the user. The user's learning effect is estimated based on the user's biometric information. For example, the processor 26 changes the AI assistant's conversation speed and volume based on the user's biometric information.

Further, when the user is participating in an online conference, a telephone conference, etc., the processor 26 estimates the state of the user's other party based on the other party's biometric information, and based on the result of the estimation, the processor 26 The voice uttered by the user may be changed in a manner that makes it easier to convey the voice uttered by the user. For example, if the user is angry and the user is making a sound that the other party is afraid of, the processor 26 may change the user's voice to a milder sound, or may change the user's voice to the other party without transmitting the sound to the other party. may be expressed using figures, text, etc. Such processing may be performed by an AI assistant.

Further, in an online conference or the like, a certain user (for example, the first user) is wearing the biological information measuring device 12, and another user (for example, the second user) is not wearing the biological information measuring device 12. There are cases. In this case, the first user's biometric information is acquired, but the second user's biometric information is not acquired. Even in such a case, if the first user can recognize the second user's facial expression through the screen, the first user's biological information may change due to changes in the second user's facial expression, voice, etc. Sometimes. When a change occurs in the first user's biometric information in this way, the processor 26 determines that the first user's biometric information changes due to the second user. The processor 26 may estimate the second user's state (for example, emotion, mental state, or psychological state) based on the judgment. For example, if the second user's facial expression is a pained expression, the first user who sees the face may feel apologetic. The second user's condition is estimated based on the biometric information. The AI assistant may also tell the first user how to interact with the second user who has such a condition.

In addition, when adding voices to animated characters or dubbing voices to dramas and movies, a user who has the authority to determine the voices listens to the voice of the voice actor temporarily attached to the character or actor, and the processor 2 may determine the official voice actor based on the user's biometric information. For example, if biometric information indicating approval is obtained from the user, processor 26 determines the voice actor who uttered the voice as the official voice actor. If biometric information indicating discomfort or discomfort is obtained from the user, processor 26 may direct the user to another voice actor.

<Example 17>
Example 17 will be described below.

This embodiment may be applied to content services such as games. For example, the game may be provided to the user via the user's terminal device, or may be provided to the user via another device (eg, a game device). The game may be an online game, or a game provided by executing a program installed on a terminal device or game device. Alternatively, a recording medium storing a game program may be used, and the game may be provided by a terminal device or a game device executing the program.

For example, the biometric information measuring device 12 measures the biometric information of the user while the user is playing a game. The processor 26 may change the progress of the game, the scenes of the game, the sound of the game, the presentation of images, etc. in accordance with the user's emotions obtained from the biometric information.

For example, for a user who is not good at scary games or things that are scary, if biometric information indicating the emotion of fear at a predetermined level or higher is measured, the processor 26 may omit violent scenes or Make it smaller, change the presentation to something simpler, or change the expression to something easier.

On the other hand, for a user who likes scarier scenes, if biometric information indicating the feeling of fear at a predetermined level or higher is not measured, the processor 26 may emphasize more violent scenes or scarier effects. The presentation of the game may be changed so that a scene appears.

Note that the user's preferences are set before starting the game. Further, the processor 26 may change the presentation of the game in conjunction with the biometric information depending on the user's chronic illness or age-related burden. Processor 26 may also change the presentation for educational reasons.

The processor 26 may determine the user's condition based on the user's biometric information at predetermined intervals. Thereby, the scene can be changed in real time and the user can proceed with the game without feeling uncomfortable or strange. Note that a scene may be set that cannot be changed due to the story of the game.

Further, if a feeling that the game is difficult or easy is detected, the processor 26 may change the difficulty level of the game depending on the feeling. For example, the processor 26 may increase or decrease the action level of the AI that is the user's enemy, or increase or decrease the number of enemy characters. Additionally, when a deduction game is being executed, processor 26 may increase or decrease the number of hints that lead to an answer. Furthermore, the processor 26 may increase or decrease set values such as the attack power and defense power of the character operated by the user.

The processor 26 may notify the user of the change in difficulty level based on biometric information. By doing so, the user can understand, for example, what difficulty setting was used to clear the game. In this case, the play time of the game, changes in the user's emotions during the play time, the difficulty level of the game, etc. may be displayed in a graphic such as a graph.

Further, when the user obtains a desired state by playing the game, information indicating the state may be stored in the memory 24 of the information processing device 10 as history information. The processor 26 may change the scene or difficulty level of the game so that the user's state becomes the state that the user desires. For example, if a goal state of wanting to relieve stress and feel refreshed is set in the information processing device 10, the processor 26 sets the game scene or difficulty level so that the user's state becomes the goal state. You may change the degree. For example, the processor 26 may change the game scene to a scene depicting a refreshing blue sky and a plain. As another example, the processor 26 may change the enemy character to a weaker character so that the enemy character can be easily defeated by the user-controlled character.

<Example 18>
Example 18 will be described below.

In Example 18, the biological information measuring device 12 is a hearable device, and is worn in the user's ear. Hearable devices may have voice assistant functionality. The voice assistant function is, for example, a function that supports the user based on the user's biometric information. The voice assistant function may be realized by AI.

The processing according to the 18th embodiment will be described below.

First, a hearable device, which is an example of the biological information measuring device 12, measures biological information (e.g., electrocardiographic waveform, brain wave, myoelectric waveform, skin potential, etc.) from the user's ear (e.g., external auditory canal), and Measuring the sound of Hearable devices store biometric information and voice information in a chronological order. This information may be stored in a hearable device, a user's terminal device, or the information processing device 10. Here, as an example, it is assumed that these pieces of information are stored in the information processing device 10.

The processor 26 of the information processing device 10 determines whether or not a large fluctuation has occurred in the user's biometric information. For example, when the biological information is brain waves, large fluctuations refer to epileptic abnormal waves, and when the biological information is electrocardiographic waveforms, large fluctuations refer to waveforms that indicate cardiac abnormalities such as arrhythmia. It is about.

When the processor 26 detects that a large fluctuation has occurred in the user's biometric information, the processor 26 collects information indicating the time when the large fluctuation occurred, the location where the user is at that time, and the state of the user (hereinafter, these information (referred to as “context information”). Processor 26 may remove noise and determine whether the large fluctuation is due to a mind-body stimulus response.

When the above-mentioned large fluctuations (ie, changes in the stimulus response of the mind and body) are a critical situation, the processor 26 takes control of the audio function of the hearable device. A critical situation is, for example, when a heart disease occurs (for example, when an arrhythmia is detected based on the pulse, which is biological information), or when the user falls (for example, when the user falls down) If the user's fall is detected based on radio waveforms or acceleration sensor data) or if the user swallows (for example, if the myoelectric waveform, which is biological information, contains irregular waveforms, (when it is detected that swallowing may be occurring based on a regular waveform), etc.

Further, the processor 26 transmits the above-mentioned context information, biological information, and voice information to a device (eg, a PC, etc.) installed at a predetermined location (eg, a hospital, etc.). At this time, the processor 26 transmits to the device the context information, biological information, and audio information measured at a time (for example, a predetermined length of time) before and after the time when the large fluctuation occurs.

Furthermore, the processor 26 may control the sound emitted from the hearable device based on the above-mentioned context information, biological information, and sound information. For example, processor 26 controls the playback speed of content played on the hearable device. Specifically, the processor 26 changes the reading speed of the learning video or electronic book. Additionally, the processor 26 may stop playing the content. Additionally, the processor 26 may reproduce content (for example, subjects, comics, music, etc.) that matches the user's preferences. Processor 26 may also perform a call-to-call function. The auto-call function here is a function that automatically notifies pre-specified emergency contacts (for example, family members, family doctor, etc.) of a phone call or message. Multiple people may be notified of the call or message at the same time, or external people may be notified of the call or message gradually. Additionally, processor 26 may change the contact information depending on the severity of the user's medical condition. Of course, the user himself/herself may be notified.

The voice assistant function described above may be switched for each hearable device or for each user account. Further, a function for setting whether or not to execute the voice assistant function based on biometric information may be provided. The voice assistant function of a hearable device may be performed without being based on biometric information. Furthermore, the situations to which the voice assistant function based on biometric information is applied may be limited to situations in which signs of illness are diagnosed, or may be limited to a reading function such as a content service. A function for setting such limitations may be provided.

The processor 26 may cause the AI that implements the voice assistant function to learn the above context information, biological information, and voice information.

For example, the processor 26 may accumulate audio information when the user is speaking in a relaxed state, and cause the AI to learn the audio information. For example, by applying voice synthesis and voice adjustment techniques, AI may synthesize a voice assistant voice optimized for biological reactions and read out the voice.

As another example, the processor 26 may accumulate audio information when the user is speaking under stress, and cause the AI to learn the audio information. For example, by applying voice synthesis and voice adjustment techniques, AI may synthesize an optimized voice assistant voice to calm the user and read it out loud.

As yet another example, the processor 26 may accumulate voice information when the user is nervous, and cause the AI to learn when, where, and under what conditions the user becomes nervous. For example, by applying voice synthesis and voice adjustment techniques, AI may synthesize an optimized voice assistant voice to calm the user and read it out loud.

As yet another example, the processor 26 may convert the user's voice into text, store the text data together with information indicating the user's happiness, anger, sadness, etc. in chronological order, and have the AI learn these data. good. It may also be stored in memory. Note that the emotions may be selected by the user. Further, the processor 26 may change the size, color, etc. of the characters converted into text according to the user's voice. By applying voice synthesis and voice conditioning techniques, AI may synthesize an optimized voice assistant voice that calms the user and read it out loud.

For example, the processor 26 links the user's voice information obtained from the user's terminal device (e.g., smartphone, PC, etc.), information such as icons, characters, pictures, etc. input by the user, and biological information measured from the user. Attach and record. The processor 26 converts the user's biometric information into emotional index data such as happiness, anger, sadness, stress, concentration, relaxation, load, understanding, mastery, or interest.

For example, electrodermal activity may be used as a parameter indicating the state of mental activity. Additionally, skin potential level (SPL), which is a direct current component of skin potential activity (SPA), may be used. SPL generally shows a negative high value when the arousal level is high (that is, when the person is excited), and a positive high value when the person is drowsy or relaxed. Potential skin reflex (SPR), which is the alternating current component of potential skin activity (SPA), may be used. SPR frequently occurs due to stimulation of pain, touch, hearing, and vision due to changes in the external environment, deep breathing, body movements, thoughts, etc.

The processor 26 collects information indicating the state of the user obtained from an interface such as a camera or a keyboard, as well as the state represented by the emotional index data (e.g., happiness, anger, sadness, stress, concentration, relaxation, load, understanding, mastery, interest, etc.). ) and the type of condition. For example, the processor 26 changes the display color, the type of font of the characters, and the size of the characters depending on the degree and type of the state. For example, in a reading function implemented by existing software, the processor 26 may change the speech style (e.g., joy, anger, sadness, fear, kindness, etc.) and the tone of voice (e.g., emphasis, speaking speed, tone of voice, etc.). pitch, intonation, volume, etc.) may be changed depending on the stimulus response (e.g., relaxed, stressed, tense, etc.). The processor 26 may change the volume and speed of the voice of the reading function depending on the level of emotion of the user.

Each of the above examples may be applied to, for example, electronic books, movie content (including content distributed via the Internet, for example), electronic learning materials, online conference tools, and the like.

As another example, when creating texts or materials, if a function is executed to change the size or color of text or graphics depending on the user's biometric information, if that function is always executed, the text There is a possibility that the uniformity of information and materials may be lost. Therefore, the processor 26 does not need to change the size or color of characters or figures in a specific area of a text or document according to biometric information. In other words, the processor 26 may change the size and color of characters and graphics within the area that is permitted to be changed, depending on the user's biometric information.

Furthermore, when the text-to-speech function is executed to read out the characters written in the text or materials, the processor 26 changes the volume and speed of the voice during the reading, depending on the user's biometric information. It's okay. Furthermore, the reading speed may be limited to a range that is neither too fast nor too slow.

The time required for processing to determine emotions, mental states, and psychological states based on biological information, and the time required to measure emotions, mental states, and psychological states, vary depending on the type of emotions, mental states, and psychological states. It may vary. For example, the measurement interval for concentration and relaxation is 1 second, the measurement interval for interest, understanding, mastery, activity load, emotional intensity, and creativity is 1 to 5 seconds, and the measurement interval for pleasure and discomfort is 1 second. The interval is 5 seconds or 10 seconds. Of course, these times are just examples, and may change depending on the performance of the biological information measuring device 12, etc.

When there are variations in time as described above, in a particular mode, the processor 26 does not determine all predetermined emotions, mental states, or psychological states, but only determines some predetermined emotions, mental states, or psychological states. The state or psychological state may be determined, or the time and scene for making the determination may be limited. For example, if the user is learning, the processor 26 may determine only the learning effect based on the user's biometric information, and may not determine emotions other than the learning effect. However, if the user's state is a specific state (for example, if the user is suffering from an underlying disease), the processor 26 may record all predetermined emotions, mental states, and psychological states. It may be determined constantly.

Note that if the user stops wearing the hearable device but is wearing another biological information measuring device 12, the processor 26, based on the biological information measured by the other biological information measuring device 12, Emotions, mental states, and psychological states may also be determined. For example, if a user wears a hearable device and a smart watch and then stops wearing the hearable device, the user's emotions, mental state, and psychological state are determined based on biological information measured by the smart watch. It's okay.

<Example 19>
Example 19 will be described below. In each of the embodiments described above, when biological information in a specific frequency band (for example, brain waves in a specific frequency band) is measured, and biological information in other frequency bands other than the specific frequency band is not measured, the processor 26 The biometric information of the other frequency band may be estimated. For example, the processor 26 estimates biological information of the same type as the measured biological information, but in a frequency band different from the measured frequency band.

For example, if only the potential in the frequency band of alpha waves, which is an example of brain waves, is measured, and the potential in frequency bands other than alpha waves is not measured, the processor 26 uses various sensors (for example, a gyro sensor, an acceleration information measured by sensors, etc.), environmental information (e.g., information indicating temperature, humidity, weather, location, etc.), information regarding the user's current behavior (e.g., information indicating whether they are exercising, working at a desk, sleeping, etc.), One or more pieces of information among information regarding the user's current illness and information regarding the history of measured α waves (for example, information indicating the shape, period, amplitude, etc. of the α wave waveform) Based on this, brain waves in frequency bands that are not actually measured (for example, brain waves other than alpha waves such as beta waves) or brain waves in frequency bands that are not clearly detected are estimated. For example, the processor 26 estimates brain waves in unmeasured frequency bands based on the correlation of the various types of information described above. The processor 26 may output the brain waves in the estimated frequency band to a terminal device or the like. For example, brain waves in the estimated frequency band are displayed on the display of the terminal device. At this time, brain waves in the measured frequency band (e.g., alpha waves) and brain waves in the estimated frequency band (e.g., beta waves) are graphed on the same scale (e.g., the horizontal axis shows time, and the vertical axis shows electric potential. may also be displayed as a graph showing The processor 26 may output the brain waves in the estimated frequency band as brain wave candidates in that frequency band. Even when alpha waves are not measured and only beta waves are measured, alpha waves may be estimated in the same manner.

When biometric information in a plurality of frequency bands is measured, the processor 26 may estimate biometric information in a frequency band that is not measured based on the biometric information in the plurality of frequency bands. For example, if alpha waves and beta waves are measured and brain waves other than these are not measured, the processor 26 measures brain waves in frequency bands other than these based on the actually measured alpha waves and beta waves. It may be estimated.

Note that when a plurality of candidates are estimated, the processor 26 may display the plurality of candidates on the display of the terminal device, or output a sound indicating that the plurality of candidates have been estimated. Further, the processor 26 may change the order in which the plurality of candidates are displayed depending on the probability of occurrence or the size of the possibility of each of the plurality of candidates, or display a score according to the size. good.

For example, when measuring brain waves with a hearable device, it may be difficult to detect bioelectrical potentials generated from a location on the head that is relatively far from the ears. In such a case, it is conceivable to estimate the brain waves in the unmeasured frequency band based on the brain waves in the measured frequency band and other information (for example, environmental information or information regarding the user).

The user may be able to turn on or off the function of estimating biological information in unmeasured frequency bands. For example, when a user turns on the function using a terminal device or the like, the function is executed. If the user turns off the function using a terminal device or the like, the function will not be executed. Unmeasured biological information (for example, brain waves in unmeasured frequency bands) may not actually be generated from the user's brain, so in such cases it may be a good idea to turn off the function. It will be done.

<Example 20>
Example 20 will be described below.

When using brain waves, etc. as a BMI for communicating between people, identify the information you want to convey and provide a means to convey the information you want to convey.

For example, the information transmitting side identifies the information to be conveyed or extracts the information to be conveyed, and transmits the information to the receiving side. The transmission target may be one sender and one receiver, or one sender and a plurality of receivers (for example, a large number of receivers). The information content to be conveyed is information about the person's thoughts and intentions, and status information, such as agreement, consent, opposition, rejection, hesitation, confusion, etc. When information is provided to someone, the person sending the information tells the recipient whether they agree with the information or not. Information can be conveyed using methods that stimulate a person's five senses, such as information transmission by voice, information transmission by light emission, stimulation transmission by vibration, etc. To give a more specific example, for example, with a hearable device, it is possible to measure biological information such as brain waves (measuring the information content to be conveyed) and transmit information through voice at the same time by utilizing the ear. Therefore, it is excellent in that BMI can be realized without preparing various devices. Here, the sound may be, for example, music, chimes, etc., as long as the meaning of the sound is mutually determined in advance in addition to linguistic information. There are also ways to convey the level and degree of intention by signaling approval with two chimes or by changing the range of the chime. There are various ways of conveying that a low-pitched chime sound indicates a low level of approval, while a high-pitched chime sound indicates a high level of approval. With the advent of BMI, human-to-human communication has expanded to include a variety of forms of communication that utilize biological information such as brain waves, in addition to the conventional language-based communication through voice and GUI (graphical user interface). It becomes possible to acquire the method of

A specific example of the process according to the 20th embodiment will be described below with reference to FIGS. 27 and 28. 27 and 28 show flowcharts showing the flow of a specific example. FIG. 27 shows the flow of processing when operating the device. FIG. 28 shows the flow of processing when communicating between people.

As shown in FIG. 27, first, an operation mode (OP mode) is selected (S01), and user operation intention information is extracted and detected (S02). As a result, the device to be operated and the operation details are extracted and linked. Furthermore, a method for determining the accuracy of the user's intention to operate is presented to the user. When the accuracy of the operation intention is high (S03, high accuracy), the operation on the device is immediately executed (S04). Thereafter, the correctness of the operation is verified (S05). For example, the verification is performed by the user. If the operation is correct (S06, Yes), the device will be operated in the same way next time (S07). For example, the operation details are set in the device, and the same operation is performed next time. If the operation is incorrect (S06, No), the process moves to step S11. Further, when the accuracy of the operation intention is low (S03, accuracy is low), the operation intention is confirmed using another method (S08). Note that even when the accuracy of the operation intention is high, the operation intention may be confirmed using another method. For example, information indicating another method may be displayed on a display or output as audio from the earphones of a hearable device. This presents the user with another method. If the confirmed operation intention is correct (S09, Yes), that is, if the confirmed operation intention matches the operation intention that the user had in mind, the operation on the device is executed (S10). If the confirmed operation intention is not correct (S09, No), the process moves to step S11. In step S11, the reason why the operation intention was incorrect is analyzed. For example, the analysis is performed by the user. Based on the analysis result, the next determination method (that is, the operation determination method) is selected (S12). Furthermore, the criterion for determining the intention to operate is changed (S13), and the changed content is reflected in the processing from step S03 onwards. For example, the threshold value registered in the management table is changed, and based on the changed threshold value, the intention of the operation is confirmed, the accuracy is determined, and the like. Note that each of the above processes may be selected by the user.

Further, when communicating between people, a communication mode (hereinafter referred to as "CM mode") is selected as shown in FIG. 28 (S20). For example, either automatic selection mode or manual selection mode is set by the user, and CM mode is selected according to the setting. Next, a communication target (hereinafter referred to as a "CM target") is selected (S21). For example, in a mode in which CM targets are automatically selected, conference participants and the like are selected as CM targets. In the mode in which the CM target person is selected by the user's manual operation, candidates for the CM target person are presented to the user by, for example, audio on a hearable device, display, etc., and the user selects the CM target person. Next, the user selects information that may be conveyed to the CM target person (S22). For example, information that may be conveyed to the CM target person is selected from a group of intention information, a group of status information, and the like. Further, information that the user does not want to convey to the CM target person may be selected. For example, information indicating emotions is selected as information that the user does not want to convey to the CM target. Next, biological information is measured or observed, and intention information, status information, etc. are extracted based on the biological information (S23). Next, information to be conveyed to the CM target person is detected from the extracted intention information, state information, etc. (S24). For example, a detection test is performed in advance, and intention information or state information that is similar to the information detected by the test (for example, intention information or state information whose degree of similarity is equal to or higher than a threshold) is extracted as intention information or state information. Detected from status information, etc. Note that the detection method may be presented to the user. The detected intention information, status information, etc. are then conveyed to the CM target person. For example, the transmission method may be presented to the user, and the user may select the transmission method. Examples of the transmission method include transmission by vibration (for example, transmission by sound), transmission by light, and the like. Information may be conveyed to the CM target person by language, music, light emitting signals, or the like. When the information has been conveyed to the CM target person, the user who is the sender may be notified that the information has been conveyed. The notification may be omitted. Moreover, after the process of step S24 is executed, the processes after step S03 shown in FIG. 27 may be executed.

Furthermore, the process according to the 20th embodiment may be performed using a hearable device. Since biometric information is measured while the hearable device is attached to a person's ear, various information can be conveyed to the user by voice while biometric information can be measured. Furthermore, since hearable devices are worn on the ears, they have the advantage of having a more natural appearance compared to electroencephalographs and the like that are worn on the head to measure brain waves and the like. Furthermore, BCI using a hearable device can improve the accuracy of the above operation mode and CM mode.

In each of the above embodiments, all or part of the processing executed by the processor 26 of the information processing device 10 may be executed by the processor 34 of the terminal device 18, or may be executed by the biological information measuring device 12. It's okay. Further, the processing may be executed by one or more devices selected from the information processing device 10, the terminal device 18, and the biological information measuring device 12.

<Example 21>
It is known that a person's psychological state (for example, decision-making) while performing a certain task or solving a problem is reflected in brain waves. For example, it is known that theta waves become larger when a person gives up on completing a certain task, and alpha waves become larger when a person is able to solve a certain problem (http://qlifepro.com/news/20211124/eeg -rhythm.html).

In the 21st embodiment, the biological information measuring device 12 measures the user's brain waves, the information processing device 10 determines the situation the user is in based on the measured brain waves, and provides intervention means (interventions) necessary for the user. determination of the type of intervention). Depending on the determination, the intervention device (e.g., terminal device 18) applies direct stimulation to the user (e.g., electrical stimulation, music playback, etc.) to influence the user's brain wave state. perform interventions that affect the user's psychological state, report the user's state to a third party (e.g., teacher, parent, user's boss, etc.), and provide the user with appropriate assistance/intervention measures. You can do it.

More generally, the biological information measuring device 12 acquires biological information of at least one user. The information processing device 10 is then configured based on biological information including at least the brain waves of at least one user and historical information including at least one time series of context information indicating the context at each point in time when the biological information was measured. and determine the status of at least one user. Then, the status of at least one user may be output to the at least one user.

The history information may include at least one time series indicating the user's behavior history or a time series of context information indicating the context at each point in time when biometric information was measured. For example, context information may include who made the measurement, when the measurement was made, where the measurement was made, what was measured, why the measurement was made, how the measurement was made, what the user did before the measurement, and what the user did before the measurement. activities performed with other users; environmental information such as the location, location, building, room, or temperature of the environment in which the user took measurements; scenes or events in which the user is participating; or the context in which the measurements were taken. may include various other information that indicates the Furthermore, the historical information may be modified with respect to the type of content it contains (e.g., searchable according to a particular event or a particular scene) and/or with respect to different time ranges (e.g., by the user or by artificial intelligence). searchable (by other automated processes). Historical information may be obtained from cameras, microphones, or information recorded on the Internet.

Historical information may be classified according to a number of different time series with different time intervals for collecting a series of biometric information or contextual information, or according to a different number of points in time during which biometric information or contextual information is collected; At least one determination regarding the condition of at least one user may be output based on analyzing biometric information and historical information for each user. For example, a first time series having multiple time points separated by a first time interval (e.g., a short time interval) and a time series having a plurality of time points separated by a first time interval (e.g., a short time interval) and a time series having a plurality of time points separated by a second time interval (e.g., a time interval having a long time interval compared to a short time interval). ) The state of at least one user can be determined based on a combination of biometric information and historical information collected in a second time series having a plurality of time points separated by .

Additionally, confidence in the determined states of at least one user may be evaluated based on the similarity or identity of the determined states for different time series. For example, a higher confidence level is assigned if the results for a short time interval and a long time interval are the same compared to when the results for a short time interval and a long time interval are different. The determined state may be output to the user or may be ignored based on the determined reliability.

Each time series may also be assigned a weight by the user depending on the context in which the user is currently placed, and the determined states may be output to the user in priority order determined based on the weight. Alternatively, a subset of the determined states may be output to the user according to the weights.

In other words, by determining which results based on which time series to output based on context information, the results that best match the user's expectations for a given situation determined by the context may be output. . Note that the determination that most matches the user's expected value may be set by the user for various contexts, or may be learned using artificial intelligence (machine learning, etc.) based on past determinations.

For example, a condition determined based on a short time interval may indicate abnormality, and a condition determined based on a long time interval may indicate normality. In addition, it may occur in a short period of time, such as when the determined state is incorrect or the information is an outlier (for example, when there is a poor contact with the device or when temporary stress is applied to the user from the outside in a short period of time). If the context indicates that the anomaly can be ignored, you may decide not to output the results to the user. In other words, in such a case, the results may be output to the user only when the determination results for the short time interval and the long time interval are the same. On the other hand, the context may indicate that short-term anomalies may be important, for example if the user is ill and any short-term changes in the user's condition need to be reported. In this case, results at short time intervals may be given greater weight and output preferentially.

Further, the length of the time interval and the number of time points can be changed or set depending on the context (scene or event) in which the user performs the measurement. The correspondence between the context and the length of the time interval or the number of time points may be set by the user, or may be learned by artificial intelligence (machine learning, etc.) based on past data.

Note that although some examples have been shown above regarding two time series having two time intervals (relatively short and long), this embodiment is not limited to this, and can be applied to any number of time series. Applicable to series.

Alternatively, the proposed intervention may be determined based on the state and/or historical information of the at least one user, and the proposed intervention may be output to the at least one user. good.

Furthermore, the biological information measuring device 12 may be in the form of a hearable device (described below) that includes an audio input/output device, and the information processing device 10 can receive the proposed intervention via the audio input/output device. The measure may be output to at least one user, the at least one user's response to the proposed intervention measure may be obtained, and the response may be stored in memory.

The proposed intervention and the at least one user's response may be stored as part of the historical information. In this way, the determination of the proposed intervention is based on past interventions, responses, and their contexts when the user was in similar or identical conditions or contexts, as well as the context in which the user is currently placed. It may be done based on. Further, the correspondence between the context and the proposed intervention means may be set by the user, or may be learned by artificial intelligence (machine learning, etc.) based on past data.

Furthermore, the information processing device 10 operates in a first mode in which a response is obtained from at least one user before executing the proposed intervention means, and in a second mode in which the proposed intervention means is automatically executed by artificial intelligence. You may choose either one.

Alternatively, an intervention device (e.g., terminal device 18) may be used to control light (by light color, light intensity, light pulses), images, images, sounds, sounds, music, vibrations, temperature changes (hot, cold), shocks (by objects, etc.). The intervention may be carried out by means of communication via at least one of the following: impact (bumping, throwing an object), stimulation (pinch, pull), smell, scent. For example, https://www. youtube. com/watch? v=Vr8hSeAjY2I shows an example in which the intervention means are carried out by light.

Other forms or modes of intervention include adjusting the air temperature (e.g. setting the temperature of an air conditioner), interacting with artificial intelligence (e.g. comforting, encouraging, helping, giving advice), or expressing comfort or It may also be a gesture (for example, a sound emitted by the pet robot or a comforting action/movement of the pet robot).

Additionally, the status of at least one user may be output to at least one other user.

When outputting to at least one other user, the determination of the state of the at least one user is performed based on the biometric information and history information of the at least one user or at least one other user. The information processing device 10 of another user may perform the processing. Similarly, the proposed intervention means may be performed by an intervention device shared by at least one user and at least one other user. Alternatively, the biometric information of at least one user may be transmitted to at least one other user, and the information processing device 10 of the at least one other user may determine the intervention means (or intervention device) to be used. good.

If at least one user and at least one other user are in the same location, the same intervention device can be selected to perform the intervention measure. More specifically, based on the location information of the at least one user and at least one other user, the range of space that shares the state of the at least one user or the intervention device to be used can be determined. .

It should be noted that such sharing of the status of at least one user may be based on the type of relationship between at least one user and at least one other user (e.g., patient-doctor relationship, worker-to-worker relationship, etc.). It may also be determined based on the relationship between the two.

The determination of the state of the at least one user or the determination of the intervention measure may be based on previous determinations made for the at least one user and at least one other user, which may be based on previous determinations made for the at least one user and at least one other user. Provides diversity in decision results that can only be made by relying on information. Additionally, if the biometric information of at least one other user is found to be similar or identical to the biometric information of at least one user, in that case, the at least one other user may be subject to the same intervention. You may add it as a possible candidate. In addition, when determining the intervention method itself or selecting which intervention device to use, the effectiveness of past intervention methods (or intervention devices) is recorded in historical information, and the most effective (high priority) intervention is selected. Priorities may be given to allow selection of means (or intervention devices). Artificial intelligence (machine learning, etc.) may be used to determine what intervention measures (or intervention devices) are effective from past data. Furthermore, if there are new intervention method candidates (or intervention device candidates), or if there are multiple intervention method candidates (or intervention device candidates), the user may be allowed to select from among them. good. Alternatively, the at least one other user may select the candidate intervention measure (or candidate intervention device) based on past effectiveness for the at least one user. Alternatively, at least one user decides not to select an interventional measure (or interventional device) and at least one user is authorized to select an interventional measure (or interventional device) on behalf of the at least one user. may allow other users to make that selection. This is because the selection of intervention methods by others may be more effective than by oneself.

<Example 22>
Further, the information processing device 10 may be used to support communication of at least one user.

In Example 22, the biological information measuring device 10 acquires biological information of at least one user. The information processing device 10 determines the state of at least one user based on the biometric information of the at least one user. The information processing device 10 then supports communication of information by at least one user based on the state of the at least one user.

Further, when supporting communication of information from one user to another user, the information processing device 10 may transmit information to the other user after obtaining consent from the one user.

Further, when one user rejects information to be transmitted to the other user, the information may be stored in a memory associated with the one user.

Alternatively, if one user rejects information to be sent to the other user, options including at least one alternative action may be offered to the other user. The alternative actions include at least one of retransmitting the information to another user after a predetermined period of time, transmitting the information to a different user, or abandoning the transmission of the information. can be included.

Further, when one user consents to transmit information to the other user, the information processing device 10 stores the information in the memory as learning data in association with the state of the other user, and stores the information in the memory as learning data. It may be used for the next judgment based on the information.

Furthermore, when one user consents to transmitting information to the other user, the information processing device 10 may store data indicating whether or not the transmission to the other user was successful.

Further, the information processing device 10 may determine the state of at least one user based on the frequency spectrum of the biopotential measured from the at least one user.

<Hearable device>
Hereinafter, a hearable device that is an example of the biological information measuring device 12 according to this embodiment will be described in detail with reference to FIGS. 29 to 37. In the following description, an earphone worn in a person's ear will be described as an example of a hearable device, but the hearable device described here may be a hearable device other than earphones.

For example, one or more electrodes are used as biopotential detection electrodes (hereinafter referred to as "sensor electrodes") for detecting potentials including biopotentials, and another one or more electrodes are used for grounding. (hereinafter referred to as "ground electrode"), and one or more other electrodes are used as electrodes for detecting a potential for comparison with the potential detected by the sensor electrode (hereinafter referred to as "reference electrode"). ).

Hereinafter, the potential detected by the sensor electrode will be referred to as the "sensor potential", the potential detected by the reference electrode will be referred to as the "reference potential", and the potential detected by the ground electrode will be referred to as the "ground potential".

FIG. 29 is a perspective view showing the entire hearable device 200. FIG. 30 is a diagram when the hearable device 200 is viewed from above. FIG. 31 is a diagram of the left earphone viewed from the left side. FIG. 32 is a perspective view of the right earphone. FIG. 33 is a perspective view of the left earphone. FIG. 34 is a diagram of the right earphone viewed from above. FIG. 35 is an exploded perspective view of the right earphone. FIG. 36 is a cross-sectional view of the right earphone. FIG. 37 is a perspective view showing the inside of the left earphone.

Here, for convenience of explanation, as shown in FIG. 29, the front-back direction, the up-down direction, and the left-right direction are defined. The front direction is the direction that a person's face faces, and the back direction is the direction opposite to the front direction. The upward direction is the direction toward which the top of a person's head faces, and the downward direction is the opposite direction to the upward direction. The right direction is the direction to the right hand side of a person, and the left direction is the direction to the left hand side of the person. The front-rear direction, the up-down direction, and the left-right direction are orthogonal to each other.

Further, here, as an example, it is assumed that the hearable device 200 measures biological information including brain waves. Of course, the hearable device 200 may measure biological information other than brain waves while measuring brain waves, or without measuring only brain waves. For example, when a right earphone 216R or a left earphone 216L, which will be described later, is attached to a person's ear, a potential representing biological information is detected. The potential signal indicating the detected potential usually includes a potential signal indicating the brain wave, but in addition to the potential signal indicating the brain wave, there is also a potential signal indicating the potential generated due to the movement of the human head. , movements caused by facial expressions, and potential signals indicating potentials generated due to movements of the neck, jaw, and eyeballs. In addition, as biological information caused by changes in blood flow due to the movement of a person's head, cerebral blood flow, pulse waves related to cardiac blood vessels, heartbeat, etc. can be significantly measured in some cases. In this way, the hearable device 200 may measure biological information other than brain waves as well as brain waves. Biological information other than brain waves may be processed as noise (for example, removed), or may be separated from brain waves and used for some processing.

As shown in FIGS. 29 and 30, the hearable device 200 can be roughly divided into a right earphone 216R that is attached to a person's right ear, a left earphone 216L that is attached to a person's left ear, and a right earphone 216R and a left earphone 216R that is attached to a person's left ear. It includes a cable 218 that electrically connects the earphone 216L. The right earphone 216R and the left earphone 216L may transmit and receive signals and data via the cable 218. The cable 218 may be provided with a remote control for operating the hearable device 200.

Either the right earphone 216R or the left earphone 216L may function as a biological information measuring device that measures biological information, or both the right earphone 216R and the left earphone 216L may function as a biological information measuring device. good.

For example, one earbud may have a sensor electrode, a reference electrode, and a ground electrode, and the other earbud may have no electrodes.

As another example, both the right earbud 216R and the left earbud 216L may have a sensor electrode, a reference electrode, and a ground electrode.

As yet another example, one earbud has one or two of the sensor electrode, reference electrode, and ground electrode, and the other earbud has an electrode that the one earbud does not have. It's okay.

As yet another example, each of the right earphone 216R and the left earphone 216L is provided with one or more electrodes, and each of the sensor electrode, reference electrode, and ground electrode may be assigned to one of the electrodes.

As yet another example, one earphone may be provided with multiple sensor electrodes, multiple reference electrodes, and multiple ground electrodes. For example, the right earphone 216R may be provided with a plurality of sensor electrodes, and the left earphone 216L may be provided with a plurality of reference electrodes.

The right earphone 216R is, for example, a canal-type earphone, and includes a right housing 220R, a right earpiece 222R, a right support 224R, a right ear hook 226R, and an electrode member 228R. Earpieces are sometimes called ear pads.

The right housing 220R has, for example, a thin rectangular parallelepiped shape, and is a housing that stores electronic circuits and the like. In the right housing 220R, a right earpiece 222R and a right support 224R are provided on the surface facing the right ear of the person when the user wears the hearable device 200. Inside the right housing 220R, there are a control device, a speaker, a communication device (for example, a communication chip), an electronic circuit that analyzes and processes biological information, and a 6-axis sensor (for example, a 3-axis sensor that detects acceleration and 3-axis sensor that detects angular velocity). A sensor having an axis sensor, etc.), a memory, a GPS (Global Positioning System) sensor, etc. are stored. The communication device is a device for realizing a wireless communication function such as Bluetooth or BLE (Bluetooth Low Energy), for example. This communication device is equipped with wireless LAN (for example, a network that uses WiFi, etc.) and cellular (3G, 4G, 5G, LPWA, etc.) modules to widen the communication area, and has a longer communication distance than Bluetooth. It is possible to send data directly to other devices or to distant devices via the Internet. The six-axis sensor detects the moving direction, orientation, and rotation of the right housing 220R. Biometric information may be stored in memory. Note that the electronic circuit for analyzing biological information does not need to be provided in the biological information measuring device 12.

The right support 224R has a columnar shape such as a cylinder, protrudes from the side surface of the right housing 220R (for example, the surface facing the right ear when the right earphone 216R is attached to the right ear), and is attached to the right housing 220R. 220R and the right earpiece 222R. For example, the outer diameter of the right support 224R is larger than the outer diameter of the right earpiece 222R. An electrode member 228R is provided on a part or all of the side surface of the right support 224R.

For example, the electrode member 228R has a ring shape and is supported by the columnar right support 224R. The whole or part of the electrode member 228R functions as an electrode. That is, an electrode may be provided on the entire surface of the electrode member 228R, or an electrode may be provided on a part of the surface of the electrode member 228R, and no electrode may be provided on the surface of that part. The electrode member 228R is made of conductive rubber made of carbon, for example. In this case as well, the entire electrode member 228R may be made of conductive rubber, and the entire electrode member 228R (for example, the entire surface) may function as an electrode, or a portion of the electrode member 228R may be made of conductive rubber. Accordingly, a portion (for example, a portion of the surface) of the electrode member 228R may function as an electrode.

The right earpiece 222R is provided at the tip of the right support 224R (that is, the end opposite to the end connected to the right housing 220R). For example, the right earpiece 222R has a cylindrical shape whose width becomes narrower toward the tip. Of course, this shape is just an example, and the right earpiece 222R may have another shape.

A sound conduit or the like (for example, a conductive tube member to be described later) is housed in the right earpiece 222R, and the sound emitted by the speaker is transmitted through the sound conduit or the like in the right earpiece 222R and externally from the right earpiece 222R. is issued. An electrode is provided on part or all of the outer surface (for example, the side surface, etc.) of the right earpiece 222R. The electrode is made of conductive rubber made of carbon, for example. The right earpiece 222R itself may be made of conductive rubber. For example, the entire or part of the right earpiece 222R may be made of conductive rubber. That is, the entire surface or a portion of the surface of the right earpiece 222R may function as an electrode.

The right earpiece 222R and the electrode member 228R may be made of, for example, an elastic member. As the elastic member, for example, resin such as rubber is used. Specifically, Si-based rubber (for example, S1734 manufactured by NOK), urethane-based rubber, or the like may be used for the right earpiece 222R and the electrode member 228R. Further, the hardness of each of the right earpiece 222R and the electrode member 228R (for example, the hardness according to the durometer type A (instantaneous) standard) is, for example, 40 to 75. As an example, a resin having a hardness of 70 may be used for the right earpiece 222R and the electrode member 228R.

As will be described later, the right ear piece 222R is inserted into the external auditory canal of the right ear and placed in the right ear so as to be in contact with the external auditory canal, and the electrode member 228R supported by the right support 224R is connected to the concha of the right ear. It is placed in the right ear so that it contacts the cavity.

The right ear hook 226R has a generally curved shape, and is a member that is hung on the user's right ear when the user wears the right earphone 216R. For example, the right ear hook 226R is hung on the helix of the right ear. To explain in more detail, the right ear hook 226R is provided in contact with the helix on the back side of the helix. One end of the right ear hook 226R is connected to the front side of the right housing 220R, and the right ear hook 226R has a curved shape from the connecting portion to the rear side of the right housing 220R. This part constitutes the curved part. The curved portion is provided in contact with the helix on the back side of the helix of the right ear. For example, the curved portion has a shape that follows the shape of the back side of the helix. The other portion of the right ear hook 226R is connected to one end of the cable 218.

The right earpiece 222R and the right support 224R are replaceable members. For example, a plurality of types (for example, 3 to 5 types) of right earpieces 222R and right side supports 224R with different shapes and sizes are prepared in advance, and the shape of the user's ear (for example, the external auditory canal, concha cavity, etc.) is prepared in advance. The right earpiece 222R and the right support 224R can be replaced according to the position of the user.

The left earphone 216L is, for example, a canal-type earphone, and includes a left housing 220L, a left earpiece 222L, a left support 224L, a left ear hook 226L, and an electrode member 228L.

The left housing 220L has, for example, a thin rectangular parallelepiped shape, and is a housing that stores a battery and the like. In the left housing 220L, a left earpiece 222L and a left support 224L are provided on a surface facing the user's left ear when the user wears the hearable device 200. Power from the battery is supplied to the right earphone 216R and the left earphone 216L, thereby driving the right earphone 216R and the left earphone 216L. For example, power from a battery is supplied to the speaker and each circuit. Note that the battery may be provided in both the right case 220R and the left case 220L, or may be provided in either the right case 220R or the left case 220L.

The left support 224L has a columnar shape such as a cylinder, protrudes from the side surface of the left housing 220L (for example, the surface facing the left ear when the left earphone 216L is attached to the left ear), and is attached to the left housing 220L. 220L and the left earpiece 222L. For example, the outer diameter of the left support 224L is larger than the outer diameter of the left earpiece 222L. An electrode member 228L is provided on a part or all of the side surface of the left support 224L.

For example, the electrode member 228L has a ring shape and is supported by the columnar left support 224L. The whole or part of the electrode member 228L functions as an electrode. That is, an electrode may be provided on the entire surface of the electrode member 228L, or an electrode may be provided on a part of the surface of the electrode member 228L, and no electrode may be provided on the surface of that part. The electrode member 228L is made of conductive rubber made of carbon, for example. In this case as well, the entire electrode member 228L may be made of conductive rubber, and the entire electrode member 228L (for example, the entire surface) may function as an electrode, or a portion of the electrode member 228L may be made of conductive rubber. A portion (for example, a portion of the surface) of the electrode member 228L may function as an electrode.

The left earpiece 222L is provided at the tip of the left support 224L (that is, the end opposite to the end connected to the left housing 220L). For example, the left earpiece 222L has a cylindrical shape whose width becomes narrower toward the tip. Of course, this shape is just an example, and the left earpiece 222L may have another shape.

A sound conduit etc. is housed in the left earpiece 222L, and the sound emitted by the speaker is transmitted through the sound conduit etc. in the left earpiece 222L and is emitted from the left earpiece 222L to the outside. An electrode is provided on a part or all of the outer surface (for example, the side surface, etc.) of the left earpiece 222L. The electrode is made of conductive rubber made of carbon, for example. The left earpiece 222L itself may be made of conductive rubber. For example, the entire or part of the left earpiece 222L may be made of conductive rubber. That is, the entire surface or a portion of the surface of the left earpiece 222L may function as an electrode.

The left earpiece 222L and the electrode member 228L may be made of, for example, an elastic member. As the elastic member, for example, resin such as rubber is used. Specifically, Si-based rubber (for example, S1734 manufactured by NOK), urethane-based rubber, or the like may be used for the left earpiece 222L and the electrode member 228L. Further, the hardness of each of the left earpiece 222L and the electrode member 228L (for example, hardness according to the durometer type A (instantaneous) standard) is, for example, 40 to 75. As an example, a resin having a hardness of 70 may be used for the left earpiece 222L and the electrode member 228L.

As will be described later, the left earpiece 222L is inserted into the external auditory canal of the right ear and placed in the right ear so as to be in contact with the external auditory canal, and the electrode member 228L supported by the left support 224L is connected to the concha of the left ear. It is placed in the left ear so that it contacts the cavity.

The left ear hook 226L has a generally curved shape, and is a member that is hung on the user's left ear when the user wears the left earphone 216L. For example, the left ear hook 226L is hung on the helix of the left ear. To explain in more detail, the left ear hook 226L is provided in contact with the helix on the back side of the helix. One end of the left ear hook 226L is connected to the front side of the left housing 220L, and the left ear hook 226L has a curved shape from the connecting portion to the rear side of the left housing 220L. This part constitutes the curved part. The curved portion is provided in contact with the helix on the back side of the helix of the left ear. For example, the curved portion has a shape that follows the shape of the back side of the helix. The other portion of the left ear hook 226L is connected to one end of the cable 218.

The left earpiece 222L and the left support 224L are replaceable members. For example, multiple types (for example, 3 to 5 types) of left earpieces 222L and left side supports 224L with different shapes and sizes are prepared in advance, and the shape of the user's ear (for example, the external auditory canal, concha cavity, etc.) is prepared in advance. The left earpiece 222L and the left support 224L can be replaced according to the location of the player.

Note that the control device, communication device, electronic circuit, 6-axis sensor, memory, etc. may be stored in either the right housing 220R or the left housing 220L, or may be stored in the right housing 220R and the left housing 220L. It may be stored in both.

Further, the right side housing 220R or the left side housing 220L is provided with a power button, a switch for adjusting the volume, and the like. A power button, a switch, etc. may be provided on both the right side housing 220R and the left side housing 220L.

For example, either the electrode provided on the right earpiece 222R or the electrode provided on the left earpiece 222L is used as a sensor electrode, and the other electrode is used as a reference electrode. Further, the electrode member 228R and the electrode member 228L are used as ground electrodes. Of course, either one of the electrode member 228R or the electrode member 228L may be used as a sensor electrode, the other may be used as a reference electrode, and the electrodes provided on the right earpiece 222R and the left earpiece 222L may be used as a ground electrode. good.

As another example, the right earpiece 222R may be provided with a plurality of electrodes separated from each other, and at least one of the plurality of electrodes may be used as a sensor electrode, a reference electrode, or a ground electrode. For example, the plurality of electrodes are used as sensor electrodes, and the potential detected by the electrode with the highest detection sensitivity, the electrode with the least noise, or the electrode with the most stable noise level among the plurality of electrodes. may be used as the sensor potential. The same applies when the plurality of electrodes are used as a reference electrode or a ground electrode. The same applies to the left earpiece 222L, the electrode member 228R, and the electrode member 228L.

As shown in FIG. 34, the electrode member 228R supported by the right support 224R is installed at a height h from the right housing 220R. In other words, the distance between the electrode member 228R and the right housing 220R is set to the height h. The height h is set, for example, to a value that prevents interference between the right side housing 220R and the ear helix and prevents poor contact. Specifically, when the right earpiece 222R is inserted into the external auditory canal of the right ear and the right earphone 216R is attached to the right ear, the height h is set such that the right housing 220R does not come into contact with the helix of the right ear. Set. In other words, the height h is set so that the right housing 220R is separated from the hemisphere to a position where it does not come into contact with the hemisphere. By doing so, for example, the upper surface of the right housing 220R does not come into contact with the helix. If the right housing 220R contacts the ear helix and interferes with the ear helix, the right ear piece 222R will not be inserted to a stable position within the ear canal, resulting in poor contact between the electrodes of the right ear piece 222R and the ear canal. may occur. Further, the electrode member 228R is not placed in a position where it can make good contact with the concha cavity of the right ear, and there is a possibility that poor contact occurs between the electrode member 228R and the concha cavity of the right ear. When such poor contact occurs, the detection sensitivity of the sensor potential, reference potential, and ground potential may decrease, and the measurement accuracy of biological information may decrease. By setting the height h so that the right housing 220R does not come into contact with the helix, the right housing 220R and the helix do not interfere, and the right earpiece 222R can be inserted to a stable position in the ear canal. Good contact between the electrode of the right earpiece 222R and the external auditory canal can be achieved. Furthermore, the electrode member 228R can be placed at a position where it makes good contact with the concha cavity of the right ear, and the contact between the electrode member 228R and the concha cavity can be made good. By doing so, the detection sensitivity of biopotential can be improved. The same applies to the left earphone 216L.

Further, as shown in FIG. 30, when the right earphone 216R is viewed from above, the right ear hook 226R is disposed offset from the position of the right housing 220R in the direction in which the right earpiece 222R is installed. . That is, when the right earphone 216R is viewed from above, a gap is formed between the right housing 220R and the right ear hook 226R. It can be said that the right ear hook 226R is arranged offset from the position of the right housing 220R in the direction in which the right earpiece 222R is installed. The gap is an ear space, and the right ear is inserted into the gap. By creating such an ear space and inserting the right ear into the ear space, the right ear can be placed in the ear space regardless of the size of the right ear, and the right earphone can be placed in the ear space. 216R can be stably worn in the right ear. As a result, it is possible to improve the contact between the electrode of the right earpiece 222R and the external auditory canal, and the contact between the electrode member 228R and the concha cavity. The same applies to the left earphone 216L.

As shown in FIG. 35, a conductive tube member 230R made of metal is provided on the right support body 224R so as to protrude in a direction away from the right housing 220R. The rear end of the conductive tube member 230R is installed on the right support 224R, and the right earpiece 222R is provided at the tip of the conductive tube member 230R. For example, the rear end of the conductive tube member 230R is attached to the tip of the right support 224R by screwing or the like. A conductive tube member 232R made of metal is provided on the side surface of the right support 224R in a direction protruding from the side surface. Further, a ring-shaped electrode member 228R is provided on the side surface of the ring-shaped electrode member 228R so as to cover the conductive tube member 232R. To explain in more detail, a groove is formed on the side surface of the right support 224R along the circumferential direction of the right support 224R, and a hole 224R1 into which one end of the conductive tube member 232R is inserted is formed in the groove. has been done. By inserting one end of the conductive tube member 232R into the hole 224R1, the conductive tube member 232R is attached to the side surface of the right support 224R. For example, the conductive tube member 232R is attached to the side surface of the right support 224R by screwing or the like. With the conductive tube member 232R attached to the side surface of the right side support 224R, the electrode member 228R was fitted into the groove on the side surface of the right side support 224R, so that the electrode member 228R came into contact with the conductive tube member 232R. In this state, it is attached to the side surface of the right support 224R. The conductive tube member 232R is connected to the electric wire within the right support 224R. As a result, a potential signal indicating the potential detected by the electrode member 228R is sent to the electric wire via the conductive tube member 232R, and output to the main board 234, which will be described later, via the electric wire. The left earphone 216L also has the same configuration as the right earphone 216R.

Further, as shown in FIG. 36, a main board 234 and a sub-board 236 are housed in the right housing 220R. Boards other than the main board 234 and the sub-board 236 may be accommodated in the left housing 220L.

Further, as shown in FIG. 37, a battery 238 and a relay board 240 are housed in the left side housing 220L. Power is supplied from the battery 238 to the right earphone 216R and the left earphone 216L via the relay board 240.

Cable 218 has a stiffness that allows cable 218 to maintain its overall shape. For example, when the hearable device 200 is worn on a person's ear, the cable 218 is routed rearward from the right housing 220R and the left housing 220L so that the cable 218 does not come into contact with the back of the person's head or the hair on the back of the head. It has a shape that extends toward the target. In this way, since the cable 218 does not come into contact with the back of the head or the hair on the back of the head, it is possible to prevent noise caused by the contact from occurring in the biological information.

Even if the cable 218 were to come into contact with the back of the head, etc., the load due to the contact would be transmitted to the cable 218, and the entire left earphone 216L would rotate toward the rear, as indicated by arrow Y in FIG. do. Thereby, the left earpiece 222L and the electrode member 228L are more firmly fixed to the left ear. In other words, the entire left earphone 216L rotates in a direction to more firmly fix the left earpiece 222L and the electrode member 228L. As a result, the contact between the electrode of the left earpiece 222L and the external auditory canal and the contact between the electrode member 228L and the concha cavity can be improved, and detection sensitivity can be improved. The same applies to the right earphone 216R.

Note that the hearable device 200 may be a device capable of bone conduction output. This allows information to be transmitted to people through bone conduction rather than sound. For example, even a deaf user can use the hearable device 200 to measure biological information such as brain waves and transmit information to the user through bone conduction.

Each of the embodiments described above may be performed using the hearable device 200. That is, the user's biometric information may be measured using the hearable device 200, and each embodiment may be executed using the measured biometric information. Of course, by using the hearable device 200 and the other biological information measuring device 12, multiple types of biological information can be measured, and based on the measured multiple types of biological information, the state of the person (e.g., emotional information, mental information, psychological information, etc.) may be obtained.

In each of the above embodiments, the processor refers to a processor in a broad sense, and may include a general-purpose processor (for example, CPU: Central Processing Unit, etc.) or a dedicated processor (for example, GPU: Graphics Processing Unit, ASIC: Application Specific Integrator). ated circuit, FPGA: field programmable gate array, programmable logic device, etc.). Further, the operation of the processor in each of the above embodiments may be performed not only by one processor, but also by a plurality of processors located at physically separate locations. Further, the order of each operation of the processor is not limited to the order described in each of the above embodiments, and may be changed as appropriate.

Reference Signs List 10 information processing device, 12 biological information measuring device, 14 environmental information measuring device, 16 equipment, 18 terminal device.

Claims (20)

The processor
obtain biometric information of at least one user;
of the at least one user based on biometric information including at least the brain waves of the at least one user, and historical information including at least one time series of context information indicating the context at each point in time when the biometric information was measured. Determine the user's status,
outputting the state of the at least one user to the at least one user;
Information processing method. The historical information is classified based on a plurality of time series, and at least one state of the at least one user is determined by analyzing the biological information and the history information for each of the plurality of time series. The information processing method according to claim 1. Each of the plurality of time series is assigned a weight according to the context in which the user is placed, and the at least one state determined for the at least one user has a priority determined by the weight. 3. The information processing method according to claim 2, wherein a subset of the at least one state determined for the at least one user is output according to the weight. The information processing method according to claim 1, wherein the state of the at least one user is determined based on a combination of the biometric information and the history information collected in a plurality of time series. The information processing method according to claim 1, wherein the at least one user's status is output to at least one other user. determining a proposed intervention measure based on state or history information of the at least one user or the at least one other user; and applying the proposed intervention measure to the at least one user or the at least one other user; further comprising outputting to one other user;
6. Information processing method according to claim 5, wherein at least one intervention device for outputting the proposed intervention means is selected from the environment in which the at least one user is present. 2. The method of claim 1, further comprising: determining a proposed intervention based on state or historical information of the at least one user; and outputting the proposed intervention to the at least one user. information processing methods. The information processing method according to claim 1, further comprising determining a state of the at least one user based on a frequency spectrum of a biopotential measured from the at least one user. The processor is included in a hearable device that includes a biological information measuring device, an audio input/output device, and a memory,
outputting the proposed intervention measure to the at least one user via the audio input/output device, obtaining a response of the at least one user to the proposed intervention measure; storing the response in said memory;
The information processing method according to claim 7. a first mode in which the response is obtained from the at least one user before executing the proposed intervention; or a second mode in which the proposed intervention is executed automatically by artificial intelligence. The information processing method according to claim 9, further comprising selecting one. Claim 7, further comprising performing the proposed intervention means by transmitting at least one of light, images, video, sounds, sounds, music, vibrations, temperature changes, bumps, stimuli, odors. The information processing method described in . By the processor
obtain biometric information of at least one user;
determining the state of the at least one user based on biometric information of the at least one user;
assisting the at least one user in communicating information based on the at least one user's status;
Information processing method. the at least one user includes a plurality of users;
13. The information processing according to claim 12, wherein when supporting communication of information from one user to another user, the processor transmits the information to the other user after obtaining approval from the one user. Method. The information processing method according to claim 13, further comprising storing the information in a memory associated with the one user if the one user rejects the information to be sent to the other user. . 14. If the one user rejects the information to be sent to the other user, the method further comprises: proposing to the one user an option that includes at least one alternative action. information processing methods. When the one user confirms the transmission of the information to the other user, the information is stored in a memory as learning data in association with the determined state of the one user, and the one user 14. The information processing method according to claim 13, further comprising using the learning data for the next determination of the information based on the state of the user. If the one user approves the transmission of the information to the other user, further comprising storing in memory data indicating whether the transmission of the information to the other user was successful. The information processing method according to claim 13. The information processing method according to claim 12, further comprising determining a state of the at least one user based on a frequency spectrum of a biopotential measured from the at least one user. obtain biometric information of at least one user;
of the at least one user based on biometric information including at least the brain waves of the at least one user, and historical information including at least one time series of context information indicating the context at each point in time when the biometric information was measured. Determine the user's status,
outputting the state of the at least one user to the at least one user;
An information processing device including a processor configured as follows. obtain biometric information of at least one user;
determining the state of the at least one user based on biometric information of the at least one user;
assisting the at least one user in communicating information based on the at least one user's status;
An information processing device including a processor configured as follows.