WO2020137352A1

WO2020137352A1 - Biosensing method, map data generation method, program, computer-readable medium, and biosensing device

Info

Publication number: WO2020137352A1
Application number: PCT/JP2019/046732
Authority: WO
Inventors: 仁村岡; 優美子大野; 聡有本
Original assignee: パナソニックＩｐマネジメント株式会社
Priority date: 2018-12-27
Filing date: 2019-11-29
Publication date: 2020-07-02

Abstract

A biosensing method according to an embodiment of the present invention includes the steps of: acquiring at least one brain activity signal indicating the activity state of the brain of the user, said brain activity signal being generated upon detection of the light component of illumination light projected on and reflected from the head of the user that has traveled through the brain of the user; and generating and outputting a signal that indicates proficiency of the user with respect to being in a mindfulness state on the basis of the least one brain activity signal.

Description

Biometric method, map data generation method, program, computer-readable recording medium, and biometric device

The present disclosure relates to a biometric method, a map data creation method, a program, a computer-readable recording medium, and a biometric device.

In recent years, research on mindfulness has been advanced in various fields such as psychology, medicine, industry, sports, welfare, and education. According to the Japan Mindfulness Society, mindfulness is defined as "just watching at this moment, without intention and without intentionally directing consciousness, evaluation." For example, as disclosed in Non-Patent Document 1, practicing mindfulness is said to have various effects such as stress reduction, disease risk reduction, and concentration improvement. .. Mindfulness techniques may be incorporated into the treatment of depression, for example. Mindfulness, which is said to be effective for the development of leadership in the industrial world, has been attracting attention in recent years. Mindfulness meditation is widely known as an exercise for achieving a state of mindfulness. Content such as audio or video for practicing mindfulness meditation is widely published, for example, on the Internet.

On the other hand, various methods for obtaining information inside the living body of a subject using light are known.

Patent Document 1 discloses a method in which a light source and a photodetector are separated from each other at a predetermined interval, and they are brought into close contact with a target portion to acquire biometric information.

[Patent Document 2] discloses a technique of acquiring information on a subject's cerebral blood flow in a non-contact manner and estimating a psychological state such as a degree of concentration or emotion of the subject.

Patent Literature 3 discloses a system that measures brain activity signals of a plurality of subjects and displays the analysis result of the biometric information of each subject and the position information in association with each other based on the positional relationship of the plurality of subjects. ing.

JP-A-11-164826 JP, 2017-124153, A JP, 2013-17734, A

Previously, there was no technology to determine whether a user is proficient in mindfulness. The present disclosure provides a novel technique that allows a user to determine whether they are proficient in entering a mindfulness state.

A biometric method according to an aspect of the present disclosure detects a component of light propagating in the brain of the user, of reflected light from the head generated by irradiating the user's head with irradiation light. And acquiring at least one brain activity signal generated by the method and indicating the activity status of the brain, and indicating the user's proficiency level of being in a mindfulness state based on the at least one brain activity signal. Generating and outputting a signal.

The comprehensive or specific aspects of the present disclosure may be realized by a recording medium such as a system, a device, a method, an integrated circuit, a computer program or a computer-readable recording disk, and the system, the device, the method, the integrated circuit, It may be realized by any combination of a computer program and a recording medium. The computer-readable recording medium may include a nonvolatile recording medium such as a CD-ROM (Compact Disc-Read Only Memory). The device may be composed of one or more devices. When the device is composed of two or more devices, the two or more devices may be arranged in one device or may be separately arranged in two or more separate devices. In this specification and claims, "device" can mean not only one device, but also a system composed of a plurality of devices.

According to an aspect of the present disclosure, it is possible to determine whether or not a user is proficient in becoming a mindfulness state.

FIG. 1A is a diagram schematically showing an example of a biometric device. FIG. 1B is a diagram showing an example of a temporal change in the intensity of light reaching the photodetector. FIG. 1C is a diagram in which the width of the input light pulse is shown on the horizontal axis and the amount of light detected by the photodetector is shown on the vertical axis. FIG. 1D is a diagram showing an example of a schematic configuration of one pixel of the photodetector. FIG. 1E is a diagram showing an example of the configuration of a photodetector. FIG. 1F is a diagram showing an example of the operation within one frame. FIG. 1G is a flowchart showing an outline of the operation of the control circuit. FIG. 2 is a diagram for explaining a method of detecting an internal scattering component of a light pulse. FIG. 3A is a diagram schematically showing an example of a timing chart when detecting a surface reflection component. FIG. 3B is a diagram schematically showing an example of a timing chart when detecting an internal scattering component. FIG. 4A is a diagram schematically showing an example of a method for detecting a change in cerebral blood flow. FIG. 4B is a diagram schematically illustrating an example of a method of simultaneously performing measurement at a plurality of locations within the target portion of the user. FIG. 5A is a diagram schematically showing an example of a light irradiation area. FIG. 5B is a diagram schematically showing a change in the measurement result due to the lateral movement of the user's head. FIG. 6 is a diagram showing on a time axis the flow of one determination operation of the proficiency level of mindfulness. FIG. 7 is a flow chart showing the flow of the determination operation of the proficiency level. FIG. 8A is a diagram showing an example of three histograms obtained for an unskilled person. FIG. 8B is a diagram showing an example of three histograms obtained for the expert. FIG. 9 is a conceptual diagram of a system for collecting and utilizing brain activity signals and position data from mobile devices of a plurality of users. FIG. 10 is a diagram showing a schematic configuration of a server and a plurality of mobile devices. FIG. 11 is a diagram showing an example of an image displayed on the display of the mobile device based on the map data. FIG. 12 is a diagram schematically showing an example of the head mount device.

Each of the embodiments described below shows a comprehensive or specific example. Numerical values, shapes, materials, constituent elements, arrangement positions and connection forms of constituent elements, steps, and order of steps shown in the following embodiments are examples, and are not intended to limit the technology of the present disclosure. Among the constituent elements in the following embodiments, the constituent elements that are not described in the independent claim indicating the highest concept are described as arbitrary constituent elements. Each drawing is a schematic view, and is not necessarily an exact illustration. Further, in each of the drawings, substantially the same or similar components are designated by the same reference numerals. Overlapping explanations may be omitted or simplified.

In the present disclosure, all or part of a circuit, unit, device, member or part, or all or part of a functional block in a block diagram may be, for example, a semiconductor device, a semiconductor integrated circuit (IC), or an LSI (large scale integration). ) Can be implemented by one or more electronic circuits including. The LSI or IC may be integrated on one chip, or may be configured by combining a plurality of chips. For example, the functional blocks other than the memory element may be integrated on one chip. Although referred to as an LSI or an IC here, the name may be changed depending on the degree of integration, and may be referred to as a system LSI, VLSI (very large scale integration), or ULSI (ultra large scale integration). The Field Programmable Gate Array (FPGA), which is programmed after the LSI is manufactured, or the reconfigurable logic device that can reconfigure the bonding relation inside the LSI or set up the circuit section inside the LSI can be used for the same purpose.

Furthermore, the functions or operations of all or some of the circuits, units, devices, members or parts can be executed by software processing. In this case, the software is recorded on a non-transitory recording medium such as one or more ROMs, optical discs, hard disk drives, etc., and when the software is executed by a processor, the functions specified by the software are recorded. It is performed by the processor and peripherals. The system or apparatus may include one or more non-transitory storage media having software recorded on it, a processor, and any required hardware devices, such as interfaces.

Before describing the embodiments of the present disclosure, the knowledge on which the present disclosure is based will be described.

As mentioned above, various contents for practicing mindfulness are published on the Internet, for example. Such content is also provided as an application for mobile devices such as smartphones or tablet computers. Such applications provide the user with audio and/or video that guides them to a state of mindfulness. The user can obtain a high relaxing effect by meditating according to the guidance, for example.

Easiness of becoming a mindfulness state varies depending on the user. Some users are proficient in mindfulness and can move to a mindfulness state with relative ease. However, other users are not proficient in mindfulness, and it is relatively difficult to shift to a mindfulness state.

Easiness of becoming a mindfulness state also depends on the surrounding environment. For example, in a place where you can experience nature, or in a historic building such as a majestic temple, temple, or temple, people are likely to be mindful. In addition, people are more likely to be in a state of mindfulness in cozy places, such as places with excellent works of art, or places with pleasant music. It can be said that people who are proficient in mindfulness easily gather in such places.

If it is possible to distinguish between those who are skilled in mindfulness and those who are not skilled in mindfulness, there is a possibility that the results of that determination can be applied to various purposes. However, heretofore, an effective method for discriminating between an expert and an inexperienced person has not been known. Therefore, the present inventors have studied a method for discriminating between an expert and an inexperienced person, and have come up with the configuration of the embodiment of the present disclosure.

The outline of the embodiments of the present disclosure will be described below.

A biometric method according to one aspect of the present disclosure includes the following steps (a) and (b).
(A) Activity of the brain generated by detecting a component of light propagating in the brain of the user among reflected light from the head generated by irradiating the user's head with irradiation light. At least one brain activity signal indicative of a condition is acquired.
(B) Generating and outputting a signal indicating the user's proficiency level for becoming a mindfulness state based on at least one brain activity signal.

Obtaining at least one brain activity signal may be performed in the state where a stimulus including at least one selected from the group consisting of audio and video is given to the user.

The at least one selected from the group consisting of the audio and the video may guide the user to the mindfulness state.

Here, the “stimulus including at least one selected from the group consisting of audio and video that guides the user to a mindfulness state” is, for example, a publicly available audio for mindfulness meditation and The content may include at least one of videos. In addition, "audio and video for mindfulness meditation" is not limited to audio and video that are intended to be used for mindfulness meditation, but it does not mean that the user is mindfulness. Includes audio and video that can be expected to induce meditation. Such audio and video may be provided, for example, as relaxing audio and video. The stimulus may be presented to the user as the user views such audio and video content.

"The component of the light that propagates in the user's brain" depends on the state of the user's brain activity. The brain activity state of the user can be estimated by detecting the light component. Here, the “light” is not limited to visible light, and may be infrared light, for example. In this specification, the term "light" is used not only for visible light but also for infrared light.

“The user's proficiency level in becoming a mindfulness state” can be determined by analyzing the acquired brain activity signal, as described in detail later.

"Brain activity signal" means a biological signal that changes depending on the brain activity of the user or the subject. For example, a signal indicating any of the oxygenated hemoglobin concentration in cerebral blood, the deoxygenated hemoglobin concentration in cerebral blood, and the oxygen saturation level in cerebral blood corresponds to a brain activity signal.

The brain activity signal is, for example, from the light intensity of the reflected light pulse of the reflected light pulse from the head generated by irradiating the user's head with the light pulse until the reduction ends. It can be generated by detecting at least a part of the components of the reflected light pulse included in the period (hereinafter, referred to as “falling period”). As will be described later, the light component included in the falling period of the reflected light pulse reflects the brain activity of the user. The state of the brain activity of the user can be estimated by detecting the light component. Furthermore, it can be determined whether the user is proficient in mindfulness.

The stimulus may be presented to the user by the user viewing the content, such as audio or video created to guide a mindfulness state. The stimuli are not limited to visual and auditory stimuli, but may be other types of stimuli, such as olfactory, tactile, or taste stimuli. The at least one brain activity signal is a first period immediately before the user is stimulated, a second period when the user is stimulated, or a third period immediately after the user is stimulated. It may be generated more than once for each of the at least two time periods, including the second time period. A signal indicating the proficiency level may be generated based on the brain activity signal acquired for each of the at least two time periods.

As will be described later, a person who is proficient in becoming mindfulness (hereinafter, referred to as “proficiency”) and a person who is not proficient (hereinafter, referred to as “non-proficiency”) are provided with images. Or, the tendency of the brain activity signal is different before and after the stimulation such as voice is given. More accurate estimation of mindfulness proficiency by analyzing not only brain activity signal during stimulation but also brain activity signal before and/or after stimulation can do.

The at least one brain activity signal includes a plurality of brain activity signals, and the plurality of brain activity signals may be obtained by detecting the light component a plurality of times in each of the first to third periods. Good. For example, the brain activity signal may be repeatedly acquired at regular intervals. In that case, the signal indicating the proficiency level may be generated based on the plurality of brain activity signals acquired for each of the first to third periods. In this way, the proficiency level of mindfulness can be estimated more accurately by acquiring multiple brain activity signals during and before and after a stimulus such as video or audio is given. be able to.

The signal indicating the proficiency level can be generated, for example, based on the degree of variation in the plurality of brain activity signals acquired for each of the first to third periods.

The at least one brain activity signal may be, for example, a signal that depends on the concentration of deoxygenated hemoglobin in blood in the brain. According to the experiments conducted by the present inventors, it has been found that the proficiency and the non-proficiency are significantly different in the tendency of the concentration change of deoxygenated hemoglobin depending on the presence or absence of stimulation. The proficiency level can be more accurately estimated by analyzing a signal depending on the concentration of deoxygenated hemoglobin in blood in the brain.

The brain activity signal may be generated based on two signals acquired by using two types of light having different wavelengths. For example, the brain activity signal is the first reflected light pulse of the first reflected light pulse from the head generated by irradiating the user's head with the light pulse of the first wavelength longer than 805 nm. The first signal generated by detecting at least a part of the components included in the falling period of the pulse and the second signal from the head generated by irradiating the optical pulse of the second wavelength shorter than 805 nm. And a second signal generated by detecting at least a part of the component included in the falling period of the reflected light pulse.

Oxygenated hemoglobin absorbs near-infrared rays with a wavelength of more than about 805 nm relatively well. In contrast, deoxygenated hemoglobin relatively well absorbs near infrared or red light with wavelengths shorter than 805 nm. The first signal can be a signal that is dependent on the concentration of oxygenated hemoglobin in cerebral blood. The second signal can be a signal that depends on the concentration of deoxygenated hemoglobin in cerebral blood. The oxygen saturation in the cerebral blood can also be determined using these two signals. By combining these two signals, the proficiency can be estimated more accurately.

A biometric device according to another aspect of the present disclosure includes a stimulator, a light source, a photodetector, and a signal processing circuit. The stimulator provides a stimulus to the user. The light source is configured to emit light toward the user's head while the stimulus is being applied to the user. The photodetector is configured to detect a component of the light that has propagated in the brain of the user in the reflected light from the head and output a brain activity signal according to the intensity of the component. The signal processing circuit is configured to generate and output, based on the brain activity signal, a signal indicating a user's level of proficiency with respect to becoming a mindfulness state.

The present disclosure also includes a program executed by a computer mounted on a biometric device that includes a stimulator, a light source, and a photodetector. The program causes a computer to execute the following operations (a) to (d).
(A) Cause the stimulator to give a stimulus to the user.
(B) The light source emits light toward the head of the user while the stimulus is presented to the user.
(C) The photodetector is caused to detect the component of the light propagating in the brain of the user in the reflected light from the head, and to output the brain activity signal according to the intensity of the component.
(D) Based on the brain activity signal, a signal indicating the user's proficiency level for becoming a mindfulness state is generated and output.

A computer-readable recording medium according to an aspect of the present disclosure stores a program that is executed by a computer installed in a biometric device that includes a stimulator, a light source, and a photodetector. When the program is executed by the computer, the stimulator is caused to give a stimulus to a user, and the stimulus is given to the user, the light source is set to By emitting light toward the head and causing the photodetector to detect the component of the light that has propagated in the brain of the user among the reflected light from the head, the intensity of the component can be increased. And outputting a brain activity signal indicating the activity state of the brain, and generating and outputting a signal indicating the user's proficiency level for becoming a mindfulness state based on the brain activity signal. And are executed.

The technology of the present disclosure can be applied to, for example, an application for a mobile device such as a smartphone or a tablet computer. A server computer connected to a plurality of mobile devices via a network such as the Internet may execute the above method. The server computer may collect a brain activity signal of each user via a network, and generate and output a signal indicating the proficiency level of each user with respect to becoming mindfulness. The signal indicating the proficiency level of each user may be transmitted to the mobile device of the user, and the determination result may be displayed on the display of the device.

A map data generation method according to an aspect of the present disclosure, a signal indicating the proficiency level of each of the plurality of users generated by the biometric method according to any one of the above from a computer of a plurality of users, and Acquiring position information of each of a plurality of users via a network, and based on the position information, generates and outputs map data in which the proficiency level of each of the plurality of users is associated with a map. And that.

Alternatively, the server computer may acquire the signal indicating the proficiency level generated by the mobile device of each user and the position information from the computers of the plurality of users via the network. Further, the server computer may generate and output map data in which the determination result of the mindfulness proficiency level of each user and the map are associated with each other based on the position information. Such map data may represent a heat map showing the distribution of mindfulness masters. For example, a heat map may be generated in which areas with high levels of proficiency are shown in red, areas with low proficiency are shown in blue, and areas with a medium number of proficiency are shown in green. The map data may be transmitted to the mobile device of each user, and the heat map may be displayed on the display of the device.

With such a configuration, each user can know in which place many experts are gathered. The place where many skilled people gather is likely to be a place where they are likely to be in a state of mindfulness, in other words, a place where they can feel comfortable and relax. By using the above application, the user can know a place where he/she can feel comfortable and relax. Further, as will be described later, by utilizing the information collected from the mobile devices of a plurality of users for marketing, it becomes possible to create a new business opportunity.

Hereinafter, embodiments of the present disclosure will be described more specifically with reference to the accompanying drawings.

(Embodiment)
[1. Biometric device 100]
FIG. 1A is a schematic diagram illustrating a biometric device 100 according to an exemplary embodiment of the present disclosure. The biometric device 100 includes a stimulator 10, a light source 20, a photodetector 30, a control circuit 60, and a signal processing circuit 70. FIG. 1A also shows a user 500 who uses the biometric device 100.

The stimulator 10 is a device that gives a stimulus to the user 500 to induce a mindfulness state. The light source 20 is a device that emits pulsed light with which the head of the user 500 is irradiated. The light source 20 is not limited to a single light emitting device, and may be realized by a combination of a plurality of light emitting devices. The photodetector 30 is a device that detects at least a part of the reflected pulsed light returning from the head of the user 500. The photodetector 30 may be, for example, an image sensor including a plurality of photodetector cells arranged two-dimensionally. The control circuit 60 is a circuit that controls the stimulator 10, the light source 20, and the photodetector 30. The signal processing circuit 70 is a circuit that processes the signal output from the photodetector 30.

The control circuit 60 in the present embodiment includes a stimulation control unit 61 that controls the stimulation device 10, a light source control unit 62 that controls the light source 20, and a detector control unit 63 that controls the photodetector 30. The stimulation control unit 61, the light source control unit 62, and the detector control unit 63 may be realized by three separate circuits or may be realized by a single circuit. The stimulus control unit 61, the light source control unit 62, and the detector control unit 63 may be realized by the control circuit 60 executing a control program stored in a memory (not shown). The functions of the control circuit 60 and the signal processing circuit 70 may be realized by a single circuit.

The stimulus control unit 61 controls, for example, at least one of hue, saturation, and brightness of a video given as a stimulus, or at least one of sound quality and loudness of sound. The light source control unit 62 controls, for example, the intensity, pulse width, emission timing, and/or wavelength of the pulsed light emitted from the light source 20. The detector control unit 63 controls the timing of signal accumulation in each photodetecting cell of the photodetector 30.

In the present specification, “biological information” means a measurable amount of a living body that changes due to stimulation. Biological information, for example, blood flow, blood pressure, heart rate, pulse rate, respiratory rate, body temperature, EEG, oxygenated hemoglobin concentration in blood, deoxygenated hemoglobin concentration in blood, blood oxygen saturation, skin Various quantities are included, such as reflectance spectra. A part of biometric information is sometimes called a vital sign.

Below, each component of the biometric device 100 will be described more specifically.

[1-1. Stimulator 10]
The stimulator 10 presents a stimulus to the user 500. The stimulation causes a biological reaction of the user 500. The stimulator 10 may include at least one of a display and a speaker. The stimulator 10 provides the user 500 with a stimulus including at least one of audio and video that guides the user 500 to a mindfulness state.

The stimulus that induces the mindfulness state may be, for example, a stimulus such as audio and/or video that has been published for mindfulness meditation.

The stimulator 10 may be, for example, a mobile device such as a smartphone or a tablet computer. In that case, the stimulus that induces the mindfulness state may be provided by an application installed on the mobile device. The stimulator 10 may be a device such as a head mounted device.

In this embodiment, visual and/or auditory stimuli are used. However, the present invention is not limited to these, and tactile, olfactory, or taste stimuli may be used. The stimulation device 10 has different structures and functions depending on the type of stimulation given to the user 500. For example, when applying a tactile stimulus to the user 500, the stimulator 10 may include a device that generates vibration or heat. When giving the user 500 a sense of smell, the stimulating device 10 may include a device that generates an odor.

[1-2. Light source 20]
The light source 20 emits light toward the target part of the user 500. The target part may be, for example, the head of the user 500, and more specifically, the forehead of the user 500. The light emitted from the light source 20 and reaching the user 500 is divided into a surface reflection component I1 reflected on the surface of the user 500 and an internal scattering component I2 scattered inside the user 500. The internal scattering component I2 is a component that is reflected or scattered once or multiple-scattered inside the living body. When the light is emitted toward the head of the user 500, the internal scattered component I2 reaches a site 8 mm to 16 mm deep from the surface of the head of the user 500, for example, the brain, and returns to the biometric device 100 again. Refers to ingredients. The surface reflection component I1 includes three components, a direct reflection component, a diffuse reflection component, and a scattered reflection component. The direct reflection component is a reflection component having the same incident angle and reflection angle. The diffuse reflection component is a component that is diffused and reflected by the uneven shape of the surface. The scattered reflection component is a component that is scattered and reflected by the internal tissue near the surface. When light is emitted toward the head of the user 500, the scattered reflection component is a component that is scattered and reflected inside the epidermis. The surface reflection component I1 that reflects off the surface of the user 500 may include these three components. The traveling directions of the surface reflection component I1 and the internal scattering component I2 change due to reflection or scattering, and part of them reaches the photodetector 30.

In the present embodiment, at least a part of the internal scattering component I2 is detected from the reflected light that is the light returning from the head of the user 500. The intensity of the internal scattered component I2 varies, reflecting the brain activity of the user 500. Therefore, the state of the brain activity of the user 500 can be estimated by analyzing the change over time of the internal scattering component I2.

Below, an example of a method for detecting the internal scattering component I2 will be described. The light source 20 repeatedly emits an optical pulse a plurality of times at a predetermined time interval or a predetermined timing according to an instruction from the control circuit 60. The light pulse emitted from the light source 20 may be, for example, a rectangular wave having a fall period close to zero. In the present specification, the “falling period” means a period from when the intensity of the light pulse starts to decrease to when the decrease ends. In general, the light incident on the user 500 propagates in the user 500 through various routes and exits from the surface of the user 500 with a time difference. Therefore, the rear end of the internal scattering component I2 of the light pulse has a spread. When the target part of the user 500 is the forehead, the spread of the rear end of the internal scattering component I2 is about 4 ns. Considering this, the falling period of the optical pulse can be set to, for example, 2 ns or less, which is half or less of that. The fall period may be half that, or 1 ns or less. The length of the rising period of the light pulse emitted from the light source 20 is arbitrary. In the present specification, the “rising period” is a period from when the intensity of the light pulse starts to increase to when the increase ends. In the detection of the internal scattered component I2 in the present embodiment, the falling portion of the light pulse is used and the rising portion is not used. The rising portion of the light pulse can be used for detecting the surface reflection component I1. The light source 20 can be, for example, a laser such as an LD. The light emitted from the laser has a steep time response characteristic in which the falling portion of the light pulse is substantially perpendicular to the time axis.

The wavelength of the light emitted from the light source 20 may be any wavelength included in the wavelength range of 650 nm or more and 950 nm or less, for example. This wavelength range is included in the wavelength range from red to near infrared. The above wavelength range is called "living body window" and has a property that light is relatively hard to be absorbed by moisture and skin in the living body. When a living body is to be detected, the detection sensitivity can be increased by using light in the above wavelength range. When detecting a change in the blood flow in the brain of the user 500 as in the present embodiment, it is considered that the light used is mainly absorbed by oxygenated hemoglobin (HbO ₂ ) and deoxygenated hemoglobin (Hb). .. Oxygenated hemoglobin and deoxygenated hemoglobin have different wavelength dependence of light absorption. Generally, when blood flow changes, the concentrations of oxygenated hemoglobin and deoxygenated hemoglobin change. Along with this change, the degree of light absorption also changes. Therefore, when the blood flow changes, the detected light amount also changes with time.

The light source 20 may emit light of a single wavelength included in the above wavelength range, or may emit light of two or more wavelengths. Light of a plurality of wavelengths may be emitted from each of a plurality of light sources. When two lights having different wavelengths are emitted from the two light sources, the optical paths of the lights having the two wavelengths returned to the photodetector 30 via the target portion of the user 500 are designed to be substantially equal to each other. obtain. In this design, for example, the distance between the photodetector 30 and one light source and the distance between the photodetector 30 and the other light source are the same, and the two light sources are centered on the photodetector 30. It may be arranged in a rotationally symmetrical position.

With the biometric device 100 according to the present embodiment, the cerebral blood flow of the user 500 is measured without contact. Therefore, the light source 20 designed in consideration of the influence on the retina can be used. For example, the light source 20 that satisfies Class 1 of the laser safety standard established in each country may be used. When the class 1 is satisfied, the user 500 is irradiated with light having low illuminance such that the exposure limit (AEL) is less than 1 mW. The light source 20 itself does not have to satisfy Class 1. For example, a laser safety standard Class 1 may be met by placing a diffuser or ND filter in front of the light source 20 to diffuse or attenuate the light.

When irradiating the head of the user 500 with light in order to measure cerebral blood flow, the amount of light of the internal scattering component I2 is several thousandth to several tens of thousands, which is very small. It can be a small value. Furthermore, considering the safety standards of lasers, the amount of light that can be emitted is extremely small. Therefore, the detection of the internal scattered component I2 is very difficult. Even in that case, if the light source 20 emits a light pulse having a relatively large pulse width, the integrated amount of the internal scattering component I2 with a time delay can be increased. Thereby, the amount of detected light can be increased and the SN ratio can be improved.

The light source 20 emits an optical pulse having a pulse width of 3 ns or more, for example. In general, the temporal spread of light scattered in a living tissue such as the brain is about 4 ns. FIG. 1B is a diagram showing an example of a temporal change in the intensity of light reaching the photodetector 30. FIG. 1B shows an example of three cases in which the width of the input light pulse emitted from the light source 20 is 0 ns, 3 ns, and 10 ns. As shown in FIG. 1B, as the width of the light pulse from the light source 20 is increased, the light amount of the internal scattered component I2 that appears at the rear end of the light pulse returned from the user 500 increases.

FIG. 1C is a diagram in which the horizontal axis represents the width of the input light pulse and the vertical axis represents the amount of light detected by the photodetector 30. The photodetector 30 includes an electronic shutter. The result of FIG. 1C was obtained under the condition that the electronic shutter was opened 1 ns after the time when the trailing edge of the light pulse was reflected by the surface of the user 500 and reached the photodetector 30. The reason for selecting this condition is that the ratio of the surface reflection component I1 is higher than that of the internal scattering component I2 immediately after the rear end of the light pulse arrives. As shown in FIG. 1C, when the pulse width of the light pulse emitted from the light source 20 is 3 ns or more, the detected light amount can be maximized.

The light source 20 may emit an optical pulse having a pulse width of 5 ns or more, and further 10 ns or more. On the other hand, if the pulse width is too large, the amount of unused light increases and it is wasted. Therefore, the light source 20 emits an optical pulse having a pulse width of 50 ns or less, for example. Alternatively, the light source 20 may emit a light pulse having a pulse width of 30 ns or less, further 20 ns or less.

The irradiation pattern of the light source 20 may be, for example, a pattern having a uniform intensity distribution in the irradiation area. In this respect, the present embodiment differs from the conventional biometric device disclosed in Patent Document 1. In the device disclosed in Patent Document 1, the detector and the light source are separated by about 3 cm, and the surface reflection component is spatially separated from the internal scattering component. Therefore, there is no choice but to use discrete light irradiation. In contrast, the biometric device 100 according to the present embodiment can reduce the surface reflection component I1 by temporally separating it from the internal scattering component I2. Therefore, the light source 20 having an irradiation pattern having a uniform intensity distribution can be used. The irradiation pattern having a uniform intensity distribution may be formed by diffusing the light emitted from the light source 20 with a diffusion plate.

In the present embodiment, the internal scattered component I2 can be detected even just below the irradiation point of the user 500. The measurement resolution can also be increased by illuminating the user 500 with light over a wide spatial range.

[1-3. Photodetector 30]
The photodetector 30 outputs a signal indicating the amount of a part of the light emitted from the light source 20 and returned from the target portion of the user 500. The signal may be a signal according to the intensity included in at least a part of the falling period of the reflected light pulse.

The photodetector 30 may include a plurality of photoelectric conversion elements 32 and a plurality of charge storage sections 34. Specifically, the photodetector 30 may include a plurality of photodetector cells arranged two-dimensionally. Such a photodetector 30 can acquire the two-dimensional biometric information of the user 500 at one time. In the present specification, the light detection cell may be referred to as a "pixel". The photodetector 30 can be, for example, any image sensor such as a CCD image sensor or a CMOS image sensor. More generally, the photodetector 30 includes at least one photoelectric conversion element 32 and at least one charge storage section 34.

The photodetector 30 may include an electronic shutter. The electronic shutter is a circuit that controls the timing of image capturing. In this embodiment, the detector control unit 63 in the control circuit 60 has a function of an electronic shutter. The electronic shutter controls a single signal accumulation period in which the received light is converted into an effective electric signal and accumulated, and a period in which the signal accumulation is stopped. The signal accumulation period can also be referred to as an “exposure period”. In the following description, the width of the exposure period may be referred to as the “shutter width”. The time from the end of one exposure period to the start of the next exposure period may be referred to as the "non-exposure period". Hereinafter, the exposure state may be referred to as “OPEN”, and the exposure stop state may be referred to as “CLOSE”.

The photodetector 30 can adjust the exposure period and the non-exposure period by sub-nanosecond, for example, in the range of 30 ps to 1 ns by using the electronic shutter. The conventional TOF camera whose purpose is to measure the distance detects all the light emitted from the light source 20 and reflected by the subject and returned. In the conventional TOF camera, the shutter width needs to be larger than the pulse width of light. On the other hand, in the biometric device 100 according to the present embodiment, it is not necessary to correct the light amount of the subject. Therefore, the shutter width need not be larger than the pulse width. The shutter width can be set to a value of 1 ns or more and 30 ns or less, for example. According to the biometric device 100 in the present embodiment, the shutter width can be reduced, so that the influence of the dark current included in the detection signal can be reduced.

When illuminating the head of the user 500 with light to detect information such as cerebral blood flow, the light attenuation rate inside the living body is very large. For example, the emitted light may be attenuated to about one millionth of the incident light. For this reason, in order to detect the internal scattered component I2, the light amount may be insufficient by irradiating only one pulse. In the case of irradiation in Class 1 of the laser safety standard, the light amount is particularly weak. In this case, the light source 20 emits the light pulse a plurality of times, and the photodetector 30 is also exposed a plurality of times by the electronic shutter in response thereto, whereby the detection signals can be integrated to improve the sensitivity.

Hereinafter, a configuration example of the photodetector 30 will be described.

The photodetector 30 may include a plurality of pixels arranged two-dimensionally on the imaging surface. Each pixel may include a photoelectric conversion element such as a photodiode and one or more charge storage units. Hereinafter, each pixel has a photoelectric conversion element that generates a signal charge according to the amount of received light by photoelectric conversion, a charge storage unit that stores the signal charge generated by the surface reflection component I1 of the light pulse, and an internal scattering component of the light pulse. An example including a charge storage unit that stores the signal charge generated by I2 will be described. In the following example, the control circuit 60 causes the photodetector 30 to detect the surface reflection component I1 by detecting the portion of the optical pulse returned from the head of the user 500 before the start of the fall. The control circuit 60 also causes the photodetector 30 to detect the internal scattered component I2 by detecting the portion of the optical pulse returning from the head of the user 500 after the start of the falling. The light source 20 in this example emits light of two types of wavelengths.

FIG. 1D is a diagram showing an example of a schematic configuration of one pixel 201 of the photodetector 30. Note that FIG. 1D schematically illustrates the structure of one pixel 201, and does not necessarily reflect the actual structure. The pixel 201 in this example includes a photodiode 203 that performs photoelectric conversion, a first floating diffusion layer (Floating Diffusion) 204 that is a charge storage unit, a second floating diffusion layer 205, a third floating diffusion layer 206, and It includes a fourth floating diffusion layer 207 and a drain 202 that drains signal charges.

Photons that have entered each pixel due to one emission of a light pulse are converted by the photodiode 203 into signal electrons that are signal charges. The converted signal electrons are discharged to the drain 202 or distributed to either the first floating diffusion layer 204 to the fourth floating diffusion layer 207 according to the control signal input from the control circuit 60.

Emission of an optical pulse from the light source 20 and accumulation of signal charges in the first floating diffusion layer 204, the second floating diffusion layer 205, the third floating diffusion layer 206, and the fourth floating diffusion layer 207, The discharge of the signal charge to the drain 202 is repeated in this order. This repetitive operation is fast, and can be repeated tens of thousands to hundreds of millions of times within one frame of a moving image, for example. The time for one frame is, for example, about 1/30 second. The pixel 201 finally generates and outputs four image signals based on the signal charges accumulated in the first floating diffusion layer 204 to the fourth floating diffusion layer 207.

The control circuit 60 in this example causes the light source 20 to repeatedly emit the first light pulse having the first wavelength and the second light pulse having the second wavelength in order. By selecting two wavelengths having different absorption rates in the internal tissue of the user 500 as the first wavelength and the second wavelength, the state of the user 500 can be analyzed. For example, a wavelength longer than 805 nm may be selected as the first wavelength and a wavelength shorter than 805 nm may be selected as the second wavelength. This makes it possible to detect changes in the oxygenated hemoglobin concentration and the deoxygenated hemoglobin concentration in the blood of the user 500.

The control circuit 60 first causes the light source 20 to emit the first light pulse. The control circuit 60 accumulates the signal charge in the first floating diffusion layer 204 during the first period in which the surface reflection component I1 of the first light pulse is incident on the photodiode 203. Subsequently, the control circuit 60 accumulates signal charges in the second floating diffusion layer 205 during the second period in which the internal scattered component I2 of the first light pulse is incident on the photodiode 203. Next, the control circuit 60 causes the light source 20 to emit the second light pulse. The control circuit 60 accumulates signal charges in the third floating diffusion layer 206 during the third period in which the surface reflection component I1 of the second light pulse is incident on the photodiode 203. Subsequently, the control circuit 60 accumulates signal charges in the fourth floating diffusion layer 207 during the fourth period in which the internal scattering component I2 of the second light pulse is incident on the photodiode 203.

As described above, the control circuit 60 starts the emission of the first optical pulse, and then leaves the first floating diffusion layer 204 and the second floating diffusion layer 205 from the photodiode 203 with a predetermined time difference. The signal charges are sequentially accumulated. After that, the control circuit 60 starts the emission of the second light pulse, and then makes a predetermined time difference to the third floating diffusion layer 206 and the fourth floating diffusion layer 207, and outputs the signal from the photodiode 203. The charges are accumulated in sequence. The above operation is repeated a plurality of times. In order to estimate the amounts of ambient light and ambient light, a period during which signal charges are accumulated in another floating diffusion layer (not shown) may be provided with the light source 20 turned off. By subtracting the signal charge amount of the other floating diffusion layer from the signal charge amount of the first floating diffusion layer 204 to the fourth floating diffusion layer 207, a signal from which ambient light and ambient light components are removed can be obtained. it can.

In this embodiment, the number of charge storage units is four, but it may be designed to be two or more depending on the purpose. For example, when the surface reflection component I1 and the internal scattering component I2 are detected by using only one type of wavelength, the number of charge accumulating portions may be two. In addition, when the number of wavelengths to be used is one and the surface reflection component I1 is not detected, the number of charge storage units for each pixel may be one. Further, even when two or more types of wavelengths are used, the number of charge storage units may be one if the imaging using each wavelength is performed in another frame. Similarly, even in the case of detecting both the surface reflection component I1 and the internal scattering component I2, the number of charge accumulating portions may be 1 in a configuration in which they are detected in different frames.

FIG. 1E is a diagram showing an example of the configuration of the photodetector 30. In FIG. 1E, a region surrounded by a two-dot chain line frame corresponds to one pixel 201. The pixel 201 includes one photodiode. Although FIG. 1E shows only four pixels arranged in two rows and two columns, a larger number of pixels may be actually arranged. The pixel 201 includes a first floating diffusion layer 204 to a fourth floating diffusion layer 207. The signals accumulated in the first floating diffusion layer 204 to the fourth floating diffusion layer 207 are treated as if they were signals of four pixels of a general CMOS image sensor and output from the photodetector 30.

Each pixel 201 has four signal detection circuits. Each signal detection circuit includes a source follower transistor 309, a row selection transistor 308, and a reset transistor 310. In this example, the reset transistor 310 corresponds to the drain 202 shown in FIG. 1D, and the pulse input to the gate of the reset transistor 310 corresponds to the drain discharge pulse. Each transistor is, for example, a field effect transistor formed on a semiconductor substrate, but is not limited to this. As shown, one of the input terminal and the output terminal of the source follower transistor 309 is connected to one of the input terminal and the output terminal of the row selection transistor 308. The one of the input terminal and the output terminal of the source follower transistor 309 is typically the source. The one of the input terminal and the output terminal of the row selection transistor 308 is typically the drain. The gate, which is the control terminal of the source follower transistor 309, is connected to the photodiode 203. The signal charge of holes or electrons generated by the photodiode 203 is stored in the floating diffusion layer which is a charge storage unit between the photodiode 203 and the source follower transistor 309.

Although not shown in FIG. 1E, the first floating diffusion layer 204 to the fourth floating diffusion layer 207 are connected to the photodiode 203. A switch may be provided between the photodiode 203 and each of the first floating diffusion layer 204 to the fourth floating diffusion layer 207. This switch switches the conduction state between the photodiode 203 and each of the first floating diffusion layer 204 to the fourth floating diffusion layer 207 in response to the signal accumulation pulse from the control circuit 60. This controls the start and stop of the accumulation of the signal charges from the first floating diffusion layer 204 to each of the fourth floating diffusion layers 207. The electronic shutter in this embodiment has a mechanism for such exposure control.

The signal charge accumulated in the first floating diffusion layer 204 to the fourth floating diffusion layer 207 is read out by turning on the gate of the row selection transistor 308 by the row selection circuit 302. At this time, the current flowing from the source follower power source 305 to the source follower transistor 309 and the source follower load 306 is amplified according to the signal potentials of the first floating diffusion layer 204 to the fourth floating diffusion layer 207. The analog signal based on this current read from the vertical signal line 304 is converted into digital signal data by the analog-digital (AD) conversion circuit 307 connected for each column. This digital signal data is read out for each column by the column selection circuit 303 and output from the photodetector 30. The row selection circuit 302 and the column selection circuit 303, after reading one row, read the next row, and similarly, read the information of the signal charges of the floating diffusion layers of all the rows. The control circuit 60 resets all floating diffusion layers by turning on the gate of the reset transistor 310 after reading all the signal charges. This completes the imaging of one frame. Similarly, the high-speed imaging of the frames is repeated to complete the imaging of a series of frames by the photodetector 30.

In the present embodiment, an example of the CMOS type photodetector 30 has been described, but the photodetector 30 may be another type of image sensor. The photodetector 30 may be, for example, a CCD type, a single photon counting type element, or an amplification type image sensor such as an EMCCD or ICCD.

FIG. 1F is a diagram showing an example of an operation within one frame in the present embodiment. As shown in FIG. 1F, the emission of the first optical pulse and the emission of the second optical pulse may be alternately switched a plurality of times within one frame. By doing so, the time difference between the acquisition timings of the detected images due to the two types of wavelengths can be reduced, and even a user who is moving can take images with the first light pulse and the second light pulse almost at the same time.

In the present embodiment, the photodetector 30 can detect the surface reflection component I1 and/or the internal scattering component I2 of the light pulse. The first biometric information of the user 500 can be acquired from the temporal or spatial change of the surface reflection component I1. The first biometric information may be, for example, the pulse of the user 500. On the other hand, the brain activity information that is the second biometric information of the user 500 can be acquired from the temporal or spatial change of the internal scattering component I2.

The first biometric information may be acquired by a method different from the method of detecting the surface reflection component I1 or may not be acquired in the first place. For example, another type of detector different from the photodetector 30 may be used to acquire the first biometric information. In that case, the photodetector 30 detects only the internal scattering component I2. Other types of detectors may be radar or thermography, for example. The first biometric information may be, for example, at least one selected from the group consisting of the pulse rate, sweating, respiration, and body temperature of the user 500. The first biometric information is biometric information other than the brain activity information obtained by detecting the internal scattered component I2 of the light pulse applied to the head of the user 500. Here, “other than brain activity information” does not mean that the first biological information does not include any information due to brain activity. The first biometric information includes biometric information resulting from a bioactivity different from the brain activity. The first biometric information may be, for example, biometric information due to autonomous or reflexive bioactivity.

[1-4. Control circuit 60 and signal processing circuit 70]
The control circuit 60 adjusts the time difference between the emission timing of the light pulse of the light source 20 and the shutter timing of the photodetector 30. In this specification, the time difference may be referred to as “phase difference”. The “emission timing” of the light source 20 is the timing at which the light pulse emitted from the light source 20 starts rising. "Shutter timing" is the timing at which exposure is started. The control circuit 60 may change the emission timing to adjust the phase difference, or may change the shutter timing to adjust the phase difference.

The control circuit 60 may be configured to remove the offset component from the signal detected by each pixel of the photodetector 30. The offset component is a signal component due to ambient light such as sunlight or fluorescent light, or ambient light. An offset component due to ambient light or ambient light is estimated by detecting a signal with the photodetector 30 in a state where the light source 20 is turned off and no light is emitted from the light source 20.

The control circuit 60 may be, for example, a combination of a processor and a memory, or an integrated circuit such as a microcontroller including the processor and the memory. The control circuit 60 adjusts the emission timing and the shutter timing, for example, by the processor executing a program recorded in the memory, for example.

The signal processing circuit 70 is a circuit that processes the image signal output from the photodetector 30. The signal processing circuit 70 performs arithmetic processing such as image processing. The signal processing circuit 70 includes, for example, a digital signal processor (DSP), a programmable logic device (PLD) such as a field programmable gate array (FPGA), a central processing unit (CPU) or an image processing arithmetic processor (GPU), and a computer program. Can be realized in combination with. The control circuit 60 and the signal processing circuit 70 may be one integrated circuit or may be separate and independent circuits. The signal processing circuit 70 may be a component of an external device such as a server provided in a remote place. In this case, an external device such as a server mutually transmits/receives data to/from the light source 20, the photodetector 30, and the control circuit 60 by wireless communication or wired communication.

The signal processing circuit 70 according to the present embodiment can generate moving image data indicating temporal changes in blood flow on the skin surface and cerebral blood flow based on the signal output from the photodetector 30. The signal processing circuit 70 is not limited to such moving image data, and may generate other information. For example, biological information such as blood flow in the brain, blood pressure, blood oxygen saturation, or heart rate may be generated by synchronizing with other devices. The signal processing circuit 70 may estimate an offset component due to ambient light and remove the offset component.

It is known that there is a close relationship between changes in cerebral blood flow or blood components such as hemoglobin and human neural activity. For example, cerebral blood flow or components in blood change due to changes in nerve cell activity in response to changes in human emotions. Therefore, if biological information such as changes in cerebral blood flow or changes in blood components can be measured, the psychological state of the user 500 can be estimated. The psychological state of the user 500 means, for example, mood, emotion, health, or temperature sensation. Mood may include moods such as pleasant or unpleasant. Emotions may include, for example, feelings of security, anxiety, sadness, or resentment. The health condition can include, for example, a condition such as healthy or tired. The temperature sensation may include, for example, a sensation of being hot, cold, or sultry. Derivatives of these may include indexes indicating the degree of brain activity, such as skill level, proficiency level, and concentration level, in the psychological state. The signal processing circuit 70 can estimate the psychological state of the user 500, such as the degree of concentration, based on, for example, a change in cerebral blood flow. In particular, the cerebral blood flow in the forehead corresponding to the prefrontal cortex region of the brain is considered to reflect the state of concentration. Mindfulness is closely related to the state of concentration. By measuring the cerebral blood flow in the forehead or its components, the degree of concentration or mindfulness of the user 500 can be estimated.

FIG. 1G is a flowchart showing an outline of the operation regarding the light source 20 and the photodetector 30 by the control circuit 60. The control circuit 60 generally performs the operations shown in FIG. 1G. Here, the operation when only the internal scattered component I2 is detected will be described.

In step S101, the control circuit 60 first causes the light source 20 to emit an optical pulse for a predetermined time. At this time, the electronic shutter of the photodetector 30 is in a state where exposure is stopped. The control circuit 60 causes the electronic shutter to stop the exposure until the period when a part of the light pulse is reflected by the surface of the user 500 and reaches the photodetector 30 is completed. Next, in step S102, the control circuit 60 causes the electronic shutter to start exposure at the timing when the other part of the light pulse is scattered inside the user 500 and reaches the photodetector 30. After the elapse of the predetermined time, in step S103, the control circuit 60 causes the electronic shutter to stop the exposure. Succeedingly, in a step S104, the control circuit 60 determines whether or not the number of times of executing the signal accumulation reaches a predetermined number. When the determination in step S104 is No, steps S101 to S103 are repeated until Yes is determined. When the determination in step S104 is Yes, in step S105, the control circuit 60 causes the photodetector 30 to generate and output a signal indicating an image based on the signal charges accumulated in each floating diffusion layer.

By the above operation, the components of the light scattered inside the measurement target can be detected with high sensitivity. In addition, the light emission and the exposure are performed a plurality of times, and they are performed as needed.

[1-5. Other]
The biometric device 100 may include an imaging optical system that forms a two-dimensional image of the user 500 on the light receiving surface of the photodetector 30. The optical axis of the imaging optical system is substantially orthogonal to the light receiving surface of the photodetector 30. The imaging optical system may include a zoom lens. When the position of the zoom lens changes, the magnification of the two-dimensional image of the user 500 changes, and the resolution of the two-dimensional image on the photodetector 30 changes. Therefore, even if the distance to the user 500 is long, it is possible to enlarge a desired measurement region and observe it in detail.

The biometric device 100 may include, between the user 500 and the photodetector 30, a bandpass filter that passes only light in the wavelength band emitted from the light source 20 or light in the vicinity thereof. As a result, the influence of disturbance components such as ambient light can be reduced. The bandpass filter can be constituted by, for example, a multilayer filter or an absorption filter. In consideration of the temperature shift of the light source 20 and the band shift caused by oblique incidence on the filter, the bandpass filter may have a bandwidth of about 20 to 100 nm.

The biometric device 100 may include polarizing plates between the light source 20 and the user 500 and between the photodetector 30 and the user 500, respectively. In this case, the polarization directions of the polarizing plate arranged on the light source 20 side and the polarizing plate arranged on the photodetector 30 side may have a crossed Nicol relationship. This prevents the specular reflection component of the surface reflection component I1 of the user 500, that is, the component having the same incident angle and reflection angle from reaching the photodetector 30. That is, the amount of light that the surface reflection component I1 reaches the photodetector 30 can be reduced.

[2. Operation of light source and photodetector]
The biometric device 100 in the present embodiment can detect the surface reflection component I1 and the internal scattering component I2 separately. When the target part of the user 500 is the forehead, the signal intensity due to the internal scattered component I2 to be detected becomes very small. As described above, in addition to the irradiation of a very small amount of light that meets the laser safety standards, the scattering and absorption of light by the scalp, cerebrospinal fluid, skull, gray matter, white matter and blood flow are large. .. Furthermore, the change in the signal intensity due to the change in the blood flow rate or the component in the blood flow during brain activity is very small, which corresponds to a magnitude of several tens of minutes. Therefore, when detecting the internal scattering component I2, the surface reflection component I1, which is thousands to tens of thousands times the signal component to be detected, is removed as much as possible during imaging.

Hereinafter, an example of the operation of the light source 20 and the photodetector 30 in the biometric device 100 that detects the internal scattered component I2 will be described.

As shown in FIG. 1A, when the light source 20 irradiates the target portion of the user 500 with a light pulse, a surface reflection component I1 and an internal scattering component I2 are generated. Part of each of the surface reflection component I1 and the internal scattering component I2 reaches the photodetector 30. The internal scattered component I2 is emitted from the light source 20 and passes through the inside of the target portion of the user 500 before reaching the photodetector 30. Therefore, the optical path length of the internal scattering component I2 is longer than the optical path length of the surface reflection component I1. Therefore, the time for the internal scattering component I2 to reach the photodetector 30 lags behind the time for the surface reflection component I1 to reach the photodetector 30 on average.

FIG. 2 is a diagram schematically showing a waveform of the light intensity of the reflected light pulse returned from the head of the user 500 when the light pulse of the rectangular wave is emitted from the light source 20. The horizontal axis represents time (t) in each of the waveforms (a) to (d) in FIG. The vertical axis represents intensity in the waveforms (a) to (c) of FIG. 2, and the waveform (d) represents the state of OPEN or CLOSE of the electronic shutter. The waveform (a) in FIG. 2 shows the surface reflection component I1. The waveform (b) in FIG. 2 shows the internal scattering component I2. The waveform (c) in FIG. 2 shows the total component of the surface reflection component I1 and the internal scattering component I2. As shown in the waveform (a) of FIG. 2, the waveform of the surface reflection component I1 maintains a substantially rectangular shape. On the other hand, the internal scattering component I2 is the sum of lights having various optical path lengths. Therefore, as shown in the waveform (b) of FIG. 2, the internal scattered component I2 has a characteristic that the rear end of the optical pulse is tailed. In other words, the falling period of the internal scattering component I2 is longer than the falling period of the surface reflection component I1. As shown in the waveform (d) of FIG. 2, in order to extract the optical signal shown in the waveform (c) of FIG. 2 while increasing the ratio of the internal scattering component I2, after the time when the rear end of the surface reflection component I1 arrives, The exposure of the electronic shutter is started. In other words, the exposure is started when the waveform of the surface reflection component I1 falls or after that. The shutter timing is adjusted by the control circuit 60.

When the measurement target is not planar, the timing of light arrival differs depending on the pixel of the photodetector 30. In this case, the shutter timing shown in the waveform (d) of FIG. 2 may be individually determined for each pixel. For example, the direction perpendicular to the light receiving surface of the photodetector 30 is the z direction. The control circuit 60 may previously acquire data indicating the two-dimensional distribution of the z coordinate on the surface of the target portion, and change the shutter timing for each pixel based on this data. Accordingly, even when the surface of the target portion is curved, it is possible to determine the optimum shutter timing at each position.

In the example shown in the waveform (a) of FIG. 2, the rear end of the surface reflection component I1 falls vertically. In other words, the time from the start of the fall of the surface reflection component I1 to the end thereof is zero. However, in reality, the rear end of the surface reflection component I1 may not fall vertically. For example, when the falling edge of the waveform of the light pulse emitted from the light source 20 is not completely vertical, when the surface of the target portion has fine irregularities, or when scattering occurs in the epidermis, the surface reflection component I1 The rear edge does not fall vertically. Further, since the user 500 is an opaque object, the amount of light of the surface reflection component I1 is much larger than the amount of light of the internal scattering component I2. Therefore, even if the rear end of the surface reflection component I1 slightly protrudes from the time of the vertical fall, the internal scattering component I2 may be buried. In addition, a time delay may occur due to the movement of electrons during the reading period of the electronic shutter. From the above, it may not be possible to realize ideal binary reading as shown in the waveform (d) of FIG. In that case, the control circuit 60 may slightly delay the shutter start timing of the electronic shutter from immediately after the fall of the surface reflection component I1. For example, it may be delayed by about 0.5 ns to 5 ns. Instead of adjusting the shutter timing of the electronic shutter, the control circuit 60 may adjust the emission timing of the light source 20. In other words, the control circuit 60 may adjust the time difference between the shutter timing of the electronic shutter and the emission timing of the light source 20. When measuring changes in the blood flow volume or blood flow component in the brain without contact, if the shutter timing is delayed too much, the originally small internal scattering component I2 is further reduced. Therefore, the shutter timing may be kept near the rear end of the surface reflection component I1. As described above, the time delay due to scattering inside the forehead is about 4 ns. In this case, the maximum delay amount of shutter timing may be about 4 ns.

The signals may be accumulated by exposing each of the plurality of light pulses emitted from the light source 20 at shutter timings with the same time difference. As a result, the detected light amount of the internal scattering component I2 is amplified.

Instead of, or in addition to, disposing a bandpass filter between the user 500 and the photodetector 30, the offset component may be estimated by photographing in the same exposure period with the light source 20 not emitting light. Good. The estimated offset component is subtracted by the difference from the signal detected by each pixel of the photodetector 30. Thereby, the dark current component generated on the photodetector 30 can be removed.

The internal scattered component I2 includes internal characteristic information of the user 500, for example, cerebral blood flow information. The amount of light absorbed by the blood changes according to the temporal change in the cerebral blood flow of the user 500. As a result, the amount of light detected by the photodetector 30 also increases or decreases correspondingly. Therefore, by monitoring the internal scattered component I2, the brain activity state can be estimated from the change in the cerebral blood flow of the user 500.

Next, an example of a method for detecting the surface reflection component I1 will be described. The surface reflection component I1 includes surface characteristic information of the user 500, for example, blood flow information of the face and scalp. The information on the surface reflection component I1 does not necessarily have to be acquired, and may be acquired as needed.

FIG. 3A is a diagram schematically showing an example of a timing chart when detecting the surface reflection component I1. To detect the surface reflection component I1, for example, as shown in FIG. 3A, the shutter is opened before the light pulse reaches the photodetector 30, and the shutter is opened before the rear end of the light pulse arrives. It may be CLOSE. By controlling the shutter in this manner, it is possible to reduce mixing of the internal scattering component I2. As a result, it is possible to increase the proportion of light that has passed near the surface of the user 500. The timing of the shutter CLOSE may be set immediately after the light reaches the photodetector 30. As a result, it becomes possible to detect a signal in which the ratio of the surface reflection component I1 having a relatively short optical path length is increased. By acquiring the signal of the surface reflection component I1, it becomes possible to detect the pulse of the user 500 or the oxygenation degree of the facial blood flow. As another acquisition method of the surface reflection component I1, the photodetector 30 may detect the entire light pulse or the continuous light emitted from the light source 20.

FIG. 3B is a diagram schematically showing an example of a timing chart when detecting the internal scattering component I2. By setting the shutter to OPEN during the period when the trailing end of the pulse reaches the photodetector 30, the signal of the internal scattered component I2 can be acquired.

3B is summarized, the control circuit 60 performs the following operations. The control circuit 60 causes the light source 20 to emit one or more light pulses. The control circuit 60 causes the photodetector 30 to detect a component included in the falling period of each optical pulse from the optical pulses returned from the target section of the user 500. The component includes the internal scattering component I2. The control circuit 60 causes the photodetector 30 to output a signal obtained by the detection. The signal processing circuit 70 generates a signal indicating the brain activity state of the user 500 based on the signal.

The surface reflection component I1 may be detected by a device other than the biometric device 100 that acquires the internal scattering component I2. An apparatus other than the apparatus for acquiring the internal scattered component I2, or another device such as a pulse wave meter or a Doppler blood flow meter may be used. In this case, the separate device is used in consideration of timing synchronization between devices, light interference, and alignment of detection points. If time-division imaging is performed by the same camera or the same sensor as in the present embodiment, temporal and spatial deviations are unlikely to occur. When acquiring the signals of both the surface reflection component I1 and the internal scattering component I2 by the same sensor, the components to be acquired may be switched for each frame as shown in FIGS. 3A and 3B. Alternatively, as described with reference to FIGS. 1D to 1F, the components to be acquired at high speed within one frame may be switched alternately. In that case, the detection time difference between the surface reflection component I1 and the internal scattering component I2 can be reduced.

Furthermore, the respective signals of the surface reflection component I1 and the internal scattering component I2 may be acquired using light of two wavelengths. For example, light pulses with two wavelengths of 750 nm and 850 nm may be used. Thereby, the change in the concentration of oxygenated hemoglobin and the change in the concentration of deoxygenated hemoglobin can be calculated from the change in the detected light amount at each wavelength. When acquiring the surface reflection component I1 and the internal scattering component I2 at two wavelengths, for example, as described with reference to FIGS. 1D to 1F, a method of rapidly switching four types of charge accumulation within one frame is used. Can be done. With such a method, it is possible to reduce the time shift of the detection signal.

Here, a more specific example of a method of acquiring cerebral blood flow information will be described.

Blood plays a major role in receiving oxygen from the lungs and transporting it to tissues, and receiving carbon dioxide from tissues and circulating it in the lungs. About 100 g of hemoglobin is present in 100 ml of blood. Hemoglobin bound to oxygen is oxygenated hemoglobin, and hemoglobin not bound to oxygen is deoxygenated hemoglobin. As mentioned above, the light absorption characteristics of oxygenated hemoglobin and deoxygenated hemoglobin are different. Oxygenated hemoglobin absorbs near-infrared rays having a wavelength of more than about 805 nm relatively well. In contrast, deoxygenated hemoglobin relatively well absorbs near infrared or red light with wavelengths shorter than 805 nm. Regarding the near-infrared ray having a wavelength of 805 nm, the absorptances of both are similar. Therefore, a first wavelength longer than 600 nm and shorter than 805 nm and a second wavelength longer than 805 nm and shorter than 1000 nm may be used. For example, the above-mentioned light having two wavelengths of 750 nm and 850 nm can be used. Based on the detected light amounts of these lights, it is possible to detect the time change of the respective concentrations of oxygenated hemoglobin and deoxygenated hemoglobin in blood. Furthermore, the oxygen saturation level of hemoglobin can be obtained. The oxygen saturation is a value indicating how much of hemoglobin in blood is associated with oxygen. The oxygen saturation is defined by the following mathematical formula, where C(Hb) is the concentration of deoxygenated hemoglobin and C(HbO ₂ ) is the concentration of oxygenated hemoglobin.
Oxygen saturation=C(HbO ₂ )/[C(HbO ₂ )+C(Hb)]×100(%)

In addition to blood, the living body contains components that absorb red light and near infrared light. However, the temporal fluctuation of the light absorption rate is mainly due to the hemoglobin in the arterial blood. Therefore, the blood oxygen saturation can be measured with high accuracy based on the change in the absorption rate.

Light that reaches the brain also passes through the scalp and face surface. Therefore, changes in blood flow in the scalp and face are also detected in a superimposed manner. In order to remove or reduce the influence, the signal processing circuit 70 may perform a process of subtracting the surface reflection component I1 from the internal scattering component I2 detected by the photodetector 30. Thereby, pure cerebral blood flow information excluding blood flow information of the scalp and face can be acquired. As the subtraction method, for example, a method of subtracting a value obtained by multiplying the signal of the surface reflection component I1 by a coefficient of 1 or more determined in consideration of the optical path length difference from the signal of the internal scattering component I2 can be used. This coefficient can be calculated by simulation or experiment, for example, based on the average value of the optical constants of the head of a general person. Such subtraction processing can be easily performed when measurement is performed using light of the same wavelength by the same camera or sensor. This is because it is easy to reduce the temporal and spatial shifts, and it is easy to match the characteristics of the scalp blood flow component included in the internal scattering component I2 and the surface reflection component I1.

There is a skull between the brain and the scalp. Therefore, the two-dimensional distribution of cerebral blood flow and the two-dimensional distribution of blood flow of the scalp and face are independent. Therefore, based on the signal detected by the photodetector 30, the two-dimensional distribution of the internal scattering component I2 and the two-dimensional distribution of the surface reflection component I1 are separated using a statistical method such as independent component analysis or principal component analysis. You may.

[3. Example of detection of changes in cerebral blood flow]
Next, an example of a method for detecting a change in the cerebral blood flow of the user 500 will be described.

FIG. 4A is a diagram schematically showing an example of temporal changes in cerebral blood flow. As shown in FIG. 4A, the target portion 500t of the user 500 is illuminated with the light from the light source 20, and the returned light is detected. In this case, the surface reflection component I1 is much larger than the internal scattering component I2. However, only the internal scattering component I2 can be extracted by the shutter adjustment described above. The graph shown in FIG. 4A shows an example of changes over time in the concentrations of oxygenated hemoglobin (HbO ₂ ) and deoxygenated hemoglobin (Hb) in cerebral blood. The internal scattering component I2 in this example is acquired using light of two wavelengths. The density shown in FIG. 4A indicates the amount of change based on the amount in normal times. This change amount is calculated by the signal processing circuit 70 based on the light intensity signal. Cerebral blood flow changes depending on the brain activity state such as normal state, concentration state, or relaxed state. For example, there is a difference in brain activity or a difference in absorption coefficient or scattering coefficient depending on the location of the target portion. Therefore, the temporal change in cerebral blood flow is measured at the same position in the target portion 500t of the user 500. When observing changes in brain activity over time, it is possible to estimate the state of the subject from the relative changes in cerebral blood flow over time, without knowing the absolute amount of cerebral blood flow.

FIG. 4B is a diagram schematically showing an example in which measurement is simultaneously performed at a plurality of locations in the target section 500t of the user 500. In this example, since the two-dimensional irradiation or the two-dimensional imaging is performed, it is possible to acquire the two-dimensional distribution of the cerebral blood flow. In this case, the irradiation pattern of the light source 20 may be, for example, a uniform distribution of uniform intensity, a dot-shaped distribution, or a donut-shaped distribution. If the irradiation has a uniform distribution with a uniform intensity, it is unnecessary or easy to adjust the irradiation position on the target portion 500t. If the irradiation has a uniform distribution, the light enters the target portion of the user 500 from a wide range. Therefore, the signal detected by the photodetector 30 can be enhanced. Further, it can be measured at any position within the irradiation area. In the case of partial irradiation such as a dot-shaped distribution or a donut-shaped distribution, the influence of the surface reflection component I1 can be reduced only by removing the target portion 500t from the irradiation area.

FIG. 5A is a diagram schematically showing an example of the light irradiation area 22. In the non-contact measurement of cerebral blood flow, the detected light amount attenuates in inverse proportion to the square of the distance from the device to the target portion. Therefore, the signal of each pixel detected by the photodetector 30 may be enhanced by integrating the signals of a plurality of neighboring pixels. By doing so, the number of integrated pulses can be reduced while maintaining the SN ratio. Thereby, the frame rate can be improved.

FIG. 5B is a diagram schematically showing a change in signal when the target portion 500t of the user 500 shifts in the horizontal direction. As described above, the change in brain activity can be read by detecting the difference between the cerebral blood flow when the brain activity changes from the normal state and the cerebral blood flow in the normal state. When the photodetector 30 including a plurality of photoelectric conversion elements arranged two-dimensionally is used, a two-dimensional brain activity distribution can be acquired as shown in the upper diagram of FIG. 5B. In this case, it is possible to detect a region where brain activity is active from the relative intensity distribution within the two-dimensional distribution without acquiring the signal in the normal state in advance. Since the measurement is performed in a non-contact manner in the present embodiment, the position of the target portion 500t may change during the measurement as shown in the lower diagram of FIG. 5B. This may occur, for example, when the user 500 has moved slightly due to breathing. Generally, the two-dimensional distribution of cerebral blood flow does not change rapidly within a very short time. Therefore, for example, the positional deviation of the target portion 500t can be corrected by pattern matching between the detected two-dimensional distribution frames. Alternatively, if it is a periodical movement such as breathing, only its frequency component may be extracted and corrected or removed. The target portion 500t does not have to be a single area, but may be a plurality of areas. The plurality of regions may be, for example, one on the left and one on the right, or may have a 2×6 matrix dot distribution.

[4. Judgment of Mindfulness Proficiency]
Next, an example of a method of determining the proficiency level of the user 500 for becoming a mindfulness state using the biometric device 100 will be described.

As described above, the brain activity of the user 500 can be estimated based on the internal scattered component I2 of the reflected light. This property can be used to estimate how proficient the user 500 is in being in a mindfulness state.

In the present embodiment, a voice guide for mindfulness meditation is used as an example of a stimulus that guides the user 500 to a mindfulness state. The user 500 performs meditation according to the voice guide output from the speaker of the stimulator 10. While the voice guide is being reproduced, the light source 20 repeatedly emits a light pulse toward the head of the user 500 at regular time intervals, and the photodetector 30 causes the signal of the internal scattered component of the reflected light pulse to be pixelated. Accumulate for each. In the present embodiment, the same operation is performed in each period immediately before and immediately after the voice guide is reproduced.

FIG. 6 is a diagram showing, on a time axis, the flow of one determination operation of the proficiency level of mindfulness in the present embodiment. One determination operation in this embodiment is composed of three periods. The first period is a period of a predetermined time length immediately before the voice guide is reproduced. The first period is referred to as a "rest A period". The second period is a period in which the voice guide is reproduced. The second period is called a "task period". The “task” refers to a task such as mindfulness meditation presented to the user 500. The third period is a period of a predetermined time length immediately after the reproduction of the voice guide is completed. The third period is referred to as a "rest B period". The biometric device 100 repeatedly generates the brain activity signal of the user 500 over the first to third periods. During the task period, the user 500 performs meditation according to the voice guide. In the example shown in FIG. 6, the length of each of the rest A period and the rest B period is 120 seconds, and the length of the task period period is 392 seconds. The length of each period is not limited to this example and can be adjusted as appropriate. The lengths of the rest A period and the rest B period may be different.

FIG. 7 is a flowchart showing the flow of the determination operation by the biometric device 100. When the user 500 performs a measurement start operation, the biometric device 100 starts measuring the brain activity signal of the user 500 (step S701). At this stage, the audio guide for mindfulness has not yet been played. The brain activity signal is repeatedly acquired for a predetermined time (for example, 120 seconds) while the user 500 is not doing anything. This is performed in order to acquire the normal brain activity signal of the user 500. This period is the above-mentioned rest A period. As described above, the measurement is performed by repeating the operation in which the light source 20 emits a light pulse toward the head of the user 500 and the photodetector 30 detects the rear end portion of the reflected light pulse. As a result, the internal scattering component I2 of the reflected light pulse is repeatedly detected. The photodetector 30 outputs a signal corresponding to the accumulated amount of reflected light pulses for each pixel. The signal processing circuit 70 repeatedly generates a brain activity signal based on the signal output from the photodetector 30. The brain activity signal may be generated, for example, for each pixel or for each pixel group.

After a lapse of a predetermined time from the start of measurement, the control circuit 60 causes the stimulator 10 to start the reproduction of the voice guide (step S702). Thereafter, the user 500 practices the mindfulness meditation, that is, the task according to the voice guide. During the task, the measurement of the brain activity signal is continued as it is. The signal processing circuit 70 repeatedly generates a brain activity signal based on the signal output from the photodetector 30 while the user 500 is performing a task.

When a certain time (392 seconds in the example of FIG. 6) has elapsed from the start of the voice guide, the voice guide ends (step S703). Even after the voice guide is finished, the measurement is continued as it is for a certain period of time (for example, 120 seconds). As a result, the brain activity signal is repeatedly acquired even after the task is completed. This period is the rest B period described above.

When the above-mentioned certain time has elapsed from the end of the voice guide, the signal processing circuit 70 ends the measurement of the brain activity signal (step S704). Next, the signal processing circuit 70 analyzes a plurality of brain activity signals acquired during the rest A period, the task period, and the rest B period, and determines the proficiency level of the user 500. First, the signal processing circuit 70 creates a histogram of the difference between the acquired brain activity signal value and the average value thereof for each of the rest A, the task, and the rest B (step S705). Specifically, the signal processing circuit 70 performs the following processing.
(1) Calculate the average value of the brain activity signal values acquired during the rest A, task, and rest B periods.
(2) Calculate the difference between each value of the brain activity signal and the average value.
(3) Obtain the frequency for each calculated difference class.

This will create three histogram data. The signal processing circuit 70 tests the significant difference between the three histograms using the existing test method (step S706). If there is a significant difference, the signal processing circuit 70 determines the user 500 as an inexpert (step S708). On the contrary, when there is no significant difference, the signal processing circuit 70 determines that the user 500 is an expert (step S709). The determination result may be displayed on the display of the stimulation device 10, for example.

8A and 8B show examples of three histograms created in step S705. FIG. 8A shows an example of the histogram obtained for the unskilled person. FIG. 8B shows an example of the histogram obtained for the expert. The notations “before”, “mindfulness”, and “after” in FIGS. 8A and 8B correspond to rest A, task, and rest B, respectively. The illustrated histogram was created using the averaged brain activity signal every 5 seconds during each period.

As shown in FIG. 8A, in the non-skilled person, the degree of variation in the brain activity signal is different between the rest period and the task period before and after the mindfulness task. It is considered that this is because the brain activity state of the unskilled person changes greatly by performing the task. On the other hand, as shown in FIG. 8B, the skilled person does not see a significant difference in the degree of variation in brain activity signals between the rest period before and after the task and the task period. It is considered that this is because the skilled person maintains the mindfulness state and the brain activity state is stable regardless of the presence or absence of the task.

From the above, the expert and the non-expert are discriminated based on whether there is a significant difference in the degree of variation in brain activity signals between the rest A period or the rest B period and the task period. be able to. In the present embodiment, the brain activity signal is acquired in both the rest A period and the rest B period, but either the rest A period or the rest B period may be omitted. The significant difference between the rest period and the task period histogram can be tested by using a known test method such as t-test.

The determination method shown in FIG. 7 is merely an example, and there are various methods for determining the proficiency level from the acquired brain activity signal. For example, the proficiency level may be determined by applying a pre-trained model trained by a machine learning algorithm to a plurality of acquired brain activity signals. In the present embodiment, a two-level evaluation of whether or not the user is proficient in mindfulness is performed, but an evaluation of three or more levels may be performed depending on the proficiency level.

Further, in the present embodiment, the user practices the task and determines the proficiency level according to the voice that guides the user to the mindfulness state, but the present disclosure is not limited to such a determination method. For example, the voice may be a stimulus that interferes with the mindfulness state. Alternatively, the proficiency may be determined by having the user perform a task in the absence of stimulation such as voice.

[5. System that collects data from multiple mobile devices]
Next, an example of a system for collecting and utilizing brain activity signals and position data from mobile devices of a plurality of users will be described.

FIG. 9 is a conceptual diagram of a system for collecting and utilizing brain activity signals and position data from mobile devices 400 of a plurality of users. This system includes a plurality of mobile devices 400 and a server computer 200. The server computer 200 is connected to a plurality of mobile devices 400 via a network such as the Internet. Although three mobile devices 400 are illustrated in FIG. 9, a larger number of mobile devices 400 may be connected to the server computer 200. The server computer 200 may be, for example, a cloud server managed by a business.

FIG. 10 is a diagram showing a schematic configuration of the server computer 200 and the plurality of mobile devices 400.

Each mobile device 400 may be a mobile information device such as a smartphone, a tablet computer, or a laptop PC. Each mobile device 400 includes a stimulator 410, a light source 420, a photodetector 430, a memory 450, a processor 460, and a communication circuit 480. The stimulator 410, the light source 420, and the photodetector 430 have the same functions as the stimulator 10, the light source 20, and the photodetector 30 shown in FIG. 1A, respectively. The processor 460 has the same functions as the control circuit 60 and the signal processing circuit 70 shown in FIG. 1A. The memory 450 includes RAM and ROM, and stores various programs executed by the processor 460 and generated data. The stimulator 410 can be realized by a combination of a display and a speaker included in the mobile device 400, for example.

The light source 420 may be built in the mobile device 400 or may be a device externally attached to the mobile device 400. The photodetector 430 may be an image sensor built in the mobile device 400 or a device externally attached to the mobile device 400.

The communication circuit 480 performs wireless communication with the server computer 200. The communication circuit 480 can receive position signals from GPS satellites. Based on the position signals obtained from the GPS satellites, the processor 460 can calculate the latitude and longitude of the current location.

Each mobile device 400 can download and install the aforementioned application for mindfulness from the server computer 200. The program of the application is recorded in the memory 450 and executed by the processor 460.

The user of each mobile device 400 can start the application and perform mindfulness meditation according to the guidance reproduced. Meanwhile, processor 460 controls light source 420 and photodetector 430 to measure brain activity in the manner described above. The measurement can be repeated, for example, at regular time intervals. This time interval can be set arbitrarily in the range of, for example, several microseconds to several seconds. The processor 460 generates a brain activity signal in a predetermined region of the target part of the user based on the signal repeatedly output from the photodetector 430. The brain activity signal can be, for example, a signal that depends on the concentration of deoxygenated hemoglobin in cerebral blood. The processor 460 determines the user's level of proficiency with respect to becoming mindfulness, for example, by executing the process shown in FIG. 7. The mobile device 400 transmits data that associates the determination result of the skill level and the position information to the server computer 200 via the network. This data may include, for example, the ID of the mobile device 400 or other information such as a user's comment. Such data can be collected as big data from the terminals of many users.

The server computer 200 includes a signal processing circuit 210 and a recording medium 220. The recording medium 220 stores the program executed by the signal processing circuit 210 and the generated data. The recording medium 220 may include a memory and a hard disk, for example. The signal processing circuit 210 collects the above-mentioned data from a plurality of mobile devices 400, and outputs information indicating the determination result of the proficiency level of each user included in the data and map information recorded in advance in the recording medium 220. Generate the associated map data. The signal processing circuit 210 transmits the generated map data to each mobile device 400.

FIG. 11 is a diagram showing an example of an image displayed on the display of the mobile device 400 based on the map data. The map data in this example represents a heat map in which a number of mindfulness masters are represented by a plurality of different colors. For example, places with a high level of proficiency are displayed in red (R), places with a moderate number of proficiency are displayed in green (G), and places with a low level of proficiency are displayed in blue (B). Places with no sign are displayed without coloring. With such map data, the user can visually know where the many mindfulness masters are gathering. The visualization method is not limited to the heat map, and may be a method of displaying information such as a mark or a character indicating that there are many experts on the map, for example. As described above, a place where many skilled people gather is likely to be a place where a mindfulness state is likely to occur. By using the application that visualizes the degree of concentration of the expert as in the present embodiment, the user can easily know the place suitable for mindfulness. With reference to the displayed map information, the user can actually go to a place where many skilled people gather, and can provide the server computer 200 with the impressions, comments, recommendations, etc. via the portable device 400. Utilizing the information collected from multiple users for marketing may lead to the creation of new business opportunities.

The biometric device is not limited to a portable computer such as a smartphone, but may be mounted on another device, for example, a head mount device. FIG. 12 is a diagram schematically showing an example of the head mount device 300. The head mount device 300 includes a display and a speaker that function as a stimulator, a light source, and a photodetector. Even when such a head mount device 300 is used, the same effect as described above can be obtained.

The technology of the present disclosure can acquire the brain activity information of the user without contact. The technology of the present disclosure can be used for various devices such as a camera, a measuring device, a smartphone, a tablet computer, and a head mount device.

10 Stimulator 20 Light source 30 Photodetector 60 Control circuit 62 Light source control unit 63 Detector control unit 70 Signal processing circuit 100 Biometric device 200 Server computer 201 Pixel 202 Drain 203 Photodiode 204 First floating diffusion layer 205 Second Floating diffusion layer 206 Third floating diffusion layer 207 Fourth floating diffusion layer 210 Signal processing circuit 220 Recording medium 300 Head mount device 302 Row selection circuit 303 Column selection circuit 304 Vertical signal line 305 Source follower power supply 306 Source follower load 307 Analog -Digital conversion circuit 308 Row selection transistor 309 Source follower transistor 310 Reset transistor 400 Portable device 410 Stimulator 420 Light source 430 Photodetector 450 Memory 460 Processor 480 Communication circuit

Claims

The reflected light from the head generated by irradiating the user's head with irradiation light is generated by detecting the component of the light propagating in the brain of the user, and indicates the activity state of the brain. Acquiring at least one brain activity signal;
Generating and outputting a signal indicative of the user's proficiency in becoming a mindfulness state based on the at least one brain activity signal;
A biometric method including:
Obtaining the at least one brain activity signal is performed with a stimulus including at least one selected from the group consisting of audio and video being given to the user.
The biometric method according to claim 1.
The at least one selected from the group consisting of the audio and the video guides the user to the mindfulness state,
The biometric method according to claim 2.
The irradiation light is a light pulse,
The reflected light is a reflected light pulse,
The light component is detected in at least a part of a period from when the light intensity of the reflected light pulse starts decreasing to when the decrease ends.
The biometric method according to any one of claims 1 to 3.
The at least one brain activity signal is
A first period immediately before the stimulus is given to the user,
A second period in which the stimulus is being given to the user,
Of the third period immediately after the stimulus is given to the user, acquired for each of at least two periods including the second period,
The biometric method according to any one of claims 2 to 4.
The at least one brain activity signal comprises a plurality of brain activity signals,
The plurality of brain activity signals are
A first period immediately before the stimulus is given to the user,
In each of the second period in which the stimulus is given to the user and the third period immediately after the stimulus is given to the user, it is acquired by detecting the component of the light a plurality of times.
The biometric method according to any one of claims 2 to 4.
The signal indicating the proficiency level is generated based on the degree of variation of the plurality of brain activity signals,
The biometric method according to claim 6.
The at least one brain activity signal is dependent on the concentration of deoxygenated hemoglobin in the blood in the brain,
The biometric method according to any one of claims 1 to 7.
The irradiation light includes a first light pulse having a first wavelength longer than 805 nm, and a second light pulse having a second wavelength shorter than 805 nm,
The reflected light includes a first reflected light pulse generated by irradiating the head with the first light pulse and a second reflected light pulse generated by irradiating the head with the second light pulse. Including reflected light pulses,
The at least one brain activity signal is
The first reflected light pulse is generated by detecting at least a part of a component included in a period from when the light intensity of the first reflected light pulse starts to decrease to when the light intensity of the first reflected light pulse ends. The first signal, and
The second reflected light pulse is generated by detecting at least a part of a component included in a period from when the light intensity of the second reflected light pulse starts to decrease to when the decrease ends. The second signal,
Generated by the operation using
The biometric method according to any one of claims 1 to 8.
A signal indicating the proficiency level of each of the plurality of users generated from the computers of the plurality of users by the biometric method according to claim 1, and position information of each of the plurality of users. Is obtained via the network,
Generating and outputting map data in which the master level and the map of each of the plurality of users are associated with each other based on the position information;
A method for generating map data, including:
A program executed by a computer mounted on a biometric device including a stimulator, a light source, and a photodetector,
Causing the stimulator to provide a stimulus to the user;
Causing the light source to emit light toward the user's head while the stimulus is being applied to the user;
A brain activity indicating the activity state of the brain according to the intensity of the component by causing the photodetector to detect a component of light that has propagated in the brain of the user among the reflected light from the head. To output a signal,
Generating and outputting a signal indicating the proficiency level of the user with respect to becoming a mindfulness state based on the brain activity signal;
A program that causes the computer to execute.
A computer-readable recording medium that stores a program executed by a computer mounted on a biometric device that includes a stimulator, a light source, and a photodetector.
When the program is executed by the computer,
Causing the stimulator to provide a stimulus to the user;
Causing the light source to emit light toward the user's head while the stimulus is being applied to the user;
A brain activity indicating the activity state of the brain according to the intensity of the component by causing the photodetector to detect a component of light that has propagated in the brain of the user among the reflected light from the head. To output a signal,
Generating and outputting a signal indicating the proficiency level of the user with respect to becoming a mindfulness state based on the brain activity signal;
A computer-readable recording medium on which is executed.
A stimulator for giving a stimulus to a user,
A light source that emits light toward the user's head while the stimulus is being applied to the user;
Of the light reflected from the head, a photodetector that detects a component of light that has propagated in the brain of the user and outputs a brain activity signal indicating the activity state of the brain according to the intensity of the component. ,
A signal processing circuit which, based on the brain activity signal, generates and outputs a signal indicating the degree of proficiency of the user for becoming a mindfulness state;
A biometric device comprising: