JP7443044B2 - Biometric information acquisition device, terminal device, biometric information acquisition method, biometric information acquisition program, and computer-readable recording medium - Google Patents

Description

The present disclosure relates to a biological information acquisition device, a terminal device, a biological information acquisition method, a biological information acquisition program, and a computer-readable recording medium that can acquire biological information.

2. Description of the Related Art A technique is known in which, when a plurality of faces are detected in an image captured by a terminal device, one person is selected from the plurality of faces. Patent Document 1 discloses a technique for comparing the sizes of images of a plurality of faces and determining a subject based on the face with the largest size. Patent Documents 2 and 3 also disclose techniques for selecting one face from a plurality of faces in an image.

Japanese Patent Application Publication No. 2005-049854 (published on February 24, 2005) Japanese Patent Application Publication No. 2007-6033 (published on January 11, 2007) Japanese Patent Application Publication No. 2009-77266 (published on April 9, 2009)

However, the above-mentioned conventional technology has the following problems.

In recent years, technology has been developed to calculate a person's pulse wave by analyzing a moving image of the person's face captured by a camera. Applications using such technology are widely used in camera-equipped mobile terminals (smartphones, etc.). When the faces of multiple people are shown in a video image, it is theoretically possible to measure the pulse waves of the multiple people.

However, attempting to calculate the pulse waves of multiple people at the same time places a large burden on the image processing ability (calculation ability) of the mobile terminal. Therefore, in order to obtain a pulse wave accurately in real time, a method can be considered in which one person is selected from a plurality of people and the pulse wave of the selected person is obtained. Patent Documents 1 to 3 disclose techniques for selecting one face from a plurality of faces in an image. On the other hand, Patent Documents 1 to 3 do not disclose a technique for calculating a pulse wave from a face in an image.

Furthermore, in general, when a person's face is imaged in a large size, the number of pixels that can be used to calculate a pulse wave increases, and as a result, it becomes easier to obtain a pulse wave with a high signal-to-noise ratio.

However, just because the face of the imaged person is large does not necessarily mean that a highly accurate pulse wave can be obtained. There are no prior art documents that point out this point as a problem.

One aspect of the present disclosure provides a biological information acquisition device, a terminal device, and a biological information acquisition method that can acquire biological information of a measurement target from an image including an image of the measurement target with higher accuracy than conventional methods. , a biological information acquisition program, and a computer-readable recording medium.

In order to solve the above problems, a biological information acquisition device according to one aspect of the present disclosure is a biological information acquisition device that acquires biological information, and includes one or more biological information included in a first image captured by an imaging unit. a face detection unit that detects the face of the first subject that is the largest among the faces of the subjects; a face direction determination unit that determines whether the face of the first subject is facing in a first direction that is a suitable direction; and a face direction determination unit that determines that the face of the first subject is facing in the first direction; , a calculation unit that calculates biological information of the first subject from a second image different from the first image captured by the imaging unit.

In order to solve the above problems, a terminal device according to an aspect of the present disclosure is a terminal device that acquires biometric information, and includes an imaging unit that captures a first image and a second image, and a terminal device that captures a first image and a second image. a face detection unit that detects the face of the first subject that is the largest among the faces of one or more subjects included in the image; and a face of the first subject detected by the face detection unit is a face direction determination unit that determines whether the face of the first subject is facing in a first direction suitable for acquiring biometric information; and a face direction determination unit that determines that the face of the first subject is facing in the first direction. a calculation unit that calculates biometric information of the first subject from the second image.

In order to solve the above problems, a biological information acquisition method according to an aspect of the present disclosure is a biological information acquisition method for acquiring biological information, the biological information acquisition method comprising one or more biological information included in a first image captured by an imaging unit. A detection step of detecting the face of the first subject that is the largest among the faces of the subjects; a determination step of determining whether the face of the first subject is facing in the first direction; and when it is determined in the determination step that the face of the first subject is facing in the first direction, the face of the first subject is imaged by the imaging unit. , a calculation step of calculating biological information of the first subject from a second image different from the first image.

According to one aspect of the present disclosure, biological information of a measurement target can be calculated from an image including an image of the measurement target with higher accuracy than conventional methods.

FIG. 2 is a configuration diagram of main parts of a terminal device including a biometric information acquisition unit according to the present disclosure. This figure shows how the pulse wave of a person whose pulse wave is to be acquired continues to be acquired when the person faces the second direction. It shows how acquisition of a pulse wave of a person is interrupted when the person whose pulse wave is to be acquired turns to the second direction. An example of an image captured by the imaging unit is shown. An example of an image in which a face has been detected by the face detection unit is shown. An example of detection results output by the face detection unit is shown. An example of an image in which a face has been detected by the face detection unit is shown. An example of the determination result by the face direction determination unit is shown. It is a flow of acquiring a pulse wave by the terminal device according to the present disclosure. It is a flow of acquiring a pulse wave by the terminal device concerning other embodiments. FIG. 7 is a configuration diagram of main parts of a terminal device including a terminal device according to another embodiment. FIG. 7 is an example of an external view of a terminal device according to another embodiment.

Embodiments of the present disclosure will be described below based on the drawings. The embodiments described below are examples of means for realizing the present disclosure, and should be modified or changed as appropriate depending on the configuration of the device or various conditions to which the present disclosure is applied. It is not limited to. Further, in the drawings described below, the same or functionally similar components are denoted by the same reference numerals, and repeated description thereof will be omitted. Furthermore, the drawings are provided for ease of understanding when viewed in conjunction with the following description and are not necessarily drawn to scale.

[Operation of biological information acquisition device 10 (overview)]
First, the operation of the biological information acquisition device 10 according to the present disclosure will be explained with reference to FIGS. 2 and 3. The biological information acquisition device 10 is a device for acquiring biological information of a person from a captured image of the person. The biological information includes pulse waves, pulse rate, blood pressure, and the like. The biological information may also include heartbeat interval (RRI), respiratory sinus arrhythmia component (RSA), respiratory rate, blood pressure fluctuation (MWSA), and the like. In the following explanation, it is assumed that the biological information is a pulse wave.

In FIGS. 2 and 3, it is assumed that when a plurality of people are detected on the screen, the sizes of the images of the plurality of faces are compared, and the pulse rate of the person with the largest face is acquired.

FIG. 2 shows how a pulse wave of a person whose pulse wave is to be obtained continues to be obtained when the person faces the second direction. FIG. 2 corresponds to a case where the biological information acquisition device 10 is not used.

In 110 of FIG. 2, two people are detected within the screen. The pulse of the person in front (hereinafter referred to as the "first person" (first subject)) is acquired. The first person is facing a first direction. The first direction is a direction suitable for obtaining pulse waves.

At 120 in FIG. 2, the first person is facing in a second direction. The second direction is a different direction than the first direction. 120 in FIG. 2 shows how the pulse wave of the first person continues to be acquired even when the first person faces the second direction. Since the first person is facing the second direction, the accuracy of the acquired pulse wave of the first person may be low.

FIG. 3 shows how the acquisition of a pulse wave of a person is interrupted when the person whose pulse wave is to be acquired turns to the second direction. FIG. 3 corresponds to the case where the biological information acquisition device 10 is used. Since 130 in FIG. 3 shows the same situation as 110 in FIG. 2, the explanation here will be omitted.

At 140 of FIG. 3, the first person is facing in a second direction. The second direction is not a direction suitable for acquiring pulse waves. Therefore, the biological information acquisition device 10 interrupts acquisition of the first person's pulse. Thereby, the biological information acquisition device 10 can acquire the first person's pulse with higher accuracy.

Details of the biometric information acquisition device 10 and the terminal device 1 equipped with the biometric information acquisition device 10 will be described below with reference to FIG. 1 and the like.

[Configuration of terminal device 1 and biological information acquisition device 10]
FIG. 1 is a configuration diagram of main parts of a terminal device 1 including a biometric information acquisition device 10 according to the present disclosure. The terminal device 1 is a terminal device that is capable of acquiring (calculating) biometric information of a person from a captured image (moving image) of the person's face. The terminal device 1 may be a smartphone, a tablet, a notebook PC, a pulse wave measurement dedicated terminal, or the like. The biological information includes pulse waves, pulse rate, blood pressure, and the like. In the following explanation, the biological information is assumed to be a pulse wave.

The terminal device 1 includes an imaging section 2 and a biological information acquisition device 10.

The imaging unit 2 captures an image and outputs image data related to the image to the face detection unit 11. The imaging unit 2 captures a moving image and outputs moving image data related to the moving image to the face detection unit 11 and the pulse wave calculation unit 13 (calculation unit). The imaging section 2 may output image data and/or video data to the face detection section 11 , the face direction determination section 12 , and the pulse wave calculation section 13 . The imaging unit 2 may continuously capture a plurality of images instead of capturing a moving image.

The imaging unit 2 is composed of an imaging device such as an RGB camera or an infrared camera. The imaging unit 2 may be any imaging device capable of calculating a pulse wave based on the moving image data output by the imaging unit 2.

The biological information acquisition device 10 is a device for acquiring biological information (pulse wave, etc.) of a person from a captured moving image of the person. The biological information acquisition device 10 includes a face detection section 11, a face direction determination section 12, and a pulse wave calculation section 13 (calculation section).

The face detection section 11 detects the person with the largest face in the image and video captured by the imaging section 2, and outputs the detection results to the face direction determination section 12 and the pulse wave calculation section 13. The detection result includes, for example, information regarding (1) the image of the face area detected by the face detection unit 11, (2) the position of the area, (3) the size of the area, and the like. When only one person appears in the image and video captured by the imaging unit 2, the face detection unit 11 outputs the detection result regarding the face of that one person to the face direction determination unit 12 and the pulse wave calculation unit 13. .

The face direction determination unit 12 determines the orientation of the face detected by the face detection unit 11 based on the detection result output by the face detection unit 11. The face direction determining section 12 outputs the determination result to the pulse wave calculating section 13. The determination result is (1) whether or not the face direction determined by the face direction determination unit 12 is the first direction (direction suitable for pulse wave acquisition); and/or (2) face direction determination unit 12 Contains information such as the angle of the face determined by the

The pulse wave calculation unit 13 calculates the pulse wave of the face detected by the face detection unit 11 from the moving image data acquired by the imaging unit 2 based on the determination result output by the face direction determination unit 12. The calculated pulse wave may be displayed on a display provided in the terminal device 1, or may be stored in a memory or the like.

The pulse wave calculation unit 13 may calculate other biological information (pulse rate, blood pressure, etc.) based on the calculated pulse wave. The pulse wave calculation unit 13 may calculate other biological information based on changes in pixel values of the moving image data due to changes in blood vessels.

The face detection unit 11, the face direction determination unit 12, and the pulse wave calculation unit 13 are configured by a calculation device such as a CPU, and may be provided inside the terminal device 1 or outside the cloud.

[Example of operation of face detection unit 11]
An example of the operation of the face detection section 11 will be explained with reference to FIGS. 4 to 7. FIG. 4 shows an example of an image captured by the imaging unit 2. There are two faces on the screen. The imaging unit 2 outputs image data related to the captured image to the face detection unit 11.

FIG. 5 shows an example of an image in which a face has been detected by the face detection unit 11. The face detection unit 11 detects the face of the person with the larger face (first person) among the two people in the image. An image in which a frame is drawn in the area of the first person's face may be displayed on the display of the terminal device 1.

FIG. 6 shows an example of the detection results output by the face detection unit 11. The detection result includes information regarding (1) the image of the face area detected by the face detection unit 11, (2) the position of the area (170, 20), and (3) the size of the area (100×100). . The position of the area may be the center point of the image of the face area, or the like. An example of a method for obtaining such detection results will be shown below.

FIG. 7 shows an example of an image in which a face has been detected by the face detection unit 11. First, the face detection unit 11 detects all the faces in the image, and detects the position and size of each face. Next, the face detection unit 11 selects the largest face from among all the detected faces. Then, the face detection unit 11 outputs the detection result including the image of the selected facial area, the position of the area, and the size of the area to the face direction determination unit 12. The face detection unit 11 does not need to acquire the positions of all detected faces. The face detection unit 11 may select the largest face and then obtain the size of the selected face.

Other examples for obtaining similar detection results will be shown below.

There are methods similar to pattern matching that detect faces by searching for areas that match a predetermined template. When this method is adopted, faces are usually detected using multiple template sizes for search. Use templates in descending order of size. As a result, the largest face is detected first, and face detection can be ended when even one face is detected. According to this method, only the largest faces are detected. This method can reduce the time required to detect the largest face compared to a method that detects multiple faces.

Other examples for obtaining similar results will be shown below.

First, one template with a fixed size is prepared. Next, a plurality of images obtained by enlarging or reducing the image captured by the imaging unit 2 are prepared. An area matching the template is searched for in order from the smallest image among the plurality of images. By using this method, the face detection unit 11 can detect the largest face first.

By using the various methods described above, the face detection unit 11 can detect the largest face appearing in the moving image captured by the imaging unit 2 and output the detection results to the face direction determination unit 12 and the pulse wave calculation unit 13. .

[Example of operation of face direction determination unit 12]
An example of the operation of the face direction determining section 12 will be explained with reference to FIG. FIG. 8 shows an example of the determination result by the face direction determination unit 12. The face direction determination unit 12 determines the orientation of the face detected by the face detection unit 11 based on the detection result output by the face detection unit 11. The determination result is (1) whether or not the face direction determined by the face direction determination unit 12 is the first direction (direction suitable for obtaining a pulse wave with high accuracy), or (2) face direction. It includes information regarding the angle of the face direction determined by the determination unit 12, and the like.

In the left diagram of FIG. 8, the face is facing directly toward the imaging unit 2. This direction is suitable for pulse wave acquisition and is referred to as the first direction. In the right figure of FIG. 8, the face is facing in a direction that does not directly face the imaging unit 2. This direction is not suitable for pulse wave acquisition and is referred to as a second direction. The first direction may be defined by the face direction determining section 12. This will be explained in detail below.

The first direction may include a wide range, and may be defined as a range of ±45 degrees in the horizontal direction from the directly facing direction, for example. The first direction may be defined by numerically representing the angle of the face direction, or may be defined by words such as "front", "diagonal", and "sideways".

The first direction may be defined by an angle in the horizontal direction with the vertical direction of the image as the axis when moving the face left and right. The first direction may be defined as a vertical angle with the horizontal direction of the image as an axis when moving the face up and down. The first direction may be defined by an angle in a rotation direction having an axis in a direction perpendicular to the image plane when tilting the head or tilting the terminal device 1. The first direction may be defined by combining a plurality of angles, such as a horizontal angle, a vertical angle, and/or a rotational angle.

The first direction may be defined based on the type of pulse wave to be obtained, the position at which the pulse wave is to be obtained, or the like. For example, when acquiring pulse waves on both left and right cheeks, the first direction may be a direction in which the face faces the terminal device 1 . When acquiring the pulse wave of the right cheek, the first direction may be a direction in which the face is rotated horizontally from a direction in which the face directly faces the terminal device 1, and in which the face faces the right side of the screen. When acquiring the pulse wave of the neck, the first direction may be a direction in which the face is rotated vertically from a direction in which the face directly faces the terminal device 1, and in which the face faces the upper side of the screen. When acquiring a pulse wave on the cheek in an environment where the face is illuminated from directly above, the first direction is when the face rotates from the direction directly facing the camera, tilts the head, or turns the body. It may be in a tilted direction.

Because the distribution of facial blood vessels varies from person to person, the angle of the face at which a pulse wave can be accurately obtained differs from person to person. Therefore, the first direction may be defined for each individual. In that case, the detection result output by the face detection unit 11 includes information for identifying the individual. The terminal device 1 can specify a person by storing the faces of a plurality of people in a memory in advance and comparing the faces stored in the memory with the detected faces. The face direction determination unit 12 defines a first direction based on information that identifies the person. That is, the first direction is associated with the first person in advance. The information that identifies an individual is a code that identifies an individual, such as a person's name, characters, symbols, or numbers.

The face direction determination unit 12 can define (determine) the various first directions described above using a known method. For example, the face direction determination unit 12 may estimate (determine) the angle of the face based on the positional relationship of facial organs (eyes, mouth, etc.). Further, the face direction determination unit 12 may use templates of various face angles and estimate (determine) the face direction (angle) according to the matching results with these templates.

[Pulse wave calculation method]
Next, a method for acquiring a pulse wave using the terminal device 1 will be explained with reference to FIG. FIG. 9 is a flowchart for acquiring a pulse wave using the terminal device 1.

First, in S10, the imaging section 2 is activated.

Next, in S20, the imaging unit 2 captures an image and obtains moving image data. The imaging unit 2 outputs moving image data to the face detection unit 11.

In S30, the face detection unit 11 detects the largest face in the image captured by the imaging unit 2. If the face detection unit 11 detects the largest face (Yes in S40), the process advances to S50. The face detection section 11 outputs the detection result to the face direction determination section 12 and the pulse wave calculation section 13. The detection result includes the image, position, and size of the detected facial area. If the face detection unit 11 does not detect any face (No in S40), the process returns to S20.

In S50, the face direction determining unit 12 determines whether the face direction of the largest face image included in the detection result is the first direction. If Yes in S50, the process advances to S60. If No in S50, the process returns to S20.

In S60, the imaging unit 2 captures a moving image and obtains moving image data. The imaging unit 2 outputs moving image data to the pulse wave calculation unit 13.

In S70, the pulse wave calculation unit 13 calculates the pulse wave of the face detected by the face detection unit 11 based on the moving image data. The position of the face in the moving image is determined based on the face position information included in the detection result.

As described above, the biological information acquisition device 10 detects the pulse wave of the person whose face is the largest among the people in the image (moving image) captured by the imaging unit 2, when the person is facing the first direction. You can get it when you want.

Generally, when a person's face is imaged in a large size, the number of pixels that can be used to calculate a pulse wave increases, and as a result, it becomes easier to obtain a pulse wave with a high signal-to-noise ratio. The terminal device 1 acquires the pulse wave of the person whose face is the largest among the one or more persons included in the image captured by the imaging unit 2 . Thereby, the terminal device 1 can acquire a highly accurate pulse wave.

When a user of the terminal device 1 attempts to obtain his or her own pulse wave, there is a high possibility that the user is at a position closest to the imaging unit 2. In other words, the closer a person is to the terminal device 1, the larger the person is likely to appear in the video image. Therefore, the terminal device 1 makes it easier to acquire the user's pulse wave.

The terminal device 1 acquires the pulse wave of the person who appears largest in the video image when that person faces the first direction. Thereby, the terminal device 1 can acquire a more accurate pulse wave.

The terminal device 1 may have the following configuration.

The face detection unit 11 can detect a face using a known technique. Known techniques include, for example, machine learning such as pattern matching.

When the imaging unit 2 outputs image data to the face direction determination unit 12, the face detection unit 11 does not need to include the image of the face area detected by the face detection unit 11 in the detection result. The face direction determination unit 12 can acquire an image of the largest face area based on the position of the face area included in the detection result.

The face direction determination section 12 outputs the determination result to the pulse wave calculation section 13. The determination result may include information indicating the position of the face included in the detection result output from the face detection section 11 to the face direction determination section 12. In this case, the face detection section 11 does not need to output the detection result to the pulse wave calculation section 13. The pulse wave calculation unit 13 can determine the position of the face for calculating the pulse wave by searching the video data output from the imaging unit 2 for a face image included in the detection result.

S60 may be performed for a predetermined period of time (for example, 10 seconds). The face detection unit 11 may detect a face appearing in an image within a moving image at the same time that the imaging unit 2 captures a moving image. The face detection unit 11 may track the detected face. The imaging unit 2 continues to capture moving images while a face is included in the image within the moving image, and when the face detection unit 11 no longer detects a face or when the face detection unit 11 is no longer able to track a face. When this happens, capturing the moving image may be terminated.

The pulse wave calculation unit 13 may calculate the pulse wave using a known technique (independent component analysis, pigment component separation, etc.) based on the temporal change in pixel values of the moving image data. The pulse wave calculation unit 13 acquires a pulse wave from a part of the face detected by the face detection unit 11 that mainly shows the skin. The pulse wave calculation unit 13 may calculate the pulse wave after the imaging unit 2 finishes capturing the moving image. The pulse wave calculation unit 13 may calculate the pulse wave at the same time as the acquisition of moving image data starts. If the determination information output from the face direction determination unit 12 to the pulse wave calculation unit 13 includes information regarding the direction of the face, the pulse wave calculation unit 13 measures the pulse wave according to the direction of the face. You may change body parts.

[Embodiment 2]
Embodiment 2 will be described below. For convenience of explanation, members having the same functions as those described in Embodiment 1 are given the same reference numerals, and the description thereof will not be repeated.

Another method of acquiring a pulse wave using the terminal device 1 according to this embodiment will be described with reference to FIG. FIG. 10 is a flowchart for acquiring a pulse wave using the terminal device 1 according to the second embodiment. Since S110 to S160 are the same as S10 to S60 in FIG. 9, the explanation thereof will be omitted.

In S170, the imaging unit 2 outputs image data included in the moving image to the face detection unit 11 while acquiring moving image data. The imaging unit 2 may output all image data included in the moving image to the face detection unit 11. The imaging unit 2 may output image data to the face detection unit 11 at predetermined intervals set in advance. The predetermined interval may be based on time (eg, every second) or number of images (eg, every 30 images). The imaging unit 2 may output image data to the face detection unit 11 every time it acquires image data to be output to the face detection unit 11. The imaging section 2 may output the image data to the face detection section 11 all at once, each time a given number of image data to be output to the face detection section 11 is accumulated.

In S180, the face detection unit 11 detects the largest face in the image captured by the imaging unit 2. If the face detection unit 11 detects the largest face (Yes in S190), the process advances to S200. At this time, the face detection section 11 outputs the detection result to the face direction determination section 12. If the face detection unit 11 does not detect any face (No in S190), the process returns to S120. Thereby, the imaging unit 2 stops (or interrupts) the acquisition of moving image data.

In S200, the face direction determining unit 12 determines whether the largest face in the moving images acquired by the imaging unit 2 is facing the first direction, based on the detection result. If Yes in S200, the process advances to S210. The face direction determination section 12 outputs the determination result to the pulse wave calculation section 13. If No in S200, the process returns to S120. The imaging unit 2 interrupts the acquisition of moving image data. The above operations are continued until the imaging unit 2 finishes acquiring the moving image data (Yes in S210).

In S220, the pulse wave calculation unit 13 calculates the pulse wave of the face detected by the face detection unit 11 from the moving image captured by the imaging unit 2.

When the imaging unit 2 interrupts acquisition of moving image data, the user may be informed of this fact. When the imaging unit 2 interrupts the acquisition of moving image data, the imaging unit 2 reacquires the image data, the face detection unit 11 detects the largest face, and the face direction determination unit 12 determines the direction of the face. good. When the imaging unit 2 interrupts the acquisition of moving image data, the biological information acquisition device 10 may ask the user whether to acquire a pulse wave. When the imaging unit 2 interrupts the acquisition of moving image data, the pulse wave calculation unit 13 may calculate the pulse wave from the moving images acquired up to that point.

According to the above configuration, while the imaging unit 2 is acquiring moving image data, the face whose pulse wave is acquired is the largest in the image, and the face is facing in the first direction. Guaranteed. Therefore, if the person's face faces in the second direction while the imaging unit 2 is acquiring moving image data, the pulse wave calculation unit 13 does not calculate the pulse wave. Thereby, the biological information acquisition device 10 can acquire a highly accurate pulse wave.

In addition, in response to the face detection unit 11 detecting a person (second person (second subject)) different from the person whose pulse wave is desired (first person), the imaging unit 2 detects the moving image data. When the acquisition of the first person is started, the face detection unit 11 detects the first person while the imaging unit 2 is acquiring the moving image data. In response to the detection of the first person by the face detection unit 11, the pulse wave calculation unit 13 calculates the pulse wave of the first person. Such a configuration is also possible. The configuration will be explained below.

If the first person wears a shield such as a mask or sunglasses, the face detection unit 11 may not be able to detect the first person's face. In such a case, if the second person is included in the image, the face detection unit 11 detects the second person's face. If the face direction determination unit 12 determines that the second person is facing the first direction, the imaging unit 2 starts acquiring moving image data, and the pulse wave calculation unit 13 detects the second person's pulse. Calculate the waves.

In the first embodiment, the pulse wave calculation unit 13 can continue to calculate the pulse wave of the second person. In the second embodiment, when the first person removes the obstructing object and the person whose face is reflected the largest is switched from the second person to the first person, the face detection unit 11 detects the first person. Thereby, the pulse wave calculation unit 13 acquires the pulse wave of the first person.

It may be possible to set in advance whether or not to operate the face detection section 11 and the face direction determination section 12 while the imaging section 2 is acquiring moving image data.

For example, while the pulse wave calculation unit 13 is calculating the pulse wave of the first person, a second person may come near the imaging unit 2. At this time, there is a possibility that the person with the largest face in the image will be switched from the first person to the second person. In this case, the imaging unit 2 interrupts the acquisition of moving image data, and the pulse wave calculation unit 13 stops calculating the pulse wave of the first person. At this time, if the operation of at least one of the face detection section 11 and the face direction determination section 12 has been set in advance to be stopped for an image in the moving image, the imaging section 2 acquires the moving image data. Then, the pulse wave calculation unit 13 can calculate the pulse wave of the first person.

In this way, the biological information acquisition device 10 can be realized with various configurations.

[Embodiment 3]
Embodiment 3 will be described below using FIGS. 11 and 12. For convenience of explanation, members having the same functions as those described in Embodiments 1 and 2 will be denoted by the same reference numerals, and the description thereof will not be repeated.

FIG. 11 is a configuration diagram of main parts of a terminal device 15 including the biological information acquisition device 10 according to the present embodiment. The terminal device 15 includes an imaging section 2, a biological information acquisition device 10, a notification section 5, and a memory 6.

The notification unit 5 provides notification based on the determination result of the face direction determination unit 12. The notification unit 5 includes, for example, (1) a display (e.g., a liquid crystal screen or an organic EL screen), (2) a sound generator (e.g., a speaker), (3) a light generator (e.g., an LED light), (4) A vibration generator (eg, a vibrator). The notification unit 5 may be configured by a combination of the above (1) to (4).

There may be a case where the user of the terminal device 15 cannot grasp the first direction. In such a case, the notification unit 5 can notify the user of the first direction using at least one of the above (1) to (4).

Furthermore, there may be a case where the user cannot see the display of the terminal device 15 when facing in the first direction. In such a case, the user cannot see through the display whether the user is facing the first direction or whether the imaging unit 2 is capturing an image (moving image). In such a case, the notification unit 5 can notify the user of the user's current situation, for example, by the methods (2) to (4) above.

For example, if the user is facing in the second direction and the user tries to force himself to view the display, the pulse wave calculation unit 13 may not be able to accurately calculate the user's pulse wave. Therefore, the notification unit 5 notifies the user that the acquisition of the moving image has been completed at the timing when the imaging unit 2 has completed the acquisition of the moving image. This allows the user to continue facing the first direction while the imaging unit 2 is capturing a moving image.

The memory 6 records images (moving images) captured by the imaging unit 2, information for identifying users, and the like. The memory 6 may naturally be included in the terminal device 1 according to the first and second embodiments. The memory 6 may be provided outside the terminal device 1 instead of inside the terminal device 1 .

FIG. 12 is an example of an external view of the terminal device 15 according to the present disclosure. The terminal device 15 includes a front camera section 2a, a sound generating section 5a, and a display 7.

The front camera section 2a is used as an imaging section 2, and obtains an image of a person as a subject.

The display 7 displays an image (moving image) captured by the imaging unit 2, a pulse wave calculated by the pulse wave calculation unit 13, or the like.

The sound generating section 5a is used as the notification section 5 and provides notification based on the determination result of the face direction determining section 12. The sound generating section 5a notifies the user of the determination result of the face direction determining section 12 by sound. For example, the sound generation unit 5a notifies by sound that the acquisition of the moving image has started in response to the determination result by the face direction determination unit 12 that the user's face is facing the first direction. Further, the sound generating section 5a continues to emit the sound upon receiving the determination result by the face direction determining section 12 that the user's face is not facing the first direction. In this case, based on the detection result of the face detection section 11, the sound generation section 5a may continue to emit the sound even when the user's face is not detected.

The display 7 may perform the above notification in place of the sound generator 5a or together with the sound generator 5a. For example, the display 7 displays the first direction. This allows the user to easily understand the first direction.

In this way, the notification unit 5 can provide various notifications to the user using various methods. The notification unit 5 may notify the user of this at the timing when the imaging unit 2 completes acquisition of the moving image or at the timing when the pulse wave calculation unit 13 finishes calculating the pulse wave.

In this way, the notification section 5 can notify the user of the operating state of at least one of the imaging section 2, the face detection section 11, the face direction determination section 12, and the pulse wave calculation section 13.

The biometric information acquisition device 10 has been described as one that acquires biometric information of a person. However, the biometric information acquisition device 10 may also acquire biometric information about people other than humans using a similar method. Therefore, "subject" may include animals other than humans.

[Example of implementation using software]
The control blocks of the biological information acquisition device 10 (particularly the face detection section 11, face direction determination section 12, and pulse wave calculation section 13) are realized by a logic circuit (hardware) formed on an integrated circuit (IC chip) or the like. or may be realized by software.

In the latter case, the biological information acquisition device 10 includes a computer that executes instructions of a program that is software that implements each function. This computer includes, for example, at least one processor (control device) and at least one computer-readable recording medium storing the above program. Then, in the computer, the processor reads the program from the recording medium and executes it, thereby achieving the object of the present disclosure. As the processor, for example, a CPU (Central Processing Unit) can be used. As the recording medium, in addition to "non-temporary tangible media" such as ROM (Read Only Memory), tapes, disks, cards, semiconductor memories, programmable logic circuits, etc. can be used. Further, the computer may further include a RAM (Random Access Memory) for expanding the above program. Furthermore, the program may be supplied to the computer via any transmission medium (communication network, broadcast waves, etc.) that can transmit the program. Note that one aspect of the present disclosure can also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

The present disclosure is not limited to the embodiments described above, and various changes can be made within the scope of the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments. are also included within the technical scope of the present disclosure. Furthermore, new technical features can be formed by combining the technical means disclosed in each embodiment.

1, 15 Terminal device 2 Imaging section 2a Front camera section 5 Notification section 5a Sound generation section 6 Memory 7 Display 10 Biological information acquisition device 11 Face detection section 12 Face direction determination section 13 Pulse wave calculation section (calculation section)

Claims (16)

A biological information acquisition device that acquires biological information,
a face detection unit that detects the largest face of the first subject among the faces of the one or more subjects included in the first image captured by the imaging unit;
a face direction determination unit that determines whether the face of the first subject detected by the face detection unit is facing a first direction that is a direction suitable for acquiring the biometric information;
When the face direction determining unit determines that the face of the first subject is facing in the first direction, a second image different from the first image, which is captured by the imaging unit, is a calculation unit that calculates biometric information of the first subject,
When the face detection unit detects that the face of a second subject different from the first subject included in the second image is the largest face, the second image by the imaging unit A biological information acquisition device that stops continuous imaging. A biological information acquisition device that acquires biological information,
a face detection unit that detects the largest face of the first subject among the faces of the one or more subjects included in the first image captured by the imaging unit;
a face direction determination unit that determines whether the face of the first subject detected by the face detection unit is facing a first direction that is a direction suitable for acquiring the biometric information;
When the face direction determining unit determines that the face of the first subject is facing in the first direction, a second image different from the first image, which is captured by the imaging unit, is a calculation unit that calculates biometric information of the first subject;
The first direction is a biological information acquisition device that is previously associated with the first subject. If the face of the first subject is facing in a second direction different from the first direction while the imaging unit is continuously capturing the second images, the face direction determination unit The biological information acquisition device according to claim 1 or 2 , wherein when the determination is made, the imaging of the second image by the imaging unit is stopped. Any one of claims 1 to 3 , wherein the calculation unit calculates the biological information at the same time as the imaging unit starts capturing the second image, or after the imaging unit finishes capturing the second image. The biological information acquisition device according to item 1. 5. At least one of the face detection section and the face direction determination section stops operating while the imaging section is capturing the second image. biological information acquisition device. The calculation unit according to any one of claims 1 to 5 , wherein the calculation unit changes the body part of the first subject from which the biometric information is to be acquired, depending on the orientation of the first subject's face. Biological information acquisition device. The first direction is any one of a direction facing the imaging unit, a horizontal angle with the vertical direction of the image as the axis, and a vertical angle with the horizontal direction of the image as the axis. The biological information acquisition device according to any one of claims 1 to 6 , defined by: The first direction includes (1) the type of the biometric information, (2) the body part of the first subject from which the biometric information is to be obtained, and (3) the subject from which the biometric information is to be captured. The biological information acquisition device according to any one of claims 1 to 7 , wherein the biological information acquisition device is changed according to at least one of them. The biological information acquisition device according to any one of claims 1 to 8 , wherein the biological information includes at least one of a pulse wave, a pulse rate, and a blood pressure. A terminal device that acquires biometric information,
an imaging unit that captures a first image and a second image;
a face detection unit that detects the face of the first subject that is the largest among the faces of the plurality of subjects included in the first image;
a face direction determination unit that determines whether the face of the first subject detected by the face detection unit is facing a first direction suitable for acquiring the biological information;
a calculation unit that calculates biometric information of the first subject from the second image when the face direction determination unit determines that the face of the first subject is facing the first direction; Equipped with
When the face detection unit detects that the face of a second subject different from the first subject included in the second image is the largest face, the second image by the imaging unit A terminal device that stops continuous imaging . A terminal device that acquires biometric information,
an imaging unit that captures a first image and a second image;
a face detection unit that detects the face of the first subject that is the largest among the faces of the plurality of subjects included in the first image;
a face direction determination unit that determines whether the face of the first subject detected by the face detection unit is facing a first direction suitable for acquiring the biological information;
a calculation unit that calculates biometric information of the first subject from the second image when the face direction determination unit determines that the face of the first subject is facing the first direction; Equipped with
A terminal device in which the first direction is associated with the first subject in advance . The terminal device according to claim 10 or 11 , further comprising a notification section that notifies a user of the operating state of at least one of the imaging section, the face detection section, the face direction determination section, and the calculation section. A biological information acquisition method for acquiring biological information, the method comprising:
a detection step of detecting the largest face of the first subject among the faces of the plurality of subjects included in the first image captured by the imaging unit;
a determining step of determining whether the face of the first subject detected in the detecting step is facing a first direction suitable for acquiring the biological information;
If it is determined in the determination step that the face of the first subject is facing the first direction, a second image different from the first image, which is captured by the imaging unit, is a calculation step of calculating biological information of the first subject ,
When it is detected in the detection step that the face of a second subject different from the first subject included in the second image is the largest face, the imaging unit detects that the face of the second subject is the largest face. A biological information acquisition method that stops continuous imaging . A biological information acquisition method for acquiring biological information, the method comprising:
a detection step of detecting the largest face of the first subject among the faces of the plurality of subjects included in the first image captured by the imaging unit;
a determining step of determining whether the face of the first subject detected in the detecting step is facing a first direction suitable for acquiring the biological information;
If it is determined in the determination step that the face of the first subject is facing the first direction, a second image different from the first image, which is captured by the imaging unit, is a calculation step of calculating biological information of the first subject ,
In the biological information acquisition method , the first direction is associated with the first subject in advance . A biological information acquisition program for causing a computer to function as the biological information acquisition device according to any one of claims 1 to 9 , wherein the computer functions as the face detection section, the face direction determination section, and the calculation section. Biometric information acquisition program for functioning. A computer-readable recording medium on which the biological information acquisition program according to claim 15 is recorded.

Images