JP2021096537A - Biological information acquisition device, terminal device, biological information acquisition method, biological information acquisition program, and computer readable recording medium - Google Patents

Abstract

To provide a biological information acquisition device capable of acquiring biological information with high accuracy.SOLUTION: A biological information acquisition device (10) comprises a face detection unit (11) which detects a face of a first person, which is largest in a captured image, a face direction determination unit (12) which determines whether the face of the first person faces a first direction or not, and a pulse wave calculation unit (13) which calculates biological information on the first person when the face direction determination unit (12) determines that the face of the first person faces the first direction.SELECTED DRAWING: Figure 1

Description

The present disclosure relates to a biometric information acquisition device, a terminal device, a biometric information acquisition method, a biometric information acquisition program, and a computer-readable recording medium capable of acquiring biometric information.

A technique is known in which when a plurality of faces are detected in an image captured by a terminal device, one person is selected from the plurality of faces. Patent Document 1 discloses a technique of comparing the sizes of images of a plurality of faces and determining a subject based on the face having the largest size. Patent Documents 2 and 3 also disclose a technique for selecting one face from a plurality of faces in an image.

Japanese Unexamined Patent Publication No. 2005-049854 (published on February 24, 2005) Japanese Unexamined Patent Publication No. 2007-6033 (published on January 11, 2007) Japanese Unexamined Patent Publication No. 2009-77266 (published on April 9, 2009)

However, the above-mentioned prior art has the following problems.

In recent years, a technique for calculating a pulse wave of a person by analyzing a moving image of the face of the person captured by a camera has been developed. Applications using such technology are widely used in camera-equipped mobile terminals (smartphones, etc.). When the faces of a plurality of persons are shown in a moving image, in principle, it is possible to measure the pulse waves of the plurality of persons.

However, trying to calculate the pulse waves of a plurality of people at the same time puts a heavy load on the image processing ability (calculation ability) of the mobile terminal. Therefore, in order to acquire the pulse wave with high accuracy in real time, a method of selecting one of a plurality of persons and acquiring the pulse wave of the selected one person can be considered. Patent Documents 1 to 3 disclose a technique for selecting one face from a plurality of faces in an image. On the other hand, Patent Documents 1 to 3 do not disclose a technique for calculating a pulse wave from a face in an image.

Further, in general, when the face of a person is photographed in a large size, the number of pixels that can be used for calculating the pulse wave increases, and as a result, it becomes easy to acquire a pulse wave having a high signal-to-noise ratio.

However, just because the captured person's face is large does not necessarily mean that a highly accurate pulse wave can be obtained. There is no prior art document that points out this point as an issue.

One aspect of the present disclosure is a biological information acquisition device, a terminal device, and a biological information acquisition method capable of acquiring biological information of the measurement target from an image including an image of the biological body to be measured with higher accuracy than before. , A biometric information acquisition program, and a computer-readable recording medium.

In order to solve the above problems, the biometric information acquisition device according to one aspect of the present disclosure is a biometric information acquisition device that acquires biometric information, and is one or more included in the first image captured by the imaging unit. Of the faces of the subject, the face detection unit that detects the face of the first subject and the face of the first subject detected by the face detection unit obtain the biometric information. When the face direction determination unit determines whether the face is facing the first direction, which is a suitable direction, and the face direction determination unit determines that the face of the first subject is facing the first direction. A calculation unit that calculates biological information of the first subject from a second image different from the first image captured by the imaging unit is provided.

In order to solve the above problems, the terminal device according to one aspect of the present disclosure is a terminal device that acquires biometric information, and includes an imaging unit that captures a first image and a second image, and the first image. Of the faces of one or more subjects included in the image, the face detection unit that detects the face of the first subject and the face of the first subject detected by the face detection unit are the above. The face direction determination unit determines whether the face is facing the first direction suitable for acquiring biological information, and the face direction determination unit determines that the face of the first subject is facing the first direction. In this case, a calculation unit for calculating the biological information of the first subject from the second image is provided.

In order to solve the above problems, the biometric information acquisition method according to one aspect of the present disclosure is a biometric information acquisition method for acquiring biometric information, and is one or more included in the first image captured by the imaging unit. Of the faces of the subject, the detection step of detecting the face of the first subject, which is the largest image, and the face of the first subject detected in the detection step are suitable for acquiring the biological information. When it was determined in the determination step that the face of the first subject was oriented in the first direction and the determination step that the face of the first subject was oriented in the first direction, the image was taken by the imaging unit. The calculation step of calculating the biological information of the first subject from the second image different from the first image is included.

According to one aspect of the present disclosure, the biological information of the measurement target can be calculated with higher accuracy than before from the image including the image of the living body of the measurement target.

It is a main part block diagram of the terminal apparatus provided with the biological information acquisition part which concerns on this disclosure. It shows how a person who acquires a pulse wave continues to acquire the pulse wave when the person turns to the second direction. It shows how the acquisition of the pulse wave of the person is interrupted when the person who acquires the pulse wave turns to the second direction. An example of the image captured by the imaging unit is shown. An example of the image detected by the face detection unit is shown. An example of the detection result output by the face detection unit is shown. An example of the image detected by the face detection unit is shown. An example of the judgment result by the face direction judgment unit is shown. This is a flow of acquiring a pulse wave by the terminal device according to the present disclosure. This is a flow of acquiring a pulse wave by a terminal device according to another embodiment. It is a main part block diagram of the terminal apparatus provided with the terminal apparatus which concerns on other embodiments. This is an example of an external view of a terminal device according to another embodiment.

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The embodiments described below are an example as means for realizing the present disclosure, and should be appropriately modified or changed depending on the configuration of the device to which the present disclosure is applied or various conditions, and the present disclosure is the following embodiments. It is not limited to. Further, in the drawings described below, the same or functionally similar components are designated by the same reference numerals, and the repeated description thereof will be omitted. Further, each drawing is shown for easy understanding when referred to together with the following description, and is not necessarily drawn at a certain ratio of scale.

[Operation of biological information acquisition device 10 (overview)]
First, the operation of the biological information acquisition device 10 according to the present disclosure will be described with reference to FIGS. 2 and 3. The biological information acquisition device 10 is a device for acquiring biological information of the person from the captured image of the person. Biological information includes pulse wave, pulse rate, blood pressure and the like. The biological information may also include a heart rate interval (RRI: RRInterval), a respiratory sinus arrhythmia component (RSA: Respiratory Sinus Arrhythmia), a respiratory rate, a blood pressure fluctuation (MWSA: Mayer Wave related Sinus Arrhythmia), and the like. In the following description, it is assumed that the biological information is a pulse wave.

In FIGS. 2 and 3, when a plurality of people are detected on the screen, the sizes of the images of the plurality of faces are compared, and the pulse of the person with the largest face is acquired.

FIG. 2 shows how a person who acquires a pulse wave continues to acquire the pulse wave when the person turns to the second direction. FIG. 2 corresponds to the case where the biological information acquisition device 10 is not used.

In 110 of FIG. 2, two people are detected in the screen. The pulse of the person in the foreground (hereinafter, referred to as "first person" (first subject)) is acquired. The first person is facing the first direction. The first direction is a direction suitable for pulse wave acquisition.

In 120 of FIG. 2, the first person is facing the second direction. The second direction is different from the first direction. 120 in FIG. 2 shows how the pulse wave of the first person continues to be acquired even when the first person faces the second direction. Since the first person is facing the second direction, the acquired pulse wave of the first person can be less accurate.

FIG. 3 shows how the acquisition of the pulse wave of the person is interrupted when the person who acquires the pulse wave turns to the second direction. FIG. 3 corresponds to the case where the biological information acquisition device 10 is used. Since 130 in FIG. 3 shows the same situation as 110 in FIG. 2, the description here will be omitted.

In 140 of FIG. 3, the first person is facing the second direction. The second direction is not suitable for pulse wave acquisition. Therefore, the biological information acquisition device 10 interrupts the acquisition of the pulse of the first person. As a result, the biological information acquisition device 10 can acquire the pulse of the first person with higher accuracy.

Hereinafter, the details of the biological information acquisition device 10 and the terminal device 1 provided with the biological information acquisition device 10 will be described with reference to FIG. 1 and the like.

[Configuration of terminal device 1 and biological information acquisition device 10]
FIG. 1 is a configuration diagram of a main part of a terminal device 1 provided with the biological information acquisition device 10 according to the present disclosure. The terminal device 1 is a terminal device capable of acquiring (calculating) biological information of a person from an image (moving image) of the captured person's face. The terminal device 1 may be a smartphone, a tablet, a notebook PC, a dedicated terminal for pulse wave measurement, or the like. Biological information includes pulse wave, pulse rate, blood pressure and the like. In the following description, biological information is a pulse wave.

The terminal device 1 includes an imaging unit 2 and a biological information acquisition device 10.

The imaging unit 2 captures an image and outputs image data related to the image to the face detection unit 11. The imaging unit 2 captures a moving image and outputs the moving image data related to the moving image to the face detection unit 11 and the pulse wave calculation unit 13 (calculation unit). The imaging unit 2 may output image data and / or moving image data to the face detection unit 11, the face direction determination unit 12, and the pulse wave calculation unit 13. The imaging unit 2 may continuously capture a plurality of images instead of capturing a moving image.

The image pickup unit 2 is composed of an image pickup device such as an RGB camera or an infrared camera. The imaging unit 2 may be any imaging device capable of calculating a pulse wave based on the moving image data output by the imaging unit 2.

The biological information acquisition device 10 is a device for acquiring biological information (pulse waves, etc.) of the person from the captured moving image of the person. The biological information acquisition device 10 includes a face detection unit 11, a face direction determination unit 12, and a pulse wave calculation unit 13 (calculation unit).

The face detection unit 11 detects the person with the largest face in the image captured by the imaging unit 2 and the moving image, and outputs the detection result to the face direction determination unit 12 and the pulse wave calculation unit 13. The detection result includes, for example, information regarding (1) an image of a face region detected by the face detection unit 11, (2) the position of the region, (3) the size of the region, and the like. When only one person is shown in the image and the moving image captured by the imaging unit 2, the face detection unit 11 outputs the detection result regarding the face of the one person to the face direction determination unit 12 and the pulse wave calculation unit 13. ..

The face direction determination unit 12 determines the orientation of the face detected by the face detection unit 11 based on the detection result output by the face detection unit 11. The face direction determination unit 12 outputs the determination result to the pulse wave calculation unit 13. The determination result is (1) whether or not the direction of the face determined by the face direction determination unit 12 is the first direction (direction suitable for pulse wave acquisition), and / or (2) the face direction determination unit 12 Includes information about the angle of face orientation determined by.

The pulse wave calculation unit 13 calculates the pulse wave of the face detected by the face detection unit 11 from the moving image data acquired by the imaging unit 2 based on the determination result output by the face direction determination unit 12. The calculated pulse wave may be displayed on a display provided in the terminal device 1 or the like, or may be stored in a memory or the like.

The pulse wave calculation unit 13 may calculate other biological information (pulse rate, blood pressure, etc.) based on the calculated pulse wave. The pulse wave calculation unit 13 may calculate other biological information based on the change in the pixel value of the moving image data due to the change in the blood vessel.

The face detection unit 11, the face direction determination unit 12, and the pulse wave calculation unit 13 are configured by an arithmetic unit such as a CPU, and may be provided inside the terminal device 1 or outside such as a cloud.

[Operation example of face detection unit 11]
An operation example of the face detection unit 11 will be described with reference to FIGS. 4 to 7. FIG. 4 shows an example of an image captured by the imaging unit 2. Two faces are shown on the screen. The image capturing unit 2 outputs image data related to the captured image to the face detection unit 11.

FIG. 5 shows an example of an image in which the face detection unit 11 detects a face. The face detection unit 11 detects the face of the person with the larger face (first person) among the two people shown in the image. An image in which a frame is drawn in the area of the face of the first person may be displayed on the display of the terminal device 1.

FIG. 6 shows an example of the detection result output by the face detection unit 11. The detection result includes information on (1) an image of the face area detected by the face detection unit 11, (2) the position of the area (170, 20), and (3) the size of the area (100 × 100). .. The position of the region may be the center point of the image of the facial region or the like. Hereinafter, an example of a method for obtaining such a detection result will be shown.

FIG. 7 shows an example of an image in which the face detection unit 11 detects a face. First, the face detection unit 11 detects all the plurality of faces in the image and detects the position and size of each. Next, the face detection unit 11 selects the largest face from all the detected faces. Then, the face detection unit 11 outputs the detection result including the image of the selected face area, the position of the area, and the size of the area to the face direction determination unit 12. The face detection unit 11 does not have to acquire the positions of all the detected faces. The face detection unit 11 may acquire the size of the selected face after selecting the largest face.

Hereinafter, another example for obtaining the same detection result will be shown.

There is a method similar to pattern matching in which face detection is performed by searching for an area that matches a predetermined template. When this method is adopted, face detection is usually performed using a plurality of template sizes used for search. Use templates in descending order of size. As a result, the largest face is detected first, and face detection can be terminated when even one face is detected. According to this method, only the largest face is detected. This method can reduce the time required to detect the largest face as compared with the method of detecting a plurality of faces.

Other examples for obtaining similar results are shown below.

First, prepare one template with a certain size. Next, a plurality of images obtained by enlarging or reducing the images captured by the imaging unit 2 are prepared. The area that matches the template is searched in order from the smallest image among the plurality of images. By using this method, the face detection unit 11 can detect the largest face first.

By the above various methods, the face detection unit 11 can detect the largest face captured in the moving image captured by the imaging unit 2 and output the detection result to the face direction determination unit 12 and the pulse wave calculation unit 13. ..

[Operation example of face direction determination unit 12]
An operation example of the face direction determination unit 12 will be described with reference to FIG. FIG. 8 shows an example of the determination result by the face direction determination unit 12. The face direction determination unit 12 determines the orientation of the face detected by the face detection unit 11 based on the detection result output by the face detection unit 11. The determination result is (1) whether or not the direction of the face determined by the face direction determination unit 12 is the first direction (direction suitable for accurately acquiring the pulse wave), or (2) the face direction. Includes information about the angle of the face orientation determined by the determination unit 12, and the like.

In the left figure of FIG. 8, the face faces the image pickup unit 2. This direction is suitable for pulse wave acquisition and is referred to as a first direction. In the right figure of FIG. 8, the face is oriented so as not to face the imaging unit 2. This direction is not suitable for pulse wave acquisition and is referred to as a second direction. The first direction may be defined by the face direction determination unit 12. Hereinafter, a specific description will be given.

The first direction may include a range having a width, and may be defined as, for example, a range of ± 45 degrees in the horizontal direction from the facing direction. The first direction may be defined by expressing the angle of the face orientation numerically, or may be defined by a language such as "front", "licking", or "horizontal".

The first direction may be defined by a horizontal angle about the vertical direction of the image when the face is moved left and right. The first direction may be defined by a vertical angle about the horizontal direction of the image when the face is moved up and down. The first direction may be defined by the angle of the rotation direction having an axis in the direction perpendicular to the image plane when the neck is bent or the terminal device 1 is tilted. The first direction may be defined by combining a plurality of angles such as a horizontal angle, a vertical angle, and / or a rotation angle.

The first direction may be defined based on the type of pulse wave to be acquired, the position where the pulse wave is acquired, or the like. For example, when acquiring pulse waves on both the left and right cheeks, the first direction may be the direction in which the face faces the terminal device 1. When acquiring the pulse wave of the right cheek, the first direction may be a direction in which the face is rotated horizontally from the direction facing the terminal device 1 and the face faces the right side of the screen. When acquiring the pulse wave of the nape of the neck, the first direction may be a direction in which the face is rotated in the vertical direction from the direction facing the terminal device 1 and the face faces the upper side of the screen. When acquiring a pulse wave on the cheek in an environment where the face is exposed to light from directly above, the first direction is that the face is rotated in the direction of rotation from the direction facing the camera, the neck is tilted, or the body is tilted. It may be tilted.

Since the distribution of blood vessels on the face varies from person to person, the angle of the face that can accurately acquire pulse waves differs from person to person. Therefore, the first direction may be defined for each individual. In that case, the detection result output by the face detection unit 11 includes information for identifying an individual. The terminal device 1 can record the faces of a plurality of persons in a memory in advance, collate the faces in the memory with the faces detected, and identify the persons. The face direction determination unit 12 defines the first direction based on the information that identifies the person. That is, the first direction is associated with the first person in advance. The information that identifies an individual is a code that identifies the individual, which is represented by a person's name, characters, symbols, numbers, or the like.

The face direction determination unit 12 can define (determine) the various first directions described above by using a known method. For example, the face direction determination unit 12 may estimate (determine) the angle of the face based on the positional relationship of the facial organs (eyes, mouth, etc.). Further, the face direction determination unit 12 may use various templates of face angles and estimate (determine) the direction (angle) of the face according to the matching result with those templates.

[Pulse wave calculation method]
Next, a method of acquiring a pulse wave by the terminal device 1 will be described with reference to FIG. FIG. 9 is a flow of acquiring a pulse wave by the terminal device 1.

First, in S10, the imaging unit 2 is activated.

Next, in S20, the imaging unit 2 captures an image and acquires moving image data. The imaging unit 2 outputs moving image data to the face detection unit 11.

In S30, the face detection unit 11 detects the largest face in the image captured by the image pickup unit 2. When the face detection unit 11 detects the largest face (Yes in S40), the process proceeds to S50. The face detection unit 11 outputs the detection result to the face direction determination unit 12 and the pulse wave calculation unit 13. The detection result includes the image, position, and size of the detected facial area. If the face detection unit 11 does not detect any face (No in S40), the process returns to S20.

In S50, the face direction determination unit 12 determines whether or not the face direction of the largest face image included in the detection result is the first direction. If Yes in S50, proceed to S60. If No in S50, the process returns to S20.

In S60, the imaging unit 2 captures a moving image and acquires moving image data. The imaging unit 2 outputs moving image data to the pulse wave calculation unit 13.

In S70, the pulse wave calculation unit 13 calculates the pulse wave of the face detected by the face detection unit 11 based on the moving image data. The position of the face in the moving image is determined based on the information on the position of the face included in the detection result.

As described above, the biological information acquisition device 10 directs the pulse wave of the person whose face is the largest among the persons captured in the image (moving image) captured by the imaging unit 2 in the first direction. You can get it when you are.

Generally, when a person's face is photographed in a large size, the number of pixels that can be used for calculating the pulse wave increases, and as a result, it becomes easy to acquire a pulse wave having a high signal-to-noise ratio. The terminal device 1 acquires the pulse wave of the person with the largest face among one or more people included in the image captured by the image pickup unit 2. As a result, the terminal device 1 can acquire a highly accurate pulse wave.

When a user of the terminal device 1 tries to acquire his / her own pulse wave, the user is likely to be closest to the image pickup unit 2. That is, the closer the person is to the terminal device 1, the larger the image is likely to appear in the moving image. Therefore, the terminal device 1 makes it easy to acquire the pulse wave of the user.

The terminal device 1 acquires the pulse wave of the person who appears in the moving image in the largest direction when the person faces the first direction. As a result, the terminal device 1 can acquire a pulse wave with higher accuracy.

The terminal device 1 may have the following configuration.

The face detection unit 11 can detect a face by using a known technique. Known techniques include, for example, machine learning such as pattern matching.

When the imaging unit 2 outputs the image data to the face direction determination unit 12, the face detection unit 11 does not have to include the image of the face region detected by the face detection unit 11 in the detection result. The face direction determination unit 12 can acquire an image of the largest face region based on the position of the face region included in the detection result.

The face direction determination unit 12 outputs the determination result to the pulse wave calculation unit 13. The determination result may include information indicating the position of the face included in the detection result output from the face detection unit 11 to the face direction determination unit 12. In this case, the face detection unit 11 does not have to output the detection result to the pulse wave calculation unit 13. The pulse wave calculation unit 13 can determine the position of the face for which the pulse wave is calculated by searching for the face image included in the detection result from the moving image data output from the imaging unit 2.

S60 may be performed for a predetermined time (for example, 10 seconds) set in advance. The face detection unit 11 may detect the face appearing in the image in the moving image at the same time as the imaging unit 2 captures the moving image. The face detection unit 11 may track the detected face. The imaging unit 2 continues to capture the moving image while the face is shown in the image in the moving image, and when the face detecting unit 11 stops detecting the face, or the face detecting unit 11 cannot track the face. At that time, the imaging of the moving image may be finished.

The pulse wave calculation unit 13 may calculate the pulse wave by a known technique (independent component analysis, dye component separation, etc.) based on the time change of the pixel value of the moving image data. The pulse wave calculation unit 13 acquires the pulse wave from the portion of the face detected by the face detection unit 11 where the skin is mainly reflected. The pulse wave calculation unit 13 may calculate the pulse wave after the imaging of the moving image by the imaging unit 2 is completed. The pulse wave calculation unit 13 may calculate the pulse wave at the same time as the acquisition of the moving image data is started. When the determination information output from the face direction determination unit 12 to the pulse wave calculation unit 13 includes information on the face orientation, the pulse wave calculation unit 13 measures the pulse wave according to the direction of the face. You may change parts of the body.

[Embodiment 2]
The second embodiment will be described below. For convenience of explanation, the members having the same functions as the members described in the first embodiment are designated by the same reference numerals, and the description thereof will not be repeated.

Another method of acquiring a pulse wave by the terminal device 1 according to the present embodiment will be described with reference to FIG. FIG. 10 is a flow of acquiring a pulse wave by the terminal device 1 according to the second embodiment. Since S110 to S160 are the same as S10 to S60 in FIG. 9, the description thereof will be omitted.

In S170, the imaging unit 2 outputs the image data included in the moving image to the face detection unit 11 while acquiring the moving image data. The imaging unit 2 may output all the image data included in the moving image to the face detection unit 11. The image pickup unit 2 may output image data to the face detection unit 11 at predetermined intervals set in advance. The predetermined interval may be based on the time (for example, every 1 second) or the number of sheets (for example, every 30 sheets). The image pickup unit 2 may output the image data to the face detection unit 11 each time the image data to be output to the face detection unit 11 is acquired. The imaging unit 2 may collectively output the image data to the face detection unit 11 every time an arbitrary number of image data to be output to the face detection unit 11 is accumulated.

In S180, the face detection unit 11 detects the largest face in the image captured by the image pickup unit 2. When the face detection unit 11 detects the largest face (Yes in S190), the process proceeds to S200. At this time, the face detection unit 11 outputs the detection result to the face direction determination unit 12. When the face detection unit 11 does not detect any face (No in S190), the process returns to S120. As a result, the imaging unit 2 stops (or interrupts) the acquisition of moving image data.

In S200, the face direction determination unit 12 determines whether the largest face in the moving image acquired by the imaging unit 2 is facing the first direction based on the detection result. If Yes in S200, proceed to S210. The face direction determination unit 12 outputs the determination result to the pulse wave calculation unit 13. If No in S200, the process returns to S120. The imaging unit 2 interrupts the acquisition of moving image data. The above operation is continued until the imaging unit 2 finishes acquiring the moving image data (Yes in S210).

In S220, the pulse wave calculation unit 13 calculates the pulse wave of the face detected by the face detection unit 11 from the moving image captured by the image pickup unit 2.

When the imaging unit 2 interrupts the acquisition of moving image data, the user may be notified to that effect. When the imaging unit 2 interrupts the acquisition of moving image data, the imaging unit 2 reacquires the image data, the face detection unit 11 detects the largest face, and the face direction determination unit 12 determines the direction of the face. Good. When the imaging unit 2 interrupts the acquisition of moving image data, the biological information acquisition device 10 may ask the user whether or not to acquire the pulse wave. When the imaging unit 2 interrupts the acquisition of the moving image data, the pulse wave calculation unit 13 may calculate the pulse wave from the moving images acquired so far.

According to the above configuration, while the imaging unit 2 is acquiring the moving image data, the face for which the pulse wave is acquired is the largest in the image, and the face is facing the first direction. Guaranteed. Therefore, if the face of the person faces the second direction while the imaging unit 2 is acquiring the moving image data, the pulse wave calculation unit 13 does not calculate the pulse wave. As a result, the biological information acquisition device 10 can acquire a pulse wave with high accuracy.

Further, in response to the fact that a person (second person (second subject)) different from the person who wants to acquire the pulse wave (first person) is detected by the face detection unit 11, the imaging unit 2 performs moving image data. When the acquisition is started, the face detection unit 11 detects the first person while the imaging unit 2 is acquiring the moving image data. In response to the fact that the face detection unit 11 has detected the first person, the pulse wave calculation unit 13 calculates the pulse wave of the first person. Such a configuration is also feasible. The configuration will be described below.

When the first person wears a shield such as a mask or sunglasses, the face detection unit 11 may not be able to detect the face of the first person. In such a case, and when the second person is shown in the image, the face detection unit 11 detects the face of the second person. Then, when the face direction determination unit 12 determines that the second person is facing the first direction, the imaging unit 2 starts acquiring moving image data, and the pulse wave calculation unit 13 determines the pulse of the second person. Calculate the wave.

In the first embodiment, the pulse wave calculation unit 13 can continue to calculate the pulse wave of the second person. In the second embodiment, when the first person removes the shield and the person with the largest face is switched from the second person to the first person, the face detection unit 11 detects the first person. As a result, the pulse wave calculation unit 13 acquires the pulse wave of the first person.

It may be possible to set in advance whether or not to operate the face detection unit 11 and the face direction determination unit 12 while the imaging unit 2 is acquiring moving image data.

For example, a second person may interrupt near the imaging unit 2 while the pulse wave calculation unit 13 is calculating the pulse wave of the first person. At this time, the person with the largest face may switch from the first person to the second person. In this case, the imaging unit 2 interrupts the acquisition of moving image data, and the pulse wave calculation unit 13 stops the calculation of the pulse wave of the first person. At this time, if the operation of at least one of the face detection unit 11 and the face direction determination unit 12 is set in advance for the image in the moving image, the imaging unit 2 acquires the moving image data. The pulse wave calculation unit 13 can calculate the pulse wave of the first person.

As described above, the biological information acquisition device 10 can be realized with various configurations.

[Embodiment 3]
The third embodiment will be described below with reference to FIGS. 11 and 12. For convenience of explanation, the members having the same functions as the members described in the first and second embodiments are designated by the same reference numerals, and the description thereof will not be repeated.

FIG. 11 is a configuration diagram of a main part of the terminal device 15 provided with the biological information acquisition device 10 according to the present embodiment. The terminal device 15 includes an imaging unit 2, a biological information acquisition device 10, a notification unit 5, and a memory 6.

The notification unit 5 gives a notification based on the determination result of the face direction determination unit 12. The notification unit 5 includes, for example, (1) a display (for example, a liquid crystal screen or an organic EL screen), (2) a sound generator (for example, a speaker), (3) a light generator (for example, an LED light), and (4). A vibration generator (for example, a vibrator). The notification unit 5 may be configured by the combination of the above (1) to (4).

It is conceivable that the user of the terminal device 15 cannot grasp the first direction. In such a case, the notification unit 5 can notify the user of the first direction by using at least one of the above (1) to (4).

It is also possible that the user cannot see the display of the terminal device 15 when facing the first direction. In such a case, the user cannot grasp through the display whether the image is facing the first direction or whether the image capturing unit 2 is capturing an image (moving image). In such a case, the notification unit 5 can notify the user of the situation in which the user is placed, for example, by the methods (2) to (4) above.

For example, when the user is facing the second direction, if the user forcibly looks at the display, the pulse wave calculation unit 13 may not be able to accurately calculate the user's pulse wave. Therefore, the notification unit 5 notifies by sound that the acquisition of the moving image is completed at the timing when the imaging unit 2 completes the acquisition of the moving image. As a result, the user can continue to face the first direction while the imaging unit 2 is capturing the moving image.

The memory 6 records an image (moving image) captured by the imaging unit 2, information for identifying a user, and the like. As a matter of course, the memory 6 may be included in the terminal device 1 according to the first and second embodiments. The memory 6 may be provided outside the terminal device 1 instead of inside the terminal device 1.

FIG. 12 is an example of an external view of the terminal device 15 according to the present disclosure. The terminal device 15 includes a front camera unit 2a, a sound generation unit 5a, and a display 7.

The front camera unit 2a is used as an imaging unit 2 to acquire an image of a person who is a subject.

The display 7 displays an image (moving image) captured by the imaging unit 2, a pulse wave calculated by the pulse wave calculation unit 13, and the like.

The sound generation unit 5a is used as the notification unit 5 and performs notification based on the determination result of the face direction determination unit 12. The sound generation unit 5a notifies the user of the determination result of the face direction determination unit 12 by sound. For example, the sound generation unit 5a notifies by sound that the acquisition of the moving image has started in response to the determination result by the face direction determination unit 12 that the user's face is facing the first direction. Further, the sound generation unit 5a continues to sound the sound in response to the determination result by the face direction determination unit 12 that the user's face is not facing the first direction. In this case, based on the detection result of the face detection unit 11, the sound generation unit 5a may continue to make a sound even when the user's face is not detected.

The display 7 may perform the above notification in place of the sound generating unit 5a or together with the sound generating unit 5a. For example, the display 7 displays the first direction. As a result, the user can easily grasp the first direction.

In this way, the notification unit 5 can give various notifications to the user in various ways. The notification unit 5 may notify the user at the timing when the imaging unit 2 completes the acquisition of the moving image or when the pulse wave calculation unit 13 finishes the calculation of the pulse wave.

In this way, the notification unit 5 can notify the user of the operating state of at least one of the imaging unit 2, the face detection unit 11, the face direction determination unit 12, and the pulse wave calculation unit 13.

The biometric information acquisition device 10 has been described as acquiring biometric information of a person. However, the biometric information acquisition device 10 may acquire biometric information other than humans by the same method. Therefore, the "subject" may include animals other than humans.

[Example of realization by software]
The control block (particularly the face detection unit 11, the face direction determination unit 12, and the pulse wave calculation unit 13) of the biological information acquisition device 10 is realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like. It may be realized by software.

In the latter case, the biometric information acquisition device 10 includes a computer that executes instructions of a program that is software that realizes each function. This computer includes, for example, at least one processor (control device) and at least one computer-readable recording medium that stores the program. Then, in the computer, the processor reads the program from the recording medium and executes it, thereby achieving the object of the present disclosure. As the processor, for example, a CPU (Central Processing Unit) can be used. As the recording medium, a "non-temporary tangible medium", for example, a ROM (Read Only Memory) or the like, a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, or the like can be used. Further, a RAM (Random Access Memory) for expanding the above program may be further provided. Further, the program may be supplied to the computer via an arbitrary transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. It should be noted that one aspect of the present disclosure can also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

The present disclosure is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims, and the embodiments obtained by appropriately combining the technical means disclosed in the different embodiments. Is also included in the technical scope of the present disclosure. Furthermore, new technical features can be formed by combining the technical means disclosed in each embodiment.

1, 15 Terminal device 2 Imaging unit 2a Front camera unit 5 Notification unit 5a Sound generation unit 6 Memory 7 Display 10 Biometric information acquisition device 11 Face detection unit 12 Face direction determination unit 13 Pulse wave calculation unit (calculation unit)

Claims (15)

It is a biometric information acquisition device that acquires biometric information.
A face detection unit that detects the face of the first subject, which is the largest of the faces of one or more subjects included in the first image captured by the imaging unit, and a face detection unit.
A face direction determination unit for determining whether the face of the first subject detected by the face detection unit is facing the first direction, which is a direction suitable for acquiring the biological information, and a face direction determination unit.
From the second image different from the first image captured by the imaging unit when the face direction determination unit determines that the face of the first subject is facing the first direction. A biological information acquisition device including a calculation unit for calculating biological information of the first subject. When the face of the first subject is facing a second direction different from the first direction while the imaging unit is continuously capturing the second image, the face direction determination unit determines. The biometric information acquisition device according to claim 1, which stops the imaging of the second image by the imaging unit when the determination is made. When the face detection unit detects that the face of the second subject different from the first subject included in the second image is the largest face, the second image by the imaging unit The biometric information acquisition device according to claim 1 or 2, which stops continuous imaging of the image. Any one of claims 1 to 3, wherein the calculation unit calculates the biological information at the same time as the start of imaging of the second image by the imaging unit or after the completion of imaging of the second image by the imaging unit. The biometric information acquisition device according to item 1. The aspect according to any one of claims 1 to 4, wherein at least one of the face detection unit and the face direction determination unit stops operating while the image pickup unit is capturing the second image. Biometric information acquisition device. The calculation unit according to any one of claims 1 to 5, wherein the calculation unit changes the body part of the first subject to acquire the biological information according to the orientation of the face of the first subject. Biological information acquisition device. The first direction is any one of a direction facing the image pickup unit, a horizontal angle about the vertical direction of the image, and a vertical angle about the horizontal direction of the image. The biometric information acquisition device according to any one of claims 1 to 6, which is defined by the above. The first direction is (1) the type of the biological information, (2) the body part of the first subject from which the biological information is acquired, and (3) the subject from which the biological information is acquired. The biometric information acquisition device according to any one of claims 1 to 7, which is changed according to at least one of them. The biometric information acquisition device according to any one of claims 1 to 8, wherein the first direction is associated with the first subject in advance. The biometric information acquisition device according to any one of claims 1 to 9, wherein the biometric information includes at least one of pulse wave, pulse rate, and blood pressure. A terminal device that acquires biological information
An imaging unit that captures the first image and the second image,
A face detection unit that detects the face of the first subject, which is the largest of the faces of one or more subjects included in the first image, and
A face direction determination unit for determining whether the face of the first subject detected by the face detection unit is facing the first direction suitable for acquiring biometric information, and a face direction determination unit.
When the face direction determination unit determines that the face of the first subject is facing the first direction, a calculation unit that calculates biological information of the first subject from the second image, and a calculation unit. Terminal device equipped with. The terminal device according to claim 11, further comprising a notification unit that notifies the user of at least one of the imaging unit, the face detection unit, the face direction determination unit, and the calculation unit. It is a biometric information acquisition method for acquiring biometric information.
A detection step for detecting the face of the first subject, which is the largest of the faces of one or more subjects included in the first image captured by the imaging unit, and
A determination step for determining whether the face of the first subject detected in the detection step is facing the first direction suitable for acquiring the biological information, and a determination step.
From the second image different from the first image captured by the imaging unit when it is determined in the determination step that the face of the first subject is facing the first direction. A method for acquiring biological information including the calculation step of calculating the biological information of the first subject. A biometric information acquisition program for operating a computer as the biometric information acquisition device according to any one of claims 1 to 10, wherein the computer is used as the face detection unit, the face direction determination unit, and the calculation unit. A biometric information acquisition program to make it work. A computer-readable recording medium on which the biometric information acquisition program according to claim 14 is recorded.

Images