CN112971712A - Biological information acquisition device, terminal device, acquisition method, and recording medium - Google Patents
Biological information acquisition device, terminal device, acquisition method, and recording medium Download PDFInfo
- Publication number
- CN112971712A CN112971712A CN202011455886.0A CN202011455886A CN112971712A CN 112971712 A CN112971712 A CN 112971712A CN 202011455886 A CN202011455886 A CN 202011455886A CN 112971712 A CN112971712 A CN 112971712A
- Authority
- CN
- China
- Prior art keywords
- face
- unit
- image
- biological information
- pulse wave
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims description 37
- 238000001514 detection method Methods 0.000 claims abstract description 120
- 238000004364 calculation method Methods 0.000 claims abstract description 55
- 238000003384 imaging method Methods 0.000 claims description 44
- 230000036772 blood pressure Effects 0.000 claims description 4
- 210000000887 face Anatomy 0.000 description 19
- 238000007796 conventional method Methods 0.000 description 5
- 238000005259 measurement Methods 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 210000004204 blood vessel Anatomy 0.000 description 2
- 210000003128 head Anatomy 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 206010005746 Blood pressure fluctuation Diseases 0.000 description 1
- 241000282412 Homo Species 0.000 description 1
- 241001465754 Metazoa Species 0.000 description 1
- 208000004301 Sinus Arrhythmia Diseases 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000012880 independent component analysis Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 230000000241 respiratory effect Effects 0.000 description 1
- 230000036387 respiratory rate Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0033—Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0077—Devices for viewing the surface of the body, e.g. camera, magnifying lens
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Veterinary Medicine (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Public Health (AREA)
- Physics & Mathematics (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Cardiology (AREA)
- Physiology (AREA)
- Measuring Pulse, Heart Rate, Blood Pressure Or Blood Flow (AREA)
- Image Analysis (AREA)
- Exposure Control For Cameras (AREA)
Abstract
A biological information acquisition device (10) is provided with: a face detection unit (11) that detects the face of the first person that is most likely to be captured; a face direction determination unit (12) that determines whether the face of a first person is oriented in a first direction; and a pulse wave calculation unit (13) that calculates biological information of the first person when the face direction determination unit (12) has determined that the face of the first person is oriented in the first direction.
Description
Technical Field
The present disclosure relates to a biological information acquisition device, a terminal device, a biological information acquisition method, and a computer-readable recording medium that can acquire biological information.
Background
A technique is known in which a plurality of faces are detected in an image captured by a terminal device, and a person is selected from the plurality of faces. Jp 2005-049854 a discloses a technique of comparing the sizes of images of a plurality of faces and determining an object based on the face with the largest size. Japanese patent laid-open nos. 2007 and 6033 and 2009 and 77266 also disclose techniques for selecting one face from a plurality of faces in an image.
Disclosure of Invention
However, the above-described conventional techniques have the following problems.
In recent years, a technique of analyzing a moving image of a face of a person photographed by a camera to calculate a pulse wave of the person has been developed. Applications using such a technique are widely used in camera-equipped mobile terminals (smart phones and the like). In principle, when faces of a plurality of persons appear in a moving image, the pulse wave of the plurality of persons can be measured.
However, if the pulse waves of a plurality of persons are to be calculated simultaneously, a large burden is imposed on the image processing capability (calculation capability) of the portable terminal. Therefore, in order to accurately acquire the pulse wave in real time, a method of selecting one pulse wave among a plurality of persons and acquiring the pulse wave of the selected one pulse wave may be considered. Patent documents 1 to 3 disclose techniques for selecting one face from a plurality of faces in an image. On the other hand, patent documents 1 to 3 do not disclose a technique for calculating a pulse wave from a face in an image.
In general, when the face of a person is imaged to be large, the number of pixels that can be used for calculation of the pulse wave increases, and as a result, the pulse wave with a high signal-to-noise ratio is easily obtained.
However, even if the face of the person to be photographed is large, a pulse wave with high accuracy cannot necessarily be obtained. No prior art document indicating this point as a subject is found.
An object of an aspect of the present disclosure is to provide a biological information acquisition device, a terminal device, a biological information acquisition method, and a computer-readable recording medium that can acquire biological information of a measurement target from an image including an image of the measurement target's biological body with higher accuracy than in the related art.
In order to solve the above problem, a biometric information acquisition apparatus according to an aspect of the present disclosure acquires biometric information, and includes: a face detection unit that detects a face of a first subject that is captured to the maximum among faces of 1 or more subjects included in the first image captured by the imaging unit; a face direction determination unit configured to determine whether or not the face of the first object detected by the face detection unit is oriented in a first direction that is a direction suitable for acquiring the biometric information; and a calculation unit that calculates biometric information of the first object from a second image different from the first image captured by the imaging unit when the face direction determination unit determines that the face of the first object is oriented in the first direction.
In order to solve the above problem, a terminal device according to an aspect of the present disclosure acquires biometric information, and includes: a photographing unit for photographing a first image and a second image; a face detection unit that detects a face of a first subject that is captured to the maximum among faces of subjects that are 1 or more included in the first image; a face direction determination unit configured to determine whether or not the face of the first object detected by the face detection unit is oriented in a first direction suitable for acquisition of the biometric information; and a calculation unit that calculates biometric information of the first object from the second image when the face direction determination unit determines that the face of the first object is oriented in the first direction.
In order to solve the above problem, a biometric information acquisition method according to an aspect of the present disclosure acquires biometric information, and includes: a detection step of detecting a face of a first subject that is captured to the maximum among faces of subjects that are 1 or more included in a first image captured by a capturing unit; a determination step of determining whether or not the face of the first subject detected in the detection step is oriented in a first direction suitable for acquisition of the biometric information; and a calculation step of calculating the biological information of the first object from a second image different from the first image photographed by the photographing unit when the determination step determines that the face of the first object is directed in the first direction.
According to an aspect of the present disclosure, it is possible to calculate biological information of a measurement target from an image including an image of a biological body of the measurement target with higher accuracy than before.
Drawings
Fig. 1 is a main part configuration diagram of a terminal device including a biological information acquisition unit according to the present disclosure.
Fig. 2 shows a case where the person who has acquired the pulse wave faces the second direction and continues to acquire the pulse wave of the person.
Fig. 3 shows a case where the person who acquired the pulse wave faces the second direction and the acquisition of the pulse wave by the person is interrupted.
Fig. 4 shows an example of an image photographed by the photographing unit.
Fig. 5 shows an example of an image in which the face detection unit performs face detection.
Fig. 6 shows an example of the detection result output by the face detection unit.
Fig. 7 shows an example of an image in which the face detection unit performs face detection.
Fig. 8 shows an example of the determination result by the face direction determination unit.
Fig. 9 is a flow of acquiring a pulse wave by the terminal device according to the present disclosure.
Fig. 10 is a flow of acquiring a pulse wave by a terminal device according to another embodiment.
Fig. 11 is a main part configuration diagram of a terminal device including a terminal device according to another embodiment.
Fig. 12 is an example of an external view showing a terminal device according to another embodiment.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described based on the drawings. The embodiments described below are examples of a method for realizing the present disclosure, and the present disclosure should be appropriately modified or changed depending on the configuration of an apparatus to which the present disclosure is applied and various conditions, and the present disclosure is not limited to the embodiments described below. In the drawings described below, the same or functionally similar components are denoted by the same reference numerals, and redundant description thereof is omitted. The drawings are shown in a manner that will be readily understood when they are referred to in conjunction with the following description, and are not necessarily drawn to scale at a certain ratio.
[ operation (outline) of biological information acquisition device 10 ]
First, the operation of the biological information acquisition apparatus 10 according to the present disclosure will be described with reference to fig. 2 and 3. The biological information acquisition device 10 is a device for acquiring biological information of a person from a photographed image of the person. The biological information includes pulse wave, pulse rate, blood pressure, or the like. The biological information may include heart beat interval (RRI), Respiratory Sinus Arrhythmia (RSA), Respiratory rate, and blood pressure fluctuation (MWSA). In the following description, the biological information is a pulse wave.
In fig. 2 and 3, when a plurality of persons are detected in the screen, the sizes of the images of the plurality of faces are compared, and the pulse rate of the person of the largest face is obtained.
Fig. 2 shows a case where the person who has acquired the pulse wave faces the second direction and continues to acquire the pulse wave of the person. Fig. 2 corresponds to a case where the biological information acquisition device 10 is not used.
At 110 in fig. 2, two people are detected within the frame. The pulse of the preceding person (hereinafter referred to as "first person" (first object)) is acquired. The first person is oriented in a first direction. The first direction is a direction suitable for acquiring a pulse wave.
In 120 of fig. 2, the first person is facing a second direction. The second direction is a different direction than the first direction. Fig. 2 shows a case 120 where the pulse wave of the first human being is continuously acquired even when the first human being is oriented in the second direction. Since the first person is directed in the second direction, the accuracy of the acquired pulse wave of the first person may be low.
Fig. 3 shows a case where the person who has acquired the pulse wave faces the second direction, and the acquisition of the pulse wave of the person is interrupted. Fig. 3 corresponds to the case where the biological information acquisition device 10 is used. Since 130 in fig. 3 represents the same situation as 110 in fig. 2, the description thereof will be omitted.
In 140 of fig. 3, the first person is facing a second direction. The second direction is not a direction suitable for acquiring a pulse wave. Therefore, the biological information acquisition device 10 interrupts the acquisition of the pulse of the first person. This enables the biological information acquisition device 10 to acquire the pulse of the first person with higher accuracy.
The details of the biological information acquisition device 10 and the terminal device 1 including the biological information acquisition device 10 will be described below with reference to fig. 1 and the like.
[ configurations of terminal device 1 and biological information acquisition device 10 ]
Fig. 1 is a main part configuration diagram of a terminal device 1 including a biological information acquisition device 10 of the present disclosure. The terminal device 1 is a terminal device capable of acquiring (calculating) biological information of a person from a photographed image (moving image) of the face of the person. The terminal device 1 may be a smartphone, a tablet computer, a notebook PC, a dedicated pulse wave measurement terminal, or the like. The biological information includes pulse wave, pulse rate, blood pressure, or the like. In the following description, the biological information is a pulse wave.
The terminal device 1 includes an imaging unit 2 and a biological information acquisition device 10.
The image pickup unit 2 picks up an image and outputs image data of the image to the face detection unit 11. The imaging unit 2 images a moving image, and outputs moving image data of the moving image to the face detection unit 11 and the pulse wave calculation unit 13 (calculation unit). The imaging unit 2 may output the image data and/or the moving image data to the face detection unit 11, the face direction determination unit 12, and the pulse wave calculation unit 13. Instead of shooting a moving image, the shooting unit 2 may continuously shoot a plurality of images.
The photographing section 2 is configured by a photographing device such as an RGB camera or an infrared camera. The imaging unit 2 may be any imaging device capable of calculating a pulse wave based on moving image data output from the imaging unit 2.
The biological information acquisition device 10 is a device for acquiring biological information (pulse wave and the like) of a person from a moving image of the person photographed. The biological information acquisition device 10 includes a face detection unit 11, a face direction determination unit 12, and a pulse wave calculation unit 13 (calculation unit).
The face detection unit 11 detects the largest face person among the images captured by the imaging unit 2 and the moving images, and outputs the detection result to the face direction determination unit 12 and the pulse wave calculation unit 13. The detection result includes, for example, information on (1) an image of the region of the face detected by the face detection unit 11, (2) the position of the region, and (3) the size of the region. When only one person is captured in the image and the moving image captured by the imaging unit 2, the face detection unit 11 outputs the detection result of the face of the person to the face direction determination unit 12 and the pulse wave calculation unit 13.
The face direction determination unit 12 determines the direction of the face detected by the face detection unit 11 based on the detection result output by the face detection unit 11. The face direction determination unit 12 outputs the determination result to the pulse wave calculation unit 13. The determination result includes information on (1) whether the direction of the face determined by the face direction determination unit 12 is the first direction (the direction suitable for acquiring the pulse wave), and/or (2) the angle of the direction of the face determined by the face direction determination unit 12.
The pulse wave calculation unit 13 calculates the face pulse wave detected by the face detection unit 11 from the moving image data acquired by the imaging unit 2 based on the determination result output by the face direction determination unit 12. The calculated pulse wave may be displayed on a display or the like included in the terminal device 1 or may be stored in a memory or the like.
The pulse wave calculation unit 13 may calculate other biological information (pulse rate, blood pressure, or the like) based on the calculated pulse wave. The pulse wave calculation unit 13 may calculate other biological information based on a change in pixel value of the moving image data accompanying a change in blood vessel.
The face detection unit 11, the face direction determination unit 12, and the pulse wave calculation unit 13 may be configured by a calculation device such as a CPU, and may be provided inside the terminal device 1 or outside the cloud.
[ operation example of the face detection unit 11 ]
An operation example of the face detection unit 11 will be described with reference to fig. 4 to 7. Fig. 4 shows an example of an image captured by the imaging unit 2. Two faces are photographed in the picture. The imaging unit 2 outputs image data of the captured image to the face detection unit 11.
Fig. 5 shows an example of an image in which the face detection unit 11 performs face detection. The face detection unit 11 detects the face of one person (first person) with a larger face, out of two persons in the captured image. An image in which a frame is drawn in the area of the first person's face may be displayed on the display of the terminal device 1.
Fig. 6 shows an example of the detection result output by the face detection unit 11. The detection result includes information on (1) the image of the region of the face detected by the face detection unit 11, (2) the positions (170, 20) of the region, and (3) the size (100 × 100) of the region. The position of the region may be a center point of an image of the region of the face, or the like. An example of a method for obtaining such a detection result is shown below.
Fig. 7 shows an example of an image in which the face detection unit 11 performs face detection. First, the face detection unit 11 detects all of a plurality of faces in a captured image, and detects the position and size of each face. Next, the face detection unit 11 selects the largest face from all the detected faces. The face detection unit 11 outputs an image of a region including the selected face, the position of the region, and the detection result of the size of the region to the face direction determination unit 12. The face detection unit 11 may not acquire the positions of all detected faces. The face detection unit 11 may select the largest face and then acquire the size of the selected face.
Other examples for obtaining the same detection result are shown below.
There is a method similar to pattern matching, which performs face detection by searching for an area that coincides with a given template. In the case of this method, face detection is generally performed using a plurality of template sizes for scout search. The templates are used in order from a large-sized template. Thereby, the largest face can be detected at the beginning, and the face detection can be terminated even at a point of time when only one face is detected. According to this method, only the largest face is detected. This method can shorten the time required to detect the largest face compared to a method of detecting a plurality of faces.
Other examples for obtaining the same result are shown below.
Initially, a template having a certain size is prepared. Next, a plurality of images obtained by enlarging or reducing the image captured by the image capturing section 2 are prepared. The areas that coincide with the template are sequentially searched from the smallest image among the plurality of images. By using this method, the face detection unit 11 can first detect the largest face.
With the above-described various methods, the face detection unit 11 can detect the largest face in the moving image captured by the imaging unit 2, and output the detection result to the face direction determination unit 12 and the pulse wave calculation unit 13.
[ operation example of the face direction determination unit 12 ]
An operation example of the face direction determination unit 12 will be described with reference to fig. 8. Fig. 8 shows an example of the determination result by the face direction determination unit 12. The face direction determination unit 12 determines the direction of the face detected by the face detection unit 11 based on the detection result output by the face detection unit 11. The determination result includes information on (1) whether the direction of the face determined by the face direction determination unit 12 is the first direction (the direction suitable for accurately acquiring the pulse wave), or (2) the angle of the direction of the face determined by the face direction determination unit 12.
The left face of fig. 8 is directed toward the image pickup unit 2. This direction is suitable for acquiring the pulse wave and is referred to as a first direction. The right face of fig. 8 is directed toward the non-aligned imaging unit 2. This direction is not suitable for acquiring the pulse wave and is referred to as a second direction. The first direction may be defined by the face direction determination unit 12. The following is a detailed description.
The first direction may include a range having a width, for example, a range that can be specified as ± 45 degrees in the horizontal direction from the directly opposite direction. The first direction may be defined by an angle indicating the direction of the face numerically, or may be defined by words such as "front", "oblique", and "lateral".
The first direction may be defined as an angle in a horizontal direction with a vertical direction of the image as an axis when the face is moved to the left and right. The first direction may be defined by an angle in a vertical direction with a horizontal direction of the image as an axis when the face is moved up and down. The first direction may be defined by an angle having a rotation direction of an axis in a direction perpendicular to an image plane when the head is tilted or when the terminal device 1 is tilted. The first direction may be defined by a combination of a plurality of angles, i.e., an angle in the horizontal direction, an angle in the vertical direction, and/or an angle in the rotational direction.
The first direction may be defined based on the type of pulse wave to be acquired, the position where the pulse wave is acquired, or the like. For example, in the case of obtaining the pulse waves of the left and right cheeks, the first direction may be a direction in which the face faces the terminal device 1. In the case of obtaining the pulse wave of the right cheek, the first direction may be a direction in which the face is turned in the horizontal direction from the direction facing the terminal device 1 and then the face is directed to the right side of the screen. In the case of obtaining the pulse wave of the neck, the first direction may be a direction in which the face is turned in a vertical direction from a direction facing the terminal device 1 and then faces upward on the screen. In the case of obtaining a pulse wave at the cheek in an environment where light is irradiated onto the face from directly above, the first direction may be a direction in which the face is rotated in a rotation direction from a direction directly facing the camera, and then the head is tilted or the body is tilted.
Since the distribution of blood vessels on the face varies among individuals, the face angle at which a pulse wave can be accurately obtained varies from person to person. Thus, the first direction may be specified for each person. In this case, the detection result output from the face detection unit 11 includes information for identifying an individual. The terminal device 1 can previously store faces of a plurality of persons in a memory, and can identify a person by comparing the face in the memory with a face detected from a face of a person. The face direction determination unit 12 defines the first direction based on information for identifying the person. That is, the first direction is pre-associated with the first person. The information for identifying an individual is represented by a name, characters, symbols, numerals, or the like of a person, and is a code (code) for identifying an individual.
The face direction determination unit 12 can specify (determine) the various first directions described above using a conventional method. For example, the face direction determination unit 12 may estimate (determine) the angle of the face based on the positional relationship of the organs (eyes, mouth, etc.) of the face. The face direction determination unit 12 may estimate (determine) the direction (angle) of the face from the matching result with the templates using various templates of the angle of the face.
[ method of calculating pulse wave ]
Next, a method for acquiring a pulse wave from the terminal device 1 will be described with reference to fig. 9. Fig. 9 is a flow of acquiring a pulse wave by the terminal device 1.
First, at S10, the image capturing unit 2 is activated.
Next, at S20, the image capturing unit 2 captures an image and acquires moving image data. The imaging unit 2 outputs the moving image data to the face detection unit 11.
In S30, the face detection unit 11 detects the largest face in the captured image captured by the imaging unit 2. If the face detection unit 11 detects the largest face (yes at S40), the process proceeds to S50. The face detection unit 11 outputs the detection result to the face direction determination unit 12 and the pulse wave calculation unit 13. The detection result includes the image, position, and size of the area of the detected face. If none of the faces is detected by the face detection unit 11 (no at S40), the process returns to S20.
In S50, the face direction determination unit 12 determines whether or not the direction of the face of the largest face image included in the detection result is the first direction. If yes at S50, the process proceeds to S60. If no at S50, the process returns to S20.
In S60, the photographing unit 2 photographs a moving image and acquires moving image data. The imaging unit 2 outputs the moving image data to the pulse wave calculation unit 13.
In S70, the pulse wave calculation unit 13 calculates the face pulse wave detected by the face detection unit 11 based on the moving image data. The position of the face in the moving image is determined based on information of the position of the face included in the detection result.
As described above, the biological information acquisition device 10 can acquire the pulse wave of the person whose face is the largest among the persons in the image (moving image) captured by the imaging unit 2 when the person is oriented in the first direction.
In general, when the face of a person is imaged largely, the number of pixels to be used for calculation of a pulse wave increases, and as a result, a pulse wave with a high signal-to-noise ratio is easily obtained. The terminal device 1 acquires the pulse wave of a person whose face is the largest among 1 or more persons included in the image captured by the image capturing section 2. Thus, the terminal device 1 can obtain a pulse wave with high accuracy.
When the user of the terminal device 1 wants to obtain his/her own pulse wave, the user is likely to be located at the position closest to the image capturing unit 2. That is, the closer the person is to the terminal device 1, the larger the person can be easily captured in the moving image. Therefore, the terminal device 1 can easily acquire the pulse wave of the user.
The terminal device 1 acquires the pulse wave of the person when the person imaged at the maximum in the moving image faces the first direction. Thus, the terminal device 1 can obtain a pulse wave with higher accuracy.
The terminal device 1 may have the following configuration.
The face detection unit 11 can detect a human face by using a conventional technique. As a conventional technique, for example, machine learning such as pattern matching is given.
When the image capturing unit 2 outputs the image data to the face direction determination unit 12, the face detection unit 11 may not include the image of the area of the face detected by the face detection unit 11 in the detection result. The face direction determination unit 12 can acquire an image of the area of the largest face based on the position of the area of the face included in the detection result.
The face direction determination unit 12 outputs the determination result to the pulse wave calculation unit 13. The determination result may include information indicating the position of the face included in the detection result output from the face detection unit 11 to the face direction determination unit 12. In this case, the face detection unit 11 may not output the detection result to the pulse wave calculation unit 13. The pulse wave calculation unit 13 can determine the position of the face at which the pulse wave is calculated by searching for the face image included in the detection result from the moving image data output from the imaging unit 2.
S60 may be performed within a predetermined time (e.g., 10 seconds). The face detection unit 11 may detect a face of an image captured in the moving image simultaneously with the capturing of the moving image by the imaging unit 2. The face detection unit 11 can track the detected face. The imaging unit 2 may continue to capture a moving image when a face is captured in an image in the moving image, and may end capturing a moving image when the face detection unit 11 does not detect a face or when the face detection unit 11 cannot track a face.
The pulse wave calculation unit 13 may calculate the pulse wave by a conventional technique (independent component analysis, separation of color elements, or the like) based on the temporal change of the pixel value of the moving image data. The pulse wave calculation unit 13 obtains a pulse wave from a region of the face detected by the face detection unit 11, the region being mainly imaged on the skin. The pulse wave calculation unit 13 may calculate the pulse wave after the imaging of the moving image by the imaging unit 2 is completed. The pulse wave calculation unit 13 may calculate the pulse wave simultaneously with the start of the acquisition of the moving image data. When the information on the direction of the face is included in the determination information output from the face direction determination unit 12 to the pulse wave calculation unit 13, the pulse wave calculation unit 13 may change the body part to which the pulse wave is to be measured, in accordance with the direction of the face.
[ embodiment 2]
Another method for obtaining a pulse wave by the terminal device 1 according to the present embodiment will be described with reference to fig. 10. Fig. 10 is a flow of acquiring a pulse wave by the terminal device 1 according to embodiment 2. Since S110 to S160 are the same as S10 to S60 of fig. 9, description thereof is omitted.
At S170, the image capturing unit 2 acquires moving image data and outputs image data included in the moving image to the face detection unit 11. The image capturing unit 2 may output all image data included in the moving image to the face detection unit 11. The image capturing unit 2 may output image data to the face detection unit 11 at predetermined intervals. The predetermined interval may be based on time (e.g., every second), or number of sheets (e.g., every 30 sheets). The image capturing unit 2 may output the image data to the face detection unit 11 every time the image data output to the face detection unit 11 is acquired. The image capturing unit 2 may output the image data output to the face detection unit 11 together every time an arbitrary number of pieces of image data are accumulated in the face detection unit 11.
In S180, the face detection unit 11 detects the largest face in the captured image captured by the imaging unit 2. When the face detection unit 11 detects the largest face (yes at S190), the process proceeds to S200. At this time, the face detection unit 11 outputs the detection result to the face direction determination unit 12. If none of the faces is detected by the face detection unit 11 (no at S190), the process returns to S120. Thereby, the photographing section 2 stops (or interrupts) the acquisition of the moving image data.
In S200, the face direction determination unit 12 determines whether or not the largest face in the image captured by the imaging unit 2 is oriented in the first direction based on the detection result. If yes at S200, the process proceeds to S210. The face direction determination unit 12 outputs the determination result to the pulse wave calculation unit 13. If S200 is no, the process returns to S120. The image capturing unit 2 interrupts the acquisition of the moving image data. The above operation continues until the imaging unit 2 finishes acquiring the moving image data (yes in S210).
In S220, the pulse wave calculation unit 13 calculates the pulse wave of the face detected by the face detection unit 11 from the moving image captured by the imaging unit 2.
The photographing unit 2 may interrupt the acquisition of the moving image data and notify the user of the contents. When the image capturing unit 2 interrupts the acquisition of the moving image data, the image capturing unit 2 may acquire the image data again, the face detecting unit 11 may detect the largest face, and the face direction determining unit 12 may determine the direction of the face. When the imaging unit 2 interrupts the acquisition of the moving image data, the biological information acquiring apparatus 10 may ask the user whether or not to acquire the pulse wave. When the imaging section 2 interrupts the acquisition of the moving image data, the pulse wave calculation section 13 may calculate the pulse wave from the moving image acquired before the interruption.
With the above configuration, while the imaging unit 2 acquires moving image data, the face that acquired the pulse wave is captured in the image to the maximum extent, and the face is kept oriented in the first direction. Therefore, when the face of the person is oriented in the second direction while the image capturing section 2 is acquiring the moving image data, the pulse wave calculating section 13 does not calculate the pulse wave. Thus, the biological information acquisition device 10 can acquire a pulse wave with high accuracy.
Further, after a person (second object)) different from the person (first person) who intends to obtain the pulse wave is detected by the face detection unit 11, the imaging unit 2 starts to obtain the moving image data, and the face detection unit 11 detects the first person while the imaging unit 2 obtains the moving image data. After the face detection unit 11 detects the first person, the pulse wave calculation unit 13 calculates the pulse wave of the first person. Such a constitution can also be realized. This structure will be described below.
The face detection unit 11 may not detect the face of the first person when the first person wears a mask such as a mask or sunglasses. In this case, and the second person takes a picture of the image, the face detection section 11 detects the face of the second person. Then, the face direction determination unit 12 determines that the second person is facing the first direction, the image capturing unit 2 starts acquiring the moving image data, and the pulse wave calculation unit 13 calculates the pulse wave of the second person.
In embodiment 1, the pulse wave calculation unit 13 continuously calculates the pulse wave of the second person. In embodiment 2, when the first person removes the blocking object and the person whose face is the largest is switched from the second person to the first person, the face detection unit 11 detects the first person. Thereby, the pulse wave calculation unit 13 obtains the pulse wave of the first person.
Whether or not to operate the face detection unit 11 and the face direction determination unit 12 may be set in advance while the image capturing unit 2 is acquiring moving image data.
For example, the second person may be inserted near the imaging unit 2 while the pulse wave calculation unit 13 calculates the pulse wave of the first person. At this time, there is a possibility that the person whose face is most photographed may be switched from the first person to the second person. In this case, the image capturing section 2 interrupts the acquisition of the moving image data, and the pulse wave calculating section 13 stops the calculation of the pulse wave of the first person. At this time, in a case where the image in the moving image is set in advance so as to stop the operation of at least one of the face detection unit 11 and the face direction determination unit 12, the image capturing unit 2 can continue to acquire moving image data, and the pulse wave calculation unit 13 can calculate the pulse wave of the first person.
In this way, the biological information acquisition device 10 can be realized in various configurations.
[ embodiment 3]
Embodiment 3 will be described below with reference to fig. 11 and 12. For convenience of explanation, the same reference numerals are given to the members and members having the same functions as those described in embodiments 1 and 2, and the explanation thereof will not be repeated.
Fig. 11 is a main part configuration diagram of a terminal device 15 including the biological information acquisition device 10 according to the present embodiment. The terminal device 15 includes an imaging unit 2, a biological information acquisition device 10, a notification unit 5, and a memory 6.
The notification unit 5 performs notification based on the determination result of the face direction determination unit 12. The notification section 5 is, for example, (1) a display (e.g., a liquid crystal screen or an organic EL screen), (2) a sound generation device (e.g., a speaker), (3) a light generation device (e.g., an LED lamp), and (4) a vibration generation device (e.g., a vibrator). The notification unit 5 may be configured by a combination of the above (1) to (4).
Consider a case where the user of the terminal device 15 cannot grasp the first direction. In this case, the notification unit 5 can notify the user of the first direction using at least one of the above-described (1) to (4).
In addition, it is also conceivable that the user cannot see the display of the terminal device 15 when facing the first direction. In this case, the user cannot grasp whether the image capturing unit 2 captures an image (moving image) or not by the display device whether the image capturing unit is oriented in the first direction. In this case, the notification unit 5 can notify the user of the situation of the user by the methods (2) to (4), for example.
For example, when the user is facing the second direction, if the user tries to look at the display with difficulty, the pulse wave calculation unit 13 may not be able to accurately calculate the pulse wave of the user. Therefore, the notification unit 5 notifies the completion of the acquisition of the moving image by sound at the timing when the imaging unit 5 completes the acquisition of the moving image. Thus, the user can continue to face the first direction while the photographing section 2 photographs a moving image.
The memory 6 stores an image (moving image) photographed by the photographing section 2, information for identifying a user, and the like. The memory 6 may be included in the terminal device 1 according to embodiments 1 and 2, as a matter of course. The memory 6 may be provided not inside the terminal apparatus 1 but outside the terminal apparatus 1.
Fig. 12 is an example of an external view showing the terminal device 15 of the present disclosure. The terminal device 15 includes a front camera section 2a, a sound generation section 5a, and a display 7.
The front camera section 2a is used as the image capturing section 2, and acquires an image of a person as a subject.
The display 7 displays an image (moving image) captured by the imaging unit 2, a pulse wave calculated by the pulse wave calculating unit 13, and the like.
The sound generation unit 5a is used as the notification unit 5, and notifies based on the determination result of the face direction determination unit 12. The sound generation unit 5a notifies the user of the determination result of the face direction determination unit 12 by sound. For example, the sound generation unit 5a receives the determination result by the face direction determination unit 12 when the face of the user is oriented in the first direction, and starts to acquire a moving image with a sound notification. The sound generation unit 5a receives the determination result from the face direction determination unit 12 when the face of the user is not oriented in the first direction, and continues to generate sound. In this case, the sound generation unit 5a may continue to generate sound even when the face of the user is not detected, based on the detection result of the face detection unit 11.
The display 7 may perform the above-described notification in place of the sound generation unit 5a, or together with the sound generation unit 5 a. For example, the display 7 displays a first direction. Thereby, the user can easily grasp the first direction.
In this way, the notification unit 5 can give various notifications to the user in various ways. The notification unit 5 notifies the user of the contents of the moving image acquisition completion timing by the imaging unit 2 or the pulse wave calculation completion timing by the pulse wave calculation unit 13.
In this way, the notification unit 5 can notify the user of the operation state of at least one of the imaging unit 2, the face detection unit 11, the face direction determination unit 12, and the pulse wave calculation unit 13.
The biometric information acquisition device 10 is explained as acquiring biometric information of a person. However, the biological information acquisition device 10 may acquire biological information other than a person by the same method. Therefore, the "subject" may include animals other than humans.
[ implementation by software ]
The control modules (particularly, the face detection unit 11, the face direction determination unit 12, and the pulse wave calculation unit 13) of the biological information acquisition apparatus 10 may be implemented by logic circuits (hardware) formed by integrated circuits (IC chips) or the like, or may be implemented by software.
In the latter case, the biological information acquisition device 10 includes a computer that executes instructions of a program of software for realizing the respective functions. The computer includes, for example, one computer-readable recording medium including at least one processor (control device) and storing the program. Also, in the computer, the processor reads the program from the recording medium and executes the program, thereby achieving the object of the present disclosure. As the processor, for example, a cpu (central Processing unit) can be used. As the recording medium, a "non-transitory tangible medium" can be used, and examples thereof include a magnetic tape, a hard disk, a card, a semiconductor memory, a programmable logic circuit, and the like, in addition to a rom (read Only memory). The present invention may further include a ram (random Access memory) for developing the above-described program. The program may be supplied to the computer via an arbitrary transmission medium (a communication network, a broadcast wave, or the like) through which the program can be transmitted. An aspect of the present disclosure may be implemented as a data signal embedded in a carrier wave, the program being electronically transmitted.
The present disclosure is not limited to the above embodiments, and various modifications can be made within the scope shown in the claims, and embodiments obtained by appropriately combining technical means disclosed in different embodiments are also included in the technical scope of the present disclosure. Further, by combining the technical methods disclosed in the respective embodiments, new technical features can be formed.
Claims (14)
1. A biological information acquisition apparatus for acquiring biological information, comprising:
a face detection unit that detects a face of a first subject that is captured to the maximum among faces of one or more subjects included in the first image captured by the imaging unit;
a face direction determination unit configured to determine whether or not the face of the first object detected by the face detection unit is oriented in a first direction that is a direction suitable for acquiring the biometric information; and
and a calculation unit that calculates biometric information of the first object from a second image different from the first image captured by the imaging unit when the face direction determination unit determines that the face of the first object is oriented in the first direction.
2. The biological information acquisition apparatus according to claim 1,
the face direction determination unit determines to stop the photographing of the second image by the photographing unit when the face of the first object is oriented in a second direction different from the first direction while the photographing unit continuously photographs the second image.
3. The biological information acquisition apparatus according to claim 1 or 2,
when the face detection unit detects that a face of a second object different from the first object included in the second image is the largest face, the continuous imaging of the second image by the imaging unit is stopped.
4. The biological information acquisition apparatus according to any one of claims 1 to 3,
the calculation unit calculates the biological information at the same time as the start of the photographing of the second image by the photographing unit or after the end of the photographing of the second image by the photographing unit.
5. The biological information acquisition apparatus according to any one of claims 1 to 4,
at least one of the face detection unit and the face direction determination unit stops operating while the image capturing unit captures the second image.
6. The biological information acquisition apparatus according to any one of claims 1 to 5,
the calculation unit changes a body part of the first subject from which the biological information is acquired, in accordance with a direction of a face of the first subject.
7. The biological information acquisition apparatus according to any one of claims 1 to 6,
the first direction is defined by a direction facing the image pickup unit, an angle in a horizontal direction with a vertical direction of the image as an axis, and an angle in a vertical direction with a horizontal direction of the image as an axis.
8. The biological information acquisition apparatus according to any one of claims 1 to 7,
the first direction is changed according to at least one of (1) a type of the biological information, (2) a body part of the first subject from which the biological information is obtained, and (3) a subject from which the biological information is obtained.
9. The biological information acquisition apparatus according to any one of claims 1 to 8,
the first direction is pre-associated with the first subject.
10. The biological information acquisition apparatus according to any one of claims 1 to 9,
the biological information includes at least one of pulse wave, pulse rate, and blood pressure.
11. A terminal device for acquiring biometric information, comprising:
an imaging unit that images a first image and a second image;
a face detection unit that detects a face of a first subject that is captured to the maximum among faces of one or more subjects included in the first image;
a face direction determination unit configured to determine whether or not the face of the first object detected by the face detection unit is oriented in a first direction suitable for acquisition of the biometric information; and
and a calculation unit that calculates biometric information of the first object from the second image when the face direction determination unit determines that the face of the first object is oriented in the first direction.
12. The terminal apparatus according to claim 11, comprising:
and a notification unit configured to notify a user of an operation state of at least one of the image capturing unit, the face detection unit, the face direction determination unit, and the calculation unit.
13. A biometric information acquisition method for acquiring biometric information, comprising:
a detection step of detecting a face of a first subject that is captured maximally among faces of one or more subjects included in a first image captured by a capturing unit;
a determination step of determining whether or not the face of the first subject detected in the detection step is oriented in a first direction suitable for acquisition of the biometric information; and
a calculation step of calculating biological information of the first object from a second image different from the first image photographed by the photographing unit when the determination step determines that the face of the first object is directed in the first direction.
14. A computer-readable recording medium storing a biological information acquisition program for causing a computer to function as the biological information acquisition apparatus according to any one of claims 1 to 10,
a biological information acquisition program for causing a computer to function as the face detection unit, the face direction determination unit, and the calculation unit is stored.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2019225919A JP7443044B2 (en) | 2019-12-13 | 2019-12-13 | Biometric information acquisition device, terminal device, biometric information acquisition method, biometric information acquisition program, and computer-readable recording medium |
| JP2019-225919 | 2019-12-13 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112971712A true CN112971712A (en) | 2021-06-18 |
Family
ID=76344944
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011455886.0A Pending CN112971712A (en) | 2019-12-13 | 2020-12-10 | Biological information acquisition device, terminal device, acquisition method, and recording medium |
Country Status (2)
| Country | Link |
|---|---|
| JP (1) | JP7443044B2 (en) |
| CN (1) | CN112971712A (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP4303533A1 (en) | 2021-03-05 | 2024-01-10 | Toppan Inc. | Distance image capturing device and distance image capturing method |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005049854A (en) * | 2003-07-15 | 2005-02-24 | Omron Corp | Object decision device and imaging device |
| JP2014198201A (en) * | 2013-03-29 | 2014-10-23 | 富士通株式会社 | Pulse wave detection program, pulse wave detection method, and pulse wave detection device |
| CN105989357A (en) * | 2016-01-18 | 2016-10-05 | 合肥工业大学 | Human face video processing-based heart rate detection method |
| CN106377241A (en) * | 2016-08-30 | 2017-02-08 | 苏州品诺维新医疗科技有限公司 | Heartbeat detection system and method of acquiring heartbeat change curve |
| WO2017085895A1 (en) * | 2015-11-20 | 2017-05-26 | 富士通株式会社 | Information processing device, information processing method, and information processing program |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005218507A (en) | 2004-02-03 | 2005-08-18 | Tama Tlo Kk | Method and apparatus for measuring vital sign |
| JP4902460B2 (en) | 2007-08-08 | 2012-03-21 | キヤノン株式会社 | Image processing apparatus, imaging apparatus, and image processing method |
| CN114795151A (en) | 2016-06-30 | 2022-07-29 | 松下知识产权经营株式会社 | Method and system |
| JP6817782B2 (en) | 2016-10-31 | 2021-01-20 | 三星電子株式会社Ｓａｍｓｕｎｇ Ｅｌｅｃｔｒｏｎｉｃｓ Ｃｏ．，Ｌｔｄ． | Pulse detection device and pulse detection method |
-
2019
- 2019-12-13 JP JP2019225919A patent/JP7443044B2/en active Active
-
2020
- 2020-12-10 CN CN202011455886.0A patent/CN112971712A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005049854A (en) * | 2003-07-15 | 2005-02-24 | Omron Corp | Object decision device and imaging device |
| JP2014198201A (en) * | 2013-03-29 | 2014-10-23 | 富士通株式会社 | Pulse wave detection program, pulse wave detection method, and pulse wave detection device |
| WO2017085895A1 (en) * | 2015-11-20 | 2017-05-26 | 富士通株式会社 | Information processing device, information processing method, and information processing program |
| CN105989357A (en) * | 2016-01-18 | 2016-10-05 | 合肥工业大学 | Human face video processing-based heart rate detection method |
| CN106377241A (en) * | 2016-08-30 | 2017-02-08 | 苏州品诺维新医疗科技有限公司 | Heartbeat detection system and method of acquiring heartbeat change curve |
Also Published As
| Publication number | Publication date |
|---|---|
| JP7443044B2 (en) | 2024-03-05 |
| JP2021096537A (en) | 2021-06-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107949863B (en) | Authentication device and authentication method using biometric information | |
| US9697415B2 (en) | Recording medium, image processing method, and information terminal | |
| JP5949319B2 (en) | Gaze detection apparatus and gaze detection method | |
| US9750420B1 (en) | Facial feature selection for heart rate detection | |
| TWI380232B (en) | ||
| JP5538160B2 (en) | Pupil detection device and pupil detection method | |
| JP2006319610A (en) | Camera | |
| KR20100075167A (en) | Apparatus and method for capturing digital image using face recognition function | |
| CN109478227A (en) | Calculate the iris in equipment or the identification of other physical feelings | |
| KR102364929B1 (en) | Electronic device, sever, and system for tracking skin changes | |
| KR20150139183A (en) | Wrist-type wearable device for vein recognition | |
| JP7420281B2 (en) | Living body detection device, control method, and program | |
| CN112971712A (en) | Biological information acquisition device, terminal device, acquisition method, and recording medium | |
| JP2011022927A (en) | Hand image recognition device | |
| JPWO2022074865A5 (en) | LIFE DETECTION DEVICE, CONTROL METHOD, AND PROGRAM | |
| RU2709185C1 (en) | Image capturing | |
| JP2009205203A (en) | Iris authentication device | |
| JP2016110444A (en) | Eyeball identification apparatus and eyeball identification method | |
| JP5004099B2 (en) | Cursor movement control method and cursor movement control apparatus | |
| Caya et al. | Development of a wearable device for tracking eye movement using pupil lumination comparison algorithm | |
| JP3732881B2 (en) | Image feature point detection method and apparatus | |
| JP2001184483A (en) | Device and method for personal identification | |
| WO2023127623A1 (en) | Terminal device | |
| JP2000292852A (en) | Face picture photographing device | |
| JP5299632B2 (en) | Personal authentication device, program and recording medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |