WO2019241982A1

WO2019241982A1 - Device and method for acquiring biological information

Info

Publication number: WO2019241982A1
Application number: PCT/CN2018/092313
Authority: WO
Inventors: Sato Shohei
Original assignee: Huawei Technologies Co., Ltd.
Priority date: 2018-06-22
Filing date: 2018-06-22
Publication date: 2019-12-26
Also published as: JP2021528169A; CN112292072A

Abstract

A device (10) and a method for acquiring biological information are provided. The method for acquiring biological information includes: acquiring a first image of an artery part of a subject imaged in a contactless manner (S11); detecting individual pulse waves corresponding to a plurality of positions in the artery part based on the first image (S12); calculating an amount of shifting between the detected individual pulse waves (S13); and calculating indication information on an artery flow of the subject, as the biological information, by using the amount of shifting (S14).

Description

DEVICE AND METHOD FOR ACQUIRING BIOLOGICAL INFORMATION

Technical Field

The present disclosure relates to a device and method for acquiring a physological parameter about the cardiovascular system by imaging photo plethysmography (iPPG) .

Background Art

A device is known that acquires an image of a subject by imaging the subject, and calculates a pulse wavelength velocity from this image. For example, there is a publicly known device that images the face of a subject, identifies two different target regions in the facial image, and calculates a pulse wavelength velocity from the amount of shifting of pulse waves in both regions.

Summary of Invention

Under the above circumstances, a technical advantage is provided by an embodiment of the present disclosure describing a device and method for acquiring biological information.

A first aspect of an embodiment provides the following method.

A method for acquiring biological information, including:

acquiring a first image of an artery part of a subject imaged in a contactless manner;

detecting individual pulse waves corresponding to a plurality of positions in the artery part based on the first image;

calculating an amount of shifting between the detected individual pulse waves; and

calculating indication information on an artery flow of the subject, as the biological information, by using the amount of shifting.

According to the first aspect, highly reliable indication information is provided.

The first aspect of the embodiment further includes:

acquiring a second image including depth information of the artery part; and

calculating a distance between artery parts associated with the amount of shifting based on the depth information included in the second image,

wherein the step of calculating the indication information calculates the indication information by using the calculated distance between the artery parts and the amount of shifting.

According to this first aspect, the distance to an artery part is acquired to calculate indication information.

A second aspect of the embodiment provides the following device.

A device for acquiring biological information, including:

a first acquisition unit for acquiring a first image of an artery part of a subject imaged in a contactless manner;

a detection unit for detecting individual pulse waves corresponding to a plurality of positions in the artery part based on the first image;

a shift amount calculating unit for calculating an amount of shifting between the detected individual pulse waves; and

an indication information calculating unit for calculating indication information on an artery flow of the subject, as the biological information, by using the amount of shifting.

According to the second aspect, highly reliable indication information is provided.

The second aspect of the embodiment further includes:

a second acquisition unit for acquiring a second image including depth information of the artery part; and

a calculation unit for calculating a distance between artery parts associated with the amount of shifting based on the depth information included in the second image,

wherein the indication information calculating unit is configured to calculate the indication information by using the calculated distance between the artery parts and the amount of shifting.

According to this second aspect, the distance to an artery part is acquired to calculate indication information.

A third aspect of the embodiment provides the following device.

A device including: the device of the second aspect; a near-infrared camera configured to image the first image; and a depth sensor used to acquire the indication information, and acquire depth information of the plurality of artery parts.

According to the third aspect, highly reliable indication information is provided.

A fourth aspect of the embodiment provides a computer readable storage medium recording a program for allowing a computer to execute a first aspect of the method according to the embodiment.

According to the fourth aspect, highly reliable indication information is provided.

A fifth aspect of the embodiment provides a computer program for allowing a computer to execute the first aspect of the method according to the embodiment.

According to the fifth aspect, highly reliable indication information is provided.

Brief Description of Drawings

To describe the technical solutions in the embodiments more clearly, the following briefly describes the accompanying drawings required for describing the present embodiments. Apparently, the accompanying drawings in the following description depict merely some of the possible embodiments, and a person of ordinary skill in the art may still derive other drawings, without creative efforts, from these accompanying drawings, in which:

[Fig. 1] Fig. 1 is a schematic diagram showing an example of the configuration of a device according to an embodiment;

[Fig. 2A] Fig. 2A is a diagram for describing an aspect in a case where light from an NIR light source is reflected at a subject in the device according to the embodiment;

[Fig. 2B] Fig. 2B is a diagram for describing an aspect for acquiring an NIR image and a depth image in the device according to the embodiment;

[Fig. 3] Fig. 3 is a diagram for describing the relationship between the wavelength of irradiated light and the transmission depth thereof;

[Fig. 4] Fig. 4 is a diagram for describing the outline of an indication information calculating process, which is implemented by the device according to the embodiment;

[Fig. 5] Fig. 5 is a diagram for describing a distance between two regions of interest (ROIs) in the device according to the embodiment;

[Fig. 6A] Fig. 6A is a diagram for describing a pixel size in the device according to the embodiment;

[Fig. 6B] Fig. 6B is a diagram showing an example of the pixel size in Fig. 6A;

[Fig. 7] Fig. 7 is a diagram showing an example of the functional configuration of the device according to the embodiment;

[Fig. 8] Fig. 8 is a flowchart illustrating an example of an overall indication information calculating process in the device according to the embodiment;

[Fig. 9] Fig. 9 is a flowchart illustrating an example of a process which is executed to calculate a distance between artery parts as detection targets in the device according to the embodiment;

[Fig. 10] Fig. 10 is a flowchart illustrating the calculation process of step S22 in Fig. 9;

[Fig. 11] Fig. 11 is a diagram for describing a separation process of step S221 in Fig. 10;

[Fig. 12] Fig. 12 is a diagram for describing an aspect in which the locations of artery parts are estimated to calculate a distance between the artery parts; and

[Fig. 13] Fig. 13 is a diagram showing a detection range in a case where a radial artery is detected.

Description of Embodiments

The following clearly describes technical solutions of the embodiments of the present disclosure with reference to the accompanying drawings of the embodiments of the present disclosure. Apparently, described embodiments are not all but just some of the embodiments of the present disclosure. It is to be noted that all other embodiments which may be obtained by a person skilled in the art based on the embodiments of the present disclosure without creative efforts shall fall within the protection scope of embodiments of the present disclosure.

A device 10 of the embodiment is configured to obtain biological information of a subject (i.e., a physological parameter about the cardiovascular system from a video image of the subject which is imaged in a contactless manner.

[Configuration of Device 10]

Fig. 1 is a schematic diagram showing an example of the hardware configuration of the device 10 according to an embodiment.

As shown in Fig. 1, the device 10 includes a processing unit 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a display device 14, an input device 15, and a ToF (time of Flight) camera 20. The ToF camera 20 includes a near-infrared (NIR) sensor 30, an NIR light source 31, and a depth sensor 40. In this embodiment, the device 10 is, for example, a cellular phone, a personal digital assistant (PDA) , a personal computer, a robot, a measuring instrument or a game machine or the like.

The processing unit 11 is connected to the individual components by a bus to transfer a control signal and data. The processing unit 11 runs various kinds of programs for implementing the general operation of the device 10, and performs arithmetic operations and timing control or the like. The aforementioned programs may be stored in a computer readable storage medium such as a DVD (Digital Versatile Disc) -ROM (Read Only Memory) or CD (Compact Disk) -ROM (Read Only Memory) .

The programs for an operating system and various kinds of data, which are needed to control the operation of the entire device 10, are stored in the ROM 12.

The RAM 13 is provided with a storage area for temporarily storing data and the programs, where the programs and data as well as other data necessary for the operation of the device 10 are stored.

The display device 14 may be, for example, a flat panel display such as a liquid crystal display or an EL (Electro-Luminescence) display. The input device 15 includes operational buttons, a touch panel, an input pen, a sensor, and the like.

The NIR camera 30 mounted in the ToF camera 20 receives light from the NIR light source 31 reflected at the subject. Thus, a near-infrared (NIR) image (first image) d10 to be described later can be acquired.

The depth sensor 40 also receives light from the NIR light source 31 reflected at the subject. Thus, in the ToF camera 20, a distance to the subject is calculated pixel by pixel from a time for irradiated light from the NIR camera 30 to return to the depth sensor 40 and the velocity of light (3×10 ⁸ m/s) . Accordingly, a depth image (second image) d20 to be described later including depth information representing the distance to the subject pixel by pixel is acquired. The NIR image d10 and the depth image d20 are both designed to be structured to be acquired in a contactless manner.

Fig. 2A shows an aspect in a case where light from the NIR light source 31 is reflected at a subject 100. In the example shown in Fig. 2A, light from the NIR light source 31 is emitted to the subject 100, and then the NIR camera 30 and the depth sensor 40 both receive light reflected at the subject 100. Accordingly, the NIR camera 30 and the depth sensor 40 acquire the NIR image d10 and the depth image d20, respectively.

Fig. 2B shows an aspect for acquiring the NIR image d10 and the depth image d20. In the example shown in Fig. 2B, the NIR image d10 and the depth image d20 are images of the same angle of view acquired from a same imaging point P. In this case, the imaging point P is the imaging point for both of the NIR camera 30 and the depth sensor 40.

The images d10, d20 are periodically output to the processing unit 11 simultaneously in a frame format. Such synchronous timings include, for example, a timing at which pulse light is emitted to the subject 100 from the NIR light source 31. The acquisition of the NIR image d10 and the depth image d20 enables it possible to execute a process of acquiring a pulse wavelength velocity (PWV) to be described later.

It should be noted that the ToF camera 20 may be externally mounted to the device 10. Further, the device 10 may be configured to achieve the same functions as the ROM 12 or the RAM 13 or the like using an external storage device such as a hard disk or an optical disc.

The ToF camera 20 may be implemented by other alternative schemes as long as the NIR image d10 and the depth image d20 can be acquired. For example, when the depth is measured with a stereo camera, a camera included in the stereo camera may acquire an NIR image. When the depth is measured with a depth camera, an image acquired by the depth camera may be treated as the NIR image d10.

[Outline of Indication Information Calculating Process]

Next, the outline of a process of calculating a PWV as information indicative of an indication relating to a blood flow, which is implemented by the device 10 will be described with reference to Figs. 1 to 6B. Fig. 3 is a diagram for describing the relationship between the wavelength of irradiated light and the transmission depth thereof. Fig. 4 is a diagram for describing the outline of measurement. Fig. 5 is a diagram for describing a distance between two regions of interest (ROIs) . Fig. 6A is a diagram for describing a pixel size. Fig. 6B is a diagram exemplifying the size of one pixel g shown in Fig. 6A.

As shown in Fig. 3, the transmission distance of light to a deep part from a body surface varies according to wavelengths d1 to d8 of light (wavelength being about 800 nm to 400 nm) . As the wavelength gets larger, the transmission distance of light to the deep part increases. d1 to d8 show a wavelength range from the wavelength of reddish purple to the wavelength of purple.

This device 10 uses near-infrared light having a wavelength range of reddish purple light with the wavelength d1, among the wavelengths d1 to d8, which has the largest distance to a deep part. The wavelength d1 is, for example, about 750 nm to 800 nm, but is not limited thereto if the NIR image d10 including a deep part of a human body to be detected can be acquired.

In the example shown in Fig. 3, near-infrared light with the wavelength d1 transmissively reaches an artery located at a deep part (at a depth of 3.0 mm or more from a skin) in a human body as a detection target. Consequently, the device 10 generates an NIR image d10 to be described later, which represents the luminance according to a change in a blood flow (artery flow) flowing through the artery, by using infrared light reflected at an artery, and calculates indication information d40 on the artery flow of a person to be examined or a subject 100 based on the result of evaluation of the NIR image d10.

The wavelength d1 has only to allow reflection at an artery, and a wavelength other than that of the aforementioned near-infrared light representing reddish purple may be available.

The indication information d40 that is handled by the device 10 of the embodiment is, but not limited to, for example, a pulse wavelength velocity (PWV) . The PWV is used as an indication for the progressing rate of arterial sclerosis. For example, the greater the value of the PWV, the more likely cardiac infarction will occur.

Since the device 10 of the embodiment uses near-infrared light whose wavelength has a large light transmission depth as mentioned above, it is possible to acquire indication information d40 of a person to be examined according to a change in a blood flow in an arterial vessel, not from a blood flow in a blood capillary. This indication information d40 is equivalent to reflection of a change in a blood flow which flows through an arterial vessel, enhancing the reliability of the indication information d40.

As shown in Fig. 4, this device 10 outputs the NIR image d10 and the depth image d20 (Fig. 2B) , imaged at the same point of view, from the ToF camera 20 to the processing unit 11 in synchronism.

In Fig. 4, for example, the depth sensor 40 in the ToF camera 20 acquires light emitted from the NIR light source 31 and later reflected at the subject 100, thereby providing the depth image d20 of the subject 100. Further, as the NIR camera 30 images the subject 100 during the period of acquisition of the depth image d20 by the depth sensor 40, the NIR image d10 of the subject 100 is acquired. As shown in Fig. 4, for example, the NIR image d10 is an image including the neck region of the person to be examined, and this image (frame image) is acquired from the ToF camera 20 in order.

The processing unit 11 which has acquired the NIR image d10 sets two regions of interest (ROIs) 1 and 2 for an artery part located in the neck of the person to be examined included in the NIR image d10 being treated as a detection target. In the example of Fig. 4, the ROI 1 includes an artery part far from a heart, and the ROI 2 includes an artery part closer to the heart. In this case, the artery part is, for example, a part of a radial artery. Accordingly, indication information reflecting a change in an artery flow in a carotid artery is acquired, which enhances the availability of indication information.

It is to be noted that one way of setting the

ROIs

1 and 2 may be, but not limited to, setting the

ROIs

1 and 2 at a preset interval therebetween. For example, shapes of or positions in artery parts may be registered in advance in the device 10. Thus, the processing unit 11 may set the

ROIs

1 and 2 from the registered information after an artery part is identified.

Further, the processing unit 11 detects time series signals f (t) and g (t) which vary according to artery flows flowing through artery parts within two

ROIs

1 and 2 included in the NIR image d10. In this case, the time series signals f(t) and g (t) are extracted in such a way that a photo plethysmography (PPG) is acquired from the NIR image d10. In the time series signals f (t) and g (t) , the lateral direction represents time t, and the longitudinal direction represents an average value of luminance of all the pixels in the corresponding ROI.

The aforementioned time series signals f (t) and g(t) may be subjected to Nx upsampling (e.g., N=8) . In this case, the number of samples representing the values of the time series signals f (t) and g (t) increases. Thus, values of the time series signals f (t) and g (t) may be given more accurately.

A cross-correlation function 111 is a function for calculating convolution of the two time series signals. A coherence indicating the degree of correlation of the two time series signals is calculated by changing the phases of the time series signals. Then, the phase deviations of the time series signals and the similarity of the periodicity of the time series signals are evaluated from the result. When two identical time series signals are input to the cross-correlation function 111, the then cross-correlation function become equivalent to an auto-correlation function, and shows a maximum. In this embodiment, the processing unit 11 outputs to a subsequent stage the value of "m" indicating the number of samples for the phase delay of the time series signal g (t) when the value of the cross-correlation function 111 shows a maximum, as a phase delay (phase offset) d30.

The cross-correlation function 111 may be expressed by, for example, the following equation (1) .

In the equation (1) , n represents the length (e.g., two cycles) of the time series signals f (t) and g (t) , and m represents the number of samples for the phase delay of the time series signal g (t)) .

Moreover, in Fig. 4, the processing unit 11 acquires a distance between two

ROIs

1 and 2 from the depth image d20 through a calculation process 112. In Fig. 5, for example, a distance L between a center Fo of the ROI 1 and a center G ₀ of the ROI 2 is set as the distance between the two

ROIs

1 and 2. The

ROIs

1 and 2 are the same as those shown in the NIR image d10

The distance L may be set to a value different from the value exemplified in Fig. 5. For example, the maximum distance or the minimum distance between the two

ROIs

1 and 2 or a distance for a preset number of pixels between the two

ROIs

1 and 2 may be used as the distance L.

In this embodiment, the distance L between the two

ROIs

1 and 2 is set as the distance between artery parts as detection targets.

In Fig. 4, a field of view (FOV) and a resolution are set in a setting unit 113 of the processing unit 11. Further, the processing unit 11 acquires the scale of each pixel in the depth image d20 through an acquisition process 114. When a depth image (image having 600 pixels in width and 360 pixels in height) d20 with a horizontal field of view of h° and a vertical field of view of v° from an imaging point P of the depth sensor 40 is acquired as shown in Fig. 6A, for example, the size (Lh, Lv) (Fig. 6B) per pixel ( "g" shown in Fig. 6A) is expressed by the following equation (2) .

Lh=2·d·tan (h/2) /600

Lv=2·d·tan (v/2) /360 (2)

In the equation (2) , d represents the distance to the depth image d20 from the imaging point P. Although an average distance to a corresponding ROI (average distance among all the pixels within the ROI) is used as one example of the distance d in the device 10 of the embodiment, as will be described later, the distance may take a different value. It is to be understood that the equation (2) merely shows an example size per pixel, which may be changed. The values of the size, Lh and Lv, may be shown according to the value of the resolution.

The processing unit 11 acquires the value of the distance L (Fig. 5) between the

ROIs

1 and 2 from the pixel size (Lh, Lv) shown in the equation (2) . When the distance L of ten pixels in the vertical direction is shown, for example, the value of "L" is given by Lv×10.

Moreover, in Fig. 4, the processing unit 11 calculates and outputs the PWV as indication information d40 relating to an artery flow of the subject. The PWV as the indication information d40 is acquired by, for example, the following equation (3) .

PWV=L/D (3)

In the equation (3) , L represents the distance (Fig. 5) between the

ROIs

1 and 2, and D indicates a time for the aforementioned phase delay d30. In this case, d is given from m/(r×N) where m represents the number of samples for the phase delay of the time series signal g (t) indicated by the phase delay d30, r represents the frame rate of the NIR camera 30, and N represents the upsampling number.

As explained above, the device 10 of the embodiment obtains the indication information d40 from the NIR image d10 and the depth image d20.

[Functional Configuration of Device 10]

Fig. 7 is a diagram showing an example of the functional configuration of the device 10 that is implemented on the hardware configuration shown in Fig. 1. The following describes the functional configuration of the device 10 with reference to Fig. 7. As shown in Fig. 7, the device 10 includes a first acquisition unit 101, a detection unit 102, a shift amount calculating unit 103, a second acquisition unit 104, a calculation unit 105, an indication information calculating unit 106, and an output unit 107.

Those components are implemented by the processing unit 11 shown in Fig. 1, and are configured as follows.

The first acquisition unit 101 acquires an NIR image d10 obtained by imaging an artery part of a subject in a contactless manner.

The detection unit 102 detects individual pulse waves corresponding to a plurality of positions of an artery part (time series signals f (t) and g (t) in Fig. 4) based on the NIR image d10.

The shift amount calculating unit 103 calculates an amount of shifting (phase delay d30 in Fig. 4) between the individual pulse waves detected by the detection unit 102.

The second acquisition unit 104 acquires a depth image d20 including depth information of the artery part.

The calculation unit 105 calculates a distance between artery parts (distance L between

ROIs

1 and 2 in Fig. 5) associated with the amount of shifting calculated by the shift amount calculating unit 103, based on the depth information included in the depth image d20.

The indication information calculating unit 106 calculates indication information (PWV) relating to an artery flow of the subject as biological information by using the amount of shifting. Further, the indication information calculating unit 106 may calculate the indication information d40 by using the distance between the artery parts calculated by the calculation unit 105 and the amount of shifting.

The output unit 107 outputs the indication information d40.

The components of the individual units 101 to 107 shown in Fig. 7 may be implemented by an ASIC (Application Specific Integrated Circuit) or FGA (Field Programmable Gate Array) or the like. Those components are referred to in the following description of the operation of the device 10 as needed.

[Operation of Device 10]

The following describes the general processing of the device 10 with reference to Figs. 1 to 8. The processing unit 11 in this embodiment can execute various processes to be described later according to a program.

Fig. 8 is a flowchart illustrating one example of the general process of calculating indication information d40.

In Fig. 8, when a subject 100 is imaged by the NIR camera 30 of the ToF camera 20, the processing unit 11 acquires an NIR image d10 of the subject 100 (step S11) . In Fig. 4, for example, an NIR image d10 including the range of the neck of the subject is acquired.

In this step, the processing unit 11 is implemented as the first acquisition unit 101.

The processing unit 11 detects individual pulse waves corresponding to a plurality of positions in an artery part of the subject based on the NIR image d10 (step S12) . The detected pulse waves are indicated as time series signals f (t) and g (t) as exemplified in Fig. 4. For example, Fig. 4 shows an example in which a change in an artery flow flowing through an artery part located in an ROI 1 with the passage of time is represented by the time series signal f (t) . Fig. 4 also shows an example in which a change in an artery flow flowing through an artery part located in an ROI 2 with the passage of time is represented by the time series signal g (t) . The

ROIs

1 and 2 are set in correspondence with the positions in an artery part as detection targets in the NIR image d10.

In step S12, the processing unit 11 may perform the detection by upsampling the time series signals f (t) and g (t) . In a case of 8x upsampling, for example, samples of the time series signals f (t) and g (t) are interpolated at timings of t=0.125, 0.25, 0.375, 0.625, 0.75, and 0.875 over an interval of t=0 to 1. Thus, the time series signals f (t) and g (t) are shown more precisely.

In this step, the processing unit 11 is implemented as the detection unit 102.

Then, the processing unit 11 calculates a phase delay d30 between the aforementioned pulse waves (step S13) . In the example of Fig. 4, with reference to the two time series signals f (t) and g (t) , the processing unit 11 determines whether the value of the cross-correlation function 111 shown in the equation (1) shows a maximum by shifting the phase of the time series signal g (t) . When the determination results in that the value of the cross-correlation function 111 shows the maximum, the processing unit 11 calculates the phase delay d30 of the time series signal g (t) (e.g., the value of "m" indicating the number of samples for the phase delay of the time series signal g(t) ) when the value of the cross-correlation function 111 shows the maximum.

In this step, the processing unit 11 is implemented as the shift amount calculating unit 103.

The processing unit 11 calculates indication information d40 (PWV in Fig. 8) on an artery part of the subject, as biological information, by using the phase delay d30 (step S14) . Further, the processing unit 11 outputs the indication information d40 (step S15) . Accordingly, the indication information d40 may be provided visually.

In step S14, the processing unit 11 is implemented as the indication information calculating unit 106. In addition, in step S15, the processing unit 11 is implemented as the output unit 107.

In the step S14, the PWV as the indication information d40 is obtained from PWV=L/D (where L is the distance between the

ROIs

1 and 2, and D is the time corresponding to the phase delay d30 resulting from the calculation of step S13) as expressed in the aforementioned equation (3) . The process of acquiring the value of "L" in the equation (3) is illustrated in a flowchart in Fig. 9.

The value of "L" mentioned above may be input via the input device 15. Even in this manner, the indication information d40 may also be obtained from the equation (3) .

Fig. 9 is a flowchart illustrating one example of the process of calculating the distance L in the equation (3) .

In Fig. 9, the processing unit 11 acquires, from the ToF camera 20, a depth image d20 having the same point of view as the NIR image d10 in synchronism with the NIR image d10 (step S21) . In this example, the depth image d20 includes depth information representing a distance to the subject pixel by pixel.

In this step, the processing unit 11 is implemented as the second acquisition unit 104.

The processing unit 11 then calculates a distance to an artery part as a detection target based on the depth image d20 (step S22) . This process is illustrated in detail in a flowchart in Fig. 10 to be described later.

Moreover, the processing unit 11 calculates a distance between the artery parts from the result of the calculation of step S22 (step S23) . In Fig. 5, for example, the distance L between the center F ₀ of the ROI 1 and the center G ₀ of the ROI 2 is calculated as the distance between the artery parts in the step S23.

In steps S22 and S23, the processing unit 11 is implemented as the calculation unit 105.

The following describes an example of the calculation process of step S22 with reference to Figs. 10 and 11. Fig. 10 is a flowchart illustrating the calculation process of step S22 in Fig. 9. Fig. 11 is a diagram for describing a separation process of step S221 in Fig. 10.

In Fig. 10, the processing unit 11 separates a foreground and a background from the depth image d20 acquired in step S21 in Fig. 9 (step S22) . In Fig. 11, a foreground G1 and a background G2 are distinguished through filter performed by determining whether the value of depth information included in the depth image d20, for example, as a target (distance to the target from the imaging point P) is equal to or greater than a threshold value. Then, a pixel region having a value equal to or greater than the threshold value is removed as a background G3 far from the imaging point P.

In Fig. 11, an edge G2 represents a portion of an active motion, and is eliminated from the foreground G1. Examples of the edge detection include threshold processing of calculating a gradient, for example.

In this embodiment, the

ROIs

1 and 2 shown in Fig. 4 are specified as the foreground through the separation process of step S221.

Next, in Fig. 10, the processing unit 11 creates a histogram (the amounts of the features of pixels) with a distance indicated by the depth information on the ROIs 1 and 2 (Fig. 4) as targets (step S222) . The processing unit 11 then calculates an average value of the distances of all the pixels in each ROI 1 and 2 (distances indicated by the depth information) as the distance to each

ROI

1 and 2 based on the histogram created in step S222 (step S223) . In this case, when the value of the target pixel has a difference of a threshold value or more from other values in the distribution of the created histogram, for example, the processing unit 11 eliminates that value as an unfit value, and then calculates the average value.

In this embodiment, the average value calculated in step S223 is set as the distance d (fig. 6) to an artery part in each ROI from the imaging point P. As a result, the size (Lh, Lv) per pixel is acquired from the equation (2) . Further, the distance between artery parts is calculated in step S23 in Fig. 9. That is, the distance L between the two ROIs 1 and 2 (Fig. 5) as the distance between the artery parts is calculated from the pixel size (Lh, Lv) acquired from the equation (2) . In Fig. 5, for example, when the distance L of ten pixels in the vertical direction is shown, for example, the value of "L" is given by Lv×10.

Consequently, the processing unit 11 substitutes the value of "L" calculated in step S23 in Fig. 9 into PWV=L/D, shown in the equation (3) in step S14 in Fig. 8 to calculate the value of the PWV as the indication information d40.

In step S14 in Fig. 8, the processing unit 11 may calculate the PWV shown in the equation (3) for the time series signals f (t) and g (t) of preset cycles (e.g., five cycles, ten cycles, or the like) . In this case, the average value, the maximum value or the minimum value of the PWV, for example, may also be used as the indication information d40. Even if the PWV cannot be properly calculated from the time series signals f(t) and g (t) at a certain timing, therefore, the adequate indication information d40 is obtained by using the average value or the like of the PWV acquired from the time series signals f(t) and g (t) for the aforementioned cycles.

According to the embodiment, as described above, the distance to an artery part within each ROI from the imaging point P is acquired based on the depth image d20, and the actual distance L or the like between the ROIs 1, w in the detection target is calculated based on this distance d. In this manner, the distance d to an artery part under a skin which cannot be directly observed is acquired at the time of acquiring the time series signals f (t) and g (t) based on the image of the artery part as the detection target. Accordingly, the time series signals of pulse waves to be detected can be made to adequately reflect the actual pulse waves. According to the related art that acquires time series signals of pulse waves based on a change in the color of a skin surface, the pulse waves of blood capillaries near a skin surface are measured. Thus, an indication such as the PWV cannot be obtained accurately. In contrast, the present embodiment, which accurately acquires an indication relating to a blood flow by obtaining the pulse waves of an artery, can surely identify an artery part by acquiring the distance d in the above-described manner.

Unlike information reflecting a change in a blood flow flowing through a blood capillary of a subject, the indication information d40 is calculated to reflect a change in a blood flow flowing through an artery vessel. This can improve the reliability of the indication information.

Moreover, the distance between artery parts (distance L between the

ROIs

1 and 2 in Fig. 5) is acquired from the depth image d20, thus eliminating the need for an operation of inputting the distance L. This eliminates an error in an input value. Thus, correct indication information d40 can be obtained.

Further, the NIR image d10 and the depth image d20 at the same field of view are output from the ToF camera 20 to the processing unit 11 in synchronism with each other. Thus, the processing unit 11 can obtain the indication information d40 through steps S21 to S23 in Fig. 9 in synchronism with one another.

Although the distance L between two ROIs 1 and 2 (Fig. 5) has been described as the distance between artery parts by way of example, the distance may be changed as needed. For example, Fig. 12 exemplifies an aspect in which with an artery part 71 being located in the

ROIs

1 and 2 shown in Fig. 4, the locations of the artery part 71 are estimated to calculate a distance between the artery parts 71 according to the result of the estimation. In Fig. 12, the processing unit 11 registers patterns of distances to artery parts around the neck of a subject in advance, and compares depth information included in the depth image d20 with the registered patterns of distance information to estimate the location of the artery part 71. Then, the processing unit 11 calculates the entire length of the artery part 71 along the estimated locations of the artery parts 71 as a distance L1. As a result, a more accurate distance L1 between the artery parts 71 may be obtained, and more accurate indication information d40 may be calculated. As the process of estimating the location of an artery part, for example, shapes of artery parts may be patterned in advance, and the location of an artery part included in the depth image d20 may be estimated from the patterns.

The aforementioned artery part as a detection target is not limited to the part of the subject 100 shown in Fig. 4. For example, Fig. 13 exemplifies a case where a radial artery within an arm range 81 of a subject is a detection target. Even in this case, indication information d40 reflecting a change in the blood flow of a radial artery in an arm is acquired.

The device-relating embodiment and the method-relating embodiment are based on the same concept, so that the technical advantages that are brought about by the device-relating embodiment are also the same as those brought about by the method-relating embodiment. For the specific principle, the foregoing description of the embodiment of the device should be referred to, and the details thereof will not be repeated herein.

It will be appreciated by those skilled in the art that the foregoing embodiments and all or part of the processes implementing the equivalent modified examples made within the scope of claims of the present invention will also fall within the scope of the present invention.

Claims

A method for obtaining biological information, comprising:

acquiring a first image of an artery part of a subject imaged in a contactless manner;

detecting individual pulse waves corresponding to a plurality of positions in the artery part based on the first image;

calculating an amount of shifting between the detected individual pulse waves; and

calculating indication information on an artery flow of the subject, as the biological information, by using the amount of shifting.
The method according to claim 1, further comprising:

acquiring a second image including depth information of the artery part; and

calculating a distance between artery parts associated with the amount of shifting based on the depth information included in the second image,

wherein the calculating the indication information calculates the indication information by using the calculated distance between the artery parts and the amount of shifting.
The method according to claim 2, wherein the calculating the distance between the artery parts includes separating a foreground and a background from the second image, calculating distances to the artery parts as a detection target based on an amount of features of pixels in the image on the foreground, and calculating the distance between the artery parts by using the calculated distances.
The method according to claim 2 or 3, wherein the calculating the distance between the artery parts includes estimating positions where the artery parts are located, based on the depth information, and calculating the distance between the artery parts by using a result of the estimation.
The method according to any one of claims 1 to 4, wherein the detecting the pulse waves includes detecting the pulse waves by upsampling time series signals corresponding to the positions where the artery parts included in the first image are located.
The method according to any one of claims 1 to 5, further comprising outputting the indication information.
The method according to any one of claims 1 to 6, wherein the indication information is a pulse wavelength velocity (PWV) .
The method according to any one of claims 1 to 7, wherein the artery part is a part of a carotid artery.
The method according to any one of claims 1 to 8, wherein the first image is acquired by using a near-infrared camera.
The method according to any one of claims 2 to 4, wherein the second image is acquired by using a depth sensor.
A device for obtaining biological information, comprising:

a first acquisition unit for acquiring a first image of an artery part of a subject imaged in a contactless manner;

a detection unit for detecting individual pulse waves corresponding to a plurality of positions in the artery part based on the first image;

a shift amount calculating unit for calculating an amount of shifting between the detected individual pulse waves; and

an indication information calculating unit for calculating indication information on an artery flow of the subject, as the biological information, by using the amount of shifting.
The device according to claim 11, further comprising:

a second acquisition unit for acquiring a second image including depth information of the artery part; and

a calculation unit for calculating a distance between artery parts associated with the amount of shifting based on the depth information included in the second image,

wherein the indication information calculating unit is configured to calculate the indication information by using the calculated distance between the artery parts and the amount of shifting.
The device according to claim 12, wherein the calculation unit is configured to separate a foreground and a background from the second image, calculate distances to the artery parts as a detection target based on an amount of features of pixels in the image on the foreground, and calculate the distance between the artery parts by using the calculated distances.
The device according to claim 12 or 13, wherein the calculation unit is configured to estimate positions where the artery parts as detection targets are located, based on the depth information, and calculate the distance between the artery parts by using a result of the estimation.
The device according to any one of claims 11 to 14, wherein the detection unit is configured to detect the pulse waves by upsampling time series signals corresponding to the positions where the artery parts included in the first image are located.
The device according to any one of claims 11 to 15, further comprising an output unit for outputting the indication information.
The device according to any one of claims 11 to 16, wherein the indication information is a pulse wavelength velocity (PWV) .
The device according to any one of claims 11 to 17, wherein the artery part is a part of a carotid artery.
The device according to any one of claims 11 to 18, wherein the first image is acquired by using a near-infrared camera.
The device according to any one of claims 12 to 14, wherein the second image is acquired by using a depth sensor.
A device comprising:

the device as recited in any one of claims 11 to 20;

a near-infrared camera configured to image the first image; and

a depth sensor used to obtain the indication information, and obtain depth information of the plurality of artery parts.
A computer readable storage medium recording a program for allowing a computer to execute the method as recited in any one of claims 1 to 10.
A computer program for allowing a computer to execute the method as recited in any one of claims 1 to 10.