JP2018534025A - Device, system and method for acquiring vital sign related information of living body - Google Patents

Abstract

The present invention relates to a device, a system, and a method for acquiring vital sign related information of a living body 3. The proposed device is an input unit 12 that receives an input signal 32 generated from light 31 received at least one wavelength interval reflected from the skin region 2 of the living body 3, wherein the input signal is An input unit representing vital sign related information from which a vital sign of a living body can be obtained; a processing unit 14 that processes the input signal 32 and obtains the vital sign related information 36 of the living body from the input signal 32; A direction estimation unit 16 for estimating the direction of the skin region 2 and a lighting unit 40 for illuminating the skin region 3 with light 41 to illuminate the skin region 2 based on the estimated direction of the skin region 2; And / or a vital sign function from the input signal obtained during a time interval selected based on the estimated direction of the skin region 2. To obtain information 36, and a control unit 18 for controlling the processing unit.

Description

  The present invention relates to a device, system and corresponding method for obtaining vital sign related information of a living body.

  Unobtrusive vital sign monitoring using a video camera or remote PPG (Photoplethysmography) has been found and proven to be important for patient monitoring. Remote photoplethysmographic imaging is described, for example, in “Remote plethmomorphic imaging using ol. Ssl, ss. It is based on the principle that temporal variations in blood volume in the skin result in variations in light absorption by the skin. Such variations can be aligned by a video camera that takes an image of a skin area, eg, the face. Meanwhile, the pixel average calculation is processed over a selected region (generally the cheek portion in this system). By paying attention to the periodic change of the average signal, the heart rate and the respiration rate can be extracted. There are additional publications and patent applications that describe details of devices, systems and methods for obtaining patient vital signs using remote PPG.

  Thus, arterial blood pulsation causes a change in light absorption. These changes observed at the photodetector (or array of photodetectors) form a PPG (Photoplethysmography) signal (also called a wave). Blood pulsations are caused by peaks in the PPG signal corresponding to the beating heart, ie, the individual beats of the heart. Thus, the PPG signal is a beating signal in itself. The normalized amplitude of this signal is different for different wavelengths, and for some wavelengths it is also a function of blood oxidation.

  Regular video data is often shown to produce sufficient vital signs (sometimes also called biometric signals, eg beating, breathing rate, Sp02 rate, etc.), but strong motion, low light intensity Image acquisition for challenging cases, such as non-white lighting, requires further improvement. Known methods, systems and devices are generally robust to motion and different lighting environments as long as one dominant light source is present. Under such conditions, PPG technology has proven to be accurate and robust in that it can be used on treadmills during fitness exercises for at least some vital signs such as heart rate. .

  One major challenge that arises in image-based (eg camera-based) vital sign surveillance occurs when no dominant light is present in the environment. Furthermore, specific irradiations are not necessarily optimal for all measurements, for example with respect to different skin types, body postures or after body movements.

  WO2014 / 083310A1 discloses a device and method for obtaining vital sign information of a living body. The disclosed device receives at least one wavelength interval of light reflected from a region of interest of a living body, generates an input signal from the received light, processes the input signal, and performs remote photoplethysmography. A processing unit for obtaining vital sign information of the living body from the input signal, an illumination unit for illuminating at least the region of interest with light, and controlling the illumination unit based on the input signal and / or the derived vital sign information Control unit. However, further improvements in exercise robustness are desired.

  WO2013 / 186696A1 discloses a system for determining a vital sign of a subject, which includes an imaging unit for obtaining the subject's video data, and a marker attached directly or indirectly to the subject's body and including a graphic pattern An image processing unit for detecting the marker in the video data; and adapted to extract a vital sign parameter associated with the target vital sign from the video data and to determine the vital sign from the vital sign parameter And an analysis unit.

  It is an object of the present invention to provide a device, system and corresponding method for obtaining vital sign related information of a living body that provides higher motion robustness with respect to the movement of the living body.

In a first aspect of the present invention, a device for obtaining vital sign related information of a living body is presented,
An input unit for receiving an input signal generated from light received at at least one wavelength interval reflected from a skin region of a living body, wherein the input signal is a vital for obtaining a vital sign of the living body An input unit representing sign-related information;
A processing unit that processes the input signal and obtains vital sign related information of the living body from the input signal;
A direction estimation unit for estimating the direction of the skin region;
In order to illuminate the skin region based on the estimated orientation of the skin region, a lighting unit that illuminates the skin region with light is controlled and / or a time interval selected based on the estimated direction of the skin region. A control unit for controlling the processing unit in order to obtain vital sign related information from the input signal obtained in the meantime.

  In a further aspect of the invention, a corresponding method for obtaining vital vital sign information is presented.

In another aspect of the present invention, a system for obtaining vital sign related information of a living body is presented,
A detection unit that receives light at at least one wavelength interval reflected from a skin region of a living body and generates an input signal representing vital sign-related information from which the biological signal of the living body can be obtained from the received light; ,
A lighting unit that illuminates the skin area with light;
A device disclosed in this document for obtaining vital sign related information of a living body from the input signal.

  In a further aspect of the invention, a computer program having program code means that, when executed on a computer, causes the computer to perform the steps of the methods described herein, and described herein when executed by a processor. A non-transitory computer readable recording medium is provided that stores a computer program that causes the method to be performed.

  Preferred embodiments of the invention are defined in the dependent claims. It is to be understood that the method, computer program and medium described in the claims have similar and / or identical preferred embodiments to the device recited in the claim and to the device defined in the dependent claims.

  In accordance with the present invention, changes in skin orientation relative to illumination (due to breathing, heart rate or other object movement) despite complete motion tracking (eg, following movement in the image plane, particularly following movement of the skin area). ) May significantly affect the quality of the acquired input signal or the PPG signal obtained from the input signal, particularly when the skin irradiation is at a large angle with the skin surface normal. I understood. This is because the projected light energy is distributed over the surface, which depends on the angle between the skin surface normal and the illumination angle.

  In the case of illumination from a certain solid angle, when the angle between the surface normal and the incident light (especially the incident principal ray) increases, the irradiated surface increases. Therefore, the irradiance (W / m 2) decreases with increasing angle. According to the present invention, a cleaner raw signal (input signal) is obtained instead of following an algorithmic approach that can clean the raw signal to some extent. This is accomplished, for example, by controlling the illumination of the skin region (also referred to as the region of interest) based on the estimated orientation of the skin region relative to the illumination or relative to a known reference plane. As a result, the skin region is illuminated in an optimized manner with respect to information acquisition, from which one or more accurate vital signs can be obtained in a robust manner. This control is preferably performed so that the change in skin orientation is irradiated at an angle that has the least effect on the intensity of the light that is irradiated and consequently reflected.

  In order to obtain vital sign related information of the living body from the input signal using photoplethysmography, a corresponding processing unit for processing the input signal is provided. As an alternative to lighting control, light from different lighting angles, for example, is time multiplexed and the processing unit determines which time interval to use for processing and optionally for vital sign extraction. Can do.

  In general, the interaction of electromagnetic radiation, particularly light and biological tissue, is complex and includes (optical) processes of (multiple) scattering, backscattering, absorption, transmission and (diffuse) reflection. The term “reflecting” as used in the context of the present invention is not limited to specular reflection but includes electromagnetic radiation, particularly the aforementioned types of interactions between light and tissue, and any combination thereof. .

  The term “vital sign” as used in the context of the present invention refers to a subject's physiological parameters (also referred to herein as a living body or a human or patient) and derived parameters. In particular, the term “vital sign” refers to blood volume pulse signal, heart rate (HR) (sometimes also called pulse rate), heart rate variability (pulse rate variability), pulsation intensity, perfusion, perfusion index, perfusion variability, Traube. This includes the concentration of substances in the blood and / or tissues such as Hering Mayer waves, respiratory rate (RR), skin temperature, blood pressure, (arterial) blood oxygen saturation or glucose levels. In addition, a “vital sign” generally includes a health condition derived from the shape of the PPG signal (eg, the shape includes some information about partial arterial occlusion (eg, applying a blood pressure cuff to the arm, If the shape obtained from the PPG signal is more sinusoidal, or contains information about, for example, skin thickness (eg, the PPG signal from the face is different from that of the hand), or even contains information about temperature There).

  The term “vital sign information” as used in the context of the present invention includes one or more measured vital signs as described above. In addition, it includes data referring to physiological parameters, corresponding waveform traces, or data referencing physiological parameters of time that may be useful for later analysis.

  To obtain a vital sign information signal of interest, the data signal of the skin pixel area in the skin area is evaluated. Here, “skin pixel region” means a region including one skin pixel or a group of adjacent skin pixels, ie, a data signal can be obtained for a single pixel or a group of skin pixels.

  According to the invention, the orientation of the skin surface is estimated, the optimal control of the illumination unit, in particular the illumination angle is determined in an automated manner, and the illumination is adapted to these estimates for optimal signal quality. Various alternatives for providing the recipe are described.

  In one embodiment, the control unit controls the intensity, direction, distribution, and / or illumination angle of at least a portion of the light emitted by the lighting unit. Based on the estimated direction of the skin area and the design of the lighting unit, the corresponding (optimal) parameters of the lighting unit can thus be appropriately controlled. Of course, other (additional or alternative) parameters can be used for control, such as illumination uniformity in the ROI, good / stable illumination in all relevant channels (wavelengths), no shadows in the ROI. . Further, in certain embodiments, only certain wavelengths (eg, wavelengths used for vital sign extraction) are changed, while other wavelengths (eg, for direction measurements) remain unaffected.

  In another embodiment, the direction estimation unit estimates the orientation of the surface normal of the skin region, and the control unit is the intensity, direction, distribution, and at least part of the light emitted by the illumination unit. Controlling the illumination angle, most or all of the light is emitted from an illumination angle close to or the same as the estimated surface normal. Thus, this embodiment minimizes the amplitude of variation in irradiance by irradiating the skin region at an angle substantially perpendicular to the average skin orientation of the region of interest (ROI), or at least a large angle, eg 45 Prevent angles greater than degrees. This reduces the variance in irradiance (noise) due to object motion, thus leading to a more robust and accurate input signal and resulting vital signs.

  In another embodiment, the direction estimation unit estimates the orientation of the skin region regularly or continuously, and the control unit adjusts the control of the lighting unit based on this. In this way, the movement of the object can be recognized quickly or immediately so that the lighting can be adapted in an automated manner.

  According to one option of the invention, the control unit controls the lighting unit based on the estimated orientation of the skin area, but according to an alternative (or additional) option it controls the processing unit. Then, vital sign related information is obtained based on the estimated direction of the skin region. In this optional embodiment, the control unit controls the illumination unit to illuminate the skin area at different time intervals from the next time interval and controls the processing unit to provide the estimation of the skin area. Vital sign related information is obtained from the input signal obtained during a time interval selected based on the determined orientation. Thus, for example, the illumination is time multiplexed and the process is controlled to use only input signals acquired at specific time intervals where illumination is performed from the best illumination angle.

  In general, it is sufficient to use one lighting element as the lighting unit (its position and / or orientation with respect to the skin area can be changed), but preferably the lighting unit comprises two or more lighting elements (both light sources). For example, having a plurality of LEDs. Thus, for example, by individually controlling separate lighting elements, the desired control of lighting can be realized in a quick, simple and flexible manner. For this reason, the two or more illumination elements are preferably arranged in different positions and / or different orientations so as to illuminate the skin area from different illumination angles. The two or more lighting elements are preferably controlled individually, such as controllable LEDs, laser diodes, conventional bulbs, neon lights, and the like.

  The control unit is preferably configured to control the lighting elements individually, in particular to control the intensity of the lighting elements individually. This makes it possible, for example, to increase the power of the lighting element that illuminates the skin area at an angle closest to the surface normal and to decrease the power of the lighting element that illuminates the skin area at a larger angle.

  Various embodiments exist for estimating the orientation of the skin region in an accurate and reliable manner. In one embodiment, the control unit controls the illumination element to illuminate the skin area in a multiplexed and / or modulated manner, and the detection unit is based on the subsequent illumination and the skin area. The reflected light is received, a reflected signal is generated from the received light, and the direction estimation unit determines the reflected signal having the strongest intensity. The control unit further controls the lighting unit such that the lighting element producing the reflected signal with the strongest intensity illuminates the skin area exclusively or with the strongest intensity of all the lighting elements; and Vital sign related from the input signal obtained during the time interval when the lighting element producing the reflected signal with the strongest intensity illuminates the skin area exclusively or with the strongest intensity of all the lighting elements The processing unit is controlled to obtain information.

  In another embodiment, an illumination unit (including one or more illumination elements) emits a spectrum that includes the wavelength used for orientation and the wavelength used for vital sign measurements. In such an embodiment, the spectrum of the lighting unit may be changed, i.e. only the intensity of the wavelength used for vital sign measurement is adapted, optionally keeping the wavelength used for direction measurement at the same level. May be. This provides the advantage that the direction measurement does not suffer from strength reduction in some directions.

  In an alternative embodiment, the control unit controls the lighting elements to illuminate the skin region in pairs, and different pairs of adjacent lighting elements are at different alternating frequencies, at different wavelengths, at different times. And / or alternately illuminating the skin area with synchronized flicker, and the detection unit receives light reflected from the skin area based on the alternating illumination and reflects a reflected signal from the received light. And the direction estimation unit determines a reflected signal with minimal intensity modulation. The control unit is further configured so that the pair of lighting elements that produce a reflected signal with the minimum intensity modulation illuminates the skin area exclusively or with the strongest intensity of all the lighting elements, or the minimum intensity Only illumination in a time slot of illumination that produces a reflected signal with modulation is used for illumination of the skin region, or only illumination in a time slot of illumination that produces a reflected signal with the minimum intensity modulation. The pairs of lighting elements that control the lighting unit and / or produce a reflected signal with the minimum intensity modulation to be used for lighting the skin area are the most exclusive or of all the lighting elements. The processing unit is adapted to obtain vital sign related information from the input signal obtained during a time interval illuminating the skin area with strong intensity Control to.

  In a further embodiment, the detection unit includes a camera that generates the input signal from received light, and a photodiode that generates the reflected signal from the received light, and the illumination unit is input by the camera. A first set of lighting elements that illuminate the skin region to allow signal generation; and a second set of lighting elements that illuminate the skin region to allow generation of a reflected signal by the photodiode. Vital sign information is obtained from the input signal, and the direction of the skin region is estimated from the reflected signal. The photodiode allows a higher modulation frequency. As a result, the light reflected from the skin can be focused to a greater extent.

  Further, the second set of lighting elements is configured to illuminate the skin region with light having a wavelength or wavelength range outside the camera sensitivity interval, so that light for direction estimation and generation of input signals is obtained. Can be performed simultaneously without interfering with each other.

  The direction estimation unit is configured to process data, particularly distance data in time-of-flight (TOF) data acquired, for example, by a TOF camera or radar unit, to determine a 3D model of the skin surface including the skin region. This provides another simple option to estimate the skin area orientation in a fairly accurate manner.

  In general, it is sufficient to use one detection element as the detection unit, but the detection unit preferably has two or more detection elements. The detection element is, for example, an image sensor, a video camera, an RGB camera, an infrared camera, or a still image camera. The detection elements generally have the same parameters but are arranged in different positions and / or different orientations. On the other hand, in one embodiment, the detection elements are different and / or have different parameters. As a result, the detection element that provides the best vital sign information can be selected for signal evaluation. In yet another embodiment, input signals from two or more detection elements can be evaluated in common, for example after averaging the input signals.

It is a figure which shows the diagram explaining the illumination of the skin surface in a different direction of the skin surface. It is a figure which shows the enlarged part of the skin surface. It is a figure which shows the diagram explaining the influence of the irradiation of the skin surface in a different direction of the skin surface. It is a figure which shows the diagram explaining the influence which the direction of a different skin has on pulsation property. It is a figure which shows the diagram explaining the influence which the direction of a different skin has on pulsation property. It is a figure which shows the diagram explaining the effect of this invention. 1 shows a schematic diagram of a first embodiment of a system according to the invention. FIG. 3 shows a schematic diagram of a second embodiment of the system according to the invention. FIG. 4 shows a schematic diagram of a third embodiment of the system according to the invention. FIG. 7 shows a schematic diagram of a fourth embodiment of the system according to the invention. Fig. 2 shows an embodiment of the method according to the invention.

  These and other aspects of the invention will be apparent from and will be elucidated with reference to the embodiments described hereinafter.

が、

と

との間で変化するという効果を持つ。 FIG. 1 is a diagram illustrating illumination of the skin surface with different orientations of the skin surface. A light source 1 with a fixed position and radiance (W / sr ⁻¹ ) illuminates the skin 2 of the subject 3 (in this example the chest wall) at an angle different from 0 ° with respect to the skin normal. provide. The orientation of the skin 2 changes over time due to heartbeat and respiration (i.e., t = 1 where the skin is oriented as shown by 2a and t = where the skin is oriented as shown by 2b). Between 2). This is the angle defined as the angle of the normals N1 and N2 respectively with respect to the direction of the illumination 4

But,

When

It has the effect of changing between.

に依存するからである。ある立体角からの照明の場合、表面法線と入射光線との間の角度が大きくなると、照射面は大きくなり、角度が大きくなると放射照度（Ｗ／ｍ２）は小さくなる。 Such changes in skin orientation can significantly affect the PPG signal quality despite motion tracking, especially when the skin irradiation is at a large angle with the skin surface normal. This is because the projected light energy is distributed over the surface and its size is the angle between the skin surface normal and the illumination

Because it depends on. In the case of illumination from a certain solid angle, as the angle between the surface normal and the incident light increases, the irradiated surface increases, and as the angle increases, the irradiance (W / m 2) decreases.

にわたる放射照度Ｉ（Ｗ／ｃｍ２）の図を示す。照射される皮膚の放射照度は、照明４の方向に垂直な表面積Ｓｐに対して最大である。表面積Ｓ１（ｔ＝１における）は、表面積Ｓｐより係数

分大きい。皮膚の向きが

から

に変化するとき、照射は図２Ｂに示されるように変化する。これにより、照度は

として表される。時間における方向変化

は、時間における放射照度の変化Ｉ（ｔ）へと変換される。 This is shown in FIG. 2B which shows the effect of illumination of the skin surface with different orientations of the skin surface. FIG. 2A shows an enlarged portion of the skin 2 and FIG.

A diagram of the irradiance I (W / cm 2) over is shown. The irradiance of the irradiated skin is maximum with respect to the surface area Sp perpendicular to the direction of the illumination 4. Surface area S1 (at t = 1) is a coefficient from surface area Sp

Minutes big. Skin orientation

From

When changing to, the illumination changes as shown in FIG. 2B. As a result, the illuminance is

Represented as: Direction change in time

Is converted to a change in irradiance I (t) over time.

となる。 In particular

It becomes.

に対する相対的な脈動性Ｐ（％）の図を示す。図から分かるように、皮膚表面の１°に対する周期的な向きの変化は、垂直照明の場合の反射光の無視できるほどの脈動性を与えるが、４５°以上の角度については急速に増加する。図３Ａの上半分において、面Ｒ１（ランバート面とも呼ばれる）は、入射光に対して垂直である（即ちα１＝９０°）。その結果、小さな面積Ａ１が最大輝度（単位面積当たりの強度）で照明される。図３Ｂの下半分において、表面Ｒ２は、入射光に対して角度α２<９０°の角度で配置される。その結果、より大きな面積Ａ２>Ａ１が、より低い輝度で照明される。 When the irradiance changes as described above, the diffuse radiation (light radiated from the skin) also changes, which may obfuscate the true PPG signal important for vital sign measurement such as HR, RR, Sp02 and the like. Assuming a skin surface rotation of 1 ° for illumination (eg, incident light) provided by, for example, heart-beating palpation, the reflected light exhibits an intensity variation in pulse frequency, as shown in FIGS. 3A and 3B. Thereby, FIG. 3A shows a diagram illustrating the effect of different skin orientations, and FIG.

The figure of relative pulsation P (%) with respect to is shown. As can be seen, the periodic orientation change to 1 ° of the skin surface gives negligible pulsation of the reflected light in the case of vertical illumination, but increases rapidly for angles above 45 °. In the upper half of FIG. 3A, the plane R1 (also called Lambertian plane) is perpendicular to the incident light (ie α1 = 90 °). As a result, the small area A1 is illuminated with the maximum luminance (intensity per unit area). In the lower half of FIG. 3B, the surface R2 is arranged at an angle α2 <90 ° with respect to the incident light. As a result, a larger area A2> A1 is illuminated with lower brightness.

、例えば４５度より大きな角度からの照明が防止される。その結果、対象の動きによる放射照度における変動（ノイズ）は、対象の固定照明を適用するシステムに比べて大幅に低減される。これは、本発明の効果を示す図４において見られることができる。 The present invention minimizes the amplitude of variation in irradiance by adapting the skin illumination angle. In one embodiment, the angle of illumination is controlled to be substantially perpendicular to or as close as possible to the average skin direction of the region of interest (ROI). In another embodiment, at least a large angle

For example, illumination from an angle larger than 45 degrees is prevented. As a result, fluctuations (noise) in the irradiance due to the movement of the object are significantly reduced compared to a system that applies fixed illumination of the object. This can be seen in FIG. 4 which shows the effect of the present invention.

、時間ｔにわたる放射照度Ｉ、及び角度

に対する放射照度Ｉを示す。ランプ位置Ａでは、放射照度は、皮膚の向きにおける周期的な変化のため、非常に周期的である。位置Ｂでは、放射照度における周期性がやや緩和される。即ち、時間的変動はより小さく、全体放射照度レベルは位置Ａよりも高い。ランプ位置Ｃでは、皮膚の向きの影響が最小化される。更に、放射照度における変動は著しく減少し、追加的に、周波数は倍増する。 FIG. 4 in particular shows the angle over time t for three different illumination angles (or lamp positions called positions A, B and C).

, Irradiance I over time t, and angle

The irradiance I with respect to is shown. At lamp position A, the irradiance is very periodic due to the periodic change in skin orientation. At position B, the periodicity in irradiance is slightly relaxed. That is, the temporal variation is smaller and the overall irradiance level is higher than position A. At ramp position C, the effect of skin orientation is minimized. Furthermore, the variation in irradiance is significantly reduced and additionally the frequency is doubled.

  FIG. 5 shows a schematic diagram of a first embodiment of the system 100 for obtaining vital sign related information of the living body 3 according to the present invention. The system 100 receives at least one wavelength interval of light 31 reflected from the skin region 2 of the living body 3 (here, the chest wall; the other regions are the hand, arm, abdominal region, forehead, check, etc.) It has a detection unit 30, for example a camera, which generates an input signal 32 representing vital sign-related information from which a signature can be obtained from the received light 31. For example, an illumination unit 40 having two or more light sources (eg, LEDs) 42 illuminates the skin region 2 with light 41. The direction estimation unit 16 estimates the direction 26 of the skin region 2. There are various options for this step, as will be described in more detail below. The control unit 18 controls the illumination unit 40 with a control signal 28 based on the estimated direction 26 of the skin region 2.

  The direction estimation unit 16 and the control unit 18 may be part of a separate device 10. This device comprises in this embodiment an input unit 12, for example a signal interface, which receives an input signal 32 from the detection unit 30. Furthermore, the device 10 may comprise a processing unit 14 that processes the input signal 32 and obtains vital sign information 36 of the living body 3 from the input signal 32, preferably using photoplethysmography.

  One or more units of device 10 may be included in one or more digital or analog processors, depending on how the invention is applied. The different units may be implemented in a personal computer that is fully or partially implemented in software and connected to one or more detectors. Some or all of the necessary functions can also be realized in hardware, for example in an application specific integrated circuit (ASIC) or in a field programmable gate array (FPGA).

  The system 100 shown in FIG. 5 can be placed in, for example, a hospital, a medical facility, an elderly care facility, and the like. In addition to patient monitoring, the present invention can also be applied in other areas such as neonatal monitoring, general surveillance applications, security monitoring, or so-called live-style environments such as fitness equipment. Unidirectional or bidirectional communication between the device 10, the detection unit 30 and the illumination unit 40 can operate via wireless or wired communication. Other embodiments of the invention can include the device 10. It is not provided as a stand-alone device, but one or more elements of the device 10, for example the direction estimation unit 16 and the control unit 18 may be integrated into the detection unit 30 or the lighting unit. The processing unit 14 can be integrated into another device, such as a nurse station workstation, or a hospital network.

  In general, the interaction of electromagnetic radiation, particularly light and biological tissue, is complex and includes (optical) processes of (multiple) scattering, backscattering, absorption, transmission and (diffuse) reflection. The term “reflecting” as used in the context of the present invention is not limited to specular reflection but includes electromagnetic radiation, particularly the aforementioned types of interactions between light and tissue, and any combination thereof. .

  The term “vital sign” as used in the context of the present invention refers to the physiological parameters and derived parameters of a subject (ie, a living body). In particular, the term “vital sign” refers to blood volume pulse signal, heart rate (HR) (sometimes also called pulse rate), heart rate variability (pulse rate variability), pulsation intensity, perfusion, perfusion index, perfusion variability, Traube. This includes the concentration of substances in the blood and / or tissues such as Hering Mayer waves, respiratory rate (RR), skin temperature, blood pressure, (arterial) blood oxygen saturation or glucose levels. In addition, a “vital sign” generally includes a health condition derived from the shape of the PPG signal (eg, the shape includes some information about partial arterial occlusion (eg, applying a blood pressure cuff to the arm, May contain more information about skin thickness (eg, PPG signal from the face is different from that of the hand) or even information about temperature ).

  The term “vital sign related information” as used in the context of the present invention includes information from which one or more vital signs as defined above can be obtained. In addition, it includes data referring to physiological parameters, corresponding waveform traces, or data referencing physiological parameters of time that may be useful for later analysis.

  In the embodiment shown in FIG. 5, the detection unit 30 (in particular for obtaining a sequence of image frames over the time of at least the skin region 2 where a photoplethysmographic signal can be obtained) Having a camera (also called an imaging unit, or camera-based or remote PPG sensor) that includes a suitable light sensor to capture remotely and unobtrusively. The image frames captured by the camera can in particular correspond to a video sequence captured using an analog or digital photosensor, for example in a (digital) camera. Such a camera typically includes a photosensor such as a CMOS or CCD sensor. It can then operate in a specific spectral range (visible, IR) or provide information regarding different spectral ranges. The camera can provide an analog or digital signal. An image frame includes a plurality of image pixels having associated pixel values. In particular, the image frame includes pixels that represent light intensity values that are captured by different photosensitive elements of the photosensor. These photosensitive elements may be sensitive in a particular spectral range (ie representing a particular color). The image frame includes at least some image pixels representing the skin portion of interest. This allows an image pixel to correspond to a single photosensitive element of the photodetector and its (analog or digital) output, or a combination of multiple photosensitive elements (eg, via binning). Can be determined based on

  To obtain a vital sign information signal of interest, the data signal of the skin pixel area in the skin area is evaluated. Here, “skin pixel region” means a region including one skin pixel or a group of adjacent skin pixels, ie, a data signal can be obtained for a single pixel or a group of skin pixels.

  The lighting unit 40 in this embodiment has at least two lighting elements 42, ie light sources such as LEDs or other light emitting elements. In the exemplary embodiment, the direction estimation unit 16 determines which of these at least lighting elements 42 is closest along the skin surface normal (or the surface method of the largest portion of the skin surface if the skin surface is not flat). Determine whether to emit light (closest to the line). When this lighting element is established, the other lighting elements are, for example, dimmed or completely extinguished. As a result, in the input signal 32 (and the resulting PPG signal) carried in the light 31 reflected (diffusely) from the skin, the motion-inducing signal reflected from the skin is minimized.

  In one embodiment, an image acquired by a camera representing detection unit 30 in system 100 to determine which of lighting elements 42 radiate closest along the surface normal of the skin (sequentially). Data can be used. When approximating the skin as a diffuse or ideally as a Lambertian reflector, the illumination element that produces the best illumination measured by the camera and evaluated by the direction estimation unit 16 is best for that skin (and part). Possible (optionally used for actual vital sign determination in the optional processing unit 14, so the direction estimation unit 16 informs the processing unit 14 accordingly). At regular or irregular intervals or continuously, after a substantial movement of the object 3, these steps representing a kind of calibration are repeated to adaptively control the illumination, in particular the irradiation angle of the skin area 2. Can be done.

  The lighting unit 40 may generally be configured as a one-dimensional or two-dimensional array of lighting elements 42. The lighting element 42 can be arranged in a flat or slightly curved frame. Preferably, the lighting elements 42 can be controlled individually.

  In general, the control unit 18 is configured to control the intensity, direction, distribution, and / or illumination angle of light emitted by the illumination unit 40.

  FIG. 6 shows a schematic diagram of a second embodiment of the system 101 according to the invention. In this embodiment, an additional distance data acquisition unit 50 is provided, and distance data 52 representing the distance between the distance data acquisition unit 50 and the skin region 2 is radiated to the object and reflected from the object. Use to get. The distance data acquisition unit 50 can be, for example, a time of flight (TOF) camera or a radar unit. From the obtained distance data 52, the direction estimation unit 16 can determine the orientation of the skin surface of the skin region 2 or generate a 3D model of the skin surface of the skin region 2. This provides another simple option to estimate the skin area orientation in a fairly accurate manner. However, in other embodiments, other methods for establishing a 3D model can be employed.

  Furthermore, in this embodiment, another realization of the lighting unit 60 is shown. The lighting unit 60 comprises a single lighting element 61 and a shutter element 62 including a switchable shutter that allows or prevents the passage of light from different areas of the lighting element 61 (mechanically or electronically). Have. For example, an LCD may be used as the lighting element 61 with an array of shutters that can be individually controlled between the light emitting element 61 and the object 3 (ie, controlled by the control signal 26).

  Such a lighting unit 60 can also be used in other embodiments of the proposed system, for example the system 101 shown in FIG. 5, and the lighting unit 40 is in the embodiment of the system 101 shown in FIG. It can also be used.

  For example, in a further embodiment, used with the elements of the system 100 shown in FIG. 5, the skin produces a specular reflection or reflection that is not an ideal Lambertian reflector, ie, does not follow Lambert's “cosine law”. It can then be assumed. This model may be particularly useful when shorter wavelengths are used where diffuse reflection is significantly absorbed. In this case, the camera can see the highest brightness from a lighting element whose incident angle is equal to the camera angle (maximum specular reflection). If the camera position is constant, then it is possible to determine the position of the illumination element that illuminates approximately along the surface normal.

  In yet another embodiment, two adjacent illumination elements 42 of the plurality of illumination elements 42 of the illumination unit 40 (eg, LED array) alternately emit equal intensity light (although only slightly relative to the surface normal). Different angles). The sets of lighting elements 42 at different positions in the lighting unit 40 alternate at different frequencies. The light 31 reflected from the skin 2 now exhibits all different frequency components, in particular all different frequency components, but the luminescent element 42 emitting closest to the surface normal gives the lowest frequency amplitude. As a result, from the (modulation) spectrum of the reflected light 31 it can be determined which orientation of the illumination element 42 is closest to the surface normal.

  In yet another embodiment, the spectrum can be measured with a camera indicated by skin region 2, although higher modulation frequencies may be possible with photodiodes where skin reflected light is focused. Yet another way is to run the camera faster (so-called “binning” option of the camera, which can be done with lower resolution).

  FIG. 7 shows a schematic diagram of a third embodiment of the system 102 according to the invention. In this embodiment, the detection unit 30 includes a camera 33 that generates the input signal 32 from the light 31 received by the camera 33, and a photodiode 34 that generates the reflected signal 35 from the light 31 received by the photodiode 34. Have In addition, the lighting unit 40 illuminates the skin area 2 with a first set 43 of illumination elements 44 that allows the camera 33 to generate an input signal 31, and illuminates the skin area 2 with the photodiode 34. And a second set 45 of illumination elements 46 that allow the generation of the reflected signal 35.

  Preferably, the wavelengths used by the first set 43 of illumination elements 44 and the second set 45 of illumination elements 46 are different. That is, the illumination element 46 is configured to illuminate the skin region 2 with light having a wavelength or wavelength range outside the sensitivity interval of the camera 33. Further, the number of lighting elements 44 and lighting elements 46 may be different.

  Two adjacent illumination elements may also be separated by means other than the modulation frequency, such as the use of slightly different wavelengths, sequential illumination, synchronous flickering at the same frequency.

  In the above-described embodiment, the control unit 18 controls the illumination unit 40 based on the estimated orientation of the skin region 2. In other embodiments, the control unit 18 may additionally or alternatively control the processing unit 14 based on the estimated orientation of the skin region 2. In particular, when the lighting unit 40 illuminates the skin area 2 from different illumination angles in subsequent time intervals, the processing unit 14 may select during the time interval selected based on the estimated orientation of the skin area, i.e. the skin area. Can be controlled to obtain vital sign related information 36 from the input signal obtained during the time interval during which is optimally illuminated. This is done by a control signal 29 from the control unit 18 to the processing unit 14, as shown in FIG. 7, but can also be provided in other embodiments.

  For example, in one embodiment, an input obtained during a time interval during which a lighting element that produces a reflected signal with the strongest intensity illuminates the skin area exclusively or with the strongest intensity of all the lighting elements. Only the signal 32 is used by the processing unit 14. In another embodiment, pairs of lighting elements that produce a reflected signal with minimum intensity modulation were obtained during a time interval that illuminates the skin area exclusively or with the strongest intensity of all lighting elements. Only the input signal 32 is used by the processing unit 14. Thus, in such an embodiment, the lighting unit can continuously illuminate the skin area 2 in the same manner and optionally without additional control.

  FIG. 8 shows a schematic diagram of a fourth embodiment of the system 103 according to the invention, in which the elements of the device 10 are not explicitly shown. In this embodiment, the illumination unit and the detection unit are embedded in the radiant warmer 60. Infants are often placed in such radiant warmers 60 to help maintain the infant's body temperature. This gives better access to the infant for treatment, care and treatment.

  The radiant warmer 60 typically has an IR warming lamp 62 centered on the infant 2. The means for holding the heating element 62 is the ideal place to integrate the detection unit 30 in the form of a camera, here as described above, at the front end with this heating element 62 in the same position as space allows. The arm 63 carrying the heating element 62 can also carry the camera 30 inside (it is essentially invisible because it only requires a few holes). In addition, the arm includes a lighting element 42 of the lighting unit 40, here in the form of an array around the camera 30. Cabling can be completely managed inside the warmer 60, in particular the arm 63. Device 10 may also be integrated into radiant warmer 60. Further, the radiant warmer 60 can share a user interface and any additional potential computing means that may be required for monitoring and warming functions. In a similar manner, some or all elements of the proposed system can be integrated into the incubator.

  A flowchart of an embodiment of the method according to the invention is shown in FIG. In the first step S10, at least the skin region 2 of the living body 3 is irradiated with light. In the second step S12, the light 31 having at least one wavelength interval reflected from the skin region 2 is received. In the third step S <b> 14, an input signal 32 representing vital sign related information for obtaining a vital sign of a living body is generated from the received light 31. In the fourth step S <b> 16, the input signal 32 is processed, and vital sign information 36 of the living body is obtained from the input signal 32. In the fifth step S18, the orientation of the skin region 2 is estimated. In a sixth step S18, the lighting unit 40 illuminates the skin region based on the estimated orientation of the skin region 2 and / or obtains vital sign related information 36 based on the estimated orientation of the skin region. Therefore, the above process is executed.

  In summary, a vital sign measurement device obtains vital sign information by measuring subtle changes in the skin region of the region of interest. This depends on the illumination. Usually dedicated lighting is required. However, it has been found that a particular preset illumination is not always optimal for measurement. For example, specular reflection is one of the difficulties, especially for SpO2 measurement, and this should be avoided in the region of interest (ROI) used for the measurement. Due to different skin types (conditions) and body postures or body movements, specular reflections may exist in the ROI or may appear in the ROI. Manually adjusting the lighting settings for each measurement or after each change in the environment or vital sign measurement device is subjective and time consuming.

  Accordingly, the present invention proposes an adaptive device and method for conservative vital sign measurements (eg, heart rate monitoring, SpO2 monitoring, etc.) that can be automatically configured for optimal measurement. Accordingly, a control unit is provided for controlling the illumination unit, eg one or more controllable light sources, and / or a processing unit for obtaining vital sign related information based on the estimated direction of the biological surface. The vital sign related information is, for example, the vital sign itself or a signal from which the vital sign can be derived. Therefore, even when conditions change, vital sign related information can be obtained with optimal accuracy and reliability.

  The present invention can be applied to various uses. Heart rate, respiratory rate, and SpO2 are very important elements in patient monitoring and home care, and remote heart rate monitoring becomes increasingly important. Furthermore, the present invention can be applied to register beats in fitness equipment. The proposed invention can be applied in particular to any application where (non-contact) camera-based vital signs monitoring is performed with varying controllable illumination or variable light conditions. Typical application areas are, for example, premature babies with very sensitive skin in NICU and patients with damaged (eg burned) skin. Application locations include NICU, emergency waiting room, general ward, nursing home, home care, psychiatric ward / prison. Usually, the collection of vital signs is very difficult and in some cases impossible, but this can be realized accurately and reliably. Furthermore, the present invention can also be applied to contact PPG devices such as contact pulse oximeters or finger clip sensors for SpO2 measurement.

  While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive. The invention is not limited to the disclosed embodiments. From a study of the drawings, the disclosure and the appended claims, other variations to the disclosed embodiments can be understood and carried out by those skilled in the art practicing the claimed invention.

  In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. A single element or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measured cannot be used to advantage.

  The computer program can be stored / distributed in suitable non-transitory media, such as optical storage media or solid media supplied with or as part of other hardware, but the Internet or other wired or wireless communications It can also be distributed in other formats, such as through a system.

  Any reference signs in the claims should not be construed as limiting the scope.

Claims (15)

A device for acquiring vital sign related information of a living body,
An input unit for receiving an input signal generated from light received at at least one wavelength interval reflected from a skin region of a living body, wherein the input signal is a vital for obtaining a vital sign of the living body An input unit representing sign-related information;
A processing unit that processes the input signal and obtains vital sign related information of the living body from the input signal;
A direction estimation unit for estimating the orientation of the skin region;
In order to illuminate the skin area based on the estimated orientation of the skin area, a lighting unit that illuminates the skin area with light is controlled and / or a time interval selected based on the estimated direction of the skin area A control unit for controlling the processing unit to obtain vital sign related information from the input signal obtained in between.   The device of claim 1, wherein the control unit controls the intensity, direction, distribution, and / or illumination angle of at least a portion of the light emitted by the lighting unit. The direction estimation unit estimates a direction of a surface normal of the skin region;
The control unit controls the intensity, direction, distribution, and / or illumination angle of at least some of the light emitted by the illumination unit, most or all of the light being close to the estimated surface normal The device of claim 1, wherein the device is emitted from the same illumination angle.   The control unit is selected based on the estimated orientation of the skin area by controlling the illumination unit to illuminate the skin area from different illumination angles in the next time interval and controlling the processing unit. The device of claim 1, wherein vital sign related information is obtained from the input signal obtained during a time interval.   The device according to claim 1, wherein the direction estimation unit estimates the orientation of the skin region regularly or continuously, and the control unit adjusts the control of the lighting unit based on this.   The device of claim 1, wherein the direction estimation unit processes distance data to determine a 3D model of a skin surface that includes the skin region. A system for acquiring vital sign related information of a living body,
A detection unit that receives light in at least one wavelength interval reflected from a skin region of a living body and generates an input signal representing vital sign related information from which the biological signal of the living body can be obtained from the received light; ,
A lighting unit for illuminating the skin area with light;
The system according to claim 1, further comprising obtaining vital sign related information of a living body from the input signal.   The system according to claim 7, wherein the lighting unit comprises two or more lighting elements.   9. The system of claim 8, wherein the two or more lighting elements are arranged at different positions and / or different orientations to illuminate the skin area from different illumination angles.   The system according to claim 8 or 9, wherein the control unit individually controls the lighting elements. The control unit controls the lighting element to illuminate the skin area in a multiplexed and / or modulated manner;
The detection unit receives light reflected from the skin region based on the subsequent illumination, generates a reflected signal from the received light,
The direction estimation unit determines the reflected signal with the strongest intensity;
The control unit further controls the lighting unit such that the lighting element producing the reflected signal with the strongest intensity illuminates the skin area exclusively or with the strongest intensity of all the lighting elements; and Vital sign related from the input signal obtained during the time interval when the lighting element producing the reflected signal with the strongest intensity illuminates the skin area exclusively or with the strongest intensity of all the lighting elements The system according to claim 8 or 9, wherein the processing unit is controlled to obtain information. The control unit controls the lighting elements to illuminate the skin area in pairs;
Different pairs of adjacent lighting elements alternately illuminate the skin area at different alternating frequencies, at different wavelengths, at different times, and / or with synchronized flicker,
The detection unit receives light reflected from the skin region based on the alternating illumination, and generates a reflected signal from the received light;
The direction estimation unit determines the reflected signal with the smallest intensity modulation;
The control unit is further adapted so that the illumination pair that produces a reflected signal with the minimum intensity modulation illuminates the skin area exclusively or with the strongest intensity of all the lighting elements, or the minimum intensity modulation Controlling the illumination unit so that only illumination in a time slot of illumination that produces a reflected signal having the following is used to illuminate the skin area and / or producing a reflected signal having the minimum intensity modulation Controlling the processing unit to obtain vital sign related information from the input signal obtained during a time interval that illuminates the skin area exclusively or with the strongest intensity of all lighting elements; The system according to claim 9. The detection unit includes a camera that generates the input signal from received light, and a photodiode that generates the reflected signal from received light;
The lighting unit includes a first set of lighting elements that illuminate the skin area to allow generation of an input signal by the camera, and illumination that illuminates the skin area to allow generation of a reflected signal by the photodiode. 13. The system of claim 12, comprising a second set of elements. In a method for obtaining a vital sign of a living body,
Receiving an input signal generated from light received at at least one wavelength interval reflected from a skin region of a living body, wherein the input signal is a vital sign capable of obtaining a vital sign of the living body Steps representing relevant information,
Processing the input signal to obtain vital sign information of the living body from the input signal;
Estimating the orientation of the skin area;
Based on the estimated orientation of the skin area, controls lighting of the skin area with light and / or vitals from the input signal obtained during a selected time interval based on the estimated orientation of the skin area Controlling the process to obtain signature related information.   15. A computer program that, when executed by a computer, causes the computer to perform the steps of the method of claim 14.

Images