WO2021045140A1 - Information processing device, pulse wave measurement system, and pulse wave measurement program - Google Patents
Information processing device, pulse wave measurement system, and pulse wave measurement program Download PDFInfo
- Publication number
- WO2021045140A1 WO2021045140A1 PCT/JP2020/033394 JP2020033394W WO2021045140A1 WO 2021045140 A1 WO2021045140 A1 WO 2021045140A1 JP 2020033394 W JP2020033394 W JP 2020033394W WO 2021045140 A1 WO2021045140 A1 WO 2021045140A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- pulse wave
- information processing
- subject
- unit
- image
- Prior art date
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
Definitions
- the present invention relates to an information processing device, a pulse wave measurement system, and a program that detect a pulse wave from an image or signal of a subject.
- Patent Document 1 discloses a device that calculates a difference in luminance information from images taken at different parts of a subject's body taken at the same time and measures a change in blood pressure according to an increase or decrease in the difference. There is.
- the RGB signal extracted from the acquired measurement target person image is converted into a YCbCr signal, and the face region and the target region ROI are detected from the luminance signal Y. Then, from the Cb + Cr signal acquired from the extracted target region ROI, a Cb + Cr trace signal in which the value at an arbitrary time is plotted is generated to detect the pulse wave.
- Patent Document 4 discloses a method of detecting a pulse wave by using an information terminal such as a smartphone.
- Patent Document 1 needs to obtain captured images of different parts of the subject's body at the same time, but depending on the posture, clothes, camera position, etc. of the subject, the subject's body at the same time. It can be difficult to obtain captured images of different parts of the body. Further, the techniques described in Patent Documents 2 and 3 are inferior in versatility because the apparatus configuration becomes large, and are not suitable for daily use.
- Patent Document 4 is suitable for daily use because information is acquired by a smartphone, but the pulse wave is calculated from the image of the fingertip, so that the detection accuracy is improved. There are challenges.
- an object of the present invention is to provide a highly accurate and general-purpose information processing device, a pulse wave measurement system, and a pulse wave measurement program.
- the information processing apparatus includes an extraction unit that extracts data in a specific range from the brightness value distribution of the target person's skin region included in an image of the target person based on the peak value, and the extraction unit.
- a normalization unit that normalizes the data so that the peak height becomes 1
- a calculation unit that calculates the standard deviation with respect to the normalized data, and the standard deviation are arranged in chronological order. It has an output unit that outputs the pulse wave of the subject.
- the information processing device of the present invention may further have a receiving unit for receiving the photographed image from the photographing device.
- the output unit can also output the pulse wave of the subject to an external device.
- the information processing apparatus of the present invention may have a removing portion for removing the luminance unevenness of the skin region of the subject included in the image, and in that case, the extracting portion may have the luminance unevenness due to the removing portion.
- the data is extracted from the removed luminance value distribution.
- the removal unit divides the skin region into a plurality of regions, calculates a luminance value distribution for each of the divided regions, aligns the peak position of each luminance value distribution with a reference position, and sums all the luminance value distributions. It is also possible to obtain the brightness value distribution from which the brightness unevenness is removed.
- the image may be taken by using two or more near-infrared lights having different wavelengths, and in that case, the extraction unit may distribute the brightness value of the near-infrared light component. Extract data from.
- the skin region of the subject included in the captured image is divided into a plurality of regions, the standard deviation of the brightness value is obtained for each of the divided plurality of regions, and the obtained standard deviation is constant. The presence or absence of the subject's pulse is determined based on the proportion of the region below the value.
- the pulse wave measurement system includes a photographing device for photographing a skin area of a subject and an information processing device described above.
- the photographing device may include a solid-state imaging device that detects at least two or more near-infrared lights having different wavelengths, and may transmit an image containing the near-infrared light component to the information processing device.
- the pulse wave measurement program has an extraction function for extracting data in a specific range from the brightness value distribution of the skin region of the subject included in an image of the subject, and the above-mentioned.
- a normalization function that normalizes the peak height to 1
- a calculation function that calculates the standard deviation for the normalized data, and the standard deviations are arranged in chronological order.
- the computer is made to execute the output function of outputting the pulse wave of the subject.
- an information processing device a pulse wave measurement system, and a pulse wave measurement program that can detect pulse waves with high accuracy and have high versatility.
- FIG. 1 A is a diagram showing a range of data extracted from the luminance value histogram in the extraction unit 302 shown in FIG. 3, and B is a diagram showing the extracted data.
- a and B are diagrams showing the data normalization process in the normalization unit 303 shown in FIG. 3, where A shows the state before the normalization process and B shows the state after the normalization process.
- FIG. 1 It is a figure which shows the output example from the output part 303. It is a block diagram which shows the structural example from the functional aspect of the information processing apparatus in the pulse wave measurement system of the 1st modification of 1st Embodiment of this invention. It is a flowchart which shows the processing process in the information processing apparatus 101 shown in FIG. It is a figure which shows the data before surface division in a luminance unevenness removal step (S12). It is a figure which shows the data after surface division in a luminance unevenness removal step (S12). It is a figure which shows the data before the peak position adjustment in a luminance unevenness removal step (S12). It is a figure which shows the data after the peak position adjustment in a luminance unevenness removal step (S12). It is a figure which shows the luminance value histogram after the luminance unevenness removal processing.
- FIG. 1 is a block diagram showing a configuration example of the pulse wave measurement system of the present embodiment.
- the pulse wave measurement system of the present embodiment includes a photographing device 110 and an information processing device 100, and the photographing device 110 and the information processing device 100 can communicate with each other via wired or wireless communication. It is connected.
- the photographing device 110 photographs the skin region of the subject and generates a captured image including the skin region.
- the photographing device 110 transmits the generated photographed image to the information processing device 100.
- a dedicated camera may be used for the photographing device 110, but a camera built in the mobile information terminal or a camera installed in the vehicle or indoors for other purposes can also be used.
- the information processing device 100 processes the captured image received from the photographing device 110 and outputs a pulse wave.
- the pulse wave is detected from the fluctuation of the brightness average value of the skin region of the subject included in the captured image, but in the pulse wave measurement system of the present embodiment, the skin region of the subject is detected. Focusing on the histogram of the brightness value of, the pulse wave is detected from the shape change.
- FIG. 2 is a block diagram showing an example of the hardware configuration of the information processing device 100.
- the information processing device 100 can be configured to include, for example, a control unit 201, a storage unit 202, an input unit 203, a display unit 204, and a communication unit 205.
- the control unit 201 controls the entire information processing device 100, and a CPU (Central Processing Unit) or the like can be used.
- a CPU Central Processing Unit
- the storage unit 202 stores a program for executing processing, data used when the control unit 201 executes processing based on the program, and the like, and is an HDD (Hard Disk Drive), ROM (Read Only Memory). , RAM (Random Access Memory) or the like is used. Then, the control unit 201 executes the process based on the program stored in the storage unit 202, thereby realizing the functions and processes of the information processing device 100 described later.
- HDD Hard Disk Drive
- ROM Read Only Memory
- RAM Random Access Memory
- the input unit 203 inputs the user's operation to the control unit 201, and is composed of buttons and the like.
- the display unit 204 displays the processing result of the control unit 201 and the like, and a display device such as a display can be used.
- the communication unit 205 connects the information processing device 100 to the network, and a NIC (Network Interface Card) or the like is used.
- NIC Network Interface Card
- FIG. 3 is a block diagram showing a configuration example from the functional aspect of the information processing apparatus 100.
- the information processing apparatus 100 when it is viewed from the functional aspect, it can be configured to include a receiving unit 301, an extracting unit 302, a normalizing unit 303, a calculating unit 304, and an output unit 305.
- the receiving unit 301 receives the captured image from the photographing device 110, and the extracting unit 302 uses the peak value as a reference from the luminance value histogram of the target person's skin area included in the captured image received by the receiving unit 301 to specify a range. Extract the data of.
- the luminance value histogram is an example of the luminance value distribution.
- the normalization unit 303 normalizes the data extracted by the extraction unit 302 so that the peak height becomes a predetermined value.
- the calculation unit 304 calculates the standard deviation of the data normalized by the normalization unit 303.
- the output unit 305 arranges the standard deviations calculated by the calculation unit 304 in chronological order and outputs and displays them as pulse waves of the target person, for example, on the display unit 204 or transmits them to an external device via the communication unit 205. To do.
- the pulse wave measurement system of the present embodiment First, an image including the skin of the subject is photographed by the photographing device 110, and the photographed image is transmitted to the information processing device 100. Next, in the information processing apparatus 100, the pulse wave is measured by capturing the peak position of the luminance value histogram and capturing the shape change of the luminance value histogram. In order to capture the shape change of the histogram of the luminance value, the information processing apparatus 100 uses the standard deviation of the data related to the histogram of the luminance value as a feature amount, and detects the pulse wave by the fluctuation of the standard deviation.
- FIG. 4 is a flowchart showing a processing process in the information processing apparatus 100. Specifically, as shown in FIG. 4, in the information processing apparatus 100, when the receiving unit 301 receives the captured image (step S1), the extraction unit 302 includes a luminance value histogram of the skin region of the target person included in the captured image. Then, data in a specific range is extracted with reference to the peak value (step S2).
- FIG. 5A is a diagram showing a range of data extracted from the luminance value histogram by the extraction unit 302
- FIG. 5B is a diagram showing the extracted data.
- the peak value of the luminance value histogram is set to 100%, and data in the range up to n% (the range surrounded by the ellipse in FIG. 5A) is extracted.
- the data shown in FIG. 5B is extracted.
- the range of the data to be extracted (n% value shown in FIG. 5A) can be appropriately set according to the shape of the luminance value histogram and the like.
- FIG. 6A and 6B are diagrams showing the data normalization process in the normalization unit 303, FIG. 6A shows the state before the normalization process, and FIG. 6B shows the state after the normalization process.
- the normalization unit 303 performs normalization so that the peak amount H of the extracted data shown in FIG. 6A is 1, for example. As a result, the data shown in FIG. 6B can be obtained. By performing this normalization process, it is possible to capture only the change in shape by the standard deviation without depending on the number of extracted data.
- the calculation unit 304 calculates the standard deviation of the normalized data (step S4).
- the result (standard deviation) is arranged in chronological order to form a waveform, which is output from the output unit 305 as a pulse wave of the subject (step S5).
- FIG. 7 is a diagram showing an example of output from the output unit 303.
- the processing result of the information processing apparatus 100 is generated, for example, on the display unit 204 by arranging the captured image showing the skin region used for pulse wave detection and the standard deviation in chronological order as shown in FIG. It is also possible to display the pulse wave side by side.
- Each of the above-mentioned steps can be carried out by creating a computer program for realizing the functions of each part of the information processing apparatus 100 and implementing it on one or more computers. That is, an extraction function that extracts data in a specific range based on the peak value from the brightness value distribution of the target person's skin area included in the image of the target person, and a predetermined peak height for the extracted data. A normalization function that normalizes to the value of, a standard deviation calculation function that calculates the standard deviation for the normalized data, and a pulse wave that outputs the pulse wave of the subject by arranging the standard deviation in chronological order.
- a program that causes a computer to execute an output function can be executed by operating the computer or an image processing device.
- the pulse wave detection system of the present embodiment simplifies the device because it is not necessary to capture a plurality of images and the pulse wave of the target person can be detected from the captured images at one place. It can be used and is highly versatile.
- a method of determining a measurement target area and detecting a pulse by averaging the brightness in the area is adopted. This method utilizes the fact that the brightness level decreases when blood is flowing by taking the average value in time series because hemoglobin in the blood absorbs light, but the absorption of light is slight. Since the change in the average luminance value is a fluctuation at a level of 1/10 or less, it is vulnerable to the movement of an object and the fluctuation of ambient light.
- the pulse wave detection system of the present embodiment extracts data in a specific range, normalizes it, and takes a standard deviation, so that it is not easily affected by the movement of an object or fluctuation of ambient light. , The pulse wave can be detected with high accuracy. Further, in general, processing such as standard deviation and dispersion is performed on a pixel-by-pixel basis, and there is a concern that the processing speed may decrease. However, since the pulse wave detection system of the present embodiment uses a luminance value histogram, the processing speed is high. Processing speed is possible.
- the pulse wave measurement system of this embodiment can be mounted on a car, for example.
- the skin region of the driver and / or the passenger is photographed by the photographing device 110, the photographed image is processed by the information processing device 100, and the pulse wave of the driver and / or the passenger is detected and output. This makes it possible to detect changes in the physical condition of the driver or the like.
- the photographing device 110 and the information processing device 100 are separate bodies, but the present invention is not limited to this, and the photographing device 110 and the information processing device 100 are integrated. It may be configured.
- the function of the information processing device 100 described above may be mounted on the photographing device 110, or both the function of the processing device 100 and the function of the photographing device 110 may be mounted on the information terminal information with a camera.
- FIG. 7 shows an example in which a part of the captured image is used as the detection area
- the present invention is not limited to this, and the entire field of view of the photographing device 110 may be set as the detection area.
- the processing by the information processing apparatus 100 may take time. Therefore, even if the subject is irradiated with infrared light, the reflecting light of the infrared light is detected by the photographing device 110, and the detection region (for example, the skin region of the subject) is automatically followed based on the amount of the light. Good. As a result, the processing time in the information processing apparatus 100 can be shortened.
- FIG. 8 is a block diagram showing a configuration example from the functional aspect of the information processing device in the pulse wave measurement system of this modified example
- FIG. 9 is a flowchart showing a processing process in the information processing device 101 shown in FIG.
- the same components as those of the information processing apparatus 100 shown in FIG. 3 are designated by the same reference numerals, and detailed description thereof will be omitted.
- the information processing device 101 is provided with a removal unit 801 in addition to the reception unit 301, the extraction unit 302, the normalization unit 304, and the output unit 305. There is.
- step S11 after the captured image is received by the receiving unit 301 of the information processing device 101 (step S11) and the brightness unevenness of the captured image is removed by the removing unit 801 (step S12).
- the extraction unit 302 data in a specific range is extracted from the luminance value histogram from which the luminance unevenness has been removed, with reference to the peak value (step S13).
- the normalization unit 303 performs the normalization process (step S14), and the calculation unit 304 calculates the standard deviation (step S15), and the obtained results are arranged in chronological order and output from the output unit 305 as a pulse wave. (Step S16).
- the removing unit 801 removes the uneven brightness of the skin region of the subject included in the captured image.
- 10 to 14 are diagrams for explaining the luminance unevenness removing process (luminance unevenness removing step S12) by the removing unit 801.
- the removing unit 801 divides the skin area (detection area) of the captured image into N pieces.
- N is a natural number of 2 or more
- each divided detection region has a size that allows the peak of the luminance histogram to be known.
- a luminance value histogram is calculated for each divided detection region.
- the removal unit 801 determines for each divided detection region whether or not the region can be used for pulse wave measurement or the like based on the shape of the luminance value histogram.
- the peak positions are adjusted to align the luminance peak positions for each luminance value histogram with the reference positions, the sum of all N histograms is calculated, and the sum of the sums shown in FIG. 14 is calculated.
- Get the data That is, the removal unit 801 creates a luminance value histogram of the entire region by combining and adding the corrected luminance value histograms. As a result, a luminance value histogram in which the luminance unevenness is removed can be obtained.
- data in a specific range is extracted by the extracting section 3023 with reference to the peak value.
- the detection area is divided, the brightness value histogram is calculated for each divided area, the peak position is adjusted, and then all the data are summed and combined.
- a luminance value histogram without is obtained.
- the light source is arranged in a fixed place as in infrared photography, and the level of reflected light fluctuates depending on the angle of the surface exposed to the light, such as when the subject is irradiated with light in a spot. Is particularly effective when using.
- the configurations and effects other than the above in the pulse wave measurement system of this modified example are the same as those in the first embodiment described above.
- the control unit 201 calculates the standard deviation of the brightness value for each of the surface-divided small regions, and determines the presence or absence of a pulse in each frame. Since the magnitude of the standard deviation indicates the degree of light reflection unevenness, the region of the luminance standard deviation larger than a certain value is a portion where the pulse component cannot be extracted well. Therefore, the control unit 201 determines that there is a pulse in the corresponding hum when the ratio of the small region in which the pulse component is successfully extracted to the whole is larger than the predetermined value, and when it is less than the predetermined value, it corresponds. Judge that there is no pulse in the humming.
- the control unit 201 divides the skin region of the subject included in the captured image into a plurality of regions, obtains the standard deviation of the brightness value for each of the divided plurality of regions, and the obtained standard deviation is a region of a certain value or less. Based on the ratio of, the presence or absence of the subject's pulse is determined and output.
- the pulse wave measurement system of the present embodiment can determine the presence or absence of a pulse.
- the configurations and effects of the pulse wave measurement system of the present embodiment other than the above are the same as those of the first embodiment described above.
- the pulse wave measurement system of the present embodiment uses an image pickup device equipped with an image pickup element capable of detecting two or more near-infrared lights having a wavelength different from that of the visible light detection pixel, and has a wavelength in addition to the visible light image.
- a near-infrared light image taken by two or more different near-infrared lights is taken.
- the imaging elements used at that time include, for example, three types of visible light pixels having different detection wavelengths described in International Publication No. 2018/155486 and International Publication No. 2018/207817, and two or more types having different detection wavelengths.
- Examples include those in which near-infrared light pixels are provided on the same substrate, but the present invention is not limited to these, and in addition to red light R, green light G, and blue light B, two or more types having different wavelengths are used. Anything that can detect near-infrared light (multi-spectrum of near-infrared light) is sufficient.
- the visible light pixel detects red light R, green light G, and blue light B
- the near-infrared light pixel detects near-infrared light that correlates with R, near-infrared light that correlates with G, and B.
- the image was taken with normal illuminance not only in an environment where the amount of visible light is small, but also in an environment where there is no visible light such as darkness (0 lux). It is possible to take an image having the same information as the case.
- the subject in the pulse wave measurement system of the present embodiment, can be photographed in a wide range of illuminance environments from normal illuminance to darkness (0 lux), and the pulse wave can be measured accurately regardless of the environment in which the subject is present. Is possible.
- the information processing apparatus 100 receives, for example, a color image generated from visible light, near-infrared light, or both of them from the photographing apparatus 110, and each component included in the color image is included.
- the pulse wave is detected from the brightness value histogram.
- pulse waves may be detected from changes in the ratio, difference, and logarithm (Log) of two near-infrared light components having different wavelengths.
- the detection signals of two types of near-infrared light having different wavelengths detected by the image sensor may be used as they are without generating a color image by the photographing apparatus.
- the output unit outputs the pulse wave detected based on the multi-spectrum of the near infrared ray.
- a pulse wave is generated by using two or more types of near-infrared light components having different wavelengths in addition to or in place of the visible light component. Since it is detected, the pulse wave can be detected more accurately.
- the configurations and effects of the pulse wave measurement system of the present embodiment other than the above are the same as those of the first embodiment and its modifications described above.
- the "part" may include, for example, a combination of hardware resources implemented by circuits in a broad sense and information processing of software that can be concretely realized by these hardware resources. .. Further, in this embodiment, various information is handled, and these information are represented by high and low signal values as a bit set of binary numbers composed of 0 or 1, and communication / calculation is executed on a circuit in a broad sense. Can be done.
- a circuit in a broad sense is a circuit realized by at least appropriately combining a circuit (Circuit), circuits (Circuitry), a processor (Processor), a memory (Memory), and the like. That is, an integrated circuit for a specific application (Application Special Integrated Circuit: ASIC), a programmable logic device (for example, a simple programmable logic device (Simple Programmable Logical Device: SPLD), a composite programmable logic device (Complex Program)) It includes a programmable gate array (Field Programmable Gate Array: FPGA) and the like.
Abstract
Provided are an information processing device, a pulse wave measurement system, and a pulse wave measurement program that are highly accurate and widely applicable. The present invention comprises: transmitting, to an information processing device 100, an image (captured image) of a subject captured by an imaging device; extracting, in an extraction unit 302 of the image processing device 100, data of a specific range from the brightness value distribution of a skin area of the subject included in the image, the specific range being relative to a peak value of the brightness value distribution; normalizing, in a normalization unit 303, the extracted data so as to have a peak height of one; calculating, in a calculation unit 304, standard deviations with respect to the normalized data; and arranging, in an output unit 305, standard deviations in a time-series order to output pulse waves of the subject.
Description
本発明は、対象者を撮影した画像又は信号から、脈波の検出を行う情報処理装置、脈波計測システム及びプログラムに関する。
The present invention relates to an information processing device, a pulse wave measurement system, and a program that detect a pulse wave from an image or signal of a subject.
従来、対象者の肌面の画像を解析し、血圧、脈波、心拍数（脈拍）などの生体情報を取得する技術が提案されている（特許文献１～４参照）。例えば、特許文献１には、同一時間に撮影された対象者の身体の異なる箇所の撮影画像から輝度情報の差分を算出し、差分の増減に応じた血圧の変動を計測する装置が開示されている。
Conventionally, a technique for analyzing an image of the skin surface of a subject and acquiring biological information such as blood pressure, pulse wave, and heart rate (pulse) has been proposed (see Patent Documents 1 to 4). For example, Patent Document 1 discloses a device that calculates a difference in luminance information from images taken at different parts of a subject's body taken at the same time and measures a change in blood pressure according to an increase or decrease in the difference. There is.
特許文献２に記載の脈波検出アルゴリズムでは、取得した計測対象者映像から抽出したＲＧＢ信号をＹＣｂＣｒ信号に変換し、輝度信号Ｙから顔領域及び対象領域ＲＯＩを検出する。そして、抽出された対象領域ＲＯＩから取得したＣｂ＋Ｃｒ信号から、任意の時刻の値をプロットしたＣｂ＋Ｃｒトレース信号を生成して脈波を検出している。
In the pulse wave detection algorithm described in Patent Document 2, the RGB signal extracted from the acquired measurement target person image is converted into a YCbCr signal, and the face region and the target region ROI are detected from the luminance signal Y. Then, from the Cb + Cr signal acquired from the extracted target region ROI, a Cb + Cr trace signal in which the value at an arbitrary time is plotted is generated to detect the pulse wave.
また、特許文献３に記載の脈波演算装置では、外乱光の輝度特性に応じた脈波測定を実現するため、可視光画像だけでなく、赤外光画像も撮影し、その少なくとも一方から抽出された波形の特徴量から生体情報を算出している。一方、特許文献４には、スマートフォンなどの情報端末を利用して、脈波を検出する方法が開示されている。
Further, in the pulse wave arithmetic unit described in Patent Document 3, not only a visible light image but also an infrared light image is captured and extracted from at least one of them in order to realize pulse wave measurement according to the luminance characteristic of ambient light. The biological information is calculated from the feature amount of the generated waveform. On the other hand, Patent Document 4 discloses a method of detecting a pulse wave by using an information terminal such as a smartphone.
しかしながら、前述した従来の脈波計測技術には、以下に示す課題がある。特許文献１に記載の技術は、同一時間に対象者の身体の異なる箇所の撮影画像を得る必要があるが、対象者の姿勢、服装、カメラの位置などによっては、同一時間に対象者の身体の異なる箇所の撮影画像を得ることは困難な場合がある。また、特許文献２、３に記載の技術は、装置構成が大がかりになるため汎用性に劣り、日常的に使用するには不向きである。
However, the above-mentioned conventional pulse wave measurement technology has the following problems. The technique described in Patent Document 1 needs to obtain captured images of different parts of the subject's body at the same time, but depending on the posture, clothes, camera position, etc. of the subject, the subject's body at the same time. It can be difficult to obtain captured images of different parts of the body. Further, the techniques described in Patent Documents 2 and 3 are inferior in versatility because the apparatus configuration becomes large, and are not suitable for daily use.
一方、特許文献４に記載の技術は、スマートフォンで情報を取得するため、日常的に使用する用途には向いているが、指先を撮影した画像から脈波を算出するため、検出精度の面で課題がある。
On the other hand, the technique described in Patent Document 4 is suitable for daily use because information is acquired by a smartphone, but the pulse wave is calculated from the image of the fingertip, so that the detection accuracy is improved. There are challenges.
そこで、本発明は、高精度でかつ汎用的な情報処理装置、脈波計測システム及び脈波計測プログラムを提供すること目的とする。
Therefore, an object of the present invention is to provide a highly accurate and general-purpose information processing device, a pulse wave measurement system, and a pulse wave measurement program.
本発明に係る情報処理装置は、対象者を撮影した画像に含まれる前記対象者の肌領域の輝度値分布から、そのピーク値を基準にして特定範囲のデータを抽出する抽出部と、前記抽出されたデータについて、ピークの高さが１になるように正規化する正規化部と、前記正規化されたデータに対して標準偏差を算出する算出部と、前記標準偏差を時系列順に並べて前記対象者の脈波を出力する出力部とを有する。
本発明の情報処理装置は、撮影装置から前記撮影画像を受信する受信部を更に有していてもよい。
前記出力部は前記対象者の脈波を外部装置に出力することもできる。
本発明の情報処理装置は、前記画像に含まれる前記対象者の肌領域の輝度むらを除去する除去部を有していてもよく、その場合、前記抽出部は、前記除去部によって輝度むらが除去された輝度値分布から前記データを抽出する。
前記除去部は、前記肌領域を複数の領域に分割し、分割した複数の領域ごとに輝度値分布を算出し、各輝度値分布のピーク位置を基準位置にあわせ、すべての輝度値分布の和を求めることで、輝度むらを除去した輝度値分布を求めることもできる。
本発明の情報処理装置において、前記画像は、波長が異なる２以上の近赤外光を用いて撮影されたものでもよく、その場合、前記抽出部は、前記近赤外光成分の輝度値分布からデータを抽出する。
本発明に係る他の情報処理装置は、撮影画像に含まれる対象者の肌領域を複数の領域に分割し、分割した複数の領域ごとに輝度値の標準偏差を求め、求めた標準偏差が一定値以下の領域の割合に基づき、前記対象者の脈の有無を判定する。 The information processing apparatus according to the present invention includes an extraction unit that extracts data in a specific range from the brightness value distribution of the target person's skin region included in an image of the target person based on the peak value, and the extraction unit. A normalization unit that normalizes the data so that the peak height becomes 1, a calculation unit that calculates the standard deviation with respect to the normalized data, and the standard deviation are arranged in chronological order. It has an output unit that outputs the pulse wave of the subject.
The information processing device of the present invention may further have a receiving unit for receiving the photographed image from the photographing device.
The output unit can also output the pulse wave of the subject to an external device.
The information processing apparatus of the present invention may have a removing portion for removing the luminance unevenness of the skin region of the subject included in the image, and in that case, the extracting portion may have the luminance unevenness due to the removing portion. The data is extracted from the removed luminance value distribution.
The removal unit divides the skin region into a plurality of regions, calculates a luminance value distribution for each of the divided regions, aligns the peak position of each luminance value distribution with a reference position, and sums all the luminance value distributions. It is also possible to obtain the brightness value distribution from which the brightness unevenness is removed.
In the information processing apparatus of the present invention, the image may be taken by using two or more near-infrared lights having different wavelengths, and in that case, the extraction unit may distribute the brightness value of the near-infrared light component. Extract data from.
In another information processing device according to the present invention, the skin region of the subject included in the captured image is divided into a plurality of regions, the standard deviation of the brightness value is obtained for each of the divided plurality of regions, and the obtained standard deviation is constant. The presence or absence of the subject's pulse is determined based on the proportion of the region below the value.
本発明の情報処理装置は、撮影装置から前記撮影画像を受信する受信部を更に有していてもよい。
前記出力部は前記対象者の脈波を外部装置に出力することもできる。
本発明の情報処理装置は、前記画像に含まれる前記対象者の肌領域の輝度むらを除去する除去部を有していてもよく、その場合、前記抽出部は、前記除去部によって輝度むらが除去された輝度値分布から前記データを抽出する。
前記除去部は、前記肌領域を複数の領域に分割し、分割した複数の領域ごとに輝度値分布を算出し、各輝度値分布のピーク位置を基準位置にあわせ、すべての輝度値分布の和を求めることで、輝度むらを除去した輝度値分布を求めることもできる。
本発明の情報処理装置において、前記画像は、波長が異なる２以上の近赤外光を用いて撮影されたものでもよく、その場合、前記抽出部は、前記近赤外光成分の輝度値分布からデータを抽出する。
本発明に係る他の情報処理装置は、撮影画像に含まれる対象者の肌領域を複数の領域に分割し、分割した複数の領域ごとに輝度値の標準偏差を求め、求めた標準偏差が一定値以下の領域の割合に基づき、前記対象者の脈の有無を判定する。 The information processing apparatus according to the present invention includes an extraction unit that extracts data in a specific range from the brightness value distribution of the target person's skin region included in an image of the target person based on the peak value, and the extraction unit. A normalization unit that normalizes the data so that the peak height becomes 1, a calculation unit that calculates the standard deviation with respect to the normalized data, and the standard deviation are arranged in chronological order. It has an output unit that outputs the pulse wave of the subject.
The information processing device of the present invention may further have a receiving unit for receiving the photographed image from the photographing device.
The output unit can also output the pulse wave of the subject to an external device.
The information processing apparatus of the present invention may have a removing portion for removing the luminance unevenness of the skin region of the subject included in the image, and in that case, the extracting portion may have the luminance unevenness due to the removing portion. The data is extracted from the removed luminance value distribution.
The removal unit divides the skin region into a plurality of regions, calculates a luminance value distribution for each of the divided regions, aligns the peak position of each luminance value distribution with a reference position, and sums all the luminance value distributions. It is also possible to obtain the brightness value distribution from which the brightness unevenness is removed.
In the information processing apparatus of the present invention, the image may be taken by using two or more near-infrared lights having different wavelengths, and in that case, the extraction unit may distribute the brightness value of the near-infrared light component. Extract data from.
In another information processing device according to the present invention, the skin region of the subject included in the captured image is divided into a plurality of regions, the standard deviation of the brightness value is obtained for each of the divided plurality of regions, and the obtained standard deviation is constant. The presence or absence of the subject's pulse is determined based on the proportion of the region below the value.
本発明に係る脈波計測システムは、対象者の肌領域を撮影する撮影装置と、前述した情報処理装置とを有する。
前記撮影装置は、波長が異なる２以上の近赤外光を少なくとも検出する固体撮像素子を備え、前記近赤外光成分を含む画像を前記情報処理装置に送信してもよい。 The pulse wave measurement system according to the present invention includes a photographing device for photographing a skin area of a subject and an information processing device described above.
The photographing device may include a solid-state imaging device that detects at least two or more near-infrared lights having different wavelengths, and may transmit an image containing the near-infrared light component to the information processing device.
前記撮影装置は、波長が異なる２以上の近赤外光を少なくとも検出する固体撮像素子を備え、前記近赤外光成分を含む画像を前記情報処理装置に送信してもよい。 The pulse wave measurement system according to the present invention includes a photographing device for photographing a skin area of a subject and an information processing device described above.
The photographing device may include a solid-state imaging device that detects at least two or more near-infrared lights having different wavelengths, and may transmit an image containing the near-infrared light component to the information processing device.
本発明に係る脈波計測プログラムは、対象者を撮影した画像に含まれる前記対象者の肌領域の輝度値分布から、そのピーク値を基準にして特定範囲のデータを抽出する抽出機能と、前記抽出されたデータについて、ピークの高さが１になるように正規化する正規化機能と、前記正規化されたデータに対して標準偏差を算出する算出機能と、前記標準偏差を時系列順に並べて前記対象者の脈波を出力する出力機能とをコンピュータに実行させる。
The pulse wave measurement program according to the present invention has an extraction function for extracting data in a specific range from the brightness value distribution of the skin region of the subject included in an image of the subject, and the above-mentioned. For the extracted data, a normalization function that normalizes the peak height to 1 and a calculation function that calculates the standard deviation for the normalized data, and the standard deviations are arranged in chronological order. The computer is made to execute the output function of outputting the pulse wave of the subject.
本発明によれば、精度よく脈波を検出でき、かつ、汎用性が高い情報処理装置、脈波計測システム及び脈波計測プログラムを実現することができる。
According to the present invention, it is possible to realize an information processing device, a pulse wave measurement system, and a pulse wave measurement program that can detect pulse waves with high accuracy and have high versatility.
以下、本発明を実施するための形態について、添付の図面を参照して、詳細に説明する。なお、本発明は、以下に説明する実施形態に限定されるものではない。
Hereinafter, a mode for carrying out the present invention will be described in detail with reference to the attached drawings. The present invention is not limited to the embodiments described below.
（第１の実施形態）
［システムの全体構成］
先ず、本発明の第１の実施形態に係る脈波計測システムについて説明する。図１は本実施形態の脈波計測システムの構成例を示すブロック図である。図１に示すように、本実施形態の脈波計測システムは、撮影装置１１０と、情報処理装置１００とを含み、撮影装置１１０と情報処理装置１００とは、有線又は無線を介して通信可能に接続されている。 (First Embodiment)
[Overall system configuration]
First, the pulse wave measurement system according to the first embodiment of the present invention will be described. FIG. 1 is a block diagram showing a configuration example of the pulse wave measurement system of the present embodiment. As shown in FIG. 1, the pulse wave measurement system of the present embodiment includes aphotographing device 110 and an information processing device 100, and the photographing device 110 and the information processing device 100 can communicate with each other via wired or wireless communication. It is connected.
［システムの全体構成］
先ず、本発明の第１の実施形態に係る脈波計測システムについて説明する。図１は本実施形態の脈波計測システムの構成例を示すブロック図である。図１に示すように、本実施形態の脈波計測システムは、撮影装置１１０と、情報処理装置１００とを含み、撮影装置１１０と情報処理装置１００とは、有線又は無線を介して通信可能に接続されている。 (First Embodiment)
[Overall system configuration]
First, the pulse wave measurement system according to the first embodiment of the present invention will be described. FIG. 1 is a block diagram showing a configuration example of the pulse wave measurement system of the present embodiment. As shown in FIG. 1, the pulse wave measurement system of the present embodiment includes a
［撮影装置１１０］
撮影装置１１０は、対象者の肌領域を撮影し、肌領域を含む撮影画像を生成するものである。撮影装置１１０は、生成した撮影画像を情報処理装置１００に送信する。撮影装置１１０には、専用のカメラを用いてもよいが、携帯情報端末に内蔵されているカメラや他の目的で車内や室内に設置されたカメラを利用することもできる。 [Photographing device 110]
The photographingdevice 110 photographs the skin region of the subject and generates a captured image including the skin region. The photographing device 110 transmits the generated photographed image to the information processing device 100. A dedicated camera may be used for the photographing device 110, but a camera built in the mobile information terminal or a camera installed in the vehicle or indoors for other purposes can also be used.
撮影装置１１０は、対象者の肌領域を撮影し、肌領域を含む撮影画像を生成するものである。撮影装置１１０は、生成した撮影画像を情報処理装置１００に送信する。撮影装置１１０には、専用のカメラを用いてもよいが、携帯情報端末に内蔵されているカメラや他の目的で車内や室内に設置されたカメラを利用することもできる。 [Photographing device 110]
The photographing
［情報処置装置１００］
情報処理装置１００は、撮影装置１１０より受信した撮影画像を処理して、脈波を出力するものである。従来の脈波検出方法では、撮影画像内に含まれる対象者の肌領域の輝度平均値の変動から脈波を検出していたが、本実施形態の脈波計測システムでは、対象者の肌領域の輝度値のヒストグラムに着目し、その形状変化から脈波を検出する。 [Information treatment device 100]
Theinformation processing device 100 processes the captured image received from the photographing device 110 and outputs a pulse wave. In the conventional pulse wave detection method, the pulse wave is detected from the fluctuation of the brightness average value of the skin region of the subject included in the captured image, but in the pulse wave measurement system of the present embodiment, the skin region of the subject is detected. Focusing on the histogram of the brightness value of, the pulse wave is detected from the shape change.
情報処理装置１００は、撮影装置１１０より受信した撮影画像を処理して、脈波を出力するものである。従来の脈波検出方法では、撮影画像内に含まれる対象者の肌領域の輝度平均値の変動から脈波を検出していたが、本実施形態の脈波計測システムでは、対象者の肌領域の輝度値のヒストグラムに着目し、その形状変化から脈波を検出する。 [Information treatment device 100]
The
〔情報処置装置１００のハードウェア構成〕
図２は情報処理装置１００のハードウェア構成の一例を示すブロック図である。図２に示すように、情報処理装置１００は、例えば、制御部２０１と、記憶部２０２と、入力部２０３と、表示部２０４と、通信部２０５とを備える構成とすることができる。制御部２０１は、情報処理装置１００の全体を制御するものであり、ＣＰＵ（Ｃｅｎｔｒａｌ Ｐｒｏｃｅｓｓｉｎｇ Ｕｎｉｔ）などを用いることができる。 [Hardware configuration of information treatment device 100]
FIG. 2 is a block diagram showing an example of the hardware configuration of theinformation processing device 100. As shown in FIG. 2, the information processing device 100 can be configured to include, for example, a control unit 201, a storage unit 202, an input unit 203, a display unit 204, and a communication unit 205. The control unit 201 controls the entire information processing device 100, and a CPU (Central Processing Unit) or the like can be used.
図２は情報処理装置１００のハードウェア構成の一例を示すブロック図である。図２に示すように、情報処理装置１００は、例えば、制御部２０１と、記憶部２０２と、入力部２０３と、表示部２０４と、通信部２０５とを備える構成とすることができる。制御部２０１は、情報処理装置１００の全体を制御するものであり、ＣＰＵ（Ｃｅｎｔｒａｌ Ｐｒｏｃｅｓｓｉｎｇ Ｕｎｉｔ）などを用いることができる。 [Hardware configuration of information treatment device 100]
FIG. 2 is a block diagram showing an example of the hardware configuration of the
記憶部２０２は、処理を実行するためのプログラムや制御部２０１がプログラムに基づき処理を実行する際に利用するデータなどを記憶するものであり、ＨＤＤ（Ｈａｒｄ Ｄｉｓｋ Ｄｒｉｖｅ）、ＲＯＭ（Ｒｅａｄ Ｏｎｌｙ Ｍｅｍｏｒｙ）、ＲＡＭ（Ｒａｎｄｏｍ Ａｃｃｅｓｓ Ｍｅｍｏｒｙ）などの記憶媒体が用いられる。そして、制御部２０１が、記憶部２０２に記憶されているプログラムに基づいて処理を実行することにより、後述する情報処理装置１００の機能や処理が実現される。
The storage unit 202 stores a program for executing processing, data used when the control unit 201 executes processing based on the program, and the like, and is an HDD (Hard Disk Drive), ROM (Read Only Memory). , RAM (Random Access Memory) or the like is used. Then, the control unit 201 executes the process based on the program stored in the storage unit 202, thereby realizing the functions and processes of the information processing device 100 described later.
入力部２０３は、ユーザの操作を制御部２０１に入力するものであり、ボタンなどで構成されている。表示部２０４は、制御部２０１の処理の結果などを表示するものであり、ディスプレイなどの表示装置を用いることができる。通信部２０５は、情報処理装置１００をネットワークに接続するものであり、ＮＩＣ（Ｎｅｔｗｏｒｋ Ｉｎｔｅｒｆａｃｅ Ｃａｒｄ）などが用いられる。
The input unit 203 inputs the user's operation to the control unit 201, and is composed of buttons and the like. The display unit 204 displays the processing result of the control unit 201 and the like, and a display device such as a display can be used. The communication unit 205 connects the information processing device 100 to the network, and a NIC (Network Interface Card) or the like is used.
〔情報処置装置１００の機能面からみた構成〕
図３は情報処理装置１００の機能面からみた構成例を示すブロック図である。図３に示すように、情報処理装置１００を機能面からみた場合、受信部３０１、抽出部３０２、正規化部３０３、算出部３０４及び出力部３０５を備える構成とすることができる。受信部３０１では、撮影装置１１０より撮影画像を受信し、抽出部３０２では、受信部３０１が受信した撮影画像に含まれる対象者の肌領域の輝度値ヒストグラムから、そのピーク値を基準とし特定範囲のデータを抽出する。なお、輝度値ヒストグラムは、輝度値分布の一例である。 [Structure from the functional aspect of the information treatment device 100]
FIG. 3 is a block diagram showing a configuration example from the functional aspect of theinformation processing apparatus 100. As shown in FIG. 3, when the information processing apparatus 100 is viewed from the functional aspect, it can be configured to include a receiving unit 301, an extracting unit 302, a normalizing unit 303, a calculating unit 304, and an output unit 305. The receiving unit 301 receives the captured image from the photographing device 110, and the extracting unit 302 uses the peak value as a reference from the luminance value histogram of the target person's skin area included in the captured image received by the receiving unit 301 to specify a range. Extract the data of. The luminance value histogram is an example of the luminance value distribution.
図３は情報処理装置１００の機能面からみた構成例を示すブロック図である。図３に示すように、情報処理装置１００を機能面からみた場合、受信部３０１、抽出部３０２、正規化部３０３、算出部３０４及び出力部３０５を備える構成とすることができる。受信部３０１では、撮影装置１１０より撮影画像を受信し、抽出部３０２では、受信部３０１が受信した撮影画像に含まれる対象者の肌領域の輝度値ヒストグラムから、そのピーク値を基準とし特定範囲のデータを抽出する。なお、輝度値ヒストグラムは、輝度値分布の一例である。 [Structure from the functional aspect of the information treatment device 100]
FIG. 3 is a block diagram showing a configuration example from the functional aspect of the
正規化部３０３では、抽出部３０２で抽出されたデータを、ピークの高さが所定の値になるように正規化する。算出部３０４では、正規化部３０３で正規化されたデータの標準偏差を算出する。出力部３０５は、算出部３０４で算出された標準偏差を時系列順に並べて対象者の脈波として、例えば表示部２０４に出力して表示したり、通信部２０５を介して外部装置に送信したりする。
The normalization unit 303 normalizes the data extracted by the extraction unit 302 so that the peak height becomes a predetermined value. The calculation unit 304 calculates the standard deviation of the data normalized by the normalization unit 303. The output unit 305 arranges the standard deviations calculated by the calculation unit 304 in chronological order and outputs and displays them as pulse waves of the target person, for example, on the display unit 204 or transmits them to an external device via the communication unit 205. To do.
［動作］
次に、本実施形態の脈波計測システムの動作について説明する。本実施形態の脈波計測システムでは、先ず、撮影装置１１０により対象者の肌を含む画像を撮影し、情報処理装置１００に撮影画像を送信する。次に、情報処理装置１００において、輝度値ヒストグラムのピーク位置を捉え、輝度値のヒストグラムの形状変化を捉えることで、脈波の測定を行う。情報処理装置１００は、輝度値のヒストグラムの形状変化を捉えるために、輝度値のヒストグラムに関するデータの標準偏差を特徴量とし、標準偏差の変動により脈波を検出する。 [motion]
Next, the operation of the pulse wave measurement system of the present embodiment will be described. In the pulse wave measurement system of the present embodiment, first, an image including the skin of the subject is photographed by the photographingdevice 110, and the photographed image is transmitted to the information processing device 100. Next, in the information processing apparatus 100, the pulse wave is measured by capturing the peak position of the luminance value histogram and capturing the shape change of the luminance value histogram. In order to capture the shape change of the histogram of the luminance value, the information processing apparatus 100 uses the standard deviation of the data related to the histogram of the luminance value as a feature amount, and detects the pulse wave by the fluctuation of the standard deviation.
次に、本実施形態の脈波計測システムの動作について説明する。本実施形態の脈波計測システムでは、先ず、撮影装置１１０により対象者の肌を含む画像を撮影し、情報処理装置１００に撮影画像を送信する。次に、情報処理装置１００において、輝度値ヒストグラムのピーク位置を捉え、輝度値のヒストグラムの形状変化を捉えることで、脈波の測定を行う。情報処理装置１００は、輝度値のヒストグラムの形状変化を捉えるために、輝度値のヒストグラムに関するデータの標準偏差を特徴量とし、標準偏差の変動により脈波を検出する。 [motion]
Next, the operation of the pulse wave measurement system of the present embodiment will be described. In the pulse wave measurement system of the present embodiment, first, an image including the skin of the subject is photographed by the photographing
図４は情報処理装置１００における処理工程を示すフローチャートである。具体的には、図４に示すように、情報処理装置１００では、受信部３０１が撮影画像を受信すると（ステップＳ１）、抽出部３０２において撮影画像に含まれる対象者の肌領域の輝度値ヒストグラムから、ピーク値を基準にして特定範囲のデータを抽出する（ステップＳ２）。
FIG. 4 is a flowchart showing a processing process in the information processing apparatus 100. Specifically, as shown in FIG. 4, in the information processing apparatus 100, when the receiving unit 301 receives the captured image (step S1), the extraction unit 302 includes a luminance value histogram of the skin region of the target person included in the captured image. Then, data in a specific range is extracted with reference to the peak value (step S2).
図５Ａは抽出部３０２において輝度値ヒストグラムから抽出されるデータの範囲を示す図であり、図５Ｂは抽出されたデータを示す図である。抽出部３０２では、例えば輝度値ヒストグラムのピーク値を１００％とし、ｎ％までの範囲（図５Ａで楕円で囲まれた範囲）のデータを抽出する。これにより、図５Ｂに示すデータが抽出される。なお、抽出するデータの範囲（図５Ａに示すｎ％の値）は、輝度値ヒストグラムの形状などに応じて適宜設定することができる。
FIG. 5A is a diagram showing a range of data extracted from the luminance value histogram by the extraction unit 302, and FIG. 5B is a diagram showing the extracted data. In the extraction unit 302, for example, the peak value of the luminance value histogram is set to 100%, and data in the range up to n% (the range surrounded by the ellipse in FIG. 5A) is extracted. As a result, the data shown in FIG. 5B is extracted. The range of the data to be extracted (n% value shown in FIG. 5A) can be appropriately set according to the shape of the luminance value histogram and the like.
次に、正規化部３０３において抽出されたデータを正規化する（ステップＳ３）。図６Ａ及びＢは正規化部３０３におけるデータ正規化処理を示す図であり、図６Ａは正規化処理前の状態を示し、図６Ｂは正規化処理後の状態を示す。正規化部３０３では、例えば図６Ａに示す抽出されたデータのピーク量Ｈが１になるよう正規化を行う。これにより、図６Ｂに示すデータが得られる。この正規化処理を行うことにより、抽出されたデータの数に依存せず、標準偏差によって形状の変化のみを捉えることが可能となる。
Next, the data extracted by the normalization unit 303 is normalized (step S3). 6A and 6B are diagrams showing the data normalization process in the normalization unit 303, FIG. 6A shows the state before the normalization process, and FIG. 6B shows the state after the normalization process. The normalization unit 303 performs normalization so that the peak amount H of the extracted data shown in FIG. 6A is 1, for example. As a result, the data shown in FIG. 6B can be obtained. By performing this normalization process, it is possible to capture only the change in shape by the standard deviation without depending on the number of extracted data.
引き続き、算出部３０４において、正規化されたデータについて標準偏差を算出する（ステップＳ４）。その結果（標準偏差）を時系列に並べて波形を形成し、それを出力部３０５から対象者の脈波として出力する（ステップＳ５）。図７は出力部３０３からの出力例を示す図である。情報処理装置１００での処理結果は、例えば、表示部２０４において、図７に示すように、撮影画像に脈波検出に用いた肌領域を示したものと、標準偏差を時系列に並べて生成された脈波とを並べて表示することもできる。
Subsequently, the calculation unit 304 calculates the standard deviation of the normalized data (step S4). The result (standard deviation) is arranged in chronological order to form a waveform, which is output from the output unit 305 as a pulse wave of the subject (step S5). FIG. 7 is a diagram showing an example of output from the output unit 303. The processing result of the information processing apparatus 100 is generated, for example, on the display unit 204 by arranging the captured image showing the skin region used for pulse wave detection and the standard deviation in chronological order as shown in FIG. It is also possible to display the pulse wave side by side.
［プログラム］
前述した各工程は、情報処理装置１００の各部の機能を実現するためのコンピュータプログラムを作成し、１又は２以上のコンピュータに実装することにより実施することができる。即ち、対象者を撮影した画像に含まれる対象者の肌領域の輝度値分布から、そのピーク値を基準に特定範囲のデータを抽出する抽出機能と、抽出されたデータについてピークの高さが所定の値になるように正規化する正規化機能と、正規化されたデータに対して標準偏差を算出する標準偏差算出機能と、標準偏差を時系列順に並べて対象者の脈波を出力する脈波出力機能をコンピュータに実行させるプログラムを、コンピュータ又は画像処理装置で動作させることにより実施することができる。 [program]
Each of the above-mentioned steps can be carried out by creating a computer program for realizing the functions of each part of theinformation processing apparatus 100 and implementing it on one or more computers. That is, an extraction function that extracts data in a specific range based on the peak value from the brightness value distribution of the target person's skin area included in the image of the target person, and a predetermined peak height for the extracted data. A normalization function that normalizes to the value of, a standard deviation calculation function that calculates the standard deviation for the normalized data, and a pulse wave that outputs the pulse wave of the subject by arranging the standard deviation in chronological order. A program that causes a computer to execute an output function can be executed by operating the computer or an image processing device.
前述した各工程は、情報処理装置１００の各部の機能を実現するためのコンピュータプログラムを作成し、１又は２以上のコンピュータに実装することにより実施することができる。即ち、対象者を撮影した画像に含まれる対象者の肌領域の輝度値分布から、そのピーク値を基準に特定範囲のデータを抽出する抽出機能と、抽出されたデータについてピークの高さが所定の値になるように正規化する正規化機能と、正規化されたデータに対して標準偏差を算出する標準偏差算出機能と、標準偏差を時系列順に並べて対象者の脈波を出力する脈波出力機能をコンピュータに実行させるプログラムを、コンピュータ又は画像処理装置で動作させることにより実施することができる。 [program]
Each of the above-mentioned steps can be carried out by creating a computer program for realizing the functions of each part of the
以上詳述したように、本実施形態の脈波検出システムは、複数画像を撮影する必要がなく、一カ所の撮影画像から対象者の脈波を検出することができるため、装置を簡素化することができ、汎用性も高い。また、従来の脈波計測方法では、計測対象エリアを決定し、そのエリア内の輝度平均により脈を検出する方法が取られている。この方法は、血中ヘモグロビンが光を吸収するため、時系列的に平均値を取ることにより、血液が流れているときに、輝度レベルが下がることを利用しているが、光の吸収は微少であり、輝度平均値の変化は１／１０以下のレベルでの変動になるため、対象物の移動及び外乱光の変動に対して弱い。
As described in detail above, the pulse wave detection system of the present embodiment simplifies the device because it is not necessary to capture a plurality of images and the pulse wave of the target person can be detected from the captured images at one place. It can be used and is highly versatile. Further, in the conventional pulse wave measurement method, a method of determining a measurement target area and detecting a pulse by averaging the brightness in the area is adopted. This method utilizes the fact that the brightness level decreases when blood is flowing by taking the average value in time series because hemoglobin in the blood absorbs light, but the absorption of light is slight. Since the change in the average luminance value is a fluctuation at a level of 1/10 or less, it is vulnerable to the movement of an object and the fluctuation of ambient light.
これに対して、本実施形態の脈波検出システムは、特定範囲のデータを抽出し、正規化した上で標準偏差をとっているため、対象物の移動や外乱光の変動の影響を受けにくく、精度よく脈波を検出することができる。更に、一般に標準偏差や分散などの処理は画素単位で行われており、処理速度の低下が懸念されるが、本実施形態の脈波検出システムでは、輝度値ヒストグラムを用いているため、高速での処理速度が可能である。
On the other hand, the pulse wave detection system of the present embodiment extracts data in a specific range, normalizes it, and takes a standard deviation, so that it is not easily affected by the movement of an object or fluctuation of ambient light. , The pulse wave can be detected with high accuracy. Further, in general, processing such as standard deviation and dispersion is performed on a pixel-by-pixel basis, and there is a concern that the processing speed may decrease. However, since the pulse wave detection system of the present embodiment uses a luminance value histogram, the processing speed is high. Processing speed is possible.
本実施形態の脈波計測システムは、例えば車に搭載することができる。その場合、撮影装置１１０により運転手及び／又は同乗者の肌領域を撮影し、その撮影画像を情報処理装置１００において処理し、運転手及び／又は同乗者の脈波を検出して出力する。これにより、運転手などの体調の変化などを察知することが可能となる。
The pulse wave measurement system of this embodiment can be mounted on a car, for example. In that case, the skin region of the driver and / or the passenger is photographed by the photographing device 110, the photographed image is processed by the information processing device 100, and the pulse wave of the driver and / or the passenger is detected and output. This makes it possible to detect changes in the physical condition of the driver or the like.
なお、図１に示す脈波計測装置では、撮影装置１１０と情報処理装置１００を別体としているが、本発明はこれに限定されるものではなく、撮影装置１１０と情報処理装置１００は一体で構成されていてもよい。例えば、上述した情報処理装置１００の機能が撮影装置１１０に実装されてもよく、また、カメラ付きの情報端末情報に処理装置１００の機能と撮影装置１１０の機能の両方を搭載することもできる。
In the pulse wave measuring device shown in FIG. 1, the photographing device 110 and the information processing device 100 are separate bodies, but the present invention is not limited to this, and the photographing device 110 and the information processing device 100 are integrated. It may be configured. For example, the function of the information processing device 100 described above may be mounted on the photographing device 110, or both the function of the processing device 100 and the function of the photographing device 110 may be mounted on the information terminal information with a camera.
また、図７では撮影画像の一部を検知領域とした例を示しているが、本発明はこれに限定されるものではなく、撮影装置１１０の視野全域を検知エリアとして設定してもよい。ただし、その場合、情報処理装置１００での処理に時間がかかることがある。そこで、対象者に赤外光を照射し、撮影装置１１０で赤外光の反射光を検出し、その光量に基づき検知領域（例えば、対象者の肌領域など）を自動追従するようにしてもよい。これにより、情報処理装置１００での処理時間を短縮することができる。
Further, although FIG. 7 shows an example in which a part of the captured image is used as the detection area, the present invention is not limited to this, and the entire field of view of the photographing device 110 may be set as the detection area. However, in that case, the processing by the information processing apparatus 100 may take time. Therefore, even if the subject is irradiated with infrared light, the reflecting light of the infrared light is detected by the photographing device 110, and the detection region (for example, the skin region of the subject) is automatically followed based on the amount of the light. Good. As a result, the processing time in the information processing apparatus 100 can be shortened.
（第１の実施形態の第１変形例）
次に、本発明の第１の実施形態の第１変形例の脈波計測システムについて説明する。脈波検出に用いられる撮影画像は、計測対象者の肌面の向きによって光の反射が大きく変化し、輝度むらが発生することがある。輝度むらの問題は、赤外光により撮影した赤外光画像を用いる場合に顕著である。そこで、本変形例の脈波計測システムでは、撮影画像から輝度むらを除去する処理を行う。 (First modification of the first embodiment)
Next, the pulse wave measurement system of the first modification of the first embodiment of the present invention will be described. In the captured image used for pulse wave detection, the reflection of light changes greatly depending on the orientation of the skin surface of the measurement target person, and uneven brightness may occur. The problem of luminance unevenness is remarkable when an infrared light image taken by infrared light is used. Therefore, in the pulse wave measurement system of this modified example, processing is performed to remove the uneven brightness from the captured image.
次に、本発明の第１の実施形態の第１変形例の脈波計測システムについて説明する。脈波検出に用いられる撮影画像は、計測対象者の肌面の向きによって光の反射が大きく変化し、輝度むらが発生することがある。輝度むらの問題は、赤外光により撮影した赤外光画像を用いる場合に顕著である。そこで、本変形例の脈波計測システムでは、撮影画像から輝度むらを除去する処理を行う。 (First modification of the first embodiment)
Next, the pulse wave measurement system of the first modification of the first embodiment of the present invention will be described. In the captured image used for pulse wave detection, the reflection of light changes greatly depending on the orientation of the skin surface of the measurement target person, and uneven brightness may occur. The problem of luminance unevenness is remarkable when an infrared light image taken by infrared light is used. Therefore, in the pulse wave measurement system of this modified example, processing is performed to remove the uneven brightness from the captured image.
図８は本変形例の脈波計測システムにおける情報処理装置の機能面からみた構成例を示すブロック図であり、図９は図８に示す情報処理装置１０１における処理工程を示すフローチャートである。なお、図８においては、図３に示す情報処理装置１００の構成要素と同じものには同じ符号を付し、その詳細な説明は省略する。図８に示すように、本変形例の脈波計測システムでは、情報処理装置１０１に、受信部３０１、抽出部３０２、正規化部３０４、出力部３０５に加えて、除去部８０１が設けられている。
FIG. 8 is a block diagram showing a configuration example from the functional aspect of the information processing device in the pulse wave measurement system of this modified example, and FIG. 9 is a flowchart showing a processing process in the information processing device 101 shown in FIG. In FIG. 8, the same components as those of the information processing apparatus 100 shown in FIG. 3 are designated by the same reference numerals, and detailed description thereof will be omitted. As shown in FIG. 8, in the pulse wave measurement system of the present modification, the information processing device 101 is provided with a removal unit 801 in addition to the reception unit 301, the extraction unit 302, the normalization unit 304, and the output unit 305. There is.
図９に示すように、この脈波計測システムでは、情報処理装置１０１の受信部３０１で撮影画像を受信し（ステップＳ１１）、除去部８０１で撮影画像の輝度むらを除去した（ステップＳ１２）後、抽出部３０２において、輝度むらが除去された輝度値ヒストグラムからピーク値を基準にして特定範囲のデータが抽出される（ステップＳ１３）。そして、正規化部３０３で正規化処理（ステップＳ１４）を、算出部３０４で標準偏差の算出（ステップＳ１５）をそれぞれ行い、得られた結果を時系列に並べて脈波として出力部３０５から出力する（ステップＳ１６）。
As shown in FIG. 9, in this pulse wave measurement system, after the captured image is received by the receiving unit 301 of the information processing device 101 (step S11) and the brightness unevenness of the captured image is removed by the removing unit 801 (step S12). In the extraction unit 302, data in a specific range is extracted from the luminance value histogram from which the luminance unevenness has been removed, with reference to the peak value (step S13). Then, the normalization unit 303 performs the normalization process (step S14), and the calculation unit 304 calculates the standard deviation (step S15), and the obtained results are arranged in chronological order and output from the output unit 305 as a pulse wave. (Step S16).
［除去部８０１］
除去部８０１では、撮影画像に含まれる対象者の肌領域の輝度むらを除去する。図１０～図１４は除去部８０１により輝度むら除去処理（輝度むら除去工程Ｓ１２）を説明する図である。除去部８０１では、先ず、図１０に示すように、撮影画像の肌領域（検知領域）をＮ個に分割する。ここで、Ｎは２以上の自然数であり、分割された各検知領域は、輝度ヒストグラムのピークがわかる大きさとする。そして、図１１に示すように、分割した検知領域ごとに輝度値ヒストグラムを算出する。除去部８０１では、分割した検知領域ごとに、脈波測定などに利用できる領域か否かを、輝度値ヒストグラムの形状により判定する。 [Removal unit 801]
The removingunit 801 removes the uneven brightness of the skin region of the subject included in the captured image. 10 to 14 are diagrams for explaining the luminance unevenness removing process (luminance unevenness removing step S12) by the removing unit 801. First, as shown in FIG. 10, the removing unit 801 divides the skin area (detection area) of the captured image into N pieces. Here, N is a natural number of 2 or more, and each divided detection region has a size that allows the peak of the luminance histogram to be known. Then, as shown in FIG. 11, a luminance value histogram is calculated for each divided detection region. The removal unit 801 determines for each divided detection region whether or not the region can be used for pulse wave measurement or the like based on the shape of the luminance value histogram.
除去部８０１では、撮影画像に含まれる対象者の肌領域の輝度むらを除去する。図１０～図１４は除去部８０１により輝度むら除去処理（輝度むら除去工程Ｓ１２）を説明する図である。除去部８０１では、先ず、図１０に示すように、撮影画像の肌領域（検知領域）をＮ個に分割する。ここで、Ｎは２以上の自然数であり、分割された各検知領域は、輝度ヒストグラムのピークがわかる大きさとする。そして、図１１に示すように、分割した検知領域ごとに輝度値ヒストグラムを算出する。除去部８０１では、分割した検知領域ごとに、脈波測定などに利用できる領域か否かを、輝度値ヒストグラムの形状により判定する。 [Removal unit 801]
The removing
次に、図１２及び図１３に示すように、ピーク位置を調整して輝度値ヒストグラムごとの輝度ピーク位置を基準位置に合わせ、Ｎ個全てのヒストグラムの和を計算し、図１４に示す和のデータを得る。即ち、除去部８０１は、補正した輝度値ヒストグラムを合成加算することで、領域全体の輝度値ヒストグラムを作成する。これにより、輝度むらを除去した輝度値ヒストグラムが得られる。除去部８０１において輝度むらが除去された輝度値ヒストグラムは、抽出部３０２３においてピーク値を基準にして特定範囲のデータが抽出される。
Next, as shown in FIGS. 12 and 13, the peak positions are adjusted to align the luminance peak positions for each luminance value histogram with the reference positions, the sum of all N histograms is calculated, and the sum of the sums shown in FIG. 14 is calculated. Get the data. That is, the removal unit 801 creates a luminance value histogram of the entire region by combining and adding the corrected luminance value histograms. As a result, a luminance value histogram in which the luminance unevenness is removed can be obtained. In the luminance value histogram in which the luminance unevenness is removed by the removing section 801, data in a specific range is extracted by the extracting section 3023 with reference to the peak value.
本変形例の脈波計測システムでは、検知領域を分割し、分割した領域毎に輝度値ヒストグラムを算出し、ピーク位置を調整した後、全てのデータの和をとり合成しているため、輝度むらのない輝度値ヒストグラムが得られる。この構成は、赤外光撮影のように光源が一定場所に配置され、対象者にスポット的に光を照射する場合などのように、光の当たる面の角度により反射光のレベルが変動する画像を用いる場合に特に有効である。なお、本変形例の脈波計測システムにおける上記以外の構成及び効果は、前述した第１の実施形態と同様である。
In the pulse wave measurement system of this modification, the detection area is divided, the brightness value histogram is calculated for each divided area, the peak position is adjusted, and then all the data are summed and combined. A luminance value histogram without is obtained. In this configuration, the light source is arranged in a fixed place as in infrared photography, and the level of reflected light fluctuates depending on the angle of the surface exposed to the light, such as when the subject is irradiated with light in a spot. Is particularly effective when using. The configurations and effects other than the above in the pulse wave measurement system of this modified example are the same as those in the first embodiment described above.
（第２の実施形態）
次に、本発明の第２の実施形態の脈波計測システムについて説明する。本実施形態の脈波計測システムでは、制御部２０１において、面分割した小領域ごとに輝度値の標準偏差を計算し、各フレームにおける脈の有無判定を行う。標準偏差の大きさは、光の反射ムラ度合いを示しているので、一定値より大きい輝度標準偏差の領域は、脈成分がうまく抽出できない部分である。そこで、制御部２０１は、全体に対する脈成分がうまく抽出できている小領域の割合が所定の値より多い場合は、該当するフームにおいて脈があると判定し、所定の値以下の場合は、該当するフームにおいて脈がないと判定する。 (Second Embodiment)
Next, the pulse wave measurement system of the second embodiment of the present invention will be described. In the pulse wave measurement system of the present embodiment, thecontrol unit 201 calculates the standard deviation of the brightness value for each of the surface-divided small regions, and determines the presence or absence of a pulse in each frame. Since the magnitude of the standard deviation indicates the degree of light reflection unevenness, the region of the luminance standard deviation larger than a certain value is a portion where the pulse component cannot be extracted well. Therefore, the control unit 201 determines that there is a pulse in the corresponding hum when the ratio of the small region in which the pulse component is successfully extracted to the whole is larger than the predetermined value, and when it is less than the predetermined value, it corresponds. Judge that there is no pulse in the humming.
次に、本発明の第２の実施形態の脈波計測システムについて説明する。本実施形態の脈波計測システムでは、制御部２０１において、面分割した小領域ごとに輝度値の標準偏差を計算し、各フレームにおける脈の有無判定を行う。標準偏差の大きさは、光の反射ムラ度合いを示しているので、一定値より大きい輝度標準偏差の領域は、脈成分がうまく抽出できない部分である。そこで、制御部２０１は、全体に対する脈成分がうまく抽出できている小領域の割合が所定の値より多い場合は、該当するフームにおいて脈があると判定し、所定の値以下の場合は、該当するフームにおいて脈がないと判定する。 (Second Embodiment)
Next, the pulse wave measurement system of the second embodiment of the present invention will be described. In the pulse wave measurement system of the present embodiment, the
即ち、制御部２０１は、撮影画像に含まれる対象者の肌領域を複数の領域に分割し、分割した複数の領域ごとに輝度値の標準偏差を求め、求めた標準偏差が一定値以下の領域の割合に基づき、対象者の脈の有無を判定し、出力する。この構成により、本実施形態の脈波計測システムは、脈の有無判定を行うことが可能となる。なお、本実施形態の脈波計測システムにおける上記以外の構成及び効果は、前述した第１の実施形態と同様である。
That is, the control unit 201 divides the skin region of the subject included in the captured image into a plurality of regions, obtains the standard deviation of the brightness value for each of the divided plurality of regions, and the obtained standard deviation is a region of a certain value or less. Based on the ratio of, the presence or absence of the subject's pulse is determined and output. With this configuration, the pulse wave measurement system of the present embodiment can determine the presence or absence of a pulse. The configurations and effects of the pulse wave measurement system of the present embodiment other than the above are the same as those of the first embodiment described above.
（第３の実施形態）
次に、本実施形態の第３の実施形態に係る脈波計測システムについて説明する。本実施形態の脈波計測システムは、可視光検出画素と波長が異なる２以上の近赤外光を検出可能な撮像素子が搭載された撮像装置を用いて、可視光画像に加えて、波長が異なる２以上の近赤外光により撮影された近赤外光画像を撮影する。その際用いられる撮像素子としては、例えば国際公開第２０１８／１５５４８６号や国際公開第２０１８／２０７８１７号に記載されている検出波長が異なる３種の可視光画素と、検出波長が異なる２種以上の近赤外光画素が同一基板上に設けられたものが挙げられるが、これらに限定されるものではなく、赤色光Ｒ、緑色光Ｇ、青色光Ｂの他に、波長が異なる２種以上の近赤外光（近赤外光のマルチスペクトル）を検出できるものであればよい。 (Third Embodiment)
Next, the pulse wave measurement system according to the third embodiment of the present embodiment will be described. The pulse wave measurement system of the present embodiment uses an image pickup device equipped with an image pickup element capable of detecting two or more near-infrared lights having a wavelength different from that of the visible light detection pixel, and has a wavelength in addition to the visible light image. A near-infrared light image taken by two or more different near-infrared lights is taken. The imaging elements used at that time include, for example, three types of visible light pixels having different detection wavelengths described in International Publication No. 2018/155486 and International Publication No. 2018/207817, and two or more types having different detection wavelengths. Examples include those in which near-infrared light pixels are provided on the same substrate, but the present invention is not limited to these, and in addition to red light R, green light G, and blue light B, two or more types having different wavelengths are used. Anything that can detect near-infrared light (multi-spectrum of near-infrared light) is sufficient.
次に、本実施形態の第３の実施形態に係る脈波計測システムについて説明する。本実施形態の脈波計測システムは、可視光検出画素と波長が異なる２以上の近赤外光を検出可能な撮像素子が搭載された撮像装置を用いて、可視光画像に加えて、波長が異なる２以上の近赤外光により撮影された近赤外光画像を撮影する。その際用いられる撮像素子としては、例えば国際公開第２０１８／１５５４８６号や国際公開第２０１８／２０７８１７号に記載されている検出波長が異なる３種の可視光画素と、検出波長が異なる２種以上の近赤外光画素が同一基板上に設けられたものが挙げられるが、これらに限定されるものではなく、赤色光Ｒ、緑色光Ｇ、青色光Ｂの他に、波長が異なる２種以上の近赤外光（近赤外光のマルチスペクトル）を検出できるものであればよい。 (Third Embodiment)
Next, the pulse wave measurement system according to the third embodiment of the present embodiment will be described. The pulse wave measurement system of the present embodiment uses an image pickup device equipped with an image pickup element capable of detecting two or more near-infrared lights having a wavelength different from that of the visible light detection pixel, and has a wavelength in addition to the visible light image. A near-infrared light image taken by two or more different near-infrared lights is taken. The imaging elements used at that time include, for example, three types of visible light pixels having different detection wavelengths described in International Publication No. 2018/155486 and International Publication No. 2018/207817, and two or more types having different detection wavelengths. Examples include those in which near-infrared light pixels are provided on the same substrate, but the present invention is not limited to these, and in addition to red light R, green light G, and blue light B, two or more types having different wavelengths are used. Anything that can detect near-infrared light (multi-spectrum of near-infrared light) is sufficient.
例えば、可視光画素で、赤色光Ｒ、緑色光Ｇ、青色光Ｂを検出し、近赤外光画素で、Ｒと相関のある近赤外光、Ｇと相関のある近赤外光、Ｂと相関のある近赤外光を検出する撮像素子を用いた場合、可視光の光量が少ない環境だけでなく、暗闇（０ルクス）のように可視光がない環境においても、通常照度で撮影した場合と同等の情報を有する画像を撮影することができる。即ち、本実施形態の脈波計測システムでは、通常照度から暗闇（０ルクス）まで広範な照度環境で対象者を撮影可能であり、対象者がいる環境によらず精度よく脈波を計測することが可能となる。
For example, the visible light pixel detects red light R, green light G, and blue light B, and the near-infrared light pixel detects near-infrared light that correlates with R, near-infrared light that correlates with G, and B. When an imaging device that detects near-infrared light that correlates with is used, the image was taken with normal illuminance not only in an environment where the amount of visible light is small, but also in an environment where there is no visible light such as darkness (0 lux). It is possible to take an image having the same information as the case. That is, in the pulse wave measurement system of the present embodiment, the subject can be photographed in a wide range of illuminance environments from normal illuminance to darkness (0 lux), and the pulse wave can be measured accurately regardless of the environment in which the subject is present. Is possible.
本実施形態の脈波計測システムでは、情報処理装置１００は、例えば、撮影装置１１０から可視光、近赤外光又はそれらの両方から生成したカラー画像を受信し、カラー画像に含まれる各成分の輝度値ヒストグラムから脈波を検出する。また、例えば、波長が異なる２つの近赤外光成分について、その比率、差分、対数（Ｌｏｇ）の変化から脈波を検出してもよい。その場合、撮影装置でカラー画像を生成せずに、撮像素子で検出した波長が異なる２種類の近赤外光の検出信号をそのまま利用してもよい。
In the pulse wave measurement system of the present embodiment, the information processing apparatus 100 receives, for example, a color image generated from visible light, near-infrared light, or both of them from the photographing apparatus 110, and each component included in the color image is included. The pulse wave is detected from the brightness value histogram. Further, for example, pulse waves may be detected from changes in the ratio, difference, and logarithm (Log) of two near-infrared light components having different wavelengths. In that case, the detection signals of two types of near-infrared light having different wavelengths detected by the image sensor may be used as they are without generating a color image by the photographing apparatus.
ここで、近赤外光のマルチスペクトルとしては、波長域（７００－１２００ｎｍ）、波長数（２～４程度）を用いることができるが、本発明はこれらに限定されるものではなく、検出波長域及び検出波長の数は適宜設定することができる。そして、本実施形態の脈波計測システムでは、出力部は、近赤外線のマルチスペクトルに基づき検出された脈波を出力する。
Here, as the multi-spectrum of near-infrared light, a wavelength range (700-1200 nm) and a wave number (about 2 to 4) can be used, but the present invention is not limited to these, and the detection wavelength is not limited thereto. The number of regions and detection wavelengths can be set as appropriate. Then, in the pulse wave measurement system of the present embodiment, the output unit outputs the pulse wave detected based on the multi-spectrum of the near infrared ray.
以上詳述したように、本実施形態の脈波計測システムでは、可視光成分に加えて、又は可視光成分に代えて、波長が異なる２種類以上の近赤外光成分も用いて脈波を検出しているので、より正確に脈波を検出することができる。なお、本実施形態の脈波計測システムにおける上記以外の構成及び効果は、前述した第１の実施形態及びその変形例と同様である。
As described in detail above, in the pulse wave measurement system of the present embodiment, a pulse wave is generated by using two or more types of near-infrared light components having different wavelengths in addition to or in place of the visible light component. Since it is detected, the pulse wave can be detected more accurately. The configurations and effects of the pulse wave measurement system of the present embodiment other than the above are the same as those of the first embodiment and its modifications described above.
上述した実施形態中で示した各種特徴事項は、互いに組み合わせ可能である。特に、本明細書において「部」とは、例えば、広義の回路によって実施されるハードウェア資源と、これらのハードウェア資源によって具体的に実現されうるソフトウェアの情報処理とを合わせたものも含みうる。また、本実施形態においては様々な情報を取り扱うが、これら情報は、０又は１で構成される２進数のビット集合体として信号値の高低によって表され、広義の回路上で通信・演算が実行されうる。
The various features shown in the above-described embodiments can be combined with each other. In particular, in the present specification, the "part" may include, for example, a combination of hardware resources implemented by circuits in a broad sense and information processing of software that can be concretely realized by these hardware resources. .. Further, in this embodiment, various information is handled, and these information are represented by high and low signal values as a bit set of binary numbers composed of 0 or 1, and communication / calculation is executed on a circuit in a broad sense. Can be done.
また、広義の回路とは、回路（Ｃｉｒｃｕｉｔ）、回路類（Ｃｉｒｃｕｉｔｒｙ）、プロセッサ（Ｐｒｏｃｅｓｓｏｒ）、及びメモリ（Ｍｅｍｏｒｙ）などを少なくとも適当に組み合わせることによって実現される回路である。すなわち、特定用途向け集積回路（Ａｐｐｌｉｃａｔｉｏｎ Ｓｐｅｃｉｆｉｃ Ｉｎｔｅｇｒａｔｅｄ Ｃｉｒｃｕｉｔ：ＡＳＩＣ）、プログラマブル論理デバイス（例えば、単純プログラマブル論理デバイス（Ｓｉｍｐｌｅ Ｐｒｏｇｒａｍｍａｂｌｅ Ｌｏｇｉｃ Ｄｅｖｉｃｅ：ＳＰＬＤ）、複合プログラマブル論理デバイス（Ｃｏｍｐｌｅｘ Ｐｒｏｇｒａｍｍａｂｌｅ Ｌｏｇｉｃ Ｄｅｖｉｃｅ：ＣＰＬＤ）、及びフィールドプログラマブルゲートアレイ（Ｆｉｅｌｄ Ｐｒｏｇｒａｍｍａｂｌｅ Ｇａｔｅ Ａｒｒａｙ：ＦＰＧＡ））などを含むものである。
Further, a circuit in a broad sense is a circuit realized by at least appropriately combining a circuit (Circuit), circuits (Circuitry), a processor (Processor), a memory (Memory), and the like. That is, an integrated circuit for a specific application (Application Special Integrated Circuit: ASIC), a programmable logic device (for example, a simple programmable logic device (Simple Programmable Logical Device: SPLD), a composite programmable logic device (Complex Program)) It includes a programmable gate array (Field Programmable Gate Array: FPGA) and the like.
１００、１０１ 情報処理装置
１１０ 撮影装置
２０１ 制御部
２０２ 記憶部
２０３ 入力部
２０４ 表示部
２０５ 通信部
３０１ 受信部
３０２ 抽出部
３０３ 正規化部
３０４ 算出部
３０５ 出力部
８０１ 除去部 100, 101Information processing device 110 Imaging device 201 Control unit 202 Storage unit 203 Input unit 204 Display unit 205 Communication unit 301 Reception unit 302 Extraction unit 303 Normalization unit 304 Calculation unit 305 Output unit 801 Removal unit
１１０ 撮影装置
２０１ 制御部
２０２ 記憶部
２０３ 入力部
２０４ 表示部
２０５ 通信部
３０１ 受信部
３０２ 抽出部
３０３ 正規化部
３０４ 算出部
３０５ 出力部
８０１ 除去部 100, 101
Claims (10)
- 対象者を撮影した画像に含まれる前記対象者の肌領域の輝度値分布から、そのピーク値を基準にして特定範囲のデータを抽出する抽出部と、
前記抽出されたデータについて、ピークの高さが１になるように正規化する正規化部と、
前記正規化されたデータに対して標準偏差を算出する算出部と、
前記標準偏差を時系列順に並べて前記対象者の脈波を出力する出力部と
を有する情報処理装置。 An extraction unit that extracts data in a specific range based on the peak value from the luminance value distribution of the subject's skin area included in the image of the subject.
A normalization unit that normalizes the extracted data so that the peak height becomes 1.
A calculation unit that calculates the standard deviation for the normalized data, and
An information processing device having an output unit that outputs the pulse wave of the subject by arranging the standard deviations in chronological order. - 撮影装置から前記撮影画像を受信する受信部を更に有する請求項１に記載の情報処理装置。 The information processing device according to claim 1, further comprising a receiving unit that receives the captured image from the photographing device.
- 前記出力部は前記対象者の脈波を外部装置に出力する請求項１又は２に記載の情報処理装置。 The information processing device according to claim 1 or 2, wherein the output unit outputs the pulse wave of the target person to an external device.
- 前記画像に含まれる前記対象者の肌領域の輝度むらを除去する除去部を有し、
前記抽出部は、前記除去部によって輝度むらが除去された輝度値分布から前記データを抽出する請求項１～３のいずれか１項に記載の情報処理装置。 It has a removing portion for removing uneven brightness in the skin area of the subject included in the image.
The information processing apparatus according to any one of claims 1 to 3, wherein the extraction unit extracts the data from a brightness value distribution in which brightness unevenness is removed by the removal unit. - 前記除去部は、前記肌領域を複数の領域に分割し、分割した複数の領域ごとに輝度値分布を算出し、各輝度値分布のピーク位置を基準位置にあわせ、すべての輝度値分布の和を求めることで、輝度むらを除去した輝度値分布を求める請求項４に記載の情報処理装置。 The removal unit divides the skin region into a plurality of regions, calculates a luminance value distribution for each of the divided regions, aligns the peak position of each luminance value distribution with a reference position, and sums all the luminance value distributions. The information processing apparatus according to claim 4, wherein the luminance value distribution from which the luminance unevenness is removed is obtained by obtaining the luminance value distribution.
- 前記画像は、波長が異なる２以上の近赤外光を用いて撮影されたものであり、
前記抽出部は、前記近赤外光成分の輝度値分布からデータを抽出する請求項１～５のいずれか１項に記載の情報処理装置。 The image was taken using two or more near-infrared lights having different wavelengths.
The information processing apparatus according to any one of claims 1 to 5, wherein the extraction unit extracts data from the brightness value distribution of the near-infrared light component. - 撮影画像に含まれる対象者の肌領域を複数の領域に分割し、分割した複数の領域ごとに輝度値の標準偏差を求め、求めた標準偏差が一定値以下の領域の割合に基づき、前記対象者の脈の有無を判定する情報処理装置。 The skin area of the subject included in the captured image is divided into a plurality of areas, the standard deviation of the brightness value is obtained for each of the divided multiple areas, and the target is based on the ratio of the area where the obtained standard deviation is a certain value or less. An information processing device that determines the presence or absence of a person's pulse.
- 対象者の肌領域を撮影する撮影装置と、
請求項１～７のいずれか１項に記載の情報処理装置と
を有する脈波計測システム。 A shooting device that captures the skin area of the subject,
A pulse wave measurement system including the information processing device according to any one of claims 1 to 7. - 前記撮影装置は、波長が異なる２以上の近赤外光を少なくとも検出する固体撮像素子を備え、前記近赤外光成分を含む画像を前記情報処理装置に送信する請求項８に記載の脈波計測システム。 The pulse wave according to claim 8, wherein the photographing apparatus includes a solid-state imaging device that detects at least two or more near-infrared lights having different wavelengths, and transmits an image containing the near-infrared light component to the information processing apparatus. Measurement system.
- 対象者を撮影した画像に含まれる前記対象者の肌領域の輝度値分布から、そのピーク値を基準にして特定範囲のデータを抽出する抽出機能と、
前記抽出されたデータについて、ピークの高さが１になるように正規化する正規化機能と、
前記正規化されたデータに対して標準偏差を算出する算出機能と、
前記標準偏差を時系列順に並べて前記対象者の脈波を出力する出力機能と
をコンピュータに実行させるプログラム。 An extraction function that extracts data in a specific range based on the peak value from the brightness value distribution of the subject's skin area included in the image of the subject.
A normalization function that normalizes the extracted data so that the peak height becomes 1.
A calculation function that calculates the standard deviation for the normalized data, and
A program that causes a computer to execute an output function that outputs the pulse wave of the subject by arranging the standard deviations in chronological order.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2021544020A JPWO2021045140A1 (en) | 2019-09-03 | 2020-09-03 | |
| CN202080059292.XA CN114270399A (en) | 2019-09-03 | 2020-09-03 | Information processing device, pulse wave measurement system, and pulse wave measurement program |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2019-160514 | 2019-09-03 | ||
| JP2019160514 | 2019-09-03 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2021045140A1 true WO2021045140A1 (en) | 2021-03-11 |
Family
ID=74853176
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/JP2020/033394 WO2021045140A1 (en) | 2019-09-03 | 2020-09-03 | Information processing device, pulse wave measurement system, and pulse wave measurement program |
Country Status (3)
| Country | Link |
|---|---|
| JP (1) | JPWO2021045140A1 (en) |
| CN (1) | CN114270399A (en) |
| WO (1) | WO2021045140A1 (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2014198202A (en) * | 2013-03-29 | 2014-10-23 | 富士通株式会社 | Pulse wave detection device, pulse wave detection method, and pulse wave detection program |
| JP2014200390A (en) * | 2013-04-02 | 2014-10-27 | 富士通株式会社 | Pulse wave detection device, pulse wave detection method, and pulse wave detection program |
| JP2016220915A (en) * | 2015-05-29 | 2016-12-28 | 株式会社リコー | Pulse wave detection device, pulse wave detection method, pulse wave detection system, and program |
-
2020
- 2020-09-03 JP JP2021544020A patent/JPWO2021045140A1/ja active Pending
- 2020-09-03 CN CN202080059292.XA patent/CN114270399A/en active Pending
- 2020-09-03 WO PCT/JP2020/033394 patent/WO2021045140A1/en active Application Filing
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2014198202A (en) * | 2013-03-29 | 2014-10-23 | 富士通株式会社 | Pulse wave detection device, pulse wave detection method, and pulse wave detection program |
| JP2014200390A (en) * | 2013-04-02 | 2014-10-27 | 富士通株式会社 | Pulse wave detection device, pulse wave detection method, and pulse wave detection program |
| JP2016220915A (en) * | 2015-05-29 | 2016-12-28 | 株式会社リコー | Pulse wave detection device, pulse wave detection method, pulse wave detection system, and program |
Also Published As
| Publication number | Publication date |
|---|---|
| JPWO2021045140A1 (en) | 2021-03-11 |
| CN114270399A (en) | 2022-04-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10758119B2 (en) | Automatic fundus image capture system | |
| CN108197546B (en) | Illumination processing method and device in face recognition, computer equipment and storage medium | |
| US10285624B2 (en) | Systems, devices, and methods for estimating bilirubin levels | |
| US8855412B2 (en) | Systems, methods, and apparatus for image processing, for color classification, and for skin color detection | |
| JP6545658B2 (en) | Estimating bilirubin levels | |
| US11069057B2 (en) | Skin diagnostic device and skin diagnostic method | |
| TWI632894B (en) | Heart rate activity detecting system based on motion images and method thereof | |
| KR101631012B1 (en) | Image processing apparatus and image processing method | |
| KR20150024769A (en) | Imaging apparatus and imaging method thereof, image processing apparatus and image processing method thereof, and program | |
| JP2010521272A (en) | A method for providing automatic quality feedback to an imaging device to achieve standardized imaging data | |
| JPWO2020059565A1 (en) | Depth acquisition device, depth acquisition method and program | |
| Finlayson | Colour and illumination in computer vision | |
| US10691924B2 (en) | Biological information detection device and biological information detection method | |
| US10346953B2 (en) | Flash and non-flash images in flash artifact removal | |
| US9147115B2 (en) | Method and device for detecting an object in an image | |
| JP2009123081A (en) | Face detection method and photographing apparatus | |
| WO2021045140A1 (en) | Information processing device, pulse wave measurement system, and pulse wave measurement program | |
| CN107527011B (en) | Non-contact skin resistance change trend detection method, device and equipment | |
| Low et al. | Experimental study on multiple face detection with depth and skin color | |
| JP2020021314A (en) | Image processing system and image processing method | |
| JP7237768B2 (en) | Biological information detector | |
| Bai et al. | An integrated design of a smartphone interface with multiple image processing methods to reduce shining light from a digital projector | |
| US10483315B2 (en) | Image sensor configured for dual mode operation | |
| US11315359B2 (en) | Biological-information detecting device and biological-information detecting method | |
| US20240080536A1 (en) | Information processing apparatus, imaging system, information processing method, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20859875 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| ENP | Entry into the national phase |
Ref document number: 2021544020 Country of ref document: JP Kind code of ref document: A |
|
| 32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 07.06.2022) |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 20859875 Country of ref document: EP Kind code of ref document: A1 |