WO2017047734A1 - Dispositif de mesure - Google Patents

Dispositif de mesure Download PDF

Info

Publication number
WO2017047734A1
WO2017047734A1 PCT/JP2016/077391 JP2016077391W WO2017047734A1 WO 2017047734 A1 WO2017047734 A1 WO 2017047734A1 JP 2016077391 W JP2016077391 W JP 2016077391W WO 2017047734 A1 WO2017047734 A1 WO 2017047734A1
Authority
WO
WIPO (PCT)
Prior art keywords
subject
processing circuit
respiratory
image processing
respiration
Prior art date
Application number
PCT/JP2016/077391
Other languages
English (en)
Japanese (ja)
Inventor
淳史 堀
Original Assignee
シャープ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by シャープ株式会社 filed Critical シャープ株式会社
Publication of WO2017047734A1 publication Critical patent/WO2017047734A1/fr

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/08—Detecting, measuring or recording devices for evaluating the respiratory organs
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb

Definitions

  • the present invention relates to a technique for measuring a biological activity caused by respiration, such as a subject's respiration rate, from a subject's video.
  • respiration such as a subject's respiration rate
  • the subject is described as being a person, but may be an animal other than a person. Animals (including people) as observation targets may be collectively referred to as “subjects”.
  • a technique in which a subject is photographed with a camera, a change in luminance value due to a biological reaction such as body movement or blood flow is detected from the moving image, and a biological activity such as a respiratory rate and a heart rate of the subject is measured.
  • the image area in which the subject is shown is specified by an observer in advance or by using a contour extraction technique.
  • a measuring device for measuring a biological activity caused by respiration there are a respiratory monitoring device and a heart rate measuring device.
  • Patent Literatures 1 and 2 disclose such a device.
  • the respiration monitoring device of Patent Literature 1 divides an image obtained by photographing a subject into local regions and analyzes lightness information of each local region. Then, using the three types of threshold values, it is determined whether the subject is observing movement around the chest or observing non-respiratory body movement such as turning over.
  • the heart rate measuring device of Patent Document 2 captures a subject's face with a camera equipped with an infrared light source, extracts a specific region between eyebrows from a face image for each frame, and corrects the average luminance.
  • the heart rate measuring device obtains a waveform of a temporal change in corrected luminance from the corrected average luminance time series, and calculates the heart rate of the subject by filtering the waveform in a frequency band corresponding to the heart rate. .
  • an appropriate threshold necessary for determining non-respiratory body movement varies greatly depending on the imaging environment.
  • the threshold value to be set can vary greatly depending on changes in the brightness of the observation location, the position of the indoor light source, the presence or absence of incident light from the outside, and the movement of people or objects other than the subject. For this reason, it is difficult to obtain an appropriate threshold value.
  • the threshold value is inappropriate, it is not possible to calculate a region for obtaining biological information such as respiration.
  • the heart rate measuring device of Patent Document 2 needs to capture the subject's face within the imaging range. Similar to Patent Document 1, when the shooting environment changes such as a change in illuminance, movement of a person, incidence of external light, etc., the luminance value of the image area in which the subject is photographed changes greatly due to a cause other than biological activity. When such disturbance noise occurs, the body movement location due to the biological reaction cannot be specified, and the biological information may not be extracted accurately.
  • the subject's face is separated from the camera, the accuracy of acquiring the subject's biological information is reduced, so the subject's face must be imaged from a relatively short distance. As a result, a feeling of pressure is given to the subject, and there is a concern about the influence on the biological activity to be measured.
  • the conventional respiratory monitoring device and heart rate measuring device can never be said to be robust against changes in the shooting environment.
  • the life activity measuring apparatus is required to be further improved in robustness against changes in the photographing environment.
  • the present invention has been made to solve the above-described problems, and is a robust measurement system for life activity caused by respiration (hereinafter simply referred to as “measurement”). System ").
  • a measurement apparatus is a measurement apparatus that measures a biological activity of a subject using a moving image of the subject generated by an imaging device that has received light emitted from a light source.
  • An input interface that receives a moving image; and an image processing circuit that measures a subject's biological activity using the moving image, wherein the moving image has a reflection pattern at a generation position of body movement accompanying breathing of the subject.
  • the image processing circuit calculates a coordinate position of the reflection pattern in at least one frame image of the plurality of frame images, and calculates a plurality of divided regions.
  • the image processing circuit measures a biological activity caused by respiration of the subject using an index for specifying a respiration start point on the respiration waveform of each divided region.
  • the image processing circuit detects an image change between two frame images of the plurality of frame images, and uses a respiratory waveform relating to the frame image from which the image change is detected as a breath of the subject. It may not be used for measurement of the resulting biological activity.
  • the image processing circuit has an average luminance of each divided region lower than an average luminance in the region of the reflection pattern and higher than an average luminance of a region outside the reflection marker in a predetermined period.
  • the plurality of divided regions may be set.
  • the image processing circuit changes the size of each divided area until at least the maximum value, the minimum value, and the average of the luminance values of the divided areas in a predetermined period satisfy a predetermined condition.
  • the plurality of divided areas may be set.
  • the image processing circuit may include a low-pass filter or a filter bank that removes noise of the respiratory waveform for each divided region.
  • the image processing circuit numerically differentiates the respiration waveform that has passed through the low-pass filter or filter bank, so that n time-series minimum points (n is an integer of 1 or more) in the respiration waveform are obtained. May be identified as a candidate for the breathing origin, which means the starting point of breathing in or breathing out.
  • the image processing circuit includes an i-th (i is an integer from 1 to n) first n out of the n candidates for the respiratory start point, and a previous one before the first candidate. Focusing on the second candidate, which is a local minimum point, tail-side amplitude indicating a difference in luminance value between the local maximum point between the first and second candidate local minimum points and the local minimum point of the first candidate , Using the index including a head side amplitude indicating a difference in luminance value between the maximum point and the minimum point of the second candidate, and a head tail ratio indicating a ratio between the tail side amplitude and the head side amplitude. Then, it may be determined whether or not the first candidate is the respiratory start point.
  • the image processing circuit has the tail side amplitude equal to or greater than a first threshold value, the head side amplitude equal to or greater than a second threshold value, and the head tail ratio is substantially equal to 1.
  • the first candidate may be determined as the respiratory start point.
  • the image processing circuit may determine whether or not the first candidate is the respiratory start point by the number corresponding to the respiratory start candidate of the respiratory waveform.
  • the image processing circuit uses the respiratory waveform that has passed through the low-pass filter or filter bank to set a minimum luminance value in the respiratory waveform as a lower limit value, and the lower limit value.
  • the threshold value of the respiratory start point may be calculated as the index by adding the amplitude addition value to the index, and the respiratory start point may be determined based on the threshold value of the respiratory start point.
  • the image processing circuit may update a threshold value of the respiratory start point every update period.
  • the image processing circuit is between two adjacent respiratory start points on the respiratory waveform in which the sign of the difference between the luminance value and the threshold value of the respiratory start point changes in the same direction.
  • a local minimum point is specified, and a local maximum point between the first local point and the second local minimum point is focused on the first local minimum point and the second local minimum point that is one time earlier than the first local minimum point.
  • a tail side amplitude indicating a difference in luminance value between a point and the first minimum point, a head side amplitude indicating a difference in luminance value between the maximum point and the second minimum point, and the tail side
  • the index including a head tail ratio indicating a ratio between an amplitude and the head-side amplitude may be calculated.
  • the image processing circuit selects a divided region from the plurality of divided regions based on the respiratory waveform and the index for each divided region, and the test processing is performed based on the respiratory start point of the selected divided region. You may measure the biological activity resulting from body respiration.
  • the head tail ratio is approximately 1 over a predetermined period, and the average value of the tail side amplitude and the head side amplitude in the predetermined period is large, or the variance is respectively
  • a small divided region may be selected from the plurality of divided regions, and the biological activity resulting from the breathing of the subject may be measured based on the respiratory start point of the selected divided region.
  • the image processing circuit selects a candidate for a divided region in which an average value of the head tail ratio is equal to or greater than a third threshold value over a predetermined period from the plurality of divided regions.
  • the biological activity resulting from the breathing of the subject may be measured based on the breathing start point of the divided region where the average value of the tail side amplitude and the head side amplitude is maximum in the predetermined period.
  • the image processing circuit calculates a weighted average value of the head tail ratio by weighting such that the weight becomes smaller as past data, and the weighted average is equal to or greater than the third threshold value. Candidates may be selected.
  • the measurement device further includes a display device that displays a measurement result of life activity resulting from the breathing of the subject, and the display device displays the respiratory rate of the subject and the trend of the respiratory rate. You may display the waveform to show and the said moving image.
  • a measurement system includes a light source that emits light, an imaging device that receives the light and generates a moving image, and an image processing circuit that measures a biological activity of a subject using the moving image.
  • a retroreflecting material having a reflection pattern is arranged at a position where body movement accompanying breathing of the subject is arranged, and the light is emitted from the light source toward the subject
  • the imaging device receives the light reflected by the retroreflecting material at a plurality of times, generates the moving image composed of a plurality of time-series frame images
  • the image processing circuit includes: The moving image is received from the imaging device, the coordinate position of the reflection pattern in at least one frame image among the plurality of frame images is calculated, and a plurality of divided regions are defined as the coordinate position Based on the respiratory waveform of each of the plurality of divided regions, the respiratory waveform is generated in each of the plurality of divided regions to generate a luminance waveform indicating a change in luminance value over the plurality of frame
  • a measurement method includes a light source that emits light, an imaging device that receives the light to generate a moving image, and an image processing circuit that measures the biological activity of a subject using the moving image.
  • a measurement system comprising: a retroreflective material having a reflection pattern at a position where a body motion accompanying breathing of the subject occurs A step of irradiating the subject with the light, and the imaging device receives reflected light reflected by the retroreflecting material at a plurality of times, and a plurality of time-series frames.
  • Generating the moving image composed of images, and the image processing circuit receives the moving image from the imaging device, and the at least one frame image among the plurality of frame images.
  • a computer program includes a light source that emits light, an imaging device that receives the light and generates a moving image, and an image processing circuit that measures a subject's biological activity using the moving image.
  • a computer program executed by the image processing circuit in a measurement system comprising: a retroreflective material having a reflection pattern is disposed at a position where body movement accompanying breathing of the subject occurs; A step of receiving a moving image generated by the imaging device when the light is emitted toward the subject, wherein the time series of the time series based on the light reflected by the retroreflecting material
  • FIG. 1 is a configuration diagram of a measurement system 100 according to a first embodiment. It is a hardware block diagram of the information processing apparatus 30 by 1st Embodiment. It is a flowchart which shows the procedure of the measurement process performed with the measurement system 100 by 1st Embodiment. It is a schematic diagram which shows the frame image 102 which image
  • FIG. 1 It is a schematic diagram which shows a mode that a respiratory waveform changes according to the influence of the body movement of persons other than the test subject 1, and the test subject 1. It is a schematic diagram which shows a mode that the respiration waveform acquired from each area
  • FIG. It is a graph which shows a respiration waveform when division area 51 is set up appropriately. It is a graph which shows the respiratory waveform when the division area 51 is not set appropriately. It is a flowchart which shows the procedure of the setting of the some division area 51.
  • FIG. It is a flowchart which shows the procedure which determines a respiratory origin by numerical differentiation.
  • FIG. 4 is a schematic view illustrating display contents displayed on a display 32.
  • FIG. It is a flowchart which shows the procedure which determines a respiratory starting point using the threshold value of a respiratory starting point.
  • FIG. It is a schematic diagram which shows the respiratory waveform of a certain division
  • FIG. It is a schematic diagram which shows a mode that the threshold value of a respiratory origin is updated for every predetermined period.
  • a measurement system includes a light source that emits light, an imaging device that receives light to generate a moving image, and an image processing circuit that measures a subject's biological activity using the moving image.
  • An infrared light source is preferably used as the light source.
  • the image processing circuit receives a moving image from the imaging device, calculates a coordinate position of the reflection pattern in at least one frame image among the plurality of frame images, and at least a part of the region of the reflection pattern in each frame image Are set based on the coordinate position of the reflection pattern.
  • the image processing circuit generates a respiratory waveform indicating a change in luminance value over a plurality of frame images in each divided region, and measures a biological activity resulting from the breathing of the subject based on the respiratory waveform in each divided region.
  • a divided region most suitable for measurement is selected from a plurality of divided regions using various techniques, and a living body caused by breathing of the subject is selected based on the respiratory waveform of the divided region. It becomes possible to measure activity.
  • the measurement system 100 identifies the respiratory start point on the respiratory waveform.
  • the respiratory start point is used to measure a biological activity resulting from the subject's breathing.
  • the breathing start point means the starting point of breathing in or breathing out.
  • the respiration rate is mainly measured based on the respiration start point.
  • the respiratory rate is an example of a biological activity resulting from the subject's breathing, and other biological activities resulting from the subject's breathing may be measured.
  • the measurement system measures the breathing motion of the subject, and derives a waveform (a waveform corresponding to the breathing waveform) resulting from breathing from body motion due to breathing.
  • biological activities that can be evaluated using the waveform, for example, biological activities such as breathing depth, turbulence, apnea periods, frequency of occurrence of apnea periods, This is the category of the biological activity to be measured.
  • the measurement system can display the respiration rate and the respiration rate trend on the display.
  • FIG. 1 schematically shows a configuration of a measurement system 100 according to the present embodiment.
  • the measurement system 100 includes a camera 10, a light source 20, an information processing device 30, and a retroreflecting material 40. Although the subject 1 is shown in FIG. 1, the subject 1 is not included in the measurement system 100.
  • the measurement system 100 is used for observing the biological activity of the subject 1.
  • the biological activity is the respiration of the subject 1, and the measurement system 100 measures the respiration rate within a predetermined period.
  • the camera 10 is a so-called imaging device having an image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) sensor and an optical system, and shoots the subject 1 to generate a moving image.
  • the camera 10 sends moving image data to the information processing apparatus 30 by wire or wirelessly.
  • CMOS Complementary Metal Oxide Semiconductor
  • the light source 20 is a light source that emits light 20a.
  • the light may be visible light or invisible light (for example, infrared light).
  • infrared light will be described as an example.
  • the light 20a is described as “infrared light 20a”.
  • the information processing apparatus 30 receives a moving image from the camera 10, specifies a respiratory start point in the respiratory waveform using a method as described later, and measures the respiratory rate of the subject 1 based on the respiratory start point. Details of the operation of the information processing apparatus 30 will be described later.
  • the retroreflective material 40 is a reflective material having an optical characteristic of reflecting incident light toward the incident direction. That is, the incident angle of light incident on the retroreflecting material 40 is equal to the emission angle of light reflected by the retroreflecting material 40. However, this property is ideal and can actually be reflected in a direction different from some incident directions.
  • the optical axis of the light source 20 and the optical axis of the camera 10 are arranged close to each other. Thereby, the infrared light 20a emitted from the light source 20 is reflected by the retroreflecting material 40, and most of the light enters the camera 10 as infrared light 20b. Therefore, the camera 10 can photograph the subject 1 with a sufficient amount of light.
  • the retroreflecting material 40 a cloth coated with glass beads can be used as the retroreflecting material 40.
  • the disturbance light 21a incident on the retroreflecting material 40 is reflected in the incident direction as reflected light 21b. Since the reflected light 21b does not substantially enter the camera 10, the moving image captured by the camera 10 is not easily affected by disturbance light.
  • FIG. 2 shows an example of the hardware configuration of the information processing apparatus 30 of the measurement system 100.
  • the information processing apparatus 30 is electrically connected to the camera 10 and the display 32.
  • the information processing apparatus 30 receives captured moving image data from the camera 10.
  • the display 32 is a result of the processing, and displays a measurement result of the respiratory rate, which is a biological activity of the subject 1, a respiratory rate trend, and the like.
  • the information processing apparatus 30 may display a warning on the display 32 when determining based on the measurement result that the shooting direction of the camera 10 is not appropriate.
  • the information processing apparatus 30 includes a CPU 301, a ROM 302, a RAM 303, a hard disk drive (HDD) 304, an interface (I / F) 305, and an image processing circuit 306.
  • the CPU 301 controls the operation of the information processing apparatus 30.
  • the ROM 302 stores a computer program.
  • the computer program is a group of instructions for causing the CPU 301 or the image processing circuit 306 to perform processing shown by a flowchart described later, for example.
  • a RAM 303 is a work memory for developing a computer program when executed by the CPU 301.
  • the HDD 304 is a storage device that stores moving image data received from the camera 10 or measured respiratory rate data of the subject 1.
  • the I / F 305 is an interface for the information processing apparatus 30 to receive moving image data from the camera 10.
  • the I / F 305 may be an Ethernet (registered trademark) terminal, for example.
  • the I / F 305 may be a transmission / reception circuit that performs communication conforming to the Wi-Fi (registered trademark) standard, for example.
  • the I / F 305 may be a wired video input terminal.
  • the image processing circuit 306 is a so-called graphics processor that analyzes moving image data.
  • the image processing circuit 306 receives a moving image from the camera 10, calculates the coordinate position of the reflection pattern of the retroreflecting material 40 in at least one frame image among the plurality of frame images, and at least one of the areas of the reflection pattern.
  • a plurality of divided regions 51 (see FIG. 9A) surrounding the part are set at the same position in each frame image based on the coordinate position.
  • the image processing circuit 306 generates a respiratory waveform indicating a change in luminance value over a plurality of frame images for each divided region 51, and measures the respiratory rate of the subject 1 based on the respiratory waveform in the divided region 51. Details of the reflection pattern and the divided region 51 will be described later.
  • the image processing circuit 306 is provided separately from the CPU 301, but this is an example.
  • An integrated circuit in which the CPU 301 and the image processing circuit 306 are integrated may be used, or the CPU 301 may perform a part of the processing of the image processing circuit 306.
  • FIG. 3 shows a procedure of measurement processing performed in the measurement system 100. This process is mainly executed by the CPU 301 and / or the image processing circuit 306. In the following description, it is assumed that the execution subject is the image processing circuit 306.
  • Step S1 First, the observer or the subject 1 arranges the retroreflecting material 40 having a predetermined reflection pattern as a reflection marker at the position where the body movement accompanying breathing of the subject 1 occurs.
  • the camera 10 receives the infrared light reflected by the retroreflecting material 40 and captures a moving image of the subject 1.
  • FIG. 4 illustrates a frame image 102 obtained by photographing the subject 1 wearing the retroreflecting material 40.
  • a high luminance area (white area) 104 corresponding to the reflection pattern in the center of the image is an area where the reflected light from the retroreflecting material 40 is detected.
  • the information processing apparatus 30 recognizes the area as a reflection marker.
  • FIG. 5 illustrates a frame image 106 obtained by photographing a subject who does not wear the retroreflecting material 40. When the retroreflecting material 40 is not present, it can be said that the luminance change in the captured frame image is very small.
  • 4 and 5 show a plurality of vertical lines and horizontal lines, which are boundary lines virtually provided for image processing. Note that the area of the image partitioned by the boundary line is different from the divided area 51.
  • FIG. 6 shows an example of the reflection pattern of the retroreflecting material 40 that functions as a reflective marker.
  • the retroreflecting material 40 can include, for example, a triangular, rectangular, and diamond-shaped reflection pattern.
  • the reflectance of the high reflection region 200 corresponding to the reflection pattern is relatively higher than the reflectance of the surrounding region.
  • the high reflection region 200 corresponds to the high luminance region 104 shown in FIG.
  • FIG. 7 shows how the respiratory waveform changes due to the influence of a person other than the subject 1 and the body movement of the subject 1.
  • FIG. 7A shows the state of the subject 1 on the bed
  • FIG. 7B shows the state where a person other than the subject 1 crosses the front of the bed
  • FIG. 7C shows the subject on the bed. 1 shows a state of turning over.
  • FIG. 7D shows a change in luminance value in each state.
  • the waveform (i) indicates a normal respiration waveform
  • the waveform (ii) indicates a respiration waveform that has fluctuated due to the influence of disturbance or body movement of the subject 1.
  • a conventional measurement system that measures a respiration rate or the like generates a respiration waveform based on a temporal change in an average luminance value of a certain area (single block) shown in FIG. 4 or 5, for example. It is necessary to acquire a respiration waveform that faithfully reproduces the respiration movement, and to accurately specify the respiration start point from the respiration waveform. However, if a person other than the subject 1 crosses the front of the bed or the subject 1 turns over during the measurement, the respiratory waveform changes greatly as shown in FIG. Including other noise components. If a breathing waveform including noise is used, the breathing start point may be specified incorrectly, and an accurate breathing rate may not be measured.
  • FIG. 8 shows a state in which the respiratory waveforms acquired from each region differ between different regions A, B, and C depending on the positional relationship between the subject 1 and the camera 10.
  • the subject 1 has various breathing modes. For this reason, even if the angle of view is fixed, the state of minute vibration of the reflective marker that occurs with body movement differs for each set region. Therefore, whether or not an optimal respiration waveform can be acquired for the measurement of the respiration rate largely depends on how the region is set.
  • the image processing circuit 306 detects whether the frame image has changed due to, for example, a person other than the subject 1 or the body movement of the subject 1. More specifically, the image processing circuit 306 detects the change using, for example, an inter-frame difference or a background difference.
  • the image processing circuit 306 is configured not to use the respiration waveform acquired in the region or period in the frame image where the change is detected for the measurement of the respiration rate.
  • an example of detection using inter-frame differences will be described.
  • the inter-frame difference is the difference between the frame image acquired in the time series and the i-th acquired frame image and the i-th acquired frame image. This is a technique for specifying an area.
  • i and n are integers of 1 or more.
  • the image processing circuit 306 obtains a difference value by taking a difference between the frame images for each pixel of the frame image. A pixel whose difference value exceeds a predetermined threshold value is described as a “change pixel”, and a region where the change pixels are gathered is described as a “change block”. The image processing circuit 306 determines that the frame image has changed if the number of changed pixels or changed blocks is equal to or greater than a predetermined threshold value.
  • the image processing circuit 306 can detect a change between frame images not in units of pixels but in units of blocks.
  • the image processing circuit 306 uses, for example, an 8 ⁇ 8 pixel region as a block (corresponding to an image region partitioned by a boundary line shown in FIGS. 4 and 5) as a unit of luminance average of pixels in the block. You may calculate and take those differences between frame images.
  • the image processing circuit 306 estimates the area from the reflection marker to the head, hand, and foot of the subject 1 and applies the above detection process only to that area. I do not care.
  • the image processing circuit 306 may estimate the region of each part (for example, face, hand, and foot) of the subject and detect whether or not the part has moved.
  • the image processing circuit 306 detects a change in the image, the image processing circuit 306 does not use the respiration waveform acquired during the period in which the change is detected to measure the respiration rate. Thereby, the influence to the respiration measurement by the test subject's 1 body movement and disturbance can be suppressed.
  • the image processing circuit 306 can filter the generated respiratory waveform using a low-pass filter. Since the data for the order of the low-pass filter (for example, 16th order) includes components other than respiration, the image processing circuit 306 uses the data for that order (for example, 16 samples) for the measurement of the respiration rate when filtering. I try not to.
  • the data for the order of the low-pass filter for example, 16th order
  • the image processing circuit 306 uses the data for that order (for example, 16 samples) for the measurement of the respiration rate when filtering. I try not to.
  • the image processing circuit 306 receives a moving image from the camera 10 and specifies a coordinate position of a reflection pattern in at least one frame image among a plurality of frame images using, for example, a known pattern recognition method.
  • the coordinate position of the reflection pattern means, for example, the coordinates of each vertex, center, or center of gravity of the high reflection region 200 shown in FIG.
  • the image processing circuit 306 can also specify the coordinate position of the reflection pattern using the corner detection method and the edge detection method.
  • a specifying method is described in Japanese Patent Application No. 2015-102726, which is an unpublished patent application filed by the present applicant. All of these disclosures are incorporated herein by reference.
  • the image processing circuit 306 specifies the coordinate position of the reflection pattern in at least one of the plurality of frame images. For example, the image processing circuit 306 updates the coordinate position every 10 seconds.
  • the image processing circuit 306 specifies the coordinate position of the reflection pattern in the image every 300 frames. In other words, the information regarding the coordinate position is updated every 300 frames.
  • the update interval corresponds to the subject's 1 breathing rate of 6 times.
  • FIG. 9A illustrates a respiratory waveform when the divided region 51 is appropriately set.
  • FIG. 9B illustrates a breathing waveform when the divided region 51 is not set appropriately.
  • the monitoring area 50 is set so as to surround the reflection marker of the retroreflecting material 40.
  • the monitoring area 50 is composed of a plurality of divided areas 51.
  • the monitoring area 50 includes a plurality of divided areas 51 that divide the reflective marker area into, for example, 3 ⁇ 3 or more.
  • the size of the divided area 51 is determined depending on the resolution and angle of view of the camera 10 and can be set to 16 ⁇ 16 pixels, for example.
  • FIG. 9A shows a waveform acquired when the divided region P is set at a position where the respiratory waveform is faithfully reproduced. It can be seen that the respiratory waveform does not include distortion and is drawn by a smooth curve. Hereinafter, such a respiratory waveform is described as “normal respiratory waveform”.
  • normal respiratory waveform By setting the divided region 51 at an appropriate position, a normal respiratory waveform is acquired, and the respiratory rate of the subject 1 can be accurately measured.
  • FIG. 9B shows a waveform obtained when the divided region P is set at a position where the respiratory waveform is not faithfully reproduced.
  • the respiratory waveform includes a distorted waveform such as a rectangular wave in a part thereof, and cannot be said to be drawn by a smooth curve.
  • a normal respiratory waveform may not be acquired depending on the position and size of the partial region 51. If a partially distorted respiratory waveform is used, there is a possibility that the respiratory start point described later will be erroneously specified. As a result, the respiratory rate of the subject 1 cannot be accurately measured.
  • the image processing circuit 306 based on the coordinate information of the reflection marker acquired in step S3, includes a plurality of divided regions 51 so as to surround at least a part of the region of the reflection pattern. Is set to the same position in each frame image.
  • the movement (up and down movement) of the reflective marker is performed in each area during the period of inhalation and exhalation (that is, half the period of one respiration period). It is preferable to employ a divided region 51 that occurs within the range of.
  • the image processing circuit 306 has a luminance average of the divided area 51 lower than the luminance average in the area of the reflective pattern and the luminance average of the area outside the reflective marker during the period of inhaling and exhaling. It is preferable to set a plurality of divided regions 51 so as to be higher. The image processing circuit 306 repeats the setting of the plurality of divided areas 51 while changing the position and size of the divided areas 51 until this condition is satisfied.
  • FIG. 10 shows an example of a procedure for setting a plurality of divided areas 51.
  • the procedure corresponds to the subroutine of step S4.
  • Step S41 can be provided arbitrarily.
  • the image processing circuit 306 generates an integrated image from the moving image acquired from the camera 10.
  • the use of the integral image can speed up the subsequent calculation processing, so it is preferable to provide step 41.
  • step 41 below, the example which performs arithmetic processing using an integral image is demonstrated.
  • the image processing circuit 306 sets a plurality of divided regions 51 so as to surround at least a part of the region of the reflection pattern in each frame image based on the coordinate information of the reflection marker acquired in step S3.
  • the image processing circuit 306 first sets the size of each divided region 51 to, for example, 8 ⁇ 8 pixels.
  • the image processing circuit 306 calculates the luminance average of the pixels for each divided region 51 and generates a respiratory waveform based on time-series data indicating a change in luminance average over a predetermined period (for example, the past 20 seconds). Alternatively, the image processing circuit 306 may generate a respiration waveform based on time-series data indicating a change in the integrated value of the pixels in the divided region during a predetermined period.
  • a predetermined period for example, the past 20 seconds
  • the image processing circuit 306 may generate a respiration waveform based on time-series data indicating a change in the integrated value of the pixels in the divided region during a predetermined period.
  • “brightness of an area” refers to, for example, an average value, an integrated value, or a representative value of pixels in the area.
  • the image processing circuit 306 generates 12 respiratory waveforms for the 3 ⁇ 4 divided region 51.
  • the predetermined period of 20 seconds corresponds to a general breathing rate of 3 bpm of the subject.
  • the image processing circuit 306 generates a respiratory waveform based on time-series data for the past 20 seconds (corresponding to 600 frames when the frame rate is 30 fps) retroactively from the current time while accessing the RAM 303 and the HDD 304.
  • the image processing circuit 306 calculates, for each divided region 51, the minimum value, maximum value, and average value of the above-mentioned “region luminance” from a respiratory waveform generated based on, for example, time-series data for the past 20 seconds.
  • the image processing circuit 306 can individually determine whether or not all of the minimum value, maximum value, and average value of the luminance satisfy a predetermined condition for each divided region 51. More specifically, the image processing circuit 306 can determine whether or not the minimum value, maximum value, and average value of the luminance are within a predetermined range. More specifically, in the image processing circuit, the maximum value and the minimum value of the luminance of the region are the maximum value ⁇ “luminance average in the region of the reflection pattern” ⁇ “threshold value” and the minimum value> “reflection”. It can be determined whether or not the condition “luminance average of the area outside the marker” + “threshold value” is satisfied.
  • step S45 If the number of blocks of the divided areas 51 that satisfy the above-described conditions among the plurality of divided areas 51 is equal to or greater than a preset threshold value (Yes in S45), the process proceeds to the next step S5 for specifying the respiratory start point. To do. If the number of the divided areas 51 satisfying this condition is less than a preset threshold value (No in S45), the process returns to step S42 again.
  • the image processing circuit 306 changes the size and position of the divided areas 51 and sets a plurality of divided areas 51 again.
  • the image processing circuit 306 repeats the processing from step S42 to S45 until the condition of step S45 is satisfied.
  • the image processing circuit 306 determines the respiratory start point for each divided region 51 by numerically differentiating the respiratory waveform.
  • the respiration waveform generated in step S4 includes noise components other than respiration. If the respiratory waveform containing noise is differentiated as it is, the noise component will be emphasized, so that the respiratory start point cannot be specified accurately. Therefore, the image processing circuit 306 removes noise from the respiratory waveform using a filter 307 (see FIG. 2) before performing numerical differentiation.
  • FIG. 11 shows a procedure for determining the respiratory start point by numerical differentiation. The procedure corresponds to the subroutine of step S5.
  • the image processing circuit 306 includes a filter 307 (see FIG. 2) that removes noise in the respiratory waveform for each divided region 51.
  • the filter 307 is a low-pass filter or a filter bank. For example, assuming a respiration rate of 0 to 150 bpm, a low-pass filter having a cutoff frequency of 2.5 Hz can be used.
  • the image processing circuit 306 uses a filter that allows a signal from 0 to 2.5 Hz to pass, there is a possibility that the breathing start point may be erroneously detected when the subject 1 breathes slowly. Considering this point, it can be said that it is preferable to use a filter bank composed of a plurality of filters, for example.
  • the image processing circuit 306 may select an optimal respiration waveform, for example, a respiration waveform having the largest amplitude, from respiration waveforms that have passed through each filter of the filter bank.
  • the respiratory waveform is differentiated and a candidate for the respiratory start point is extracted: Step S52)
  • the image processing circuit 306 numerically differentiates (primary differentiation) the respiratory waveform for each divided region 51 that has passed through the filter 307.
  • the image processing circuit 306 extracts n time-sequential minimum points (n is an integer of 1 or more) in the respiratory waveform obtained by differentiation as candidates for the respiratory start point.
  • the minimum point corresponding to the candidate for the respiratory start point is a so-called upward zero-cross point where the differential value changes from minus to plus.
  • Step S53 Determining whether the breathing start point or not: Step S53. If the respiratory start candidate obtained in step S52, that is, the upward zero-cross point is determined as it is as the respiratory start point, erroneous detection increases in the measurement of the respiratory rate. The reason is that an upward zero-cross point may occur due to subtle vibrations other than breathing, disturbance light, ringing due to filtering, and the like. In order to prevent erroneous detection, the image processing circuit 306 determines whether or not the respiratory start candidate is the respiratory start point.
  • FIG. 12 shows a state where a specific respiration start point is determined from a plurality of respiration start candidates in the respiration waveform.
  • FIG. 13 shows a tail point, a head point, a max point, a tail side amplitude, and a head side amplitude in the respiratory waveform.
  • the image processing circuit 306 selects the i-th first candidate among the n candidates for the respiratory start point and the second candidate that is one time earlier in time series than the first candidate.
  • i is an integer from 1 to n.
  • the first candidate point is expressed as “tail point”
  • the second candidate point is expressed as “head point”
  • the first and second candidate minimum points that is, between the tail point and the head point
  • the maximum point is expressed as “Max Point”.
  • the luminance value difference between the max point and the tail point is expressed as “tail side amplitude (“ TA ”in FIG. 13)”
  • the luminance value difference between the max point and the head point is “ The head-side amplitude (“HA” in FIG. 13) is expressed.
  • the ratio between the tail side amplitude and the head side amplitude is referred to as a “head tail ratio”.
  • the image processing circuit 306 determines whether or not the first candidate, that is, the tail point (i), is the respiration start point, using three indexes for each divided region 51. Specifically, the image processing circuit 306 determines the tail point (i) as the respiratory start point when the tail side amplitude and the head side amplitude are each equal to or greater than a predetermined threshold and the head tail ratio is substantially equal to 1. Is determined.
  • the predetermined threshold is determined according to the size of the reflective marker on the image and the size of the divided area 51. For example, when the luminance value of a pixel is indicated by an 8-bit signal, the predetermined threshold value can be set to 3.
  • the head tail ratio may be tail side amplitude / head side amplitude, or head side amplitude / tail side amplitude.
  • the image processing circuit 306 selects, from the plurality of divided areas 51, a divided area 51 that is optimal for measuring the respiration rate based on the respiratory waveform and respiratory start information for each divided area 51.
  • the image processing circuit 306 specifies a respiration start point used for respiration rate measurement in the respiration waveform of the selected divided region 51.
  • FIG. 14 shows an example of a processing procedure for selecting the optimum divided area 51.
  • the procedure corresponds to the subroutine of step S6.
  • each average value of the head-side amplitude and the tail-side amplitude is calculated for each divided region 51: Step S61)
  • the image processing circuit 306 averages the head-side amplitude and tail-side amplitude for the past several beats (for example, 60 seconds) for each divided region 51. Note that the image processing circuit 306 may obtain variances of the head-side amplitude and the tail-side amplitude for the past several beats. In the divided area 51 that is optimal for measuring the respiration rate, each average value is large and the variance is small.
  • the image processing circuit 306 calculates an average value of head tail ratios for the past several beats for each divided region 51.
  • the image processing circuit 306 may calculate the weighted average for each divided region 51 by weighting the head tail ratio for the past several beats. At that time, weighting is performed so that the weight becomes smaller as the past data is obtained.
  • the divided region 51 that is most suitable for measuring the respiratory rate has a feature that the change in the head tail ratio for the past several beats is small and the head tail ratio is substantially equal to one. In other words, the dispersion is small.
  • the image processing circuit 306 can select the optimum divided region 51 on condition of any of the following (A) to (C).
  • the image processing circuit 306 specifies a respiration start point used for respiration rate measurement in the respiration waveform of the selected divided region 51.
  • A The head tail ratio for the past several beats is substantially constant, and the average values of the tail side amplitude and the head side amplitude for the past several beats are large.
  • B The head tail ratio for the past several beats is substantially constant, and the tail-side amplitude and head-side amplitude variance for the past several beats are small.
  • the head tail ratio for the past several beats is substantially constant at 1, and the tail-side amplitude and head-side amplitude variance for the past several beats are equal to or greater than a predetermined threshold, and the past In several beats, the average value of the tail side amplitude and the head side amplitude is the largest.
  • the predetermined threshold value can be set to 0.1.
  • the image processing circuit 306 selects a candidate for a divided region in which the average of the head tail ratios for the past several beats is equal to or greater than a predetermined threshold value.
  • the image processing circuit 306 may select, as the optimum divided region 51, the divided region 51 in which the average values of the tail side amplitude and the head side amplitude for the past several beats are the largest among the divided region candidates. .
  • the head tail ratio represented by the above formula (1) is used, and the predetermined threshold value can be halved, for example.
  • the image processing circuit 306 has a large tail side amplitude and a head side amplitude among the divided regions A, B, and C, and a small dispersion of the head tail ratio.
  • the divided area A can be determined as the optimum divided area 51.
  • FIG. 15 shows the respiratory start point specified in step S6 and the respiratory cycle (s) in the respiratory waveform, which are used for measuring the respiratory rate.
  • the time difference between two adjacent breathing origins is the breathing cycle (s) of the subject 1 breathing.
  • the respiration rate (bpm) is expressed as 60 / respiration cycle (s).
  • the image processing circuit 306 measures the respiration rate of the subject 1 based on the respiration waveform of the optimum divided region 51 selected in step S6 and the respiration start information. For example, in the plurality of respiration waveforms shown in FIG. 8, the image processing circuit 306 measures the respiration rate of the subject 1 based on the respiration waveform of the divided area A and the respiration start information. The image processing circuit 306 can count the respiration rate within a predetermined period.
  • FIG. 16 shows an example of display contents displayed on the display 32.
  • the display 32 displays the measurement result of the biological activity resulting from the respiration of the subject 1.
  • the information regarding the measurement result of the biological activity includes information indicating the respiratory rate of the subject 1 and the trend of the respiratory rate.
  • Information indicating the trend of the respiratory rate is displayed on the display 32 as a waveform indicating a temporal change in the respiratory rate.
  • the information on the respiration rate is updated at a predetermined interval.
  • the display 32 also displays a breathing waveform indicating a change in the luminance value of the optimum divided area 51. Further, system information indicating the state of the measurement system 100 is displayed.
  • the information means, for example, the state of a program that is running internally such as during search, during measurement, and during stop.
  • the moving image captured by the camera 10 is displayed on the display 32 in real time.
  • a rectangular frame that identifies the detection position of the reflective marker and the divided region 51 used for measuring the respiration rate can also be displayed superimposed on the moving image.
  • an operator for example, a doctor of the measurement system 100 can confirm on the display 32 that the reflective marker is accurately recognized. If the reflective marker is not accurately arranged, the detection position of the reflective marker and the respiration rate are not displayed on the display 32, so the operator can surely confirm this problem.
  • a measurement system having high robustness against changes in the surrounding environment is provided.
  • the image processing circuit 306 according to the second embodiment is configured to determine the respiratory start point using a threshold value for determining the respiratory start point (hereinafter referred to as “the threshold value of the respiratory start point”). This is different from the image processing circuit 306 according to the first embodiment.
  • the respiratory start point according to the present embodiment refers to a point on the respiratory waveform where the luminance value is equal to the threshold value of the respiratory start point.
  • description of common parts will be omitted, and description will be made focusing on differences.
  • FIG. 17 shows a procedure for determining the respiratory start point using the threshold value of the respiratory start point.
  • the procedure corresponds to a subroutine of step S6 different from the first embodiment.
  • the image processing circuit 306 determines the respiratory start point according to the procedure of FIG.
  • Step S51 Removal of respiratory waveform noise for each divided region 51: Step S51.
  • the image processing circuit 306 removes respiratory waveform noise for each divided region 51.
  • FIG. 18 shows a respiratory waveform of a certain divided area 51.
  • FIG. 19 shows a state in which the threshold value of the respiratory start point is updated every predetermined period.
  • the image processing circuit 306 sets a lower limit value for calculating a threshold value of the respiratory start point based on data for a predetermined period. For example, the image processing circuit 306 sets the minimum value of the luminance value in the past 20 seconds as the lower limit value.
  • the lower limit value is updated every predetermined period, for example, every 20 seconds.
  • the image processing circuit 306 adds the added value to the lower limit value to obtain the threshold value for the respiratory start point.
  • the added value can be “1”.
  • the threshold value of the respiratory start point is also updated every predetermined period.
  • FIG. 19 shows how the DC component of the respiratory waveform varies in three sections. It can be seen that the DC component of the respiratory waveform varies from section to section. In the present embodiment, the threshold value of the respiratory start point is updated for each section according to the fluctuation of the DC component. As a result, the respiratory start point can be accurately determined following the fluctuation of the DC component.
  • the image processing circuit 306 determines the respiratory start point based on the threshold value of the respiratory start point. More specifically, for example, the image processing circuit 306 determines a point on the respiration waveform where the sign of the difference between the luminance value and the threshold value of the respiration start point changes from negative to positive as the respiration start point. Alternatively, the image processing circuit 306 may determine a point on the respiration waveform where the sign of the sign changes to the respiration start point.
  • the image processing circuit 306 determines the optimal divided region, specifies the respiration start point (step S6), and determines the respiration rate of the subject, as in the first embodiment. Is measured (step S7). More specifically, for example, the image processing circuit 306 is located between two adjacent respiratory start points on a respiratory waveform in which the sign of the difference between the luminance value and the threshold value of the respiratory start point changes from negative to positive. A certain local minimum point is grasped, and the head side amplitude, the tail side amplitude, and the head tail ratio are calculated based on the local minimum points. The image processing circuit 306 can determine an optimum divided region based on the same conditions described in the first embodiment using those indices.
  • the measurement system 100 according to the first embodiment is less susceptible to fluctuations in the DC component of the respiratory waveform than the measurement system 100 according to the second embodiment. For this reason, it can be said that it is preferable to use the measurement system 100 according to the first embodiment in a measurement environment where the fluctuation of the DC component is remarkable.
  • a measurement system having high robustness against changes in the surrounding environment is provided.
  • a measurement system that is not easily affected by minute noise is provided.
  • the measurement system 100 according to the third embodiment is different from the measurement system 100 according to the first embodiment in that the camera 10 is equipped with an optical filter.
  • the camera 10 is equipped with an optical filter.
  • the camera 10 may be equipped with an optical filter (not shown) that blocks the wavelength in the visible light region.
  • the optical filter is also called an infrared filter, for example.
  • the optical filter transmits infrared light emitted from the light source 20 and reflected by the retroreflecting material 40, but blocks visible light.
  • the present embodiment by providing an optical filter, it is possible to prevent light other than infrared light, more specifically, visible light from entering the camera 10, thereby changing the luminance value of the captured moving image. Can be reduced. Since the fluctuation of the luminance value of each frame image due to visible light can be suppressed, it is possible to effectively reduce the generation of disturbance noise that is caused only by visible light and is not caused by a biological reaction. In other words, a change in luminance value due to only infrared light reflected by the retroreflecting material 40 can be reliably captured.
  • the measurement system 100 that assumes real-time processing of a moving image captured by the camera 10 has been described, but the present invention is not limited to this.
  • the moving image of the subject 1 captured by the camera 10 may be temporarily stored in an external memory or the like.
  • the image processing circuit 306 may read the moving image data from the external memory and measure the respiration rate of the subject 1 afterwards based on the data.
  • the information processing apparatus 30 including the image processing circuit 306 may be a component of the measurement system 100 or may be a single measurement apparatus that can be distributed to the market independently.
  • This specification discloses a measurement apparatus, a measurement system, a measurement method of a biological activity caused by respiration of a subject, and a computer program described in the following items.
  • a measuring device that measures a subject's biological activity using a moving image of a subject generated by an imaging device that has received light emitted from a light source, An input interface for receiving the moving image; An image processing circuit for measuring the biological activity of the subject using the moving image; With In the moving image, a retroreflecting material having a reflection pattern is arranged at a position where a body movement accompanying breathing of the subject is arranged, and when the light is emitted from the light source toward the subject, a plurality of the moving images are provided.
  • the image processing circuit includes: Calculating a coordinate position of the reflection pattern in at least one frame image of the plurality of frame images, and setting a plurality of divided regions in each frame image based on the coordinate position; In each of the plurality of divided regions, a respiration waveform indicating a change in luminance value over the plurality of frame images is generated, and a living body resulting from respiration of the subject based on the respiration waveform of each of the plurality of divided regions A measuring device that measures activity.
  • a robust measurement device for life activity caused by breathing is provided in which measurement conditions for life activity are not easily influenced by the surrounding environment.
  • the respiratory start point can be accurately identified.
  • the image processing circuit detects an image change between two frame images of the plurality of frame images, and generates a respiratory waveform related to the frame image from which the image change has been detected, as a result of the biological activity caused by the breathing of the subject.
  • Item 3 The measuring device according to Item 2, which is not used for measurement.
  • the measurement device described in item 3 it is possible to suppress the influence on the respiratory measurement due to the body movement or disturbance of the subject.
  • the image processing circuit is configured so that, in a predetermined period, the average luminance of each divided region is lower than the average luminance in the region of the reflective pattern and higher than the average luminance in the region outside the reflective marker. 4.
  • a normal respiration waveform can be acquired and the respiration rate of the subject can be accurately measured.
  • the image processing circuit is configured to change the size of each divided region until the at least maximum value, minimum value, and average of luminance values of the divided regions in a predetermined period satisfy a predetermined condition. 4. The measuring device according to item 2 or 3, wherein
  • a normal respiration waveform can be acquired and the respiration rate of the subject can be accurately measured.
  • noise can be removed from the respiratory waveform.
  • the image processing circuit numerically differentiates the respiratory waveform that has passed through the low-pass filter or filter bank, thereby obtaining n time-series minimum points (n is an integer of 1 or more) in the respiratory waveform.
  • Item 7 The measuring device according to item 6, wherein the measuring device is specified as a candidate for the breathing start point, which means a start point of breathing in or breathing out.
  • a measurement device that is less susceptible to fluctuations in the DC component of the respiratory start point is provided.
  • the image processing circuit is an i-th (i is an integer from 1 to n) first candidate out of n candidates for the respiratory start point, and a first minimum point one prior to the first candidate. Focus on the two candidates, A tail-side amplitude indicating a difference in luminance value between a maximum point between the first and second candidate minimum points and the first candidate minimum point; A head-side amplitude indicating a difference in luminance value between the maximum point and the minimum point of the second candidate; The measuring device according to item 7, wherein the first candidate is determined to be the respiratory start point using the index including a head tail ratio indicating a ratio between the tail side amplitude and the head side amplitude.
  • the measurement device described in item 8 it is possible to more accurately determine whether or not the candidate for the respiratory start is the respiratory start using the three indicators.
  • the measurement device described in item 9 it is possible to more accurately determine whether or not the candidate for the respiratory start point is the respiratory start point using the three indicators.
  • the image processing circuit uses the respiratory waveform that has passed through the low-pass filter or the filter bank, sets a minimum luminance value in the respiratory waveform as a lower limit value, and sets an amplitude addition value to the lower limit value. Add and calculate the breathing origin threshold as the indicator, Item 7.
  • a measurement device that is not easily affected by minute noise is provided.
  • a measurement device that is not easily affected by minute noise is provided.
  • the image processing circuit identifies a minimum point between two adjacent respiratory start points on the respiratory waveform in which the sign of the difference between the luminance value and the threshold value of the respiratory start point changes in the same direction. , Paying attention to the first local minimum point and the second local minimum point in time series before the first local minimum point, A tail-side amplitude indicating a difference in luminance value between a maximum point between the first and second minimum points and the first minimum point; A head-side amplitude indicating a difference in luminance value between the maximum point and the second minimum point; The measuring device according to item 11 or 12, wherein the index including a head tail ratio indicating a ratio between the tail side amplitude and the head side amplitude is calculated.
  • the image processing circuit selects a divided region from the plurality of divided regions based on the respiratory waveform and the index for each divided region, and is caused by the subject's breathing based on the respiratory start point of the selected divided region. 14.
  • the measuring device according to any one of items 7 to 13, which measures a biological activity to be performed.
  • the head tail ratio is approximately 1 over a predetermined period, and an average value of the tail side amplitude and the head side amplitude in the predetermined period is respectively large, or the divided regions having small dispersion are 14.
  • the measuring device according to any one of items 7 to 10, and 13 selected from a plurality of divided regions and measuring a biological activity caused by the breathing of the subject based on the respiratory start point of the selected divided region.
  • the measurement device described in item 15 it is possible to accurately measure the biological activity resulting from the breathing of the subject based on the breathing start point of the selected divided region.
  • the image processing circuit selects, from the plurality of divided regions, a candidate for a divided region in which an average value of the head tail ratio is equal to or greater than a third threshold value over a predetermined period.
  • a candidate for a divided region in which an average value of the head tail ratio is equal to or greater than a third threshold value over a predetermined period.
  • the biological activity caused by the breathing of the subject is measured based on the respiratory start point of the divided area where the average value of the tail-side amplitude and the head-side amplitude is maximized in the predetermined period.
  • the measuring device according to any one of items 7 to 10, and 13.
  • the image processing circuit calculates a weighted average value of the head tail ratio by weighting that decreases in weight as past data is selected, and selects a candidate for the divided region in which the weighted average is greater than or equal to the third threshold value.
  • [Item 18] Further comprising a display device for displaying a measurement result of the biological activity resulting from the breathing of the subject, The measurement device according to any one of items 1 to 17, wherein the display device displays a respiratory rate of the subject, a waveform indicating a trend of the respiratory rate, and the moving image.
  • the operator or the subject can check the measurement result on the display device.
  • a light source that emits light
  • An imaging device that receives the light and generates a moving image
  • An image processing circuit for measuring a subject's biological activity using the moving image, and a measurement system comprising: When a retroreflecting material having a reflection pattern is arranged at a position where a body movement accompanying breathing of the subject is arranged, and when the light is emitted from the light source toward the subject, The imaging device receives the light reflected by the retroreflecting material at a plurality of times, and generates the moving image composed of a plurality of time-series frame images,
  • the image processing circuit includes: The moving image is received from the imaging device, the coordinate position of the reflection pattern in at least one frame image of the plurality of frame images is calculated, and a plurality of divided regions are included in each frame image based on the coordinate position.
  • a respiration waveform indicating a change in luminance value over the plurality of frame images is generated, and a living body resulting from respiration of the subject based on the respiration waveform of each of the plurality of divided regions
  • a robust measurement system for life activity caused by breathing is provided in which measurement conditions for life activity are not easily affected by the surrounding environment.
  • a light source that emits light
  • An imaging device that receives the light and generates a moving image
  • a measurement method comprising: a measurement system comprising an image processing circuit for measuring a subject's life activity using the moving image; and a measurement method for measuring a life activity resulting from respiration of the subject, Placing a retroreflective material having a reflection pattern at a position where body movement associated with breathing of the subject occurs; and The light source irradiating the subject with the light;
  • the imaging device receives reflected light reflected by the retroreflecting material at a plurality of times, and generates the moving image composed of a plurality of time-series frame images;
  • the image processing circuit receives the moving image from the imaging device, calculates a coordinate position of the reflection pattern in at least one frame image of the plurality of frame images, and sets a plurality of divided regions as the coordinate positions.
  • the image processing circuit generates a respiratory waveform indicating a change in luminance value over the plurality of frame images in each of the plurality of divided regions, and the subject is based on the respiratory waveform of each of the plurality of divided regions.
  • a light source that emits light
  • An imaging device that receives the light and generates a moving image
  • a computer program executed by the image processing circuit in a measurement system comprising: an image processing circuit that measures a subject's biological activity using the moving image; When a retroreflecting material having a reflection pattern is arranged at a position where body movement occurs due to breathing of the subject, and when the light is emitted from the light source toward the subject, Receiving a moving image generated by the imaging device, the moving image including a plurality of time-series frame images based on the light at a plurality of times reflected by the retroreflecting material; When, Calculating a coordinate position of the reflection pattern in at least one frame image of the plurality of frame images, and setting a plurality of divided regions in each frame image based on the coordinate positions; In each of the plurality of divided regions, a respiration waveform indicating a change in luminance value over the plurality of frame images is generated, and a living body resulting from respiration of the subject
  • the present invention can be used as a method for analyzing a moving image obtained by photographing a subject and measuring the life activity of the subject, particularly the number of breaths, in a non-contact manner.
  • the present invention can also be used as an apparatus, system, and computer program for analyzing such moving images and measuring life activity.

Abstract

[Problème] Fournir un système de mesure robuste pour mesurer une activité organique causée par la respiration, dans lequel les conditions de mesure d'activité organique ne sont pas aisément affectées par l'environnement ambiant. [Solution] La présente invention concerne un système de mesure (100) qui est pourvu d'une source de lumière (20), un dispositif d'imagerie (10) et un dispositif de traitement d'informations (30). Un matériau réfléchissante récursif (40) ayant un motif réfléchissant est disposé à une position où le mouvement qui accompagne la respiration d'un sujet (1) est généré. Lorsque de la lumière est émise depuis une source de lumière vers le sujet, le dispositif de traitement d'informations reçoit une image animée depuis le dispositif d'imagerie, calcule les positions de coordonnées d'un motif réfléchissant dans au moins une image de trame parmi une pluralité d'images de trame, et définit une pluralité de régions de division dans les images de trame sur la base des positions de coordonnées. L'activité organique causée par la respiration du sujet est mesurée sur la base d'une forme d'onde de respiration générée dans chacune des régions de division.
PCT/JP2016/077391 2015-09-16 2016-09-16 Dispositif de mesure WO2017047734A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2015-182737 2015-09-16
JP2015182737A JP2017055949A (ja) 2015-09-16 2015-09-16 計測装置、計測システム、計測方法およびコンピュータプログラム

Publications (1)

Publication Number Publication Date
WO2017047734A1 true WO2017047734A1 (fr) 2017-03-23

Family

ID=58289498

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2016/077391 WO2017047734A1 (fr) 2015-09-16 2016-09-16 Dispositif de mesure

Country Status (2)

Country Link
JP (1) JP2017055949A (fr)
WO (1) WO2017047734A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114831356A (zh) * 2022-07-05 2022-08-02 深圳市恒尔创科技有限公司 基于移动终端的电子烟控制方法及相关设备

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7094184B2 (ja) * 2018-09-03 2022-07-01 リンナイ株式会社 監視装置
JP7405527B2 (ja) * 2019-07-12 2023-12-26 グローリー株式会社 変化タイミング検知装置、変化タイミング検知方法及び変化タイミング検知プログラム
JPWO2022168246A1 (fr) * 2021-02-05 2022-08-11

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11276443A (ja) * 1998-03-27 1999-10-12 Toshiba Corp 被介護者観測装置及びその方法
JP2006507088A (ja) * 2002-11-25 2006-03-02 バリアン・メディカル・システムズ・テクノロジーズ・インコーポレイテッド 対象の呼吸活動をモニターする方法およびシステム
JP2008522522A (ja) * 2004-11-30 2008-06-26 ザ ユニバーシティー コート オブ ザ ユニバーシティー オブ セント アンドリューズ ビデオフィンガープリントのシステム、方法、及びコンピュータプログラム製品
US20140334697A1 (en) * 2013-05-08 2014-11-13 Koninklijke Philips N.V. Device for obtaining a vital sign of a subject
JP2015523132A (ja) * 2012-06-12 2015-08-13 コーニンクレッカ フィリップス エヌ ヴェ カメラによる生命徴候測定システム

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4703049B2 (ja) * 2001-07-17 2011-06-15 住友大阪セメント株式会社 監視装置
WO2014136027A1 (fr) * 2013-03-06 2014-09-12 Koninklijke Philips N.V. Système et procédé pour déterminer des informations de signes vitaux
US20140276104A1 (en) * 2013-03-14 2014-09-18 Nongjian Tao System and method for non-contact monitoring of physiological parameters
US20150073281A1 (en) * 2013-09-11 2015-03-12 Xerox Corporation Generating a flow-volume loop for respiratory function assessment

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH11276443A (ja) * 1998-03-27 1999-10-12 Toshiba Corp 被介護者観測装置及びその方法
JP2006507088A (ja) * 2002-11-25 2006-03-02 バリアン・メディカル・システムズ・テクノロジーズ・インコーポレイテッド 対象の呼吸活動をモニターする方法およびシステム
JP2008522522A (ja) * 2004-11-30 2008-06-26 ザ ユニバーシティー コート オブ ザ ユニバーシティー オブ セント アンドリューズ ビデオフィンガープリントのシステム、方法、及びコンピュータプログラム製品
JP2015523132A (ja) * 2012-06-12 2015-08-13 コーニンクレッカ フィリップス エヌ ヴェ カメラによる生命徴候測定システム
US20140334697A1 (en) * 2013-05-08 2014-11-13 Koninklijke Philips N.V. Device for obtaining a vital sign of a subject

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114831356A (zh) * 2022-07-05 2022-08-02 深圳市恒尔创科技有限公司 基于移动终端的电子烟控制方法及相关设备

Also Published As

Publication number Publication date
JP2017055949A (ja) 2017-03-23

Similar Documents

Publication Publication Date Title
US20200260996A1 (en) Method and apparatus for processing a cyclic physiological signal
EP3057487B1 (fr) Dispositif et procédé pour l'obtention d'un signe vital chez un sujet
EP3373804B1 (fr) Dispositif, système et procédé de guidage de position de capteur
EP2560549B1 (fr) Appareil, procédé et programme d'ordinateur de détection de mouvement respiratoire
CN105190691B (zh) 用于获得对象的生命体征的设备
EP2898477B1 (fr) Surveillance de signal vital robuste de mouvement
JP6521845B2 (ja) 心拍に連動する周期的変動の計測装置及び計測方法
WO2016006027A1 (fr) Procédé de détection d'onde d'impulsion, programme de détection d'onde d'impulsion et dispositif de détection d'onde d'impulsion
WO2017047734A1 (fr) Dispositif de mesure
US20210153756A1 (en) Reliable acquisition of photoplethysmographic data
JP2016193021A (ja) 脈波検出装置、及び脈波検出プログラム
JPWO2020054122A1 (ja) 情報処理装置、プログラム及び情報処理方法
Wiede et al. Signal fusion based on intensity and motion variations for remote heart rate determination
JP2016214937A (ja) 計測システムおよびコンピュータプログラム
JP6280650B2 (ja) 被験体の呼吸に起因する生体活動の計測方法および計測システム
JP6002811B1 (ja) 計測システム、被験体の呼吸に起因する生体活動の計測方法およびコンピュータプログラム
US20240127469A1 (en) Touchless volume waveform sampling to determine respiration rate
Nijholt Contactless respiration monitoring using a 3D camera system
CN116115214A (zh) 一种呼吸速率的检测方法、装置、电子设备及存储介质
WO2016076009A1 (fr) Système de mesure de l'activité biologique due à la respiration d'un sujet et dispositif de traitement d'image
CN117137454A (zh) 生物信息估算装置以及生物信息估算方法
JP2017123918A (ja) 呼吸モニタリング装置、呼吸数計測装置、呼吸数計測方法およびコンピュータプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16846605

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16846605

Country of ref document: EP

Kind code of ref document: A1