WO2016009901A1 - Method for measuring biological activity resulting from respiration of subject, measurement system, and computer program - Google Patents

Method for measuring biological activity resulting from respiration of subject, measurement system, and computer program Download PDF

Info

Publication number
WO2016009901A1
WO2016009901A1 PCT/JP2015/069550 JP2015069550W WO2016009901A1 WO 2016009901 A1 WO2016009901 A1 WO 2016009901A1 JP 2015069550 W JP2015069550 W JP 2015069550W WO 2016009901 A1 WO2016009901 A1 WO 2016009901A1
Authority
WO
WIPO (PCT)
Prior art keywords
light
processing circuit
subject
image processing
retroreflecting material
Prior art date
Application number
PCT/JP2015/069550
Other languages
French (fr)
Japanese (ja)
Inventor
智之 市座
貴行 山内
池田　豊
蔭地　謙作
三木　成一郎
Original Assignee
シャープ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by シャープ株式会社 filed Critical シャープ株式会社
Priority to JP2016534383A priority Critical patent/JP6280650B2/en
Publication of WO2016009901A1 publication Critical patent/WO2016009901A1/en

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/08—Detecting, measuring or recording devices for evaluating the respiratory organs
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb

Definitions

  • the present invention relates to a technique for measuring a biological activity caused by respiration, such as a subject's respiration rate, from a subject's video.
  • a technique in which a subject is photographed with a camera, a change in luminance value due to a biological reaction such as body movement or blood flow is detected from the moving image, and a biological activity such as a subject's respiratory rate or heart rate is measured (for example, Patent Documents 1 and 2).
  • the image area in which the subject is photographed is specified by an observer in advance or by using a contour extraction technique.
  • the respiration monitoring device of Patent Literature 1 divides an image obtained by photographing a subject into local regions and analyzes lightness information of each local region. Then, using the three types of threshold values, it is determined whether the subject is observing movement around the chest or observing non-respiratory body movement such as turning over.
  • the heart rate measuring device of Patent Document 2 captures a subject's face with a camera equipped with an infrared light source, extracts a specific region between eyebrows from a face image for each frame, and corrects the average luminance.
  • the heart rate measuring device obtains a waveform of a temporal change in corrected luminance from the corrected average luminance time series, and calculates the heart rate of the subject by filtering the waveform in a frequency band corresponding to the heart rate. .
  • an appropriate threshold necessary for determining non-respiratory body movement varies greatly depending on the imaging environment.
  • the threshold value to be set can vary greatly depending on changes in the brightness of the observation location, the position of the indoor light source, the presence or absence of incident light from the outside, and the movement of people or objects other than the subject. Since there is no method for always obtaining an appropriate threshold value, an area for obtaining biological information such as respiration cannot be calculated if the threshold value is inappropriate.
  • the heart rate measuring device of Patent Document 2 needs to capture the subject's face within the imaging range. Similar to Patent Document 1, when the shooting environment changes such as a change in illuminance, movement of a person, incidence of external light, etc., the luminance value of the image area in which the subject is photographed changes greatly due to a cause other than biological activity. When such disturbance noise occurs, the body movement location caused by the biological reaction cannot be specified, and the biological information may not be extracted accurately.
  • the subject's face moves away from the camera, the accuracy of acquiring the subject's biological information is reduced, so the subject's face must be imaged from a relatively short distance. As a result, a feeling of pressure is given to the subject, and there is a concern about the influence on the biological activity to be measured.
  • the present invention has been made in order to solve the above-described problems, and is a measurement system for biological activity caused by respiration, a measurement method for biological information, etc. I will provide a.
  • a measurement method includes a light source that emits light, an imaging device that receives the light to generate a moving image, and an image processing circuit that measures the biological activity of a subject using the moving image.
  • a biological activity measurement system comprising: (a) a predetermined reflection pattern at a generation position of body movement associated with respiration of the subject. Placing a retroreflecting material; (b) the light source irradiating the subject with the light; and (c) the imaging device receiving the light reflected by the retroreflecting material.
  • the image processing circuit measures the life activity based on a change in luminance value of the plurality of frame images.
  • the image processing circuit divides each frame image into a plurality of partial areas and measures the biological activity based on a change in luminance value of the plurality of partial areas. .
  • the image processing circuit is a partial region of each frame image using the predetermined reflection pattern, the retroreflective material is included, and The partial area including the position where the body movement is generated is specified, and the biological activity is measured using a change in luminance value of the specified partial area.
  • the image processing circuit holds information for specifying the predetermined reflection pattern in advance, and the image processing circuit uses the information to perform pattern matching processing.
  • the predetermined reflection pattern is detected in each frame image, and the partial region including the retroreflecting material is specified.
  • the image processing circuit specifies a partial region of each frame image including a boundary of the retroreflecting material.
  • the image processing circuit specifies a first direction that is a direction of the body movement, and along a second direction different from the first direction, Each frame image is divided into a plurality of partial areas.
  • the predetermined reflection pattern of the retroreflecting material includes a first portion that reflects the light and a second portion that does not reflect the light in the direction of the imaging device.
  • the second portion is provided around the retroreflecting material.
  • the first portion is provided around the retroreflecting material.
  • the predetermined reflection pattern of the retroreflecting material includes a predetermined pattern.
  • the predetermined reflection pattern of the retroreflective material includes a plurality of portions having different light reflectivities.
  • the predetermined reflection pattern of the retroreflecting material includes a portion where the reflectance of the light continuously changes.
  • the measurement method further includes the step of (e) blocking visible light incident on the imaging device using an optical filter.
  • the wavelength of the light when the wavelength of the light is ⁇ , (f) the light reflected by the retroreflecting material is passed through an optical filter that mainly transmits light of the wavelength ⁇ .
  • the method further includes a step of passing.
  • the light source is a light emitting diode that emits light having a wavelength ⁇ of 850 nm or 940 nm.
  • the measurement method includes: (g) passing the light emitted from the light source through a polarizing element having a predetermined polarization direction; and (h) the light reflected by the retroreflecting material. Passing light through a polarizing element having the same polarization direction as the predetermined polarization direction.
  • the measurement method includes (i) a step of further providing a retroreflecting material having a predetermined shape as a position marker at a fixed position; and (j) the imaging device includes the position marker.
  • the imaging device includes the position marker.
  • the measurement method includes: (l) a warning in step (k) when the image processing circuit determines that the imaging device is not shooting a predetermined direction or position. Is further included.
  • the retroreflective material is made of a non-metallic material.
  • a measurement system includes a light source that emits light, an imaging device that receives the light and generates a moving image, and an image processing circuit that measures a biological activity of a subject using the moving image.
  • a retroreflecting material is disposed at a position where body movement occurs due to breathing of the subject, and when the light is emitted from the light source toward the subject, the measurement system includes: The imaging device receives the light reflected by the retroreflecting material at a plurality of times and generates the moving image composed of a plurality of time-series frame images, and the image processing circuit includes the imaging device The moving image is received from the frame, and the biological activity caused by the breathing of the subject is measured based on the change in the luminance value of the plurality of frame images.
  • the image processing circuit specifies a partial region that includes the retroreflecting material and includes the position where the body motion is generated from each frame image, and the luminance value of the specified partial region is determined.
  • the life activity is measured using the change.
  • each of the frame images has a first partial area having a relatively high luminance value corresponding to the light reflected by the retroreflecting material, and a relative luminance value higher than that of the first partial area.
  • the second partial area is low, and the image processing circuit changes over time the luminance value of the coordinate position across the boundary between the first partial area and the second partial area across the plurality of frame images. The life activity is measured by utilizing this.
  • the image processing circuit includes a partial region including a coordinate position crossed by the boundary, and a coordinate position where a ratio of the first partial region and the second partial region changes with time. With respect to the partial area, a change in the luminance value is detected.
  • the image processing circuit sets the processing unit area at the same coordinate position of each frame image.
  • the size of the partial area is variable.
  • the image processing circuit when the image processing circuit detects that the position of the region including the retroreflecting material has moved by a predetermined amount or more, the image processing circuit again performs the recursion from each frame image.
  • a new partial area that includes a reflective reflector and that includes the position of occurrence of the body movement is specified, and the biological activity is measured using a change in luminance value of the specified new partial area.
  • a computer program includes a light source that emits light, an imaging device that receives the light and generates a moving image, and an image processing circuit that measures a subject's biological activity using the moving image.
  • a computer program executed by an image processing circuit in a measurement system comprising: a retroreflecting material disposed at a position where a body motion accompanying breathing of the subject occurs, and from the light source toward the subject
  • the computer program receives, in the image processing circuit, a moving image generated by the imaging device, and the light reflected by the retroreflecting material is received at a plurality of times.
  • a step of measuring the biological activity resulting from respiration of the subject based on.
  • the light source and the retroreflective material are used in combination, even if the surroundings are dark or bright, and even if the distance between the subject and the imaging device is sufficiently separated, it can be used for biological activities such as breathing.
  • the resulting body movement can be captured from changes in a plurality of frame images. Since the imaging device can be set apart from the subject and the imaging environment can be set to be dark, it is possible to acquire biological information with sufficiently high accuracy while sufficiently reducing the feeling of pressure applied to the subject.
  • FIG. It is a figure which shows the structure of the biological activity measurement system 100 by Embodiment 1.
  • FIG. It is a figure which shows the frame image 102 which image
  • FIG. It is a figure which shows the vibration of the high-intensity area
  • FIG. 2 is a diagram illustrating an example of a hardware configuration of an information processing apparatus 30 of the life activity measurement system 100 mainly.
  • FIG. 4 is a flowchart illustrating a procedure of processing performed in the biological activity measurement system 100.
  • (A) And (b) is a figure which shows the example of the partial area Q in two frame images image
  • (c) is a figure which shows the change of the luminance value of the partial area Q. is there.
  • (A)-(f) is a figure which shows the example of the shape of the retroreflection material 40, respectively.
  • FIG. 1 is a figure which shows the example which mounted
  • (b) is a figure which shows the example of an imaging.
  • (A1) to (c1) are diagrams showing a retroreflecting material 40 provided with a reflecting material functioning as a marker around a triangular reflecting portion, and (a2) to (c2) are respectively shown in the marker. It is a figure which shows the two partial area
  • FIG. (A) And (b) is a figure which shows the example in which the area
  • FIG. (A) And (b) is a figure which shows the example which ensured the partial area
  • (C) is a figure which shows the change of the luminance value of the partial area Q.
  • FIG. (A) And (b) is a figure which shows the example which ensured the partial area Q as a process unit area smaller than the partial area Q of FIG. 8
  • (c) is a figure which shows the change of the luminance value of the partial area Q.
  • FIG. 10 is a flowchart illustrating an operation procedure of the biological information monitoring apparatus 300 according to the second embodiment. It is a figure which shows the life activity measurement system 111 by the modification of the life activity measurement system.
  • FIG. 1 shows a configuration of a life activity measurement system 100 according to the present embodiment.
  • the life activity measurement system 100 includes a camera 10, a light source 20, an information processing device 30, and a retroreflecting material 40. Although the subject 1 is shown in FIG. 1, the subject 1 is not included in the life activity measurement system 100.
  • the life activity measurement system 100 is used for observing the life activity of the subject 1.
  • the life activity is the respiration of the subject 1, and the life activity measurement system 100 measures the respiration rate within a predetermined time.
  • the subject 1 is described as being a person, it may be an animal other than a person. Animals (including people) as observation targets may be collectively referred to as “subjects”.
  • the camera 10 is a so-called imaging device, and shoots the subject 1 to generate a moving image.
  • the camera 10 sends moving image data to the information processing apparatus 30 by wire or wirelessly.
  • the light source 20 is a light source that emits light 20a.
  • the light may be visible light or invisible light (for example, infrared light).
  • infrared light will be described as an example.
  • the light 20a is described as “infrared light 20a”.
  • the information processing apparatus 30 receives moving image data captured by the camera 10 and measures the respiration rate of the subject 1 using changes in images between a plurality of frame images constituting the moving image. Details of the operation of the information processing apparatus 30 will be described later.
  • the retroreflective material 40 is a reflective material having an optical characteristic of reflecting incident light toward the incident direction. That is, the incident angle of light incident on the retroreflecting material 40 is equal to the emission angle of light reflected by the retroreflecting material 40. However, this property is ideal and can actually be reflected in a direction different from some incident directions.
  • the optical axis of the light source 20 and the optical axis of the camera 10 are arranged close to each other. Thereby, the infrared light 20a emitted from the light source 20 is reflected by the retroreflecting material 40, and most of the light enters the camera 10 as infrared light 20b. Therefore, the camera 10 can photograph the subject 1 with a sufficient amount of light.
  • a cloth coated with glass beads is used as the retroreflecting material 40.
  • the disturbance light 21a incident on the retroreflecting material 40 is reflected in the incident direction as reflected light 21b. Since the reflected light 21b does not substantially enter the camera 10, the moving image photographed by the camera 10 is hardly affected by disturbance light.
  • the overall operation of the life activity measurement system 100 is outlined as follows.
  • the observer or the subject 1 arranges the retroreflecting material 40 having a predetermined reflection pattern at the position where the body movement accompanying the breathing of the subject 1 occurs.
  • the camera 10 receives the infrared light reflected by the retroreflecting material 40 and captures a moving image of the subject 1.
  • FIG. 2 shows a frame image 102 obtained by photographing the subject 1 wearing the retroreflecting material 40.
  • a high brightness area (white area) 104 in the center of the image is an area where the reflected light from the retroreflecting material 40 is detected.
  • FIG. 3 shows a frame image 106 obtained by photographing a subject who does not wear the retroreflecting material 40. When the retroreflecting material 40 is not present, it can be said that the luminance change in the captured frame image is very small.
  • FIG. 2 and FIG. 3 show a plurality of vertical lines and horizontal lines, which are boundaries virtually provided for image processing. In this specification, an area of an image divided by a boundary line is referred to as a “partial area” of the image.
  • FIG. 2 illustrates the partial region P. Note that the boundary lines of the partial areas P are highlighted for convenience of understanding.
  • a predetermined reflection pattern is provided on the retroreflecting material 40, and the high-intensity region 104 is detected more reliably and more easily using the reflection pattern.
  • the shape of the reflection pattern does not appear.
  • the information processing apparatus 30 analyzes each of a plurality of time-series frame images constituting the moving image as shown in FIG. 2 and determines the body movement of the subject 1 based on the change in the luminance value of the plurality of frame images. To detect. More specifically, the information processing apparatus 30 detects the high luminance region 104 shown in FIG. 2 over a plurality of frame images. Since the body movement of the subject 1 during calm occurs due to breathing, the position of the high-intensity region 104 changes (vibrates) in accordance with the breathing cycle. The information processing apparatus 30 can measure the respiratory rate of the subject 1 during the period by counting the number of respiratory cycles over a predetermined period, where one period of vibration of the high-intensity region 104 is one respiratory period.
  • the respiratory rate is an example of a biological activity resulting from the subject's breathing
  • other biological activities resulting from the subject's breathing may be measured.
  • a subject's breathing motion is measured, and a waveform resulting from breathing (a waveform corresponding to the breathing waveform) is derived from body motion due to breathing.
  • other biological activities that can be evaluated using the waveform, for example, biological activities such as breathing depth, turbulence, apnea periods, frequency of occurrence of apnea periods, This is the category of the biological activity to be measured.
  • FIG. 4 shows the vibration of the high luminance region 104 measured based on the change of the luminance value of the plurality of frame images.
  • Waveforms observed using the retroreflecting material 40 can accurately measure body movement due to respiration even in a dark imaging environment. That is, it is possible to measure body movement, that is, respiration, using the luminance value.
  • FIG. 5 shows changes in luminance values of a plurality of frame images taken in a dark shooting environment without providing the retroreflecting material 40. Due to the absence of the retroreflecting material 40, the luminance change in the image is originally small, and therefore the influence of noise is very large even if the change in the luminance value is observed over a plurality of frame images.
  • the scale of the vertical axis differs by several times.
  • the scale in FIG. 5 is larger than that in FIG.
  • the direction using the retroreflecting material 40 means that the signal-to-noise ratio (SNR) is superior to the direction not using it (FIG. 5).
  • the retroreflecting material 40 by using the retroreflecting material 40, it is possible to perform imaging while ensuring a sufficient amount of reflected infrared light.
  • the subject 1 and the camera 10 can be set apart by, for example, about 6 m.
  • the room to be measured can be darkened.
  • the environment is less susceptible to changes in the brightness of the observation location, the position of the indoor light source, and the presence or absence of incident light from outside, that is, an environment in which the influence of noise is small. It becomes possible to shoot with. Therefore, it becomes possible to measure the biological activity more accurately.
  • FIG. 6 shows an example of the hardware configuration of the information processing apparatus 30 mainly in the life activity measurement system 100.
  • the information processing apparatus 30 is connected to the camera 10 and the display 32.
  • the information processing apparatus 30 receives captured moving image data from the camera 10.
  • the display 32 displays a measurement result of the number of breaths that is the biological activity of the subject 1 as a result of the processing.
  • the information processing apparatus 30 may display a warning on the display 32.
  • the information processing apparatus 30 includes a CPU 301, a ROM 302, a RAM 303, a hard disk drive (HDD) 304, an interface (I / F) 305, and an image processing circuit 306.
  • the CPU 301 controls the operation of the information processing apparatus 30.
  • the ROM 302 stores a computer program.
  • the computer program is a group of instructions for causing the CPU 301 or the image processing circuit 306 to perform processing shown by a flowchart described later, for example.
  • a RAM 303 is a work memory for developing a computer program when executed by the CPU 301.
  • the HDD 304 is a storage device that stores moving image data received from the camera 10 or measured respiratory rate data of the subject 1.
  • the I / F 305 is an interface for the information processing apparatus 30 to receive moving image data from the camera 10.
  • the I / F 305 is, for example, an Ethernet (registered trademark) terminal.
  • the I / F 305 is a transmission / reception circuit that performs communication based on, for example, the Wi-Fi (registered trademark) standard.
  • the I / F 305 may be a wired video input terminal.
  • the image processing circuit 306 is a so-called graphics processor that analyzes moving image data.
  • the image processing circuit 306 detects a high luminance area of each frame image of the moving image, detects a body movement based on the vibration of the high luminance area, and counts the respiration rate based on the vibration waveform of the body movement.
  • the image processing circuit 306 is provided separately from the CPU 301, but this is an example.
  • the CPU 301 may perform processing of an image processing circuit 306 described later.
  • FIG. 7 shows a procedure of processing performed in the life activity measurement system 100.
  • step S1 the camera 10 photographs the subject 1 wearing the retroreflecting material 40.
  • the captured moving image is sent to the information processing apparatus 30.
  • step S2 the image processing circuit 306 of the information processing apparatus 30 divides each of a plurality of frame images constituting the captured moving image into a plurality of partial areas.
  • the partial region (for example, the partial region P in FIG. 2) has, for example, a size of 64 pixels in the horizontal direction and 64 pixels in the vertical direction. Note that “divide” does not require division as an actual operation. For example, the operation of setting the size of the partial area as a unit for cutting out an image or a unit for performing processing may be included in the “dividing” operation described here.
  • step S3 the image processing circuit 306 specifies a partial region where the retroreflective material 40 exists and a partial region including a body movement location due to a biological reaction based on the luminance value of each partial region. This will be described more specifically.
  • the partial region where the retroreflecting material 40 exists can be specified in each frame image.
  • the partial region including the body movement location can be specified over a plurality of frame images, that is, between the plurality of frame images.
  • the partial area where the retroreflective material 40 exists is specified by the following processing.
  • the image processing circuit 306 holds information on luminance values of partial areas observed when the retroreflecting material 40 exists in the ROM 302 in advance. Using this information as a threshold value of the luminance value, a partial region having a luminance value equal to or higher than the threshold value is specified as a partial region where the retroreflecting material 40 exists.
  • the luminance value at this time may be the sum of the luminance values of the pixels included in the partial area, or may be an average value. Depending on whether the sum of luminance values or the average value is adopted, the threshold may also change. Since the arithmetic processing for calculating the sum is smaller than the arithmetic processing for calculating the average value and the calculation load is small and the processing speed can be increased, the present embodiment is the sum of the luminance values of the pixels included in the partial area. And
  • the partial area including the body movement location is specified by the following process.
  • the region where the reflected light from the retroreflecting material 40 is observed varies due to body movement caused by a biological reaction (respiration).
  • a partial region Q existing at a certain common coordinate position FIGS. 8A and 8B show examples of partial areas Q in two frame images taken at different times.
  • a region R shown in FIGS. 8A and 8B is assumed to be a high luminance region in which reflected light from the retroreflecting material 40 is detected.
  • the partial area Q may or may not become a high luminance area due to body movement accompanying breathing.
  • FIG. 8C shows a change in the luminance value of the partial region Q at this time.
  • step S3 of FIG. 7 the partial area including the body movement location is specified as the partial area Q of the coordinate position indicating the luminance change shown in FIG. Then, a luminance value calculation (image processing) is performed with a group of pixels included in the partial region Q as a group.
  • step S4 the image processing circuit 306 calculates an average luminance value of the partial region Q including the body movement location.
  • step S5 the image processing circuit 306 counts the breathing rate of the subject using each calculated average luminance value.
  • Each calculated average luminance value is expressed as a waveform shown in FIG.
  • the image processing circuit 306 counts the number of breaths within a predetermined period, with one period of body movement specified by the average luminance value of the partial region Q oscillating as one breath.
  • the life activity measurement system 100 can acquire respiratory rate information with high accuracy. Since the amount of infrared light reflected from the retroreflecting material 40 is sufficiently large, when a moving image taken by the camera 10 is used, a partial region including body movement due to respiration is specified from the retroreflecting material 40 in the image. It is easy.
  • FIGS. 9 (a) to 9 (f) show examples of the shape of the retroreflecting material 40, respectively.
  • the reference numeral 40 is omitted.
  • the retroreflecting material 40 in FIGS. 9A to 9D has a triangular shape, and the retroreflecting material 40 in FIGS. 9E and 9F is configured by combining rectangular reflecting materials. .
  • FIG. 9A shows an example of a retroreflecting material 40 in which reflecting materials having different directivities are combined.
  • Some prism type retroreflective materials have directivity in the retroreflecting direction. Therefore, as shown in FIG. 9A, the reflectors a1 and a2 whose reflection directions are shifted by 90 ° are alternately arranged in a strip shape. This eliminates the need to consider directivity when installing the retroreflecting material 40.
  • FIG. 9B shows an example of the retroreflecting material 40 including a part b1 that does not reflect infrared light and a part b2 that reflects infrared light.
  • the part b 1 that does not reflect infrared light is provided at the outer edge of the retroreflecting material 40
  • the part b 2 that reflects infrared light is provided in an area inside the retroreflecting material 40.
  • the frame image is divided into a plurality of partial areas using the retroreflecting material 40 and processing is performed using the partial areas with high luminance, it may be difficult to capture body movements due to respiration. is there.
  • the brightness of the clothes or the background wall being worn is high, it is difficult to determine a partial region including body movement due to respiration based on the brightness. Therefore, a portion that does not reflect infrared light is formed in a part of the retroreflecting material 40, and the luminance change is surely generated by body movement due to respiration.
  • FIG. 9C shows an example of the retroreflecting material 40 whose reflectance changes in a gradation.
  • the intensity of the reflected light from the retroreflective material 40 changes in a gradation. Therefore, for example, it is possible to set a partial region Q that is completely included in the region R of the reflected light from the retroreflecting material 40.
  • FIG. 9D shows a retroreflecting material 40 including a predetermined pattern d1.
  • the pattern d1 can be formed using a material that does not reflect infrared light. Since this pattern also appears in the frame image, the position including the direction of the retroreflecting material 40 can be specified.
  • FIG. 9E shows another example of the retroreflective member 40 including a part e1 that does not reflect infrared light and a part e2 that reflects infrared light. Unlike FIG. 9B, in this example, a portion e ⁇ b> 1 that does not reflect infrared light is provided inside the retroreflecting material 40. Since the number of boundaries increases, the conditions for setting the partial areas are easily adjusted.
  • FIG. 9 (f) shows a retroreflecting material 40 in which the portions f1 and f2 that reflect infrared light are dispersed and provided at a plurality of locations.
  • the portions f2 that reflect infrared light are provided at the four corners of the portion f1 that reflects infrared light.
  • the number of boundaries increases, so the conditions for setting the partial areas are easy to be prepared.
  • FIG. 10A shows an example in which the retroreflecting material 40 of FIG. 9A is attached to the subject 1, and FIG. 10B shows an imaging example.
  • the retroreflective material 40 is installed so that the edge portion of the retroreflective material 40 is located at a location (for example, the abdomen or chest) where body movement due to the breathing of the subject 1 occurs.
  • the image processing circuit 306 of the information processing apparatus 30 holds in advance, for example, in the ROM 302 information indicating that the retroreflecting material 40 having such a pattern is used and the characteristics of the pattern.
  • the image processing circuit 306 identifies a region having a high luminance value in a strip shape as shown in FIG. 10B in each frame image of the obtained moving image.
  • the image processing circuit 306 performs a pattern matching process on each frame image using the characteristics of the pattern held in advance, and specifies the position of the retroreflecting material 40.
  • the image processing circuit 306 divides each frame image into two or more.
  • the image processing circuit 306 sets a dividing line (boundary line) at a position across the retroreflecting material 40. Further, the dividing line is set in a direction different from the body movement direction. For example, it is assumed that body movement is recognized in the vertical direction in the frame image.
  • the image processing circuit 306 sets a boundary line in the horizontal direction, divides each frame image into two or more, and sets a partial region.
  • the image processing circuit 306 can count the respiration rate of the subject 1 using the average luminance value of the partial area.
  • 11 (a1) to 11 (c1) each show a retroreflecting material 40 provided with a reflecting material functioning as a marker around a triangular reflecting portion.
  • a reflecting material functioning as a marker around a triangular reflecting portion.
  • the reflective portion 40a and the marker 40b are shown in FIG. 11 (a1).
  • the shape of the illustrated marker is an example. The shape is arbitrary as long as it can be identified in relation to the reflective portion 40a.
  • the processing when such a retroreflective material is used is generally as described above.
  • the position may be specified by using a retroreflecting material 40 having a known shape including the reflecting portion 40 a and the marker 40 b and divided into partial regions around the retroreflecting material 40.
  • the image processing circuit 306 may divide the marker 40b into two or more partial areas based on the searched surrounding marker 40b.
  • FIG. 11 (a2) shows two partial regions Q1 and Q2 provided in the marker 40b. Note that the shapes or sizes of the partial regions Q1 and Q2 may be different from each other. Depending on what type of retroreflecting material 40 is used, the shapes or sizes of the partial regions Q1 and Q2 may be determined in advance.
  • FIGS. 11B2 and 11C2 are the same as those in FIG. 11 (a1). Further, the partial region may be set in a shape exemplified in FIGS. 11B2 and 11C2.
  • the life activity measurement system 100 can be used in measurement / inspection using a magnetic resonance imaging apparatus by appropriately selecting the retroreflecting material 40.
  • the above-described retroreflecting material 40 may be formed by depositing aluminum.
  • the retroreflective material 40 includes a metal material, the retroreflective material 40 cannot be used for measurement / inspection (so-called MRI measurement / inspection) using a magnetic resonance imaging apparatus. This is because the captured image may be affected in the gantry of the magnetic resonance imaging apparatus.
  • a non-metallic retroreflecting material 40 In an environment where MRI measurement / inspection is performed, it is necessary to employ a non-metallic retroreflecting material 40.
  • a PET prism sheet can be used as the retroreflecting material 40.
  • FIG. 12 shows the camera 10 equipped with an optical filter 11 that blocks the wavelength in the visible light region.
  • This optical filter 11 is also called an infrared filter, for example.
  • the filter 11 is provided to photograph the subject 1.
  • the optical filter 11 transmits infrared light emitted from the light source 20 and reflected by the retroreflecting material 40, but blocks visible light.
  • light other than infrared light more specifically visible light, is prevented from entering the camera 10, thereby affecting the change in the luminance value of the captured moving image. Can be reduced. Since fluctuations in the luminance value of each frame image due to visible light can be suppressed, it is possible to reduce the occurrence of disturbance noise due to only visible light and not due to biological reactions.
  • the inventors of the present application consider that it is very useful to provide the optical filter 11 that blocks the wavelength in the visible light region. This is because it is often difficult to realize a complete dark room environment during actual photographing. For example, when the life activity measurement system 100 is operated in a hospital, a night light, an evacuation guide light, etc. are lit in the hospital even at night. In such a photographing environment, it is preferable to block visible light by the optical filter 11.
  • an optical filter that blocks not only visible light but also unnecessary infrared light may be provided.
  • a band pass filter that allows infrared light emitted from the light source 20 to pass therethrough may be provided as the optical filter 11.
  • an LED light source having a steep wavelength characteristic is adopted as the light source 20.
  • the wavelength is, for example, 850 nm or 940 nm and their vicinity.
  • the “steep wavelength characteristic” means that the fluctuation of the wavelength of the emitted infrared light is small here.
  • the camera 10 is provided with an optical filter having a bandpass characteristic that allows infrared light emitted from the light source 20 to pass therethrough as the optical filter 11.
  • the optical filter 11 that transmits infrared light having a wavelength of 850 nm is provided.
  • the camera 10 By matching the wavelength of the light source 20 with the pass band of the optical filter 11, the camera 10 has sensitivity only to light having the same wavelength as that of infrared light emitted from the light source 20. Since not only visible light but also unnecessary infrared light can be blocked, the captured moving image is not affected by disturbance light. Since the retroreflective material 40 is used, the amount of infrared light emitted from the light source 20 and reflected by the retroreflective material 40 is large. Therefore, the ease of capturing the reflected light is the same as that described above.
  • FIG. 13 shows the light source 20 provided with the polarizing filter 12a and the camera 10 provided with the polarizing filter 12b.
  • the camera 10 is provided with the optical filter 11 described above, but the optical filter 11 is not essential.
  • the polarizing filters 12a and 12b are installed in the camera 10 and the light source 20 so that their polarization directions coincide.
  • the camera 10 is sensitive only to light having the same polarization direction as the infrared light emitted from the light source 20 by the polarizing filters 12a and 12b.
  • the infrared light having a predetermined polarization direction that has passed through the polarizing filter 12 a out of the infrared light emitted from the light source 20 is reflected by the retroreflecting material 40 and further enters the camera 10. Therefore, the influence of disturbance light (visible light and infrared light) having a polarization direction different from the polarization direction can be eliminated.
  • FIG. 14 shows the configuration of the life activity measurement system 101 that uses the position marker 41 to determine the imaging direction. Except for the use of the position marker 41 and the processing of the information processing apparatus 30 regarding the position marker 41, the description is the same as the above description.
  • the position marker 41 is a retroreflecting material provided in a fixed place such as a wall surface of a room where the life activity measurement system 101 is constructed. In this example, it is assumed that the position marker 41 has a pentagonal shape. However, this is an example.
  • the position marker 41 may be circular, elliptical, rectangular, or the like.
  • the shape and size of the position marker 41 are arbitrary as long as the camera 10 can identify the shape of the position marker 41. That is, the shape and size of the position marker 41 can be arbitrarily selected according to the resolution of the camera 10 in consideration of the distance between the position marker 41 and the camera 10.
  • the infrared light emitted from the light source 20 spreads in a conical shape and reaches the position marker 41, for example.
  • a region through which infrared light passes is shown as a space S.
  • Part of the infrared light reflected by the retroreflecting material that is the position marker 41 reaches the camera 10. Thereby, the camera 10 can photograph the position marker 41.
  • the image processing circuit 306 of the information processing apparatus 30 holds, in advance, information on the shape of the position marker 41 and information on the position where the position marker 41 is installed, for example, in the ROM 302.
  • the position where the position marker 41 is installed is, for example, where the installation position of the camera 10 and the shooting direction are appropriately determined and when the position marker 41 is shot, in which position of the image where the position marker 41 is shot. This information identifies whether it exists.
  • the image processing circuit 306 analyzes the captured moving image and performs, for example, pattern matching processing using the stored information to determine whether or not the position marker 41 is included in the frame image of the moving image, and If it is included, the position is specified.
  • the image processing circuit 306 determines whether or not the specified position matches the position where the position marker 41 is to be installed with reference to information held in advance. If they do not match, it means that the camera 10 is not installed in the proper position and / or shooting direction.
  • the image processing circuit 306 notifies a warning indicating that they do not match. For example, the image processing circuit 306 sends a video signal to the display 32 to present a message “Camera position or shooting direction is shifted. Please check”. Alternatively, the CPU 301 may present a warning sound or a warning message via a voice processing circuit (not shown).
  • the life activity measurement system 100 detects the situation and issues a warning. Can do.
  • the present embodiment relates to processing for setting a partial region in a frame image including a place where body movement due to respiration has occurred.
  • the life activity measurement system 100 shown in FIGS. 1 and 6 is referred to.
  • the difference between the present embodiment and the first embodiment is mainly the processing of the information processing apparatus 30 (FIGS. 1 and 6).
  • the same configurations and operations as those of the first embodiment, including the modified example of the retroreflecting material 40, can be applied to the present embodiment.
  • the expected respiration waveform may not be obtained due to the relationship between the position and size of the retroreflecting material 40 and the size of the partial area that is one unit of image processing.
  • the partial region Q as shown in FIGS. 8A and 8B may not be specified.
  • a non-ideal region may be set as a unit region for performing luminance value calculation (image processing).
  • FIGS. 15A and 15B show an example in which a region at a coordinate position where a change in luminance value due to body movement does not appear is set as the partial region Q.
  • FIG. Due to body movement caused by respiration the position of the reflected light region R changes with the states of FIGS. 15A and 15B as the lower limit and the upper limit, respectively.
  • the partial area Q is always in the reflected light area R, the luminance value does not change due to body movement.
  • FIG. 15C shows a change in the luminance value of the set partial area Q. It is understood that the state of high brightness continues regardless of whether there is breathing. Since the fluctuation of the luminance value is small, it is very difficult to extract respiration information using the luminance value.
  • FIGS. 16A and 16B show another example in which a region at a coordinate position where a change in luminance value due to body movement does not appear is set as the partial region Q.
  • FIG. Assume that body movement is recognized in the vertical direction of the drawing.
  • the partial region Q is set so as to always include the upper and lower edges of the reflected light region R, the luminance value of the partial region Q does not change even if the reflected light region R changes. Even when the partial area Q is set in this way, it is very difficult to extract respiration information using the luminance value.
  • the image processing circuit 306 (FIG. 6) dynamically changes the size of the partial region Q for detecting a change in luminance value.
  • the image processing circuit 306 (FIG. 6) dynamically changes the size of the partial region Q for detecting a change in luminance value.
  • an example of setting the partial area Q will be described.
  • an area including a group of pixels for calculating a luminance value is referred to as a “processing unit area”.
  • the edge of the reflected light region R is specified, and when the coordinate position of the edge varies according to time, the processing unit region is set so as to include the region that the edge crosses.
  • the edge of the reflected light region R here is assumed to be one edge when there are a plurality of edges facing each other with respect to the fluctuation direction. For example, an area that simultaneously crosses two edges facing each other with respect to the changing direction is not set as a processing unit area.
  • FIGS. 17A and 17B show an example in which the partial region Q as the processing unit region is secured approximately equal to or larger than the region R of the reflected light from the retroreflecting material 40.
  • FIGS. 17A and 17B show two frame images taken at different times.
  • Each frame image includes a region R of reflected light.
  • the coordinate position of the edge (boundary) of the region R changes with time. Therefore, the partial region Q including the region crossed by the edge of the region R is set as the processing unit region.
  • the ratio at which the reflected light region R and the partial region Q overlap changes with time. This change over time appears as a change in the luminance value of the partial region Q. Therefore, an area including such a change over time may be set as a processing unit area.
  • the luminance value of the processing unit region will not change, and counting the respiration rate using the change in luminance value is not possible. Can not.
  • FIG. 17C shows a change in luminance value of the partial region Q. Since the partial area Q is larger than the reflected light area R, the dynamic range of the luminance change of the reflected light from the retroreflecting material 40 can be increased.
  • the size of the partial region Q as the processing unit region in FIGS. 8A and 8B is 64 pixels ⁇ 64 pixels
  • the size of the partial region Q is 256 pixels. x256 pixels.
  • FIGS. 18A and 18B show an example in which the partial area Q as the processing unit area is secured smaller than the partial area Q of FIG.
  • the partial region Q includes the edge (boundary) of the region R of the reflected light. That is, also in this example, the ratio of the reflected light region R and the partial region Q changes with time.
  • FIG. 18C shows a change in luminance value of the partial region Q. According to FIG.18 (c), it is possible to count a respiration rate using the change of a luminance value.
  • the size of the partial area Q as the processing unit area in FIGS. 8A and 8B is 64 pixels ⁇ 64 pixels
  • the size of the partial area Q is 32 pixels. x32 pixels.
  • FIG. 19 shows the fluctuation range of the reflected light region R that varies with time, and the minimum settable processing unit regions Qa and Qb.
  • Both the processing unit areas Qa and Qb are areas that cross the edge of the area R when the area R of the reflected light fluctuates. In other words, both of the processing unit areas Qa and Qb exist at positions where part or all of them enter or leave the area R.
  • the ratio at which the reflected light region R and the partial region Qa or Qb overlap varies with time.
  • the processing unit area Qa includes a boundary (edge) of the reflected light area R when the change of the reflected light area R reaches the upper limit. That is, the processing unit area Qa is an area that exists at a position where a part of the processing unit area Qa enters or does not enter the area R.
  • the processing unit region Qb is a region that exists at a position where the entire region R enters or does not enter the region R as the reflected light region R changes.
  • One region Qc is a region that is always located in the region R of the reflected light regardless of changes in the region R of the reflected light.
  • the partial area Qc and the reflected light area R always have the same ratio of overlapping areas. Therefore, the partial area Qc is not suitable as a processing unit area.
  • the region Qa and / or Qb in the position satisfying the above-described conditions may be specified, and the processing unit region may be set in consideration of the performance of the information processing apparatus 30 and the like.
  • FIG. 20 shows examples of suitable processing unit areas Qd, Qe, and Qf.
  • Each region is a region that crosses the edge of the region R when the region R of the reflected light fluctuates.
  • FIG. 21 shows an operation procedure of the biological information monitoring apparatus 300 according to the present embodiment. This process is mainly executed by the CPU 301 and / or the image processing circuit 306. In the following description, the execution subject is the image processing circuit 306.
  • the image processing circuit 306 specifies a region R where the retroreflective material 40 exists from the acquired image.
  • the image processing circuit 306 holds in advance in the ROM 302 information on the luminance value of the region where the reflected light from the retroreflecting material 40 is observed. This information is used as a threshold value of the luminance value, and a partial region having a luminance value equal to or higher than the threshold value is specified as a region where the retroreflecting material 40 exists.
  • the image processing circuit 306 specifies an area including the edge of the retroreflecting material 40.
  • the image processing circuit 306 specifies a portion where the difference in luminance value between adjacent pixels is equal to or greater than a predetermined value as an edge. Since the technique for detecting the edge is well known, detailed description thereof is omitted.
  • step S13 the image processing circuit 306 selects a region including an edge as a processing unit region.
  • the edge here refers to one edge when there are a plurality of edges facing each other with respect to the fluctuation direction.
  • step S14 the image processing circuit 306 adds and averages the luminance values of the pixels existing in the selected processing unit area.
  • step S15 the image processing circuit 306 determines a temporal change in the calculated average result as a respiratory waveform, and in step S16, counts the respiratory rate from the respiratory waveform.
  • step S17 the image processing circuit 306 sends a video signal to the display 32 to display information on the respiration rate.
  • the above steps S11 to S13 can be performed at any time even after the edge is detected once and the measurement of the respiration rate is started.
  • the image processing circuit 306 may perform the processing of steps S11 to S13 again to reset the processing unit area resulting from the body movement.
  • body movements such as turning over are very large, and the coordinate position where reflected light is detected changes greatly.
  • the image processing circuit 306 performs the processes of steps S11 to S13 again. Note that the image processing circuit 306 continues to monitor the user's body movement in step S15. Therefore, when body movements other than breathing occur in the subject 1, the image processing circuit 306 can observe changes in body movements quickly and easily.
  • the life activity measurement system 100 shown in FIG. 1 has been described as an example.
  • the configuration of the life activity measurement system 100 is an example.
  • FIG. 22 shows a life activity measurement system 111 according to a modification of the life activity measurement system 100.
  • a plurality of cameras 10 are connected to the information processing apparatus 30 via the network 110.
  • the information processing apparatus 30 obtains moving image data output from the plurality of cameras 10 and individually performs the above-described processing.
  • the life activity measurement system 111 is laid, for example, in a hospital.
  • the camera 10 and the light source 20 may be installed in each patient's home, and the information processing apparatus 30 may be installed in a hospital or the like.
  • This specification discloses a respiration rate measurement method, a measurement system, and a computer program described in the following items.
  • a light source that emits light
  • An imaging device that receives the light and generates a moving image
  • a measurement method for measuring a biological activity resulting from respiration of the subject using a biological activity measurement system comprising an image processing circuit for measuring the biological activity of the subject using the moving image, (A) disposing a retroreflecting material having a predetermined reflection pattern at a position where body movement accompanying breathing of the subject occurs; (B) the light source irradiating the subject with the light; (C) The imaging device receives the light reflected by the retroreflecting material and generates a moving image composed of a plurality of frame images; (D)
  • the image processing circuit includes a step of measuring a biological activity caused by respiration of the subject based on changes in the plurality of frame images.
  • the light source and the retroreflecting material are used in combination, even if the surroundings are dark or bright, and the distance between the subject and the imaging device is sufficiently separated, a living body such as breathing
  • the body movement caused by the activity can be grasped from the change of a plurality of frame images. Since the imaging device can be set apart from the subject and the imaging environment can be set to be dark, it is possible to acquire biological information with sufficiently high accuracy while sufficiently reducing the feeling of pressure applied to the subject.
  • the image processing circuit divides each frame image into a plurality of partial areas, and measures the biological activity based on a change in luminance values of the plurality of partial areas. The measurement method described.
  • the image processing circuit includes: Using the predetermined reflection pattern, identify a partial region of each frame image, including the retroreflective material, and including the position where the body movement occurs, Item 3.
  • the image processing circuit holds in advance information specifying the predetermined reflection pattern, From the item 1, the image processing circuit detects the predetermined reflection pattern in each frame image by performing a pattern matching process using the information, and identifies a partial region including the retroreflecting material. 4. The measuring method according to any one of 4 above.
  • the image processing circuit specifies a first direction that is the direction of the body movement, and a plurality of the frame images are arranged along a second direction different from the first direction.
  • Item 4 The measurement method according to Item 3, wherein the method is divided into partial areas.
  • the predetermined reflection pattern of the retroreflecting material includes any one of items 1 to 7, including a first portion that reflects the light and a second portion that does not reflect the light toward the imaging device. The measurement method described.
  • Item 9 The measurement method according to Item 8, wherein the predetermined reflection pattern of the retroreflective material includes a predetermined pattern.
  • Item 16 The measurement method according to Item 15, wherein in the step (b), the light source is a light emitting diode that emits light of 850 nm or 940 nm as the wavelength ⁇ .
  • the image processing circuit further includes a step of notifying a warning when it is determined that the imaging device has not photographed a predetermined direction or position. 18. The measuring method according to 18.
  • a light source that emits light
  • An imaging device that receives the light and generates a moving image
  • An image processing circuit for measuring a subject's biological activity using the moving image, and a measurement system comprising: When a retroreflecting material is disposed at a position where body movement occurs due to breathing of the subject and the light is emitted from the light source toward the subject, The imaging device receives the light reflected by the retroreflecting material at a plurality of times, and generates the moving image composed of a plurality of time-series frame images, The said image processing circuit is a measuring system which receives the said moving image from the said imaging device, and measures the biological activity resulting from the said subject's respiration based on the change of the luminance value of these frame images.
  • the light source and the retroreflective material are used in combination, even if the surrounding is dark and the distance between the subject and the imaging device is sufficiently separated, it is caused by biological activity such as respiration. Body movement can be captured from changes in a plurality of frame images. Since the imaging device can be set apart from the subject and the imaging environment can be set to be dark, it is possible to acquire biological information with sufficiently high accuracy while sufficiently reducing the feeling of pressure applied to the subject.
  • the image processing circuit specifies a partial area including the retroreflecting material and including the position where the body motion is generated from each frame image, and uses a change in luminance value of the specified partial area.
  • Item 22 The measurement system according to Item 21, wherein the biological activity is measured.
  • Each of the frame images includes a first partial region having a relatively high luminance value corresponding to the light reflected by the retroreflecting material, and a second lower luminance value than the first partial region.
  • the image processing circuit measures the biological activity using a change in luminance value of a coordinate position crossing a boundary between the first partial region and the second partial region over time over the plurality of frame images.
  • the image processing circuit is a partial region including a coordinate position crossed by the boundary, and the partial region including a coordinate position where a ratio of the first partial region and the second partial region changes with time. 24.
  • the image processing circuit detects that the position of the region including the retroreflecting material has moved by a predetermined amount or more, the image processing circuit again includes the retroreflecting material from each frame image.
  • a light source that emits light
  • An imaging device that receives the light and generates a moving image
  • a computer program executed by an image processing circuit in a measurement system comprising: an image processing circuit that measures a subject's biological activity using the moving image; When a retroreflecting material is disposed at a position where body movement occurs due to breathing of the subject and the light is emitted from the light source toward the subject, The computer program is stored in the image processing circuit.
  • Receiving the moving image generated by the imaging device receiving the light reflected by the retroreflecting material at a plurality of times and receiving the moving image composed of a plurality of time-series frame images; Steps, A step of measuring a biological activity caused by respiration of the subject based on a change in luminance values of the plurality of frame images.
  • the present invention can be used as a method for analyzing a moving image obtained by photographing a subject and measuring the life activity of the subject, particularly the number of breaths, in a non-contact manner.
  • the present invention can also be used as an apparatus, system, and computer program for analyzing such moving images and measuring life activity.

Abstract

Provided is a method for measuring biological activity resulting from the respiration of a subject. The method for measuring biological activity includes: a step (a) in which a recursive reflective material having a predetermined reflection pattern is arranged at a position at which body movement that accompanies the respiration of a subject occurs; a step (b) in which a light source irradiates the subject with light; a step (c) in which an imaging device receives light reflected by the recursive reflective material and generates a moving image comprising a plurality of frame images; and a step (d) in which an image processing circuit measures biological activity resulting from the respiration of the subject on the basis of change in the plurality of frame images.

Description

被験体の呼吸に起因する生体活動の計測方法、計測システムおよびコンピュータプログラムMeasuring method, measuring system and computer program for life activity caused by breathing of subject
　本発明は、被験者の映像から被験者の呼吸数等の、呼吸に起因する生体活動を計測するための技術に関する。 The present invention relates to a technique for measuring a biological activity caused by respiration, such as a subject's respiration rate, from a subject's video.
　カメラで被験者を撮影し、その動画像から体動や血流などの生体反応による輝度値の変化を検出し、被験者の呼吸数、心拍数等の生体活動を計測する技術が知られている（たとえば特許文献１および２）。被験者が写る画像領域は、観測者が予め指定したり、輪郭抽出技術を用いることによって特定される。 A technique is known in which a subject is photographed with a camera, a change in luminance value due to a biological reaction such as body movement or blood flow is detected from the moving image, and a biological activity such as a subject's respiratory rate or heart rate is measured ( For example, Patent Documents 1 and 2). The image area in which the subject is photographed is specified by an observer in advance or by using a contour extraction technique.
　特許文献１の呼吸モニタリング装置は、被験者を撮影した画像を局所領域に分割し、それぞれの局所領域の明度情報を解析する。そして、三種類のしきい値を用いて、被験者の胸部周辺の動きを観測しているのか、寝返りなどの非呼吸体動を観測しているかを判定する。 The respiration monitoring device of Patent Literature 1 divides an image obtained by photographing a subject into local regions and analyzes lightness information of each local region. Then, using the three types of threshold values, it is determined whether the subject is observing movement around the chest or observing non-respiratory body movement such as turning over.
　特許文献２の心拍数計測装置は、赤外線光源を搭載したカメラで被験者の顔面を撮影し、フレームごとの顔画像から、眉間の特定領域を抽出してその平均輝度を補正する。心拍数計測装置は、補正された平均輝度の時系列から補正輝度の時間的変化の波形を得て、この波形を心拍数に対応する周波数帯でフィルタリングすることで、被験者の心拍数を算出する。 The heart rate measuring device of Patent Document 2 captures a subject's face with a camera equipped with an infrared light source, extracts a specific region between eyebrows from a face image for each frame, and corrects the average luminance. The heart rate measuring device obtains a waveform of a temporal change in corrected luminance from the corrected average luminance time series, and calculates the heart rate of the subject by filtering the waveform in a frequency band corresponding to the heart rate. .
特開平１１－２７６４４３号公報Japanese Patent Laid-Open No. 11-276443 特開２０１１－１３０９９６号公報JP 2011-130996 A
　特許文献１の呼吸モニタリング装置においては、非呼吸体動の判定に必要な適切なしきい値は、撮影環境に応じて大きく変動する。たとえば観測場所の明るさの変化、室内光源の位置、外部からの入射光の有無、被撮影者以外の人や物の移動により、設定すべきしきい値が大きく変動し得る。常に適切なしきい値を求める方法がないため、しきい値が不適切な場合は、呼吸などの生体情報を求めるための領域を算出することができない。 In the respiratory monitoring device disclosed in Patent Document 1, an appropriate threshold necessary for determining non-respiratory body movement varies greatly depending on the imaging environment. For example, the threshold value to be set can vary greatly depending on changes in the brightness of the observation location, the position of the indoor light source, the presence or absence of incident light from the outside, and the movement of people or objects other than the subject. Since there is no method for always obtaining an appropriate threshold value, an area for obtaining biological information such as respiration cannot be calculated if the threshold value is inappropriate.
　特許文献２の心拍数計測装置は、被験者の顔面を撮像範囲に捉えて撮影する必要がある。特許文献１と同様、照度の変化、人の動き、外部光の入射など撮影環境が変化すると、生体活動以外の原因で、被験者が写る画像領域の輝度値が大きく変化する。このような外乱ノイズが発生すると、生体反応起因の体動個所を特定できず、生体情報が正確に抽出できないことがある。 The heart rate measuring device of Patent Document 2 needs to capture the subject's face within the imaging range. Similar to Patent Document 1, when the shooting environment changes such as a change in illuminance, movement of a person, incidence of external light, etc., the luminance value of the image area in which the subject is photographed changes greatly due to a cause other than biological activity. When such disturbance noise occurs, the body movement location caused by the biological reaction cannot be specified, and the biological information may not be extracted accurately.
　さらに、カメラから被験者の顔面が離れると被験者の生体情報を取得する精度が落ちるため、比較的近距離から被験者の顔面を撮像し続けなければならない。その結果、被験者に圧迫感を与えてしまい、計測対象となる生体活動への影響が懸念される。 Furthermore, if the subject's face moves away from the camera, the accuracy of acquiring the subject's biological information is reduced, so the subject's face must be imaged from a relatively short distance. As a result, a feeling of pressure is given to the subject, and there is a concern about the influence on the biological activity to be measured.
　本発明は、上記課題を解決するためになされたものであって、生体活動の計測条件が周囲の環境の影響を受けにくい、呼吸に起因する生体活動の計測システム、および生体情報の計測方法等を提供する。 The present invention has been made in order to solve the above-described problems, and is a measurement system for biological activity caused by respiration, a measurement method for biological information, etc. I will provide a.
　本発明の実施形態による計測方法は、光を放射する光源と、前記光を受けて動画像を生成する撮像装置と、前記動画像を利用して被験体の生体活動を計測する画像処理回路とを備えた生体活動計測システムを用いて、前記被験体の呼吸に関する生体活動を計測する方法であって、（ａ）前記被験体の呼吸に伴う体動の発生位置に、所定の反射パターンを有する再帰性反射材を配置するステップと、（ｂ）前記光源が、前記光で前記被験体を照射するステップと、（ｃ）前記撮像装置が、前記再帰性反射材で反射された前記光を受けて、複数のフレーム画像から構成される動画像を生成するステップと、（ｄ）前記画像処理回路が、前記複数のフレーム画像の変化に基づいて前記被験体の呼吸に起因する生体活動を計測するステップとを包含する。 A measurement method according to an embodiment of the present invention includes a light source that emits light, an imaging device that receives the light to generate a moving image, and an image processing circuit that measures the biological activity of a subject using the moving image. Using a biological activity measurement system comprising: (a) a predetermined reflection pattern at a generation position of body movement associated with respiration of the subject. Placing a retroreflecting material; (b) the light source irradiating the subject with the light; and (c) the imaging device receiving the light reflected by the retroreflecting material. A step of generating a moving image composed of a plurality of frame images, and (d) the image processing circuit measures a biological activity caused by respiration of the subject based on a change in the plurality of frame images. Including steps That.
　ある実施形態においては、前記ステップ（ｄ）において、前記画像処理回路は、前記複数のフレーム画像の輝度値の変化に基づいて前記生体活動を計測する。 In one embodiment, in the step (d), the image processing circuit measures the life activity based on a change in luminance value of the plurality of frame images.
　ある実施形態においては、前記ステップ（ｄ）において、前記画像処理回路は、各フレーム画像を複数の部分領域に分割し、前記複数の部分領域の輝度値の変化に基づいて前記生体活動を計測する。 In one embodiment, in the step (d), the image processing circuit divides each frame image into a plurality of partial areas and measures the biological activity based on a change in luminance value of the plurality of partial areas. .
　ある実施形態においては、前記ステップ（ｄ）において、前記画像処理回路は、前記所定の反射パターンを利用して、各フレーム画像の部分領域であって、前記再帰性反射材が含まれ、かつ前記体動の発生位置を含む部分領域を特定し、特定した前記部分領域の輝度値の変化を利用して前記生体活動を計測する。 In one embodiment, in the step (d), the image processing circuit is a partial region of each frame image using the predetermined reflection pattern, the retroreflective material is included, and The partial area including the position where the body movement is generated is specified, and the biological activity is measured using a change in luminance value of the specified partial area.
　ある実施形態においては、前記ステップ（ｄ）において、前記画像処理回路は、前記所定の反射パターンを特定する情報を予め保持しており、前記画像処理回路は、前記情報を利用してパターンマッチング処理を行うことにより、各フレーム画像において前記所定の反射パターンを検出し、前記再帰性反射材が含まれる部分領域を特定する。 In one embodiment, in the step (d), the image processing circuit holds information for specifying the predetermined reflection pattern in advance, and the image processing circuit uses the information to perform pattern matching processing. By performing the above, the predetermined reflection pattern is detected in each frame image, and the partial region including the retroreflecting material is specified.
　ある実施形態においては、前記ステップ（ｄ）において、前記画像処理回路は、前記各フレーム画像の、前記再帰性反射材の境界を含む部分領域を特定する。 In one embodiment, in the step (d), the image processing circuit specifies a partial region of each frame image including a boundary of the retroreflecting material.
　ある実施形態においては、前記ステップ（ｄ）において、前記画像処理回路は、前記体動の方向である第１の方向を特定し、前記第１の方向と異なる第２の方向に沿って、前記各フレーム画像を複数の部分領域に分割する。 In one embodiment, in the step (d), the image processing circuit specifies a first direction that is a direction of the body movement, and along a second direction different from the first direction, Each frame image is divided into a plurality of partial areas.
　ある実施形態においては、前記再帰性反射材の前記所定の反射パターンは、前記撮像装置の方向へ、前記光を反射する第１部分と、前記光を反射しない第２部分とを含む。 In one embodiment, the predetermined reflection pattern of the retroreflecting material includes a first portion that reflects the light and a second portion that does not reflect the light in the direction of the imaging device.
　ある実施形態においては、前記第２部分は、前記再帰性反射材の周囲に設けられている。 In one embodiment, the second portion is provided around the retroreflecting material.
　ある実施形態においては、前記第１部分は、前記再帰性反射材の周囲に設けられている。 In one embodiment, the first portion is provided around the retroreflecting material.
　ある実施形態においては、前記再帰性反射材の前記所定の反射パターンは、所定の模様を含む。 In one embodiment, the predetermined reflection pattern of the retroreflecting material includes a predetermined pattern.
　ある実施形態においては、前記再帰性反射材の前記所定の反射パターンは、前記光の反射率が異なる複数の部分を含む。 In one embodiment, the predetermined reflection pattern of the retroreflective material includes a plurality of portions having different light reflectivities.
　ある実施形態においては、前記再帰性反射材の前記所定の反射パターンは、前記光の反射率が連続的に遷移する部分を含む。 In one embodiment, the predetermined reflection pattern of the retroreflecting material includes a portion where the reflectance of the light continuously changes.
　ある実施形態において、前記計測方法は、（ｅ）光学フィルタを用いて、前記撮像装置に入射する可視光を遮断するステップをさらに包含する。 In one embodiment, the measurement method further includes the step of (e) blocking visible light incident on the imaging device using an optical filter.
　ある実施形態においては、前記計測方法は、前記光の波長をλとしたときにおいて、（ｆ）前記波長λの光を主として通過させる光学フィルタに、前記再帰性反射材で反射された前記光を通過させるステップをさらに包含する。 In one embodiment, when the wavelength of the light is λ, (f) the light reflected by the retroreflecting material is passed through an optical filter that mainly transmits light of the wavelength λ. The method further includes a step of passing.
　ある実施形態においては、前記ステップ（ｂ）において、前記光源は、前記波長λとして８５０ｎｍまたは９４０ｎｍの光を放射する発光ダイオードである。 In one embodiment, in the step (b), the light source is a light emitting diode that emits light having a wavelength λ of 850 nm or 940 nm.
　ある実施形態において、前記計測方法は、（ｇ）前記光源から放射された前記光を、所定の偏光方向を有する偏光素子に通過させるステップと、（ｈ）前記再帰性反射材で反射された前記光を、前記所定の偏光方向と同じ偏光方向を有する偏光素子に通過させるステップとをさらに包含する。 In one embodiment, the measurement method includes: (g) passing the light emitted from the light source through a polarizing element having a predetermined polarization direction; and (h) the light reflected by the retroreflecting material. Passing light through a polarizing element having the same polarization direction as the predetermined polarization direction.
　ある実施形態において、前記計測方法は、（ｉ）固定された位置に、予め定められた形状を有する再帰性反射材を位置マーカとしてさらに設けるステップと、（ｊ）前記撮像装置が、前記位置マーカで反射された前記光を受け、複数のフレーム画像から構成される動画像を生成するステップと、（ｋ）前記画像処理回路が、前記位置マーカで反射された前記複数のフレーム画像の少なくとも１つに基づいて前記撮像装置が予め定められた方向を撮影しているか否かを判定するステップとをさらに包含する。 In one embodiment, the measurement method includes (i) a step of further providing a retroreflecting material having a predetermined shape as a position marker at a fixed position; and (j) the imaging device includes the position marker. Receiving the light reflected at step (a) and generating a moving image composed of a plurality of frame images; and (k) at least one of the plurality of frame images reflected by the position marker by the image processing circuit. And determining whether or not the imaging device is photographing a predetermined direction based on the above.
　ある実施形態において、前記計測方法は、（ｌ）前記ステップ（ｋ）において、前記画像処理回路が、前記撮像装置が予め定められた方向または位置を撮影していないと判定した場合には、警告を報知するステップをさらに包含する。 In one embodiment, the measurement method includes: (l) a warning in step (k) when the image processing circuit determines that the imaging device is not shooting a predetermined direction or position. Is further included.
　ある実施形態において、前記再帰性反射材は非金属の材料によって形成されている。 In one embodiment, the retroreflective material is made of a non-metallic material.
　本発明の実施形態による計測システムは、光を放射する光源と、前記光を受けて動画像を生成する撮像装置と、前記動画像を利用して被験体の生体活動を計測する画像処理回路とを備えた計測システムであって、前記被験体の呼吸に伴う体動の発生位置に再帰性反射材が配置され、前記光源から、前記被験体に向けて前記光が放射されたときにおいて、前記撮像装置は、前記再帰性反射材で反射された前記光を複数の時刻において受けて、時系列の複数のフレーム画像から構成される前記動画像を生成し、前記画像処理回路は、前記撮像装置から前記動画像を受け取り、前記複数のフレーム画像の輝度値の変化に基づいて前記被験体の呼吸に起因する生体活動を計測する。 A measurement system according to an embodiment of the present invention includes a light source that emits light, an imaging device that receives the light and generates a moving image, and an image processing circuit that measures a biological activity of a subject using the moving image. A retroreflecting material is disposed at a position where body movement occurs due to breathing of the subject, and when the light is emitted from the light source toward the subject, the measurement system includes: The imaging device receives the light reflected by the retroreflecting material at a plurality of times and generates the moving image composed of a plurality of time-series frame images, and the image processing circuit includes the imaging device The moving image is received from the frame, and the biological activity caused by the breathing of the subject is measured based on the change in the luminance value of the plurality of frame images.
　ある実施形態において、前記画像処理回路は、各フレーム画像の中から、前記再帰性反射材を含み、かつ前記体動の発生位置を含む部分領域を特定し、特定した前記部分領域の輝度値の変化を利用して前記生体活動を計測する。 In one embodiment, the image processing circuit specifies a partial region that includes the retroreflecting material and includes the position where the body motion is generated from each frame image, and the luminance value of the specified partial region is determined. The life activity is measured using the change.
　ある実施形態において、前記各フレーム画像は、前記再帰性反射材で反射された前記光に対応する、輝度値が相対的に高い第１部分領域と、前記第１部分領域よりも輝度値が相対的に低い第２部分領域とを含み、前記画像処理回路は、前記複数のフレーム画像にわたって、前記第１部分領域と前記第２部分領域との境界が横切る座標位置の輝度値が経時的に変化することを利用して前記生体活動を計測する。 In one embodiment, each of the frame images has a first partial area having a relatively high luminance value corresponding to the light reflected by the retroreflecting material, and a relative luminance value higher than that of the first partial area. The second partial area is low, and the image processing circuit changes over time the luminance value of the coordinate position across the boundary between the first partial area and the second partial area across the plurality of frame images. The life activity is measured by utilizing this.
　ある実施形態において、前記画像処理回路は、前記境界が横切る座標位置を含む部分領域であって、かつ前記第１部分領域と前記第２部分領域との割合が経時的に変化する座標位置を含む部分領域に関して、前記輝度値の変化を検出する。 In one embodiment, the image processing circuit includes a partial region including a coordinate position crossed by the boundary, and a coordinate position where a ratio of the first partial region and the second partial region changes with time. With respect to the partial area, a change in the luminance value is detected.
　ある実施形態において、前記画像処理回路は、前記各フレーム画像の同じ座標位置に前記処理単位領域を設定する。 In one embodiment, the image processing circuit sets the processing unit area at the same coordinate position of each frame image.
　ある実施形態において、前記部分領域のサイズは可変である。 In one embodiment, the size of the partial area is variable.
　ある実施形態において、前記画像処理回路は、前記再帰性反射材を含む領域の位置が、予め定められた量以上移動したことを検出した場合には、再度、各フレーム画像の中から、前記再帰性反射材を含み、かつ前記体動の発生位置を含む新たな部分領域を特定し、特定した前記新たな部分領域の輝度値の変化を利用して前記生体活動を計測する。 In one embodiment, when the image processing circuit detects that the position of the region including the retroreflecting material has moved by a predetermined amount or more, the image processing circuit again performs the recursion from each frame image. A new partial area that includes a reflective reflector and that includes the position of occurrence of the body movement is specified, and the biological activity is measured using a change in luminance value of the specified new partial area.
　本発明の実施形態によるコンピュータプログラムは、光を放射する光源と、前記光を受けて動画像を生成する撮像装置と、前記動画像を利用して被験体の生体活動を計測する画像処理回路とを備えた計測システムにおける画像処理回路によって実行されるコンピュータプログラムであって、前記被験体の呼吸に伴う体動の発生位置に再帰性反射材が配置され、前記光源から、前記被験体に向けて前記光が放射されたときにおいて、前記コンピュータプログラムは前記画像処理回路に、前記撮像装置によって生成された動画像を受け取るステップであって、前記再帰性反射材で反射された前記光を複数の時刻において受けて時系列の複数のフレーム画像から構成される前記動画像を受け取るステップと、前記複数のフレーム画像の輝度値の変化に基づいて前記被験体の呼吸に起因する生体活動を計測するステップとを実行させる。 A computer program according to an embodiment of the present invention includes a light source that emits light, an imaging device that receives the light and generates a moving image, and an image processing circuit that measures a subject's biological activity using the moving image. A computer program executed by an image processing circuit in a measurement system comprising: a retroreflecting material disposed at a position where a body motion accompanying breathing of the subject occurs, and from the light source toward the subject When the light is radiated, the computer program receives, in the image processing circuit, a moving image generated by the imaging device, and the light reflected by the retroreflecting material is received at a plurality of times. Receiving the moving image composed of a plurality of time-series frame images received in step, and changing luminance values of the plurality of frame images. And a step of measuring the biological activity resulting from respiration of the subject based on.
　本発明によれば、光源と再帰性反射材とを組み合わせて使用するため、周囲が暗い場合でも明るい場合でも、かつ被験者と撮像装置との距離を十分離したとしても、呼吸等の生体活動に起因する体動を、複数のフレーム画像の変化から捉えることができる。撮像装置を被験体から離して設置し、かつ撮影環境を暗く設定できるため、被験体へ与える圧迫感を十分低減しつつ、十分高い精度で生体情報を取得することが可能である。 According to the present invention, since the light source and the retroreflective material are used in combination, even if the surroundings are dark or bright, and even if the distance between the subject and the imaging device is sufficiently separated, it can be used for biological activities such as breathing. The resulting body movement can be captured from changes in a plurality of frame images. Since the imaging device can be set apart from the subject and the imaging environment can be set to be dark, it is possible to acquire biological information with sufficiently high accuracy while sufficiently reducing the feeling of pressure applied to the subject.
実施の形態１による生体活動計測システム１００の構成を示す図である。It is a figure which shows the structure of the biological activity measurement system 100 by Embodiment 1. FIG. 再帰性反射材４０を装着した被験者１を撮影したフレーム画像１０２を示す図である。It is a figure which shows the frame image 102 which image | photographed the test subject 1 with which the retroreflection material 40 was mounted | worn. 再帰性反射材４０を装着しない被験者を撮影したフレーム画像１０６を示す図である。It is a figure which shows the frame image 106 which image | photographed the test subject which does not wear the retroreflection material 40. FIG. 複数のフレーム画像の輝度値の変化に基づいて測定された、高輝度領域１０４の振動を示す図である。It is a figure which shows the vibration of the high-intensity area | region 104 measured based on the change of the luminance value of a some frame image. 再帰性反射材４０を設けずに暗い撮影環境下で撮影された複数のフレーム画像の輝度値の変化を示す図である。It is a figure which shows the change of the luminance value of the some frame image image | photographed in the dark imaging environment, without providing the retroreflection material 40. FIG. 生体活動計測システム１００の、主として情報処理装置３０のハードウェア構成の例を示す図である。2 is a diagram illustrating an example of a hardware configuration of an information processing apparatus 30 of the life activity measurement system 100 mainly. FIG. 生体活動計測システム１００で行われる処理の手順を示すフローチャートである。4 is a flowchart illustrating a procedure of processing performed in the biological activity measurement system 100. （ａ）および（ｂ）は、時刻の異なる時刻に撮影された２枚のフレーム画像における部分領域Ｑの例を示す図であり、（ｃ）は部分領域Ｑの輝度値の変化を示す図である。(A) And (b) is a figure which shows the example of the partial area Q in two frame images image | photographed at the time from which time differs, (c) is a figure which shows the change of the luminance value of the partial area Q. is there. （ａ）～（ｆ）は、それぞれ、再帰性反射材４０の形状の例を示す図である。(A)-(f) is a figure which shows the example of the shape of the retroreflection material 40, respectively. （ａ）は図９（ａ）の再帰性反射材４０を被験者１へ装着した例を示す図であり、（ｂ）は撮像例を示す図である。(A) is a figure which shows the example which mounted | wore the test subject 1 with the retroreflection material 40 of Fig.9 (a), (b) is a figure which shows the example of an imaging. （ａ１）～（ｃ１）はそれぞれ、三角形の反射部分の周囲にマーカとして機能する反射材を設けた再帰性反射材４０を示す図であり、（ａ２）～（ｃ２）はそれぞれ、マーカ内に設けられた２つ部分領域を示す図である。(A1) to (c1) are diagrams showing a retroreflecting material 40 provided with a reflecting material functioning as a marker around a triangular reflecting portion, and (a2) to (c2) are respectively shown in the marker. It is a figure which shows the two partial area | regions provided. 可視光領域の波長を遮る光学フィルタ１１を装着したカメラ１０を示す図である。It is a figure which shows the camera 10 equipped with the optical filter 11 which interrupts | blocks the wavelength of visible light area | region. 偏光フィルタ１２ａが設けられた光源２０と、偏光フィルタ１２ｂが設けられたカメラ１０とを示す図である。It is a figure which shows the light source 20 provided with the polarizing filter 12a, and the camera 10 provided with the polarizing filter 12b. 位置マーカ４１を利用して撮影方向を判定する生体活動計測システム１０１の構成を示す図である。It is a figure which shows the structure of the biological activity measurement system 101 which determines an imaging | photography direction using the position marker 41. FIG. （ａ）および（ｂ）は、体動による輝度値の変化が現れない座標位置の領域が部分領域Ｑとして設定された例を示す図であり、（ｃ）は部分領域Ｑの輝度値の変化を示す図である。(A) And (b) is a figure which shows the example in which the area | region of the coordinate position where the change of the luminance value by a body motion does not appear is set as the partial area Q, (c) is a change of the luminance value of the partial area Q FIG. （ａ）および（ｂ）は、体動による輝度値の変化が現れない座標位置の領域が部分領域Ｑとして設定された他の例を示す図である。(A) And (b) is a figure which shows the other example in which the area | region of the coordinate position where the change of the luminance value by a body motion does not appear was set as the partial area | region Q. （ａ）および（ｂ）は、処理単位領域としての部分領域Ｑを、再帰性反射材４０からの反射光の領域Ｒと概ね同じ、または領域Ｒよりも大きく確保した例を示す図であり、（ｃ）は部分領域Ｑの輝度値の変化を示す図である。(A) And (b) is a figure which shows the example which ensured the partial area | region Q as a process unit area | region substantially the same as the area | region R of the reflected light from the retroreflection material 40, or larger than the area | region R, (C) is a figure which shows the change of the luminance value of the partial area Q. FIG. （ａ）および（ｂ）は、処理単位領域としての部分領域Ｑを、図８の部分領域Ｑよりも小さく確保した例を示す図であり、（ｃ）は部分領域Ｑの輝度値の変化を示す図である。(A) And (b) is a figure which shows the example which ensured the partial area Q as a process unit area smaller than the partial area Q of FIG. 8, (c) is a figure which shows the change of the luminance value of the partial area Q. FIG. 時間的に変動する反射光の領域Ｒの変動範囲と、設定可能な最小の処理単位領域ＱａおよびＱｂを示す図である。It is a figure which shows the fluctuation range of the area | region R of the reflected light which fluctuates temporally, and the minimum process unit area | region Qa and Qb which can be set. 好適な処理単位領域Ｑｄ、ＱｅおよびＱｆの例を示す図である。It is a figure which shows the example of suitable process unit area | region Qd, Qe, and Qf. 実施の形態２による生体情報モニタリング装置３００の動作の手順を示すフローチャートである。10 is a flowchart illustrating an operation procedure of the biological information monitoring apparatus 300 according to the second embodiment. 生体活動計測システム１００の変形例による生体活動計測システム１１１を示す図である。It is a figure which shows the life activity measurement system 111 by the modification of the life activity measurement system.
　以下、添付の図面を参照しながら、本発明による生体活動計測システムおよび計測方法の実施形態を説明する。 Hereinafter, embodiments of a life activity measurement system and a measurement method according to the present invention will be described with reference to the accompanying drawings.
　（実施の形態１）
　図１は、本実施の形態による生体活動計測システム１００の構成を示す。生体活動計測システム１００は、カメラ１０と、光源２０と、情報処理装置３０と、再帰性反射材４０とを含む。図１には被験者１が示されているが、被験者１は生体活動計測システム１００に含まれない。 (Embodiment 1)
FIG. 1 shows a configuration of a life activity measurement system 100 according to the present embodiment. The life activity measurement system 100 includes a camera 10, a light source 20, an information processing device 30, and a retroreflecting material 40. Although the subject 1 is shown in FIG. 1, the subject 1 is not included in the life activity measurement system 100.
　生体活動計測システム１００は、被験者１の生体活動を観察するために利用される。本実施の形態では、生体活動は被験者１の呼吸であるとし、生体活動計測システム１００は所定時間内の呼吸数を計測する。なお、被験者１は人であるとして説明するが、人以外の動物であってもよい。観測対象としての動物（人を含む。）を総称して「被験体」と呼ぶことがある。 The life activity measurement system 100 is used for observing the life activity of the subject 1. In the present embodiment, it is assumed that the life activity is the respiration of the subject 1, and the life activity measurement system 100 measures the respiration rate within a predetermined time. Note that although the subject 1 is described as being a person, it may be an animal other than a person. Animals (including people) as observation targets may be collectively referred to as “subjects”.
　カメラ１０は、いわゆる撮像装置であり、被験者１を撮影して動画像を生成する。カメラ１０は、有線または無線で動画像のデータを情報処理装置３０に送る。 The camera 10 is a so-called imaging device, and shoots the subject 1 to generate a moving image. The camera 10 sends moving image data to the information processing apparatus 30 by wire or wirelessly.
　光源２０は光２０ａを放射する光源である。光は可視光であってもよく、不可視光（たとえば赤外光）であってもよい。本実施の形態では、赤外光を例に挙げて説明する。以下では、光２０ａを「赤外光２０ａ」と記述する。 The light source 20 is a light source that emits light 20a. The light may be visible light or invisible light (for example, infrared light). In this embodiment, infrared light will be described as an example. Hereinafter, the light 20a is described as “infrared light 20a”.
　情報処理装置３０は、カメラ１０が撮影した動画像のデータを受け取り、動画像を構成する複数のフレーム画像間の画像の変化を利用して被験者１の呼吸数を計測する。情報処理装置３０の動作の詳細は後述する。 The information processing apparatus 30 receives moving image data captured by the camera 10 and measures the respiration rate of the subject 1 using changes in images between a plurality of frame images constituting the moving image. Details of the operation of the information processing apparatus 30 will be described later.
　再帰性反射材４０は、入射してきた光を、その入射方向に向けて反射する光学特性を有する反射材である。つまり、再帰性反射材４０に入射する光の入射角と、再帰性反射材４０によって反射された光の出射角とは等しい。ただしこの性質は理想的であり、実際には一部の入射方向とは異なる方向に反射され得る。本実施の形態では、光源２０の光軸とカメラ１０の光軸とを近接して配置させている。これにより、光源２０から放射された赤外光２０ａは再帰性反射材４０に反射され、その多くが赤外光２０ｂとしてカメラ１０に入射する。よって、カメラ１０は十分な光量で被験者１を撮影することができる。本実施形態では、再帰性反射材４０として、ガラスビーズを塗布した布を用いた。 The retroreflective material 40 is a reflective material having an optical characteristic of reflecting incident light toward the incident direction. That is, the incident angle of light incident on the retroreflecting material 40 is equal to the emission angle of light reflected by the retroreflecting material 40. However, this property is ideal and can actually be reflected in a direction different from some incident directions. In the present embodiment, the optical axis of the light source 20 and the optical axis of the camera 10 are arranged close to each other. Thereby, the infrared light 20a emitted from the light source 20 is reflected by the retroreflecting material 40, and most of the light enters the camera 10 as infrared light 20b. Therefore, the camera 10 can photograph the subject 1 with a sufficient amount of light. In the present embodiment, a cloth coated with glass beads is used as the retroreflecting material 40.
　なお、再帰性反射材４０を設けることにより、再帰性反射材４０に入射した外乱光２１ａは、反射光２１ｂとしてその入射方向に反射される。反射光２１ｂは実質的にカメラ１０に入射しないため、カメラ１０によって撮影される動画像は外乱光の影響を受けにくくなる。 In addition, by providing the retroreflecting material 40, the disturbance light 21a incident on the retroreflecting material 40 is reflected in the incident direction as reflected light 21b. Since the reflected light 21b does not substantially enter the camera 10, the moving image photographed by the camera 10 is hardly affected by disturbance light.
　生体活動計測システム１００の全体の動作を概説すると以下のとおりである。 The overall operation of the life activity measurement system 100 is outlined as follows.
　まず、観測者または被験者１が、被験者１の呼吸に伴う体動の発生位置に、所定の反射パターンを有する再帰性反射材４０を配置する。光源２０が赤外光で被験者１を照射すると、カメラ１０は再帰性反射材４０で反射された赤外光を受けて、被験者１の動画像を撮影する。 First, the observer or the subject 1 arranges the retroreflecting material 40 having a predetermined reflection pattern at the position where the body movement accompanying the breathing of the subject 1 occurs. When the light source 20 irradiates the subject 1 with infrared light, the camera 10 receives the infrared light reflected by the retroreflecting material 40 and captures a moving image of the subject 1.
　たとえば図２は、再帰性反射材４０を装着した被験者１を撮影したフレーム画像１０２を示す。画像中央部の高輝度領域（白い領域）１０４が、再帰性反射材４０からの反射光が検出された領域である。参考として、図３は、再帰性反射材４０を装着しない被験者を撮影したフレーム画像１０６を示す。再帰性反射材４０が存在しない場合には撮影されたフレーム画像内の輝度変化は非常に小さいと言える。図２および図３には、複数の縦線および横線が示されているが、これは画像処理のために仮想的に設けられた境界線である。本明細書では、境界線によって区画される画像の領域を、画像の「部分領域」と呼ぶ。図２には、部分領域Ｐが例示されている。なお、部分領域Ｐの境界線は理解の便宜のため強調して表示されている。 For example, FIG. 2 shows a frame image 102 obtained by photographing the subject 1 wearing the retroreflecting material 40. A high brightness area (white area) 104 in the center of the image is an area where the reflected light from the retroreflecting material 40 is detected. For reference, FIG. 3 shows a frame image 106 obtained by photographing a subject who does not wear the retroreflecting material 40. When the retroreflecting material 40 is not present, it can be said that the luminance change in the captured frame image is very small. FIG. 2 and FIG. 3 show a plurality of vertical lines and horizontal lines, which are boundaries virtually provided for image processing. In this specification, an area of an image divided by a boundary line is referred to as a “partial area” of the image. FIG. 2 illustrates the partial region P. Note that the boundary lines of the partial areas P are highlighted for convenience of understanding.
　なお、後述するように、本明細書では、再帰性反射材４０に所定の反射パターンを設け、その反射パターンを利用してより確実かつより簡易に高輝度領域１０４を検出するが、図２は特にその反射パターンの形状は現れていない。 As will be described later, in this specification, a predetermined reflection pattern is provided on the retroreflecting material 40, and the high-intensity region 104 is detected more reliably and more easily using the reflection pattern. In particular, the shape of the reflection pattern does not appear.
　情報処理装置３０は、図２に示されるような、動画像を構成する時系列の各複数のフレーム画像を解析して、複数のフレーム画像の輝度値の変化に基づいて被験者１の体動を検出する。より具体的に説明すると、情報処理装置３０は、図２に示す高輝度領域１０４を複数のフレーム画像にわたって検出する。平静時の被験者１の体動は呼吸に起因して発生するため、高輝度領域１０４の位置が呼吸の周期に合わせて変化（振動）する。情報処理装置３０は、高輝度領域１０４の振動の１周期を１呼吸周期として、所定期間にわたって呼吸周期の数をカウントすることにより、その期間における被験者１の呼吸数を計測することができる。 The information processing apparatus 30 analyzes each of a plurality of time-series frame images constituting the moving image as shown in FIG. 2 and determines the body movement of the subject 1 based on the change in the luminance value of the plurality of frame images. To detect. More specifically, the information processing apparatus 30 detects the high luminance region 104 shown in FIG. 2 over a plurality of frame images. Since the body movement of the subject 1 during calm occurs due to breathing, the position of the high-intensity region 104 changes (vibrates) in accordance with the breathing cycle. The information processing apparatus 30 can measure the respiratory rate of the subject 1 during the period by counting the number of respiratory cycles over a predetermined period, where one period of vibration of the high-intensity region 104 is one respiratory period.
　本明細書においては、主として呼吸数を計測する例を説明する。しかしながら、呼吸数は被験者の呼吸に起因する生体活動の一例であり、被験者の呼吸に起因する他の生体活動を計測してもよい。本明細書では、被験者の呼吸動作を計測し、呼吸による体動から呼吸に起因する波形（呼吸波形に相当する波形）を導出する。典型的には、その波形を利用して評価可能な他の生体活動、たとえば、呼吸の深さ、乱れ、無呼吸期間、無呼吸期間が発生する頻度などの生体活動は、本明細書において、計測対象である生体活動の範疇である。 In this specification, an example in which the respiration rate is mainly measured will be described. However, the respiratory rate is an example of a biological activity resulting from the subject's breathing, and other biological activities resulting from the subject's breathing may be measured. In this specification, a subject's breathing motion is measured, and a waveform resulting from breathing (a waveform corresponding to the breathing waveform) is derived from body motion due to breathing. Typically, other biological activities that can be evaluated using the waveform, for example, biological activities such as breathing depth, turbulence, apnea periods, frequency of occurrence of apnea periods, This is the category of the biological activity to be measured.
　図４は、複数のフレーム画像の輝度値の変化に基づいて測定された、高輝度領域１０４の振動を示す。再帰性反射材４０を用いて観測される波形は暗い撮影環境下でも呼吸による体動を精度よく測定することが可能である。つまり、輝度値を利用して体動、すなわち呼吸を測定することが可能である。参考として、図５は、再帰性反射材４０を設けずに暗い撮影環境下で撮影された複数のフレーム画像の輝度値の変化を示す。再帰性反射材４０が存在しないことにより、画像内の輝度変化はもともと小さく、そのため複数のフレーム画像にわたって輝度値の変化を観測してもノイズの影響が非常に大きい。よって計測する必要がある体動の波形がノイズに埋もれている。なお、図４と図５では、縦軸のスケールは数倍程度異なっていることに留意されたい。理解の便宜のため、図５でのスケールは図４よりも大きくしている。換言すれば、再帰性反射材４０を用いる方（図４）が、用いない方（図５）よりも、信号対雑音比（ＳＮＲ）に優れていることを意味する。 FIG. 4 shows the vibration of the high luminance region 104 measured based on the change of the luminance value of the plurality of frame images. Waveforms observed using the retroreflecting material 40 can accurately measure body movement due to respiration even in a dark imaging environment. That is, it is possible to measure body movement, that is, respiration, using the luminance value. For reference, FIG. 5 shows changes in luminance values of a plurality of frame images taken in a dark shooting environment without providing the retroreflecting material 40. Due to the absence of the retroreflecting material 40, the luminance change in the image is originally small, and therefore the influence of noise is very large even if the change in the luminance value is observed over a plurality of frame images. Therefore, the body motion waveform that needs to be measured is buried in noise. Note that in FIG. 4 and FIG. 5, the scale of the vertical axis differs by several times. For convenience of understanding, the scale in FIG. 5 is larger than that in FIG. In other words, the direction using the retroreflecting material 40 (FIG. 4) means that the signal-to-noise ratio (SNR) is superior to the direction not using it (FIG. 5).
　本実施の形態の計測方法によれば、再帰性反射材４０を利用することにより、赤外光の反射光量を十分確保して撮影を行うことができる。その結果、被験者１とカメラ１０とを、たとえば６ｍ程度離して設置することができる。また、計測する部屋を暗くすることができる。これにより、被験者１への圧迫感を軽減しつつ、観測場所の明るさの変化、室内光源の位置、外部からの入射光の有無の影響を受けにくい環境下、つまりノイズの影響が小さい環境下で撮影を行うことが可能になる。よって生体活動をより正確に計測することが可能になる。 According to the measurement method of the present embodiment, by using the retroreflecting material 40, it is possible to perform imaging while ensuring a sufficient amount of reflected infrared light. As a result, the subject 1 and the camera 10 can be set apart by, for example, about 6 m. Moreover, the room to be measured can be darkened. As a result, while reducing the feeling of pressure on the subject 1, the environment is less susceptible to changes in the brightness of the observation location, the position of the indoor light source, and the presence or absence of incident light from outside, that is, an environment in which the influence of noise is small. It becomes possible to shoot with. Therefore, it becomes possible to measure the biological activity more accurately.
　図６は、生体活動計測システム１００の、主として情報処理装置３０のハードウェア構成の例を示す。本実施の形態では、情報処理装置３０はカメラ１０、およびディスプレイ３２と接続されている。情報処理装置３０は、カメラ１０から、撮影された動画像のデータを受け取る。またディスプレイ３２は、処理の結果である、被験者１の生体活動である呼吸の数の計測結果を表示する。高輝度領域が検出されないことにより、カメラ１０の撮影方向が適切でないと判断した場合には、情報処理装置３０はディスプレイ３２に警告を表示してもよい。 FIG. 6 shows an example of the hardware configuration of the information processing apparatus 30 mainly in the life activity measurement system 100. In the present embodiment, the information processing apparatus 30 is connected to the camera 10 and the display 32. The information processing apparatus 30 receives captured moving image data from the camera 10. In addition, the display 32 displays a measurement result of the number of breaths that is the biological activity of the subject 1 as a result of the processing. When it is determined that the shooting direction of the camera 10 is not appropriate due to the absence of the high brightness area, the information processing apparatus 30 may display a warning on the display 32.
　情報処理装置３０は、ＣＰＵ３０１と、ＲＯＭ３０２と、ＲＡＭ３０３と、ハードディスクドライブ（ＨＤＤ）３０４と、インタフェース（Ｉ／Ｆ）３０５と、画像処理回路３０６とを有する。ＣＰＵ３０１は情報処理装置３０の動作を制御する。ＲＯＭ３０２は、コンピュータプログラムを格納している。コンピュータプログラムは、たとえば後述するフローチャートによって示される処理をＣＰＵ３０１または画像処理回路３０６に行わせるための命令群である。ＲＡＭ３０３は、ＣＰＵ３０１による実行にあたって、コンピュータプログラムを展開するためのワークメモリである。ＨＤＤ３０４は、カメラ１０から受信した動画像のデータ、または計測された被験者１の呼吸数のデータを格納する記憶装置である。 The information processing apparatus 30 includes a CPU 301, a ROM 302, a RAM 303, a hard disk drive (HDD) 304, an interface (I / F) 305, and an image processing circuit 306. The CPU 301 controls the operation of the information processing apparatus 30. The ROM 302 stores a computer program. The computer program is a group of instructions for causing the CPU 301 or the image processing circuit 306 to perform processing shown by a flowchart described later, for example. A RAM 303 is a work memory for developing a computer program when executed by the CPU 301. The HDD 304 is a storage device that stores moving image data received from the camera 10 or measured respiratory rate data of the subject 1.
　Ｉ／Ｆ３０５は、情報処理装置３０がカメラ１０から動画像のデータを受け取るためのインタフェースである。情報処理装置３０が有線のネットワーク経由で動画像のデータを受け取る場合には、Ｉ／Ｆ３０５はたとえばイーサネット（登録商標）端子である。情報処理装置３０が無線のネットワーク経由で動画像のデータを受け取る場合には、Ｉ／Ｆ３０５はたとえばＷｉ－Ｆｉ（登録商標）規格に準拠した通信を行う送受信回路である。またはＩ／Ｆ３０５は、有線の映像入力端子であってもよい。 The I / F 305 is an interface for the information processing apparatus 30 to receive moving image data from the camera 10. When the information processing apparatus 30 receives moving image data via a wired network, the I / F 305 is, for example, an Ethernet (registered trademark) terminal. When the information processing apparatus 30 receives moving image data via a wireless network, the I / F 305 is a transmission / reception circuit that performs communication based on, for example, the Wi-Fi (registered trademark) standard. Alternatively, the I / F 305 may be a wired video input terminal.
　画像処理回路３０６は、動画像のデータを解析する、いわゆるグラフィックスプロセッサである。画像処理回路３０６は、動画像の各フレーム画像の高輝度領域を検出し、高輝度領域の振動に基づいて体動を検出し、体動の振動波形に基づいて呼吸数をカウントする。 The image processing circuit 306 is a so-called graphics processor that analyzes moving image data. The image processing circuit 306 detects a high luminance area of each frame image of the moving image, detects a body movement based on the vibration of the high luminance area, and counts the respiration rate based on the vibration waveform of the body movement.
　本実施の形態ではＣＰＵ３０１とは別に画像処理回路３０６を設けているが、これは一例である。後述する画像処理回路３０６の処理を、ＣＰＵ３０１が行ってもよい。 In this embodiment, the image processing circuit 306 is provided separately from the CPU 301, but this is an example. The CPU 301 may perform processing of an image processing circuit 306 described later.
　図７は、生体活動計測システム１００で行われる処理の手順を示す。 FIG. 7 shows a procedure of processing performed in the life activity measurement system 100.
　ステップＳ１において、カメラ１０が再帰性反射材４０を装着した被験者１を撮影する。撮影された動画像は情報処理装置３０に送られる。 In step S1, the camera 10 photographs the subject 1 wearing the retroreflecting material 40. The captured moving image is sent to the information processing apparatus 30.
　ステップＳ２において、情報処理装置３０の画像処理回路３０６は、撮影した動画像を構成する複数のフレーム画像の各々を、複数の部分領域に分割する。部分領域（たとえば図２の部分領域Ｐ）は、たとえば横６４画素、縦６４画素の大きさを有する。なお「分割する」とは、実際の動作として分割する必要はない。たとえば画像を切り出す単位または処理を行う単位として部分領域のサイズを設定する、という動作も、ここで言う「分割する」動作に含まれ得る。 In step S2, the image processing circuit 306 of the information processing apparatus 30 divides each of a plurality of frame images constituting the captured moving image into a plurality of partial areas. The partial region (for example, the partial region P in FIG. 2) has, for example, a size of 64 pixels in the horizontal direction and 64 pixels in the vertical direction. Note that “divide” does not require division as an actual operation. For example, the operation of setting the size of the partial area as a unit for cutting out an image or a unit for performing processing may be included in the “dividing” operation described here.
　ステップＳ３において、画像処理回路３０６は、各部分領域の輝度値に基づいて、再帰性反射材４０が存在する部分領域、および生体反応による体動箇所を含む部分領域を特定する。より具体的に説明する。再帰性反射材４０が存在する部分領域は、各フレーム画像内で特定され得る。一方、体動箇所を含む部分領域は、複数のフレーム画像にわたって、すなわち複数のフレーム画像間で特定され得る。 In step S3, the image processing circuit 306 specifies a partial region where the retroreflective material 40 exists and a partial region including a body movement location due to a biological reaction based on the luminance value of each partial region. This will be described more specifically. The partial region where the retroreflecting material 40 exists can be specified in each frame image. On the other hand, the partial region including the body movement location can be specified over a plurality of frame images, that is, between the plurality of frame images.
　再帰性反射材４０が存在する部分領域は以下の処理によって特定される。たとえば、画像処理回路３０６は、再帰性反射材４０が存在する場合に観測される部分領域の輝度値の情報を、予めＲＯＭ３０２に保持している。この情報を輝度値の閾値として利用し、閾値以上の輝度値を有する部分領域を、再帰性反射材４０が存在する部分領域として特定する。 The partial area where the retroreflective material 40 exists is specified by the following processing. For example, the image processing circuit 306 holds information on luminance values of partial areas observed when the retroreflecting material 40 exists in the ROM 302 in advance. Using this information as a threshold value of the luminance value, a partial region having a luminance value equal to or higher than the threshold value is specified as a partial region where the retroreflecting material 40 exists.
　このときの輝度値は、部分領域に含まれる各画素の輝度値の和であってもよいし、平均値であってもよい。輝度値の和および平均値のいずれを採用するかに応じて、閾値もまた変化し得る。平均値を算出する演算処理よりも和を算出する演算処理の方が、計算負荷が小さく高速化が可能であるため、本実施の形態では部分領域に含まれる各画素の輝度値の和であるとする。 The luminance value at this time may be the sum of the luminance values of the pixels included in the partial area, or may be an average value. Depending on whether the sum of luminance values or the average value is adopted, the threshold may also change. Since the arithmetic processing for calculating the sum is smaller than the arithmetic processing for calculating the average value and the calculation load is small and the processing speed can be increased, the present embodiment is the sum of the luminance values of the pixels included in the partial area. And
　一方、体動箇所を含む部分領域は以下の処理によって特定される。上述のように、再帰性反射材４０からの反射光が観測される領域は、生体反応（呼吸）による体動により変動する。いま、各フレーム画像に関して、ある共通の座標位置に存在する部分領域Ｑに着目する。図８（ａ）および（ｂ）は、時刻の異なる時刻に撮影された２枚のフレーム画像における部分領域Ｑの例を示す。図８（ａ）および（ｂ）に示す領域Ｒは、再帰性反射材４０からの反射光が検出されている高輝度領域であるとする。呼吸に伴う体動により、部分領域Ｑが、高輝度領域になったりならなかったりする。 On the other hand, the partial area including the body movement location is specified by the following process. As described above, the region where the reflected light from the retroreflecting material 40 is observed varies due to body movement caused by a biological reaction (respiration). Now, with respect to each frame image, attention is paid to a partial region Q existing at a certain common coordinate position. FIGS. 8A and 8B show examples of partial areas Q in two frame images taken at different times. A region R shown in FIGS. 8A and 8B is assumed to be a high luminance region in which reflected light from the retroreflecting material 40 is detected. The partial area Q may or may not become a high luminance area due to body movement accompanying breathing.
　図８（ｃ）は、このときの部分領域Ｑの輝度値の変化を示す。複数のフレーム画像にわたって時系列的に部分領域Ｑの輝度を観測すると、ある閾値Ｔを超えるフレーム画像群と、超えないフレーム画像群とが交互に存在する。 FIG. 8C shows a change in the luminance value of the partial region Q at this time. When the luminance of the partial area Q is observed in time series over a plurality of frame images, a group of frame images exceeding a certain threshold T and a group of frame images not exceeding exist alternately.
　図７のステップＳ３において、体動箇所を含む部分領域は、図８（ｃ）に示す輝度変化を示す座標位置の部分領域Ｑとして特定される。そして、部分領域Ｑに含まれる画素群を一まとまりとして、輝度値の演算（画像処理）が行われる。 In step S3 of FIG. 7, the partial area including the body movement location is specified as the partial area Q of the coordinate position indicating the luminance change shown in FIG. Then, a luminance value calculation (image processing) is performed with a group of pixels included in the partial region Q as a group.
　再び図７を参照する。 Refer to FIG. 7 again.
　ステップＳ４において、画像処理回路３０６は、体動箇所を含む部分領域Ｑの平均輝度値を算出する。 In step S4, the image processing circuit 306 calculates an average luminance value of the partial region Q including the body movement location.
　そしてステップＳ５において、画像処理回路３０６は、算出した各平均輝度値を利用して被験者の呼吸数をカウントする。算出した各平均輝度値は、図４に示す波形として表現される。画像処理回路３０６は、部分領域Ｑの平均輝度値が振動することによって特定される体動の１周期を１呼吸として、所定期間内の呼吸数をカウントする。 In step S5, the image processing circuit 306 counts the breathing rate of the subject using each calculated average luminance value. Each calculated average luminance value is expressed as a waveform shown in FIG. The image processing circuit 306 counts the number of breaths within a predetermined period, with one period of body movement specified by the average luminance value of the partial region Q oscillating as one breath.
　以上の処理により、生体活動計測システム１００において、高い精度で呼吸数の情報を取得することができる。再帰性反射材４０から反射された赤外光の光量が十分大きいため、カメラ１０が撮影した動画像を用いると、画像内の再帰性反射材４０から呼吸による体動を含む部分領域を特定することは容易である。 Through the above processing, the life activity measurement system 100 can acquire respiratory rate information with high accuracy. Since the amount of infrared light reflected from the retroreflecting material 40 is sufficiently large, when a moving image taken by the camera 10 is used, a partial region including body movement due to respiration is specified from the retroreflecting material 40 in the image. It is easy.
　（再帰性反射材４０に関する変形例）
　次に、再帰性反射材４０に特定のパターンを与え、そのパターンの情報を利用することで、より確実に、かつより簡易に、呼吸による体動を含む部分領域を特定できることを説明する。あるいは、再帰性反射材４０を被験者１に装着する際の装着のしやすさを向上させることも可能になる。 (Modification regarding retroreflective material 40)
Next, it will be described that a partial pattern including body movement due to breathing can be specified more reliably and more easily by giving a specific pattern to the retroreflecting material 40 and using the information of the pattern. Or it becomes possible to improve the ease of mounting | wearing at the time of mounting | wearing the test subject 1 with the retroreflection material 40. FIG.
　図９（ａ）～（ｆ）は、それぞれ、再帰性反射材４０の形状の例を示す。図９では参照符号４０の記載は省略している。図９（ａ）～（ｄ）の再帰性反射材４０は三角形状であり、図９（ｅ）および（ｆ）の再帰性反射材４０は、矩形状の反射材を組み合わせて構成されている。 9 (a) to 9 (f) show examples of the shape of the retroreflecting material 40, respectively. In FIG. 9, the reference numeral 40 is omitted. The retroreflecting material 40 in FIGS. 9A to 9D has a triangular shape, and the retroreflecting material 40 in FIGS. 9E and 9F is configured by combining rectangular reflecting materials. .
　図９（ａ）は、指向性が異なる反射材を組み合わせた再帰性反射材４０の例を示す。プリズム型の再帰性反射材は再帰反射する方向に関して指向性を有するものが存在する。そこで、図９（ａ）に示すように、反射方向が９０°ずれている反射材ａ１およびａ２を交互に短冊状に並べる。これにより、再帰性反射材４０の設置時に指向性を考慮する必要がなくなる。 FIG. 9A shows an example of a retroreflecting material 40 in which reflecting materials having different directivities are combined. Some prism type retroreflective materials have directivity in the retroreflecting direction. Therefore, as shown in FIG. 9A, the reflectors a1 and a2 whose reflection directions are shifted by 90 ° are alternately arranged in a strip shape. This eliminates the need to consider directivity when installing the retroreflecting material 40.
　図９（ｂ）は、赤外光を反射しない部分ｂ１と、赤外光を反射する部分ｂ２とを含む再帰性反射材４０の例を示す。この例では、赤外光を反射しない部分ｂ１は再帰性反射材４０の外縁に設けられており、赤外光を反射する部分ｂ２は再帰性反射材４０の内側の領域に設けられている。これにより、エッジ部分のコントラストを高め、呼吸による体動から輝度の変化を大きくすることが可能になる。 FIG. 9B shows an example of the retroreflecting material 40 including a part b1 that does not reflect infrared light and a part b2 that reflects infrared light. In this example, the part b 1 that does not reflect infrared light is provided at the outer edge of the retroreflecting material 40, and the part b 2 that reflects infrared light is provided in an area inside the retroreflecting material 40. As a result, it is possible to increase the contrast of the edge portion and increase the change in luminance due to body movement due to breathing.
　再帰性反射材４０を利用してフレーム画像を複数の部分領域に分割し、そのうちの輝度の高い部分領域を用いて処理を行うとしても、呼吸による体動を輝度で捉えることが困難な場合がある。たとえば、装着している衣服や背景の壁の輝度が高い場合には、輝度に基づいて呼吸による体動を含む部分領域を決定することが難しい。そこで、再帰性反射材４０の一部に赤外光を反射しない箇所を作り、呼吸による体動で確実に輝度変化が発生させている。 Even if the frame image is divided into a plurality of partial areas using the retroreflecting material 40 and processing is performed using the partial areas with high luminance, it may be difficult to capture body movements due to respiration. is there. For example, in the case where the brightness of the clothes or the background wall being worn is high, it is difficult to determine a partial region including body movement due to respiration based on the brightness. Therefore, a portion that does not reflect infrared light is formed in a part of the retroreflecting material 40, and the luminance change is surely generated by body movement due to respiration.
　図９（ｃ）は、反射率がグラデーション状に変化する再帰性反射材４０の例を示す。このような再帰性反射材４０を用いると、再帰性反射材４０からの反射光の強度がグラデーション状に変化する。そのため、たとえば再帰性反射材４０からの反射光の領域Ｒに完全に含まれている部分領域Ｑを設定することも可能になる。 FIG. 9C shows an example of the retroreflecting material 40 whose reflectance changes in a gradation. When such a retroreflective material 40 is used, the intensity of the reflected light from the retroreflective material 40 changes in a gradation. Therefore, for example, it is possible to set a partial region Q that is completely included in the region R of the reflected light from the retroreflecting material 40.
　図９（ｄ）は、所定の模様ｄ１を含む再帰性反射材４０を示す。模様ｄ１は、赤外光を反射しない材質を用いて形成され得る。この模様はフレーム画像にも出現するため、再帰性反射材４０の向きを含む位置を特定することができる。 FIG. 9D shows a retroreflecting material 40 including a predetermined pattern d1. The pattern d1 can be formed using a material that does not reflect infrared light. Since this pattern also appears in the frame image, the position including the direction of the retroreflecting material 40 can be specified.
　図９（ｅ）は、赤外光を反射しない部分ｅ１と、赤外光を反射する部分ｅ２とを含む再帰性反射材４０の他の例を示す。図９（ｂ）と異なり、この例では、再帰性反射材４０の内部に赤外光を反射しない部分ｅ１が設けられている。境界数が多くなるため、部分領域を設定する条件が整いやすい。 FIG. 9E shows another example of the retroreflective member 40 including a part e1 that does not reflect infrared light and a part e2 that reflects infrared light. Unlike FIG. 9B, in this example, a portion e <b> 1 that does not reflect infrared light is provided inside the retroreflecting material 40. Since the number of boundaries increases, the conditions for setting the partial areas are easily adjusted.
　図９（ｆ）は、赤外光を反射する部分ｆ１およびｆ２を複数箇所に分散して設けた再帰性反射材４０を示す。たとえば赤外光を反射する部分ｆ１の四隅に赤外光を反射する部分ｆ２が設けられている。この例でも境界数が多くなるため、部分領域を設定する条件が整いやすい。 FIG. 9 (f) shows a retroreflecting material 40 in which the portions f1 and f2 that reflect infrared light are dispersed and provided at a plurality of locations. For example, the portions f2 that reflect infrared light are provided at the four corners of the portion f1 that reflects infrared light. In this example as well, the number of boundaries increases, so the conditions for setting the partial areas are easy to be prepared.
　図１０（ａ）は、図９（ａ）の再帰性反射材４０を被験者１へ装着した例を示し、図１０（ｂ）は撮像例を示す。 FIG. 10A shows an example in which the retroreflecting material 40 of FIG. 9A is attached to the subject 1, and FIG. 10B shows an imaging example.
　再帰性反射材４０のエッジ部分が、被験者１の呼吸による体動が発生する箇所（たとえば腹部または胸部）に位置するように再帰性反射材４０が設置される。情報処理装置３０の画像処理回路３０６は、予め、このようなパターンを有する再帰性反射材４０が用いられること、およびそのパターンの特徴を示す情報を、たとえばＲＯＭ３０２に保持している。 The retroreflective material 40 is installed so that the edge portion of the retroreflective material 40 is located at a location (for example, the abdomen or chest) where body movement due to the breathing of the subject 1 occurs. The image processing circuit 306 of the information processing apparatus 30 holds in advance, for example, in the ROM 302 information indicating that the retroreflecting material 40 having such a pattern is used and the characteristics of the pattern.
　このような再帰性反射材４０を利用したときの生体活動計測システム１００における処理を説明する。図７の処理手順におけるステップＳ２およびＳ３に代えて、以下の処理が行われる。 A process in the biological activity measurement system 100 when such a retroreflecting material 40 is used will be described. The following processing is performed instead of steps S2 and S3 in the processing procedure of FIG.
　カメラ１０が被験者１を撮影すると、画像処理回路３０６は、得られた動画像の各フレーム画像において、図１０（ｂ）に示されるような、短冊状に輝度値が高い領域を特定する。画像処理回路３０６は、予め保持していたパターンの特徴を利用して各フレーム画像にパターンマッチング処理を行い、再帰性反射材４０の位置を特定する。 When the camera 10 captures the subject 1, the image processing circuit 306 identifies a region having a high luminance value in a strip shape as shown in FIG. 10B in each frame image of the obtained moving image. The image processing circuit 306 performs a pattern matching process on each frame image using the characteristics of the pattern held in advance, and specifies the position of the retroreflecting material 40.
　次に、画像処理回路３０６は各フレーム画像を２つ以上に分割する。このとき、画像処理回路３０６は再帰性反射材４０をまたぐ位置に分割線（境界線）を設定する。さらに、分割線は、体動方向とは異なる方向に設定される。たとえばフレーム画像内で体動が上下方向に認められるとする。このとき画像処理回路３０６は、たとえば水平方向に境界線を設定して各フレーム画像を２つ以上に分割し、部分領域を設定する。画像処理回路３０６は、部分領域の平均輝度値を利用して被験者１の呼吸数をカウントすることができる。 Next, the image processing circuit 306 divides each frame image into two or more. At this time, the image processing circuit 306 sets a dividing line (boundary line) at a position across the retroreflecting material 40. Further, the dividing line is set in a direction different from the body movement direction. For example, it is assumed that body movement is recognized in the vertical direction in the frame image. At this time, for example, the image processing circuit 306 sets a boundary line in the horizontal direction, divides each frame image into two or more, and sets a partial region. The image processing circuit 306 can count the respiration rate of the subject 1 using the average luminance value of the partial area.
　既知の形状の再帰性反射材４０を利用して位置を特定し、再帰性反射材４０周辺を部分領域に分割することにより、呼吸による輝度の変化を１か所以上で測定できるようになる。よって検出精度を向上することができる。 By specifying the position using the retroreflecting material 40 having a known shape and dividing the periphery of the retroreflecting material 40 into partial areas, it becomes possible to measure a change in luminance due to respiration at one or more locations. Therefore, detection accuracy can be improved.
　ここで、再帰性反射材４０に関する更に他の例を説明する。 Here, still another example regarding the retroreflecting material 40 will be described.
　図１１（ａ１）～（ｃ１）はそれぞれ、三角形の反射部分の周囲にマーカとして機能する反射材を設けた再帰性反射材４０を示す。参考として、図１１（ａ１）のみに、反射部分４０ａ、およびマーカ４０ｂを示す。なお、図示されたマーカの形状は一例である。反射部分４０ａとの関係で識別可能であれば、その形状は任意である。 11 (a1) to 11 (c1) each show a retroreflecting material 40 provided with a reflecting material functioning as a marker around a triangular reflecting portion. For reference, only the reflective portion 40a and the marker 40b are shown in FIG. 11 (a1). In addition, the shape of the illustrated marker is an example. The shape is arbitrary as long as it can be identified in relation to the reflective portion 40a.
　このような再帰性反射材を利用したときの処理は、概ね上述した通りである。反射部分４０ａ、およびマーカ４０ｂを含む、既知の形状の再帰性反射材４０を利用して位置を特定し、再帰性反射材４０の周辺に部分領域に分割すればよい。 The processing when such a retroreflective material is used is generally as described above. The position may be specified by using a retroreflecting material 40 having a known shape including the reflecting portion 40 a and the marker 40 b and divided into partial regions around the retroreflecting material 40.
　部分領域に分割する際には、画像処理回路３０６は、検索した周囲のマーカ４０ｂを基準に、そのマーカ４０ｂ内を２つ以上の部分領域に分割すればよい。たとえば図１１（ａ２）は、マーカ４０ｂ内に設けられた２つ部分領域Ｑ１およびＱ２を示す。なお、部分領域Ｑ１およびＱ２の形状またはサイズは互いに異なっていてもよい。どのような再帰性反射材４０を利用するかに応じて、部分領域Ｑ１およびＱ２の形状またはサイズを予め決定しておいてもよい。 When dividing into partial areas, the image processing circuit 306 may divide the marker 40b into two or more partial areas based on the searched surrounding marker 40b. For example, FIG. 11 (a2) shows two partial regions Q1 and Q2 provided in the marker 40b. Note that the shapes or sizes of the partial regions Q1 and Q2 may be different from each other. Depending on what type of retroreflecting material 40 is used, the shapes or sizes of the partial regions Q1 and Q2 may be determined in advance.
　図１１（ｂ１）および（ｃ１）の各々についても図１１（ａ１）と同様である。また、図１１（ｂ２）および（ｃ２）に例示されるような形状で、部分領域が設定されてもよい。 11 (b1) and (c1) are the same as those in FIG. 11 (a1). Further, the partial region may be set in a shape exemplified in FIGS. 11B2 and 11C2.
　マーカを設けることにより、再帰性反射材４０に関する部分領域を限定して設定することができるため、計算処理の負荷を大きく軽減できる。 By providing a marker, it is possible to limit and set a partial area related to the retroreflecting material 40, so that the load of calculation processing can be greatly reduced.
　続いて、再帰性反射材４０の更なる変形例を説明する。 Subsequently, a further modification of the retroreflecting material 40 will be described.
　本実施の形態による生体活動計測システム１００は、再帰性反射材４０を適切に選択することにより、磁気共鳴画像診断装置を用いた測定・検査においても利用することが可能である。 The life activity measurement system 100 according to the present embodiment can be used in measurement / inspection using a magnetic resonance imaging apparatus by appropriately selecting the retroreflecting material 40.
　上述した再帰性反射材４０は、アルミを蒸着させて形成されているものが存在する。再帰性反射材４０が金属材料を含む場合には、その再帰性反射材４０を、磁気共鳴画像診断装置を利用した測定・検査（いわゆるＭＲＩ測定・検査）に用いることはできない。磁気共鳴画像診断装置のガントリ内で撮影画像に影響を与える恐れが生じるからである。 The above-described retroreflecting material 40 may be formed by depositing aluminum. When the retroreflective material 40 includes a metal material, the retroreflective material 40 cannot be used for measurement / inspection (so-called MRI measurement / inspection) using a magnetic resonance imaging apparatus. This is because the captured image may be affected in the gantry of the magnetic resonance imaging apparatus.
　ＭＲＩ測定・検査を行う環境下では、非金属材質の再帰性反射材４０を採用する必要がある。たとえばＰＥＴ製のプリズムシートを再帰性反射材４０として使用することが可能である。 In an environment where MRI measurement / inspection is performed, it is necessary to employ a non-metallic retroreflecting material 40. For example, a PET prism sheet can be used as the retroreflecting material 40.
　これにより、ＭＲＩ環境下においても再帰性反射材４０を患者に装着して、周囲環境のノイズの影響を低減して生体情報を取得することが可能となる。 This makes it possible to wear the retroreflecting material 40 on the patient even in an MRI environment, and to obtain biological information by reducing the influence of noise in the surrounding environment.
　（カメラに関する変形例）
　次に、カメラに関する変形例を説明する。なお、以下に特に説明する構成および動作を除いては、生体活動計測システムの構成および動作は、実施の形態１の生体活動計測システム１００（図１）と同じである。 (Camera variation)
Next, a modification regarding the camera will be described. Except for the configuration and operation specifically described below, the configuration and operation of the life activity measurement system are the same as those of the life activity measurement system 100 (FIG. 1) of the first embodiment.
　図１２は、可視光領域の波長を遮る光学フィルタ１１を装着したカメラ１０を示す。この光学フィルタ１１は、たとえば赤外フィルタとも呼ばれる。 FIG. 12 shows the camera 10 equipped with an optical filter 11 that blocks the wavelength in the visible light region. This optical filter 11 is also called an infrared filter, for example.
　本実施の形態では、フィルタ１１を設けて被験者１を撮影する。光学フィルタ１１は、光源２０から放射され、再帰性反射材４０において反射された赤外光は透過するが、可視光は遮断する。光学フィルタ１１を設けることにより、赤外光以外の光、より具体的には可視光、がカメラ１０に入射することを防ぎ、それにより、撮影された動画像の輝度値の変化への影響を低減できる。可視光に起因する各フレーム画像の輝度値の変動を抑制できるため、可視光のみに起因し、生体反応に起因しない外乱ノイズの発生を低減できる。 In this embodiment, the filter 11 is provided to photograph the subject 1. The optical filter 11 transmits infrared light emitted from the light source 20 and reflected by the retroreflecting material 40, but blocks visible light. By providing the optical filter 11, light other than infrared light, more specifically visible light, is prevented from entering the camera 10, thereby affecting the change in the luminance value of the captured moving image. Can be reduced. Since fluctuations in the luminance value of each frame image due to visible light can be suppressed, it is possible to reduce the occurrence of disturbance noise due to only visible light and not due to biological reactions.
　本願発明者らは、可視光領域の波長を遮る光学フィルタ１１を設けることは非常に有用であると考えている。その理由は、実際の撮影時には、完全な暗室環境を実現することが困難な場合が多いからである。たとえば病院で生体活動計測システム１００を動作させる場合には、夜間であったとしても常夜灯、避難誘導灯などが院内に点灯する。そのような撮影環境では光学フィルタ１１によって可視光を遮断することが好適である。 The inventors of the present application consider that it is very useful to provide the optical filter 11 that blocks the wavelength in the visible light region. This is because it is often difficult to realize a complete dark room environment during actual photographing. For example, when the life activity measurement system 100 is operated in a hospital, a night light, an evacuation guide light, etc. are lit in the hospital even at night. In such a photographing environment, it is preferable to block visible light by the optical filter 11.
　さらに、図１２に示す光学フィルタ１１として、可視光のみならず不要な赤外光をも遮断する光学フィルタを設けてもよい。換言すれば、光学フィルタ１１として、光源２０が放射する赤外光を通過させるバンドパスフィルタを設けてもよい。 Furthermore, as the optical filter 11 shown in FIG. 12, an optical filter that blocks not only visible light but also unnecessary infrared light may be provided. In other words, a band pass filter that allows infrared light emitted from the light source 20 to pass therethrough may be provided as the optical filter 11.
　まず、光源２０として急峻な波長特性を有するＬＥＤ光源を採用する。波長は、たとえば８５０ｎｍまたは９４０ｎｍ、およびそれらの近傍である。「急峻な波長特性」とは、ここでは放射される赤外光の波長の変動が小さいことを意味する。 First, an LED light source having a steep wavelength characteristic is adopted as the light source 20. The wavelength is, for example, 850 nm or 940 nm and their vicinity. The “steep wavelength characteristic” means that the fluctuation of the wavelength of the emitted infrared light is small here.
　光源２０に対応して、光学フィルタ１１として、光源２０から放射される赤外光を通過させるバンドパス特性を有する光学フィルタをカメラ１０に設ける。たとえば、光源２０から放射される赤外光の波長が８５０ｎｍの場合には、８５０ｎｍの波長の赤外光を透過させる光学フィルタ１１を設ける。 Corresponding to the light source 20, the camera 10 is provided with an optical filter having a bandpass characteristic that allows infrared light emitted from the light source 20 to pass therethrough as the optical filter 11. For example, when the wavelength of infrared light emitted from the light source 20 is 850 nm, the optical filter 11 that transmits infrared light having a wavelength of 850 nm is provided.
　光源２０の波長と、光学フィルタ１１の通過帯域とを一致させることにより、カメラ１０は、光源２０から放射される赤外光の波長と同じ波長の光のみに感度を持つことになる。可視光のみならず、不要な赤外光をも遮断できるため、撮影された動画像は外乱光の影響を受けない。なお、再帰性反射材４０を利用しているため、光源２０から放射され、再帰性反射材４０で反射された赤外光の光量は大きい。よって、反射光の捉え易さは先に説明した態様と同じである。 By matching the wavelength of the light source 20 with the pass band of the optical filter 11, the camera 10 has sensitivity only to light having the same wavelength as that of infrared light emitted from the light source 20. Since not only visible light but also unnecessary infrared light can be blocked, the captured moving image is not affected by disturbance light. Since the retroreflective material 40 is used, the amount of infrared light emitted from the light source 20 and reflected by the retroreflective material 40 is large. Therefore, the ease of capturing the reflected light is the same as that described above.
　図１３は、偏光フィルタ１２ａが設けられた光源２０と、偏光フィルタ１２ｂが設けられたカメラ１０とを示す。この例では、カメラ１０には上述した光学フィルタ１１が設けられているが、光学フィルタ１１は必須ではない。 FIG. 13 shows the light source 20 provided with the polarizing filter 12a and the camera 10 provided with the polarizing filter 12b. In this example, the camera 10 is provided with the optical filter 11 described above, but the optical filter 11 is not essential.
　偏光フィルタ１２ａおよび１２ｂは、その偏光方向が一致するようカメラ１０および光源２０に設置される。 The polarizing filters 12a and 12b are installed in the camera 10 and the light source 20 so that their polarization directions coincide.
　この構成によれば、偏光フィルタ１２ａおよび１２ｂにより、カメラ１０は、光源２０から放射された赤外光と同じ偏光方向の光に対してのみ感度を持つ。これにより、光源２０から放射された赤外光のうち、偏光フィルタ１２ａを通過した、所定の偏光方向を有する赤外光のみが再帰性反射材４０で反射され、さらにカメラ１０に入射する。よって、その偏光方向と異なる偏光方向の外乱光（可視光および赤外光）の影響を排除できる。 According to this configuration, the camera 10 is sensitive only to light having the same polarization direction as the infrared light emitted from the light source 20 by the polarizing filters 12a and 12b. Thereby, only the infrared light having a predetermined polarization direction that has passed through the polarizing filter 12 a out of the infrared light emitted from the light source 20 is reflected by the retroreflecting material 40 and further enters the camera 10. Therefore, the influence of disturbance light (visible light and infrared light) having a polarization direction different from the polarization direction can be eliminated.
　（位置マーカの導入による撮影方向の判定）
　図１４は、位置マーカ４１を利用して撮影方向を判定する生体活動計測システム１０１の構成を示す。位置マーカ４１を利用すること、および位置マーカ４１に関する情報処理装置３０の処理を除いては、これまでの説明と同じである。 (Judging the shooting direction by introducing a position marker)
FIG. 14 shows the configuration of the life activity measurement system 101 that uses the position marker 41 to determine the imaging direction. Except for the use of the position marker 41 and the processing of the information processing apparatus 30 regarding the position marker 41, the description is the same as the above description.
　位置マーカ４１は、生体活動計測システム１０１が構築されている部屋の壁面等の予め固定されている場所に設けられた再帰性反射材である。本例では、位置マーカ４１は五角形状であるとする。しかしながらこれは一例である。位置マーカ４１は、円形、楕円形、矩形などであってもよい。 The position marker 41 is a retroreflecting material provided in a fixed place such as a wall surface of a room where the life activity measurement system 101 is constructed. In this example, it is assumed that the position marker 41 has a pentagonal shape. However, this is an example. The position marker 41 may be circular, elliptical, rectangular, or the like.
　位置マーカ４１の形状および大きさは、カメラ１０が位置マーカ４１の形状を識別可能であれば任意である。すなわち、位置マーカ４１の形状および大きさは、位置マーカ４１とカメラ１０との間の距離を考慮した、カメラ１０の解像度に応じて任意に選択可能である。 The shape and size of the position marker 41 are arbitrary as long as the camera 10 can identify the shape of the position marker 41. That is, the shape and size of the position marker 41 can be arbitrarily selected according to the resolution of the camera 10 in consideration of the distance between the position marker 41 and the camera 10.
　光源２０が放射した赤外光は、たとえば円錐状に広がって位置マーカ４１に到達する。図１４には、赤外光が通過する領域を空間Ｓとして示している。位置マーカ４１である再帰性反射材で反射された赤外光の一部は、カメラ１０に到達する。これにより、カメラ１０は位置マーカ４１を撮影できる。 The infrared light emitted from the light source 20 spreads in a conical shape and reaches the position marker 41, for example. In FIG. 14, a region through which infrared light passes is shown as a space S. Part of the infrared light reflected by the retroreflecting material that is the position marker 41 reaches the camera 10. Thereby, the camera 10 can photograph the position marker 41.
　情報処理装置３０の画像処理回路３０６は、予め位置マーカ４１の形状の情報、および位置マーカ４１の設置される位置に関する情報を、たとえばＲＯＭ３０２に保持している。位置マーカ４１の設置される位置とは、たとえばカメラ１０の設置位置および撮影方向が適正に決定されて、位置マーカ４１が撮影されたときに、その位置マーカ４１が撮影された画像のどの位置に存在するかを特定する情報である。 The image processing circuit 306 of the information processing apparatus 30 holds, in advance, information on the shape of the position marker 41 and information on the position where the position marker 41 is installed, for example, in the ROM 302. The position where the position marker 41 is installed is, for example, where the installation position of the camera 10 and the shooting direction are appropriately determined and when the position marker 41 is shot, in which position of the image where the position marker 41 is shot. This information identifies whether it exists.
　画像処理回路３０６は撮影された動画像を解析し、保持していた情報を利用して、たとえばパターンマッチング処理を行い、動画像のフレーム画像中に位置マーカ４１が含まれているか否か、および含まれている場合にはその位置を特定する。 The image processing circuit 306 analyzes the captured moving image and performs, for example, pattern matching processing using the stored information to determine whether or not the position marker 41 is included in the frame image of the moving image, and If it is included, the position is specified.
　そして画像処理回路３０６は、予め保持している情報を参照して、特定した位置と、位置マーカ４１の設置されるべき位置とが一致しているか否かを判定する。一致していない場合には、カメラ１０が、適正な位置および／または撮影方向に設置されていないことを意味する。 Then, the image processing circuit 306 determines whether or not the specified position matches the position where the position marker 41 is to be installed with reference to information held in advance. If they do not match, it means that the camera 10 is not installed in the proper position and / or shooting direction.
　画像処理回路３０６は、一致していないことを示す警告を報知する。たとえば画像処理回路３０６は、ディスプレイ３２に映像信号を送り、「カメラの位置または撮影方向がずれています。確認して下さい。」というメッセージを呈示させる。またはＣＰＵ３０１は、図示されない音声処理回路を介して、警告音または警告メッセージの音声を呈示させてもよい。 The image processing circuit 306 notifies a warning indicating that they do not match. For example, the image processing circuit 306 sends a video signal to the display 32 to present a message “Camera position or shooting direction is shifted. Please check”. Alternatively, the CPU 301 may present a warning sound or a warning message via a voice processing circuit (not shown).
　物が当たるなどの原因で、カメラ１０の位置および／または撮影方向がずれ、被験者１を適切に撮影できない状況にあるときは、生体活動計測システム１００はその状況を検出し、警告を報知することができる。 When the position of the camera 10 and / or the shooting direction is shifted due to an object hitting or the like, and the subject 1 is in a situation where the subject 1 cannot be properly photographed, the life activity measurement system 100 detects the situation and issues a warning. Can do.
　（実施の形態２）
　本実施の形態は、呼吸による体動が生じた箇所を含むフレーム画像に部分領域を設定する処理に関する。 (Embodiment 2)
The present embodiment relates to processing for setting a partial region in a frame image including a place where body movement due to respiration has occurred.
　本実施の形態においても、図１および図６に示す生体活動計測システム１００を参照する。本実施の形態と、実施の形態１との相違点は、主として情報処理装置３０（図１および図６）の処理である。その他の構成および動作に関しては、再帰性反射材４０の変形例等を含め、実施の形態１と同じ構成および動作が、本実施の形態にも適用され得る。 Also in this embodiment, the life activity measurement system 100 shown in FIGS. 1 and 6 is referred to. The difference between the present embodiment and the first embodiment is mainly the processing of the information processing apparatus 30 (FIGS. 1 and 6). Regarding the other configurations and operations, the same configurations and operations as those of the first embodiment, including the modified example of the retroreflecting material 40, can be applied to the present embodiment.
　上述した実施の形態では、再帰性反射材４０の位置、大きさ、および画像処理の１単位である部分領域のサイズの関係で、期待通りの呼吸波形を得ることが出来ないことがある。たとえば、ある被験者１の呼吸による体動が比較的小さく、さらに処理単位である部分領域も比較的小さいと仮定する。このような仮定の下では、図８（ａ）および（ｂ）に示されるような部分領域Ｑを特定できないことがある。その結果、理想的ではない領域を、輝度値の演算（画像処理）を行う単位領域として設定してしまう可能性がある。 In the above-described embodiment, the expected respiration waveform may not be obtained due to the relationship between the position and size of the retroreflecting material 40 and the size of the partial area that is one unit of image processing. For example, it is assumed that the body movement due to breathing of a subject 1 is relatively small and that the partial area as a processing unit is also relatively small. Under such an assumption, the partial region Q as shown in FIGS. 8A and 8B may not be specified. As a result, a non-ideal region may be set as a unit region for performing luminance value calculation (image processing).
　図１５（ａ）および（ｂ）は、体動による輝度値の変化が現れない座標位置の領域が部分領域Ｑとして設定された例を示す。呼吸に起因する体動により、反射光の領域Ｒの位置は、図１５（ａ）および（ｂ）の状態をそれぞれ下限および上限として変化する。この場合、部分領域Ｑは、常に反射光の領域Ｒの中に入っているため、体動による輝度値の変化が現れない。図１５（ｃ）は、設定された部分領域Ｑの輝度値の変化を示す。呼吸の有無にかかわらず輝度が高い状態が続いていることが理解される。輝度値の変動が小さいため、輝度値を利用して呼吸の情報を抽出することは非常に困難である。 FIGS. 15A and 15B show an example in which a region at a coordinate position where a change in luminance value due to body movement does not appear is set as the partial region Q. FIG. Due to body movement caused by respiration, the position of the reflected light region R changes with the states of FIGS. 15A and 15B as the lower limit and the upper limit, respectively. In this case, since the partial area Q is always in the reflected light area R, the luminance value does not change due to body movement. FIG. 15C shows a change in the luminance value of the set partial area Q. It is understood that the state of high brightness continues regardless of whether there is breathing. Since the fluctuation of the luminance value is small, it is very difficult to extract respiration information using the luminance value.
　図１６（ａ）および（ｂ）は、体動による輝度値の変化が現れない座標位置の領域が部分領域Ｑとして設定された他の例を示す。体動が図面の上下方向に認められるとする。このとき、反射光の領域Ｒの上下のエッジを常に含むように部分領域Ｑを設定してしまうと、反射光の領域Ｒが変動したとしても、部分領域Ｑの輝度値は変化しない。このように部分領域Ｑを設定してしまった場合にも、輝度値を利用して呼吸の情報を抽出することは非常に困難である。 FIGS. 16A and 16B show another example in which a region at a coordinate position where a change in luminance value due to body movement does not appear is set as the partial region Q. FIG. Assume that body movement is recognized in the vertical direction of the drawing. At this time, if the partial region Q is set so as to always include the upper and lower edges of the reflected light region R, the luminance value of the partial region Q does not change even if the reflected light region R changes. Even when the partial area Q is set in this way, it is very difficult to extract respiration information using the luminance value.
　そこで本実施の形態では、画像処理回路３０６（図６）が、輝度値の変化を検出するための部分領域Ｑのサイズを動的に変更する。以下、部分領域Ｑの設定例を説明する。なお、以下では、輝度値の演算を行う一まとまりの画素群を含む領域を「処理単位領域」と呼ぶ。 Therefore, in the present embodiment, the image processing circuit 306 (FIG. 6) dynamically changes the size of the partial region Q for detecting a change in luminance value. Hereinafter, an example of setting the partial area Q will be described. In the following, an area including a group of pixels for calculating a luminance value is referred to as a “processing unit area”.
　本実施の形態では、反射光の領域Ｒのエッジを特定し、エッジの座標位置が時刻に応じて変動する場合に、そのエッジが横切る領域を含むよう、処理単位領域を設定する。ただし、ここでいう反射光の領域Ｒのエッジは、変動方向に関して対向する複数のエッジが存在する場合にはその一方のエッジであるとする。たとえば変動方向に関して対向する２本のエッジを同時に横切る領域は処理単位領域として設定しない。 In the present embodiment, the edge of the reflected light region R is specified, and when the coordinate position of the edge varies according to time, the processing unit region is set so as to include the region that the edge crosses. However, the edge of the reflected light region R here is assumed to be one edge when there are a plurality of edges facing each other with respect to the fluctuation direction. For example, an area that simultaneously crosses two edges facing each other with respect to the changing direction is not set as a processing unit area.
　以下、具体的に説明する。 The details will be described below.
　図１７（ａ）および（ｂ）は、処理単位領域としての部分領域Ｑを、再帰性反射材４０からの反射光の領域Ｒと概ね同じ、または領域Ｒよりも大きく確保した例を示す。 FIGS. 17A and 17B show an example in which the partial region Q as the processing unit region is secured approximately equal to or larger than the region R of the reflected light from the retroreflecting material 40.
　図１７（ａ）および（ｂ）は、異なる時刻に撮影された２枚のフレーム画像を示している。各フレーム画像は、反射光の領域Ｒを含む。そして領域Ｒのエッジ（境界）の座標位置は経時的に変化している。そこで、領域Ｒのエッジが横切る領域を含む部分領域Ｑを、処理単位領域として設定する。反射光の領域Ｒと部分領域Ｑとが重複する比率は経時的に変化する。この経時的な変化は、部分領域Ｑの輝度値の変化として現れる。よってそのような経時的な変化を含む領域を処理単位領域として設定すればよい。 FIGS. 17A and 17B show two frame images taken at different times. Each frame image includes a region R of reflected light. The coordinate position of the edge (boundary) of the region R changes with time. Therefore, the partial region Q including the region crossed by the edge of the region R is set as the processing unit region. The ratio at which the reflected light region R and the partial region Q overlap changes with time. This change over time appears as a change in the luminance value of the partial region Q. Therefore, an area including such a change over time may be set as a processing unit area.
　なお、反射光の領域Ｒと部分領域Ｑとが重複する比率が同じであれば、処理単位領域の輝度値は変化しないことになり、輝度値の変化を利用して呼吸数をカウントすることはできない。 If the ratio of overlap of the reflected light region R and the partial region Q is the same, the luminance value of the processing unit region will not change, and counting the respiration rate using the change in luminance value is not possible. Can not.
　図１７（ｃ）は、部分領域Ｑの輝度値の変化を示す。部分領域Ｑが反射光の領域Ｒよりも大きいため、再帰性反射材４０からの反射光の輝度変化のダイナミックレンジを大きくとることが可能となる。 FIG. 17C shows a change in luminance value of the partial region Q. Since the partial area Q is larger than the reflected light area R, the dynamic range of the luminance change of the reflected light from the retroreflecting material 40 can be increased.
　たとえば図８（ａ）および（ｂ）の処理単位領域としての部分領域Ｑのサイズが６４画素ｘ６４画素であるとすると、図１７（ａ）および（ｂ）では、部分領域Ｑのサイズは２５６画素ｘ２５６画素である。 For example, assuming that the size of the partial region Q as the processing unit region in FIGS. 8A and 8B is 64 pixels × 64 pixels, in FIGS. 17A and 17B, the size of the partial region Q is 256 pixels. x256 pixels.
　次に、図１８（ａ）および（ｂ）は、処理単位領域としての部分領域Ｑを、図８の部分領域Ｑよりも小さく確保した例を示す。 Next, FIGS. 18A and 18B show an example in which the partial area Q as the processing unit area is secured smaller than the partial area Q of FIG.
　図１８（ａ）では、部分領域Ｑは反射光の領域Ｒに含まれているため、その輝度値は非常に大きい。一方、図１８（ｂ）では、部分領域Ｑは反射光の領域Ｒのエッジ（境界）を含んでいる。つまり、この例でも、反射光の領域Ｒと部分領域Ｑとが重複する比率は経時的に変化する。 In FIG. 18A, since the partial area Q is included in the reflected light area R, the luminance value is very large. On the other hand, in FIG. 18B, the partial region Q includes the edge (boundary) of the region R of the reflected light. That is, also in this example, the ratio of the reflected light region R and the partial region Q changes with time.
　図１８（ｃ）は、部分領域Ｑの輝度値の変化を示す。図１８（ｃ）によれば、輝度値の変化を利用して呼吸数をカウントすることは可能である。 FIG. 18C shows a change in luminance value of the partial region Q. According to FIG.18 (c), it is possible to count a respiration rate using the change of a luminance value.
　たとえば図８（ａ）および（ｂ）の処理単位領域としての部分領域Ｑのサイズが６４画素ｘ６４画素であるとすると、図１７（ａ）および（ｂ）では、部分領域Ｑのサイズを３２画素ｘ３２画素である。 For example, assuming that the size of the partial area Q as the processing unit area in FIGS. 8A and 8B is 64 pixels × 64 pixels, in FIGS. 17A and 17B, the size of the partial area Q is 32 pixels. x32 pixels.
　図１９は、時間的に変動する反射光の領域Ｒの変動範囲と、設定可能な最小の処理単位領域ＱａおよびＱｂを示す。処理単位領域ＱａおよびＱｂはいずれも、反射光の領域Ｒが変動することによって領域Ｒのエッジに横切られる領域である。換言すれば、処理単位領域ＱａおよびＱｂはいずれも、その一部または全部が領域Ｒに入ったり入らなかったりする位置に存在する。反射光の領域Ｒと部分領域ＱａまたはＱｂとが重複する比率は経時的に変化する。 FIG. 19 shows the fluctuation range of the reflected light region R that varies with time, and the minimum settable processing unit regions Qa and Qb. Both the processing unit areas Qa and Qb are areas that cross the edge of the area R when the area R of the reflected light fluctuates. In other words, both of the processing unit areas Qa and Qb exist at positions where part or all of them enter or leave the area R. The ratio at which the reflected light region R and the partial region Qa or Qb overlap varies with time.
　処理単位領域Ｑａは、反射光の領域Ｒの変化が上限に達したときの反射光の領域Ｒの境界（エッジ）を含む。つまり処理単位領域Ｑａは、その一部が領域Ｒに入ったり入らなかったりする位置に存在する領域である。 The processing unit area Qa includes a boundary (edge) of the reflected light area R when the change of the reflected light area R reaches the upper limit. That is, the processing unit area Qa is an area that exists at a position where a part of the processing unit area Qa enters or does not enter the area R.
　処理単位領域Ｑｂは、反射光の領域Ｒの変化に伴って、その全部が領域Ｒに入ったり入らなかったりする位置に存在する領域である。 The processing unit region Qb is a region that exists at a position where the entire region R enters or does not enter the region R as the reflected light region R changes.
　一方の領域Ｑｃは、反射光の領域Ｒの変化にかかわらず、常に反射光の領域Ｒ内に位置する領域である。部分領域Ｑｃと反射光の領域Ｒとは、常に重複する領域の比率が同じである。よって部分領域Ｑｃは処理単位領域として好適ではない。 One region Qc is a region that is always located in the region R of the reflected light regardless of changes in the region R of the reflected light. The partial area Qc and the reflected light area R always have the same ratio of overlapping areas. Therefore, the partial area Qc is not suitable as a processing unit area.
　上述した条件を満たす位置にある領域Ｑａおよび／またはＱｂを特定し、情報処理装置３０の性能等を考慮して、処理単位領域を設定すればよい。 The region Qa and / or Qb in the position satisfying the above-described conditions may be specified, and the processing unit region may be set in consideration of the performance of the information processing apparatus 30 and the like.
　図２０は、好適な処理単位領域Ｑｄ、ＱｅおよびＱｆの例を示す。いずれの領域も、反射光の領域Ｒが変動することによって領域Ｒのエッジに横切られる領域である。 FIG. 20 shows examples of suitable processing unit areas Qd, Qe, and Qf. Each region is a region that crosses the edge of the region R when the region R of the reflected light fluctuates.
　図２１は、本実施の形態による生体情報モニタリング装置３００の動作の手順を示す。この処理は主としてＣＰＵ３０１および／または画像処理回路３０６によって実行される。以下では、実行主体は画像処理回路３０６である例に挙げて説明する。 FIG. 21 shows an operation procedure of the biological information monitoring apparatus 300 according to the present embodiment. This process is mainly executed by the CPU 301 and / or the image processing circuit 306. In the following description, the execution subject is the image processing circuit 306.
　ステップＳ１１において、画像処理回路３０６は、取得した画像から再帰性反射材４０が存在する領域Ｒを特定する。たとえば、画像処理回路３０６は、再帰性反射材４０からの反射光が観測される領域の輝度値の情報を、予めＲＯＭ３０２に保持している。この情報を輝度値の閾値として利用し、閾値以上の輝度値を有する部分領域を、再帰性反射材４０が存在する領域として特定する。 In step S11, the image processing circuit 306 specifies a region R where the retroreflective material 40 exists from the acquired image. For example, the image processing circuit 306 holds in advance in the ROM 302 information on the luminance value of the region where the reflected light from the retroreflecting material 40 is observed. This information is used as a threshold value of the luminance value, and a partial region having a luminance value equal to or higher than the threshold value is specified as a region where the retroreflecting material 40 exists.
　ステップＳ１２において、画像処理回路３０６は再帰性反射材４０のエッジを含む領域を特定する。たとえば画像処理回路３０６は、隣接する画素の輝度値の差が予め定められた値以上の部分をエッジとして特定する。なお、エッジを検出する技術は公知であるため、詳細の説明は省略する。 In step S12, the image processing circuit 306 specifies an area including the edge of the retroreflecting material 40. For example, the image processing circuit 306 specifies a portion where the difference in luminance value between adjacent pixels is equal to or greater than a predetermined value as an edge. Since the technique for detecting the edge is well known, detailed description thereof is omitted.
　ステップＳ１３において、画像処理回路３０６は、エッジを含む領域を処理単位領域として選定する。ここで言うエッジとは、変動方向に関して対向する複数のエッジが存在する場合にはその一方のエッジをいう。 In step S13, the image processing circuit 306 selects a region including an edge as a processing unit region. The edge here refers to one edge when there are a plurality of edges facing each other with respect to the fluctuation direction.
　ステップＳ１４において、画像処理回路３０６は、選定した処理単位領域内に存在する各画素の輝度値を加算平均する。 In step S14, the image processing circuit 306 adds and averages the luminance values of the pixels existing in the selected processing unit area.
　ステップＳ１５において、画像処理回路３０６は算出した加算平均結果の時間変化を呼吸波形と判断し、ステップＳ１６において、呼吸波形から呼吸数をカウントする。 In step S15, the image processing circuit 306 determines a temporal change in the calculated average result as a respiratory waveform, and in step S16, counts the respiratory rate from the respiratory waveform.
　ステップＳ１７において、画像処理回路３０６はディスプレイ３２に映像信号を送り、呼吸数の情報を表示させる。 In step S17, the image processing circuit 306 sends a video signal to the display 32 to display information on the respiration rate.
　上述のステップＳ１１～Ｓ１３は、一度エッジが検出され、呼吸数の計測が始まった後でも随時行われ得る。たとえば被験者１が寝返りを打つことにより、再帰性反射材４０からの反射光の検出位置が変化する場合がある。その場合には画像処理回路３０６は改めてステップＳ１１～Ｓ１３の処理を行い、体動に起因する処理単位領域の再設定を行えばよい。呼吸による体動と比較すると、寝返りなどの体動は非常に大きいため、反射光が検出される座標位置は大きく変化する。画像処理回路３０６は、反射光が検出される座標位置が所定量以上移動した場合には、改めてステップＳ１１～Ｓ１３の処理を行う。なお、画像処理回路３０６は、ステップＳ１５においてユーザの体動をモニタし続けている。よって、呼吸以外の体動が被験者１に生じた場合には、画像処理回路３０６は体動の変化を迅速かつ容易に観測できる。 The above steps S11 to S13 can be performed at any time even after the edge is detected once and the measurement of the respiration rate is started. For example, when the subject 1 turns over, the detection position of the reflected light from the retroreflecting material 40 may change. In that case, the image processing circuit 306 may perform the processing of steps S11 to S13 again to reset the processing unit area resulting from the body movement. Compared with body movements due to respiration, body movements such as turning over are very large, and the coordinate position where reflected light is detected changes greatly. When the coordinate position where the reflected light is detected has moved by a predetermined amount or more, the image processing circuit 306 performs the processes of steps S11 to S13 again. Note that the image processing circuit 306 continues to monitor the user's body movement in step S15. Therefore, when body movements other than breathing occur in the subject 1, the image processing circuit 306 can observe changes in body movements quickly and easily.
　以上、本発明の実施の形態を説明した。 The embodiment of the present invention has been described above.
　上述の実施の形態では、図１に示す生体活動計測システム１００を例にして説明した。ただし、生体活動計測システム１００の構成は一例である。 In the above-described embodiment, the life activity measurement system 100 shown in FIG. 1 has been described as an example. However, the configuration of the life activity measurement system 100 is an example.
　たとえば図２２は、生体活動計測システム１００の変形例による生体活動計測システム１１１を示す。生体活動計測システム１１１では、複数のカメラ１０がネットワーク１１０を介して情報処理装置３０と接続されている。情報処理装置３０は、複数のカメラ１０から出力される動画像のデータを所得して、個々に上述の処理を行う。 For example, FIG. 22 shows a life activity measurement system 111 according to a modification of the life activity measurement system 100. In the life activity measurement system 111, a plurality of cameras 10 are connected to the information processing apparatus 30 via the network 110. The information processing apparatus 30 obtains moving image data output from the plurality of cameras 10 and individually performs the above-described processing.
　生体活動計測システム１１１は、たとえば病院に敷設される。または生体活動計測システム１１１は、カメラ１０および光源２０は各患者の自宅に設置され、情報処理装置３０は病院等に設置されてもよい。 The life activity measurement system 111 is laid, for example, in a hospital. Alternatively, in the life activity measurement system 111, the camera 10 and the light source 20 may be installed in each patient's home, and the information processing apparatus 30 may be installed in a hospital or the like.
　本明細書は、以下の項目に記載の呼吸数の計測方法、計測システム、およびコンピュータプログラムを開示している。 This specification discloses a respiration rate measurement method, a measurement system, and a computer program described in the following items.
　［項目１］
　光を放射する光源と、
　前記光を受けて動画像を生成する撮像装置と、
　前記動画像を利用して被験体の生体活動を計測する画像処理回路と
　を備えた生体活動計測システムを用いて、前記被験体の呼吸に起因する生体活動を計測する計測方法であって、
（ａ）前記被験体の呼吸に伴う体動の発生位置に、所定の反射パターンを有する再帰性反射材を配置するステップと、
（ｂ）前記光源が、前記光で前記被験体を照射するステップと、
（ｃ）前記撮像装置が、前記再帰性反射材で反射された前記光を受けて、複数のフレーム画像から構成される動画像を生成するステップと、
（ｄ）前記画像処理回路が、前記複数のフレーム画像の変化に基づいて前記被験体の呼吸に起因する生体活動を計測するステップと
　を包含する、計測方法。 [Item 1]
A light source that emits light;
An imaging device that receives the light and generates a moving image;
A measurement method for measuring a biological activity resulting from respiration of the subject using a biological activity measurement system comprising an image processing circuit for measuring the biological activity of the subject using the moving image,
(A) disposing a retroreflecting material having a predetermined reflection pattern at a position where body movement accompanying breathing of the subject occurs;
(B) the light source irradiating the subject with the light;
(C) The imaging device receives the light reflected by the retroreflecting material and generates a moving image composed of a plurality of frame images;
(D) The image processing circuit includes a step of measuring a biological activity caused by respiration of the subject based on changes in the plurality of frame images.
　項目１の計測方法によると、光源と再帰性反射材とを組み合わせて使用するため、周囲が暗い場合でも明るい場合でも、かつ被験者と撮像装置との距離を十分離したとしても、呼吸等の生体活動に起因する体動を、複数のフレーム画像の変化から捉えることができる。撮像装置を被験体から離して設置し、かつ撮影環境を暗く設定できるため、被験体へ与える圧迫感を十分低減しつつ、十分高い精度で生体情報を取得することが可能である。 According to the measurement method of Item 1, since the light source and the retroreflecting material are used in combination, even if the surroundings are dark or bright, and the distance between the subject and the imaging device is sufficiently separated, a living body such as breathing The body movement caused by the activity can be grasped from the change of a plurality of frame images. Since the imaging device can be set apart from the subject and the imaging environment can be set to be dark, it is possible to acquire biological information with sufficiently high accuracy while sufficiently reducing the feeling of pressure applied to the subject.
　［項目２］
　前記ステップ（ｄ）において、前記画像処理回路は、前記複数のフレーム画像の輝度値の変化に基づいて前記生体活動を計測する、項目１に記載の計測方法。 [Item 2]
The measurement method according to item 1, wherein in the step (d), the image processing circuit measures the life activity based on a change in luminance values of the plurality of frame images.
　［項目３］
　前記ステップ（ｄ）において、前記画像処理回路は、各フレーム画像を複数の部分領域に分割し、前記複数の部分領域の輝度値の変化に基づいて前記生体活動を計測する、項目１または２に記載の計測方法。 [Item 3]
In the step (d), the image processing circuit divides each frame image into a plurality of partial areas, and measures the biological activity based on a change in luminance values of the plurality of partial areas. The measurement method described.
　［項目４］
　前記ステップ（ｄ）において、前記画像処理回路は、
　前記所定の反射パターンを利用して、各フレーム画像の部分領域であって、前記再帰性反射材が含まれ、かつ前記体動の発生位置を含む部分領域を特定し、
　特定した前記部分領域の輝度値の変化を利用して前記生体活動を計測する、項目１または２に記載の計測方法。 [Item 4]
In the step (d), the image processing circuit includes:
Using the predetermined reflection pattern, identify a partial region of each frame image, including the retroreflective material, and including the position where the body movement occurs,
Item 3. The measurement method according to Item 1 or 2, wherein the life activity is measured using a change in luminance value of the identified partial region.
　［項目５］
　前記ステップ（ｄ）において、前記画像処理回路は、前記所定の反射パターンを特定する情報を予め保持しており、
　前記画像処理回路は、前記情報を利用してパターンマッチング処理を行うことにより、各フレーム画像において前記所定の反射パターンを検出し、前記再帰性反射材が含まれる部分領域を特定する、項目１から４のいずれかに記載の計測方法。 [Item 5]
In the step (d), the image processing circuit holds in advance information specifying the predetermined reflection pattern,
From the item 1, the image processing circuit detects the predetermined reflection pattern in each frame image by performing a pattern matching process using the information, and identifies a partial region including the retroreflecting material. 4. The measuring method according to any one of 4 above.
　［項目６］
　前記ステップ（ｄ）において、前記画像処理回路は、前記各フレーム画像の、前記再帰性反射材の境界を含む部分領域を特定する、項目５に記載の計測方法。 [Item 6]
6. The measurement method according to item 5, wherein in the step (d), the image processing circuit specifies a partial region of each frame image including a boundary of the retroreflecting material.
　［項目７］
　前記ステップ（ｄ）において、前記画像処理回路は、前記体動の方向である第１の方向を特定し、前記第１の方向と異なる第２の方向に沿って、前記各フレーム画像を複数の部分領域に分割する、項目３に記載の計測方法。 [Item 7]
In the step (d), the image processing circuit specifies a first direction that is the direction of the body movement, and a plurality of the frame images are arranged along a second direction different from the first direction. Item 4. The measurement method according to Item 3, wherein the method is divided into partial areas.
　［項目８］
　前記再帰性反射材の前記所定の反射パターンは、前記撮像装置の方向へ、前記光を反射する第１部分と、前記光を反射しない第２部分とを含む、項目１から７のいずれかに記載の計測方法。 [Item 8]
The predetermined reflection pattern of the retroreflecting material includes any one of items 1 to 7, including a first portion that reflects the light and a second portion that does not reflect the light toward the imaging device. The measurement method described.
　［項目９］
　前記第２部分は、前記再帰性反射材の周囲に設けられている、項目８に記載の計測方法。 [Item 9]
The measurement method according to item 8, wherein the second portion is provided around the retroreflecting material.
　［項目１０］
　前記第１部分は、前記再帰性反射材の周囲に設けられている、項目８に記載の計測方法。 [Item 10]
The measurement method according to item 8, wherein the first portion is provided around the retroreflecting material.
　［項目１１］
　前記再帰性反射材の前記所定の反射パターンは、所定の模様を含む、項目８に記載の計測方法。 [Item 11]
Item 9. The measurement method according to Item 8, wherein the predetermined reflection pattern of the retroreflective material includes a predetermined pattern.
　［項目１２］
　前記再帰性反射材の前記所定の反射パターンは、前記光の反射率が異なる複数の部分を含む、項目１から７のいずれかに記載の計測方法。 [Item 12]
The measurement method according to any one of items 1 to 7, wherein the predetermined reflection pattern of the retroreflective material includes a plurality of portions having different reflectances of the light.
　［項目１３］
　前記再帰性反射材の前記所定の反射パターンは、前記光の反射率が連続的に遷移する部分を含む、項目１２に記載の計測方法。 [Item 13]
The measurement method according to item 12, wherein the predetermined reflection pattern of the retroreflective material includes a portion where the reflectance of the light continuously changes.
　［項目１４］
　（ｅ）光学フィルタを用いて、前記撮像装置に入射する可視光を遮断するステップをさらに包含する、項目１から１３のいずれかに記載の計測方法。 [Item 14]
(E) The measurement method according to any one of items 1 to 13, further including a step of blocking visible light incident on the imaging device using an optical filter.
　［項目１５］
　前記光の波長をλとしたときにおいて、
　（ｆ）前記波長λの光を主として通過させる光学フィルタに、前記再帰性反射材で反射された前記光を通過させるステップをさらに包含する、項目１から１３のいずれかに記載の計測方法。 [Item 15]
When the wavelength of the light is λ,
(F) The measurement method according to any one of items 1 to 13, further including a step of allowing the light reflected by the retroreflecting material to pass through an optical filter that mainly transmits the light having the wavelength λ.
　［項目１６］
　前記ステップ（ｂ）において、前記光源は、前記波長λとして８５０ｎｍまたは９４０ｎｍの光を放射する発光ダイオードである、項目１５に記載の計測方法。 [Item 16]
Item 16. The measurement method according to Item 15, wherein in the step (b), the light source is a light emitting diode that emits light of 850 nm or 940 nm as the wavelength λ.
　［項目１７］
　（ｇ）前記光源から放射された前記光を、所定の偏光方向を有する偏光素子に通過させるステップと、
　（ｈ）前記再帰性反射材で反射された前記光を、前記所定の偏光方向と同じ偏光方向を有する偏光素子に通過させるステップと
　をさらに包含する、項目１から１３のいずれかに記載の計測方法。 [Item 17]
(G) passing the light emitted from the light source through a polarizing element having a predetermined polarization direction;
(H) The measurement according to any one of items 1 to 13, further comprising: passing the light reflected by the retroreflecting material through a polarizing element having the same polarization direction as the predetermined polarization direction. Method.
　［項目１８］
　（ｉ）固定された位置に、予め定められた形状を有する再帰性反射材を位置マーカとしてさらに設けるステップと、
　（ｊ）前記撮像装置が、前記位置マーカで反射された前記光を受け、複数のフレーム画像から構成される動画像を生成するステップと、
　（ｋ）前記画像処理回路が、前記位置マーカで反射された前記複数のフレーム画像の少なくとも１つに基づいて前記撮像装置が予め定められた方向を撮影しているか否かを判定するステップと
　をさらに包含する、項目１に記載の計測方法。 [Item 18]
(I) further providing a retroreflecting material having a predetermined shape as a position marker at a fixed position;
(J) the imaging device receiving the light reflected by the position marker and generating a moving image composed of a plurality of frame images;
(K) the image processing circuit determining whether or not the imaging device is photographing a predetermined direction based on at least one of the plurality of frame images reflected by the position marker; The measurement method according to item 1, further including:
　［項目１９］
　（ｌ）前記ステップ（ｋ）において、前記画像処理回路が、前記撮像装置が予め定められた方向または位置を撮影していないと判定した場合には、警告を報知するステップをさらに包含する、項目１８に記載の計測方法。 [Item 19]
(L) In the step (k), the image processing circuit further includes a step of notifying a warning when it is determined that the imaging device has not photographed a predetermined direction or position. 18. The measuring method according to 18.
　［項目２０］
　前記再帰性反射材は非金属の材料によって形成されている、項目１から１９のいずれかに記載の計測方法。 [Item 20]
The measurement method according to any one of items 1 to 19, wherein the retroreflective material is formed of a non-metallic material.
　［項目２１］
　光を放射する光源と、
　前記光を受けて動画像を生成する撮像装置と、
　前記動画像を利用して被験体の生体活動を計測する画像処理回路と
　を備えた計測システムであって、
　前記被験体の呼吸に伴う体動の発生位置に再帰性反射材が配置され、前記光源から、前記被験体に向けて前記光が放射されたときにおいて、
　前記撮像装置は、前記再帰性反射材で反射された前記光を複数の時刻において受けて、時系列の複数のフレーム画像から構成される前記動画像を生成し、
　前記画像処理回路は、前記撮像装置から前記動画像を受け取り、前記複数のフレーム画像の輝度値の変化に基づいて前記被験体の呼吸に起因する生体活動を計測する、計測システム。 [Item 21]
A light source that emits light;
An imaging device that receives the light and generates a moving image;
An image processing circuit for measuring a subject's biological activity using the moving image, and a measurement system comprising:
When a retroreflecting material is disposed at a position where body movement occurs due to breathing of the subject and the light is emitted from the light source toward the subject,
The imaging device receives the light reflected by the retroreflecting material at a plurality of times, and generates the moving image composed of a plurality of time-series frame images,
The said image processing circuit is a measuring system which receives the said moving image from the said imaging device, and measures the biological activity resulting from the said subject's respiration based on the change of the luminance value of these frame images.
　項目２１の計測システムによれば、光源と再帰性反射材とを組み合わせて使用するため、周囲が暗く、かつ被験者と撮像装置との距離を十分離したとしても、呼吸等の生体活動に起因する体動を、複数のフレーム画像の変化から捉えることができる。撮像装置を被験体から離して設置し、かつ撮影環境を暗く設定できるため、被験体へ与える圧迫感を十分低減しつつ、十分高い精度で生体情報を取得することが可能である。 According to the measurement system of item 21, since the light source and the retroreflective material are used in combination, even if the surrounding is dark and the distance between the subject and the imaging device is sufficiently separated, it is caused by biological activity such as respiration. Body movement can be captured from changes in a plurality of frame images. Since the imaging device can be set apart from the subject and the imaging environment can be set to be dark, it is possible to acquire biological information with sufficiently high accuracy while sufficiently reducing the feeling of pressure applied to the subject.
　［項目２２］
　前記画像処理回路は、各フレーム画像の中から、前記再帰性反射材を含み、かつ前記体動の発生位置を含む部分領域を特定し、特定した前記部分領域の輝度値の変化を利用して前記生体活動を計測する、項目２１に記載の計測システム。 [Item 22]
The image processing circuit specifies a partial area including the retroreflecting material and including the position where the body motion is generated from each frame image, and uses a change in luminance value of the specified partial area. Item 22. The measurement system according to Item 21, wherein the biological activity is measured.
　［項目２３］
　前記各フレーム画像は、前記再帰性反射材で反射された前記光に対応する、輝度値が相対的に高い第１部分領域と、前記第１部分領域よりも輝度値が相対的に低い第２部分領域とを含み、
　前記画像処理回路は、前記複数のフレーム画像にわたって、前記第１部分領域と前記第２部分領域との境界が横切る座標位置の輝度値が経時的に変化することを利用して前記生体活動を計測する、項目２２に記載の計測システム。 [Item 23]
Each of the frame images includes a first partial region having a relatively high luminance value corresponding to the light reflected by the retroreflecting material, and a second lower luminance value than the first partial region. Including partial areas,
The image processing circuit measures the biological activity using a change in luminance value of a coordinate position crossing a boundary between the first partial region and the second partial region over time over the plurality of frame images. The measurement system according to Item 22,
　［項目２４］
　前記画像処理回路は、前記境界が横切る座標位置を含む部分領域であって、かつ前記第１部分領域と前記第２部分領域との割合が経時的に変化する座標位置を含む部分領域に関して、前記輝度値の変化を検出する、項目２３に記載の計測システム。 [Item 24]
The image processing circuit is a partial region including a coordinate position crossed by the boundary, and the partial region including a coordinate position where a ratio of the first partial region and the second partial region changes with time. 24. The measurement system according to item 23, which detects a change in luminance value.
　［項目２５］
　前記画像処理回路は、前記各フレーム画像の同じ座標位置に前記処理単位領域を設定する、項目２４に記載の計測システム。 [Item 25]
25. The measurement system according to item 24, wherein the image processing circuit sets the processing unit region at the same coordinate position of each frame image.
　［項目２６］
　前記部分領域のサイズは可変である、項目２５に記載の計測システム。 [Item 26]
26. The measurement system according to item 25, wherein the size of the partial area is variable.
　［項目２７］
　前記画像処理回路は、前記再帰性反射材を含む領域の位置が、予め定められた量以上移動したことを検出した場合には、再度、各フレーム画像の中から、前記再帰性反射材を含み、かつ前記体動の発生位置を含む新たな部分領域を特定し、特定した前記新たな部分領域の輝度値の変化を利用して前記生体活動を計測する、項目２２に記載の計測システム。 [Item 27]
When the image processing circuit detects that the position of the region including the retroreflecting material has moved by a predetermined amount or more, the image processing circuit again includes the retroreflecting material from each frame image. The measurement system according to item 22, wherein a new partial area including the occurrence position of the body motion is specified, and the life activity is measured using a change in luminance value of the specified new partial area.
　［項目２８］
　光を放射する光源と、
　前記光を受けて動画像を生成する撮像装置と、
　前記動画像を利用して被験体の生体活動を計測する画像処理回路と
　を備えた計測システムにおける画像処理回路によって実行されるコンピュータプログラムであって、
　前記被験体の呼吸に伴う体動の発生位置に再帰性反射材が配置され、前記光源から、前記被験体に向けて前記光が放射されたときにおいて、
　前記コンピュータプログラムは前記画像処理回路に、
　前記撮像装置によって生成された動画像を受け取るステップであって、前記再帰性反射材で反射された前記光を複数の時刻において受けて時系列の複数のフレーム画像から構成される前記動画像を受け取るステップと、
　前記複数のフレーム画像の輝度値の変化に基づいて前記被験体の呼吸に起因する生体活動を計測するステップと
　を実行させる、コンピュータプログラム。 [Item 28]
A light source that emits light;
An imaging device that receives the light and generates a moving image;
A computer program executed by an image processing circuit in a measurement system comprising: an image processing circuit that measures a subject's biological activity using the moving image;
When a retroreflecting material is disposed at a position where body movement occurs due to breathing of the subject and the light is emitted from the light source toward the subject,
The computer program is stored in the image processing circuit.
Receiving the moving image generated by the imaging device, receiving the light reflected by the retroreflecting material at a plurality of times and receiving the moving image composed of a plurality of time-series frame images; Steps,
A step of measuring a biological activity caused by respiration of the subject based on a change in luminance values of the plurality of frame images.
　本発明は、被験体を撮影した動画像を解析して、被験体の生体活動、特に呼吸の数を非接触で計測する方法として利用することができる。また本発明は、そのような動画像の解析および生体活動の計測のための装置、システム、コンピュータプログラムとして利用することができる。 The present invention can be used as a method for analyzing a moving image obtained by photographing a subject and measuring the life activity of the subject, particularly the number of breaths, in a non-contact manner. The present invention can also be used as an apparatus, system, and computer program for analyzing such moving images and measuring life activity.
　１　被験者
　１０　カメラ
　２０　光源
　３０　情報処理装置
　３２　ディスプレイ
　４０　再帰性反射材
　３０１　ＣＰＵ
　３０２　ＲＯＭ
　３０３　ＲＡＭ
　３０４　ＨＤＤ
　３０５　Ｉ／Ｆ
　３０６　画像処理回路
　１００、１０１、１１１　生体活動計測システム DESCRIPTION OF SYMBOLS 1 Test subject 10 Camera 20 Light source 30 Information processing apparatus 32 Display 40 Retroreflective material 301 CPU
302 ROM
303 RAM
304 HDD
305 I / F
306 Image processing circuit 100, 101, 111 Life activity measurement system

Claims (22)

  1. 　光を放射する光源と、
    　前記光を受けて動画像を生成する撮像装置と、
    　前記動画像を利用して被験体の生体活動を計測する画像処理回路と
    　を備えた生体活動計測システムを用いて、前記被験体の呼吸に起因する生体活動を計測する方法であって、
    （ａ）前記被験体の呼吸に伴う体動の発生位置に、所定の反射パターンを有する再帰性反射材を配置するステップと、
    （ｂ）前記光源が、前記光で前記被験体を照射するステップと、
    （ｃ）前記撮像装置が、前記再帰性反射材で反射された前記光を複数の時刻において受けて、時系列の複数のフレーム画像から構成される動画像を生成するステップと、
    （ｄ）前記画像処理回路が、前記複数のフレーム画像の輝度値の変化に基づいて前記被験体の呼吸に起因する生体活動を計測するステップと
    　を包含する、呼吸に起因する生体活動の計測方法。     A light source that emits light;
    An imaging device that receives the light and generates a moving image;
    A method for measuring a biological activity resulting from respiration of the subject using a biological activity measurement system comprising an image processing circuit for measuring the biological activity of the subject using the moving image,
    (A) disposing a retroreflecting material having a predetermined reflection pattern at a position where body movement accompanying breathing of the subject occurs;
    (B) the light source irradiating the subject with the light;
    (C) the imaging device receiving the light reflected by the retroreflecting material at a plurality of times and generating a moving image composed of a plurality of time-series frame images;
    (D) The method for measuring a biological activity caused by respiration, wherein the image processing circuit includes a step of measuring a biological activity caused by respiration of the subject based on a change in luminance values of the plurality of frame images. .
  2. 　前記ステップ（ｄ）において、前記画像処理回路は、各フレーム画像を複数の部分領域に分割し、前記複数の部分領域の輝度値の変化に基づいて前記生体活動を計測する、請求項１に記載の計測方法。 The said image processing circuit divides | segments each frame image into several partial area | region in the said step (d), The said biological activity is measured based on the change of the luminance value of these partial area | regions. Measurement method.
  3. 　前記ステップ（ｄ）において、前記画像処理回路は、
    　前記所定の反射パターンを利用して、各フレーム画像の部分領域であって、前記再帰性反射材が含まれ、かつ前記体動の発生位置を含む部分領域を特定し、
    　特定した前記部分領域の輝度値の変化を利用して前記生体活動を計測する、請求項１に記載の計測方法。     In the step (d), the image processing circuit includes:
    Using the predetermined reflection pattern, identify a partial region of each frame image, including the retroreflective material, and including the position where the body movement occurs,
    The measurement method according to claim 1, wherein the life activity is measured using a change in luminance value of the identified partial region.
  4. 　前記ステップ（ｄ）において、前記画像処理回路は、前記所定の反射パターンを特定する情報を予め保持しており、
    　前記画像処理回路は、前記情報を利用してパターンマッチング処理を行うことにより、各フレーム画像において前記所定の反射パターンを検出し、前記再帰性反射材が含まれる部分領域を特定する、請求項１から３のいずれかに記載の計測方法。     In the step (d), the image processing circuit holds in advance information specifying the predetermined reflection pattern,
    The image processing circuit detects a predetermined reflection pattern in each frame image by performing a pattern matching process using the information, and identifies a partial region including the retroreflecting material. To 4. The measuring method according to any one of 3.
  5. 　前記ステップ（ｄ）において、前記画像処理回路は、前記各フレーム画像の、前記再帰性反射材の境界を含む部分領域を特定する、請求項４に記載の計測方法。 The measurement method according to claim 4, wherein in the step (d), the image processing circuit specifies a partial region of each frame image including a boundary of the retroreflecting material.
  6. 　前記ステップ（ｄ）において、前記画像処理回路は、前記体動の方向である第１の方向を特定し、前記第１の方向と異なる第２の方向に沿って、前記各フレーム画像を複数の部分領域に分割する、請求項２に記載の計測方法。 In the step (d), the image processing circuit specifies a first direction that is the direction of the body movement, and a plurality of the frame images are arranged along a second direction different from the first direction. The measuring method according to claim 2, wherein the measuring method is divided into partial areas.
  7. 　前記再帰性反射材の前記所定の反射パターンは、前記撮像装置の方向へ、前記光を反射する第１部分と、前記光を反射しない第２部分とを含む、請求項１から６のいずれかに記載の計測方法。 The predetermined reflection pattern of the retroreflective material includes a first portion that reflects the light and a second portion that does not reflect the light in the direction of the imaging device. Measurement method described in 1.
  8. 　前記第２部分は、前記再帰性反射材の周囲に設けられている、請求項７に記載の計測方法。 The measurement method according to claim 7, wherein the second part is provided around the retroreflecting material.
  9. 　前記再帰性反射材の前記所定の反射パターンは、前記光の反射率が異なる複数の部分を含む、請求項１から６のいずれかに記載の計測方法。 The measurement method according to any one of claims 1 to 6, wherein the predetermined reflection pattern of the retroreflecting material includes a plurality of portions having different reflectances of the light.
  10. 　（ｅ）光学フィルタを用いて、前記撮像装置に入射する可視光を遮断するステップをさらに包含する、請求項１から９のいずれかに記載の計測方法。 (E) The measurement method according to any one of claims 1 to 9, further comprising a step of blocking visible light incident on the imaging device using an optical filter.
  11. 　前記光の波長をλとしたときにおいて、
    　（ｆ）前記波長λの光を主として通過させる光学フィルタに、前記再帰性反射材で反射された前記光を通過させるステップをさらに包含する、請求項１から１０のいずれかに記載の計測方法。     When the wavelength of the light is λ,
    The measurement method according to claim 1, further comprising: (f) passing the light reflected by the retroreflecting material through an optical filter that mainly transmits the light having the wavelength λ.
  12. 　（ｇ）前記光源から放射された前記光を、所定の偏光方向を有する偏光素子に通過させるステップと、
    　（ｈ）前記再帰性反射材で反射された前記光を、前記所定の偏光方向と同じ偏光方向を有する偏光素子に通過させるステップと
    　をさらに包含する、請求項１から１１のいずれかに記載の計測方法。     (G) passing the light emitted from the light source through a polarizing element having a predetermined polarization direction;
    The step (h) further comprising: passing the light reflected by the retroreflecting material through a polarizing element having the same polarization direction as the predetermined polarization direction. Measurement method.
  13. 　（ｉ）固定された位置に、予め定められた形状を有する再帰性反射材を位置マーカとしてさらに設けるステップと、
    　（ｊ）前記撮像装置が、前記位置マーカで反射された前記光を受け、複数のフレーム画像から構成される動画像を生成するステップと、
    　（ｋ）前記画像処理回路が、前記位置マーカで反射された前記複数のフレーム画像の少なくとも１つに基づいて前記撮像装置が予め定められた方向を撮影しているか否かを判定するステップと
    　をさらに包含する、請求項１から１２のいずれかに記載の計測方法。     (I) further providing a retroreflecting material having a predetermined shape as a position marker at a fixed position;
    (J) the imaging device receiving the light reflected by the position marker and generating a moving image composed of a plurality of frame images;
    (K) the image processing circuit determining whether or not the imaging device is photographing a predetermined direction based on at least one of the plurality of frame images reflected by the position marker; The measurement method according to claim 1, further comprising:
  14. 　前記再帰性反射材は非金属の材料によって形成されている、請求項１から１３のいずれかに記載の計測方法。 The measurement method according to any one of claims 1 to 13, wherein the retroreflecting material is formed of a non-metallic material.
  15. 　光を放射する光源と、
    　前記光を受けて動画像を生成する撮像装置と、
    　前記動画像を利用して被験体の生体活動を計測する画像処理回路と
    　を備えた計測システムであって、
    　前記被験体の呼吸に伴う体動の発生位置に所定の反射パターンを有する再帰性反射材が配置され、前記光源から、前記被験体に向けて前記光が放射されたときにおいて、
    　前記撮像装置は、前記再帰性反射材で反射された前記光を複数の時刻において受けて、時系列の複数のフレーム画像から構成される前記動画像を生成し、
    　前記画像処理回路は、前記撮像装置から前記動画像を受け取り、前記複数のフレーム画像の輝度値の変化に基づいて前記被験体の呼吸に起因する生体活動を計測する、計測システム。     A light source that emits light;
    An imaging device that receives the light and generates a moving image;
    An image processing circuit for measuring a subject's biological activity using the moving image, and a measurement system comprising:
    When a retroreflecting material having a predetermined reflection pattern is arranged at a position where a body movement accompanying breathing of the subject is arranged, and when the light is emitted from the light source toward the subject,
    The imaging device receives the light reflected by the retroreflecting material at a plurality of times, and generates the moving image composed of a plurality of time-series frame images,
    The said image processing circuit is a measuring system which receives the said moving image from the said imaging device, and measures the biological activity resulting from the said subject's respiration based on the change of the luminance value of these frame images.
  16. 　前記画像処理回路は、各フレーム画像の中から、前記再帰性反射材を含み、かつ前記体動の発生位置を含む部分領域を特定し、特定した前記部分領域の輝度値の変化を利用して前記生体活動を計測する、請求項１５に記載の計測システム。 The image processing circuit specifies a partial area including the retroreflecting material and including the position where the body motion is generated from each frame image, and uses a change in luminance value of the specified partial area. The measurement system according to claim 15, wherein the biological activity is measured.
  17. 　前記各フレーム画像は、前記再帰性反射材で反射された前記光に対応する、輝度値が相対的に高い第１部分領域と、前記第１部分領域よりも輝度値が相対的に低い第２部分領域とを含み、
    　前記画像処理回路は、前記複数のフレーム画像にわたって、前記第１部分領域と前記第２部分領域との境界が横切る座標位置の輝度値が経時的に変化することを利用して前記生体活動を計測する、請求項１６に記載の計測システム。     Each of the frame images includes a first partial region having a relatively high luminance value corresponding to the light reflected by the retroreflecting material, and a second lower luminance value than the first partial region. Including partial areas,
    The image processing circuit measures the biological activity using a change in luminance value of a coordinate position crossing a boundary between the first partial region and the second partial region over time over the plurality of frame images. The measurement system according to claim 16.
  18. 　前記画像処理回路は、前記境界が横切る座標位置を含む部分領域であって、かつ前記第１部分領域と前記第２部分領域との割合が経時的に変化する座標位置を含む部分領域に関して、前記輝度値の変化を検出する、請求項１７に記載の計測システム。 The image processing circuit is a partial region including a coordinate position crossed by the boundary, and the partial region including a coordinate position where a ratio of the first partial region and the second partial region changes with time. The measurement system according to claim 17, wherein a change in luminance value is detected.
  19. 　前記画像処理回路は、前記各フレーム画像の同じ座標位置に前記処理単位領域を設定する、請求項１８に記載の計測システム。 The measurement system according to claim 18, wherein the image processing circuit sets the processing unit area at the same coordinate position of each frame image.
  20. 　前記部分領域のサイズは可変である、請求項１９に記載の計測システム。 The measurement system according to claim 19, wherein the size of the partial area is variable.
  21. 　前記画像処理回路は、前記再帰性反射材を含む領域の位置が、予め定められた量以上移動したことを検出した場合には、再度、各フレーム画像の中から、前記再帰性反射材を含み、かつ前記体動の発生位置を含む新たな部分領域を特定し、特定した前記新たな部分領域の輝度値の変化を利用して前記生体活動を計測する、請求項１６に記載の計測システム。 When the image processing circuit detects that the position of the region including the retroreflecting material has moved by a predetermined amount or more, the image processing circuit again includes the retroreflecting material from each frame image. The measurement system according to claim 16, wherein a new partial area including the generation position of the body movement is specified, and the biological activity is measured using a change in luminance value of the specified new partial area.
  22. 　光を放射する光源と、
    　前記光を受けて動画像を生成する撮像装置と、
    　前記動画像を利用して被験体の生体活動を計測する画像処理回路と
    　を備えた計測システムにおける画像処理回路によって実行されるコンピュータプログラムであって、
    　前記被験体の呼吸に伴う体動の発生位置に所定の反射パターンを有する再帰性反射材が配置され、前記光源から、前記被験体に向けて前記光が放射されたときにおいて、
    　前記コンピュータプログラムは前記画像処理回路に、
    　前記撮像装置によって生成された動画像を受け取るステップであって、前記再帰性反射材で反射された前記光を複数の時刻において受けて時系列の複数のフレーム画像から構成される前記動画像を受け取るステップと、
    　前記複数のフレーム画像の輝度値の変化に基づいて前記被験体の呼吸に起因する生体活動を計測するステップと
    　を実行させる、コンピュータプログラム。     A light source that emits light;
    An imaging device that receives the light and generates a moving image;
    A computer program executed by an image processing circuit in a measurement system comprising: an image processing circuit that measures a subject's biological activity using the moving image;
    When a retroreflecting material having a predetermined reflection pattern is arranged at a position where a body movement accompanying breathing of the subject is arranged, and when the light is emitted from the light source toward the subject,
    The computer program is stored in the image processing circuit.
    Receiving the moving image generated by the imaging device, receiving the light reflected by the retroreflecting material at a plurality of times and receiving the moving image composed of a plurality of time-series frame images; Steps,
    A step of measuring a biological activity caused by respiration of the subject based on a change in luminance values of the plurality of frame images.
PCT/JP2015/069550 2014-07-15 2015-07-07 Method for measuring biological activity resulting from respiration of subject, measurement system, and computer program WO2016009901A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2016534383A JP6280650B2 (en) 2014-07-15 2015-07-07 Measuring method and measuring system of life activity caused by breathing of subject

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
JP2014-145475 2014-07-15
JP2014145475 2014-07-15
JP2014-145476 2014-07-15
JP2014145476 2014-07-15

Publications (1)

Publication Number Publication Date
WO2016009901A1 true WO2016009901A1 (en) 2016-01-21

Family

ID=55078397

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2015/069550 WO2016009901A1 (en) 2014-07-15 2015-07-07 Method for measuring biological activity resulting from respiration of subject, measurement system, and computer program

Country Status (2)

Country Link
JP (1) JP6280650B2 (en)
WO (1) WO2016009901A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016186172A1 (en) * 2015-05-20 2016-11-24 シャープ株式会社 Measuring system, method of measuring biological activity caused by breathing of subject, and computer program

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016076009A1 (en) * 2014-11-10 2016-05-19 シャープ株式会社 System for measuring biological activity due to breathing of subject and image processing device

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012120648A (en) * 2010-12-07 2012-06-28 Alpha Co Posture detection apparatus
WO2013186696A1 (en) * 2012-06-12 2013-12-19 Koninklijke Philips N.V. System for camera-based vital sign measurement
JP2014512905A (en) * 2011-03-30 2014-05-29 コーニンクレッカ フィリップス エヌ ヴェ Non-contact sleep disorder screening system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012120648A (en) * 2010-12-07 2012-06-28 Alpha Co Posture detection apparatus
JP2014512905A (en) * 2011-03-30 2014-05-29 コーニンクレッカ フィリップス エヌ ヴェ Non-contact sleep disorder screening system
WO2013186696A1 (en) * 2012-06-12 2013-12-19 Koninklijke Philips N.V. System for camera-based vital sign measurement

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016186172A1 (en) * 2015-05-20 2016-11-24 シャープ株式会社 Measuring system, method of measuring biological activity caused by breathing of subject, and computer program

Also Published As

Publication number Publication date
JP6280650B2 (en) 2018-02-14
JPWO2016009901A1 (en) 2017-04-27

Similar Documents

Publication Publication Date Title
JP6863408B2 (en) Information processing equipment, information processing methods and programs
RU2616175C2 (en) Object distance determination by image
CN105142516B (en) Device and method for determining the vital sign information from object
JPWO2009019886A1 (en) Normal information generating apparatus and normal information generating method
US10552675B2 (en) Method and apparatus for eye detection from glints
JP2020183965A (en) System and method for specular reflection detection and reduction
CN109564382A (en) Filming apparatus and image pickup method
US11666247B2 (en) Method, device and computer program for capturing optical image data of patient surroundings and for identifying a patient check-up
JP7122157B2 (en) Rockfall detector and rockfall detection system
JP6280650B2 (en) Measuring method and measuring system of life activity caused by breathing of subject
WO2017047734A1 (en) Measurement device
JP2019000474A (en) Pulse period detection device, pulse period detection method, pulse period detection program, and pulse wave detection device
JP4587304B2 (en) Image processing device
JP2016214937A (en) Measuring system and computer program
JP2012115505A (en) Visual line detection device and visual line detection method
JP6185090B2 (en) Measuring system
JP4729325B2 (en) Image processing device
JP6002811B1 (en) Measuring system, measuring method of biological activity resulting from breathing of subject, and computer program
JP2019021189A (en) Object detection device
JP6244960B2 (en) Object recognition apparatus, object recognition method, and object recognition program
JP2013251826A (en) Mobile communication device and imaging system
WO2016076009A1 (en) System for measuring biological activity due to breathing of subject and image processing device
CN106778617A (en) A kind of sleep monitor method and system based on acquisition technology
JP2006285699A (en) Image-processing device
CN103198290A (en) Method for detecting number, positions and moving of human bodies through video

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15821998

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2016534383

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15821998

Country of ref document: EP

Kind code of ref document: A1