WO2016098308A1 - 撮像装置、画像処理装置及び画像処理方法 - Google Patents

撮像装置、画像処理装置及び画像処理方法 Download PDF

Info

Publication number
WO2016098308A1
WO2016098308A1 PCT/JP2015/006062 JP2015006062W WO2016098308A1 WO 2016098308 A1 WO2016098308 A1 WO 2016098308A1 JP 2015006062 W JP2015006062 W JP 2015006062W WO 2016098308 A1 WO2016098308 A1 WO 2016098308A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
unit
image data
frame
specific color
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2015/006062
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
亮司 荻野
中村　剛
純哉 桑田
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Intellectual Property Management Co Ltd
Original Assignee
Panasonic Intellectual Property Management Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panasonic Intellectual Property Management Co Ltd filed Critical Panasonic Intellectual Property Management Co Ltd
Priority to US15/529,672 priority Critical patent/US10492678B2/en
Publication of WO2016098308A1 publication Critical patent/WO2016098308A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/02—Detecting, measuring or recording for evaluating the cardiovascular system, e.g. pulse, heart rate, blood pressure or blood flow
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/041—Capsule endoscopes for imaging
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/313—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor for introducing through surgical openings, e.g. laparoscopes
    • A61B1/3137—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor for introducing through surgical openings, e.g. laparoscopes for examination of the interior of blood vessels
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
    • A61B3/102—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for optical coherence tomography [OCT]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G06V40/162—Detection; Localisation; Normalisation using pixel segmentation or colour matching
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/40—Spoof detection, e.g. liveness detection
    • G06V40/45—Detection of the body part being alive
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/15—Biometric patterns based on physiological signals, e.g. heartbeat, blood flow

Definitions

  • the present disclosure relates to an imaging apparatus, an image processing apparatus, and an image processing method that perform image processing on an image.
  • Vital sensing technology that estimates human biological information is not limited to the field of home medical care and health management, but includes a variety of methods such as detection of sleepiness while driving, acquisition of a user's psychological state during a game, and detection of abnormal persons in a monitoring system. Application to a wide range of fields is expected. Currently, a device that senses biological information is mainly used in contact with a human body and needs to be worn by the user, so that the application range is limited.
  • a technique for estimating a pulse as an example of biological information from an image obtained by imaging with a camera has been proposed.
  • this technology it is possible to sense biological information without making the user aware of it, and the application range can be expected to be expanded. For example, it is possible to detect a suspicious person whose pulse fluctuation has increased due to stress by performing image processing on an image obtained by imaging while photographing with a surveillance camera. It is also a great merit that a plurality of people appearing on one camera can be sensed simultaneously by image processing. Compared to the contact type, it is not necessary to prepare an individual device for each user, and the troublesomeness of attaching to the body can be reduced.
  • a pulse measuring device shown in Patent Document 1 calculates a feature amount of a captured input image, detects a pulse wave peak interval from the calculated feature amount, and calculates a pulse rate from the detected pulse wave peak interval.
  • the pulse measurement device has an estimated maximum error between a pulse rate having an effective peak interval based on an adoption rate indicating a ratio of an effective peak interval and a true pulse rate within a calculated value.
  • the frame rate indicating the number of frames captured per unit time is controlled so that
  • This disclosure is intended to effectively suppress the distribution of biological information unintended by the creator and distributor of video content, and to accurately protect the privacy of persons appearing in the video content.
  • An image processing apparatus includes an image input unit that inputs image data obtained by capturing a person, an area detection unit that detects an area of a specific color of the image data, and a frame of image data input to the image input unit. Using each frame of image data before and after input to the image input unit, a filter unit for smoothing pixel values in a specific color region detected by the region detection unit, and an output image of the filter unit are encoded An encoding unit; and an output unit that outputs an output image of the filter unit encoded by the encoding unit.
  • an imaging apparatus includes an imaging unit that captures a person as a subject, an area detection unit that detects an area of a specific color of image data including the person captured by the imaging unit, and an image captured by the imaging unit.
  • an imaging unit that captures a person as a subject
  • an area detection unit that detects an area of a specific color of image data including the person captured by the imaging unit
  • an image captured by the imaging unit Using the frame of data and the frame of each image data before and after imaging in the imaging unit, a filter unit for smoothing the pixel value in the specific color region detected by the region detection unit, and an output image of the filter unit are encoded
  • an output unit that outputs an output image of the filter unit encoded by the encoding unit.
  • the image processing method of the present disclosure is an image processing method in an image processing apparatus, and includes a step of inputting image data obtained by imaging a person, a step of detecting a specific color area of the image data, Using the image data frame and the image data frames before and after the input, the step of smoothing the pixel value in the detected specific color area, and the output image in which the pixel value in the specific color area is smoothed And a step of outputting a coded output image.
  • FIG. 1A is a diagram schematically illustrating an example of the relationship between the contraction of a human heart and the amount of light absorbed in a blood vessel.
  • FIG. 1B is a diagram illustrating an example of a time-series change in light intensity.
  • FIG. 2 is a diagram illustrating an example of an absorptance for each wavelength of light in hemoglobin.
  • FIG. 3 is a block diagram illustrating an example of an internal configuration of the image processing apparatus according to the present embodiment.
  • FIG. 4 is a block diagram illustrating an example of an internal configuration of the imaging apparatus according to the present embodiment.
  • FIG. 5A is an explanatory diagram relating to detection of a human skin color region.
  • FIG. 5B is an explanatory diagram relating to detection of a human skin color region.
  • FIG. 5A is an explanatory diagram relating to detection of a human skin color region.
  • FIG. 6 is an explanatory diagram showing an outline of the operation of the photoelectric pulse wave fluctuation removal filter.
  • FIG. 7 is an explanatory diagram illustrating an operation example of the photoelectric pulse wave fluctuation removal filter when the subject person is stationary.
  • FIG. 8 is an explanatory diagram showing an operation example of the photoelectric pulse wave fluctuation removal filter when the subject person is moving.
  • FIG. 9 is a flowchart illustrating an example of an operation procedure of the image processing apparatus according to the present embodiment.
  • the imaging apparatus of the present embodiment is, for example, a DSC (Digital Still Camera), a digital camcorder, a smartphone, a mobile phone, a tablet terminal, or a surveillance camera having a camera function.
  • the image processing apparatus according to the present embodiment is, for example, an electronic device (for example, a desktop or laptop PC) in which an encoder application is installed, an encoder device as a dedicated device, and a recorder.
  • the image processing apparatus inputs image data obtained by capturing a person from the outside (for example, an external apparatus connected to the image processing apparatus), and detects a specific color area of the input image data.
  • the pixel values in a specific color (for example, skin color) region in the image data frame are smoothed using the input image data frame and the respective image data frames before and after the input time point.
  • the image processing apparatus encodes the output image after the pixel values are smoothed, and outputs the encoded output image to the outside (for example, an external apparatus connected to a network or a display).
  • FIG. 1A is a diagram schematically illustrating an example of the relationship between the contraction of a human heart and the amount of light absorbed in a blood vessel.
  • FIG. 1B is a diagram illustrating an example of a time-series change in light intensity.
  • FIG. 1A shows that the volume of a blood vessel changes in synchronization with the systole of a human heart.
  • the amount of absorption of light for example, light in a specific wavelength region shown in FIG. 2
  • the intensity of light also decreases (see FIG. 1B).
  • the pulse wave indicates a wave motion when the pressure change in the blood vessel generated when blood is pushed out to the aorta due to the contraction of the heart is transmitted in the peripheral direction.
  • the horizontal axis indicates time
  • the vertical axis indicates the intensity of a signal (photoelectric pulse wave) obtained by a change in the amount of absorbed light. That is, in FIG. 1B, when the peak appears, the amount of light absorption is small, so the volume of the blood vessel is not increased, and when the minimum value appears, the amount of light absorption is large, so The volume is increasing. It should be noted that although there is a slight delay due to the distance between the heart and the peripheral portion, a slight delay is seen, but the heart contraction and the change in the intensity of the photoelectric pulse wave basically fluctuate in synchronization.
  • FIG. 2 is a diagram showing an example of the absorption rate for each wavelength of light in hemoglobin.
  • FIG. 2 shows that, for example, hemoglobin (blood) easily absorbs a wavelength of 400 nm (that is, green).
  • the image processing apparatus in order to accurately protect the privacy of a captured person (for example, a TV program performer), the image processing apparatus includes a single frame constituting the image data obtained by the imaging.
  • the smoothing of pixel values in a specific color (for example, skin color) region will be described.
  • the specific color is not limited to the skin color.
  • the image processing apparatus may smooth the pixel value in the red region (wavelength exceeding 1000 nm shown in FIG. 2).
  • FIG. 3 is a block diagram illustrating an example of an internal configuration of the image processing apparatus 1 according to the present embodiment.
  • FIG. 4 is a block diagram illustrating an example of an internal configuration of the imaging apparatus 1A according to the present embodiment. 4, the same reference numerals are given to the same components as those of the image processing apparatus 1 shown in FIG. 3, and the description thereof is simplified or omitted.
  • the image processing apparatus 1 shown in FIG. 3 includes an image input unit 11, an image storage unit 13, a skin color region detection unit 15, a photoelectric pulse wave fluctuation removal filter 17, an image encoding unit 19, and an image output control unit 21. And a recording medium 23 and an image output unit 25.
  • An imaging apparatus 1A illustrated in FIG. 4 includes an imaging unit 27, an image storage unit 13A, a signal processing unit 29, a skin color region detection unit 15, a photoelectric pulse wave fluctuation removal filter 17, an image encoding unit 19, and an image.
  • the configuration includes an output control unit 21, a recording medium 23, and an image output unit 25.
  • the image input unit 11 continuously inputs (acquires) frames of image data in which a person (for example, a performer of a television program) is captured by an external device (not illustrated) (for example, a camera that captures images at a predetermined frame rate) from the external device. And stored in the image storage unit 13.
  • a person for example, a performer of a television program
  • an external device for example, a camera that captures images at a predetermined frame rate
  • the image storage unit 13 as an example of the storage unit is configured by using a semiconductor memory such as a DRAM (Dynamic Random Access Memory) or a hard disk, for example, and the image data input by the image input unit 11 and the photoelectric pulse wave fluctuation removal filter 17. Save the output image data.
  • a semiconductor memory such as a DRAM (Dynamic Random Access Memory) or a hard disk, for example, and the image data input by the image input unit 11 and the photoelectric pulse wave fluctuation removal filter 17. Save the output image data.
  • the skin color region detection unit 15 as an example of the region detection unit reads the image data stored in the image storage unit 13 by the image input unit 11, and the specific color (for example, skin color) region (for example, FIG. 5A and FIG. The face area FC and hand area HD of the person HM shown in FIG. 5B are detected.
  • the skin color area detection unit 15 outputs information (for example, coordinates) regarding the specific color area detected in the frame of the image data to the photoelectric pulse wave fluctuation removal filter 17.
  • a method for detecting a specific color (for example, skin color) region in the skin color region detection unit 15 will be described later with reference to FIGS. 5A and 5B.
  • the photoelectric pulse wave fluctuation removal filter 17 as an example of the filter unit reads image data stored in the image storage unit 13 by the image input unit 11 for each partial region, and corresponds to a frame of image data input by the image input unit 11. Using the pixel value of the partial area and the pixel value of the corresponding partial area of each image data frame before and after input to the image input unit 11, the image data of the specific color area detected by the skin color area detection unit 15 Are smoothed (see FIG. 6). The pixel value is indicated by luminance (Y) and color difference (U, V).
  • the photoelectric pulse wave fluctuation removing filter 17 stores the image data after smoothing the pixel values of the image data of the specific color area in the image storage unit 13. A smoothing method in the photoelectric pulse wave fluctuation removal filter 17 will be described later with reference to FIGS.
  • An image encoding unit 19 as an example of an encoding unit is image data of the output image of the photoelectric pulse wave fluctuation removal filter 17 stored in the image storage unit 13 (that is, smoothed by the photoelectric pulse wave fluctuation removal filter 17). Using the image data having the pixel values of the specific color area, encoded data for conversion into a predetermined data format capable of storing and transmitting the image data is generated. The image encoding unit 19 outputs the encoded data of the image data to the image output control unit 21.
  • the image output control unit 21 stores the image data encoded by the image encoding unit 19 in the recording medium 23 or outputs it to the image output unit 25.
  • the recording medium 23 as an example of a recording unit is configured by using a semiconductor memory such as a DRAM or a hard disk, for example, and records encoded data of the image data generated by the image encoding unit 19.
  • the recording medium 23 is not limited to a semiconductor memory or a hard disk built in the image processing apparatus 1 or the imaging apparatus 1A.
  • an external connection medium for example, a DRAM or the like
  • connectable via a USB (Universal Serial Bus) terminal is used.
  • Semiconductor memory Semiconductor memory
  • the image output unit 25 uses the encoded data of the image data generated by the image encoding unit 19 in response to an instruction from the image output control unit 21, for example, an external device (transmission destination). Packet generation processing for transmission to an unillustrated) is performed, and a packet of encoded data of image data is transmitted to an external device. Thereby, the image output unit 25 can transmit image data (in other words, image data in which it is difficult to analyze the pulse rate of the person appearing in the image data) in which the pixel value of the specific color region is smoothed to the external device.
  • the network is a wireless network or a wired network.
  • the wireless network is, for example, NFC (Near Field Communication), Bluetooth (registered trademark), IrDA, wireless LAN (Local Area Network), 3G, LTE (Long Term Term Evolution), or WiGig.
  • the wired network is, for example, an intranet or the Internet.
  • the image output unit 25 also outputs image data of the output image of the photoelectric pulse wave fluctuation removal filter 17 stored in the image storage unit 13 (that is, the photoelectric pulse wave fluctuation removal filter 17 in accordance with an instruction from the image output control unit 21.
  • the image data having the pixel value of the specific color area smoothed by the above is displayed on a display (not shown), for example.
  • the imaging unit 27 includes at least a lens and an image sensor for imaging a person (for example, a performer of a television program) as a subject.
  • the lens collects ambient light incident from the outside of the imaging apparatus 1A and forms an image on a predetermined imaging surface of an image sensor (not shown).
  • the image sensor is configured using, for example, a CCD (Charged-Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) solid-state imaging device, and converts an optical image formed on the imaging surface into an electrical signal.
  • the output of the image sensor is input and stored in the image storage unit 13A.
  • the image storage unit 13A as an example of a storage unit is configured using a semiconductor memory such as a DRAM or a hard disk, for example, and stores the output of the image sensor of the imaging unit 27 and the image data of the output image of the photoelectric pulse wave fluctuation removal filter 17. To do.
  • a semiconductor memory such as a DRAM or a hard disk
  • the signal processing unit 29 uses the output of the image sensor stored in the image storage unit 13A to recognize image data in RGB (Red Green Blue) format or YUV (luminance (Y) / color difference (U, V). A frame of the image data defined by ()) is generated, and the generated frame is stored in the image storage unit 13A.
  • RGB Red Green Blue
  • YUV luminance (Y) / color difference (U, V)
  • FIGS. 5A and 5B are explanatory diagrams regarding detection of a human skin color region.
  • 5A and 5B for example, two types of methods will be described as a method by which the skin color region detection unit 15 detects a region of a specific color (for example, skin color) in a frame of image data.
  • the skin color area detection unit 15 includes colors in which pixel values (for example, color differences) in a frame of image data are included in a range (for example, a predetermined range between 0 and 60) designated in advance in the HSV color space. Is determined to be skin tone. Accordingly, the skin color area detection unit 15 detects the face area FC and the hand area HD as skin color areas for the image data in which the person HM shown in FIG. 5A is shown.
  • the skin color area detection unit 15 performs image processing (for example, face detection processing) based on the luminance signal on the frame of the image data in which the person HM shown in FIG. And the color of an area that is the same as the detected face area FC or has a color difference within a predetermined range from the color difference of the face area FC is determined to be a skin color. Therefore, the skin color area detection unit 15 sets the hand area HD having a color difference that is the same as or close to the face area FC to the image data in which the person HM shown in FIG. To detect.
  • image processing for example, face detection processing
  • FIG. 6 is an explanatory diagram showing an outline of the operation of the photoelectric pulse wave fluctuation removing filter 17.
  • FIG. 7 is an explanatory diagram illustrating an operation example of the photoelectric pulse wave fluctuation removal filter 17 when the subject person is stationary.
  • FIG. 8 is an explanatory diagram illustrating an operation example of the photoelectric pulse wave fluctuation removal filter 17 when the subject person is moving.
  • the photoelectric pulse wave fluctuation removal filter 17 has a minute change amount of the pixel value of the skin color area (for the frame of the image data IM1 in which the skin color area of the person is detected by the skin color area detection unit 15 ( Removal and smoothing of the graph GPH1 shown in FIG. 6 are performed.
  • the pixel value may also be an average value of pixel values in one block that can be composed of (1 pixel ⁇ 1 pixel) or (32 pixels in the horizontal direction of the frame ⁇ 32 pixels in the vertical direction of the same frame). Good. Note that the number of pixels constituting one block is not limited to (32 horizontal directions of the frame ⁇ 32 vertical directions of the same frame).
  • frames P of images (input images) input by the image input unit 11 are continuously input at every time T.
  • the photoelectric pulse wave fluctuation removal filter 17 performs the specific color in the frame P (t) as the smoothing in the time direction of the pixel value P (t, i, j) of the region TG of the specific color (for example, skin color).
  • the pixel value P (t, i, j) of the region TG (for example, skin color) and the pixel value P (t of the region TG of the specific color (for example, skin color) in the corresponding frames P (t ⁇ 1) and P (t + 1) ⁇ 1, i, j) and P (t + 1, i, j) are averaged (see formula (1)).
  • the photoelectric pulse wave fluctuation removal filter 17 does not perform smoothing according to the equation (1) for pixel values other than the specific color (for example, skin color) region TG.
  • a predetermined value for example, 4
  • the frame P of the image (input image) input by the image input unit 11 is continuously input at every time T.
  • the photoelectric pulse wave fluctuation removal filter 17 divides a frame of image data into blocks (for example, 4 pixels ⁇ 4 pixels, 8 pixels ⁇ 8 pixels, 16 pixels ⁇ 16 pixels, etc. A search is performed by scanning where a block having the same or similar pixel value is located in adjacent frames.
  • the photoelectric pulse wave fluctuation removal filter 17 selects a block in which a minimum sum of absolute differences of pixel values at positions corresponding to the blocks is obtained, and is a region of a specific color (for example, skin color) of the current frame in adjacent frames. Is determined as a block having the same or similar pixel value.
  • the photoelectric pulse wave fluctuation removing filter 17 calculates a vector value up to the position of a block having the same or similar pixel value in adjacent frames before and after as a motion vector.
  • An average value (see Equation (3)) with t + 1, k, l) is calculated. Note that the photoelectric pulse wave fluctuation removal filter 17 does not perform smoothing according to the equation (3) for pixel values other than the specific color (for example, skin color) region TG.
  • the coordinates (g, h) of the pixel value corresponding to t, i, j) are expressed by Equation (4).
  • FIG. 9 is a flowchart illustrating an example of an operation procedure of the image processing apparatus 1 according to the present embodiment.
  • the image input unit 11 continuously receives frames of image data in which a person (for example, a performer of a TV program) is captured by an external device (not illustrated) (for example, a camera that captures images at a predetermined frame rate) from the external device. Is input (acquired) and stored in the image storage unit 13.
  • the skin color area detection unit 15 reads the image data stored in the image storage unit 13 by the image input unit 11, and the specific color (for example, skin color) area (for example, the person HM shown in FIGS. 5A and 5B) in the frame of the image data.
  • a face area FC and a hand area HD are detected.
  • the skin color area detection unit 15 outputs information (for example, coordinates) regarding the specific color area detected in the frame of the image data to the photoelectric pulse wave fluctuation removal filter 17.
  • the photoelectric pulse fluctuation removal filter 17 reads the image data stored in the image storage unit 13 by the image input unit 11 for each partial region (S1), and outputs the skin color region detection unit 15 (that is, in the frame of the image data). Based on the detected information (for example, coordinates) regarding the specific color area, it is determined whether the read partial area is a skin color (S2). As described above, the pixel value is one block that can be composed of (1 pixel ⁇ 1 pixel) or (32 pixels in the horizontal direction of the frame ⁇ 32 pixels in the vertical direction of the same frame). The average value of the pixel values at.
  • the photoelectric pulse wave fluctuation removal filter 17 reads the partial area read in step S1. And the pixel values of the image data of the specific color area detected by the skin color area detecting unit 15 (more specifically, the luminance) (Y) and the color difference (U, V)) are smoothed (S3).
  • step S3 for example, when the person shown in the image data is stationary, the smoothing process is performed by the method shown in FIG. 7, while when the person shown in the image data is moving, Smoothing is performed by the method shown in FIG.
  • the photoelectric pulse wave fluctuation removal filter 17 stores the image data of the partial area after smoothing the pixel value of the image data of the specific color area in the image storage unit 13.
  • step S1 when the pixel value data of the partial area read in step S1 is not a color (for example, skin color) designated in advance (S2, NO), the processing of the image processing apparatus 1 proceeds to step S4.
  • a color for example, skin color
  • the image encoding unit 19 stores the image data of the output image of the photoelectric pulse fluctuation removal filter 17 stored in the image storage unit 13 (that is, the pixel value of the partial region read by the photoelectric pulse fluctuation removal filter 17 in step S1).
  • the smoothed partial region image data is encoded (S4).
  • the image processing apparatus 1 inputs image data obtained by capturing a person in the image input unit 11, extracts a specific color region of the input image data in the skin color region detection unit 15, and inputs it.
  • the pixel value in a specific color (for example, skin color) region is smoothed by the photoelectric pulse wave fluctuation removal filter 17 using the frame of the image data and the frame of each image data before and after the input time.
  • the image processing apparatus 1 encodes the output image of the photoelectric pulse wave fluctuation removal filter 17 after the pixel value is smoothed, and the encoded output image is externally (for example, an external device connected to a network, or Output to the display.
  • the image processing apparatus 1 smoothes the pixel values in a specific color (for example, skin color) region in the frame of the image data, and thus a biological content that is not intended by the producer or distributor of the video content composed of the image data. Since the distribution of information (for example, pulse rate) can be effectively suppressed, the privacy of the person appearing in the video content can be protected accurately.
  • the image processing apparatus 1 increases the pulse rate of an announcer who misrepresents a statement when the announcer appears in a television program, or increases the pulse rate of an athlete in a shogi game. It is possible to prevent the viewer from knowing, and it is possible to make video content creators and distributors produce video content that can accurately protect the privacy of performers.
  • the image processing apparatus 1 uses the pixel value of the specific color area in the frame of the input image data and the pixel value of the corresponding specific color area of each frame of the image data before and after the input. Is larger than a predetermined value (for example, 4), the smoothing of the pixel values in the specific color region in the frame of the input image data is omitted, and the data is output as it is.
  • a predetermined value for example, 4
  • the image processing apparatus 1 performs smoothing using the frame at the time of input and the frames before and after the input because the motion clearly appears in the frame at the time of input by the image input unit 11. This can reduce the influence (for example, image quality degradation) on the image quality of the frame.
  • the pixel values in the specific color region are obtained by using a plurality of pixels (for example, 32 pixels) in the horizontal direction and a plurality of pixels in the vertical direction of the frame of the image data input to the image input unit 11.
  • An average value of (for example, 32 blocks) is used.
  • the image processing apparatus 1 uses the average value of the pixel values of the block unit having a plurality of pixels as the pixel value in the specific color region, so that the specific value can be obtained as compared with the case where the pixel value of a single pixel is used.
  • the reliability of the pixel value in the color region can be improved.
  • the present disclosure is useful as an imaging device, an image processing device, and an image processing method that effectively suppress the distribution of biological information that is not intended by the creator or distributor of video content, and accurately protect the privacy of a person appearing in the video content. It is.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • Pathology (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • General Physics & Mathematics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Human Computer Interaction (AREA)
  • Optics & Photonics (AREA)
  • Physiology (AREA)
  • Cardiology (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Ophthalmology & Optometry (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)
  • Measuring Pulse, Heart Rate, Blood Pressure Or Blood Flow (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
PCT/JP2015/006062 2014-12-17 2015-12-07 撮像装置、画像処理装置及び画像処理方法 Ceased WO2016098308A1 (ja)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US15/529,672 US10492678B2 (en) 2014-12-17 2015-12-07 Image capturing apparatus, image processing apparatus and image processing method for secure processing of biological information

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2014255461A JP6436385B2 (ja) 2014-12-17 2014-12-17 撮像装置、画像処理装置及び画像処理方法
JP2014-255461 2014-12-17

Publications (1)

Publication Number Publication Date
WO2016098308A1 true WO2016098308A1 (ja) 2016-06-23

Family

ID=56126216

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2015/006062 Ceased WO2016098308A1 (ja) 2014-12-17 2015-12-07 撮像装置、画像処理装置及び画像処理方法

Country Status (3)

Country Link
US (1) US10492678B2 (https=)
JP (1) JP6436385B2 (https=)
WO (1) WO2016098308A1 (https=)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6436385B2 (ja) * 2014-12-17 2018-12-12 パナソニックＩｐマネジメント株式会社 撮像装置、画像処理装置及び画像処理方法
US20170262987A1 (en) * 2016-03-09 2017-09-14 Xerox Corporation Method and apparatus for non-contact estimation of pulse transmit time
US10335045B2 (en) 2016-06-24 2019-07-02 Universita Degli Studi Di Trento Self-adaptive matrix completion for heart rate estimation from face videos under realistic conditions

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH02155381A (ja) * 1988-12-08 1990-06-14 Canon Inc 動画／静止画変換装置
JP2000197011A (ja) * 1998-12-28 2000-07-14 Nec Corp 動画像符号化方法、動画像符号化装置及び動画像符号化プログラムを記憶した記憶媒体
JP2013240039A (ja) * 2012-04-11 2013-11-28 Sharp Corp 映像処理装置、映像処理方法、テレビジョン受像機、プログラム、及び記録媒体
JP2014502187A (ja) * 2010-11-17 2014-01-30 コーニンクレッカ フィリップス エヌ ヴェ 入力信号を出力信号に変換するためのシステム及び方法

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050054935A1 (en) * 2003-09-08 2005-03-10 Rice Robert R. Hyper-spectral means and method for detection of stress and emotion
DE602007002068D1 (de) * 2006-05-04 2009-10-01 Trinity College Dublin Handwaschüberwachungssystem
JP5332406B2 (ja) 2008-08-28 2013-11-06 富士通株式会社 脈拍計測装置、脈拍計測方法および脈拍計測プログラム
JP5685598B2 (ja) * 2009-10-06 2015-03-18 コーニンクレッカ フィリップス エヌ ヴェ 画素値に基づく少なくとも値の変化を表す時間変化する信号の形成
NO334029B1 (no) * 2011-09-30 2013-11-18 Cisco Tech Inc System og fremgangsmåte for etablering av videokonferansesesjon med justerbart filter for markering av tilstedeværelsesnivå i endepunkter
DE112013000762T5 (de) * 2012-01-31 2014-12-11 Panasonic Corporation Bildverarbeitungsvorrichtung und Bildverarbeitungsverfahren
US8897522B2 (en) * 2012-05-30 2014-11-25 Xerox Corporation Processing a video for vascular pattern detection and cardiac function analysis
KR20150095661A (ko) * 2012-11-11 2015-08-21 켄코우 게엠베하 활력 매개변수들을 측정하기 위한 방법 및 장치
US9633482B2 (en) * 2014-08-12 2017-04-25 Toshiba Medical Systems Corporation Apparatus and method for restricting image data visualization
JP6436385B2 (ja) * 2014-12-17 2018-12-12 パナソニックＩｐマネジメント株式会社 撮像装置、画像処理装置及び画像処理方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH02155381A (ja) * 1988-12-08 1990-06-14 Canon Inc 動画／静止画変換装置
JP2000197011A (ja) * 1998-12-28 2000-07-14 Nec Corp 動画像符号化方法、動画像符号化装置及び動画像符号化プログラムを記憶した記憶媒体
JP2014502187A (ja) * 2010-11-17 2014-01-30 コーニンクレッカ フィリップス エヌ ヴェ 入力信号を出力信号に変換するためのシステム及び方法
JP2013240039A (ja) * 2012-04-11 2013-11-28 Sharp Corp 映像処理装置、映像処理方法、テレビジョン受像機、プログラム、及び記録媒体

Also Published As

Publication number Publication date
US10492678B2 (en) 2019-12-03
JP6436385B2 (ja) 2018-12-12
JP2016112325A (ja) 2016-06-23
US20170325680A1 (en) 2017-11-16

Similar Documents

Publication Publication Date Title
EP3593524B1 (en) Image quality assessment
JP6049448B2 (ja) 被写体領域追跡装置、その制御方法及びプログラム
JP6653467B2 (ja) 脈拍推定装置、脈拍推定システムおよび脈拍推定方法
KR101913837B1 (ko) 파노라마 영상 생성 방법 및 이를 적용한 영상기기
US9286944B2 (en) Methods and systems for providing a combination of media data and metadata
WO2016006027A1 (ja) 脈波検出方法、脈波検出プログラム及び脈波検出装置
WO2014030439A1 (ja) 生体状態観察システム、生体状態観察方法、およびプログラム
JP6574878B2 (ja) 画像処理装置及び画像処理方法、撮像装置、プログラム、並びに記憶媒体
US10270977B2 (en) Imaging apparatus and a method of tracking a subject in the imaging apparatus
JP5909147B2 (ja) 撮像装置、撮像装置の制御方法、およびプログラム
US11647913B2 (en) Image processing apparatus and pulse estimation system provided therewith, and image processing method
US20160150158A1 (en) Photographing apparatus and method for controlling thereof
JP2014221172A (ja) 脈波検出装置、脈波検出プログラム、脈波検出方法及びコンテンツ評価システム
JP6436385B2 (ja) 撮像装置、画像処理装置及び画像処理方法
JP5955170B2 (ja) 表示制御装置、表示制御方法、及びプログラム
JP6087615B2 (ja) 画像処理装置およびその制御方法、撮像装置、および表示装置
JP6450107B2 (ja) 画像処理装置及び画像処理方法、プログラム、記憶媒体
US20150116568A1 (en) Reduction of spatial resolution for temporal resolution
KR101597095B1 (ko) 모니터링 시스템에서의 모니터링 데이터 처리
US20230222765A1 (en) Image processing device, image processing method, and storage medium
JP7232014B2 (ja) 画像のノイズを低減する画像処理装置および画像処理方法ならびにプログラム
JP2018180714A (ja) 動体検知システム、動体検知装置、動体検知方法、およびプログラム
EP3370411B1 (en) Image output method and image capturing device
JP2017228942A (ja) ヘッドマウントディスプレイ、送信制御プログラムおよび送信制御方法
JP2021166389A (ja) 画像処理装置及びこれを備えたバイタル情報取得システムならびに画像処理方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15869517

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 15529672

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15869517

Country of ref document: EP

Kind code of ref document: A1