WO2021261141A1

WO2021261141A1 - Object detection device and object detection method

Info

Publication number: WO2021261141A1
Application number: PCT/JP2021/019505
Authority: WO
Inventors: 真也阪田
Original assignee: オムロン株式会社
Priority date: 2020-06-22
Filing date: 2021-05-24
Publication date: 2021-12-30
Also published as: JP2022002019A; JP7435298B2

Abstract

This object detection device comprises an object detection means for detecting, from a captured image, an object region which is a region of an object, a detection means for detecting, from the object region detected by the object detection means in the image, a motion region which is a region including a motion, and a correction means for correcting the object region on the basis of the motion region detected by the detection means.

Description

Object detection device and object detection method

The present invention relates to a technique for detecting an area of an object from a captured image.

Various techniques have been proposed as conventional techniques for object detection and motion detection. For example, Patent Document 1 proposes a technique related to face detection.

Japanese Unexamined Patent Publication No. 2006-293720

However, in the conventional object detection, the detected area may be too large or too small with respect to the object area. That is, the area of the object cannot be detected with high accuracy. Not only is it too large, but it can also be too small, so it is not appropriate to resize (always enlarge or always reduce) the detected area at a given magnification. Even in motion detection, a plurality of regions may be detected as regions with motion for one moving object (the regions of one moving object may be detected by being divided into a plurality of regions). That is, the region of the moving object cannot be detected with high accuracy.

And, if the area of the object (including the moving body) is not detected accurately, other processing based on the area may not be suitable.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique capable of detecting a region of an object with high accuracy.

In order to achieve the above object, the present invention adopts the following configuration.

The first aspect of the present invention is that there is movement from an object detection means that detects an object region, which is a region of an object, from an captured image, and from the object region that is detected by the object detection means in the image. Provided is an object detection device comprising a detection means for detecting a motion region which is a region and a correction means for correcting the object region based on the motion region detected by the detection means. Objects are, for example, vehicles (cars, planes, etc.), natural objects (trees, flowers, mountains, etc.), human bodies, faces, and the like. The motion region is, for example, an region composed of only motion pixels (pixels with motion), a smallest rectangular region including an region composed of only motion pixels, and the like.

When the detected object area is too large, the motion area detected in it more accurately represents the true object area than the detected object area. According to the above-described configuration, since the object region is corrected based on the motion region detected from the object region, it is possible to obtain a region that more accurately represents the true object region as the corrected object region. .. As a result, other processing based on the object region (AE, AF, exclusion of erroneously detected regions, etc.) can be preferably performed.

When multiple movement areas are detected from the detected object area, it is highly possible that each movement area is a part of the same object. Therefore, when the detection means detects a plurality of movement regions, the correction means may correct the object region based on the smallest region including the plurality of movement regions. By doing so, even when a plurality of motion regions are detected, it is possible to obtain a region that more accurately represents the true object region as the corrected object region.

The detection means has a motion detection means for detecting a motion region from the image and a selection means for selecting a motion region located in the object region from the detection result of the motion detection means, and the correction means includes the correction means. The object region may be corrected based on the movement region selected by the selection means. At this time, the selection means may select a movement region whose center position is included in the object region. The selection means may select a movement region that is entirely included in the object region.

There is a high possibility that the small movement area is an area where noise or the like is erroneously detected. Therefore, the correction means may correct the object region based on the movement region whose size is equal to or larger than the threshold value detected by the detection means. By doing so, it is possible to suppress erroneous correction of the object area. Here, the detection means may not detect the movement area whose size is less than the threshold value, or the correction means may not use the movement area whose size is less than the threshold value. The size of the motion region may be the size of the entire motion region (the total number of pixels in the motion region), the number of motion pixels, or the like.

The detecting means may detect the moving region by the background subtraction method. The background subtraction method is, for example, a method of detecting a pixel in an image in which the difference (absolute value) of the pixel value from a predetermined background image is equal to or greater than a predetermined threshold value as motion pixels.

The detection means may detect the movement region by the inter-frame difference method. In the inter-frame difference method, for example, among the captured current images (current frames), the pixels whose pixel value difference from the captured past images (past frames) is equal to or larger than a predetermined threshold are motion pixels. It is a method to detect as.

The detection means may detect the movement region from a region including the object region and a region around the object region. By doing so, even if the detected object area is too small, it is possible to obtain a region that more accurately represents the true object region as the corrected object region. Here, the peripheral area is, for example, an area obtained by excluding the object area from an area obtained by enlarging the detected object area by a predetermined time, or an area from the edge of the detected object area to a position outside by a predetermined number of pixels. And so on.

The object may be a human body.

The second aspect of the present invention is the object detection step of detecting the object area which is the area of the object from the captured image, and the motion from the object area detected in the object detection step in the image. Provided is an object detection method comprising a detection step for detecting a motion region which is a region and a correction step for correcting the object region based on the motion region detected in the detection step.

The present invention can be regarded as an object detection system having at least a part of the above configuration or function. The present invention also provides, a program for causing a computer to execute an object detection method or an object detection system control method, including at least a part of the above processing, or such a program non-temporarily. It can also be regarded as a recorded computer-readable recording medium. Each of the above configurations and processes can be combined with each other to construct the present invention as long as no technical contradiction occurs.

According to the present invention, the region of an object can be detected with high accuracy.

FIG. 1 is a block diagram showing a configuration example of an object detection device to which the present invention is applied. FIG. 2A is a schematic diagram showing a rough configuration example of an object detection system according to an embodiment of the present invention, and FIG. 2B is a configuration example of a PC (object detection device) according to the embodiment. It is a block diagram which shows. FIG. 3 is a flowchart showing an example of a processing flow of a PC according to an embodiment of the present invention. FIG. 4 is a diagram showing a specific example of the operation according to the embodiment of the present invention. FIG. 5 is a diagram showing a specific example of the operation according to the embodiment of the present invention. FIG. 6 is a diagram showing a specific example of the operation according to the embodiment of the present invention.

<Application example>
An application example of the present invention will be described.

In conventional object detection, the detected area may be too large or too small for the object area. That is, the area of the object cannot be detected with high accuracy. Not only is it too large, but it can also be too small, so it is not appropriate to resize (always enlarge or always reduce) the detected area at a given magnification. Even in motion detection, a plurality of regions may be detected as regions with motion for one moving object (the regions of one moving object may be detected by being divided into a plurality of regions). That is, the region of the moving object cannot be detected with high accuracy.

And, if the area of the object (including the moving body) is not detected accurately, other processing based on the area may not be suitable. For example, if the detected area contains a large amount of the background of the object, the exposure may be controlled to be suitable for the background in AE (automatic exposure), and may not be controlled to the exposure suitable for the object. .. Similarly, in AF (autofocus), the focus may be on the background and the object may not be in focus. Further, in the process of excluding an area that is too large or an area that is too small as an erroneously detected area, the area corresponding to the object may be unintentionally excluded.

FIG. 1 is a block diagram showing a configuration example of an object detection device 100 to which the present invention is applied. The object detection device 100 includes an object detection unit 101, a detection unit 102, and a correction unit 103. The object detection unit 101 detects an object area, which is an object area, from the captured image. The detection unit 102 detects a motion region, which is a motion region, from the object region detected by the object detection unit 101 in the captured image. The correction unit 103 corrects the object area detected by the object detection unit 101 based on the movement area detected by the detection unit 102. The object detection unit 101 is an example of the object detection means of the present invention, the detection unit 102 is an example of the detection means of the present invention, and the correction unit 103 is an example of the correction means of the present invention. Here, the object is, for example, a vehicle (car, airplane, etc.), a natural object (tree, flower, mountain, etc.), a human body, a face, or the like. The motion region is, for example, an region composed of only motion pixels (pixels with motion), a smallest rectangular region including an region composed of only motion pixels, and the like.

<Embodiment>
An embodiment of the present invention will be described.

FIG. 2A is a schematic diagram showing a rough configuration example of the object detection system according to the present embodiment. The object detection system according to the present embodiment includes a camera 10 and a PC200 (personal computer; object detection device). The camera 10 and the PC 200 are connected to each other by wire or wirelessly. The camera 10 captures an image and outputs it to the PC 200. The camera 10 is not particularly limited, and for example, an IR that detects natural light, a camera that measures distance (for example, a stereo camera), and a camera that measures temperature (for example, IR (Infrared Ray) light). It may be either a camera) or the like. The captured image is not particularly limited, and may be, for example, an RGB image, an HSV image, a gray scale image, or the like. The PC 200 detects an object from an image captured by the camera 10. The PC200 displays an object detection result (presence / absence of an object, an area where an object is detected, etc.) on a display unit, records an object detection result on a storage medium, and records an object detection result on another terminal (remote). Output to the administrator's smartphone etc. in the ground).

Although the PC 10 is a device separate from the camera 10 in the present embodiment, the PC 200 may be built in the camera 10. The display unit and the storage medium described above may or may not be a part of the PC200. Further, the installation location of the PC 200 is not particularly limited. For example, the PC 200 may or may not be installed in the same room as the camera 10. The PC 200 may or may not be a computer on the cloud. The PC 200 may be a terminal such as a smartphone carried by an administrator.

FIG. 2B is a block diagram showing a configuration example of the PC200. The PC 200 has an input unit 210, a control unit 220, a storage unit 230, and an output unit 240.

In this embodiment, it is assumed that the camera 10 captures a moving image. The input unit 210 sequentially performs a process of acquiring an captured image (frame of a moving image) from the camera 10 and outputting it to the control unit 220. The camera 10 may sequentially capture still images, and in that case, the input unit 210 sequentially acquires the captured still images from the camera 10 and outputs them to the control unit 220. conduct.

The control unit 220 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), etc., and controls each component and performs various information processing. In the present embodiment, the control unit 220 detects an object from the captured image and outputs the detection result of the object (presence / absence of the object, region where the object is detected, etc.) to the output unit 240.

The storage unit 230 stores programs executed by the control unit 220, various data used by the control unit 220, and the like. For example, the storage unit 230 is an auxiliary storage device such as a hard disk drive or a solid state drive.

The output unit 240 displays the detection result (object detection result) output by the control unit 220 on the display unit, records it on a storage medium, or uses another terminal (such as a smartphone of an administrator at a remote location). Output to.

The control unit 220 will be described in more detail. The control unit 220 includes an object detection unit 221, a detection unit 222, and a correction unit 223.

The object detection unit 221 acquires an image captured by the camera 10 from the input unit 210, and detects an object area, which is an object area, from the acquired image. Then, the object detection unit 221 outputs the detection result of the object region to the detection unit 222. The object area detected by the object detection unit 221 may be too large or too small with respect to the true object area. The object detection unit 221 is an example of the object detection means of the present invention.

Any algorithm may be used for object detection by the object detection unit 221. For example, an object region may be detected using a detector (identifier) that combines boosting with image features such as HoG and Haar-like. An object region may be detected using a trained model generated by existing machine learning, specifically generated by deep learning (eg, R-CNN, Fast R-CNN, YOLO, SSD, etc.). The trained model may be used to detect the object region.

The detection unit 222 acquires the image captured by the camera 10 from the input unit 210, and acquires the detection result of the object region from the object detection unit 221. The detection unit 222 detects a motion region, which is a motion region, from the object region detected by the object detection unit 221 in the acquired image. Then, the detection unit 222 outputs the detection result of the movement region to the correction unit 223. The detection unit 222 is an example of the detection means of the present invention.

Any algorithm may be used for motion detection by the detection unit 222. For example, the detection unit 222 may detect the motion region by the background subtraction method or may detect the motion region by the interframe subtraction method. The background subtraction method is, for example, a method of detecting a pixel in an image in which the difference (absolute value) of the pixel value from a predetermined background image is equal to or greater than a predetermined threshold value as motion pixels. In the inter-frame difference method, for example, among the captured current images (current frames), the pixels whose pixel value difference from the captured past images (past frames) is equal to or larger than a predetermined threshold are motion pixels. It is a method to detect as. In the inter-frame difference method, for example, the past frame is a frame before a predetermined number of the current frame, and the predetermined number is 1 or more. The predetermined number (the number of frames from the current frame to the past frame) is determined according to the frame rate of the processing of the control unit 220 (the processing of detecting and correcting the object area), the frame rate of imaging by the camera 10, and the like. You may.

Further, as described above, the motion region may be a region consisting of only motion pixels (pixels with motion), or may be a minimum rectangular region including an region consisting of only motion pixels. For example, the detection unit 222 may detect a rectangular contour circumscribing a region consisting of only motion pixels, and may detect the region having the contour as a motion region. According to this method, the smallest rectangular area including the area consisting of only the moving pixels is detected as the moving area. The detection unit 222 may detect the movement region by labeling. In labeling, each moving pixel of the captured image is selected as a pixel of interest. Then, when the movement pixel to which the label (the number of the movement region) is attached exists around the attention pixel, the same label as the movement pixel is attached to the attention pixel. If the moving pixel with the label does not exist around the pixel of interest, a new label is attached to the pixel of interest. According to this method, a region consisting only of motion pixels is detected as a motion region. The motion pixels (moving pixels around the pixel of interest) referred to in labeling are not particularly limited. For example, 8 pixels adjacent to the pixel of interest may be referred to, or 18 pixels separated by 2 pixels from the pixel of interest may be referred to.

The correction unit 223 corrects the object area detected by the object detection unit 221 based on the detection result of the detection unit 222. Then, the correction unit 223 outputs the information of the corrected object area to the output unit 240 as the detection result of the object. The correction unit 223 is an example of the correction means of the present invention.

For example, when the detected object area is too large, the motion area detected in the detected object area more accurately represents the true object area than the detected object area. Therefore, when the detection unit 222 detects one movement region, the correction unit 223 corrects the object region detected by the object detection unit 221 based on the movement region. By doing so, it is possible to obtain a region that more accurately represents the true object region as the corrected object region.

Also, when multiple movement areas are detected from the detected object area, it is highly possible that each movement area is a part of the same object. Therefore, when the detection unit 222 detects a plurality of movement regions, the correction unit 223 corrects the object region detected by the object detection unit 221 based on the minimum region including the plurality of movement regions. By doing so, even when a plurality of motion regions are detected, it is possible to obtain a region that more accurately represents the true object region as the corrected object region. The shape of the corrected object region may or may not be a predetermined shape (rectangle or the like).

The detection unit 222 will be described in more detail. The detection unit 222 has a motion detection unit 222-1 and a selection unit 222-2.

The motion detection unit 222-1 acquires an image captured by the camera 10 from the input unit 210, and detects a motion region from the acquired image. Then, the motion detection unit 222-1 outputs the detection result of the motion region to the selection unit 222-2. The motion detection unit 222-1 may detect the motion region from the entire acquired image, or may detect the motion region from a part (predetermined region) of the acquired image. As described above, any algorithm may be used for motion detection by the motion detection unit 222-1. The motion detection unit 222-1 is an example of the motion detection means of the present invention.

The selection unit 222-2 acquires the detection result of the motion region from the motion detection unit 222-1, and acquires the detection result of the object region from the object detection unit 221. The selection unit 222-2 selects a movement region located in the object region detected by the object detection unit 221 from one or more movement regions detected by the motion detection unit 222-1. The method of selecting the movement region is not particularly limited, and for example, the selection unit 222-2 may select a movement region whose center position is included in the object region, or may select a movement region whose entire center position is included in the object region. You may choose. Then, the selection unit 222-2 outputs the selection result of the movement region to the correction unit 223 as the detection result of the movement region by the detection unit 222. The selection unit 222-2 is an example of the selection means of the present invention.

FIG. 3 is a flowchart showing an example of the processing flow of the PC200. The PC 200 repeatedly executes the processing flow of FIG. The repetition cycle of the processing flow of FIG. 3 is not particularly limited, but in the present embodiment, it is assumed that the processing flow of FIG. 3 is repeated at the frame rate (for example, 30 fps) of the image taken by the camera 10.

First, the input unit 210 acquires the captured image from the camera 10 (step S301). FIG. 4 shows an example of the image 400 captured by the camera 10. The image 400 shows the human body 410.

Next, the object detection unit 221 detects the object region from the image acquired in step S301 (step S302). For example, from the image 400 of FIG. 4, the object region 420 including the human body 410 is detected as the region of the human body 410. The object area 420 is much larger than the human body 410.

Next, the motion detection unit 222-1 detects the motion region from the image acquired in step S301 (step S303). For example, the motion regions 431 to 435 are detected from the image 400 of FIG.

Next, the selection unit 222-2 selects a movement region located in the object region detected in step S302 from one or more movement regions detected in step S303 (step S304). For example, among the movement areas 431 to 435 in FIG. 4, the movement areas 431 to 434 included in the object area 420 are selected.

Next, the correction unit 223 corrects the object region detected in step S302 based on the smallest region including the movement region selected in step S304 (step S305). For example, the object region 420 in FIG. 4 is corrected to the smallest rectangular region 440 including the motion regions 431 to 434. The rectangular region 440 (corrected object region) more accurately represents the true region of the human body 410 than the object region 420. That is, the object region can be detected with high accuracy by the correction in step S305. In the correction of the object area, it is sufficient that the object area is brought close to the reference area, and it is not necessary to completely match the object area with the reference area. For example, the object region 420 may be corrected to a region slightly different from the rectangular region 440.

Next, the output unit 240 outputs the correction result (corrected object area) of step S305 to the display unit, the storage medium, the smartphone, and the like (step S306).

As described above, according to the present embodiment, the object region is corrected based on the motion region detected from the object region, so that the true object region is more accurately represented as the corrected object region. You can get the area. As a result, other processing based on the object region (AE, AF, exclusion of erroneously detected regions, etc.) can be preferably performed.

It is highly possible that the small movement area is an area where noise or the like is erroneously detected. Therefore, the correction unit 223 may correct the object area based on the movement area whose size is equal to or larger than the threshold value detected by the detection unit 222. By doing so, it is possible to suppress erroneous correction of the object area. Here, the detection unit 222 may not detect the movement region whose size is less than the threshold value, or the correction unit 223 may not use the movement region whose size is less than the threshold value. The size of the motion region may be the size of the entire motion region (the total number of pixels in the motion region), the number of motion pixels, or the like.

A specific example will be described with reference to FIG. In FIG. 5, the same objects and regions as those in FIG. 4 are designated by the same reference numerals as those in FIG. In the example of FIG. 5, in the object region 420 of the captured image 500, in addition to the motion regions 431 to 434 of FIG. 4, the motion regions 531 and 532 due to noise are detected. When the size of the motion region is not taken into consideration, the object region 420 is corrected to the smallest rectangular region 540 including the motion regions 431 to 434, 531, 532, so that there is almost no change (erroneous correction). Considering the size of the motion region, the object region 420 can be corrected to the rectangular region 440 by using the motion regions 431 to 434 excluding the motion regions 531, 532.

Further, the detection unit 222 may detect a motion region from a region including an object region detected by the object detection unit 221 and a region around the object region. In other words, the selection unit 222-2 may select a movement region located in a region including an object region detected by the object detection unit 221 and a region around the object region. By doing so, even if the detected object area is too small, it is possible to obtain a region that more accurately represents the true object region as the corrected object region. Here, the peripheral area is, for example, an area obtained by excluding the object area from an area obtained by enlarging the detected object area by a predetermined time, or an area from the edge of the detected object area to a position outside by a predetermined number of pixels. And so on. The predetermined magnification or the predetermined number of pixels may differ between the horizontal direction (horizontal direction) and the vertical direction (vertical direction) of the image.

A specific example will be described with reference to FIG. In the example of FIG. 6, the object region 620, which is much smaller than the human body 610, is detected as the region of the human body 610 from the captured image 600, and only the motion region 631 is detected in the object region 620. Therefore, if the area 650 around the object area 620 is not taken into consideration, the object area 420 is reduced to the motion area 631 (wrong correction). Since the motion regions 632 to 634 are detected in the peripheral region 650, the object region 420 can be expanded to the smallest rectangular region 640 including the motion regions 631 to 634, considering the peripheral region 650. The region 640 (corrected object region) more accurately represents the true region of the human body 610 than the object region 620.

<Others>
The above-described embodiment is merely an example of a configuration example of the present invention. The present invention is not limited to the above-mentioned specific form, and various modifications can be made within the scope of its technical idea.

<Appendix 1>
An object detection means (101,221) that detects an object area, which is an object area, from the captured image, and
Among the images, the detection means (102, 222) for detecting the motion region, which is a motion region, from the object region detected by the object detection means.
An object detection device (100, 200) comprising a correction means (103, 223) for correcting the object region based on the movement region detected by the detection means.

<Appendix 2>
An object detection step (S302) for detecting an object area, which is an object area, from the captured image, and
Among the images, a detection step (S303, S304) for detecting a motion region, which is a motion region, from the object region detected in the object detection step.
An object detection method comprising a correction step (S305) for correcting the object region based on the movement region detected in the detection step.

100: Object detection device 101: Object detection unit 102: Detection unit 103: Correction unit 10: Camera 200: PC (object detection device)
210: Input unit 220: Control unit 230: Storage unit 240: Output unit 221: Object detection unit 222: Detection unit 223: Correction unit 222-1: Motion detection unit 222-2: Selection unit 400: Image 410: Human body 420: Object area 431 to 435: Motion area 440: Rectangular area (corrected object area)
500: Image 531 and 532: Motion area 540: Rectangular area (corrected object area)
600: Image 610: Human body 620: Object area 631 to 634: Movement area 640: Rectangular area (corrected object area)
650: Area (area around the detected object area)

Claims

An object detection means that detects an object area, which is an object area, from the captured image,
A detection means for detecting a motion region, which is a motion region, from the object region detected by the object detection means in the image.
An object detection device comprising a correction means for correcting the object region based on the movement region detected by the detection means.
The first aspect of claim 1, wherein when the detecting means detects a plurality of motion regions, the correction means corrects the object region based on the smallest region including the plurality of motion regions. Object detector.
The detection means
A motion detecting means for detecting a motion region from the image,
It has a selection means for selecting a movement region located in the object region from the detection result of the movement detection means.
The object detection device according to claim 1 or 2, wherein the correction means corrects the object region based on the movement region selected by the selection means.
The object detection device according to claim 3, wherein the selection means selects a movement region whose center position is included in the object region.
The object detection device according to claim 3, wherein the selection means selects a motion region as a whole included in the object region.
The object according to any one of claims 1 to 5, wherein the correction means corrects the object region based on a motion region whose size is equal to or larger than a threshold value detected by the detection means. Detection device.
The object detection device according to any one of claims 1 to 6, wherein the detection means detects the motion region by a background subtraction method.
The object detection device according to any one of claims 1 to 6, wherein the detection means detects the movement region by an inter-frame difference method.
The object detection device according to any one of claims 1 to 8, wherein the detection means detects the movement region from a region including the object region and a region around the object region.
The object detection device according to any one of claims 1 to 9, wherein the object is a human body.
An object detection step that detects an object area, which is an object area, from the captured image,
In the image, a detection step of detecting a motion region, which is a motion region, from the object region detected in the object detection step,
A method for detecting an object, which comprises a correction step for correcting the object region based on the motion region detected in the detection step.
A program for causing a computer to execute each step of the object detection method according to claim 11.