JP2015158796A - Object detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an object detection device which suppresses rain streaks reflected in an imaged video from being erroneously recognized as a moving object such as a person even when an object is detected from an image captured by an infrared camera.SOLUTION: A temporal change extraction part 2 calculates a luminance dispersion value S1 (image M2a) from a reference frame K to a past frame K-N, a luminance dispersion value S2 (image M2b) from the reference frame K to a future image frame K+N, a luminance dispersion value S3 (image M3a) from an image frame K-1 to a past image frame K-N, and a luminance dispersion value S4 (image M3b) from an image frame K+1 to a future image frame K+N. Pixel state determination means determines that a pixel with S1 and S2 larger than a threshold and S3 or S4 larger than the threshold is in the dynamic state and other pixels are in the static state. An inter-frame feature extraction part 3 calculates the feature quantity. A detection part 4 detects an object by a cascade discriminator by AdaBoost.

Description

  The present invention relates to an object detection device that detects an object from a captured image of a camera.

In recent years, with the increase and sophistication of crime, the demand for security cameras with high security functions is increasing even in ordinary households. Such a conventional security camera is configured so that the camera is activated and starts imaging when the human sensor senses, so that even if a suspicious person is in the imaging range of the camera, it may be located away from the camera. Since it was not captured, it was insufficient for crime prevention.
For this reason, the inventors of the present invention have proposed a technique for detecting a moving object by image processing with image processing reduced as much as possible by image processing.

The object detection device of Patent Document 1 calculates the variance value of the luminance of the same part up to a reference image frame and a predetermined number of past image frames, while calculating the variance value of the same part up to a reference image frame and a predetermined number of future image frames. By calculating the variance value of the brightness, if the calculated variance value satisfies a predetermined condition, it is determined that the moving state and others are static states, and the moving amount / static state of the pixels of the entire image is normalized to extract the feature amount. An object was detected.
This image processing makes it possible to detect an object without placing a heavy burden on the CPU to be calculated even when a suspicious person or the like is located far away from the camera, and can be applied to a security camera installed in a general household. It was possible.

JP 2013-190943 A

However, when this technology is applied to a night vision camera equipped with infrared illumination that can be monitored even at night, rain during night vision is captured as white streaks, so when determining the pixel status, it is close to the night vision camera. In some cases, the rain picked up by the camera is erroneously determined as a moving state of a person or the like.
For monitoring by a surveillance camera, nighttime monitoring is often important, and such a malfunction is not preferable.

  Therefore, in view of such a problem, the present invention causes a rain streak reflected in a captured image to be erroneously recognized as a moving object such as a person even when an object is detected from an image captured by a night vision camera. The object of the present invention is to provide an object detection device that reduces the number of objects.

In order to solve the above problems, an object detection apparatus according to a first aspect of the present invention includes an image frame group generation unit that generates a group of image frames continuous on a time axis from a captured image of a camera, and a plurality of generated image frames Any image frame is used as a reference frame, and the variance value is calculated by calculating the amount of change in luminance of the same part for each pixel from the reference frame to a predetermined number of past image frames or a predetermined number of future image frames. A dispersion value calculation means, a pixel state determination means for comparing the obtained dispersion value with a predetermined threshold value to determine a static state / motion state for each pixel, and a cell composed of a plurality of neighboring pixels are formed in a reference frame. A feature amount calculation means for creating a histogram composed of binary values of static state / motion state for each cell and normalizing the histogram to calculate a feature amount of motion; An object detection means for detecting the presence of a moving object from a strong classifier using a learning sample and a variance value calculating means for calculating a luminance value from a reference frame to a predetermined number of past image frames. A luminance variance value (first variance value) based on the change amount, a luminance variance value (second variance value) based on the luminance change amount from the reference frame to a predetermined number of future image frames, and the immediately preceding reference frame A luminance variance value (third variance value) based on a luminance change amount from a predetermined number of image frames to a predetermined number of past image frames, and a luminance change amount from an image frame immediately after the reference frame to a predetermined number of future image frames While the pixel state determination unit calculates the luminance variance value (fourth variance value) based on the first and second variance values, and the third variance value and the fourth variance value Out of At least one of determining the larger pixel than the threshold value and the dynamic state, the dispersion value and judging the pixels of other values and static state.
According to this configuration, even if rain or the like appears in the reference frame as white streaks, an image taken with a night vision camera is used to determine the presence of an object in consideration of past image data and future image data excluding this image. Even when an object is detected from the image, it is possible to reduce misjudgment that a white line due to rain is determined as a moving object such as a person.

An object detection apparatus according to a second aspect of the present invention is an image frame group generation unit that generates a group of image frames continuous on a time axis from a captured image of a camera, and an arbitrary image frame among a plurality of generated image frames as a reference Luminance calculation means for calculating the maximum value of the change in luminance of the same part for each pixel from the reference frame to a predetermined number of past image frames or a predetermined number of future image frames as a frame, and the obtained luminance change amount A pixel state determination unit that determines a static state / motion state for each pixel by comparing the maximum value of the pixel with a predetermined threshold value, and a cell composed of a plurality of neighboring pixels is formed in the reference frame to form a static state / motion for each cell. A feature amount calculating means for creating a histogram composed of binary values of states and normalizing the histogram to calculate a feature amount of motion; and a calculated feature amount of motion Object detection means for detecting the presence of a moving object from the strong classifier using the training sample, and the luminance calculation means has a maximum value of the luminance change amount from the reference frame to a predetermined number of past image frames (first Maximum luminance difference), the maximum luminance change amount from the reference frame to a predetermined number of future image frames (second maximum luminance difference), and the luminance from the image frame immediately before the reference frame to a predetermined number of past image frames. While calculating the maximum value of the variable (third maximum luminance difference) and the maximum value of the luminance change amount (fourth maximum luminance difference) from the image frame immediately after the reference frame to a predetermined number of future image frames, The state determination unit determines that a pixel in which the first and second maximum luminance differences are larger than a threshold and at least one of the third maximum luminance difference and the fourth maximum luminance difference is larger than the threshold is a moving state. Characterized in that the maximum luminance difference is determined for other values the pixels of the static state.
According to this configuration, even if rain or the like appears in the reference frame as white streaks, an image taken with a night vision camera is used to determine the presence of an object in consideration of past image data and future image data excluding this image. Even when an object is detected from the image, it is possible to reduce misjudgment that a white line due to rain is determined as a moving object such as a person.

  According to the present invention, even if rain or the like appears in the reference frame as white streaks, an image taken with a night vision camera is used to determine the presence of an object by taking into account past image data and future image data excluding this image. Even when an object is detected from the image, it is possible to reduce misjudgment that a white line due to rain is determined as a moving object such as a person.

It is a functional lineblock diagram showing an example of an object detection device concerning the present invention. It is explanatory drawing of the image frame group which shows the relationship between an image frame and a brightness | luminance. It is explanatory drawing which shows the flow of the calculation which detects a moving object in an image, and has shown the case where rain is imaged with the night vision camera with the white line. FIG. 4 is an enlarged view of a reference frame in FIG. 3. FIG. 4 is an enlarged view of an M4 image in FIG. 3. It is explanatory drawing corresponding to FIG. 3, and has shown the detection of the person imaged with natural light.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments embodying the present invention will be described below in detail with reference to the drawings.
<First form>
FIG. 1 is a functional configuration diagram showing a first embodiment of an object detection apparatus according to the present invention, in which an image frame composed of continuous images on a time axis at a predetermined interval is inputted with a video signal captured by a camera 1a. An image output unit 1 that generates and outputs, a time change extraction unit 2 that extracts a time change amount of luminance for each pixel using an arbitrary image frame as a reference frame, and a frame that calculates a feature amount that becomes a difference feature between frames for each pixel An inter-feature extraction unit 3, a detection unit 4 that detects an object in the image frame from the learning sample and the calculated feature amount, and a result output unit 5 that outputs a detection result are provided.
Of these components, the image output unit 1, the time change extraction unit 2, the interframe feature extraction unit 3, and the detection unit 4 constitute an object detection device that detects a moving object, and the CPU or DSP executes a predetermined program. It is realized by executing. The whole including the result output unit 5 can be realized by a personal computer, for example.

  Hereinafter, the operation of each unit will be described. Here, the night vision camera 10 that is equipped with infrared illumination and can be monitored at night is used as the camera 1a, and the image output unit 1 receives an image captured by the night vision camera 10, and is composed of digital data every 0.03 seconds, for example. Generate and output an image frame (still image).

The time change extraction unit 2 calculates a luminance dispersion value S from an arbitrary pixel of the image frame output by the image output unit 1. This operation is finally performed on all pixels in the frame.
FIG. 2 is an explanatory diagram of an image frame group showing the relationship between the image frame and the luminance. The reference frame is K, and the past image frame that is the image frame output immediately before the reference frame K and the image frame that is output immediately after the reference frame K are illustrated. The relationship of the future image frame which is an image frame is shown. The luminance of a pixel at an arbitrary coordinate in the reference frame K is denoted by Ik, and the luminance at the same coordinate in each image frame is denoted by Ik (± n). Hereinafter, a description will be given with reference to FIG.

The calculation for obtaining the variance value S of the luminance of the pixels includes a first variance value S1 from the reference frame K to N past image frames K-N, and a second variance from the reference frame K to N future image frames K + N. A value S2, a third variance value S3 from the immediately preceding image frame K-1 to N past image frames KN, and a fourth variance value from the immediately following image frame K + 1 to N future image frames K + N. Four values of S4 are calculated.
Equation 1 shows an arithmetic expression for the luminance dispersion value S, and represents an expression for calculating the above-described first dispersion value S1 as a representative. In Equation 1, j is an integer from 1 to n.

Next, threshold processing is performed on the obtained first to fourth variance values S1 to S4 of each pixel to determine a static state / moving state. If the first variance value S1 and the second variance value S2 are equal to or greater than a preset threshold value and one of the third variance value S3 and the fourth variance value S4 is equal to or greater than the threshold value, the moving state is determined. Is determined as a static state.
Further, the time change extraction unit 2 outputs luminance information binarized (two gradations) for each pixel in this way.

 Note that the threshold value may be a fixed value set in advance, but in the case of a fixed value, the determination varies greatly depending on the installation environment and brightness of the camera 1a. It is preferable to determine an appropriate value based on the group.

FIG. 3 is an explanatory diagram specifically showing the flow of calculation for extracting a moving object as an image. The above calculation shows a state in which rain appears as a white line on the reference frame K of the image captured by the night vision camera 10. The state of being deleted is specifically shown. Here, the moving object is a person.
The center image row M1 is a captured image of the night vision camera 10, and shows the image frames from the past image frame K-4 to the future image frame K + 4 of the four images centering on the reference frame K, and the luminance of the pixels in the meantime The flow which determines a static state / dynamic state from information is shown.
The upper image row M2 and the lower image row M3 indicate the static state / moving state obtained from the variance value of each pixel as an image, M2a is the first variance value S1, and M2b is the second variance value S2, M3a. Is an image corresponding to the third dispersion value S3, and M3b is an image corresponding to the fourth dispersion value S4.

The image M2c shows the result of ANDing the image M2a consisting of the first variance value S1 and the image M2b consisting of the second variance value S2, and the image M3c is the image M3a consisting of the third variance value S3 and the fourth The result of performing OR processing with the image M3b composed of the variance value S4 is shown.
Furthermore, the image M4 shows the AND processing result of the image M2c and the image M3c. 4 is an enlarged view of the image of the reference frame K, and FIG. 5 is an enlarged view of the M4 image.

Based on the static state / moving state information for each pixel thus obtained, the inter-frame feature extraction unit 3 performs a predetermined calculation to calculate the feature amount. The inter-frame feature extraction unit 3 creates a cell composed of a plurality of adjacent pixels, creates a static / moving state histogram for each pixel for each cell, and forms the histogram by a plurality of adjacent cells. To calculate the feature amount (not detailed).
One cell is composed of, for example, 5 × 5 pixels, and one block is composed of, for example, 3 × 3 cells.

  The detection unit 4 calculates a learning sample classified according to a person and a background to detect a person, or a learning sample classified according to a vehicle such as a bicycle or an automobile and a background to detect a vehicle. The object is detected by using a well-known cascade booster that is composed of a strong classifier that has been learned using the feature amount. As a result, when a moving object such as a person or a vehicle is detected, an object detection signal including object coordinate information, region information, and the like is output.

  The result output unit 5 includes a sound report unit that receives an object detection signal output from the detection unit 4 and reports a warning, a video display unit such as an LCD that displays the detected video, a report unit that reports to the outside, and the like. In response to the coordinate information and the like output from the detection unit 4, the video display unit displays a detection window (not shown) for detecting a person in addition to displaying the input video.

  In this way, even if rain or the like appears in the reference frame K as white streaks, the image taken by the night vision camera 10 is used to determine the presence of the object in consideration of the data of the past image and the future image excluding this image. Even when an object is detected from the image, it is possible to reduce misjudgment that a white line due to rain is determined as a moving object such as a person.

<Second Embodiment>
In the above embodiment, the luminance dispersion value is obtained to determine the static state / moving state for each pixel. However, the static state / moving state may be determined based on the maximum value of the luminance change amount for each pixel. This form will be described below. The configuration of the object detection apparatus is the same as that of FIG.
Hereinafter, the operation of each unit will be described. Since the operation of the time change extraction unit 2 is greatly different from the above form, the operation of the time change extraction unit 2 will be mainly described.

The time change extraction unit 2 extracts a luminance time change amount from an arbitrary pixel of the image frame output by the image output unit 1. This operation is finally performed on all pixels in the frame.
As shown in FIG. 2 above, paying attention to the reference frame K, the calculation is performed between the past N-th image frames and the future N-th image frames. Perform detection. The detection of pixels whose luminance changes rapidly is performed between the frames up to the past N frames centering on the reference frame K and the frames up to the future of the N frames.
For example, assuming that the maximum amount of change in luminance between the luminances Ik-n and Ik is the variation amount T, the variation amount T is calculated by the following equation (2). In Equation 2, j is an integer from 1 to n.

  However, the pixel brightness change amount (maximum change amount) T is calculated from the reference frame K to the N previous image frames K-N (first change amount) T1, and from the reference frame K to the future N. The second change amount T2 from the previous image frame K + N, the third change amount T3 from the previous image frame K-1 to the N previous image frames K-N, and the next image frame K + 1 to the N future images. Four values of the fourth variation T4 up to the frame K + N are calculated.

Next, threshold processing is performed on the obtained first to fourth change amounts T1 to T4 of each pixel to determine a static state / moving state. If the first change amount T1 and the second change amount T2 are equal to or greater than the threshold value and one of the third change amount T3 and the fourth change amount T4 is equal to or greater than the threshold value, the moving state is determined. Is determined.
Then, the time change extraction unit 2 outputs luminance information binarized for each pixel (with two gradations).

 Note that the threshold value may be a fixed value set in advance, but in the case of a fixed value, the determination varies greatly depending on the installation environment and brightness of the camera 1a. It is preferable to determine an appropriate value based on the above.

Based on the static state / moving state information for each pixel thus obtained, the inter-frame feature extraction unit 3 performs a predetermined calculation to calculate the feature amount. The inter-frame feature extraction unit 3 creates a cell composed of a plurality of adjacent pixels as in the above-described form, creates a static / moving state histogram for each pixel for each cell, and uses this histogram as a plurality of adjacent cells. The feature amount is calculated by normalization in units of blocks formed in (not detailed).
The operations of the detection unit 4 and the result output unit 5 are the same as those in the above embodiment, and the detection unit 4 detects an object using a cascade type discriminator by Adaboost, and the object detected by the result output unit 5 is a detection window or the like. Is displayed.

  In this way, even if rain or the like appears in the reference frame K as white streaks, the image taken by the night vision camera 10 is used to determine the presence of the object in consideration of the data of the past image and the future image excluding this image. Even when an object is detected from the image, it is possible to reduce misjudgment that a white line due to rain is determined as a moving object such as a person.

In the above embodiment, the night vision camera 10 is used as the video signal acquisition source, but the technique of the present invention is a normal surveillance camera that can be imaged by sunlight or night illumination. FIG. 6 shows a flow of detecting a person from a captured image by natural light, and shows an image corresponding to FIG.
In addition, the calculation is performed using data of image frames that are continuous with the reference frame K as the center, but discontinuous data such as only data of odd frames may be used. it can.
Furthermore, although the image output unit 1 is separate from the camera 1a, the image output unit 1 may be integrated with the camera 1a, or the configuration up to the detection unit 4 may be integrated with the camera 1a.

  1. Image output unit (image frame group generating unit), 1a, camera, 2. Time change extracting unit (luminance calculating unit, variance value calculating unit, pixel state determining unit), 3. Inter-frame feature extracting unit (Characteristic amount calculation means), 4 .... detection unit (object detection means), 5..result output unit, 10..night vision camera, S..luminance dispersion value, T..maximum change amount of luminance.

Claims (2)

Image frame group generation means for generating image frame groups continuous on the time axis from the captured image of the camera;
Using an arbitrary image frame among a plurality of generated image frames as a reference frame, the amount of change in luminance of the same part for each pixel from the reference frame to a predetermined number of past image frames or a predetermined number of future image frames is calculated. Dispersion value calculating means for calculating the dispersion value
A pixel state determination unit that compares the obtained dispersion value with a predetermined threshold to determine a static state / motion state for each pixel;
In the reference frame, a cell composed of a plurality of neighboring pixels is formed, a histogram composed of binary values of the static state / moving state is generated for each cell, and the histogram is normalized to calculate a feature amount of motion A calculation means;
An object detection means for detecting the presence of an object moving from the strong classifier using the calculated feature quantity of the motion and the learning sample;
The variance value calculation means includes a luminance variance value (first variance value) based on a luminance change amount from the reference frame to a predetermined number of past image frames, and
A luminance variance value (second variance value) based on a luminance change amount from the reference frame to a predetermined number of future image frames;
A luminance variance value (third variance value) based on a luminance change amount from an image frame immediately before the reference frame to the predetermined number of past image frames;
Further, while calculating a luminance variance value (fourth variance value) based on the luminance change amount from the image frame immediately after the reference frame to the predetermined number of future image frames,
The pixel state determination unit moves a pixel in which the first and second variance values are larger than the threshold value and at least one of the third variance value and the fourth variance value is larger than the threshold value. An object detection apparatus characterized by determining a state and determining a pixel having a dispersion value other than that as a static state. Image frame group generation means for generating image frame groups continuous on the time axis from the captured image of the camera;
The maximum value of the amount of change in the luminance of the same part for each pixel from the reference frame to a predetermined number of past image frames or a predetermined number of future image frames from any of the generated image frames as a reference frame Brightness calculating means for calculating
A pixel state determination unit that determines a static state / moving state for each pixel by comparing the obtained maximum value of the amount of change in luminance with a predetermined threshold;
In the reference frame, a cell composed of a plurality of neighboring pixels is formed, a histogram composed of binary values of the static state / moving state is generated for each cell, and the histogram is normalized to calculate a feature amount of motion A calculation means;
An object detection means for detecting the presence of an object moving from the strong classifier using the calculated feature quantity of the motion and the learning sample;
The luminance calculation means, a maximum value of the luminance change amount from the reference frame to a predetermined number of past image frames (first maximum luminance difference),
A maximum value of the amount of change in luminance from the reference frame to a predetermined number of future image frames (second maximum luminance difference);
A maximum value of the luminance variable (third maximum luminance difference) from the image frame immediately before the reference frame to the predetermined number of past image frames;
While calculating the maximum value (fourth maximum luminance difference) of the luminance change amount from the image frame immediately after the reference frame to the predetermined number of future image frames,
The pixel state determining means has the first and second maximum luminance differences larger than the threshold, and at least one of the third maximum luminance difference and the fourth maximum luminance difference is larger than the threshold. An object detection apparatus characterized in that a pixel is determined to be in a moving state, and a pixel having a value other than the maximum luminance difference is determined to be a static state.

Images