JP3465632B2 - Object detection device and object detection method - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an object detecting device and an object detecting method for detecting an object included in an image taken by a surveillance camera or the like.

[0002]

2. Description of the Related Art When detecting an object from a moving image photographed by a surveillance camera or the like, it is important to be able to detect the object reliably even if the brightness of the background changes due to sunshine conditions or the like.
Therefore, the inventors of the present invention have devised an object detection technique that is less susceptible to the influence of the illumination variation even when it frequently occurs, and have already applied for a patent thereof (Japanese Patent Application No. 11-078641). ).

In this object detection technique, a reference point is provided at a position different from the point of interest in the background image created in advance and the current image, and the object is discriminated from the correlation information of the brightness value change of the reference point. It is a thing. Hereinafter, the principle will be described in detail with reference to the drawings. FIG. 8 is an explanatory diagram showing a background image, in which the X axis is in the horizontal direction and the Y axis is in the vertical direction.
Each axis is set. However, in order to avoid unnecessarily complicated images, the background image has no texture and has a constant brightness regardless of location. In FIG. 8, P _A is a point of interest and P _B is a reference point provided in the vicinity thereof. Then, the brightness value at the attention point P _A is I _A , and the reference point P A
The luminance values in _B and I _B.

FIG. 9 shows the brightness values of the target point and the reference point.
It is explanatory drawing which shows the two-dimensional brightness | luminance space formed. this
A two-dimensional coordinate system is set in the luminance space, and the horizontal axis is
Note P_ABrightness value of_ARepresents the reference point P on the vertical axis._B
Brightness value of_BIs represented. Here, attention point P_ABrightness value
Is I_A1, Reference point P_BBrightness value of I_B1, Then
Points in the luminance space corresponding to these luminance value pairs,
That is, the luminance point (I _A1, I_B1) Is, for example, white in FIG.
This is the position of the dot 102. And the background brightness changes
Point of interest P_AIf the brightness value of
Then, the reference point P_BThe luminance value of is also reduced at the same rate. Well
Attention point P_AIs 0, the reference point P_BThe brightness value of
It becomes 0. Therefore, attention point P_AAnd the reference point P_BShine
Degree I_A, I_BThe points in the luminance space corresponding to the
When the ruggedness changes, along the straight line 104 shown in FIG.
Moving. Here, such a straight line is
It is called a scenery model.

On the other hand, for example, the object has entered the position of the attention point P _A, the luminance value of the attention point P _A is the dropped to I _A2, the luminance point in this case (I _A2, I _B1), for example white circle dots The position is 106. Therefore, by checking the distance 107 between the brightness point and the straight line 104, when the brightness point is far from the straight line 104, it can be determined that the object has entered the position of the target point. Then, if illumination fluctuation occurs and, for example, the brightness value of the reference point P _B becomes large, the brightness value of the target point P _A on the object also becomes large in the same manner, so that the brightness points (I _A2 , I _B1 ) become Move to the upper right direction (arrow A) in parallel with the straight line 104, and straight line 1
The distance to 04 does not change much. Therefore, it is possible to reliably detect the object even if the illumination changes.

[0006]

However, in the above-mentioned prior art, when the reference point is arranged close to the point of interest, only the contour portion of the object is extracted, and the extracted object image is in a hollow state. , There was room for improvement in this respect. Hereinafter, a specific description will be given with reference to FIG. FIG. 10 is an explanatory diagram showing a captured image when an object enters the capturing area of the surveillance camera, for example. As in the case of FIG. 8, the background has a constant brightness regardless of the place. For this image,
As shown in FIG. 10, if the attention point P _A is set and the reference point P _B is set close to each other, all of these points are on the object 108 in the state of FIG. 10, and the brightness of both points is set. together to lower to the same extent, the point of interest P _a and the reference point P _B corresponding luminance points in the luminance space (I _a, I _B) is,
This is the position of the black dot 110 on the straight line 104 shown in FIG. Since the distance between the black dot 110 and the straight line 104 is zero, it is determined that the attention point P _A is not on the object in this case, and the attention point P _A is on the object 108. However, the object cannot be detected.

On the other hand, even when the attention point P _A and the reference point P _B are set at the same interval, for example, the attention point P _A is on the object 108 and the reference point P _B is on the background at the edge of the object 108. When each point is set at the position, the luminance difference occurs between the two points, and the corresponding luminance point is located away from the straight line 104 in FIG. 9, so that the object 108 can be detected. Therefore, in the technique of Japanese Patent Application No. 11-078641, when the target point and the reference line are arranged close to each other, only the edge portion of the object, that is, the contour portion is detected.

Therefore, in order to detect not only the outline of the object but the whole, it is necessary to dispose the reference point sufficiently away from the target point. Further, in the technique of Japanese Patent Application No. 11-078641, the reference point must be set outside the area where an object may enter. About this
This will be described with reference to FIG. 11A is an explanatory diagram showing the image surface, FIG. 11B is a schematic diagram when an object enters the photographing area, and FIG. 11C is a schematic diagram showing a detection result.

As shown in FIG. 11A, when the attention point P _A and the attention point P _B are set apart from each other on the image, as shown in FIG.
As shown in the (B), when an object 108 enters the position of the target point P _A, the object 108 generated luminance difference between the reference point P _B and the point of interest P _A is detected. However, the object 108
A similar brightness difference also occurs when the object enters the position of the reference point P _B , so that the object 108 is also detected in this case. Therefore, when the object 108 invades the attention point P _A and is detected, it is not possible to specify whether the object 108 exists at the position of the attention point P _{A or} the position of the reference point P _B. As a result, although the object 108 exists only at the position of the attention point P _A , the object image 109 is generated such that the object 108 exists at both positions as shown in FIG. 11C. Will be done.

Therefore, in order to avoid such a phenomenon,
The reference point must always be set outside the area where the object may enter. The present invention has been made to eliminate such drawbacks, and its purpose is to detect an object with high accuracy and with stability even when the illumination fluctuation is large and frequently occur, and to determine a reference point. It is an object to provide an object detection device and an object detection method that can set an arbitrary position to reliably detect an object.

[0011]

In order to achieve the above-mentioned object, the present invention is an object detecting apparatus for detecting an object using a photographed image, in which the brightness value of each pixel of the photographed image is captured and held. An image input unit and a plurality of reference pixels that select a plurality of reference pixels for each target pixel of the captured image and obtain the brightness values of the selected plurality of reference pixels and the target pixel from the image input unit for each target pixel Point selection means,
Background image holding means for holding the luminance value of each pixel of the background image forming the background of the captured image, and the target pixel selected by the plurality of reference point selection means and the plurality of reference pixels corresponding to the same pixel respectively The luminance value of a plurality of pixels in the background image is acquired from the background image holding means, and the illumination variation background model when the background luminance changes under the influence of illumination variation is the same pixel as the pixel of interest. A background model creating means for creating each set with each reference pixel corresponding to, in the luminance space, the background model creating means creates the illumination variation background model for each set of the target pixel and the reference pixel, The distance between each of the brightness values of the plurality of reference pixels acquired for each of the target pixels by the plurality of reference point selecting means and the point represented by the set of the brightness values of the target pixel, respectively. For each set of the distance calculating means for calculating and the target pixel and each reference pixel, it is determined whether or not the target pixel is a pixel representing an object based on the distance calculated by the distance calculating means. And an object discriminating means for finally discriminating whether or not the target pixel is a pixel representing an object.

In the object detecting apparatus of the present invention, the image input means captures and holds the luminance value of each pixel of the photographed image, and the plural reference point selecting means selects a plurality of reference pixels for each target pixel of the image. Then, the brightness values of the selected plurality of reference pixels and the target pixel are acquired for each target pixel from the image input means. Then, the background model creating means obtains from the background image holding means brightness values of a plurality of pixels in the background image respectively corresponding to the target pixel selected by the plurality of reference point selecting means and the plurality of reference pixels corresponding to the same pixel. An illumination variation background model in the case where the background brightness is changed by being affected by the illumination variation is created for each set of the target pixel and each reference pixel corresponding to the same pixel.

Further, the distance calculation means is configured such that, in the brightness space, the illumination variation background model created by the background model creation means for each set of the target pixel and the reference pixel, and the plurality of reference point selection means for each target pixel. The distances between the points represented by the respective sets of the brightness values of the plurality of reference pixels and the brightness value of the target pixel acquired in step S1 are calculated. Then, the object discriminating means discriminates, for each group of the target pixel and each reference pixel, based on the distance calculated by the distance calculating means, whether the target pixel is a pixel representing an object or not, and discriminating for each group. A logical operation is performed on the result to finally determine whether the pixel of interest is a pixel representing an object.

Further, the present invention is an object detecting device for detecting an object using a photographed image, comprising image input means for fetching and holding a luminance value of each pixel of the photographed image, and a horizontal direction in the photographed image. Horizontal reference point selection for selecting a plurality of reference pixels included in the target horizontal pixel row and acquiring the brightness value of each reference pixel from the image input means for each target horizontal pixel row composed of a plurality of pixels arranged in And a plurality of reference pixels included in the target vertical pixel row for each target vertical pixel row composed of a plurality of pixels arranged in the vertical direction in the captured image, and the brightness value of each reference pixel is set to the image. Vertical reference point selecting means acquired from the input means, background image holding means for holding the luminance value of each pixel of the image forming the background of the photographed image, and the plurality of reference points selected by the horizontal reference point selecting means. A horizontal background that obtains luminance values of a plurality of pixels in the background image, which correspond to pixels, from the background image holding means and creates an illumination variation background model when the background luminance changes under the influence of illumination variation. The brightness values of the plurality of pixels in the background image, which correspond respectively to the plurality of reference pixels selected by the model creating means and the vertical reference point selecting means, are acquired from the background image holding means, and the background brightness is changed by illumination. Vertical background model creating means for creating an illumination fluctuation background model in the case of a change due to the influence of, and the illumination fluctuation background model created by the horizontal background model creating means in a multidimensional luminance space, and the horizontal reference point selection. Horizontal distance calculation means for calculating a distance between the points represented by the plurality of brightness values acquired by the means, and the vertical background model in the multidimensional brightness space. Vertical distance calculating means for calculating the distance between the illumination fluctuation background model created by the creating means and the points represented by the plurality of brightness values acquired by the vertical reference point selecting means, the horizontal distance calculating means, and Based on the distance calculated by the vertical distance calculating means, it is determined whether or not the target horizontal pixel row and the target vertical pixel row include a pixel representing an object. And an object discriminating means for performing a logical operation to specify a rectangular area including the object.

In the object detecting device of the present invention, the image input means captures and holds the luminance value of each pixel of the photographed image, and the horizontal reference point selecting means is composed of a plurality of pixels arranged in the horizontal direction in the photographed image. For each target horizontal pixel row, a plurality of reference pixels included in the target horizontal pixel row are selected and the brightness value of each reference pixel is acquired from the image input means. Further, the vertical reference point selecting means selects a plurality of reference pixels included in the target vertical pixel row for each target vertical pixel row composed of a plurality of pixels arranged in the vertical direction in the captured image, and determines the brightness of each reference pixel. The value is obtained from the image input means. Then, the horizontal background model creating means acquires the brightness values of a plurality of pixels in the background image corresponding to the plurality of reference pixels selected by the horizontal reference point selecting means from the background image holding means and determines the background brightness. An illumination variation background model is created when it is changed under the influence of the illumination variation, and the vertical background model creating means corresponds to the plurality of reference pixels selected by the vertical reference point selecting means. The brightness value of the pixel is acquired from the background image holding means, and an illumination fluctuation background model is created when the background brightness changes under the influence of the illumination fluctuation.

After that, the horizontal distance calculating means, in the multi-dimensional brightness space, defines the illumination fluctuation background model created by the horizontal background model creating means and the points represented by the plurality of brightness values acquired by the horizontal reference point selecting means. The vertical distance calculation means calculates the distance between the points, and the point represented by the illumination fluctuation background model created by the vertical background model creation means and the plurality of brightness values acquired by the vertical reference point selection means in the multidimensional brightness space. Calculate the distance between. Then, the object discriminating means discriminates based on the distance calculated by the horizontal distance calculating means and the vertical distance calculating means whether or not the target horizontal pixel row and the target vertical pixel row include a pixel representing an object, A logical operation is performed on the discrimination result relating to both pixel rows to specify a rectangular area including an object.

Further, the present invention is an object detecting method for detecting an object using a photographed image, comprising an image input step of fetching a luminance value of each pixel of the photographed image and holding the luminance value in the first storage means. A plurality of reference point selecting step of selecting a plurality of reference pixels for each target pixel of the captured image and acquiring the brightness values of the selected plurality of reference pixels and the target pixel from the first storage means for each target pixel A background image holding step of holding the luminance value of each pixel of the background image forming the background of the captured image in the second storage means, and the target pixel and the same pixel selected in the plurality of reference point selecting steps. The brightness values of the plurality of pixels in the background image corresponding to the plurality of reference pixels are acquired from the second storage unit, and the background brightness is changed by the influence of illumination fluctuation. And the target pixel illumination variation background model case, the background model creation step of creating for each set and each reference pixel corresponding to the same pixel, in the luminance space,
The illumination fluctuation background model created for each of the set of the target pixel and the reference pixel in the background model creating step,
A distance calculation step of calculating a distance between each of the luminance values of the plurality of reference pixels acquired for each of the attention pixels in the plurality of reference point selection steps and a point represented by a set of the luminance values of the attention pixel; For each set of the target pixel and each reference pixel, it is determined whether or not the target pixel is a pixel representing an object based on the distance calculated in the distance calculation step, and the determination result for each set is logically determined. And an object determining step for finally determining whether or not the pixel of interest is a pixel representing an object by performing a calculation.

[0018]

[0019]

[0020]

[0021]

Further, the present invention is an object detecting method for detecting an object using a photographed image, comprising an image input step of fetching a luminance value of each pixel of the photographed image and storing the luminance value in the first storage means. For each target horizontal pixel row composed of a plurality of pixels arranged in the horizontal direction in the captured image, a plurality of reference pixels included in the target horizontal pixel row are selected, and the brightness value of each reference pixel is stored in the first storage. A horizontal reference point selecting step obtained from the means, and a plurality of reference pixels included in the target vertical pixel row are selected for each target vertical pixel row composed of a plurality of pixels arranged in the vertical direction in the captured image. Vertical reference point selecting step of obtaining the brightness value of a reference pixel from the first storage means, and background image holding for holding the brightness value of each pixel of the image forming the background of the photographed image in the second storage means Step, and the brightness values of a plurality of pixels in the background image corresponding to the plurality of reference pixels selected in the horizontal reference point selecting step are acquired from the second storage unit to reduce the background brightness to the illumination variation. Horizontal background model creating step of creating an illumination variation background model when affected and changed, and brightness of a plurality of pixels in the background image respectively corresponding to the plurality of reference pixels selected in the vertical reference point selecting step A vertical background model creating step of obtaining a value from the second storage means and creating an illumination fluctuation background model when the background brightness changes under the influence of the illumination fluctuation; and the horizontal background in the multidimensional brightness space. Of the illumination fluctuation background model created in the model creating step and the points represented by the plurality of brightness values acquired in the horizontal reference point selecting step A horizontal distance calculation step of calculating the distance, in a multidimensional intensity space,
A vertical distance calculation step of calculating a distance between the illumination fluctuation background model created in the vertical background model creation step and a point represented by the plurality of brightness values acquired in the vertical reference point selection step, and the horizontal distance. Based on the distance calculated in the calculation step and the vertical distance calculation step, it is determined whether or not the target horizontal pixel row and the target vertical pixel row include a pixel representing an object, and a determination relating to both pixel rows. And an object determining step of performing a logical operation on the result to identify a rectangular area including the object.

In the object detecting method of the present invention, the brightness value of each pixel of the photographed image is fetched and held in the first storage means in the image input step, and is arranged in the horizontal direction in the photographed image in the horizontal reference point selecting step. A plurality of reference pixels included in the target horizontal pixel row are selected for each target horizontal pixel row including a plurality of pixels, and the brightness value of each reference pixel is acquired from the first storage unit. Further, in the vertical reference point selecting step, the plurality of reference pixels included in the target vertical pixel row are selected for each target vertical pixel row composed of a plurality of pixels arranged in the vertical direction in the captured image, and the brightness of each reference pixel is selected. The value is acquired from the first storage means. Then, in the horizontal background model creating step, the brightness values of the plurality of pixels in the background image respectively corresponding to the plurality of reference pixels selected in the horizontal reference point selecting step are acquired from the first storage unit to obtain the background brightness. Creates an illumination variation background model in the case where it changes due to the influence of illumination variation, and in the vertical background model creating step, a plurality of background images corresponding to the plurality of reference pixels selected in the vertical reference point selecting step are provided. The luminance value of the pixel is acquired from the second storage means and an illumination fluctuation background model is created when the background luminance changes under the influence of the illumination fluctuation.

Then, in the horizontal distance calculation step, in the multidimensional brightness space, the illumination fluctuation background model created in the horizontal background model creation step and the points represented by the plurality of brightness values acquired in the horizontal reference point selection step are defined. In the vertical distance calculation step, in the vertical distance calculation step, the points represented by the illumination fluctuation background model created in the vertical background model creation step and the plurality of brightness values acquired in the vertical reference point selection step. Calculate the distance between. Then, in the object determining step, based on the distance calculated in the horizontal distance calculating step and the vertical distance calculating step, it is determined whether or not the target horizontal pixel row and the target vertical pixel row include a pixel representing an object, A logical operation is performed on the discrimination result relating to both pixel rows to specify a rectangular area including an object.

[0025]

BEST MODE FOR CARRYING OUT THE INVENTION Next, embodiments of the present invention will be described with reference to the drawings. First, the principle of object detection in the first embodiment will be described. 3A is an explanatory diagram showing a positional relationship between a target point and a reference point on the image surface, FIG. 3B is a schematic diagram when an object enters the photographing region, and FIG. 3C is a schematic diagram showing a detection result. , (D) is another explanatory diagram showing the positional relationship between the target point and the reference point on the image surface, (E) is another schematic diagram showing the detection result, and (F) is a schematic diagram showing the final detection result. Is.

As shown in FIG. 3A, when the attention point P _A and the attention point P _B are set apart from each other on the image, as shown in FIG. and enters the position of the attention point P _a, the luminance difference between the reference point P _B and the point of interest P _a occurs, detects the object 108 performs detection in the same manner as conventional using the illumination variation background model . Then, as shown in FIG. 3C, the detection result indicates that the object 108 may exist at both the attention point P _A and the reference point P _B as in the conventional case.

On the other hand, in this embodiment, (D) of FIG.
As shown in, and sets a different reference point P _C is the reference point P _B. Then, with respect to the set of the attention point P _A and the reference point P _C , the object is detected by the same method as the conventional one. As a result, when the object 108 invades the attention point P _A as described above, the detection result shows that the object 108 is present at both the attention point P _A and the reference point P _C , as shown in FIG. Is likely to be present.

In the present embodiment, the two detection results, that is, the logical product of (C) and (E) in FIG. Accordingly, as shown in (F) in FIG. 3, leaving only the object image 111 in the position of the attention point P _A that is present commonly in both of Figures 3 and (C) (E), point of interest P _A The object 108 that has entered the vehicle is specified and reliably detected. Further, in the present embodiment, the attention point and the reference point can be arranged to be apart from each other to some extent, so that the image of the detected object is an image in which not only the contour portion but also the inner region is drawn.

Generally, when N different reference points (N is an integer of 3 or more) including the target point are set for one target point, the object The existence area can be specified.

[0030]

## EQU1 ## dist (P _i , P _j )> ObjectSize, and 1 ≦ i, j ≦ N (i ≠ j), and dist
(A, B) represents the distance between points A and B. Also, Obje
ctSize represents the size of the object on the image.

Next, a first embodiment of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing an object detecting apparatus according to the first embodiment, and FIG. 2 is a flow chart showing the operation of the object detecting apparatus shown in FIG. Hereinafter, an example of the object detection apparatus of the present invention will be described with reference to these drawings, and at the same time, an embodiment of a computer-readable information recording medium recording the object detection method and the object detection program of the present invention will be described. .

The object detecting apparatus 2 shown in FIG. 1 comprises an image input means 4, a plurality of reference point selecting means 6, a background image holding means 8,
The background model creating unit 10, the distance calculating unit 12, the object discriminating unit 14, and the background creating unit 16 are included, and the object is detected from an image captured by a surveillance camera, for example. Specifically, the object detection device 2 is, for example, provided with a monitor camera, an interface with a video tape recorder, and the like, and is configured by a computer in which a predetermined operating system is incorporated. A CD-R equipped with, for example, a CD-ROM as a computer-readable information recording medium recording the object detection program of the present invention
This is realized by reading the program data from the OM, loading the program data into the memory of the computer, and operating the CPU of the computer based on the program data.

The image input means 4 takes in an image captured by the surveillance camera, or an image recorded in the video tape recorder after the capture, from the surveillance camera or VTR, and holds it. More specifically, the image input means 4 captures and holds the brightness value of each pixel of the captured image. The image captured by the image input means 4 may be a moving image or a still image. Further, the image input unit 4 may be configured to include a camera that captures an image by itself.

The plurality of reference point selecting means 6 selects and selects a plurality of predetermined reference pixels (pixels at the reference point positions) for each target pixel (pixels at the position of the target points) of the image. The brightness values of the plurality of reference pixels and the target pixel are acquired from the image input unit 4 for each target pixel. In addition,
The plurality of reference pixels corresponding to one target pixel is also referred to as a reference pixel group. The background image holding unit 8 holds the brightness value of each pixel of the background image in the shooting area. In the present embodiment, the brightness value held by the background image holding means 8 is appropriately updated by the background creating means 16 as described later.
The background model creating means 10 stores the brightness values of a plurality of pixels in the background image, which correspond to the target pixel selected by the plurality of reference point selecting means 6 and the plurality of reference pixels corresponding to the same pixel, respectively, to the background image holding means 8 , And an illumination variation background model in the case where the background brightness changes due to the influence of illumination variation is created for each set of the target pixel and each reference pixel corresponding to the same pixel.

Then, the distance calculating means 12 has the illumination fluctuation background model created by the background model creating means 10 for each of the groups of the target pixel and the reference pixel in the luminance space,
The plural reference point selecting means 6 calculates the distance between each point represented by the set of the luminance values of the plurality of reference pixels acquired for each target pixel and the luminance value of the target pixel.

The object discriminating means 14 discriminates, for each set of the target pixel and each reference pixel, based on the distance calculated by the distance calculating means 12, whether or not the target pixel is a pixel representing an object. Finally, it is determined whether or not the target pixel is a pixel representing an object by performing a logical operation on the determination result for each set of the reference pixel and each reference pixel. Then, the image output unit 15 outputs the image data of the determination result output by the object determination unit 14 to a display device (not shown) or the like. Also, the background creating means 16
Is the luminance value of the pixel of interest and the pixel of the background image corresponding to each of the plurality of reference pixels corresponding to the pixel of interest determined as not being a pixel representing the object. It is updated based on the brightness value and held in the background image holding means 8.

Next, the operation of the first embodiment thus constructed will be described. As shown in FIG.
First, the image input unit 4 of the object detection device 2 reads an image captured by a surveillance camera or the like, and holds a brightness value for each pixel forming the image in a memory (not shown) provided in the image input unit 4. (Step A1). The image is composed of a large number of pixels arranged in a matrix, and the image input means 4 holds the luminance values of such pixels as two-dimensional array data. In addition, each pixel is x in the memory.
It can be specified by specifying the address and y address. The x address corresponds to the column number of the matrix, and y
The address corresponds to the row number of the matrix.

Then, the plurality of reference point selecting means 6 selects a plurality of predetermined reference pixels for each target pixel of the image and inputs the brightness values of the selected plurality of reference pixels and the target pixel as an image. Each pixel of interest is acquired from the memory of the means 4 (step A2). Each of the reference pixels is set at a position separated from the pixel of interest by a distance larger than the size of the intruding object to be detected. For example, if the size of the intruding object on the screen is less than 100 pixels and the current pixel of interest PA and another pixel point PB different from this pixel are used as reference points, (A) in FIG. ), The reference point PB is x from the point of interest PA.
It is set at a position separated by 100 pixels in the axial direction. At this time,
The brightness value at the attention point PA is I (x, y), and the reference point PB
The luminance value at can be expressed as I (x + 100, y). Similarly, when setting another reference point PC, as shown in FIG. 3D, the reference point PC is set at a position 100 pixels away from the target point PA in the direction opposite to the x-axis. The brightness value at the reference point PC is I (x-100,
y).

In the present embodiment, the plural reference point selecting means 6 has two reference points P _{B for} the point of interest P _{A in} addition to the point of interest.
It is assumed that P _C is set, and accordingly, one target pixel and two reference pixels respectively corresponding to the target point and each reference point are set. Next, the plural reference point selecting means 6 supplies the luminance values of the target pixel and the reference pixel obtained in step A2 to the background creating means 16 (step A).
3). At this stage, since the brightness value of each pixel forming the background image is not yet stored in the memory forming the background image holding unit 8, the background creating unit 16 is supplied from the multiple reference point selecting unit 6. The background image holding unit 8 sets each luminance value as the luminance value of the pixel of the corresponding background image.
To hold.

Next, the background model creating means 10 creates an illumination variation background model using the brightness value of each pixel of the background image held by the background image holding means 8. For example, if the target pixel is p _a corresponding to the target point P _A in FIG. 3 and the reference pixels are p _b and p _c corresponding to the reference points P _B and P _C , the background model creating means 10 first Pixel of interest p _a
And the luminance values of the two pixels in the background image corresponding to the reference pixel p _b are acquired from the background image holding means 8.

Then, using these two luminance values, as the one corresponding to the set of the target pixel p _a and the reference pixel p _b ,
An illumination fluctuation background model is created when the background brightness changes under the influence of lighting fluctuation. Specifically, as shown in FIG. 9, the horizontal axis the luminance value of the background pixel corresponding to the pixel of interest p _a, in the luminance space to the vertical axis the luminance value of the background pixel corresponding to the reference pixel p _b, A straight line passing through a point represented by a set of brightness values of background pixels corresponding to the target pixel p _a and the reference pixel p _b and the origin is obtained, and the straight line is set as the illumination variation background model. The background model creating means 10 similarly creates an illumination variation background model for a set of the target pixel p _a and the reference pixel p _c .

If the number of pixels to be referred to for each target pixel is N, including the target pixel, the brightness values of N background image pixels are used for each target pixel. The set of background luminance values for each pixel can be represented as an N-dimensional vector. In the present embodiment, two sets of the pixel of interest and one reference point are used, so the background brightness value is two sets of two-dimensional vectors.

After that, the distance calculating means 12 determines the target pixel p.
_{For the set of a} and the reference pixel p _b , in the brightness space, calculate the distance between the illumination fluctuation background model and the point represented by the set of brightness values of the pixel of interest p _a and the reference pixel p _b , Further, similar distances are calculated for the set of the target pixel p _a and the reference pixel p _c (step A4).

Here, the target pixel p _a , the reference pixels p _b and p _c
Let the brightness values of the background pixels corresponding to i _a0 , i _b0 , i _c0 ,
The luminance values of the pixel of interest p _a and the reference pixels p _b and p _c are i _a1 ,
Let i _b1 and i _{c 1} be the distance D ₁ between the illumination fluctuation background model corresponding to the set of the target pixel p _a and the reference pixel p _b and the point corresponding to the brightness values i _a1 and i _b1 , and The distance D ₂ between the illumination fluctuation background model corresponding to the set of the target pixel p _a and the reference pixel p _c and the points corresponding to the luminance values i _a1 and i _c1 is [Equation 2] and [Equation 4], respectively. It can be calculated by the formula shown in.

[0045]

[Equation 2]

[0046]

[Equation 3] Next, the object discriminating means 14 compares the distances D ₁ and D ₂ obtained in step A4 with preset threshold values T ₁ and T ₂ (step A5), and the distances and the thresholds are compared. When the value satisfies [Equation 4], the target pixel p _a is a pixel representing an object, and it is determined that the object exists at the corresponding position. On the other hand, when the value does not satisfy [Equation 4], the target pixel p a It is determined that _a is a pixel representing the background (steps A6, S7, S8, S9).

[0047]

[Equation 4] More specifically, in the object discriminating means 14, the distances D ₁ and D ₂ calculated by the plural distance calculating means are the threshold values T ₁ and
It is checked whether it is larger than T ₂ (step A5). For example, when the distance D ₁ is larger than the threshold value T ₁ , the object discriminating means 14 determines that the target pixel p _a and the reference pixel p _b may be pixels representing an object, and the detection result is logically determined by both pixels. Correlate "1". If the distance D _{2 is} also larger than the threshold value T ₂ , the object discriminating unit 14 determines that the target pixel p _a and the reference pixel p _c may be pixels representing an object, and both of them are detected as the detection result. The logic "1" is associated with the pixel. On the other hand, when the distances D ₁ and D ₂ are smaller than T ₁ and T ₂ , each pixel is associated with the logic “0”.

Then, the object discriminating means 14 determines whether these
The logical product of the detection results is taken (step A6), and the value is
If judged (step A7) and the result is logic "1",
Pixel of interest p_aIs a pixel representing an object (step A
8), if the logic is “0”, the pixel of interest p_aRepresents an object
Instead, it is assumed that the pixel represents the background (step A9). Was
For example, the distance D₁Is the threshold T₁If larger, object size
Another means 14 is each pixel p_a, P_b, P_cThe value associated with
1 ”,“ 1 ”,“ 0 ”, and the distance D₂Threshold
Value T₂If it is larger, the object discriminating means 14 determines that each pixel p_a,
p_b, P_cThe values associated with are "1", "0", and "1".
It Therefore, the logical product of these values is calculated for each pixel.
When taken, they become “1”, “0”, and “0”, respectively.
p_aOnly the logical value for is "1". Therefore,
The object discriminating means 14 determines the target pixel p. _aIs a pixel that represents an object
Judge that there is.

Generally, when N reference points other than the pixel of interest are taken, the distance D _i is calculated for each reference point and the threshold T _i for each reference point is used to satisfy [Equation 5]. Whether or not the target pixel is a pixel representing an object can be determined.

[0050]

[Equation 5] By performing the processing from steps A2 to S9 by setting all the pixels held by the image input unit 4 as the target pixel one after another (step A10), it is possible to detect the object on the captured image. it can. The detected object is output by the image output means 15 and displayed on the display device. At that time, in steps A7 and A8, if the display color of the pixel of interest that is determined to represent the object is displayed in a color different from the background, the shape of the detected object can be more clearly understood on the screen. .

When it is determined in step A10 that all pixels have been processed, the number of pixels determined to be an object is counted, and the count value is set to a preset threshold value. It may be determined that the object exists only when it is large. This makes it possible to detect only objects of a certain size or larger.

On the other hand, when it is determined in steps A7 and A9 that the pixel of interest is not a pixel representing an object, the background creating means 16 determines, based on the luminance values of the pixel of interest and a plurality of corresponding reference pixels. The brightness value of the pixel of the corresponding background image is updated and held in the background image holding means 8 (step A3). If the first input image does not include a moving object, basically it is not necessary to update the brightness value of each pixel of the background image. However, due to the influence of noise or the like, the luminance value of the background region may vary due to factors other than the illumination variation. Therefore, by updating the luminance value of the pixel regarded as the background by the object discriminating unit 14 at any time, an error may occur. Can be suppressed.

Further, in updating such a background image, it is possible to adopt a method of multiplying the old and new luminance values by a weight and gradually updating. That is, the object discriminating means 14
If the pixel of interest p _a is regarded as a background pixel in, then [Equation 6]
It is possible to weight the newly added data and gradually update it.

[0054]

[Equation 6] Here, P _{A —} new is the target pixel p _a and the reference pixels p _b and p _c
A three-dimensional luminance value matrix of the luminance values of the background pixels corresponding to
P _{A —} old is the pixel of interest p _a and the reference pixels p _b and p _{c that} have been obtained so far.
3 is a three-dimensional background luminance matrix based on the luminance value of the background pixel corresponding to. I _A is the pixel of interest p _a and the reference pixels p _b and p _c.
Is a luminance value matrix, and α is the weighting constant.

Next, when it is determined in step A10 that the above-described determination processing has been completed for all pixels, the image input means 4 determines whether or not there is a next image (step A11). If there is a next image, the process returns to step A1, the image is acquired, and thereafter, the above-described processes are repeated. Then, when the processing is completed for all the images, the object detection processing is completed. By performing the above processing each time an image is captured from the surveillance camera,
Object detection can be performed sequentially.

FIG. 4A is a schematic view showing an original image of a black car running on a road, FIG. 4B is an explanatory view showing a detection result of a car according to the prior art, and FIG. 4C is this embodiment. It is explanatory drawing which shows the detection result of the vehicle by an example. FIG. 4B is the same as FIG.
8A shows a result of detecting a vehicle based on the conventional image described with reference to FIG. 8 and the like using the original image and the background image shown in FIG. 8A. Since the object detection is performed with the attention point and the reference point adjacent to each other, Only the car contour is detected.

On the other hand, in (C) of FIG. 4 which is the detection result according to the present embodiment, not only the contour portion of the vehicle but
The entire vehicle is detected, including the inside of the contour, and the vehicle is detected more clearly and therefore reliably. In the present embodiment, as described above, the reference pixel selected for each target pixel is set apart from the target pixel by at least the size of the object. Therefore, unlike the case where the target pixel and the reference pixel are set close to each other, only the contour portion of the object is not detected. Then, a plurality of reference pixels are set for one target pixel, object detection is performed for each set of the target pixel and each reference pixel, and finally, a logical operation is performed to determine whether the target pixel is an object pixel. Since the determination is made, the position of the object can be reliably specified even if the target pixel and the reference pixel are set separately.

As in the prior art, it is determined whether or not the pixel of interest represents an image based on the distance between the illumination fluctuation background model and the luminance point in the luminance space. Therefore, the illumination variation is large and frequent. The object can be detected in a stable manner and with high accuracy even in the case of occurrence.

Next, a second embodiment of the present invention will be described. FIG. 5 is a block diagram showing the object detecting device 2 of the second embodiment, and FIG. 6 is a flowchart showing the operation of the object detecting device of FIG. Hereinafter, an object detecting apparatus according to a second embodiment of the present invention will be described with reference to these drawings, and at the same time, a computer that records an object detecting method and an object detecting program according to the second embodiment of the present invention. A readable information recording medium will be described.
In FIG. 5, the same elements as those in FIG. 1 are designated by the same reference numerals, and a description thereof will be omitted here.

The object detecting device 18 shown in FIG. 5 is different from the above object detecting device 2 in that the circumscribed rectangle of the object is obtained instead of detecting the object itself. As a result, the object is detected with a small amount of calculation. It becomes possible to detect. The object detection device 18 includes an image input unit 4, a plurality of reference point selection units 20 functioning as horizontal reference point selection unit and vertical reference point selection unit according to the present invention, a background image holding unit 8, and a horizontal background model generation according to the present invention. Background model creating means 22 functioning as means and vertical background model creating means, distance calculating means 24 functioning as horizontal distance calculating means and vertical distance calculating means according to the present invention, object discriminating means 26, background creating means 28, and image output. The means 15 is included and configured to detect an object from an image captured by a surveillance camera, for example.

Like the object detecting device 2, the object detecting device 18 is specifically constituted by a computer having an interface with a surveillance camera or a video tape recorder as an example, and is an input device constituting the computer. , For example, a CD-ROM as a computer-readable information recording medium recording the object detection program of the present invention is mounted, the program data is read from the CD-ROM, loaded into the memory of the computer,
It is realized by operating the CPU of the computer based on the program data.

Plural reference point selecting means 2 of the object detecting device 18
0 functions as a horizontal reference point selecting unit, selects a plurality of reference pixels included in the target horizontal pixel row for each target horizontal pixel row composed of a plurality of pixels arranged in the horizontal direction in the captured image, and selects each reference pixel. The brightness value of the reference pixel is acquired from the image input means 4. Further, the plurality of reference point selection means 20 functions as a vertical reference point selection means, and for each target vertical pixel row composed of a plurality of pixels arranged in the vertical direction in the captured image, a plurality of target vertical pixel rows are included. The reference pixel is selected and the brightness value of each reference pixel is acquired from the image input unit 4.

The background model creating means 22 functions as the horizontal background model creating means, and sets the brightness values of a plurality of pixels in the background image corresponding to the plurality of reference pixels selected by the horizontal reference point selecting means to the background. Image holding means 8
Create a lighting fluctuation background model when the background brightness changes due to the fluctuation of lighting. The background model creating means 22 functions as a vertical background model creating means, and stores the brightness values of a plurality of pixels in the background image, which correspond to the plurality of reference pixels selected by the vertical reference point selecting means, to the background image holding means. 8 to obtain the illumination fluctuation background model in the case where the background brightness changes under the influence of the lighting fluctuation.

The plural distance calculating means 2 functions as a horizontal distance calculating means, and in the multidimensional luminance space, the illumination fluctuation background model created by the background model creating means 22 as the horizontal background model creating means and the horizontal reference point. The distance between the points represented by the plurality of brightness values acquired by the plurality of reference point selecting means 20 as the selecting means is calculated. Further, the plural distance calculating means 2 functions as a vertical distance calculating means, and in the multidimensional luminance space, the illumination fluctuation background model created by the background model creating means 22 as the vertical background model creating means and the vertical reference point selection. Multiple reference point selection means 2 as means
The distance between 0 and the point represented by the plurality of acquired brightness values is calculated.

The object discriminating means 26 is a plural distance calculating means 2 as the horizontal distance calculating means and the vertical distance calculating means.
On the basis of the distance calculated by the above, it is determined whether or not the pixel representing the object is included in the target horizontal pixel row and the target vertical pixel row, and the logical operation is performed on the determination result related to both pixel rows to determine the object. Specify a rectangular area that contains. The background creating means 28, based on the luminance value of each pixel forming the target horizontal pixel row and the target vertical pixel row, which the object discriminating means 26 determines not to include a pixel representing an object, the corresponding pixel of the background image. The background image holding means 8 by updating the luminance value of
To hold.

Next, the operation of the object detecting device 18 thus constructed will be described. 7A and 7B are diagrams for explaining the operation of the second embodiment, and FIG. 7A is an explanatory diagram showing detection of an object possible region in the vertical direction,
(B) is an explanatory view showing detection of an object possible area in the horizontal direction, and (C) is an explanatory view showing a detection result. Hereinafter, description will be given with reference to FIGS. 6 and 7 together with FIG. 5.

As shown in FIG. 6, first, the image input means 4 reads an image picked up by a surveillance camera or the like, and the brightness value of each pixel forming the image is provided in the image input means 4. It is held in a memory (not shown) (step B1). Next, the plural reference point selecting means 20 sets one row of pixels at the vertical address y in the captured image as a target horizontal pixel column, and selects this one row of pixels as a reference pixel (reference pixel group). Then, the brightness value of each reference pixel is acquired from the image input means 4 (step B4). Therefore, assuming that the width of the captured image is Width pixels (Width is a positive integer), Wi
The dth number of pixels are included, and the multiple reference point selection unit 20 is
In this embodiment, all these pixels are used as reference pixels.

Then, the background model creating means 22 obtains the brightness value of each pixel in the background image corresponding to each of the reference pixels selected by the plural reference point selecting means 20 from the background image holding means 8 to obtain the background brightness. We will create a background model of the lighting fluctuation when the value changes due to the lighting fluctuation.
After that, the plural distance calculating means 24, in the multidimensional luminance space formed by the luminance values of the reference pixels, the illumination variation background model created by the background model creating means 22 and the plurality of the plurality of reference point selecting means 20 acquired. The distance to the point represented by the luminance value is calculated (step B6).

Then, the object discriminating means 26 compares the distance calculated by the plural distance calculating means 24 with a threshold value, and if the distance is larger than the threshold value, the object horizontal pixel row is selected. Correspondingly, the logic "1" is held, and if the distance is smaller than the threshold value, the logic "0" is held (step B8). The plural reference point selecting means 20, the background model creating means 22, and the plural distance calculating means 2 perform such processing in the entire range in the vertical direction of the image while increasing the value of the address y by 1 (step B10). ).
As a result, as shown in FIG. 7A, when the object 108 exists as shown in the figure, the object discriminating unit 26 sets the logic "1" to each horizontal pixel row of interest within the range of H. Will hold.

The plural reference point selecting means 20, the background model creating means 22, and the plural distance calculating means 2 perform the same processing in the horizontal direction of the image. That is,
The plural-reference-point selecting unit 20 sets a pixel for one column at the address x in the horizontal direction in the captured image as a vertical pixel column of interest, and selects a pixel for this one column as a reference pixel (reference pixel group). The brightness value of the reference pixel is acquired from the image input means 4 (step B2). Therefore, if the height of the captured image is Height pixels (Height is a positive integer), the vertical pixel array of interest includes Height pixels, and the plurality of reference point selecting means 20 is configured to include the plurality of reference point selecting means 20. Then, all these pixels are used as reference pixels.

Then, the background model creating means 22 obtains the brightness value of each pixel in the background image corresponding to each of the reference pixels selected by the plural reference point selecting means 20 from the background image holding means 8 to obtain the background brightness. We will create a background model of the lighting fluctuation when the value changes due to the lighting fluctuation.
After that, the plural distance calculating means 24, in the multidimensional luminance space formed by the luminance values of the reference pixels, the illumination variation background model created by the background model creating means 22 and the plurality of the plurality of reference point selecting means 20 acquired. The distance to the point represented by the luminance value of is calculated (step B5).

Subsequently, the object discriminating means 26 compares the distance calculated by the plural distance calculating means 2 with a threshold value, and if the distance is larger than the threshold value, the vertical pixel column of interest is selected. Correspondingly, the logic "1" is held, and if the distance is smaller than the threshold value, the logic "0" is held (step B7). The plural reference point selecting means 20, the background model creating means 22, and the plural distance calculating means 24 perform such processing in the entire horizontal range of the image while increasing the value of the address y by 1 (step B9). ). As a result, as shown in FIG. 7B, the object discriminating unit 26 logically determines that for each vertical pixel column of interest within the range W.
1 ”will be retained.

After that, the object discriminating means 26 obtains the image of the rectangular area as the detection result by taking the logical product of the logical values held for each target horizontal pixel row and each target vertical pixel row (step B11). In a memory (not shown) for holding the value, for each pixel, it is determined whether the value is logic "1" or logic "0". Specifically, when both the logical "1" is held for the target horizontal pixel row and the target vertical pixel row including the pixel of the captured image at the position corresponding to the pixel whose value is to be determined, The value is set to logic "1". On the other hand, when the logical "0" is held for at least one of the noticeable horizontal pixel column and the noticeable vertical pixel column including the pixels of the corresponding captured image, the value of the pixel is set to the logic "0". (Steps B12, B13, B1
4). As a result, in the memory that holds the detection result, as shown in FIG. 7C, the value of the pixel in the rectangular area R that may include an object is logical “1”, and the other pixels are The value of is logical "0".

Therefore, when the contents of this memory are displayed on the display device by the image output means 15, the rectangular area including the object is displayed on the screen. Also, the rectangular area R
The four sides of are the circumscribed rectangles of the object 108. And
When the width of the captured image is the Width pixel and the height is the Height pixel, the background model is created and the distance is calculated for each pixel in the above-described first embodiment. It is necessary to perform (Width × Height) times. However, in the second embodiment, the calculation process may be performed for each horizontal pixel column, and the calculation process may be performed for each vertical pixel column. Therefore, the number of calculation processes is (Width + Height) times in total. Becomes Therefore, the amount of calculation is significantly reduced as compared with the case of the first embodiment, and it is very advantageous when performing object detection at high speed and reducing the cost of the device.

As in the case of the object detecting device 2, the background creating means 28 makes each pixel constituting the noticeable horizontal pixel row and the noticeable vertical pixel row which the object discriminating means 26 has determined not to include a pixel representing the object. The brightness value of the pixel of the corresponding background image is updated based on the brightness value of the background image holding means 8
To hold. This makes it possible to perform highly accurate object detection using the latest background image. If there is another image to be processed, the above procedure is repeated for the next image as in the case of the object detection device 2 (step B15).

In the second embodiment, all pixels in each pixel row in the noticed horizontal pixel row and the noticed vertical pixel row are used as the reference pixels. It is also possible to further reduce the calculation amount by performing the above. Further, for example, if the area in which the object can exist is first determined in one of the horizontal direction and the vertical direction and then the processing in the other direction is performed only in that area, the area in which the object is not likely to exist Since it is not necessary to perform arithmetic processing, it is possible to further reduce the amount of arithmetic.

Although the object detecting device 2 and the object detecting device 18 are composed of computers here, it is of course possible to compose the object detecting device 2 and the object detecting device 18 with dedicated hardware. . Further, among the functions according to the present invention, some functions are realized by loading a predetermined program on a computer, other functions are realized by dedicated hardware, and further,
It is also possible to configure the object detection device 2 and the object detection device 18 by using a combination of programs already installed in the computer.

The computer-readable information recording medium is not limited to the above-mentioned CD-ROM, but may be a portable medium such as a floppy disk, a magneto-optical disk, a ROM, or a storage device such as a hard disk built in the computer. Contains. Further, the computer-readable information recording medium dynamically holds the program for a short time like a communication line when transmitting the program through a communication network such as the Internet or a communication line such as a telephone line. It also includes things, such as a volatile memory inside a computer system that serves as a server or a client in that case, that holds a program for a certain period of time.

[0079]

As described above, in the object detecting apparatus of the present invention, the image inputting means captures and holds the luminance value of each pixel of the photographed image, and the plural reference point selecting means sets each pixel of interest of the image. , A plurality of reference pixels are selected, and the brightness values of the selected plurality of reference pixels and the target pixel are acquired for each target pixel from the image input unit. Then, the background model creating means obtains from the background image holding means brightness values of a plurality of pixels in the background image respectively corresponding to the target pixel selected by the plurality of reference point selecting means and the plurality of reference pixels corresponding to the same pixel. An illumination variation background model in the case where the background brightness is changed by being affected by the illumination variation is created for each set of the target pixel and each reference pixel corresponding to the same pixel.

In addition, the distance calculation means is configured such that, in the luminance space, the background model generation means creates the illumination fluctuation background model for each of the set of the target pixel and the reference pixel, and the plural reference point selection means sets each target pixel for the target pixel. The distances between the points represented by the respective sets of the brightness values of the plurality of reference pixels and the brightness value of the target pixel acquired in step S1 are calculated. Then, the object discriminating means discriminates, for each group of the target pixel and each reference pixel, based on the distance calculated by the distance calculating means, whether the target pixel is a pixel representing an object or not, and discriminating for each group. A logical operation is performed on the result to finally determine whether the pixel of interest is a pixel representing an object.

As described above, in the object detecting device of the present invention,
A plurality of reference pixels are set for one target pixel, object detection is performed for each set of the target pixel and each reference pixel, and finally a logical operation is performed to determine whether the target pixel is an object pixel. Therefore, even if the target pixel and the reference pixel are set apart from each other,
The position of the object can be specified with certainty. Therefore, the reference pixel to be selected for each target pixel can be set apart from the target pixel by at least the size of the object, and as a result, it is possible to set the target pixel and the reference pixel close to each other. It is unlikely that only the outline of the object will be detected. Further, similar to the above-described conventional technology, since it is determined whether or not the pixel of interest represents an image based on the distance between the illumination variation background model and the luminance point in the luminance space, the illumination variation is large,
Moreover, even when it occurs frequently, it is stable and can detect an object with high accuracy.

Further, in the object detecting device of the present invention, the image input means captures and holds the luminance value of each pixel of the photographed image, and the horizontal reference point selecting means comprises the plurality of pixels arranged in the horizontal direction in the photographed image. A plurality of reference pixels included in the noticed horizontal pixel row are selected for each noticed horizontal pixel row, and the brightness value of each reference pixel is acquired from the image input unit. Further, the vertical reference point selecting means selects a plurality of reference pixels included in the target vertical pixel row for each target vertical pixel row composed of a plurality of pixels arranged in the vertical direction in the captured image, and determines the brightness of each reference pixel. The value is obtained from the image input means. Then, the horizontal background model creating means acquires the brightness values of a plurality of pixels in the background image corresponding to the plurality of reference pixels selected by the horizontal reference point selecting means from the background image holding means and determines the background brightness. An illumination variation background model is created when it is changed under the influence of the illumination variation, and the vertical background model creating means corresponds to the plurality of reference pixels selected by the vertical reference point selecting means. The brightness value of the pixel is acquired from the background image holding means, and an illumination fluctuation background model is created when the background brightness changes under the influence of the illumination fluctuation.

After that, the horizontal distance calculating means divides the illumination fluctuation background model created by the horizontal background model creating means and the point represented by the plurality of brightness values acquired by the horizontal reference point selecting means into the multidimensional brightness space. The vertical distance calculation means calculates the distance between the points, and the point represented by the illumination fluctuation background model created by the vertical background model creation means and the plurality of brightness values acquired by the vertical reference point selection means in the multidimensional brightness space. Calculate the distance between. Then, the object discriminating means discriminates based on the distance calculated by the horizontal distance calculating means and the vertical distance calculating means whether or not the target horizontal pixel row and the target vertical pixel row include a pixel representing an object, A logical operation is performed on the discrimination result relating to both pixel rows to specify a rectangular area including an object.

In the object detecting apparatus of the present invention, the calculation process such as the background model creation for each horizontal pixel column and the calculation of the distance between the background model and the luminance point is performed, and the similar process is performed for each vertical pixel column. Since the arithmetic processing may be performed, the total number of arithmetic processing is a value obtained by adding the number of pixels in the vertical direction of the image to the number of pixels in the horizontal direction of the image. On the other hand, when the background model is created for each pixel and the calculation processing such as the calculation of the distance between the background model and the luminance point is performed, the total number of calculation processing is the number of pixels in the horizontal direction of the image. , Multiplied by the number of pixels in the vertical direction of the image. Therefore, according to the present invention, even when the illumination variation is large and stable even when it frequently occurs, the area where the object exists can be detected with high accuracy, and the amount of calculation is significantly reduced. This is very advantageous for detection and cost reduction of the device.

In the object detecting method of the present invention , in the image input step, the brightness value of each pixel of the photographed image is fetched and stored in the first storage means, and in the plural reference point selecting step, for each pixel of interest of the image. A plurality of reference pixels are selected, and the brightness values of the selected plurality of reference pixels and the target pixel are acquired from the first storage unit for each target pixel. Then, in the background model creating step, the brightness values of a plurality of pixels in the background image respectively corresponding to the target pixel selected in the plurality of reference point selecting step and the plurality of reference pixels corresponding to the same pixel are stored in the second storage means. Get from
An illumination variation background model in the case where the background brightness changes under the influence of illumination variation is created for each set of the target pixel and each reference pixel corresponding to the same pixel.

Further, in the distance calculation step, in the luminance space, the illumination fluctuation background model created for each set of the target pixel and the reference pixel in the background model creation step, and each target pixel in the multiple reference point selection step. The distances between the points represented by the respective sets of the brightness values of the plurality of reference pixels and the brightness value of the target pixel acquired in step S1 are calculated.
Then, in the object determination step, for each set of the target pixel and each reference pixel, it is determined whether or not the target pixel is a pixel representing an object based on the distance calculated in the distance calculation step, and the determination for each group is performed. A logical operation is performed on the result to finally determine whether the pixel of interest is a pixel representing an object.

As described above, in the present invention, a plurality of reference pixels are set for one target pixel, object detection is performed for each set of the target pixel and each reference pixel, and finally the target pixel is determined by logical operation. Since it is determined whether or not is the pixel of the object, the position of the object can be surely specified even if the target pixel and the reference pixel are set separately. Therefore, the reference pixel to be selected for each target pixel can be set apart from the target pixel by at least the size of the object, and as a result, it is possible to set the target pixel and the reference pixel close to each other. It is unlikely that only the outline of the object will be detected. Further, similar to the above-described conventional technique, it is determined whether or not the pixel of interest represents an image based on the distance between the illumination variation background model and the luminance point in the luminance space, and thus when the illumination variation is large and frequently occurs. Moreover, the object can be detected with high stability and with high stability.

Further, in the object detecting method of the present invention, the brightness value of each pixel of the photographed image is fetched and held in the first storage means in the image input step, and in the horizontal reference point selecting step, the luminance value of the photographed image is changed in the horizontal direction. A plurality of reference pixels included in the noticed horizontal pixel row are selected for each noticed horizontal pixel row composed of a plurality of arranged pixels, and the brightness value of each reference pixel is acquired from the first storage means. Further, in the vertical reference point selecting step, the plurality of reference pixels included in the target vertical pixel row are selected for each target vertical pixel row composed of a plurality of pixels arranged in the vertical direction in the captured image, and the brightness of each reference pixel is selected. The value is acquired from the first storage means. Then, in the horizontal background model creating step, the brightness values of the plurality of pixels in the background image respectively corresponding to the plurality of reference pixels selected in the horizontal reference point selecting step are acquired from the first storage unit to obtain the background brightness. Creates an illumination variation background model in the case where it changes due to the influence of illumination variation, and in the vertical background model creating step, a plurality of background images corresponding to the plurality of reference pixels selected in the vertical reference point selecting step are provided. The luminance value of the pixel is acquired from the second storage means and an illumination fluctuation background model is created when the background luminance changes under the influence of the illumination fluctuation.

Then, in the horizontal distance calculation step, in the multidimensional brightness space, the illumination fluctuation background model created in the horizontal background model creation step and the points represented by the plurality of brightness values acquired in the horizontal reference point selection step are displayed. In the vertical distance calculation step, in the vertical distance calculation step, the points represented by the illumination fluctuation background model created in the vertical background model creation step and the plurality of brightness values acquired in the vertical reference point selection step. Calculate the distance between. Then, in the object determining step, based on the distance calculated in the horizontal distance calculating step and the vertical distance calculating step, it is determined whether or not the target horizontal pixel row and the target vertical pixel row include a pixel representing an object, A logical operation is performed on the discrimination result relating to both pixel rows to specify a rectangular area including an object.

As described above, according to the present invention, the background model is created for each horizontal pixel row, the calculation processing such as the calculation of the distance between the background model and the luminance point is performed, and the similar processing is performed for each vertical pixel row. Since the arithmetic processing may be performed, the total number of arithmetic processing is a value obtained by adding the number of pixels in the vertical direction of the image to the number of pixels in the horizontal direction of the image. On the other hand, when the background model is created for each pixel and the calculation processing such as the calculation of the distance between the background model and the luminance point is performed, the total number of calculation processing is the number of pixels in the horizontal direction of the image. , Multiplied by the number of pixels in the vertical direction of the image. Therefore, in the present invention, the illumination fluctuation is large and stable even when it frequently occurs,
In addition, it is possible to detect the existence region of the object with high accuracy, and the amount of calculation is significantly reduced. As a result, it is very advantageous in performing the object detection at high speed and reducing the cost of the apparatus.

[Brief description of drawings]

FIG. 1 is a block diagram showing an object detection device according to a first exemplary embodiment.

FIG. 2 is a flowchart showing an operation of the object detection device of FIG.

3A is an explanatory diagram showing a positional relationship between a target point and a reference point on an image surface, FIG. 3B is a schematic diagram when an object enters an imaging region, and FIG. 3C is a schematic diagram showing a detection result. Figure,
(D) is another explanatory diagram showing the positional relationship between the target point and the reference point on the image surface, (E) is another schematic diagram showing the detection result,
(F) is a schematic diagram showing a final detection result.

4A is a schematic diagram showing an original image of a black car running on a road, FIG. 4B is an explanatory diagram showing a detection result of a car according to a conventional technique, and FIG. 4C is according to this embodiment. It is explanatory drawing which shows the detection result of a vehicle.

FIG. 5 is a block diagram showing an object detection device according to a second exemplary embodiment.

6 is a flowchart showing an operation of the object detection device of FIG.

7A and 7B are diagrams for explaining the operation of the second embodiment, where FIG. 7A is an explanatory diagram showing detection of an object possible area in a vertical direction, and FIG. 7B is an object in a horizontal direction. FIG. 4C is an explanatory diagram showing detection of a possible existence area of FIG.

FIG. 8 is an explanatory diagram illustrating a background image.

FIG. 9 is an explanatory diagram showing a two-dimensional luminance space formed by the luminance values of an attention point and a reference point.

FIG. 10 is an explanatory diagram showing a captured image when an object enters the capturing area of the surveillance camera.

11A is an explanatory diagram showing an image surface, FIG. 11B is a schematic diagram when an object enters an imaging region, and FIG. 11C is a schematic diagram showing a detection result.

[Explanation of symbols]

2 ... Object detecting device, 4 ... Image input means, 6 ... Multiple reference point selecting means, 8 ... Background image holding means, 10 ... Background model creating means, 12 ... Distance calculating means, 14 ... Object Discriminating means, 16 ... Background creating means, 18 ... Object detecting device, 20 ... Multiple reference point selecting means, 22 ... Background model creating means, 24 ... Distance calculating means, 26 ... Object determining means, 28 ... ... background creating means, 102 ... white dots, 1
04: straight line, 106: white circle dot, 108: object, 110: black dot.

─────────────────────────────────────────────────── ─── Continuation of the front page (58) Fields investigated (Int.Cl. ⁷ , DB name) G06T 1/00 280 G06T 7/20 G08B 13/196 H04N 5/235 H04N 7/18 JISST file (JOIS)

Claims (8)

(57) [Claims] 1. An object detection device for detecting an object using a captured image, comprising: an image input unit that captures and holds a luminance value of each pixel of the captured image; and a plurality of pixels for each target pixel of the captured image. A plurality of reference point selecting means for selecting a reference pixel and acquiring the brightness values of the selected plurality of reference pixels and the target pixel for each of the target pixels from the image input means; and a background image forming a background of the captured image. Background image holding means for holding the brightness value of each pixel, and a plurality of pixels in the background image respectively corresponding to the target pixel selected by the plurality of reference point selection means and the plurality of reference pixels corresponding to the pixel A luminance value is obtained from the background image holding means, and an illumination variation background model in the case where the background luminance changes under the influence of illumination variation is the target pixel and each reference corresponding to the same pixel. A background model creating means for creating each pair with a pixel, and the background model creating means for the brightness variation background model created for each pair of the target pixel and the reference pixel in the luminance space, and the plurality of reference point selections Distance calculation means for calculating a distance between each of the brightness values of the plurality of reference pixels acquired for each pixel of interest and a point represented by a set of brightness values of the pixel of interest; For each group with each reference pixel, it is determined whether or not the pixel of interest is a pixel representing an object based on the distance calculated by the distance calculation means, and a logical operation is performed on the determination result for each group to perform the logical operation. An object detecting device comprising: an object determining unit that finally determines whether or not the pixel of interest is a pixel representing an object. 2. The brightness value of the pixel of the background image corresponding to each of the target pixel determined to be not a pixel representing an object by the object determining means and each of the plurality of reference pixels corresponding to the pixel, 2. The object detecting apparatus according to claim 1, further comprising a background creating unit that updates the pixel of interest and brightness values of the plurality of reference pixels corresponding to the pixel and holds the background image in the background image holding unit. 3. An object detecting device for detecting an object using a captured image, comprising image input means for capturing and holding a brightness value of each pixel of the captured image, and horizontally arranged in the captured image. Horizontal reference point selecting means for selecting a plurality of reference pixels included in the horizontal pixel row of interest for each horizontal horizontal pixel row of a plurality of pixels and acquiring the brightness value of each reference pixel from the image input means; For each target vertical pixel row composed of a plurality of pixels arranged in the vertical direction in the captured image, a plurality of reference pixels included in the target vertical pixel row are selected to obtain the brightness value of each reference pixel from the image input means. Vertical reference point selecting means, a background image holding means for holding the luminance value of each pixel of the image forming the background of the captured image, and the plurality of reference pixels selected by the horizontal reference point selecting means A corresponding horizontal background model is created by obtaining the brightness values of a plurality of pixels in the background image from the background image holding means and creating an illumination fluctuation background model when the background brightness changes under the influence of the illumination fluctuation. Means, and luminance values of a plurality of pixels in the background image, which correspond respectively to the plurality of reference pixels selected by the vertical reference point selecting means, are acquired from the background image holding means, and the background luminance is affected by the illumination variation. A vertical background model creating means for creating an illumination variation background model in the case where the illumination variation background model changes, and in the multidimensional luminance space, the illumination variation background model created by the horizontal background model creating means and the horizontal reference point selecting means. Horizontal distance calculating means for calculating the distance between the points represented by the acquired plurality of brightness values, and the vertical background model creation procedure in a multidimensional brightness space. A vertical distance calculating means for calculating a distance between the illumination fluctuation background model created by the vertical reference point selecting means and the points represented by the plurality of luminance values acquired by the vertical reference point selecting means, the horizontal distance calculating means and the vertical distance. Based on the distance calculated by the calculating means, it is determined whether or not the target horizontal pixel row and the target vertical pixel row include a pixel representing an object, and a logical operation is performed on the determination results relating to both pixel rows. And an object discriminating means for identifying a rectangular area including an object. 4. The pixel of the background image corresponding to the pixel of the corresponding background image is determined based on the luminance value of each pixel forming the target horizontal pixel row and the target vertical pixel row, which is determined by the object determining unit not to include a pixel representing an object. The object detecting apparatus according to claim 3, further comprising: a background creating unit that updates a brightness value and holds the background image in the background image holding unit. 5. An object detection method for detecting an object using a captured image, comprising: an image input step of capturing a brightness value of each pixel of the captured image and storing it in a first storage means; A plurality of reference pixels for each pixel of interest, a plurality of reference point selecting steps of acquiring the brightness values of the selected plurality of reference pixels and the pixel of interest from the first storage means for each pixel of interest; A background image holding step of holding the luminance value of each pixel of the background image forming the background of the image in the second storage means; the target pixel selected in the plurality of reference point selecting steps and the plurality of references corresponding to the same pixel The brightness values of a plurality of pixels in the background image, which respectively correspond to the pixels, are acquired from the second storage unit, and the illumination change when the background brightness changes under the influence of the illumination change. A background model creating step for creating a background model for each set of the target pixel and each reference pixel corresponding to the same pixel; and, in a luminance space, for each set of the target pixel and the reference pixel in the background model creating step. Between the illumination fluctuation background model created in step S1 and the point represented by each of the brightness values of the target pixel and the brightness values of the plurality of reference pixels acquired for each target pixel in the plurality of reference point selection steps. A distance calculation step for calculating the respective distances, and for each set of the target pixel and each reference pixel, it is determined whether or not the target pixel is a pixel representing an object based on the distance calculated in the distance calculation step. And an object discriminating step of finally discriminating whether or not the target pixel is a pixel representing an object by performing a logical operation on the discrimination result for each set. Detection method. 6. The luminance value of the pixel of the background image corresponding to each of the target pixel determined to be not a pixel representing an object in the object determining step and each of the plurality of reference pixels corresponding to the pixel, 6. The object detecting method according to claim 5, further comprising a background creating step of updating based on the luminance value of the pixel of interest and the plurality of reference pixels corresponding to the pixel and holding the background value in the first storage unit. . 7. An object detection method for detecting an object using a captured image, comprising: an image input step of capturing a brightness value of each pixel of the captured image and holding the same in a first storage means; For each target horizontal pixel row composed of a plurality of pixels arranged in the horizontal direction, a plurality of reference pixels included in the target horizontal pixel row are selected and the brightness value of each reference pixel is acquired from the first storage means. Horizontal reference point selecting step, and for each target vertical pixel column composed of a plurality of pixels arranged in the vertical direction in the captured image, select a plurality of reference pixels included in the target vertical pixel column to determine the brightness of each reference pixel. A vertical reference point selecting step of acquiring a value from the first storage means; and a second brightness value of each pixel of an image forming a background of the captured image.
And a luminance value of a plurality of pixels in the background image corresponding to each of the plurality of reference pixels selected in the horizontal reference point selecting step, from the second storage means. Then, a horizontal background model creating step of creating an illumination variation background model when the background brightness changes under the influence of the illumination variation, and respectively corresponding to the plurality of reference pixels selected in the vertical reference point selecting step, A vertical background model creating step of creating the illumination variation background model when the luminance values of a plurality of pixels in the background image are acquired from the second storage means and the background luminance changes under the influence of the illumination variation; In the multidimensional luminance space, in the illumination fluctuation background model created in the horizontal background model creating step, and in the horizontal reference point selecting step. The horizontal distance calculation step of calculating the distance between the points represented by the plurality of obtained brightness values, the illumination fluctuation background model created in the vertical background model creation step in the multidimensional brightness space, and the vertical reference point Based on the vertical distance calculation step of calculating a distance between the points represented by the plurality of brightness values obtained in the selection step, the horizontal distance calculation step and the distance calculated in the vertical distance calculation step, the target horizontal An object discrimination step of discriminating whether or not a pixel representing an object is included in the pixel row and the vertical pixel row of interest, and performing a logical operation on the discrimination results relating to both pixel rows to identify a rectangular area including the object. An object detection method comprising: 8. The pixel of the corresponding background image based on the luminance value of each pixel forming the target horizontal pixel row and the target vertical pixel row, which is determined not to include a pixel representing an object in the object determining step. 8. The object detecting method according to claim 7, further comprising: a background creating step of updating a brightness value and holding the brightness value in the second storage means.