JP3419919B2 - Moving object detection device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device for detecting a moving object from a time-series image obtained by photographing.

This type of device is used in a system for observing traffic on a road and a security system for monitoring an intruder.

[0003]

2. Description of the Related Art A background image that does not include a moving object is prepared in advance, a time-series image obtained by camera shooting is input, and each time the image is input, the input image is compared with the background image, and there is a mismatch. The image area is defined as the existence portion of the moving object.

Alternatively, a time-series image obtained by photographing with a camera is input, the latest input image is compared with the already input image, and the inconsistent image area is set as the existence portion of the moving object.

In Japanese Patent Laid-Open No. 282709/1989, an input image is binarized, edges are extracted from the binarized input image, and the extracted edges having a certain length or more are defined as the contour line of the object. , A device for recognizing a contour of a pair of images whose position has changed as the contour of a moving object is shown.

[0006]

In the prior art, since all the portions of the input screen that do not match the background image or the already input image are detected as moving objects, the moving objects and their shadows are distinguished. Not done.

Therefore, in the case of a system for observing traffic on a road, a traveling vehicle whose shadow is included in the field of view of the camera is also mistakenly recognized as a passing vehicle, and the result of observation of traffic is excessive than the actual one. Becomes Further, at night, the headlight light reflected on the road surface also moves together with the vehicle and is therefore mistakenly recognized as a traveling vehicle.

The present invention has been made in view of the above conventional circumstances, and an object of the present invention is to provide a device that does not mistake a moving shadow as a moving object.

[0009]

In FIG. 1, an apparatus according to a first aspect of the present invention includes an image input means 10 for continuously inputting time-series images obtained by photographing, and a background image which does not include a moving object. Background image creating means 12 for creating from a series of input images; edge point extracting means 14 for extracting edge points having large changes in brightness from the input image and the background image;
A moving edge point selecting means 16 for selecting an edge point existing only on the input image side among the edge points extracted from the input image and the background image, and an area having a high density of the selected edge points are set. And a moving object specifying unit 18 that specifies the moving object as an existing portion.

In the figure, the device according to the second invention is also
Image input means 10 for continuously inputting time-series images obtained by shooting, background image creation means 12 for creating a background image that does not include a moving object from time-series input images, input image and background image The edge point extracting means 14 for extracting an edge point having a large change in luminance from and the edge point existing only on the input image side among the edge points extracted from the input image and the background image are selected. It has moving edge point selecting means 16 and moving object specifying means 18 for specifying a region with a high density of the selected edge points as a moving object existing portion.

However, the background image creating means 12 sets an initial background image setting means 121 for setting the image input at the beginning of the time series as an initial image of a background image that does not include a moving object, the input image and the background image. The brightness difference calculation means 122 for calculating the brightness difference between the corresponding points in 1) and the background image update means 123 for updating the brightness of each point in the background image in the direction in which the calculated brightness difference decreases.

Further, the edge point extraction means 14 extracts the edge points having a large change in brightness from the input image and generates the input edge image by the input edge image generation means 141.
And a background edge image generation unit 142 that generates a background edge image by extracting an edge point having a large change in luminance from the updated background image.

Furthermore, the moving edge point selecting means 16 searches the changing edge point searching means 161 for searching for edge points existing in the input edge image but not in the background edge image, and changing edge image for all the searched edge points. And a changing edge image creating means 162 for creating.

Then, the moving object specifying means 18 scans the changing edge image in the direction traversing the object moving path, calculates the sum of the edge points for each scanning, and sequentially calculates the calculated sum of the respective edge points. And a moving object region determining unit 182 that determines a portion where a certain number or more of edge points are densely continuous in the extending direction of the object moving path as an existing region of the moving object. Including.

In the figure, the device according to the third invention is
Image input means 10 for continuously inputting time-series images obtained by shooting, background image creation means 12 for creating a background image that does not include a moving object from time-series input images, input image and background image The edge point extracting means 14 for extracting an edge point having a large change in luminance from and the edge point existing only on the input image side among the edge points extracted from the input image and the background image are selected. The moving edge point selecting unit 16 and the moving object specifying unit 18 that specifies a region where the density of the selected edge points is high as a moving object existing portion, Passing object number counting means 20 for counting the number
And also have.

Also in the apparatus according to the third aspect of the present invention, the background image creating means 12 inputs the initial background image setting means 121 for setting the image input at the beginning of the time series as the initial image of the background image not including the moving object, and the input. Difference calculation means 12 for calculating the difference in brightness between corresponding points in the captured image and the background image
2 and the background image update means 123 for updating the brightness of each point in the background image in the direction in which the calculated difference in brightness decreases.
And, including.

Further, the edge point extraction means 14 extracts the edge points having a large change in brightness from the input image and generates the input edge image, and the input edge image generation means 141.
And a background edge image generation unit 142 that generates a background edge image by extracting an edge point having a large change in luminance from the updated background image.

Further, the moving edge point selecting means 16 searches the changing edge point searching means 161 for searching the edge points existing in the input edge image but not in the background edge image, and the changing edge image in all the searched edge points. And a changing edge image creating means 162 for creating.

Then, the moving object specifying means 18 scans the changing edge image in the direction traversing the object moving path, calculates the sum of the edge points for each scanning, and sequentially calculates the calculated sum of the respective edge points. And a moving object region determining unit 182 that determines a portion where a certain number or more of edge points are densely continuous in the extending direction of the object moving path as an existing region of the moving object. Including.

Here, the passing object number counting means 20 is
Object position storage means 201 that stores one end of the determined moving object existing area as the position of the moving object, and inter-screen object associating means that associates the stored position with the previously stored position and the distance between the two positions. 202 and a new imaged object counting unit 203 that counts the number of stored positions that are not associated with previously stored positions to obtain the number of moving objects that have passed through the field of view of the image capturing.
And, including.

[0021]

In the device according to the first aspect of the present invention, the time-series images obtained by photographing are continuously input, a background image containing no moving object is created from the time-series input images, and the input image and the background image are input. The edge points having a large change in brightness are extracted from and the contours are extracted, and the edge points existing only on the side of the input image are selected to extract the contours of the moving object and its shadow.

Here, since the density of the edge points is high in the portion where the moving object exists, and the density of the edge points is low in the portion of the shadow of the traveling vehicle and the headlight light, an area where the selected high edge point density is continuous. Is specified as the existing part of the moving object.

In the apparatus according to the second aspect of the present invention, the image input at the beginning of the time series is used as the initial image of the background image that does not include a moving object, and the brightness difference between corresponding points in the input image and the background image is determined. The brightness of each point in the background image is calculated in a direction in which the calculated difference in brightness decreases, and the change in illuminance is reflected in the background image.

Even if the outdoor illuminance changes greatly depending on the time of day and the weather, since the change in illuminance is reflected in the background image in real time, a stable detection result can always be obtained despite the large change in illuminance. .

Further, the input edge image is generated by extracting edge points having a large change in luminance from the input image, and the background edge image is extracted by extracting edge points having a large change in luminance from the updated background image. Is generated, an edge point existing only in the input edge image is searched, and a changed edge image is created with all the searched edge points.

Then, the changing edge image is scanned in the direction traversing the object moving path, the total sum of the edge points is calculated for each scan, and a histogram is created which is arranged in order in the extending direction of the object moving path. A histogram portion in which the sum of the edge points is densely continuous in the extension direction of the object moving path is determined as the moving object existing area, and the shadow and the headlight light portion are removed.

As described above, since both the vertical edges and the horizontal edges are used for detecting the moving object, the reliability of the detection result is improved (more horizontal edges than vertical edges are extracted from the photographed image of the vehicle. Therefore, it is best to use both edges when detecting a moving vehicle).

In the device according to the third aspect of the invention, one end (for example, the tail of the traveling vehicle) in the determined moving object existing area is stored as the position of the moving object, and the stored position is set as the previously stored position. The moving object is tracked in association with the distance between the two positions (positions that are close to each other in distance between two consecutive images are related to the same moving object), and are stored in the previously stored positions. The number of positions that are not associated with the position is counted as the number of passing objects (the number of vehicles that have newly entered the field of view is integrated).

[0029]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Referring to FIG. 2, a photographing output of a camera 2000 (a consumer video camera can be used) is input to a video board 2004 of a moving image processing system 2002, and an image output of the video board 2004 (a photographed image of a road). And its traffic information) are given to the monitor 2006.

The workstation 2008 is also connected to the video board 2004, and the workstation 2008 displays road traffic information on the video board 200.
4 and sends it to a remote host computer (not shown) via a line 2010.

The moving image processing system 2002 includes five image processing boards 2012_1, 1212_2, 2
A linear array processor in which 012_3, 2012_4, and 2012_5 are pipeline-connected is provided.

The linear array processor includes a camera 200
The shooting output of 0 is input from the video board 2004, and the linear array processor sends traffic information (the number of vehicles, traveling speed) to the video board 2004.

Image processing boards 2012_1, 2012_2, 2012_3, 2 of the linear array processor.
Workstations 012_4 and 2012_5
08, the program is downloaded, the image processing board 2012_1, 2012_2, 2012_3, 201
2_4 and 2012_5 individually execute the programs downloaded to themselves from the workstation 2008.

For this reason, the algorithm of the entire apparatus is the image processing boards 2012_1, 2012_2, 2012_.
3, 2012_4, 2012_5, and therefore pipeline processing is performed at high speed by the linear array processor without depending on the processing speed of the workstation 2008. As a result, traffic information can be obtained in real time.

In FIGS. 3 and 4, the camera 2000 is oriented slightly in the direction of extension of the white line that divides the road into two lanes, and is set slightly downward, and is provided at a position distant from the white line. As a result, the camera 2000 has a field of view of a road portion (observation section: see FIG. 3) extending over several tens of meters, and a vehicle traveling on the road portion is imaged from diagonally above and behind.

In FIG. 2, a moving image processing system 200
The video board 2004 of No. 2 A / D-converts a captured image each time a captured image is input from the camera 2000, stores the image in the frame memory, and then the image processing board 2012.
Hand over to _1.

The image processing board 2012_1 removes portions other than roads from the delivered image 50 as shown in FIG.
A white line at the center of the road is detected from the remaining image portion, and the remaining image portion is divided into left and right driving lanes (see FIG. 4) by the white line (divided into areas 500 and 501), and these divided areas 500 and 501 are divided. The image processing boards 2012_2, 2012_3, 2012_4, 201 are configured to perform the processing for each.
Instruct 2_5.

The image processing board 2012_2 stores the areas 500 and 501 of the first input photographed image as they are in the memory as a background image, and performs the background update processing of FIG. 6 every time the photographed image is input.

In this process, the luminance difference between the input image side and the background image side is calculated for all the coordinates of the areas 500 and 501 (step 600) (step 602), and the difference in the background image is reduced. The brightness of each coordinate is updated by a fixed amount (steps 604, 606, 608, 6).
10).

Therefore, even if the outdoor illuminance greatly changes depending on time and weather, the change in illuminance is reflected in the background image. Therefore, it is possible to always obtain stable traffic information despite changes in the external environment in which the illuminance fluctuates greatly.

The image processing board 2012_3 performs the processing shown in FIG. 7, and calculates the amount of change in luminance for all coordinates of the areas 500 and 501 (step 700) (step 70).
2, 704, 706), a background edge image having a value "1" at a coordinate point where the amount of change in luminance is equal to or larger than a predetermined value and a value "0" at other coordinate points is generated (steps 708, 710,
712).

The image processing board 2012_4 is shown in FIG.
Processing is performed to calculate the brightness change amount for all coordinates of the regions 500 and 501 (step 800) (steps 802, 804, and 806), and the value '1 is obtained at the coordinate point where the brightness change amount is a predetermined value or more. ', An input edge image having a value of "0" at other coordinate points is generated (steps 808, 81)
0,812).

Further, the image processing board 2012_4 performs the processing shown in FIG. 9 and checks the values of the coordinate points corresponding to the background edge image and the input edge image for all coordinates of the areas 500 and 501 (step 900) (step 902). 90
4), the coordinate point having the value "0" on the side of the background edge image and the value "1" on the side of the input edge image is set to the value "1" (step 906), and the other coordinate points are set to the value "0" (step 906). Step 908), a changing edge image is generated at edge points that are present in the input edge image but not in the background edge image.

For example, when the background edge image of FIG. 10A and the input edge image of FIG.
The changed edge image of (C) is generated, and the changed edge image is generated at the edge points of the vehicle and its shadow which are present in the input edge image but not in the background edge image.

Then, the image processing board 2012_4 performs the processing of FIG. 11 and records the rear position of the vehicle traveling in the areas 500 and 501 (in the field of view of the camera image: on the traveling lane of the observation section).

In the processing of FIG. 11, the measurement start point Y1 (see FIG.
(See) to the measurement end point Y2 (see FIG. 5),
The change edge image is scanned in the direction X across the road, and the sum of the edge points is calculated (the value of the edge histogram: H (y) = ΣE2
(x, y)) is calculated, and the sum of the edge points is compared with the first set value (predetermined value 1) for each scan, and the position y at which the sum of the edge points is equal to or more than the first set value is set to the vehicle position. It is detected as the tail position (steps 1100, 1102, 1104, 1105, and
1106, 1108).

Further, while moving toward the measurement end point Y2, the changing edge image is scanned in the direction X crossing the road to calculate the total sum of the edge points, and this is integrated (integrated value = S) to obtain the second set value. Compared with (predetermined value 2), the position y at which the total sum of the edge points is equal to or less than the second set value is detected as the head position of the vehicle (steps 1108, 1110, 1112, 111).
4, 1116).

When the head position of the vehicle is detected, the value S obtained by integrating the total sum of the number of edge points is set to the third set value (predetermined value 3).
When it is confirmed that the integrated value S is larger than the third set value and the total sum of the number of edge points is dense in the direction Y by comparing with (step 1118), the portion from the detected rear position to the leading position is present by the traveling vehicle. The area is confirmed and this position is recorded (step 1120).

When it is confirmed that the integrated value S is smaller than the third set value and the total sum of the number of edge points is rough in the direction Y, the shadow of the traveling vehicle exists in the portion from the detected rear position to the leading position. Judge as the area to When the traveling vehicle or its shadow is detected, the above-described processing for searching the rear position of the next vehicle is restarted (step 1118->).
Step 1102).

In the center portion of FIG. 12, a change edge image (see FIG. 10) when one vehicle is traveling in the right lane of the observation section (see FIGS. 3 and 4) is shown.
The edge histogram of the vehicle is on the right side of the figure,
Further, in the left part of the figure, the edge histrums of the shadow of the vehicle projected on the left lane are respectively shown.

As can be seen from FIG. 12, when the vehicle moves on the right lane from the front side, its edge histogram advances from the lower side to the upper side in the region 501 of FIG. The edge histogram also advances from the lower side to the upper side in the area 500 of FIG.

FIG. 13 shows an enlarged edge histogram of a portion where a vehicle exists. As can be understood from FIG. 13, a large histogram value (that is, edge The sum of points) appears densely.

FIG. 14 shows an enlarged edge histogram of the shadow portion of the vehicle. As can be seen from FIG. 14, although the histogram value is high at the boundary of the vehicle shadow portion, It becomes lower inside.

Therefore, after detecting the tail of the vehicle (step 1104-> step 1108), the coordinate value y
While changing (step 1112), the histogram value is compared with the second installation value (step 1116), and thereafter, the time point when the head of the vehicle is detected (N in step 1116).
In O), the integrated value S of the total number of edge points is compared with the third installation value (predetermined value 3) (step 1118),
Whether this portion is due to the traveling vehicle (see FIG. 13) or the shadow thereof (see FIG. 14) can be reliably determined.

FIG. 15 illustrates the processing performed by the image processing board 2012_5, and the observation section (see FIGS. 3 and 4).
The number of vehicles passing through (see) is counted by this processing.

In the process of FIG. 15, all vehicle positions recorded last time (positions of the vehicle tail) are sequentially taken out (steps 1500 and 1502), and each time the previously recorded vehicle position is taken out, they are recorded this time. The determined vehicle position (position of the vehicle tail) is sequentially extracted (step 1504).
1506).

Then, every time the vehicle position recorded this time is taken out, it is checked whether or not the distance between the vehicle position and the vehicle position recorded previously (the one taken out) is a fixed value or less (step 1508).

At this time, when it is confirmed that the distance between the two positions is equal to or less than a certain value (step 1508).
YES), it is determined that these positions belong to the same vehicle (step 1510).

If it is confirmed that the distance between the two vehicles is not less than a certain value (N at step 1508).
O), it is determined that the positions of the vehicles are different, and the next vehicle position (currently recorded) is taken out.

Further, if the vehicle position of the current record that is the same as the vehicle position of the previously recorded record retrieved is not retrieved (YES in step 1504), the position of the current record retrieved at that time is determined. It is determined that the vehicle has newly entered the observation section (step 1512), and the value indicating the number of vehicles that have passed the observation section is counted up (step 1514).

The image processing board 2012_5 also calculates the speed of the vehicle passing through the observation section, and the vehicle speed and the number of vehicles passing through the observation section are given to the video board 2004.

The video board 2004 is the image processing board 2
The vehicle speed and the number of vehicles given from 012_5 are taken by the camera 20.
00 is output to the monitor 2006 together with the latest image captured by 00, and is displayed on the monitor 2006.

Further, the workstation 2008 acquires the observed vehicle speed and the number of vehicles from the video board 2004 and transmits them to the remote host computer via the line 2010.

As described above, according to this embodiment, the density of the edge points is continuously high in the portion where the vehicle actually exists, and the density of the edge points is high in the portion of the shadow (and headlight light). Focusing on the lowness, only the region where the density of the edge points is continuously high is specified as the existence portion of the moving object, so that the number of vehicles passing through can be accurately observed.

The input image at the beginning of the time series is set as the initial background image, and the brightness of the background image is updated in the direction in which the brightness difference from the subsequent input images decreases, so that the outdoor illuminance changes depending on time and weather. Even if it is largely changed, the change in the illuminance is reflected in the background image, and thus it is possible to always obtain stable observation results.

Further, since both the vertical edges and the horizontal edges are used for detecting the vehicle, the vehicle detection is performed as compared with the case where only the vertical edges are used (there is a small number of edges, which may cause detection omission depending on illuminance and vehicle). It is possible to greatly improve the reliability of the.

Image processing boards 2012_1, 2012_2, 2012_3 of the linear array processor.
Workstations 20 to 2012_4 and 2012_5
08, the program is downloaded, the image processing board 2012_1, 2012_2, 2012_3, 201
2_4 and 2012_5 individually execute the programs downloaded from the workstation 2008 to execute the algorithm of the entire apparatus on the image processing boards 2012_1 and 2012_2.
012_2, 2012_3, 2012_4, 2012_
Since it is shared by 5, it is possible to obtain road information in real time.

[0068]

As described above, according to the present invention, a moving object can be reliably and accurately in spite of a change in illuminance (a change in sunshine) in a photographing field, a shadow of a moving object, or strong reflected light of an object moving path. It becomes possible to detect.

[Brief description of drawings]

FIG. 1 is a diagram illustrating the principle of the invention.

FIG. 2 is an explanatory diagram of a configuration of an embodiment.

FIG. 3 is an explanatory diagram of installation of a camera.

FIG. 4 is an explanatory diagram of installation of a camera.

FIG. 5 is an explanatory diagram of mask processing.

FIG. 6 is a flowchart illustrating background update processing.

FIG. 7 is a flowchart illustrating background edge image generation processing.

FIG. 8 is a flowchart illustrating a process of generating an input edge image.

FIG. 9 is a flowchart illustrating a changing edge image generation process.

FIG. 10 is an explanatory diagram of a generation operation of a changed edge image.

FIG. 11 is a flowchart illustrating a vehicle position determination process.

FIG. 12 is an explanatory diagram of an edge histogram.

FIG. 13 is an explanatory diagram of a histogram of vehicle partial edges.

FIG. 14 is an explanatory diagram of a histogram of a background partial edge.

FIG. 15 is a flowchart illustrating a vehicle counting process.

[Explanation of symbols]

2000 camera 2002 moving image processing system 2004 video board 2006 monitor 2008 workstation 2010 line 2012_1 image processing board (area mask setting) 2012_2 image processing board (background image creation) 2012_3 image processing board (background image edge extraction) 2012_4 image processing board (Creation of histogram) 2012_5 Image processing board (calculation of number of vehicles and speed)

Continuation of the front page (56) References JP-A 1-211083 (JP, A) JP-A 3-144797 (JP, A) JP-A 5-62094 (JP, A) JP-A 6-223157 (JP , A) JP 5-28396 (JP, A) JP 4-84300 (JP, A) JP 63-9813 (JP, A) International publication 94/011852 (WO, A1) (58) Survey Areas (Int.Cl. ⁷ , DB name) G06T ⁷ /00-7/60 G06T 1/00 H04N 7/18 G08G 1/00-1/16 B60R 21/00

Claims (3)

(57) [Claims] 1. An image input means for continuously inputting time-series images obtained by photographing, a background image creation means for creating a background image containing no moving object from time-series input images, and an input image. Edge point extraction means for extracting an edge point having a large change in brightness from the background image and the edge point existing only on the input image side among the edge points extracted from the input image and the background image. A moving object detecting device, comprising: a moving edge point selecting means for selecting; and a moving object specifying means for specifying an area having a high density of the selected edge points as a portion where a moving object exists. 2. An image input means for continuously inputting time-series images obtained by photographing, a background image creation means for creating a background image containing no moving object from time-series input images, and an input image. Edge point extraction means for extracting an edge point having a large change in brightness from the background image and the edge point existing only on the input image side among the edge points extracted from the input image and the background image. The background image creating means includes a moving edge point selecting means to be selected, and a moving object specifying means to specify an area having a high density of the selected edge points as an existing portion of the moving object. An initial background image setting means that uses the captured image as an initial image of a background image that does not include a moving object, and a brightness difference calculating means that calculates a brightness difference between corresponding points in the input image and the background image. The background image updating means for updating the brightness of each point in the background image in the direction in which the difference in the brightness is reduced, and the edge point extracting means extracts the edge points having a large brightness change from the input image. Input edge image generation means for generating an input edge image according to the present invention, and a background edge image generation means for generating a background edge image by extracting edge points having a large change in brightness from the updated background image. The selecting means includes a changing edge point searching means for searching for an edge point existing in the input edge image and not existing in the background edge image, and a changing edge image creating means for creating a changing edge image with all the searched edge points. The moving object specifying means scans the changing edge image in the direction traversing the object moving path, calculates the sum of the edge points for each scan, and calculates the calculated sum of the edge points. A histogram creating means for aligning in the extension direction of the object moving path, and a moving object area determining means for determining a portion where a certain number of edge point sums are densely continuous in the extending direction of the object moving path as an existing area of the moving object. A moving object detection device comprising: 3. An image input means for continuously inputting time-series images obtained by photographing, a background image creation means for creating a background image containing no moving object from the time-series images, and the input image and background. An edge point extracting means for extracting an edge point having a large change in brightness from the image, and a movement for selecting an edge point existing only on the input image side among the edges extracted from the input image and the background image. Edge point selecting means, moving object specifying means for specifying a region where the density of the selected edge points is high as the existence portion of the moving object, and passing object number counting means for counting the number of moving objects that have passed through the visual field range of the photographing. The background image creating means includes an initial background image setting means that uses the image input at the beginning of the time series as an initial image of a background image that does not include a moving object, and the input image and the background image. A brightness difference calculating means for calculating a brightness difference between corresponding points in the scene image; and a background image updating means for updating the brightness of each point in the background image in a direction in which the calculated brightness difference decreases, The point extracting means extracts an edge point having a large luminance change from the input image to generate an input edge image, and an edge point having a large luminance change from the updated background image. A moving edge point selecting means for searching for an edge point existing in the input edge image but not in the background edge image, and a moving edge point searching means for generating a background edge image. A moving edge image creating means for creating a changing edge image at all edge points, and the moving object specifying means is a changing edge image in a direction traversing the object moving path. Histogram creating means for scanning and calculating the sum total of edge points for each scan, and aligning the calculated sum total of edge points in the extension direction of the object moving path in order; A moving object area determining unit that determines a portion that is densely continuous in a direction as a moving object existing area, and the passing object number counting unit stores one end in the determined moving object existing area as a position of the moving object. Object position storage means, inter-screen object associating means for associating the stored position with the previously stored position and the distance between both positions, and the previously stored position among the stored positions. A moving object detection device, comprising: a new imaged object counting unit that counts the number of positions that have not existed and obtains the number of moving objects that have passed through the field of view of imaging.