JP2001043458A - System for generating background picture - Google Patents

System for generating background picture

Info

Publication number
JP2001043458A
JP2001043458A JP11215064A JP21506499A JP2001043458A JP 2001043458 A JP2001043458 A JP 2001043458A JP 11215064 A JP11215064 A JP 11215064A JP 21506499 A JP21506499 A JP 21506499A JP 2001043458 A JP2001043458 A JP 2001043458A
Authority
JP
Japan
Prior art keywords
background
pixel
value
image
mask
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
JP11215064A
Other languages
Japanese (ja)
Inventor
Naohiro Amamoto
直弘 天本
Original Assignee
Oki Electric Ind Co Ltd
沖電気工業株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oki Electric Ind Co Ltd, 沖電気工業株式会社 filed Critical Oki Electric Ind Co Ltd
Priority to JP11215064A priority Critical patent/JP2001043458A/en
Publication of JP2001043458A publication Critical patent/JP2001043458A/en
Withdrawn legal-status Critical Current

Links

Abstract

PROBLEM TO BE SOLVED: To provide a background picture generating system by which a background picture excluding a mobile object is generated even when the mobile object exists on a screen and the proper background picture is obtained. SOLUTION: In the system, a prescribed area to be monitored is photographed, the inputted picture is stored in a frame buffer 102, the absolute value of a difference between the input picture of the time and the input picture one time before is binarized concerning the picture, a change area mask 104 where the value of a pixel with detected change is adopted as zero and the value of the pixel without detected change is adopted as one is generated, the value of a count buffer is counted-up by one concerning the pixel where change is not detected, the value of the count buffer is reset to zero concerning the pixel where change is detected, the values of the respective pixels in the count buffer are larger than a stored threshold, the pixel value of a new register mask 108 is set to one with respect to the pixel which is not established in a background establishing mask, the pixel value of the background establishing mask is set to zero and the pixel value of the input picture of the time is copied on the background picture with respect to the pixels which is one in value in a background mask updating means 107 and the new register mask 108.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a system using a background image as a reference image such as a traffic flow monitor or an intrusion monitor using a TV camera, and removes a moving object even if the moving object exists on a screen. The present invention relates to a background image generation system capable of generating and updating a background image and obtaining a suitable background image.

[0002]

2. Description of the Related Art Conventionally, techniques in such a field include:
For example, (1) Japanese Patent Application Laid-Open No. 4-273586, (2)
There is one disclosed in JP-A-6-274786.

According to the vehicle detection device disclosed in the above (1), an image consisting of only a background is input in advance, stored as a background image, and a difference between the background image and the input image is obtained. The extremely dark area is a shaded area immediately below the vehicle, and the presence or absence of the vehicle is detected.

According to the road traffic abnormality monitoring system disclosed in the above (2), a measurement area is set in advance, and a background image corresponding to the measurement area is created based on an input image where a vehicle is not reflected. The difference between the background image and the input image is obtained, the vehicle is detected, and its coordinates are obtained. Thereafter, according to each section of the measurement area in the road width direction, the coordinate distribution is frequency-ized for each vehicle, and the number of passing vehicles for a certain period is accumulated.
Then, the cumulative frequency is divided by the number of passing vehicles to normalize, thereby obtaining a traveling position spectrum of the vehicle. Further, time-series spectrum data is input to automatically determine the type and degree of abnormality.

[0005]

However, in any of the techniques described above, it is a precondition that an image in which the vehicle is not running is taken in advance and this image is registered as a background image. Although this is possible in a rural road environment with a small traffic volume, it is difficult to capture an image without vehicles in a road environment in an urban city with a large traffic volume.

The present invention uses a background image as a reference image for traffic flow monitoring, intrusion monitoring, and the like using a TV camera, and generates a background image excluding a moving object even when a moving object exists in the screen. An object of the present invention is to provide a background image generation system that can obtain a suitable background image.

[0007]

In order to achieve the above object, the present invention provides: (1) an image pickup apparatus for taking an image of a predetermined monitoring area in a background image generation system;
A background image generation device that generates a background image; and an image display device that displays the background image and presents it to a user, wherein the background image generation device stores an input image captured by the imaging device. And binarizing an absolute value of a difference between an input image at the time and an input image one time before the input image with respect to the image stored in the frame buffer,
Change detection means for creating a change area mask in which the value of a pixel in which a change is detected is set to 0 and the value of a pixel in which a change is not detected is set to 1; Is incremented by one, and the pixel in which a change is detected is set to 0 in the count buffer.
Count buffer updating means for resetting the threshold value, a threshold value storing means for storing a threshold value used for comparing each pixel value of the count buffer, a value of each pixel of the count buffer being larger than the threshold value of the threshold value storing means, and setting of the background. For a pixel which has not been determined in the mask, the pixel value of the newly registered mask is set to 1 and the pixel value of the background determined mask is set to 0.
In the background registration means for copying the pixel value of the input image at that time into the background image, and in the background determination mask, it is determined whether or not all the pixels have been determined. The method further comprises a background creation end determining means for outputting a background image if the setting is confirmed and ending the processing.

[2] In the background image generation system according to [1], the background image generation device determines whether or not undetermined pixels are noise in the background determination mask. It is characterized by further comprising a threshold value changing means for changing the threshold value stored in the storage means to a smaller value.

[3] In a background image generation system, an imaging device for capturing an image of a predetermined monitoring area, a background image generation device for processing the captured image and generating a background image,
An image display device that displays the background image and presents it to a user, wherein the background image generation device includes a frame buffer that stores an input image captured by the imaging device, and an image stored in the frame buffer. On the other hand, the absolute value of the difference between the input image at the time and the input image one time before is binarized, and the value of a pixel where a change is detected is set to 0, and the value of a pixel where a change is not detected is set to 1 Detecting means for creating a changed area mask, and a pixel in which no change is detected by the change detecting means counts up the value of the count buffer by one, and a pixel in which a change is detected resets the value of the count buffer to zero. Count buffer updating means, threshold value storage means for storing a threshold value used for comparing each pixel value of the count buffer, and a value of each pixel of the count buffer For a pixel larger than the threshold value of the value storage means and not determined in the background fixed mask, the pixel value of the newly registered mask is set to 1 and the pixel value of the background fixed mask is set to 0; A pixel registered in the image is updated using the input image at the time, a pixel having a value of 1 with the newly registered mask is a background update unit that copies the pixel value of the input image at the time into the background image, In the finalized mask, it is determined whether or not all the pixels have been finalized. If the finalized image has not been finalized, the process is continuously performed. It is characterized by having.

[4] In the background image generation system according to the above [3], the background creation end determination means determines whether or not all pixels are determined in the background mask,
If it is not determined, the process is continuously performed, and an image obtained by combining the input image, the background image, and the background determination mask is sequentially output,
When a user inputs a command, the command is interpreted, and if all background images have been determined, the process is terminated.

[0011]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a block diagram of a background image generating system showing a first embodiment of the present invention, and FIG. 2 is an explanatory diagram of the operation of the count buffer.

A background image generating system according to the present invention includes an ITV installed to capture an image composed of a background and an object.
A moving image is input to a background image generation device 115 from an imaging device (input image) 101 such as a camera. Image display device 11
A display device 4 such as a TV monitor displays the background image generated by the background image generation device 115. The background image generation device 115 includes a frame buffer 102, a change detection unit 103, a change area mask 104, a count buffer update unit 105, a count buffer 106, a background mask update unit 107, a new registration mask 108, a background determination mask 109, and a threshold storage unit. 110, background registration means 11
1. Background image storage means 112, background creation end determination means 1
13.

[0014] The frame buffer 102 is
The input image obtained from step 1 is stored. The change detection unit 103 detects a moving object that changes with time with respect to the input image, and stores the detection result as a binary image in the change area mask 104. In the count buffer updating means 105,
The change area mask 104 is input and the count buffer 1
06 is updated. Also, the background mask updating means 107
In the count buffer 106, it is determined whether there is a pixel that satisfies the new registration condition for the background image. If there is a pixel that satisfies the condition, the pixel is registered in the new registration mask 108 and registered. A registered flag is recorded in the background determination mask 109 to indicate that the pixel has been registered.

The threshold value storage means 110 stores a threshold value for making the above determination. In the background registration means 111,
For the pixels registered in the newly registered mask 108, the pixel value of the input image at that time is recorded in the background image recording unit 112. The background creation end determination means 113 determines whether or not the background image has been completely created. If there is a pixel that has not been created yet, the process is continued. If all the pixels are determined, the background image I is determined. BG is output and the process ends. The image display device 114 is a display device such as a television monitor,
Background image I output from background creation end determination means 113
Display BG .

Hereinafter, the operation of the background image generating system will be described.

The imaging device 101 captures an image composed of a background and an object such as a road and a vehicle running on the road. A moving image captured by the imaging device 101 is input to the frame buffer 102. The frame buffer 102 is a ring buffer capable of storing m (m ≧ 2) images.

The change detecting means 103 calculates the difference between the input image at the time t and the input image one time before (t-1), and if the absolute value of the difference value of each pixel is equal to or greater than the threshold value, 0; In the following case, a binary image that is 1 is created and stored in the change area mask 104.

Next, the count buffer updating means 105 receives the input of the change area mask 104 and updates the contents of the count buffer 106. The count buffer 106
It has the same size (number of pixels) as the input image size, and is composed of one pixel nbit (n bits). Usually n = 8
(0-255) is sufficient. At the start of the process, all are initialized to zero.

In the count buffer updating means 105, the value of the pixel corresponding to the pixel whose change is not detected by the change detecting means 103 (the pixel having the value 1 in the change area mask 104) is set to 1 in the count buffer 106.
Count up. Further, for a pixel for which a change has been detected by the change detecting means 103 (a pixel having a value of 0 in the change area mask 104), the value of a position corresponding to the pixel in the count buffer 106 is reset to 0.

Accordingly, the value of each pixel in the count buffer 106 represents a period during which no change is detected for each pixel in the input moving image. That is, if the pixel value of the count buffer 106 is 11, it indicates that the pixel value at that position in the input moving image has not changed from the time (t−10) to the current time t.

FIG. 2 is a diagram for explaining the operation of the count buffer of the background image generating system according to the first embodiment of the present invention.

This figure shows a count buffer updating means 10
5 shows an example of a procedure for updating the count buffer 106 when a new change area mask 104 is input. In other words, the updated count buffer is as shown in FIG. 2C by the change area mask as shown in FIG. 2A and the count buffer before update shown in FIG. 2B. Note that the cells in FIG.
And the 3 × 3 area of the count buffer 106 is enlarged, and the numbers each represent a pixel value.

The background mask updating means 107 determines whether or not each pixel in the count buffer 106 satisfies the registration condition for the background image, and if so, registers the pixel in the new registration mask 108 and determines the background. Mask 1
At 09, a flag indicating that registration with the background image has been confirmed is recorded.

Here, the new registration mask 108 and the background fixed mask 109 are each a binary image having the same size as the input image, and all pixel values of the background fixed mask 109 are initialized to 1 when the system is started. A pixel having a value of 1 indicates that the pixel has not been registered in the background image, and a pixel having a value of 0 indicates that the pixel has been registered in the background image. The new registration mask 108 is initialized to 0 each time a new input image is input. The value of the count buffer 106 is equal to or greater than the threshold value T stored in the threshold value storage unit 110 in advance, and the background determination mask 10
If there is a pixel that satisfies the condition that the value of 9 is 1, the pixel value at the position corresponding to the pixel in the newly registered mask 108 is set to 1 and the pixel value of the background fixed mask 109 is set to 0. That is, a pixel that has not been detected in the input image for a predetermined time and has not been registered in the background image in the past is newly registered in the background image.

If a pixel having a value of 1 exists in the newly registered mask 108, the background registration unit 111 records the pixel value at the position corresponding to the pixel in the input image at that time stored in the frame buffer 102 into the background image. Means 112
Copy to

Finally, the background creation end determining means 113 determines whether or not a pixel having the value 1 exists in the background determination mask 109. If at least one pixel having the value 1 exists, the background image is not determined. Processing continues as complete. If the pixel having the value 1 does not exist, it is determined that the creation of the background image has been completed, the image is output, and the process ends. Image display device 1
At 14, the background image recording means 112 output from the background creation end determination means 113 is displayed.

With this configuration, according to the background image generation system of the first embodiment, it is difficult to obtain an image without a moving object on a road with a large traffic volume such as an urban area (always a moving object). In this case, it is possible to generate a good background image using a moving image obtained from an imaging device such as an ITV camera.

Next, a second embodiment of the present invention will be described.

The second embodiment can provide a background image generation system capable of generating a background image at high speed even when a picked-up image (input image) has a lot of noise and an unstable pixel value.

FIG. 3 is a block diagram of a background image generation system showing a second embodiment of the present invention.

The background image generating system according to the present embodiment is provided with an I / O set to capture an image composed of a background and an object.
A moving image is input to the background image generation device 216 from the imaging device 201 such as a TV camera. The image display device 215 is a TV
It is a display device such as a monitor, and displays the background image generated by the background image generation device 216. The background image generation device 216 includes a frame buffer 202, a change detection unit 2
03, change area mask 204, count buffer updating means 205, count buffer 206, background mask updating means 207, new registration mask 208, background fixed mask 20
9, a threshold storage unit 210, a threshold change unit 211, a background registration unit 212, a background image storage unit 213, and a background creation end determination unit 214.

The frame buffer 202 is the imaging device 20.
The input image obtained from step 1 is stored. The change detection unit 203 detects a moving object that changes with time with respect to the input image, and the detection result is stored in the change area mask 204 as a binary image.

The count buffer updating means 205 receives the change area mask 204 as an input and
6 is updated. The background mask updating unit 207 determines whether there is a pixel in the count buffer 206 that satisfies the new registration condition for the background image.
If there is a pixel that satisfies the condition, the pixel is registered in the new registration mask 208, and a registered flag is recorded in the background determination mask 209 to indicate that the registered pixel has been registered.

The threshold value storage means 210 stores a threshold value for making the above determination. In the threshold value changing means 211,
The threshold value of the threshold value storage unit 210 is adjusted according to the state of the pixel not determined by the background determination mask 209.

The background registration unit 212 records the pixel value of the input image at the time of the pixel registered in the new registration mask 208 in the background image recording unit 213. The background creation end determination means 214 determines whether or not the background image has been completely created. If there is a pixel that has not been created yet, the process is continued. If all the pixels are determined,
The background image is output from the background image recording unit 213, and the process ends. The image display device 215 is a display device such as a television monitor, and displays the background image IBG output from the background creation end determination unit 214.

The operation of the background image generation system according to the second embodiment will be described below.

The imaging device 201 captures a moving image composed of a background and an object such as a road and a vehicle running on the road. A moving image captured by the imaging device 201 is input to the frame buffer 202. The frame buffer 202 is a ring buffer capable of storing m (m ≧ 2) images.

The change detecting means 203 calculates a difference between the input image at the time t and the input image one time before (t-1), and if the absolute value of the difference value of each pixel is equal to or larger than the threshold value, 0; In the following case, a binary image that is 1 is created and stored in the change area mask 204.

Next, the count buffer updating means 205 receives the change area mask 204 and updates the contents of the count buffer 206. The count buffer 206
It has the same size (number of pixels) as the input image size, and is composed of one pixel nbit (n bits). Usually n = 8
(0-255) is sufficient. At the start of the process, all are initialized to zero.

In the count buffer updating means 205, for a pixel for which no change is detected by the change detecting means 203 (a pixel having a value of 1 in the change area mask 204), the value of a position corresponding to the pixel in the count buffer 206 is set to 1 Count up. Further, for a pixel for which a change has been detected by the change detecting means 203 (a pixel having a value of 0 in the change area mask 204), the value of a position corresponding to the pixel is reset to 0 in the count buffer 206.

Accordingly, the value of each pixel in the count buffer 206 represents a period during which no change is detected for each pixel in the input moving image. That is, if the pixel value of the count buffer 206 is 11, it indicates that the pixel value at that position in the input moving image has not changed from the time (t−10) to the current time t.

The background mask updating means 207 determines whether or not each pixel in the count buffer 206 satisfies the registration condition for the background image, and if so, registers the pixel in the new registration mask 208 and determines the background. Mask 2
At 09, a flag indicating that registration with the background image has been confirmed is recorded.

Here, each of the newly registered mask 208 and the background fixed mask 209 is a binary image having the same size as the input image, and all pixel values of the background fixed mask 209 are initialized to 1 when the system is started. A pixel having a value of 1 indicates that the pixel has not been registered in the background image, and a pixel having a value of 0 indicates that the pixel has been registered in the background image.

The new registration mask 208 is initialized to 0 each time a new input image is input. Count buffer 2
If there is a pixel that satisfies the condition that the value of “06” is equal to or larger than the threshold value T stored in the threshold value storage unit 210 in advance and the value of the background confirmation mask 209 is 1, the newly registered mask 208 corresponds to the pixel. The pixel value of the position to be set is set to 1 and the pixel value of the background fixed mask 209 is set to 0. That is, a pixel that has not been detected in the input image for a predetermined time and has not been registered in the background image in the past is newly registered in the background image.

The threshold value changing means 211 determines whether or not a pixel not determined by the background determination mask 209 is noise. If the pixel is noise, the value of the threshold value T stored in the threshold value storage means 210 is reduced by k. (T−k) is newly stored in the threshold value storage unit 210 as a threshold value. If it is not noise, the threshold is not changed.

The determination as to whether or not the pixel not determined by the background determination mask 209 is noise is made by the background determination mask 2
09 is subjected to contraction processing (a basic technique of image processing, a technique introduced in many commentary books such as Technical Review Company, Hasegawa and Koshimizu, etc.), and a pixel having a value of 1 is obtained in the resulting image. It is checked whether or not there is any pixel, and if there is no pixel, all pixels that have not been determined in the background determination mask 209 are determined to be noise. Here, the contraction means that the background fixed mask 209 is f (i, j) and the processing result is g
When (i, j) is set, the following processing is performed.

G (i, j) = {0: When f (i, j) = 0 or any of the four neighborhoods (or eight neighborhoods) of f (i, j) is 0. 1 indicates other times. By this processing, a noise component having a small area is removed.

As described above, it is determined whether or not a pixel that has not been determined by the background determination mask 209 is noise. If it is determined that the pixel is noise, the placement T is reduced to determine the background. It is possible to haveten it.

Although the method for determining whether noise is present has been described above, the background determination mask 209 is simply described.
The threshold value may be changed when the total number of pixels that have not been determined in the step becomes smaller than a predetermined number. Also,
The threshold value can be changed not in one step but in several steps.

If a pixel having a value of 1 exists in the newly registered mask 208, the background registration unit 212 records the pixel value at the position corresponding to the pixel of the input image at that time stored in the frame buffer 202 as the background image. Means 213
Copy to

Finally, the background creation end determination means 214 determines whether or not a pixel having a value of 1 exists in the background determination mask 209. If at least one pixel having a value of 1 exists, the background image is not determined. Processing continues as complete. If the pixel having the value 1 does not exist, it is determined that the creation of the background image has been completed, the image is output, and the process ends. Image display device 2
At 15, the background image IBG output from the background creation end determination means 214 is displayed.

As described above, according to the background image generation system of the second embodiment, it is difficult to obtain an image without a moving object on a road with a large traffic volume such as an urban area. (Where only images to be captured can be captured), it is possible to generate a good background image at high speed using a moving image obtained from an imaging device such as an ITV camera. Further, even when the picked-up image has much noise and the pixel value is unstable, a good background image can be generated at high speed.

Next, a third embodiment of the present invention will be described.

In the third embodiment, even when the brightness of the background changes due to a change in weather or the like while the background is being created, the change is absorbed and an accurate background image is generated. .

FIG. 4 is a block diagram of a background image generation system showing a third embodiment of the present invention.

The background image generation system according to the present embodiment is provided with an
A moving image is input to the background image generation device 315 from the imaging device 301 such as a TV camera. The image display device 314 is a TV
It is a display device such as a monitor, and displays the background image generated by the background image generation device 315. The background image generation device 315 includes the frame buffer 302, the change detection unit 3
03, change area mask 304, count buffer updating means 305, count buffer 306, background mask updating means 307, new registration mask 308, background fixed mask 30
9, a threshold storage unit 310, a background update unit 311, a background image storage unit 312, and a background creation end determination unit 313.

[0058] The frame buffer 302 is
The input image obtained from step 1 is stored. The change detection unit 303 detects a moving object that changes with time in the input image, and stores the detection result in the change area mask 304 as a binary image.

The count buffer updating means 305 receives the change area mask 304 as an input, and
6 is updated. The background mask updating unit 307 determines whether or not there is a pixel in the count buffer 306 that satisfies the new registration condition for the background image.
If there is a pixel that satisfies the condition, the pixel is registered in the new registration mask 308, and a registered flag is recorded in the background determination mask 309 to indicate that the registered pixel has been registered. The threshold value storage unit 310 stores a threshold value for making the above determination.

The background updating unit 311 records the pixel value of the input image at the time of the pixel registered in the new registration mask 308 in the background image recording unit 312. In addition, a pixel already registered as a background image is updated according to the input image.

The background creation completion determination means 313 determines whether or not the background image has been completely created. If there is a pixel that has not been created yet, the process is continued. The background image IBG is output, and the process ends.

The image display device 314 is a display device such as a television monitor, and displays the background image IBG output from the background creation end determination means 313.

Hereinafter, the operation of the background image generation system will be described.

The imaging device 301 captures a moving image composed of a background and an object such as a road and a vehicle running on the road. A moving image captured by the imaging device 301 is input to the frame buffer 302. The frame buffer 302 is a ring buffer capable of storing m (m ≧ 2) images.

The change detecting means 303 obtains a difference between the input image at the time t and the input image one time before (t-1), and 0 if the absolute value of the difference value of each pixel is equal to or larger than the threshold, In the following case, a binary image that is 1 is created and stored in the change area mask 304.

Next, the count buffer updating means 305 receives the change area mask 304 as an input and updates the contents of the count buffer 306. The count buffer 306
It has the same size (number of pixels) as the input image size, and is composed of one pixel nbit (n bits). Usually n = 8
(0-255) is sufficient. At the start of the process, all are initialized to zero.

The count buffer updating means 305 sets the value of the pixel corresponding to the pixel whose change is not detected by the change detecting means 303 (the pixel having a value of 1 in the change area mask 304) to 1 in the count buffer 306.
Count up. Further, for a pixel for which a change has been detected by the change detecting means 303 (a pixel having a value of 0 in the change area mask 304), the value of a position corresponding to the pixel is reset to 0 in the count buffer 306.

Accordingly, the value of each pixel in the count buffer 306 represents a period during which no change is detected for each pixel in the input moving image. That is, if the pixel value of the count buffer 306 is 11, it indicates that the pixel value at that position in the input moving image has not changed from the time (t−10) to the current time t.

The background mask updating means 307 determines whether or not each pixel of the count buffer 306 satisfies the registration condition for the background image, and if so, registers the pixel in the new registration mask 308 and determines the background. Mask 3
At 09, a flag indicating that registration with the background image has been confirmed is recorded. Here, the new registration mask 308 and the background determination mask 309 are each a binary image having the same size as the input image, and all pixel values of the background determination mask 309 are initialized to 1 when the system is started. The pixel has not been registered in the background image, and a pixel having a value of 0 indicates that the pixel has already been registered in the background image.

The new registration mask 308 is initialized to 0 each time a new input image is input. Count buffer 3
If there is a pixel satisfying the condition that the value of “06” is equal to or greater than the threshold value T stored in the threshold value storage unit 310 in advance and the value of the background determination mask 309 is 1, the newly registered mask 308 corresponds to the pixel The pixel value at the position to be set is set to 1 and the pixel value of the background fixed mask 309 is set to 0. That is, a pixel that has not been detected in the input image for a predetermined time and has not been registered in the background image in the past is newly registered in the background image.

If there is a pixel having a value of 1 in the newly registered mask 308, the background updating means 311 records the pixel value at the position corresponding to the pixel of the input image at that time stored in the frame buffer 302 in the background image. Means 312
Copy to Further, the pixels already registered in the background image recording unit 312 are updated. The updating of the background image is performed as follows.

First, the difference between the input image at that time stored in the frame buffer 302 and the background image is obtained.
A binary image is created such that the absolute value of the difference value of each pixel is equal to or greater than a threshold value and is equal to 0 if the absolute value is equal to or less than the threshold value. In this image, the pixel value is 1 and the background fixed mask 30
The background image is updated by the following equation for the pixel whose value is 0 in 9:

B t (i, j) = I t (i, j) × [1-
exp (−1 / C)] + B t−1 (i, j) × exp (−
1 / C) where B t (i, j) is the updated background image,
t-1 (i, j) is updated before the background image, I t (i, j) is the input image of the time, C is represent time constants.

As described above, by updating the value of the pixel already determined as the background image by using the latest input image, even if the background fluctuates due to the fluctuation of the weather during the creation of the background, the fluctuation can be reduced. It becomes possible to absorb.

Finally, the background creation end determination means 313 determines whether or not a pixel having a value of 1 exists in the background determination mask 309. If at least one pixel having a value of 1 exists, the background image is not determined. Processing continues as complete. If the pixel having the value 1 does not exist, it is determined that the creation of the background image has been completed, the image is output, and the process ends. Image display device 3
At 14, the background image IBG output from the background creation end determination means 313 is displayed.

As described above, according to the background image generation system of the third embodiment, it is difficult to obtain an image without a moving object on a road with a large traffic volume such as an urban area, and a moving object always exists. Even when only images can be taken,
Using a moving image obtained from an imaging device such as an ITV camera, a good background image can be generated at high speed.

Even if the background fluctuates during the generation of the background image due to a change in the weather or the like, the already registered pixels are updated using the latest input image. Even if the pixel is registered as a background image, the fluctuation can be absorbed.

Next, a fourth embodiment of the present invention will be described.

In the fourth embodiment, it is possible to input a command from the user, such as re-creating a background image and forcibly terminating generation of a background image, and generate a background image desired by the user.

FIG. 5 is a block diagram of a background image generating system showing a fourth embodiment of the present invention.

The background image generating system according to the present embodiment is provided with an I / O system installed to capture an image composed of a background and an object.
A moving image is input to a frame buffer 402 in an imaging device 401 such as a TV camera. A moving image is input from the imaging device 401 to the background image generation device 416. The image display device 414 is a display device such as a TV monitor, and displays the background image generated by the background image generation device 416. The background image generation device 416 includes the frame buffer 40
2. Change detection means 403, change area mask 404, count buffer update means 405, count buffer 40
6. Background mask updating means 407, newly registered mask 40
8, a background determination mask 409, a threshold storage unit 410, a background update unit 411, a background image storage unit 412, and a background creation end determination unit 413.

The frame buffer 402 includes the image pickup device 40
The input image obtained from step 1 is stored. The change detection unit 403 detects a moving object that changes with time in the captured image, and stores the detection result as a binary image in the change area mask 404. The count buffer updating means 405 receives the change area mask 404 as an input and
6 is updated.

The background mask updating means 407 determines whether or not there is a pixel which satisfies the condition for newly registering a background image in the count buffer 406. The registered pixel is registered in the newly registered mask 408, and a registered flag is recorded in the background fixed mask 409 to indicate that the registered pixel has been registered. The threshold value storage unit 410 stores a threshold value for making the determination.

The background updating unit 411 records the pixel value of the input image at the time of the pixel registered in the new registration mask 408 in the background image recording unit 412. In addition, a pixel already registered as a background image is updated according to the input image.

The background creation completion determination means 413 determines whether or not the background image has been completely created. If there is a pixel that has not been created yet, the process is continued. The background image is output, and the process ends.

[0086] The image display device 414 is a display device such as a television monitor. The input unit 415 is an input device such as a keyboard, and enables a user to input a command.

Hereinafter, the operation of the background image generation system will be described.

The imaging device 401 captures a moving image composed of a background and an object such as a road and a vehicle running on the road. A moving image captured by the imaging device 401 is input to the frame buffer 402. The frame buffer 402 is a ring buffer capable of storing m (m ≧ 2) images.

The change detecting means 403 obtains the difference between the input image at the time t and the input image one time before (t-1), and if the absolute value of the difference value of each pixel is equal to or larger than the threshold value, 0; In the following case, a binary image that is 1 is created and stored in the change area mask 404.

Next, the count buffer updating means 405 receives the change area mask 404 as input and updates the contents of the count buffer 406. The count buffer 406
It has the same size (number of pixels) as the input image size, and is composed of one pixel nbit (n bits). Usually n = 8
(0-255) is sufficient. At the start of the process, all are initialized to zero.

The count buffer updating means 405 sets the value of the pixel corresponding to the pixel whose change is not detected by the change detecting means 403 (the pixel having the value 1 in the change area mask 404) to 1 in the count buffer 406.
Count up. Also, for the pixel for which a change has been detected by the change detecting means 403 (the pixel having a value of 0 in the change area mask 404), the value of the position corresponding to the pixel in the count buffer 406 is reset to 0.

Accordingly, the value of each pixel in the count buffer 406 represents a period during which no change is detected for each pixel in the input moving image. That is, if the pixel value of the count buffer 406 is 11, it indicates that the pixel value at that position in the input moving image has not changed from the time (t−10) to the current time t.

The background mask updating means 407 determines whether or not each pixel in the count buffer 406 satisfies the registration condition for the background image, and if so, registers the pixel in the new registration mask 408 and determines the background. Mask 4
At 09, a flag indicating that registration with the background image has been confirmed is recorded.

Here, the new registration mask 408 and the background determination mask 409 are each a binary image having the same size as the input image, and all pixel values of the background determination mask 409 are initialized to 1 when the system is started. A pixel having a value of 1 indicates that the pixel has not been registered in the background image, and a pixel having a value of 0 indicates that the pixel has been registered in the background image.

The new registration mask 408 is initialized to 0 each time a new input image is input. Count buffer 4
If there is a pixel satisfying the condition that the value of 06 is equal to or greater than the threshold value T stored in the threshold value storage means 410 in advance and the value of the background confirmation mask 409 is 1, the newly registered mask 408 corresponds to the pixel. The pixel value of the position to be set is set to 1 and the pixel value of the background fixed mask 409 is set to 0. That is, a pixel that has not been detected in the input image for a predetermined time and has not been registered in the background image in the past is newly registered in the background image.

If a pixel having a value of 1 exists in the newly registered mask 408, the background updating means 411 records the pixel value at the position corresponding to the pixel of the input image at that time stored in the frame buffer 402 as the background image. Means 412
Copy to Further, the pixel already registered in the background image recording unit 412 is updated. The updating of the background image is performed as follows.

First, the difference between the input image at that time stored in the frame buffer 402 and the background image IBG is obtained, and if the absolute value of the difference value of each pixel is equal to or greater than the threshold value, 0; Create a binary image that is 1. For a pixel having a pixel value of 1 in this image and a value of 0 in the background confirmation mask 409, the background image is updated by the following equation.

B t (i, j) = I t (i, j) × [1-
exp (−1 / C)] + B t−1 (i, j) × exp (−
1 / C) where B t (i, j) is the updated background image,
t-1 (i, j) is updated before the background image, I t (i, j) is the input image of the time, C is represent time constants.

As described above, by updating the value of a pixel already determined as a background image using the latest input image, even if the background fluctuates due to a change in weather or the like during creation of the background, the fluctuation can be reduced. It becomes possible to absorb.

The background creation end determination means 413 determines whether or not a pixel having a value of 1 exists in the background determination mask 409. If at least one pixel having a value of 1 exists, the background image is determined to be incomplete. Continue processing. If the pixel having the value 1 does not exist, it is determined that the creation of the background image has been completed, and the process ends. In addition, the background creation end determination unit 413 combines the input image at that time, the intermediate result of the background image IBG , and the background determination mask 409 stored in the frame buffer 402 and sequentially outputs them.

The image display device 414 displays the composite image output from the background creation end determination means 413 as a moving image.

FIG. 6 shows an example of an image displayed on the image display device 414. The upper left is an input image 501, the upper right is a background image 502, and the lower right is a background fixed mask 503. Further, a region painted black in the background image and a region painted white (value 1) by the background confirmation mask are background image undetermined regions 504, which indicate that the background image is being created. I have.

The input means 415 provides an interface for receiving an input from the user. Input means 415
Is realized by a keyboard, for example, necessary commands are assigned such as an R (Redo) key for a background reproduction command and an I (Interrupt) key for a background generation end command. If the R key is pressed,
The command is interpreted by the background creation end determination unit 413, and the background image recording unit 412, the background fixed mask 409,
The new registration mask 408 and the count buffer 406 are initialized, and the background generation processing is performed again.

Similarly, when the I key is pressed, the command is interpreted by the background creation end determination means 413, and if there is an undetermined (value 1) pixel in the background determination mask 409,
The background image recording unit 412 copies the pixel value of the input image at that time stored in the frame buffer 402 to a position corresponding to the pixel, and ends the background generation processing.

As described above, the user operates the image display device 414.
Is visually checked, and if the completed background image is different from the desired background image or an erroneous background image is created for some reason, an instruction to recreate the background is issued. be able to. If the background image of the area desired by the user has already been determined and the creation has been completed, but one undetermined pixel remains in the other area, the end of the background generation processing may be instructed. it can.

As described above, according to the background image generation system of the fourth embodiment, it is difficult to obtain an image without a moving object on a road with a large traffic volume such as an urban area (when a moving object always exists). (Where only images to be captured can be captured), it is possible to generate a good background image at high speed using a moving image obtained from an imaging device such as an ITV camera.

Further, when the obtained background image is different from the one desired by the user, the background image is regenerated, the background image of the area desired by the user has already been generated, and the background image has not been set in other areas. If the determined area exists, it is possible to instruct the end of the background image generation processing, and it is possible to provide an interface that allows the user to perform simple control of the apparatus.

Further, the present invention is not limited to the above-described embodiment, and various modifications can be made based on the gist of the present invention, and they are not excluded from the scope of the present invention.

[0109]

As described above, according to the present invention, the following effects can be obtained.

(A) In a system using a background image as a reference image such as a traffic flow monitor or an intrusion monitor using a TV camera, a background image excluding a moving object is generated even if a moving object exists in the screen. Thus, a suitable background image can be obtained.

(B) Even when it is difficult to obtain an image without a moving object on a road with a lot of traffic such as an urban area (when only an image with a moving object can be photographed at all times), an ITV camera or the like may be used. Using the moving image obtained from the imaging device, it is possible to generate a good background image.

(C) When it is difficult to obtain an image without a moving object on a road with a large traffic volume such as an urban area (when only an image with a moving object can always be captured)
Also, it is possible to generate a good background image at high speed using a moving image obtained from an imaging device such as an ITV camera. Further, even when the picked-up image has much noise and the pixel value is unstable, a good background image can be generated at high speed.

(D) Even when it is difficult to obtain an image without a moving object on a road with a lot of traffic such as an urban area and can always photograph only an image with a moving object, an imaging device such as an ITV camera can be used. It is possible to generate a good background image at high speed using the moving image obtained from. Also, even if the background fluctuates during the generation of the background image due to weather fluctuation or the like, the already registered pixels are updated using the latest input image,
Even if a pixel is registered as a background image before the background change, the change can be absorbed.

(E) Even when it is difficult to obtain an image without a moving object on a road with a large traffic volume such as an urban area (when only an image with a moving object can be photographed at all times), an ITV camera or the like may be used. A good background image can be generated at high speed using a moving image obtained from the imaging device. If the obtained background image is different from the one desired by the user, the background image is regenerated. The background image of the area desired by the user has already been generated, and the undetermined background image is set in other areas. Exists, it is possible to instruct the end of the background image generation processing, and it is possible to supply an interface that allows the user to easily control the apparatus.

[Brief description of the drawings]

FIG. 1 is a block diagram of a background image generation system showing a first embodiment of the present invention.

FIG. 2 is an explanatory diagram of an operation of a count buffer of the background image generation system according to the first embodiment of the present invention.

FIG. 3 is a block diagram of a background image generation system showing a second embodiment of the present invention.

FIG. 4 is a block diagram of a background image generation system showing a third embodiment of the present invention.

FIG. 5 is a block diagram of a background image generation system showing a fourth embodiment of the present invention.

FIG. 6 is a diagram illustrating an example of an image displayed on an image display device of a background image generation system according to a fourth embodiment of the present invention.

[Explanation of symbols]

101, 201, 301, 401 Imaging device 102, 202, 302, 402 Frame buffer 103, 203, 303, 403 Change detection means 104, 204, 304, 404 Change area mask 105, 205, 305, 405 Count buffer update means 106 , 206, 306, 406 Count buffer 107, 207, 307, 407 Background mask updating means 108, 208, 308, 408 New registration mask 109, 209, 309, 409, 503 Background determination mask 110, 210, 310, 410 Threshold value storage Means 111, 212 Background registration means 112, 213, 312, 412 Background image recording means 113, 214, 313, 413 Background creation end determination means 114, 215, 314, 414 Image display device 115, 216, 315, 41 6 Background Image Generation Device 211 Threshold Value Changing Unit 311, 411 Background Update Unit 415 Input Unit 501 Input Image 502 Background Image 504 Background Image Undetermined Area

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. 7 Identification symbol FI Theme coat ゛ (Reference) G06F 15/70 405 F-term (Reference) 5B057 AA19 BA02 DA16 DA20 DC32 5C054 CC00 CE12 FC01 FC04 FC12 FC15 FF03 GA04 GB14 HA18 HA30 5C084 AA01 AA06 BB01 BB31 DD12 GG52 GG54 GG65 GG78 5C087 AA08 AA19 EE06 EE07 EE14 GG02 GG06 GG18 GG19 GG31 GG43 5L096 BA02 CA02 EA37 GA10 GA51 HA02 LA05 LA08

Claims (4)

[Claims]
1. An imaging device that captures an image of a predetermined monitoring area, (b) a background image generation device that processes the captured image and generates a background image, and (c) displaying the background image. And an image display device for presenting to a user. (D) the background image generation device includes: a frame buffer that stores an input image captured by the imaging device; The absolute value of the difference between the input image at the time and the input image one time before the current time is binarized, and the value of the pixel whose change is detected is 0, and the value of the pixel whose change is not detected is 1 A change detecting means for creating an area mask; a pixel in which no change is detected by the change detecting means counts up the value of the count buffer by one; and a pixel in which a change is detected resets the value of the count buffer to zero. Count buffer updating means; threshold value storing means for storing a threshold value used for comparing each pixel value of the count buffer; and a value of each pixel of the count buffer being larger than the threshold value of the threshold value storing means, and determined in a background fixed mask. A background mask updating means for setting the pixel value of the newly registered mask to 1 for a pixel which has not been set and setting the pixel value of the background fixed mask to 0; In the background registration means for copying the value to the background image, and in the background determination mask, it is determined whether or not all the pixels have been determined. If the determination has not been performed, the process is continuously performed. A background image generation system comprising: a background creation end determination unit that outputs and ends processing.
2. The background image generation system according to claim 1, wherein the background image generation device determines whether or not undetermined pixels in the background determination mask are noise. A background image generation system, further comprising a threshold changing unit that changes a threshold stored in the background image in a direction of decreasing the threshold.
3. An imaging device for capturing an image of a predetermined monitoring area; (b) a background image generation device for processing the captured image to generate a background image; and (c) displaying the background image. And an image display device for presenting to a user. (D) the background image generation device includes: a frame buffer that stores an input image captured by the imaging device; The absolute value of the difference between the input image at the time and the input image one time before the current time is binarized, and the value of the pixel whose change is detected is 0, and the value of the pixel whose change is not detected is 1 A change detecting means for creating an area mask; a pixel in which no change is detected by the change detecting means counts up the value of the count buffer by one; and a pixel in which a change is detected resets the value of the count buffer to zero. Count buffer updating means; threshold value storing means for storing a threshold value used for comparing each pixel value of the count buffer; and a value of each pixel of the count buffer being larger than the threshold value of the threshold value storing means, and determined in a background fixed mask. For a pixel that has not been set, the pixel value of the newly registered mask is set to 1, and the pixel value of the background fixed mask is set to 0. For a pixel that has already been registered in the background image, Update using the new registration mask and the value 1
The pixel of (1) determines whether or not all the pixels have been determined in the background updating unit that copies the pixel value of the input image at the time to the background image, and if the pixels have not been determined, continues the processing. A background image generation system comprising: a background creation end determination unit that outputs a background image if the determination is made and ends the processing.
4. The background image generation system according to claim 3, wherein the background creation end determination means determines whether or not all pixels are determined in the background mask, and if not determined, continues processing. To sequentially output images obtained by synthesizing the input image, the background image, and the background determination mask, interpret the command when a user inputs a command, and terminate the process if all the background images have been determined. A background image generation system, characterized in that:
JP11215064A 1999-07-29 1999-07-29 System for generating background picture Withdrawn JP2001043458A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP11215064A JP2001043458A (en) 1999-07-29 1999-07-29 System for generating background picture

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP11215064A JP2001043458A (en) 1999-07-29 1999-07-29 System for generating background picture

Publications (1)

Publication Number Publication Date
JP2001043458A true JP2001043458A (en) 2001-02-16

Family

ID=16666166

Family Applications (1)

Application Number Title Priority Date Filing Date
JP11215064A Withdrawn JP2001043458A (en) 1999-07-29 1999-07-29 System for generating background picture

Country Status (1)

Country Link
JP (1) JP2001043458A (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006203444A (en) * 2005-01-19 2006-08-03 Inc Corp Background image formation device and program
US8098897B2 (en) 2006-11-07 2012-01-17 Olympus Corporation Multi dimensional imaging system
JP2012060299A (en) * 2010-09-07 2012-03-22 Canon Inc Image pickup device, control method thereof, program, and recording medium
US8179471B2 (en) 2009-01-30 2012-05-15 Olympus Corporation Focusing device and image pickup apparatus
JP2014074623A (en) * 2012-10-03 2014-04-24 Denso Corp Vehicle navigation system
JP2014081676A (en) * 2012-10-12 2014-05-08 Nippon Telegr & Teleph Corp <Ntt> Foreground detection device, foreground detection method, and computer program
JP2016503913A (en) * 2012-12-17 2016-02-08 ボリーメディアコミュニケーションズ(シンチェン)カンパニーリミテッドBoly Media Communications (Shenzhen)Co., Ltd Security monitoring system and corresponding alarm triggering method
US9500495B2 (en) 2012-10-03 2016-11-22 Denso Corporation Vehicular navigation system

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006203444A (en) * 2005-01-19 2006-08-03 Inc Corp Background image formation device and program
JP4596929B2 (en) * 2005-01-19 2010-12-15 日本放送協会 Background image generation apparatus and background image generation program
US8098897B2 (en) 2006-11-07 2012-01-17 Olympus Corporation Multi dimensional imaging system
US8179471B2 (en) 2009-01-30 2012-05-15 Olympus Corporation Focusing device and image pickup apparatus
JP2012060299A (en) * 2010-09-07 2012-03-22 Canon Inc Image pickup device, control method thereof, program, and recording medium
JP2014074623A (en) * 2012-10-03 2014-04-24 Denso Corp Vehicle navigation system
US9500495B2 (en) 2012-10-03 2016-11-22 Denso Corporation Vehicular navigation system
JP2014081676A (en) * 2012-10-12 2014-05-08 Nippon Telegr & Teleph Corp <Ntt> Foreground detection device, foreground detection method, and computer program
JP2016503913A (en) * 2012-12-17 2016-02-08 ボリーメディアコミュニケーションズ(シンチェン)カンパニーリミテッドBoly Media Communications (Shenzhen)Co., Ltd Security monitoring system and corresponding alarm triggering method

Similar Documents

Publication Publication Date Title
US8462218B2 (en) Real-time image stabilization
EP1805715B1 (en) A method and system for processing video data
JP3103894B2 (en) Apparatus and method for correcting camera shake of video data
US8027534B2 (en) Image processing apparatus, image processing method, and program
US6909806B2 (en) Image background replacement method
CN102348128B (en) Surveillance camera system having camera malfunction detection function
US7035430B2 (en) Intruding object detection method and intruding object monitor apparatus which automatically set a threshold for object detection
US8687072B2 (en) System and method for creating composite images by utilizing an imaging device
JP3406313B2 (en) Method and apparatus for changing the content of a continuous image
CN1929602B (en) Virtual mask for use in autotracking video camera images
JP3679512B2 (en) Image extraction apparatus and method
US8050459B2 (en) System and method for detecting pedestrians
US7082209B2 (en) Object detecting method and object detecting apparatus and intruding object monitoring apparatus employing the object detecting method
EP1570652B1 (en) Image stabilisation system and method
KR101611440B1 (en) Method and apparatus for processing image
JP3461626B2 (en) Specific image region extraction method and specific image region extraction device
EP2278549A1 (en) Method and apparatus for extracting segmentation mask
JP4666706B2 (en) Moving image real-time conversion method, key frame selection method, and moving image conversion system
JP2007143131A (en) Method and device for image signal processing
US7535501B1 (en) Testing of digital cameras
DE602004005358T2 (en) Object detection in pictures
US7936945B2 (en) System, method, device, and computer program product for providing image correction
US20120019728A1 (en) Dynamic Illumination Compensation For Background Subtraction
US20060077262A1 (en) Imaging system and imaging method
JP5281446B2 (en) Imaging apparatus, control method thereof, and program

Legal Events

Date Code Title Description
A300 Withdrawal of application because of no request for examination

Free format text: JAPANESE INTERMEDIATE CODE: A300

Effective date: 20061003