JP2012118680A - Image processor and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To stabilize the operation of detecting a foreign matter from a photographed image by a background difference method.SOLUTION: A difference image is binarized by threshold processing (S103), foreign matters and others (noise) other than the foreign matters are discriminated, and then a threshold is adjusted for each pixel (S104). The threshold is switchable between a threshold TH1 predetermined corresponding to the magnification of the noise generated in a photographing environment and a threshold TH2 a prescribed value lower than the threshold TH1. The prescribed value is determined on the basis of the variation of the actually measured noise. In the threshold adjustment (S104), the threshold adjustment of performing adjustment to the value of the threshold TH2 when a foreign matter is discriminated at the initial threshold TH1 and returning it to the threshold TH1 thereafter when the foreign matter is not discriminated anymore in consecutively executed binarization processing by the threshold TH2 is executed for each pixel. By the adjustment, a foreign matter detecting operation is prevented from becoming instable without being affected by almost all generated noise.

Description

  The present invention relates to an image processing apparatus and an image processing method for processing a captured image of a camera that takes a moving image and detecting foreign matter appearing in the image.

As a so-called foreign object detection device, for example, an imaging device such as a video camera that captures a moving image is installed at a monitoring location, and an image appearing in the captured screen is detected as a foreign object by processing a captured image of the video camera. A moving body detection apparatus is already known (see Patent Document 1).
In this moving body detection apparatus, a technique is used in which a foreign object is detected from an image signal that is captured by a video camera using a CCD (Charge Coupled Device) or the like and converted in units of pixels. This detection method is a so-called background difference method, i.e., taking a difference between a background image of a monitoring location that has been previously captured in a state where there is no foreign object to be monitored and a captured image at the time of monitoring that location. This is a technique for detecting a foreign object from a difference value that appears when a foreign object to be monitored is present.

  By the way, it is known that noise corresponding to the shooting environment is more or less superimposed on the input image of the camera as a fluctuation component. For this reason, in the foreign object detection using the background difference method, processing for removing noise is performed so that a noise component added in addition to the signal component of the foreign object included in the obtained difference is not detected as a foreign object. That is, by this process, the difference value obtained for each pixel is subjected to a binarization process using a threshold value, so that pixels corresponding to noise are excluded from the difference image including noise, and the presence of a pixel including a foreign component is expressed as a binary signal. In other words, the foreign object can be detected as a foreign object if it is greater than or equal to the threshold value, and as a noise if it is less than the threshold value.

  Here, the threshold used for binarization may be a fixed value as long as the change in the shooting environment is small. However, when the shooting environment changes greatly, it is necessary to dynamically change the threshold in accordance with the change. That is, in general, noise fluctuation tends to be small in a bright environment and noise fluctuation tends to be large in a dark environment. Accordingly, binarization is performed so that noise is not erroneously detected as a foreign object. Therefore, it is necessary to reduce the false detection rate due to noise and increase the accuracy of detecting foreign matter by effectively changing the noise by dynamically changing the threshold value for.

  However, the conventional foreign object detection method based on the binarization process based on the threshold value has a problem that when detecting the foreign material component of the difference image, the detection operation becomes unstable due to noise generated in the foreign material itself of the captured image. In other words, if the difference value of the foreign matter is approximately equal to the binarization threshold value, the difference value of the foreign matter pixel fluctuates due to noise, so the difference value for each pixel of the difference image goes back and forth between the threshold value and the threshold value. Can be in a state of being. Therefore, accurate foreign object detection becomes difficult. As a result, the foreign object detection notification is repeatedly sent to the control unit of the monitoring system, which becomes an obstacle to the normal operation of the system. In addition, when leaving or carrying out detection is performed in which a notification is made on condition that foreign matter has been continuously detected for a certain period of time, there is a problem that the notification is not performed.

JP-A-7-160858

  The present invention has been made to solve the above-described problems of the prior art, and an object of the present invention is to detect the presence / absence of a foreign substance when performing a process of detecting the foreign substance from a captured image by a so-called background subtraction method. Even when a foreign object having a difference value equal to the threshold value exists, it is possible to accurately detect the foreign object while suppressing the influence of noise.

The present invention provides a difference image between a background image captured by a camera that takes a moving image installed at a predetermined location and converted in units of pixels, and a captured image obtained by capturing the same location by the camera after the background capture. An image processing device that performs processing for detecting foreign matter based on the threshold value processing means that compares the difference image for each pixel with a predetermined threshold value, and discriminates between the foreign pixel and other pixels based on the comparison result; A foreign matter determination means for determining whether or not the foreign object is a detection target based on an image made up of foreign matter pixels determined by the threshold processing means, a means for outputting the determination result of the foreign matter determination means as a foreign matter detection result, The threshold value used in the threshold processing means is a first threshold value that is set in advance so as not to detect noise according to the magnitude of noise that occurs in the shooting environment, and a second threshold value that is lower than the first threshold value by a predetermined value. If the threshold value used when the pixel is determined to be a foreign pixel is the first threshold value, the threshold value used for processing by the threshold value processing unit for the corresponding pixel is switched to the second threshold value. On the other hand, if the threshold value used when it is determined that the pixel is not a foreign pixel is the second threshold value, there is provided a threshold adjustment means for switching the setting to the first threshold value and adjusting the threshold value.
The present invention provides a difference image between a background image captured by a camera that takes a moving image installed at a predetermined location and converted in units of pixels, and a captured image obtained by capturing the same location by the camera after the background capture. An image processing method for performing processing for detecting foreign matter based on the threshold value processing step of comparing the difference image for each pixel with a predetermined threshold value, and discriminating between the foreign pixel and other pixels based on the comparison result, A foreign substance determination step for determining whether or not a foreign object is a detection target based on an image formed of the foreign substance pixels determined in the threshold processing step, a step of outputting the determination result of the foreign substance determination step as a foreign object detection result, The threshold value used in the threshold processing step is set between a first threshold value that is set in advance so as not to detect noise according to the magnitude of noise that occurs in the shooting environment, and a second threshold value that is lower than the first threshold value by a predetermined value. If the threshold value used when it is determined that the pixel is a foreign pixel is the first threshold value, the threshold value used in the next threshold processing for the corresponding pixel is switched to the second threshold value, but not the foreign pixel. If the threshold value used when it is determined that the threshold value is the second threshold value, the threshold value adjusting step of adjusting the threshold value by switching the setting to the first threshold value is provided.

  According to the present invention, even when there is a foreign object having a difference value equivalent to a threshold value for detecting the presence or absence of a foreign object, foreign object detection can be performed stably and accurately while suppressing the influence of noise. it can.

1 is a block diagram showing a schematic configuration of an image processing apparatus according to the present invention. It is a flowchart of the process which detects the foreign material which the image processing apparatus (FIG. 1) which concerns on this invention performs. It is a flowchart of the subsequence of the threshold value adjustment step of the process (FIG. 2) which detects a foreign material. It is a figure explaining the difference value calculated | required in the determination process of a threshold value adjustment value. It is a flowchart which concerns on one Embodiment of the process which detects the foreign material which added the calculation step of noise amount. It is a flowchart which concerns on other embodiment of the process which detects the foreign material which added the calculation step of noise amount. It is a flowchart which concerns on other embodiment of the process which detects the foreign material which added the calculation step of noise amount.

Prior to the description of the embodiments of the present invention, first, the principle of the present invention will be described.
The present invention is a background subtraction method, that is, a background image captured by a camera that captures a moving image installed at a predetermined location and converted in units of pixels, and a captured image obtained by capturing the same location after the background image is captured by the camera. A process for detecting a foreign object is performed based on an image obtained by a background difference method that obtains a difference in pixel value for each pixel.
If there is no noise, the difference image obtained by the background difference method is an image of only the difference that appears due to the presence of a foreign object from which the background has been removed. The difference image is binarized using a threshold set in consideration of not detecting the difference value, and foreign matter and other pixels are discriminated.
However, since the foreign object image itself also contains noise, if the difference value of the foreign object pixel is about the threshold value, the foreign object detection operation becomes unstable due to the effect of noise generated on the difference value, and accurate foreign object detection is performed. Becomes difficult.

Stabilization method
Therefore, in the present invention, the detection operation is stabilized by adjusting (changing) the threshold value, that is, by adjusting (changing) the binarization threshold value for discriminating a foreign substance and other pixels for each pixel.
In this threshold value adjustment, the threshold value is changed during the foreign pixel detection operation, that is, the threshold value for the next captured image is changed based on the binarization result for one captured image.
The threshold value is a value set when the foreign object detection processing system is activated (referred to as “first threshold value TH1”) and a predetermined value (hereinafter referred to as “threshold adjustment value” or simply “adjustment value”) than the first threshold value TH1. And a threshold value having a low luminance value (referred to as “second threshold value TH2”).

For the first threshold TH1, a value set when the foreign object detection processing system is started is commonly used for each pixel. The first threshold value TH1 to be set has a difference value corresponding to noise generated in the shooting environment, and therefore a predetermined threshold value is applied so as not to detect noise according to the magnitude of noise generated in the shooting environment. The method for determining the first threshold value TH1 can be implemented by adopting an existing method. For example, an experiment is performed in an on-site shooting environment, and a noise amount that changes in accordance with a change in the shooting environment at that time is calculated. Based on this, a value suitable for discriminating foreign objects and other pixels without error is determined.
The adjustment value is determined by a statistical method based on the amount of noise in the image (detailed in “determination of adjustment value” in the examples described later). That is, the amount of fluctuation due to noise in the captured image included in the foreign pixel itself is taken into account.

  When there is a pixel that is determined to be a foreign object by the first threshold value TH1, the threshold value is switched to the second threshold value TH2 that is lower than the first threshold value TH1 by a predetermined value. When processing is performed with the threshold value TH2 and exceeding this threshold is interrupted, that is, when foreign matter determination is not performed, the threshold value of the pixel is returned to the first threshold value TH1 again.

The reason for switching the threshold value is that when all the foreign matter determination processing is performed only with the first threshold value TH1, the difference value exceeds the first threshold value TH1 due to the noise included in the foreign matter pixel itself. This is because the difference value may be less than the first threshold TH1. Further, the threshold value is lowered to the second threshold value TH2, even when the pixel that has been determined to be a foreign object after exceeding the first threshold value is less than the first threshold value TH1 in the difference value in the difference image that is subsequently processed. This is because the difference value of the pixel exceeds the second threshold value TH2 so that the pixel of the foreign object image can be reliably identified.
The reason why the threshold value TH2 is returned to the first threshold value TH1 again when exceeding the second threshold value TH2 is not a threshold value determined so as not to detect noise, is to detect foreign matters stably. Since there is a possibility that the noise is misidentified as a foreign object, the possibility is reduced if the noise is returned to the first threshold TH1 which is determined not to detect the noise. Therefore, based on the first threshold TH1 This is because the foreign pixel detection is performed again.

  When the threshold value is lowered more than necessary during the adjustment from the first threshold value TH1 to the second threshold value TH2, in the worst case, switching to the first threshold value TH1 cannot be performed (that is, foreign object pixels are continuously detected). Since there is a possibility, it is necessary to determine the value to be applied in consideration of this point.

By adjusting the threshold value in this way, it is possible to stabilize the output of the binarization process for discriminating foreign objects and other pixels.
In particular, as described above, the effectiveness can be further improved in the leaving or removal detection in which notification is made on the condition that foreign matter is continuously detected for a certain period of time.
As described above, the stabilization method of this embodiment is a method of stabilizing the foreign object detection by adjusting the threshold value for each pixel. It can be used as a function of

Hereinafter, embodiments according to an image processing apparatus and an image processing method using the above principle will be described with reference to the drawings.
Here, an outline of an image processing apparatus that performs foreign object detection processing based on a difference image obtained by the background difference method will be described.
FIG. 1 is a block diagram illustrating a schematic configuration of an image processing apparatus according to the present embodiment.
As shown in FIG. 1, the image processing apparatus 10 includes a control unit 11, an image processing unit 12 that operates under the control of the control unit 11, a photographing environment detection unit 13, a storage unit 14, a display unit 15, an input unit 16, and In order to exchange data with each part of the interface part 17, it has the bus | bath 18 which connects each said part 11-17 mutually.

Although not shown, the control unit 11 controls the entire image processing apparatus and performs data processing for executing foreign object detection processing or the like, although not shown, a CPU (Central Processing Unit), a program and data necessary for processing of the CPU A RAM (Random Access Memory) for temporarily storing and the like, and a ROM (Read Only Memory) storing a program for driving the CPU to execute operations and processes, etc. .
The image processing unit 12 is a unit that performs a process of detecting a foreign object from a captured image, and includes a unit that performs a detection process that will be described in detail later with reference to the process flows of FIGS. The image processing unit 12 may be configured by a dedicated image processing circuit that performs foreign object detection processing, but may be implemented by driving a program that executes foreign object detection processing by the computer of the control unit 11.

The shooting environment detection unit 13 is provided to detect a change in the shooting environment, and processing conditions (for example, binary described later) according to changes in the shooting environment such as indoors, outdoors, daytime, night, presence / absence of sunlight, etc. This is means for notifying the control unit 11 of information necessary for the change when the threshold value used for conversion is changed. In the embodiment described later, means for detecting based on a change in the entire captured image is employed.
The storage unit 14 stores various data including image data such as a photographed image and a processed image. In addition, the storage unit 14 stores various programs necessary for foreign object detection processing, image processing, and the like used by the computer of the control unit 11. .

The input unit 16 is used to input various data including setting data necessary for foreign object detection, such as an instruction or operation for the foreign object detection process, or setting of processing (control) conditions for the foreign object detection process. Such as a touch panel.
The display unit 15 is a means for providing a function as a user interface together with the input unit 16, such as data input through the input unit 16, a captured image by the camera 1, or a processed image processed by the image processing unit 12. It is a display means such as a monitor for displaying data, and is used when displaying the result of foreign object detection processing on a captured image, when displaying setting data at the time of input, detection result, and the like.
The interface unit 17 is a means for exchanging data with an external device, and in this embodiment, the captured image from the camera 1 that captures the detection target is captured in the image processing unit 12 and the like.

"Foreign matter detection process"
The process example of the foreign material detection process which the image processing apparatus 10 performs is shown. The processing example shown below is an example of an image processing apparatus that can be implemented in a system that monitors a foreign object appearing at a predetermined monitoring location by a camera 1 installed. As soon as this monitoring system is started up, The foreign object detection process is started to start the process, and the process ends when the system is shut down.

FIG. 2 is a process flow diagram in the foreign object detection process of the present embodiment.
As shown in FIG. 2, the control unit 11 starts processing when the monitoring system is activated. First, an image photographed by the camera 1 is input through the interface unit 17 (step S101).
Next, under the control of the control unit 11, the image processing unit 12 performs a difference value (absolute value) between the input captured image and the pixel value of each pixel of the background image that is created in advance and stored in the storage unit 14. For each pixel, and a difference image is generated (step S102).
Next, the image processing unit 12 binarizes the difference image obtained in the previous stage with a threshold set for each pixel in order to discriminate foreign objects and other pixels, and obtains an intended binary image (step S103). ).

Next, the control unit 11 adjusts the threshold value used in step S103 for each pixel in order to stabilize the foreign object detection operation (step S104).
Here, an example of processing performed in this step will be described in the following “threshold adjustment processing”, and an example of processing for determining an adjustment value will be described in the following “determination of adjustment value”.

“Threshold adjustment process”
FIG. 3 is a flowchart of the subsequence of the threshold adjustment step (step S104) in the foreign object detection processing flow (FIG. 2). This sub-sequence is a procedure for performing the foreign object detection process by applying the above-described “stabilization method”.

As shown in the sequence of FIG. 3, first, the control unit 11 checks whether or not the pixel to be discriminated is discriminated as a foreign object in the binarization process using the threshold value in step S103 (step S201).
Here, when it is confirmed that it is determined as a foreign object (step S201-YES), it is confirmed whether or not the threshold value at that time is the first threshold value TH1 (see “stabilization method” above) ( Step S202).
As a result of this confirmation, if the threshold is the first threshold TH1 (step S202-YES), the second threshold TH2 that is lower than the first threshold TH1 by a predetermined value (see “stabilization method” above) for the reason already described. ) To adjust the threshold value (step S203), and this sequence is completed.
On the other hand, if the threshold value is not the first threshold value TH1 as a result of the confirmation in step S202 (step S202-NO), the threshold value in this case is the second threshold value TH2, so that the threshold value is not changed (the second threshold value). TH2) and finish this sequence.

If the pixel to be determined is not determined to be a foreign object in step S201 (step S201-NO), it is confirmed whether or not the threshold value at that time is the second threshold value TH2 (step S204).
As a result of this confirmation, if the threshold value is the second threshold value TH2 (step S204-YES), this is also changed to the first threshold value TH1 for the reason already described, and the threshold value is returned to the initial value (step S205). Finish.
On the other hand, if the threshold value is not the second threshold value TH2 as a result of the confirmation in step S204 (step S204-NO), the threshold value in this case is the first threshold value TH1, so that the threshold value is not changed (the first threshold value). TH1).

“Determination of threshold adjustment value”
In order to stabilize the foreign object detection operation by the above-described threshold adjustment (FIG. 3), the threshold adjustment value (when adjusted to the second threshold value TH2 whose luminance value is lower than the first threshold value TH1 by the predetermined value) A method for determining the predetermined value will be described.
This adjustment value is required to be adapted to the amount of noise generated in the shooting environment at the monitoring location. Specifically, the amount of noise is measured based on the captured image of the camera 1 installed at the monitoring location. Then, an adjustment value that can reliably detect foreign matter without being affected by noise, that is, stably detect foreign matter, is determined from the actually measured value.
The following determination processing example shows a method of determining the monitoring location by a statistical method based on the noise amount of the image actually captured by the camera 1. Further, the timing at which the threshold adjustment value based on the actual measurement value is determined in order to adapt to the shooting environment will be described in “noise amount calculation process” to be described later.

Two examples of <Processing example 1> and <Processing example 2> are shown as examples of adjustment value determination processing.
<Processing example 1>
This processing example is as follows. ~ 5. Execute according to the processing procedure shown in
1. A small area where no foreign matter enters is set as an image area to be detected for the amount of noise in the captured image. The reason for performing this setting is that stable noise amount information can be acquired in an image area that does not contain foreign matter. In addition, the small area setting method is automatically performed by a method of manually setting an image area that satisfies the above conditions on the monitor image by a mouse click or by image processing using a known algorithm for automatically setting an image area that does not contain a foreign object. However, any method may be used.
2. Above 1. By obtaining the inter-frame difference for the small area set in step 1, the amount of change in the luminance value for each pixel is obtained as the difference value (absolute value).
3. 2. In this order, the number of pixels having the corresponding difference value is added in order from the smaller difference value among the pixels that have obtained the difference value, and the total number is the proportion of the majority of the number of pixels in the small area (predetermined proportion As the experience value that varies depending on the shooting environment, for example, when the difference value of the corresponding pixel is occupied when 80 to 90% of the number of pixels in the small area is occupied, the obtained difference value is stored.

FIG. 4 is a graph for explaining the difference value obtained by the above method. The graph in FIG. 4 shows the distribution of the number of pixels (horizontal axis) with respect to the difference value (vertical axis). As shown in the figure, the difference value generated based on the noise gradually decreases as the difference value increases with the number of pixels having a difference value of 0 being maximized, and rapidly increases when the difference value is larger than a certain difference value. The distribution tends to be reduced in the number of pixels. A difference value exceeding a value at which the number of pixels rapidly decreases can be regarded as an abnormal value, and it is appropriate to determine this value as a limit value. Pixels having difference values that fall within this limit value occupy most of the image area to be detected, and most noise that occurs normally has difference values that fall within this limit value. Therefore, this value is set as a threshold adjustment value as a difference value for obtaining this limit value. However, here, based on the obtained difference value, the following 4. Use to determine the adjustment value.
Note that the difference value obtained by this method differs depending on the shooting environment such as brightness.

  4). 3. above. This process is repeated for a few seconds, and based on all the values obtained in the last few seconds, one of these values is calculated as an average value, a mode value, or an intermediate value. Based on the calculated value, A threshold adjustment value is determined. The foreign object detection operation can be stabilized by the adjustment value obtained by this method. 3. And 4. In order to improve the accuracy of the threshold adjustment value obtained by this process, a predetermined upper limit value is set to the threshold adjustment value obtained by this process to ensure the accuracy. That is, when the obtained threshold adjustment value exceeds a predetermined upper limit value, foreign matter enters a small area set as a detection target, or changes in the situation that affect the entire image (illumination change, change in sunlight, camera aperture Since it can be inferred that a change has occurred), 3. And 4. It is possible to deal with re-processing.

  5). 4. above. The difference value (difference value that defines the limit occupied by the majority of pixels) as the measured noise amount of the photographed image is used as the threshold adjustment value that lowers the value indicating this noise amount from the initial first threshold TH1. . That is, the second threshold value TH2 is a value obtained by subtracting this adjustment value from the first threshold value TH1.

<Processing example 2>
In <Processing Example 1>, a small area that does not contain a foreign object in the captured image is set as an image area that is a target for detecting the amount of noise. In this processing example, the entire screen of the captured image is detected. . Even in the case of a full screen, it is necessary that a foreign object does not appear in the photographed image during processing. If this condition is satisfied, it is not necessary to perform processing by the same method as in <Processing Example 1>. Does not occur.
However, when this determination process is performed at a fixed timing, such as every certain time, if the entire screen is the detection target, some foreign matter appears in the captured image obtained to obtain the difference value (limit value) due to noise. Therefore, there is a risk that the accuracy will drop accordingly. In a situation where some kind of foreign matter is reflected in the amount of noise to be obtained, considering this point, the limit value of the difference value due to noise is calculated in the above <Processing example 1> 3. In the above procedure, it is possible to suppress a decrease in accuracy by setting the ratio of the majority of the number of pixels in the target image area to be lower than that determined in the above <Processing Example 1>.

Returning to the flow of FIG. 2, if the threshold adjustment step (subsequence of FIG. 3) in step S <b> 104 is exited, then the foreign matter obtained by the binarization process of step S <b> 103 and other pixels are discriminated. Therefore, labeling is performed on the binarized image (step S105). In this step, an image determined to be a foreign object is divided into connected pixel regions (image regions in which foreign pixels are connected to form a single block), a label for identifying the region is attached to each region, and used as management information.
Next, it is determined whether or not the image of each labeled region is a foreign object to be detected (step S106). In this step, in order to detect an image that is considered to be a foreign object to be detected (for example, a human) from images that are determined to be foreign objects and are labeled, An image that matches the characteristics such as is determined, and the result is obtained.

  Next, the foreign substance determination result performed in the previous stage is output to the use destination (step S107). As the output form, for example, a method of displaying the foreign matter portion obtained as a detection result on the screen displaying the original captured image of the display unit 15 so as to be identifiable can be adopted. Further, data obtained as a detection result may be used in another control system that manages foreign matter.

After outputting the detection result, the process proceeds to the foreign object detection process for the next photographed image, but before that, the background image is updated (step S108). The background image is stored in the storage unit 14 for use when generating the difference image in step S102. However, when the photographing environment is assumed to change over time, the foreign object detection operation is performed as in this processing flow. It is necessary to update the data stored in the storage unit 14 with the latest background image taken each time in order to keep detection with high accuracy. A known algorithm can be used for the background update method.
After performing the background update, the process returns to step S101 in order to execute the foreign object detection process for the next photographed image.

  In the foreign matter detection processing flow (FIG. 2), when the foreign matter is determined by the first threshold value TH1 of the initial value in the threshold adjustment step (subsequence of FIG. 3), the above “threshold adjustment value than the first threshold value TH1. The threshold value adjusted by the method described in “Determination of the threshold value” is adjusted to a second threshold value TH2 that is lower, and pixels that are determined to be foreign objects are no longer detected in the subsequent binarization process using the second threshold value TH2. Sometimes, threshold adjustment is performed again to return to the first threshold TH1, and this threshold adjustment is performed for each pixel, so that it is not affected by most of the generated noise and prevents the foreign object detection operation from becoming unstable. And accurate foreign object detection can be performed. In particular, in the present embodiment, detection is not intermittently caused by noise, so that the effectiveness can be further improved in the detection of leaving or taking away on the condition that foreign matter is continuously detected.

Next, the timing at which the calculation of the noise amount necessary for “determination of the adjustment value” (hereinafter referred to as “noise amount calculation” processing) is performed will be described.
In the noise amount calculation process, as described above, the measured noise amount is calculated using a statistical processing method based on the pixel value of the image obtained by photographing the monitoring place in order to determine the adjustment value. A burden arises. In order to reduce this processing burden, it is desirable to reduce as much as possible if there is an imaging environment in which the accuracy of foreign object detection is maintained. Therefore, in the following embodiment, a method of performing the noise amount calculation process at a predetermined timing that can reduce the processing load without performing the foreign object detection process each time will be described.

"Noise amount calculation process"
There are mainly two methods for performing the noise amount calculation processing in the foreign object detection processing at a predetermined timing at which the processing load can be reduced.
One is a method of performing a noise amount calculation process at a predetermined timing determined in advance, and the other is a method of performing a noise amount calculation process at a timing when the shooting environment changes.
Hereinafter, the above two methods are named “processing at a predetermined timing” and “processing corresponding to a change in photographing environment”, and respective processing examples are shown.

“Processing at a specified timing”
First, as a procedure for performing the noise amount calculation process at a predetermined timing, it is performed only when the foreign object detection process is activated. For example, in a shooting environment where there is no outside light and the shooting location is illuminated under certain conditions such as indoor lighting, the foreign object detection operation is based on the assumption that there is almost no noise fluctuation until the operation is completed. This process is suitable for performing the above. In this processing example, the noise amount calculation process is performed only at the time of startup, and the threshold value is adjusted by continuing to use the calculated noise amount thereafter.

FIG. 5 is a flowchart according to a processing example of the foreign object detection process in which the noise amount is calculated only when the foreign object detection process is started.
According to the processing flow of FIG. 5, the control unit 11 starts processing when the monitoring system is activated, first performs noise amount calculation processing (step S <b> 301), and sets a threshold adjustment value (determining the above “threshold adjustment value”). ", See). The obtained threshold adjustment value is used for adjustment from the first threshold TH1 to the second threshold TH2 in the subsequent threshold adjustment (step S305).
After the process of step S301, the foreign object detection operation is performed according to the foreign object detection process flow of steps S302 to S309. The processing in steps S302 to S309 is the same as the foreign matter detection basic processing (steps S101 to S108) described with reference to FIGS. 2 and 3 in the above “foreign matter detection processing”. Therefore, the description is omitted here.

  Regarding the procedure for performing the noise amount calculation process at a predetermined timing, in the following processing example, the noise amount calculation process is performed when the foreign object detection process is activated, and then the noise amount calculation process is performed at a constant time period. For example, even in a shooting environment where the shooting location is affected by sunlight, if there is no sudden change in brightness, the amount of noise is calculated following the change at a fixed time period, and the calculated threshold adjustment value is used. Threshold adjustment is performed with an accuracy within an allowable range. According to this processing example, the processing load is increased as compared with the above processing example in which the noise amount calculation processing is performed only at the time of activation, but the limitation of the shooting environment can be more relaxed.

FIG. 6 is a flowchart according to a processing example of the foreign object detection processing in which the amount of noise is calculated at a constant time period.
The foreign object detection operation is performed in accordance with the foreign object detection processing flow in steps S402 to S411. However, the processing of steps S402 to S411 except for steps S403 and S404 is the same as the foreign matter detection basic processing (steps S101 to S108) described with reference to FIGS. 2 and 3 in the “foreign matter detection processing”. Therefore, the previous description will be referred to and the description will be omitted here.

As shown in the processing flow of FIG. 6, the control unit 11 starts processing when the monitoring system is activated, and first performs noise amount calculation processing (step S <b> 401) to obtain a threshold adjustment value (the above “threshold value”). See “Determining Adjustment Values”.
Here, when the noise amount calculation process in step S401 is performed, a timer is started in order to manage the time for the next noise amount calculation process. The threshold adjustment value obtained in this step is used for adjustment from the first threshold value TH1 to the second threshold value TH2 in the subsequent threshold adjustment (step S407).

Further, in the processing flow of this embodiment, a step for confirming the timer that was started when the noise amount calculation process was performed first is included, and the noise amount is again measured when a predetermined time set in the timer has elapsed. Perform the calculation process. As a flow, the elapse of a certain time is confirmed by a timer in step S403, and when the certain time has elapsed (YES in step S403), a noise amount calculation process is performed. Even when the noise amount calculation process is performed, the timer is reset and started again in order to manage the time for the next noise amount calculation process.
On the other hand, the elapse of the predetermined time is confirmed by a timer in step S403, and if the predetermined time has not elapsed (step S403-NO), the foreign object detection process proceeds without performing the noise amount calculation process.

“Processing for changes in shooting environment”
In the processing corresponding to the change in the shooting environment, the change in the shooting environment is checked every time the foreign object detection process is performed, and it is confirmed whether or not the noise amount is recalculated based on the checked result. A quantity calculation process is performed. For example, in a shooting environment where the shooting location is affected by sunlight, when the sunlight is blocked by clouds and sudden changes in brightness occur, if the noise amount calculation is performed at regular intervals as described above, Inconvenient inability to follow changes, but in this processing example, the amount of noise following the changes in the shooting environment is calculated by checking the change in the shooting environment during the foreign object detection process and confirming the necessity. Can always be detected with high accuracy. According to this processing example, there is a possibility that the processing load may be increased as compared with the above processing example in which the noise amount calculation processing is performed at a predetermined timing, but it is possible to further ease the limitation of the shooting environment.

In the present embodiment, a photographing environment detection unit 13 is provided in order to examine changes in the photographing environment (see FIG. 1). The shooting environment detection unit 13 uses various means including a method of estimating the change as a means for detecting changes in the shooting environment such as indoors, outdoors, daytime, night, presence / absence of sunshine, and transfer of the shooting location. However, here, it depends on the method of detecting the brightness of the shooting location.
In addition, as a method for detecting the brightness of the shooting location, in this embodiment, a method is used in which this detection is performed based on a change in the entire image based on the image taken by the camera 1.

A processing example of a method for performing a change in shooting environment based on detection of a change in the entire shot image will be described below.
This processing example is as follows. ~ 5. Execute according to the processing procedure shown in
1. A small area that does not contain a foreign object is set as an image area to be processed in the captured image. The reason for performing this setting is that a stable detection process can be performed in an image area where no foreign matter enters.
2. Above 1. By obtaining the background difference for the small region set in step 1, the amount of change in luminance value for each pixel is obtained as the difference value (absolute value). The background image used when obtaining the background difference is stored for use in the foreign object detection process.
3. 2. In this order, the number of pixels having the corresponding value from the smaller difference value is added in order, and the total number is the ratio of the majority of the number of pixels in the small area (predetermined ratio) Yes, as an experience value, for example, a value that is 90% of the number of pixels in a small area is assigned), and a difference value is obtained, and the obtained value is stored.

4). 3. above. The above process is repeated for several seconds, and based on all the values obtained in these several seconds, one of these values is calculated, and the average value, mode value, or intermediate value is calculated, and the calculated value is obtained. A value indicating the environmental change.
5. 4. above. The difference value (difference value that determines the limit occupied by most pixels) is compared with a threshold value set to determine whether or not to recalculate the noise amount, and this determination is made based on the comparison result. This determination is performed every time the foreign object detection process is performed. If the threshold value is exceeded for a certain period of time or more while the process is repeated, the brightness of the entire image is changed, that is, the intention is calculated (a noise amount is calculated). Yes) It may be a procedure for determining that a change in the shooting environment has occurred.

FIG. 7 is a flowchart according to a processing example of the foreign object detection processing in which the amount of noise is calculated in response to a change in the shooting environment.
In the illustrated flow, the processing of steps S502 to S512 excluding steps S503 to S505 is the same as the foreign matter detection basic processing (steps S101 to S108) described with reference to FIGS. 2 and 3 in the “foreign matter detection processing”. The same. Therefore, the previous description will be referred to and the description will be omitted here.

  As shown in the processing flow of FIG. 7, the control unit 11 starts processing when the monitoring system is activated. First, the control unit 11 performs noise amount calculation processing (step S <b> 501), and sets a threshold adjustment value (the above “threshold adjustment value”). The initial value is determined. The threshold adjustment value obtained in this step is used for the adjustment from the first threshold TH1 to the second threshold TH2 in the subsequent threshold adjustment (step S508).

In the processing flow of this embodiment, a change in the shooting environment is determined every time the foreign object detection process is performed, and a step of determining whether to recalculate the noise amount from the determination result is included. An amount calculation process is performed.
As a processing flow, a photographed image is input (step S502), and then it is determined whether or not the amount of noise is calculated based on a change in the brightness of the entire photographed image (photographing environment) (step S503).
Next, the determination result in step S503 is confirmed (step S504). If it is determined that the noise amount is to be calculated (step S504-YES), a noise amount calculation process is performed (step S505). At this time, the stored threshold adjustment value obtained by the noise amount calculation process and updated at this time is updated.
On the other hand, if it is determined in step S504 that it is not necessary to calculate the amount of noise (step S504-NO), the foreign object detection processing is performed without performing the noise amount calculation processing.

In the foreign matter detection processing flow (FIGS. 5 to 7), a noise amount calculation process is performed based on the captured image of the camera 1 installed at the monitoring location, and the threshold adjustment value determined based on the calculation result is set as the foreign matter detection operation. By reflecting, foreign object detection can be maintained with high accuracy while reducing the influence of noise, and foreign object detection can be performed more stably.
Further, according to the processing flow (FIG. 5) in which the noise amount calculation process is performed only at the time of startup, the burden of the noise amount calculation process can be reduced.
Further, according to the processing flow (FIG. 6) for performing the noise amount calculation processing at a constant time period, the processing load is increased as compared with the processing only at the time of activation, but the restriction of the photographing environment can be relaxed. According to the processing flow corresponding to the change of the shooting environment (FIG. 7), there is a possibility that the processing load becomes larger than the processing examples of FIGS. 5 and 6 in which the noise amount calculation processing is performed at a predetermined timing. Can be further relaxed.

  DESCRIPTION OF SYMBOLS 1 ... Camera 10 ... Image processing apparatus 11 ... Control part 12 ... Image processing part 13 ... Imaging environment detection part 14 ... Storage part 15 ... Display part 16 ... Input part , 17 · Interface part, 18 · Bus.

Claims (6)

A foreign object is captured based on a difference image between a background image that is captured by a camera that captures a moving image installed at a predetermined location and the background image is converted in units of pixels and a captured image that can be captured by the camera after the background is captured. An image processing apparatus that performs processing to detect,
A threshold processing means for comparing the difference image for each pixel with a predetermined threshold, and discriminating between the foreign pixel and the other pixels based on the comparison result;
Foreign matter determination means for determining whether or not the foreign object is a detection target based on an image made up of foreign matter pixels determined by the threshold processing means;
Means for outputting the determination result of the foreign matter determination means as a foreign matter detection result;
The threshold value used by the threshold value processing means is set to a first threshold value that is set in advance so as not to detect noise according to the magnitude of noise that occurs in the shooting environment, and a second threshold value that is lower than the first threshold value by a predetermined value. If the threshold value used when it is determined that the pixel is a foreign pixel is the first threshold value, the threshold value used for the next processing by the threshold value processing unit for the corresponding pixel is switched to the second threshold value. If the threshold value used when it is determined that the pixel is not a foreign pixel is the second threshold value, threshold adjustment means for switching the setting to the first threshold value and adjusting the threshold value;
An image processing apparatus comprising: The image processing apparatus according to claim 1,
Most of the difference values of pixels excluding abnormal values in the inter-frame difference value of each pixel obtained by obtaining the inter-frame difference value for pixels in the image area where no foreign matter enters among the images photographed by the camera An image processing apparatus, comprising: a second threshold value generating unit configured to generate a second threshold value that is used by the threshold value processing unit as a predetermined value related to a second threshold value. The image processing apparatus according to claim 2,
The second threshold generation unit is a unit that generates the second threshold based on any one of an average value, a mode value, and an intermediate value of the plurality of predetermined values obtained by repeating the process of calculating the predetermined value a plurality of times. An image processing apparatus comprising: In the image processing device according to claim 2 or 3,
An image processing apparatus that generates the second threshold value by operating the second threshold value generation unit at a predetermined timing. In the image processing device according to claim 2 or 3,
A photographing environment detecting means for detecting that the luminance value of the entire image photographed by the camera has changed,
When the change in luminance value of the entire image is detected by the photographing environment detection unit, the second threshold value generation unit operates the second threshold value generation unit to detect the second threshold value when the change of the entire image is detected. An image processing apparatus that generates the image processing apparatus. A foreign object is captured based on a difference image between a background image that is captured by a camera that captures a moving image installed at a predetermined location and the background image is converted in units of pixels and a captured image that can be captured by the camera after the background is captured. An image processing method for performing detection processing,
A threshold processing step of comparing the difference image for each pixel with a predetermined threshold, and determining a foreign pixel and other pixels according to the comparison result;
A foreign matter determination step for determining whether or not the foreign matter is a detection target based on the image formed of the foreign matter pixels determined in the threshold processing step;
Outputting the determination result of the foreign matter determination step as a foreign matter detection result;
The threshold value used in the threshold value processing step is set to a first threshold value that is determined in advance so as not to detect noise in accordance with the magnitude of noise that occurs in the shooting environment, and a second threshold value that is lower than the first threshold value by a predetermined value. If the threshold value used when the pixel is determined to be a foreign pixel is the first threshold value, the threshold value used in the threshold processing to be performed next for the corresponding pixel is switched to the second threshold value. If the threshold value used when it is determined that the threshold value is not the second threshold value, a threshold adjustment step of adjusting the threshold value by switching the setting to the first threshold value;
An image processing method characterized by comprising:

Images