JP5933482B2 - Image processing device for monitoring - Google Patents

Description

  The present invention relates to a monitoring image processing apparatus that detects the appearance of a notification event such as an intruder.

In a conventional image processing apparatus, for example, as disclosed in Patent Literature 1, a silhouette (change area) of a moving object is extracted using a “luminance background difference” method, and the feature amount of the change area is the feature amount. The notification processing is performed depending on whether or not the parameter setting value is met. With this configuration, the background difference logic can be configured simply.
In addition, as disclosed in Patent Document 2, for example, the conventional image processing apparatus employs a “brightness template matching” method to perform complicated inspection and positioning at high speed.

JP 2003-274395 A JP 7-129770 A

  However, in the conventional monitoring image processing apparatus, even if the change area is a regular report target to be reported, that is, a moving object having a different color from the background, if the brightness is similar to the background, The feature quantity does not match the feature quantity parameter (for example, area value), and it is difficult to extract the change area. Due to the inaccuracy of the change area, it is not possible to issue the report even though it is a regular report target. There was a problem of being unreported. Also in the template matching method, in brightness data matching, even for moving objects of different colors, accurate template matching cannot be performed as long as the brightness is similar to that of the background. There was a problem of becoming information.

  For example, in a case where a person in red clothes moves on a black background, the luminance is almost the same for both black and red. Therefore, in the conventional “brightness background difference” method and “brightness template matching” method, a difference cannot be obtained, and it is difficult to extract a change region. However, the human eye clearly has a difference in color between “black” and “red”, and oversight of moving objects in this scene, that is, misrepresentation, appears as strange.

  The present invention has been made to solve the above-described problems, and is to obtain a monitoring image processing apparatus that can always perform high-precision background subtraction without reducing accuracy and avoid misreporting. Objective.

  A monitoring image processing apparatus according to the present invention includes a video input unit that receives a current monitoring image of a monitoring target area as a video signal, and an A / A that quantizes the video signal received by the video input unit to obtain a current image. A luminance background difference that compares a D converter, the current image, and a comparison image indicating the monitoring image in a state where no alerting event appears in the current image, and extracts a luminance background difference in the monitoring image; A color background difference unit that compares the current image with the comparison image and extracts a color background difference in the monitoring image, creates a color histogram from the current image, and distributes the color histogram Based on a colorful determination unit that determines whether the current image is in a monotone state or a colorful state, and the color background difference when the colorful determination unit determines that the state is colorful Selected, in which the colorful determination section and a selector section for selecting the luminance background subtraction when it is determined that the a monotone situation.

  According to the present invention, after determining whether the current image is in a colorful state or a monotone state, the background difference method can be selected from “brightness background difference” and “color background difference”. Different moving objects can be accurately extracted as a change area even if the brightness is similar to that of the background, and misreporting can be avoided. In addition, two types of template matching, “Luminance matching” and “Color matching”, have been installed so that you can select which method to use, so moving objects with different backgrounds and colors are similar to the background. Can be extracted with high accuracy, and misreporting can be avoided.

1 is a configuration diagram of a monitoring image processing apparatus according to Embodiment 1 of the present invention. FIG. It is a flowchart of the alerting | reporting process of the monitoring image processing apparatus which concerns on Embodiment 1 of this invention. It is a flowchart of the colorful determination by the colorful determination part of Embodiment 1 of this invention. It is a figure which shows the example of the image condition whose current image is a monotone. It is a figure which shows the example of a video condition where the present image is colorful. It is a figure which shows the example of a histogram of a monotone image (FIG. 4). It is a figure which shows the example of a histogram of a colorful image (FIG. 5). It is a figure explaining a change field. It is the flowchart which showed the determination by the recognition process part 14 in detail. It is a figure which shows a color information histogram in the condition where the monotone condition and the colorful condition were mixed. It is a flowchart of the colorful determination by the colorful determination part of Embodiment 2 of this invention. It is a block diagram of the image processing apparatus for monitoring which concerns on Embodiment 3 of this invention. It is a flowchart of the alerting | reporting process of the monitoring image processing apparatus which concerns on Embodiment 3 of this invention. It is a figure explaining the brightness | luminance template arrangement | positioning by a brightness | luminance template arrangement | positioning memory | storage part. It is a figure which shows the front image of the present image of Fig.4 (a). It is a figure explaining the brightness | luminance template arrangement | positioning by a color template arrangement | positioning memory | storage part. It is a figure which shows the front image of the present image of Fig.5 (a). It is a figure which shows the example which performed template matching with the brightness | luminance template. It is a figure which shows the example which performed template matching with the color template.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
Embodiment 1 FIG.
FIG. 1 is a block diagram of a monitoring image processing apparatus according to Embodiment 1 of the present invention.
The monitoring image processing apparatus includes a video input unit 1, an A / D converter 2, a colorful determination unit 3, a current image storage unit 4, a background image storage unit 5, a background update unit 6, and a luminance background difference unit. 7, color background difference unit 8, selector unit 9, threshold value calculation unit 10, binarization unit 11, feature amount calculation unit 12, feature amount parameter setting value storage unit 13, and recognition processing unit 14 And a reporting unit 15.

The video input unit 1 receives a current monitoring video of a monitoring target area output from an imaging device (not shown) such as a TV camera installed in an electric station or the like as a video signal.
The A / D converter 2 quantizes the video signal received from the video input unit 1. That is, the analog data received from the video input unit 1 is converted into digital data.
The colorful determination unit 3 creates a color histogram from the current video data output from the A / D converter 2 (hereinafter referred to as the current image), and the current image is vividly colored (hereinafter referred to as the colorful state) due to the degree of dispersion. Or, conversely, it is determined whether the situation is similar to a monotone in which colors are suppressed (hereinafter referred to as a monotone situation).
The current image storage unit 4 receives and stores the current image output from the A / D converter 2. The current image data may be recorded as necessary, and the received current image data may be output as it is to the luminance background difference unit 7 and the color background difference unit 8 without accumulating the current image data.

  The background image storage unit 5 receives the current image output from the A / D converter 2, performs a corresponding calculation, and indicates a monitoring target area in a state where no alerting event (for example, an intruder) has appeared. A background image (for example, image data acquired by the A / D converter 2 at a timing when a notification event does not appear, hereinafter referred to as a background image) is created and stored. The background image may be recorded as necessary, the background image is not stored, the background image is generated from the current image data received from the A / D converter 2, and the luminance background difference unit 7 and the color background are directly generated. It is good also as outputting to the difference part 8, and it is good also as acquiring the background image created artificially from the outside and outputting to the brightness | luminance background difference part 7 and the color background difference part 8 as it is.

The background update unit 6 calculates the data in the background image storage unit 5 according to a certain rule, and updates the background image. In general, for example, updating is performed with a simple rule, for example, an average of the past five background images is used as a background image.
The luminance background difference unit 7 compares the two images of the current image stored in the current image storage unit 4 and the background image stored in the background image storage unit 5, and detects the moving object and the change object in the monitoring image. Extracted as luminance difference data (referred to as luminance background difference).
The color background difference unit 8 compares two images, the current image stored in the current image storage unit 4 and the background image stored in the background image storage unit 5, and Cb (blue component), Cr (red component) ) (Or color information such as RGB) is performed, and the moving object and the changed object in the monitoring image are extracted as color difference data (referred to as color background difference).
The selector unit 9 determines which of the luminance background difference and the color background difference is passed to the next process according to the determination result of the colorful determination unit 3.

The threshold value calculation unit 10 calculates a threshold value for binarizing the background difference data of the luminance background difference or the color background difference determined by the selector unit 9. For example, the optimum threshold value is generally obtained by a simple rule based on the background difference data, such as using the average value of the background difference data as a threshold value.
The binarization unit 11 binarizes the background difference data of the luminance background difference or the color background difference based on the threshold obtained by the threshold calculation unit 10. The binarized data is called a change area. However, depending on the apparatus, the background difference data may be directly recognized without performing this binarization.

The feature amount calculation unit 12 receives the binary data output from the binarization unit 11 and calculates the feature amount, for example, the area of the change region, the vertical and horizontal dimensions, and the like.
The feature parameter setting value storage unit 13 stores feature parameter parameters to be reported in advance. The feature amount parameter is a range or condition of a feature amount (for example, upper limit, lower limit of area, upper limit, lower limit of vertical and horizontal dimensions, movement amount) of a regular report target (for example, a person or a car) determined in advance by an operator.
The recognition processing unit 14 compares the feature amount data output from the feature amount calculation unit 12 with the feature amount parameter setting value of the notification target stored in the feature amount parameter setting value storage unit 13 to determine whether or not an event has occurred. Determine whether. In addition, a notification for notifying the occurrence of an event (reported event occurrence notification) is output. Note that a case where the recognition processing unit 14 determines that the feature parameter setting value is met is referred to as an event occurrence, and a notification target that generated the event is referred to as an event target.
When receiving the notification event occurrence from the recognition processing unit 14, the notification unit 15 performs the notification process. The alerting process refers to a process of making a BEEP sound, for example.

Next, the operation will be described.
FIG. 2 is a flowchart of the reporting process of the monitoring image processing apparatus according to the first embodiment of the present invention. The operation up to issue will be described with reference to FIG.
The video input unit 1 receives a video signal (video signal) output from an external imaging device such as a TV camera installed in an electric station or the like, and sends it to the A / D converter 2 (step ST1).
The A / D converter 2 converts the video signal received from the video input unit 1 into digital data, and converts the converted current video data (current image) into a colorful determination unit 3, a current image storage unit 4, and a background image storage unit 5. (Step ST2). Specifically, conversion into multi-value data of 8 bits per pixel is common.

The current image storage unit 4 stores the current image received from the A / D converter 2 (step ST3). Note that the current image may be recorded as necessary, and the received current image may be output to the luminance background difference unit 7 or the color background difference unit 8 as it is without accumulating the current image.
Based on the current image received from the A / D converter 2, the background image storage unit 5 is a background image for comparison (for example, indicating a monitoring target area in a state where a reporting event (for example, an intruder) does not appear). The image data acquired by the A / D converter 2 at the timing when the notification event does not appear is created and stored (step ST4). The background image may be recorded as necessary, the background image is not stored, the background image is generated from the current image data received from the A / D converter 2, and the luminance background difference unit 7 and the color background are directly generated. It is good also as outputting to the difference part 8, and it is good also as acquiring the background image created artificially from the outside and outputting to the brightness | luminance background difference part 7 and the color background difference part 8 as it is.

The background update unit 6 updates the background image stored in the background image storage unit 5 with a certain rule (step ST5). In general, arithmetic processing is performed based on a periodically sampled background image. For example, a method of updating an average image obtained by averaging the values of each pixel of the background images of the past five scenes as a background image, or a predetermined time A method of replacing the previous background image is common.
The luminance background difference unit 7 compares the output images from the current image storage unit 4 and the background image storage unit 5 to obtain a luminance difference for each pixel, and an image composed of the difference data (hereinafter referred to as a luminance difference image). Output (step ST6). The luminance difference image generally represents a portion where the luminance has changed when an object moves or the amount of light changes in the video. Usually, moving objects such as people and vehicles are mainly used, but when the surroundings are dark, an accurate luminance difference image may not be obtained in rare cases.

The color background difference unit 8 compares the output images from the current image storage unit 4 and the background image storage unit 5, performs the color background difference, obtains the color difference for each pixel, and displays an image (hereinafter referred to as the difference data). (Referred to as color difference image) is output (step ST7). The color difference image generally represents a portion where a color has changed when an object moves or a color amount changes in a video. Usually, moving objects such as people and vehicles are mainly used. However, when the background image and the moving object have similar colors, an accurate color difference image may not be obtained in rare cases.
An example of the color background difference calculation formula is shown below.
Color Background Difference = | Background Cb−Current Picture Cb | + | Background Cr−Current Picture Cr |
This is merely an example, and the color background difference may be obtained using another arithmetic expression.

The colorful determination unit 3 creates a color histogram based on the current image received from the A / D converter 2, and performs a colorful determination to determine whether the current image is a colorful state or a monotone state based on the distribution state (step ST8). ).
FIG. 3 is a flowchart of colorful determination by the colorful determination unit 3 according to Embodiment 1 of the present invention.
The colorful determination will be described in detail with reference to FIG.
First, the colorful determination unit 3 acquires Cb values of all the pixels of the current image received from the A / D converter 2 and creates a histogram (step ST21).
Similar to step ST21, the colorful determination unit 3 acquires the Cr values of all the pixels of the current image received from the A / D converter 2 and creates a histogram (step ST22).
The colorful determination unit 3 calculates the average of the Cb histogram created in step ST21 and calculates the deviation of Cb, that is, the degree of Cb component dispersion (step ST23).
Similar to step ST23, the colorful determination unit 4 calculates the average of the Cr histogram created in step ST22, and calculates the Cr deviation, that is, the dispersion degree of the Cr component (step ST24).

The colorful determination unit 3 compares the deviation threshold value determined in advance and stored in itself with the Cb and Cr deviation results calculated in steps ST23 and ST24, respectively, and one of them exceeds the threshold value. It is determined whether it is not (step ST25).
In step ST25, when either one exceeds the threshold value (in the case of “YES” in step ST25), the colorful determination unit 3 determines that the state is “colorful” and ends the process (step ST26).
On the other hand, if neither of the thresholds is exceeded in step ST25 (in the case of “NO” in step ST25), the colorful determination unit 3 determines that the state is “monotone” and ends the process (step ST27).

  The selector unit 9 receives the determination result of the colorful determination by the colorful determination unit 3 and adopts any background difference, either the luminance background difference output by the luminance background difference unit 7 or the color background difference output by the color background difference unit 8. The determined background difference is output to the threshold value calculation unit 10 (step ST9).

Here, the processing from step ST6 to step ST9 will be specifically described using an image example.
FIG. 4 is a diagram illustrating an example of a video situation in which the current image is monotone. First, output of a luminance difference image and a color difference image when the current image is monotone will be described with reference to FIG.
FIG. 4A is a current image stored in the current image storage unit 4 and is an image with a pedestrian. FIG. 4B is a background image stored in the background image storage unit 5 and is an image in a situation where there is no pedestrian.

In step ST6, the luminance background difference unit 7 compares the current image in FIG. 4A with the background image in FIG. 4B, and outputs a luminance background difference image. FIG. 4C is a luminance difference image, in which pixels having a certain amount of luminance difference are filled and overlaid on the current image.
In step ST7, the color background difference unit 8 compares the current image of FIG. 4A and the background image of FIG. 4B, and outputs a color background difference image. FIG. 4D is a color background difference image, in which pixels having a certain amount of color difference are filled and overlaid on the current image.

Since FIG. 4 is a snowy landscape, there is almost no difference between the “color” of the background image and the pedestrian, and the images are gray and black. Therefore, the color difference is very small.
On the contrary, the difference in “luminance” between the background and the pedestrian is clear, and the luminance difference can be confirmed between the snow ground and the pedestrian. Therefore, the difference in luminance is relatively large.
Therefore, the area of the portion shown by the filling in FIG. 4D is small, and the area of the portion shown by the filling in FIG. 4C is relatively large. That is, the luminance background difference image has a larger change area than the color background difference image.

FIG. 5 is a diagram illustrating an example of a video situation in which the current image is colorful. (Although no color appears on the drawing, it represents a colorful image.)
The output of the luminance difference image and the color difference image when the current image is monotone will be described with reference to FIG.
FIG. 5A is a current image stored in the current image storage unit 4 and is an image with a pedestrian. FIG. 5B is a background image stored in the background image storage unit 5 and is an image in a situation where there is no pedestrian.

In step ST6, the luminance background difference unit 7 compares the current image in FIG. 5A and the background image in FIG. 5B, and outputs a luminance background difference image. FIG. 5C is a luminance difference image, in which pixels having a certain amount of luminance difference are filled and overlaid on the current image.
In step ST7, the color background difference unit 8 compares the current image in FIG. 5A with the background image in FIG. 5B, and outputs a color background difference image. FIG. 5D is a color background difference image, in which pixels having a certain amount of color difference are filled and overlaid on the current image.

Since FIG. 5 is a daytime scenery, the difference between the background image and the “color” of the pedestrian is clear. Therefore, the background is green, and the pedestrian is blue. There is no similar color. The color difference is relatively large.
Conversely, the difference in “brightness” between the background and the pedestrian is small. Therefore, both the green background and the blue color of the pedestrian have similar values in terms of the luminance value. The difference in brightness is very small.
Therefore, the area of the portion indicated by the fill in FIG. 5C is small, and the area of the portion indicated by the fill in FIG. 5D is relatively large. That is, the color background difference image has a larger change area than the luminance background difference image.
Therefore, as shown in FIGS. 4 and 5, it can be said that using a luminance difference in a monotone situation and a color difference in a colorful situation is a method for obtaining larger difference information.

FIG. 6 is a diagram showing an example of a histogram of a monotone image (FIG. 4).
FIG. 7 is a diagram illustrating an example of a histogram of a colorful image (FIG. 5).
Here, the colorful determination in the colorful determination unit 3 will be specifically described with reference to FIGS.
In FIG. 6, since the whole color is biased to blue and there are few other colors, the histogram of the Cb component (blue component) is gathered at the center (FIG. 6A). Further, since there are few Cr components (red components), there is almost no Cr histogram (FIG. 6B).
In the monotone situation, the histograms of the Cb and Cr components gather together in this way, and the variance is small.

In FIG. 7, since green, blue, and red colors are mixed throughout, the Cb component (blue component) and Cr component (red component) histograms are scattered and dispersed in each color (FIG. 7 ( a), (b)).
In a colorful situation, the histogram of the Cb and Cr components is thus widely dispersed and the deviation is large.
The colorful determination unit 3 acquires the Cb value and Cr value of all pixels of the current image received from the A / D converter 2 and creates histograms as shown in FIGS. 6 and 7 (steps ST21 and ST22). , Cr is calculated from the dispersion values of Cr (steps ST23 and ST24), and a status of “colorful status” is determined if a constant (threshold value) determined in advance is exceeded (step ST25). -ST27).
In response to this determination, the selector unit 9 determines which background difference is to be adopted.

Specifically, for example, when the current image is FIG. 4A, the histogram is as shown in FIG. 6, and the colorful determination unit 3 determines that both the deviations of Cb and Cr are below the threshold, and Cb , Cr component histograms are gathered together, and it is determined that the state is “monotone” in which the variance is smaller than a certain value. In response to this, the selector unit 9 determines that the luminance background difference image (FIG. 4C) output from the luminance background difference unit 7 is adopted.
On the other hand, for example, when the current image is FIG. 5A, the histogram is as shown in FIG. 7, and the colorful determination unit 3 determines that the deviation of Cb and Cr exceeds the threshold, and the Cb and Cr components It is determined that the “colorful situation” in which the histogram is widely distributed than a certain value. In response to this, the selector unit 9 determines to adopt the color background difference image (FIG. 5D) output from the color background difference unit 8.

Returning to the flow of FIG.
In step ST9, when the selector unit 9 outputs the determined background difference of the luminance background difference and the color background difference to the threshold value calculation unit 10, the threshold value calculation unit 10 binarizes the received background difference data. A threshold value is calculated (step ST10). Since the background difference data is multi-valued data and it is uncertain to perform the recognition process as it is, it is general that the difference data is binarized and converted into binarized data in order to perform the subsequent processing at high speed. . As for the threshold value, for example, an optimum threshold value is generally obtained by a simple rule based on the background difference data, for example, using an average value of background difference data as a threshold value. This is only an example, and the threshold value may be set by another method.

The binarization unit 11 binarizes the background difference data with the threshold calculated by the threshold calculation unit 10 in step ST10 (step ST11). Hereinafter, the background difference data is referred to as a change area. However, depending on the apparatus, the background difference data may be directly recognized without performing this binarization.
A change area | region is demonstrated based on FIG.
FIG. 8A shows a normal main gate image. As shown in FIG. 8A, the person is now moving in the image. In this case, the position of the moving person becomes the change area as it is.
FIG. 8B is a video showing the current image superimposed on the change area of FIG.

  The feature amount calculation unit 12 receives the change area binarized by the binarization unit 11 in step ST11, and calculates the feature amount (step ST12). The feature amount refers to the area of the change region, vertical and horizontal dimensions, and the like. In the first embodiment, for example, the feature amount is “continuity”, “area”, “vertical / horizontal dimension”, and “speed”. “Continuity” refers to the continuous time of how many frame change areas have survived, “Area” refers to the number of pixels in the change area, and “Vertical / Horizontal dimensions” refers to the length and width of the circumscribed rectangle of the change area. “Speed” represents the moving speed of the change region.

  The recognition processing unit 14 compares the feature amount data calculated by the feature amount calculation unit 12 in step ST12 with the feature amount parameter setting value stored in the feature amount parameter setting value storage unit 13, and determines whether or not an event has occurred. That is, it is determined whether or not a notification target has occurred, and if it is a notification target, a notification target notification is output (step ST13). Specifically, whether or not the feature amount data calculated by the feature amount calculation unit 12 matches a range or condition indicated in the feature amount parameter setting value stored in the feature amount parameter setting value storage unit 13. If they match, the event target is found, and an event notification (notification event occurrence notification) is output.

FIG. 9 is a flowchart showing in detail the determination by the recognition processing unit 14.
When the feature amount of the change region calculated by the feature amount calculation unit 12 in step ST12 is received, the recognition processing unit 14 determines whether or not the change region exists continuously for a predetermined time (step ST31). Specifically, it is determined that the continuous time in which the change area exists is OK when the range or condition stored in the feature parameter setting value storage unit 13 is satisfied, and is NG when it does not match. For example, when the range of the feature parameter setting value is “minimum 4 frames and maximum 10 frames”, it is determined that the continuous time is 4 to 10 frames, and NG is determined otherwise.

If “NG” is determined in step ST31, the subsequent processing is skipped and the processing ends.
If “OK” is determined in step ST31, the recognition processing unit 14 determines whether or not the area is a predetermined value among the feature values of the changed region (step ST32). Specifically, it is determined to be OK when the area of the change area matches the range and condition of the feature parameter setting value, and NG when it does not match. For example, when the range of the feature parameter setting value is “minimum 500 pixels, maximum 1000 pixels”, it is determined to be OK if the area is 500 to 1000 pixels, and NG otherwise.

If “NG” is determined in step ST32, the subsequent processing is skipped and the processing ends.
If “OK” is determined in step ST32, the recognition processing unit 14 determines whether or not the vertical and horizontal dimensions of the feature amount of the change area are a predetermined value (step ST33). Specifically, it is determined to be OK when the vertical and horizontal dimensions match the range and condition of the feature parameter setting value, and when it does not match, it is determined as NG. For example, when the range of the feature parameter setting value is “minimum 50 pixels vertical, 100 pixels maximum” “minimum horizontal 50 pixels, maximum 100 pixels”, if the vertical and horizontal dimensions are 50 to 100 pixels, it is OK. Is judged as NG.

If “NG” is determined in step ST33, the subsequent processing is skipped and the processing ends.
If “OK” is determined in step ST33, the recognition processing unit 14 determines whether or not the moving speed is a predetermined value among the feature amounts of the change area (step ST34). Specifically, it is determined that the speed is OK when it matches the range or condition of the feature parameter setting value, and NG when it does not match. For example, when the range of the feature parameter setting value is “minimum 50 pixels / second, maximum 100 pixels / second”, it is determined to be OK if the vertical and horizontal dimensions are 50 to 100 pixels / second, and NG otherwise. The
If “NG” is determined in step ST34, the subsequent processing is skipped and the processing ends.
If “OK” is determined in step ST34, the recognition processing unit 14 determines that the target is a regular notification target, and outputs an event notification, that is, a notification event occurrence notification (step ST35).

The “area”, “vertical / horizontal dimensions”, and “moving speed” are compared with the feature parameter setting values by calculating the average of the feature values in each image during the continuous existence time of the change area shown in step ST31. It is common.
Further, the processes in steps ST31 to ST35 are generally performed in S / W.
The notification target notification output determination in the recognition processing unit 14 is the same as the notification target notification output determination in the conventional monitoring image processing apparatus. However, what is shown here is only an example, and the notification target notification output determination may be performed by another procedure or method.

Returning to FIG. 2, in step ST <b> 13, when the recognition processing unit 14 determines that a notification target has occurred and a notification event notification is output (in the case of “YES” in step ST <b> 13), the notification unit 15. Performs a notification process (step ST14). The alerting process is, for example, generally a BEEP sound output, lamp lighting, and notification / communication to a host computer. Note that the notification process is not limited as long as it notifies the occurrence of the notification target, and other methods may be used.
In step ST13, when the recognition processing unit 14 determines that an issue target has not occurred (in the case of “NO” in step ST13), the issue process is skipped and the process ends.

  As described above, according to the first embodiment, depending on whether the current image is monotone or colorful, it is switched whether to determine the reporting target based on the luminance background difference or to determine the reporting target based on the color background difference. Therefore, even in situations where the background (darkness) and moving objects (red coat figures) are both low in luminance and it is difficult to obtain a luminance difference background, such as a person in a red coat walking in the darkness at night, the change area It can be judged accurately. As a result, it is possible to carry out highly accurate notification processing without always reducing accuracy, and thus it is possible to avoid missing information.

Embodiment 2. FIG.
FIG. 10 is a diagram showing a color information histogram in a situation where a monotone situation and a colorful situation are mixed.
FIG. 10A shows the current image, where the upper half of the image is colorful and the lower half of the image is grass and is in a monotone situation.
When the histogram of the lower half of FIG. 10A is created, the histograms of the Cb component (blue component) and the Cr component (red component) are gathered at the center. (Refer to FIGS. 10B and 10C)
When the histogram of the upper half of FIG. 10A is created, the histograms of the Cb component (blue component) and Cr component (red component) are scattered and dispersed in each color. (See FIGS. 10 (d) and 10 (e))

In the situation shown in FIG. 10, when the background difference method is switched as described in the first embodiment, when the color background difference is used, accuracy degradation occurs in the lower half of the image, and the luminance background difference is changed. If it is used, accuracy degradation occurs in the upper half of the image, and any of these will cause a problem.
Therefore, in the second embodiment, a description will be given of an embodiment in which a highly accurate notification process is performed without causing deterioration in accuracy even in a situation where a monotone situation and a colorful situation are mixed as shown in FIG.

Since the configuration of the monitoring image processing apparatus according to the second embodiment of the present invention is the same as that shown in FIG. 1 in the first embodiment, illustration and repeated description thereof are omitted.
The monitoring image processing apparatus according to the second embodiment of the present invention differs from the monitoring image processing apparatus according to the first embodiment only in the operation of the colorful determination unit 3. Therefore, illustration and redundant description of the operation shown in the first embodiment will be omitted, and only the operation of the colorful determination unit 3 will be described.

FIG. 11 is a flowchart of colorful determination by the colorful determination unit 3 according to the second embodiment of the present invention.
In the following description, it is assumed that the current image shown in FIG. 10A is received from the A / D converter 2.
The colorful determination unit 3 divides the upper half and the lower half of the current image received from the A / D converter 2 into blocks A and B, respectively (step ST41).
Here, the colorful determination unit 3 divides the current image into the upper half and the lower half. However, the present invention is not limited to this, and it may be divided into the right half and the left half. Can be set as appropriate. In addition, here, the block is divided into two blocks, but the number of blocks is not limited to two, and two or more blocks can be set as appropriate. Further, each block does not need to have the same shape, and may be dynamically changed as appropriate.

The colorful determination unit 3 reads the Cb and Cr values of all the pixels of the current image in the upper half range of the current image shown in FIG. Is created (step ST42). The histograms of Cb and Cr values created in step ST42 are as shown in FIGS. 10 (d) and 10 (e).
The colorful determination unit 3 reads the Cb and Cr values of all the pixels of the current image in the lower half of the current image shown in FIG. Is created (step ST43). The histograms of Cb and Cr values created in step ST42 are as shown in FIGS. 10 (b) and 10 (c).

The colorful determination unit 3 obtains an average from the histogram of the Cb and Cr values in the range of the block A created in step ST42, and calculates each deviation of Cb and Cr, that is, the degree of dispersion of the Cb and Cr components (step ST44).
The colorful determination unit 3 calculates the average from the histogram of the Cb and Cr values in the range of the block B created in step ST43, and calculates each deviation of Cb and Cr, that is, the degree of dispersion of the Cb component (step ST45).
The colorful determination unit 3 compares each deviation calculated in step ST44 with a threshold value of a deviation determined in advance and stored in itself, and one of the deviations calculated in step ST44 exceeds the threshold value. It is determined whether or not there is (step ST46).

In step ST46, when one of the deviations exceeds the threshold value (in the case of “YES” in step ST46), the colorful determination unit 3 determines that the range of block A has a widely dispersed histogram of the Cb and Cr components. It is determined that the “colorful situation” in which the histograms of the Cb and Cr components are widely dispersed than a certain value (step ST47).
On the other hand, if none of the threshold values is exceeded in step ST46 (in the case of “NO” in step ST46), the colorful determination unit 3 has the Cb and Cr component histograms gathered together in the block A range, and the variance Is a “monotone situation” that is smaller than a certain value (step ST48).
In the case of FIG. 10, the colorful determination unit 3 determines that the respective deviations of the Cb and Cr values in the range of the block A exceed the threshold value and are in “colorful status”.

The colorful determination unit 3 compares each deviation calculated in step ST45 with a threshold value of a deviation determined in advance and stored in itself, and one of the deviations calculated in step ST45 exceeds the threshold value. It is determined whether or not there is (step ST49).
In step ST49, when one of the deviations exceeds the threshold value (in the case of “YES” in step ST49), the colorful determination unit 3 determines that the range of block B is that the histogram of the Cb and Cr components is wider than a certain value. It is determined that the “colorful situation” is dispersed (step ST50).
On the other hand, if none of the threshold values is exceeded in step ST49 (in the case of “NO” in step ST50), the colorful determination unit 3 determines that the range of the block B is “monotone situation” (step ST51).
In the case of FIG. 10, the colorful determination unit 3 determines that the deviations of the Cb and Cr values in the range of the block B do not exceed the threshold, the histograms of the Cb and Cr components gather together, and the variance is less than a certain value. Judged to be a small “monotone situation”.

  When the colorful determination unit 3 performs colorful determination for each block, the selector unit 9 determines a background difference to be adopted according to the background difference determined by the colorful determination unit 3 for each block. The subsequent processing is the same as that after step ST9 in FIG. 2 of the first embodiment.

  As described above, according to the second embodiment, even in a situation where colorful and monotone are mixed, the notification process can be performed without degrading the accuracy, so that the misreport can be avoided.

Embodiment 3 FIG.
FIG. 12 is a block diagram of a monitoring image processing apparatus according to Embodiment 3 of the present invention.
In addition, the overlapping description is abbreviate | omitted about the structure similar to what was demonstrated in Embodiment 1,2.
The difference between the third embodiment and the first embodiment is that the threshold value calculation unit 10 and the binarization unit 11 shown in FIG. 1 are not provided, but the luminance template arrangement storage unit 16 and the color as shown in FIG. The template arrangement storage unit 17 and the template matching unit 18 are provided.

The luminance template arrangement storage unit 16 arranges a luminance template around a large portion of the movement of the object based on the luminance difference image output from the luminance background difference unit 7, and the luminance template data is at least one frame. Remember the previous one.
Based on the color difference image output from the color background difference unit 8, the color template arrangement storage unit 17 arranges a color template around a large portion of the movement of the object, and the color template data is at least one frame. Remember the previous one.

  The template matching unit 18 includes template data stored in the luminance template arrangement storage unit 16 or the color template arrangement storage unit 17, and luminance difference image data of the latest current image (hereinafter referred to as a previous image) captured next, Alternatively, template matching is performed with color difference image data.

The operation will be described.
FIG. 13 is a flowchart of the reporting process of the monitoring image processing apparatus according to the third embodiment of the present invention.
Note that step ST1 to step ST8 in FIG. 13 are the same as step ST1 to step ST8 in FIG. 2 described in the first and second embodiments, and thus redundant description is omitted. Here, the first and second embodiments are omitted. Only the operation different from that will be described.
When the luminance difference image output from the luminance background difference unit 7 is received, the luminance template arrangement storage unit 16 specifies and stores the position of the luminance template (step ST15). The luminance template arrangement storage unit 16 stores the template image data for at least one frame.
When the color difference image output from the color background difference unit 8 in step ST7 is received, the color template arrangement storage unit 17 specifies and stores the position of the color template (step ST16). The color template arrangement storage unit 17 stores the template image data for at least one frame.

  When the colorful determination unit 3 performs colorful determination in step ST8, the selector unit 9 stores the luminance template stored in the luminance template arrangement storage unit 16 or the color template arrangement storage unit 17 in step ST9. It is determined which color template or which template is to be used for template matching, and the determined template is output to the template matching unit 18 (step ST17).

Here, the processing from step ST6 to step ST8 and step ST17 will be specifically described using image examples. As in the first embodiment, FIG. 4 shows an example of a video situation in which the current image is monotone, FIG. 5 shows an example of a video situation in which the current image is colorful, and FIG. 5 shows an example of a histogram of the monotone image (FIG. 4). FIG. 6 will be described with reference to FIG. 6, and a histogram example of the colorful image (FIG. 5) will be described with reference to FIG.
As described in the first embodiment, as shown in FIGS. 4 and 5, using a luminance difference in a monotone situation and a color difference in a colorful situation is a method for obtaining larger difference information. This also means that it is easier to distinguish the background from the moving object. In other words, the template data of the object has little similarity with the background image.
In other words, luminance template matching in monotone situations and color template matching in colorful situations means that template matching accuracy is better, and as a result, it is possible to accurately calculate feature quantities such as movement vectors of objects. To do.

Since the colorful determination by the colorful determination unit 3 is the same as in the first and second embodiments, detailed description thereof is omitted. For example, when the current image is FIG. 4A, the histogram is as shown in FIG. Thus, the colorful determination unit 3 determines that the deviations of Cb and Cr are both lower than the threshold value, and determines that the histogram of Cb and Cr components is gathered together and is a “monotone” situation with small dispersion. In response to this, the selector unit 9 determines to use the luminance template output from the luminance template arrangement storage unit 16.
On the other hand, for example, when the current image is FIG. 5A, the histogram is as shown in FIG. 7, and the colorful determination unit 3 determines that the deviation of Cb and Cr exceeds the threshold, and the Cb and Cr components It is determined that the “colorful situation” is widely distributed in the histogram. In response to this, the selector unit 9 determines to use the color template stored in the color template arrangement storage unit 17.

FIG. 14 is a diagram for explaining the luminance template arrangement by the luminance template arrangement storage unit 16. It is assumed that a luminance template is arranged based on the current image in FIG.
The luminance template arrangement storage unit 16 specifies the position of the template based on the luminance background difference image received from the luminance background difference unit 7 (or change area data after binarization thereof). Normally, as shown in FIG. 14 (a), the change area (filled portion. Here, the change area is binarized with an appropriate threshold value and converted into binary data, and is painted and superimposed. The square blocks are densely arranged at the position that overlaps with (1) and covers the change area.
As a result, the position where each block is arranged becomes a normal template position as shown in FIG.
Further, the luminance template arrangement storage unit 16 stores and stores the image data of the plurality of templates for at least one frame. This is because in order to perform template matching with the next image (previous image), it is necessary to continue storing during that time. FIG. 15 shows a previous image of the current image shown in FIG.

FIG. 16 is a diagram illustrating the luminance template arrangement by the color template arrangement storage unit 17. It is assumed that a luminance template is arranged based on the current image in FIG.
The color template arrangement storage unit 17 specifies the position of the template based on the color background difference image received from the color background difference unit 8 (or the change area data after binarizing it). Normally, as shown in FIG. 16 (a), the change area (filled portion. Here, the change area is binarized with an appropriate threshold value and converted into binary data, and is painted and superimposed. The square blocks are densely arranged at the position that overlaps with (1) and covers the change area.
As a result, the position where each block is arranged becomes a normal template position as shown in FIG.
Further, the color template arrangement storage unit 17 stores and stores the image data of the plurality of templates for at least one frame. This is because in order to perform template matching with the next image (previous image), it is necessary to continue storing during that time. FIG. 17 shows a previous image of the current image shown in FIG.

  As described above, in step ST17, the selector unit 9 adopts the luminance template as shown in FIG. 14 that is output from the luminance template arrangement storage unit 16 if the state is “monotone”, and if the state is “colorful state”. The color template output from the color template arrangement storage unit 17 is determined to be adopted as shown in FIG.

Returning to the flow of FIG.
In step ST <b> 17, when the selector unit 9 outputs the template determined between the luminance template and the color template to the template matching unit 18, the template matching unit 18 stores it in the luminance template arrangement storage unit 16 or the color template arrangement storage unit 17. As soon as the previous image is obtained from the current image storage unit 4 based on the template data, the luminance template data and the luminance image data of the previous image, or between the color template data and the color image data of the previous image. Template matching is performed between them (step ST18).

The general template method is as follows.
A brightness / color image of the same size as the brightness / color template image is cut out from the previous image, and the difference between the pixel values is calculated between the extracted previous image and the pixels at the same position in the brightness / color template, and the total is small It is determined that the similarity is higher.
The similarity with the brightness / color template image is calculated while sequentially shifting the cut-out position from the previous image, and the place with the highest similarity can be determined as the position where the previous template data has moved.

This will be specifically described with reference to the drawings.
FIG. 18 is a diagram illustrating an example in which template matching is performed using a luminance template.
18, FIG. 18 (a) is a diagram in which templates are arranged in the current image, and FIG. 18 (b) is a diagram showing positions where template data of the current image has moved.
In FIG. 18, it is assumed that FIG. 4A is the current image and FIG. 15 is the previous image, and the luminance template as shown in FIG. 14 is arranged and stored in the luminance template arrangement storage unit 16.

FIG. 19 is a diagram illustrating an example in which template matching is performed using a color template.
19, FIG. 19 (a) is a diagram in which templates are arranged in the current image, and FIG. 19 (b) is a diagram showing positions where template data of the current image has moved.
In FIG. 19, it is assumed that FIG. 5A is the current image and FIG. 17 is the previous image, and the brightness template as shown in FIG. 16 is arranged and stored in the color template arrangement storage unit 17.

In step ST18, the template matching unit 18 receives the data subjected to template matching, and the feature amount calculation unit 12 calculates the feature amount (step ST12). In Embodiments 1 and 2, for example, the feature quantity is “continuity”, “area”, “vertical / horizontal dimension”, and “speed”. However, in Embodiment 2, the vector in which the object is moved is used as the feature quantity. And
Hereinafter, the processing of step ST13 and step ST14 is the same as step ST13 and step ST14 of FIG. However, the feature amount compared by the recognition processing unit 14 in step ST13 is a movement vector, and the recognition processing unit 14 is a feature stored in the movement vector calculated from the template matching data and the feature amount parameter setting value storage unit 13. When the movement vector calculated from the matching data is a predetermined value, it is determined as a regular notification target, and an event notification, that is, a notification event occurrence notification is output.

  As described above, according to the third embodiment, the template matching method can be switched to an effective one depending on whether the current image is monotone or colorful. For example, a background such as a person in a red coat walking in the dark at night The moving object can be accurately extracted even in a situation where both the (darkness) and the moving object (the person in the red coat) have low luminance and it is difficult to obtain a luminance difference background. As a result, it is possible to carry out highly accurate notification processing without always reducing accuracy, and thus it is possible to avoid missing information.

Embodiment 4 FIG.
In the third embodiment, it has been described that the notification processing is performed by switching to an effective template matching method depending on whether the current image is monotone or colorful.
However, when the template matching method is switched by the method of Embodiment 3, for example, in a situation where a monotone situation and a colorful situation are mixed as shown in FIG. When accuracy degradation occurs and luminance template matching is used, accuracy degradation occurs in the upper half of the image, and any selection has a problem.
Therefore, in the fourth embodiment, even if the monotone situation and the colorful situation are mixed as shown in FIG. 10, the notification processing adopting the highly accurate template matching method is performed without causing the accuracy deterioration. Will be described.

Since the configuration of the monitoring image processing apparatus according to the fourth embodiment of the present invention is the same as that shown in FIG. 12 in the third embodiment, illustration and repeated description thereof are omitted.
The monitoring image processing apparatus according to the fourth embodiment of the present invention differs from the monitoring image processing apparatus according to the third embodiment only in the operation of the colorful determination unit 3. The operation of colorful determination unit 3 in the third embodiment of the present invention is the same as the operation of colorful determination 3 shown in FIG. 11 in the second embodiment.

As described in FIG. 11, the colorful determination unit divides the upper half and the lower half of the current image into blocks A and B (step ST41), and determines whether each block is in a colorful state or a monotone state (step ST41). Step ST42 to step ST51).
Here, the colorful determination unit 3 is not limited to dividing the current image into the upper half and the lower half, and may be divided into the right half and the left half, and the division unit can be set as appropriate. To do. In addition, here, the block is divided into two blocks, but the number of blocks is not limited to two, and two or more blocks can be set as appropriate. Each block does not need to have the same shape and may be dynamically changed as appropriate.

  When the colorful determination unit 3 performs colorful determination for each block, the selector unit 9 determines a template to be adopted according to the background difference determined by the colorful determination unit 3 for each block. The subsequent processing is the same as that after step ST17 in FIG. 13 of the third embodiment.

  As described above, according to the second embodiment, even when colorful and monotone are mixed, the current image is divided into a plurality of blocks, and the template matching method is switched for each block. The difference (moving object) can be extracted with higher accuracy than in the case where the processing in the screen is unified. As a result, the reporting process can be performed without degrading the accuracy, so that the misreporting can be avoided.

  In the present invention, within the scope of the invention, any combination of the embodiments, or any modification of any component in each embodiment, or omission of any component in each embodiment is possible. .

  DESCRIPTION OF SYMBOLS 1 Video input part, 2 A / D converter, 3 Colorful determination part, 4 Current image storage part, 5 Background image storage part, 6 Background update part, 7 Luminance background difference part, 8 Color background difference part, 9 Selector part, 10 Threshold calculation unit, 11 binarization unit, 12 feature quantity calculation unit, 13 feature quantity parameter setting value storage unit, 14 recognition processing unit, 15 reporting unit, 16 luminance template arrangement storage unit, 17 color template arrangement storage unit, 18 Template matching part.

Claims (3)

A video input unit for receiving a current monitoring image of the monitoring target area as a video signal;
An A / D converter that quantizes the video signal received by the video input unit into a current image;
A luminance background difference unit that compares the current image with a comparison image indicating the monitoring image in a state where a reporting event does not appear in the current image, and extracts a luminance background difference in the monitoring image;
A color background difference unit that compares the current image with the comparison image and extracts a color background difference in the monitoring image;
A color determination unit that creates a color histogram from the current image and determines whether the current image is in a monotone state or a colorful state based on the variance of the color histogram;
A selector that selects the color background difference when the colorful determination unit determines that the colorful state is present, and that selects the luminance background difference when the colorful determination unit determines that the state is the monotone state; Monitoring image processing apparatus. A video input unit for receiving a current monitoring image of the monitoring target area as a video signal;
An A / D converter that quantizes the video signal received by the video input unit into a current image;
A luminance background difference unit that compares the current image with a comparison image indicating the monitoring image in a state where a reporting event does not appear in the current image, and extracts a luminance background difference in the monitoring image;
A luminance template arrangement storage unit that arranges a luminance template based on the luminance background difference extracted by the luminance background difference unit, and stores the luminance template up to at least one frame before;
A color background difference unit that compares the current image with the comparison image and extracts a color background difference in the monitoring image;
A color template arrangement storage unit that arranges a color template based on the color background difference output by the color background difference unit and stores the color template up to at least one frame before;
A color determination unit that creates a color histogram from the current image and determines whether the current image is in a monotone state or a colorful state based on the variance of the color histogram;
A selector for selecting the color template when the colorful determination unit determines that the colorful state is selected, and a selector for selecting the luminance template when the colorful determination unit determines that the state is monotone;
A template matching unit for performing template matching between the current image newly received from the video input unit and the template selected by the selector unit;
A recognition processing unit that determines whether a notification event has occurred based on the template matching result by the template matching unit and a feature value of a notification target set in advance, and outputs a notification event notification When,
An image processing apparatus for monitoring, comprising: a notification unit that performs a notification process based on the notification of the notification event occurrence. The colorful determination unit
3. The current image is divided into a plurality of blocks, and it is determined for each of the plurality of blocks whether the current image is in the colorful state or the monotone state. An image processing apparatus for monitoring.

Images