JP3454979B2 - Abnormal state detection method and abnormal state detection device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention
The present invention relates to an abnormal state detection method and an abnormal state detection device for detecting an abnormal state in an image by monitoring an image in a monitoring area using an image pickup means such as a camera.

[0002]

2. Description of the Related Art For example, in the case of monitoring an image in a monitoring area by using an image pickup means such as an ITV camera and detecting an abnormal state in the image, first, a continuous difference between input images temporally continuous or a background image are detected. Find the difference between and. Next, by binarizing this result, a candidate for a change region that is considered to be caused by the movement of the object is extracted.

In this case, an area where an object does not exist spatially is checked in advance from the installation conditions of the ITV camera, and if it is known that the object enters the surveillance area, the object Data such as the shape is stored as a template. Too small of the extracted change areas are considered to be noise, so they are removed and the remaining candidate areas are compared with the template. Is a change that is known in advance. If there is no corresponding item, it is determined that the region is in an abnormal state because the change in the region is due to an object or the like that is unknown in advance.

[0004]

However, since the conventional abnormal state detecting method is easily affected by noise due to the vibration of the ITV camera, when the difference binary image between the images is obtained, these noises are not detected. The effect will appear. Therefore, some noise removal processing is required.

Further, it is necessary to give information on the installation position of the ITV camera in advance. Furthermore, as a result of the differential binarization process, even when extracting from the obtained candidate regions, it is necessary to compare with many templates corresponding to the objects that can occur in the image, so the calculation amount is inevitable. Tends to increase.

Further, in determining whether the generated change is due to an abnormal state or a steady state, there are many template types for possible objects for the purpose of determination based on the shape of the change region. Therefore, it is not easy to distinguish a steady change in an image from an abnormal state.

Therefore, an object of the present invention is to provide an abnormal state detecting method and an abnormal state detecting apparatus which can extract a change in an image, occurrence of an abnormal state, or the like without comparing with a template. To do.

It is another object of the present invention to provide an abnormal state detecting method and an abnormal state detecting device which do not require processing such as noise removal and can perform high speed processing.

[0009]

An abnormal state detecting method according to the present invention captures images in a monitoring area, continuously captures the captured images, and captures a plurality of captured temporally continuous images. Each is divided into a plurality of areas, the image feature amount is extracted for each of these divided areas, and each extracted area is extracted .
Extracting an abnormal state of the region by determining the duration of the change in state of the region from the feature of the temporal change of the, different for the entire region by integrating the basis of this extracted result
An abnormal state in the monitoring area is detected by checking the presence or absence of a normal state .

Further, the abnormal state detecting apparatus of the present invention includes an image pickup means for picking up an image in the monitoring area, an image fetching means for continuously fetching the images picked up by the image pickup means, and this image fetching. Image dividing means for dividing a plurality of temporally consecutive images captured by the means into a plurality of areas, respectively,
A feature extraction unit that extracts a feature amount of an image for each region divided by the image dividing unit, and a change state in the region based on a temporal change of the feature amount for each region extracted by the feature extraction unit. An abnormality extraction unit that extracts an abnormal state in the area by determining the duration and an integration result based on the extraction result of the abnormality extraction unit, and
An abnormality detecting means for detecting an abnormal state in the monitoring area by checking whether or not there is an abnormal state is provided.

[0011]

[0012]

[0013]

[0014]

[0015]

[0016]

[0017]

[0018]

According to the present invention, the image is divided into a plurality of areas and the processing using the internal feature amount is performed instead of the process using the information of the entire image, and the feature amount changes with time. By analyzing the above, it is possible to extract the change in the image, the occurrence of the abnormal state, and the like without performing the comparison with the template.

In addition, when the ITV is divided into a plurality of areas,
Since it is possible to absorb the influence of vibration of the image pickup means such as a camera, it is not necessary to perform processing such as noise removal. Further, depending on the feature amount of the image to be handled, the feature amount can be extracted in real time at the stage of capturing the image, so that the processing can be performed at high speed.

[0021]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. First, the first embodiment will be described. FIG. 1 schematically shows the configuration of the abnormal state detection device according to the first embodiment. That is, the ITV camera 1 as an image pickup means is installed, for example, as shown in FIG. 2, and continuously picks up images in the monitoring area E as shown in the drawing and converts them into electric signals. I
Image signal (analog signal) output from TV camera 1
Is digitized by a sampling pulse having a predetermined sampling rate which is input to the A / D conversion unit 2 and output from the sampling pulse generation unit 3, and then sequentially stored in the image memory 4 as image data. These input images can be confirmed by displaying them on the display device 5, and by recording and saving them in the recording device 6 such as a video tape recorder, it is possible to refer to the images later.

The processing unit 7 is mainly composed of, for example, a CPU and the like, and a plurality of images (for example, two images) captured in time series in the image memory 4 will be described in detail later. By performing the processing as described above, the abnormal state is extracted from the input image, and when it is determined that the abnormal state has occurred, the alarm device 8 is operated to notify the staff or the like. On the other hand, by recording the images before and after it is determined that an abnormal state has occurred in the recording device 6 and displaying the images on the display device 5, the attendant can confirm the images.

The processing unit 7 is loaded into the image memory 4,
For example, an input image (M × N pixels) as shown in FIG. 3 (a) is divided into a plurality of vertically and horizontally equally divided parts (eg, m × n) as shown in FIG. 3 (b). Divide into areas. Each divided area has the number of the area with respect to the total number of areas and the coordinate values at the upper left and lower right as position information, for example, as parameters (see FIG. 4).

When dividing an area, as shown in FIG. 5A, a division method that does not overlap each other, and
As shown in (b), there is a division method in which they overlap each other. Also, the sizes of the divided regions are all the same size here, but it is also possible to change the size as necessary, for example, by allocating a large number of small regions to the region that is desired to be monitored in detail. . At this time, the area is allocated to the entire image.

When actually processing an image, the input image at each time is divided into a plurality of areas as described above, and each area is subjected to the processing described below. The division of the region has been described here as being divided into rectangles for convenience, but the shape of the region is not limited to this in any case.

The process of extracting the characteristic change over time and the abnormal state in each divided area will be described in detail below. FIG. 6 is an example of a process of extracting an abnormal state from the temporal change of the feature amount in each region after dividing the input image into a plurality of regions, and according to the change of the feature amount inside the divided regions. This is a display of the changed area. The rough flow of this processing will be described with reference to the flowchart of the overall processing shown in FIG. 6 (a) to 6 (d) show the input image, and FIGS. 6 (e) to 6 (g) show how the characteristic amount of the region changes with respect to the input image.

First, with respect to each area to be divided, the time change of the feature quantity is set as a parameter in advance as an initial setting (S1). Next, at each time, an input image converted into digital image data is taken in (S2), the taken input image is divided into a plurality of areas (S3), and each divided area has some characteristics of the image. The amount is calculated (S4). In each area, the change in the characteristic amount up to now is compared with the parameters given to the area, and the presence or absence of an abnormal state is checked for each area (S4). Next, the results of the processing for each area are integrated, the presence or absence of an abnormal state is checked for all areas, and if it is determined that there is an abnormality, that fact is notified (S5).

Here, as the feature quantity in each of the divided areas, for example, in the case of a monochrome image, the average density value, variance value of the image, and the messiness (complexity) using the result of the binarization of the difference. , And the like can be used. In the case of a color image, each hue, saturation, lightness, etc. can be used.

If an average density value or a variance value is used as the feature value, the average value in the area is once obtained by the usual variance value calculation method, and the variance value is calculated again. However, when the image capture interval is short and the processing speed is high, it is considered that the change in the field of view between consecutive images is not so large, so the variance value is calculated using the average value at the previous time. It is also possible to use a so-called pseudo variance value.

Next, the process of extracting an abnormal state at each area level in step S4 in the flowchart of FIG. 7 will be described in detail with reference to the flowchart of FIG.

The feature amount d is sequentially applied to each of the divided areas.
(T) is extracted (S11), and the feature amount d (t- at the previous time is extracted.
If the absolute value of the difference from 1) is obtained and it is determined by the threshold processing that there is a change (S12), the feature amount and the parameter are compared (S13), and a predetermined value is determined by comparison with the parameter allowable range. When it exceeds the threshold value (S14), it is determined that the change in the characteristic amount is due to the abnormal state, and the abnormal state exists in the area (S15). This is performed for all areas (S16).

The threshold for extracting the above-mentioned change and the threshold for the allowable range of the parameters are set in advance by examining the change of the feature quantity with a test image and using the result. Alternatively, it may be appropriately obtained so that the change in the feature amount can be extracted well in the process of processing.

Hereinafter, some examples will be described. For example, consider FIG. 10A as an example of the temporal change of the feature amount in each area. In this case, the parameters are
As shown in FIG. 11A, the value when stable (B), the height when the change occurs (H), the duration of the changed state (L), and β as the error,
Consider 6 of η and λ.

As shown in FIG. 11 (a), these parameters may be given by investigating from the result of the analysis of the image data in the monitoring area E in advance, or as described above. The processing system may be provided with a learning function so that learning can be performed well, and learning can be performed, and the result can be given from the result.

As an example of the change over time in the feature quantity of this region, consider six examples of FIGS. 10 (b) to 10 (g). In FIG. 10B, attention is paid at the rising stage, but since it immediately returns to the original stable state, it is considered as noise.

In FIG. 10 (c), the overall shape and duration are almost the same, but the height of the peak exceeds the allowable range, so it is determined to be abnormal. In FIG. 10 (d), attention is paid at the stage of rising, and when the state returns to the original state after a lapse of the continuous time,
Since the change is similar to the parameter that is held in advance, it is a normal change.

In FIG. 10 (e), paying attention to the stage of rising, it drops once before the duration has passed, but it returns to the height of the mountain again, and finally returns to the original state after the duration has passed, Since the change is similar to the parameter that is held in advance, it is a normal change.

In FIG. 10 (f), while paying attention to the start-up stage and starting to look at the continuation time, since the change exceeds the height of the mountain from the middle, it differs from the parameters that it has in advance. , Judge that there is something wrong.

In FIG. 10 (g), while paying attention to the start-up stage and starting to look at the continuation time, since the original state does not return even after the continuation time has elapsed, something remains in the area. There is a possibility of, and it is judged to be abnormal.

As described above, if any change in the area is determined to be abnormal, the area is abnormal. Note that the above six parameters may be given assuming that some kind of change occurs in all areas, but for areas where it is known that there is no change in advance, parameters other than B have no meaning and are not given. Good. Further, in the case of monitoring a completely periodical object (such as an object on a conveyor belt conveyor), FIG.
As shown in (b), a parameter S (± σ) may be given as the change occurrence interval.

Further, in the above description, a rectangular wave that changes in the positive direction is used as an example of the temporal change of the characteristic amount, but it goes without saying that the processing corresponding thereto is performed even in other cases.

Next, the process of extracting the abnormal state at the whole area level in step S5 in the flowchart of FIG. 7 will be described in detail with reference to the flowchart of FIG.

The processing for each area described above ends (S
After 16), it is first checked whether or not the whole process is necessary (S21). If the whole process is needed,
The result of the peripheral area is checked for the area showing the abnormality in the entire area (S22), and if the area has the abnormal state (S23), it is similarly checked whether the area showing the abnormal state exists in the periphery ( S24), and if they exist, they are connected to each other (S25). At this time, when the regions indicating the abnormal state are isolated and scattered, the size of the changing regions is too small and is considered to be noise (S2).
6).

After the presence of an abnormal state is checked for all areas (S27), when the number of elements when finally connecting the areas showing the abnormal state exceeds a predetermined threshold value ( S28), and set the area group as an abnormal area (S2
9) Assuming that an abnormal state exists in the image, for example, it is notified by means such as processing stop. Even if the specified value is not exceeded, some caution will be called for.

For example, the following method can be considered for determining whether or not to process the entire area in step S21. That is, as described above, after checking whether or not an abnormal state has occurred in each area, (1) as a result of performing processing of each area, if there is an area exhibiting one or more abnormal states, (2) When the ratio of the number of areas determined to be abnormal to the total number of areas exceeds a predetermined value, if any of the following is satisfied, an abnormal state is searched for in all areas.

When the processing speed is high, it is considered that the changes in the imaging field of view are not large. Therefore, when calculating the variance value, the average value in the region is calculated once and the variance value is calculated again. It is possible to use a so-called pseudo variance value in which the variance value is obtained by using the average value of the previous time, instead of the usual calculation method.

Next, a second embodiment will be described. The difference between the second embodiment and the first embodiment is that, as shown in the flowchart of the overall processing in FIG. 12, the processing in the entire area (S
5) is not performed, and instead, the area division / fusion processing (S
6) is performed, and the division / fusion processing of the area will be described in detail below.

First, the area division processing will be described.
As shown in FIG. 13, when the temporal change of the feature amount in the divided areas has a noise-like state, it is determined that the area always has noise as it is. In order to prevent this, each divided area is divided into smaller areas to see the temporal change of the feature amount.

That is, regarding the area in the image, the temporal change of the feature amount within the area is noise-like as shown in FIG. 13, and when the change frequency is high, if this state continues for a predetermined time or more, Perform further division of the area.

This area division processing will be described with reference to the flowchart shown in FIG. After the extraction of the feature amount d (t) for each divided area in step S4 is completed, the absolute value of the difference from the feature amount d (t−1) at the previous time is obtained, and the difference data is obtained from the threshold Ts. If it is also large, there is a change (S31), and the number of times (Cs)
Is counted (S32). When this change exists continuously and the number of changes exceeds a predetermined threshold value Tcs (S33), the area is divided (S34).

As a result, since the parameter is not set in advance for the newly increased area, the parameter of the area before division is assigned (S35). Next, the counted number of times of occurrence of change (Cs) is reset (S3
6) Then, proceed to the next processing.

In dividing the area in step S34, for example, division as shown in FIG. 14 is performed. Here, the area is divided into four equal parts. For the divided areas, as in the example shown in FIG. 4, the number of the new area and the coordinate values of the upper left and lower right as position information are given as parameters, for example. In addition, depending on the division method in the initial state described above, the regions are divided so as not to have an overlap or to have an overlap (see FIG. 5).

Next, the area fusion processing will be described.
As for the region in the image, the temporal change of the feature amount in the region is hardly seen as shown in FIG. 15, and if the occurrence frequency is low and this state continues for a predetermined time or longer, the surrounding similar state Fusion with the area in.

This area fusion processing will be described with reference to the flowchart shown in FIG. After the above-described area division processing is completed, first, the state counter that counts that there is no change state is incremented by "+1" (S41).
Next, the feature quantities d (t) and d extracted as described above
The absolute value of the difference from (t-1) is obtained, and it is assumed that there is a change when the difference data is larger than the threshold value Tm (S4
2) The state counter is reset (S43), and the process for the next image is performed.

When there are continuous unchanging states and the value of the state counter exceeds a predetermined threshold Th (S44),
It is assumed that the attribute information given to the areas can be [fused] (S45), and after checking all the areas, the fusing process is performed (S46).

Regarding the area fusion processing in step S46,
This will be described with reference to the flowchart shown in FIG. First, the entire area is searched (S51), the attribute information of the area is read (S52), and it is checked whether the areas can be merged (S5).
3). If it is possible, it is checked whether the attribute information of the neighboring area can also be merged (S54), and if possible, the parameter of the processing area is given to the other party's area to be merged (S55).
For areas that have already been merged, the attribute information is returned to the normal state, and these operations are performed for all areas (S56, S57).

When merging regions, for example, FIG.
Perform fusion as shown in. In this case, the merging of the regions is performed so that the new region becomes rectangular again. That is, three neighboring areas, that is, right adjacent, lower, right adjacent, and lower than the attention area are examined.

The area dividing and merging processing has been described above. However, regarding each area in an image, when the frequency of change in the area is low, the area is not particularly subjected to the merging processing as described above. It is possible to reduce the total amount of processing by reducing the number of times the processing is performed.

When the number of areas changes by repeating the division and fusion of the areas, the areas are renumbered as necessary. When changing the
For example, there is a method in which the upper left coordinate value that each area has as a parameter is examined in the order of the upper left corner to the lower right corner of the image, and numbers are assigned.

In the above description, the case where the area fusion processing is performed after the area division processing has been described. However, the order of the processing may be reversed, or the area division processing and the area division processing may be performed. The fusion processing may be performed in parallel.

As described above, according to the above-described embodiment, the image is divided into a plurality of areas and the processing using the feature amount in each area is performed instead of the processing using the information of the entire image. In addition, by analyzing the temporal change of the feature amount, it is possible to absorb small changes such as vibration by intentionally dividing the image roughly, and to detect larger changes (such as movement of the object in the image). It becomes possible to focus only on the extraction, and it is possible to extract the change in the image or the occurrence of the abnormal state without performing the comparison with the template.

Further, since the influence of the vibration of the ITV camera or the like can be absorbed at the stage of dividing into areas, processing such as noise removal is not necessary. Further, depending on the feature amount of the image to be handled, the feature amount can be extracted in real time at the stage of capturing the image, so that the processing can be performed at high speed.

[0063]

As described in detail above, according to the present invention, an abnormal state detecting method and an abnormal state detecting method that can extract a change in an image, occurrence of an abnormal state, and the like without performing comparison with a template and the like. A device can be provided.

Further, according to the present invention, it is possible to provide an abnormal state detecting method and an abnormal state detecting device which do not require processing such as noise removal and can perform the processing at high speed.

[Brief description of drawings]

FIG. 1 is a block diagram schematically showing a configuration of an abnormal state detection device according to a first embodiment of the present invention.

FIG. 2 is a diagram showing a monitoring environment and an installation example of an ITV camera.

FIG. 3 is a diagram illustrating a process of dividing an input image into a plurality of areas.

FIG. 4 is a diagram for explaining how to give a parameter for a region.

FIG. 5 is a diagram illustrating a method of dividing a region.

FIG. 6 is a diagram showing how an input image and a feature amount of a region change.

FIG. 7 is a flowchart illustrating an overall process.

FIG. 8 is a flowchart illustrating a region level process.

FIG. 9 is a flowchart illustrating processing in all areas.

FIG. 10 is a diagram showing an example of a temporal change of a feature amount.

FIG. 11 is a diagram illustrating an example of how to determine a characteristic amount parameter.

FIG. 12 is a flowchart illustrating the overall processing of the abnormal state detection device according to the second embodiment of the present invention.

FIG. 13 is a diagram showing an example of a temporal change of a feature amount such as noise.

FIG. 14 is a diagram illustrating an example of area division.

FIG. 15 is a diagram showing an example of a temporal change of a feature amount that hardly changes.

FIG. 16 is a diagram illustrating an example of region fusion.

FIG. 17 is a flowchart illustrating region division processing.

FIG. 18 is a flowchart illustrating a region fusion process.

FIG. 19 is a flowchart illustrating region fusion processing.

[Explanation of symbols]

E ... Monitoring area, 1 ... ITV camera (imaging means), 2
... A / D converter, 3 ... Sampling pulse generator,
4 ... Image memory, 5 ... Display device, 6 ... Recording device,
7 ... Processing unit, 8 ... Alarm device.

─────────────────────────────────────────────────── ─── Continuation of the front page (58) Fields surveyed (Int.Cl. ⁷ , DB name) G08B 13/00-31/00 H04N 7/18 G06T 7/20

Claims (2)

(57) [Claims] 1. An image in a monitoring area is picked up, the picked-up image is continuously captured, the plurality of temporally continuous images thus captured are each divided into a plurality of areas, and each of the divided areas is divided. The feature quantity of the image is extracted for each
The abnormal state in the relevant area is extracted by judging the duration of the change state in the relevant area from the temporal change of the feature amount for each area, and integrated based on this extraction result to the entire area.
The abnormal state detecting method is characterized by detecting the abnormal state in the monitoring area by checking the presence or absence of the abnormal state. 2. An image pickup means for picking up an image in the monitoring area, an image fetching means for continuously fetching the images picked up by the image pickup means, and a temporally continuous fetch for the image fetching means. an image dividing means for dividing the plurality of images into a plurality of regions, a feature extraction means for extracting a feature value of the image for each of the areas divided by the image dividing means, each extracted by the feature extracting means Features for each area
Abnormality extraction means for extracting an abnormal state in the area by determining the duration of the change state in the area from the temporal change of the area, and integrating all the areas based on the extraction result of the abnormality extraction means.
An abnormal state detecting device for detecting an abnormal state in the monitoring area by checking the presence or absence of the abnormal state.