JP3081396B2 - Image processing device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus, and more specifically, to a feature quantity for determining whether or not a predetermined object exists in image data taken by an image monitoring apparatus or the like. It relates to improvements such as extraction.

[0002]

2. Description of the Related Art When detecting that an intruder has entered a site such as a substation, an image of a monitoring area is captured by a monitoring camera such as an ITV and the intruder is included in the captured image data. It is necessary to determine the presence or absence of the image. To automatically perform the determination, use an image monitoring device that performs predetermined image processing (feature extraction) on the image data and detects an abnormality in the monitoring area. Sometimes.

[0003] Such a monitoring apparatus takes in images, for example, at regular time intervals, calculates the difference between several continuous monitoring images, or stores a reference image such as a background in advance, and takes in the images at regular time intervals. By performing an image difference between the monitoring image and the reference image, a changing area in the image data is obtained. By the way, if such a change area exists, it can be expected that an intruder to be detected has entered the monitoring area. However, there are cases where noise is detected or the detection is not performed on small animals such as dogs and cats.

Therefore, the image data of such a change area is represented by X
The projection is performed in the directions of the axis and the Y-axis, a common part having a large peak of the projection pattern is obtained, and the size and aspect ratio of the common part are obtained. It is determined that there is an intruder (abnormal) in the monitoring area.

[0005]

However, in the conventional method for detecting anomalies by extracting the size and aspect ratio of a lump obtained from a projected pattern as the above-mentioned feature amount, for example, a plurality of intruders are detected. If they move while overlapping, the size of the chunks and the aspect ratio become large, and detection becomes impossible.

The present invention has been made in view of the above background, and an object of the present invention is to extract a predetermined feature amount even when a plurality of intruders come in overlappingly. It is intended to provide an image processing apparatus capable of performing detection and determination with certainty.

[0007]

According to the present invention, there is provided an image processing apparatus for detecting whether or not a predetermined detection target exists in image data captured by an imaging apparatus. Means for removing unnecessary portions from the captured image data to extract predetermined judgment target data; means for receiving the judgment target data to generate an X-axis projection pattern; and receiving the judgment target data and receiving a Y-axis projection pattern. Means for generating a predetermined feature amount such as the number of objects constituting the data to be determined from the X-axis projection pattern;
A means for generating a Y-axis projection comparison pattern when the object constituting the X-axis projection pattern is a detection target from the feature quantity obtained by the means for extracting the feature quantity; A determination unit is provided for comparing the axis projection comparison pattern with the Y-axis projection pattern, and determining whether the determination target data is the detection target based on the degree of coincidence between the two patterns.

[0008]

An X-axis projection pattern of the target data to be determined is generated, and the number of objects constituting the pattern is determined from the number of peaks of the pattern. When there are a plurality of Y-axis projection comparison patterns, a Y-axis projection pattern to be obtained assuming that the X-axis projection pattern is a detection target is generated for each object by means for generating a Y-axis projection comparison pattern. By combining them, a Y-axis projection comparison pattern is generated. When there is only one object constituting the pattern, there is no need to combine them, so that the X-axis projection pattern is assumed to be the detection target, and the Y-axis projection comparison pattern is generated.

Next, the Y-axis projection comparison pattern generated as described above is compared with the Y-axis projection pattern obtained from the actually captured image data. Then, since the comparison pattern is generated assuming that it is the detection target, when the determination target data is the detection target, the shapes of both patterns are the same or similar, and when the determination target data is different from the detection target, the two patterns are compared. The shapes are different. Therefore, the degree of coincidence between the two patterns is determined, and if they match, the determination target is determined as the detection target.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of an image processing apparatus according to the present invention will be described below in detail with reference to the accompanying drawings. FIG.
1 shows a block diagram of an image monitoring apparatus according to an embodiment of the present invention. As shown in FIG. 1, after image data captured by the ITV camera 1 is digitized via an A / D converter 2, the image data is sent to a first noise processing unit 3 and is subjected to predetermined processing to obtain an isolated point. Noise is removed. In this example, the zoom (magnification) of the ITV camera 1 is fixed, and the monitoring area for capturing the image is also fixed (the direction of the camera is fixed).

The output of the first noise processing unit 3 is sent to a difference processing unit 4, where the output is compared with reference data of a background image previously captured and stored and subjected to difference processing. That is, for example, assuming that the image data of the captured n-th image is as shown in FIG.
Compared with the reference data shown in (B), a determination target (intruder candidate) M which is a changed area different from the reference data is extracted from the difference processing unit 4 as shown in FIG. Is output. Note that the processing method in the difference processing unit 4 is not limited to this, and may be any method as long as it can extract a determination target object, such as obtaining a difference between frames that are continuously captured. Can be used.

Further, the output of the difference processing unit 4 is sent to a binarization unit 5, where the image data is binarized with a predetermined threshold value, and the obtained binarized data is subjected to noise by a second noise processing unit 6. Is sent to the determination processing unit 7 according to the present invention, where it is determined whether the determination target M extracted by the difference processing unit 4 is an intruder to be detected. .

As shown in FIG. 3, the determination processing unit 7 has a Y-axis projection pattern generation unit 11 and an X-axis projection pattern generation unit 12 arranged in parallel on its input side. The converted data is input to both the generating units 11 and 12, and a projection pattern for each axis is generated, and the pattern is stored in the connected Y-axis projection memory unit 13 and X-axis projection memory unit 14, respectively.

Further, the X-axis projected pattern generation unit 12 extracts the position (y coordinate) of the head (uppermost position in the image) of each peak value while generating the pattern. That is, the binary image is sequentially scanned for pattern generation, but the value of the binary data (“1”, “0”) changes for each horizontal scan (“1” in this example).
(Hereinafter referred to as "data change"), and the X and Y coordinates at that time are stored. This data change
If it occurs in a predetermined number or more of continuous scans and its X coordinate value is within a predetermined range, it is determined that there is an object in the vicinity, and the Y coordinate in the scan where the first data change occurred Is determined as a peak. The coordinate data (y) is also stored in the X-axis projection memory unit 14.

The X-axis projection memory unit 14 is connected to an X-axis projection feature amount extraction unit 15.
From the X-axis projection pattern stored in the storage area, the number (number of persons) of the objects constituting the determination target object M, the location of the objects, the degree of overlap of the plurality of objects when there are a plurality of objects, and the movement of the objects. A predetermined feature amount for detecting a direction or the like is extracted. Further, the y-coordinate stored in the X-axis projection memory unit 14 is read at the same time. The specific processing is as follows.

First, the peak position of the peak of the X-axis projection pattern is detected. Then, the X coordinate data (x) of the peak location is extracted as the existing position in the X direction. The number of objects is calculated from the number of peaks. Then, when there are a plurality of such peaks, the degree of overlap d of the objects is detected from the distance between the peaks. That is, assuming that the binarized image is as shown in FIG. 4A, there is only one peak of the X-axis projection pattern, and thus it can be seen that there is only one object in the image. Also, as shown in FIG. 3B, it can be understood that the number of objects is two when there are two peaks. When a plurality of objects exist as shown in FIG. 7B, the difference d (d = x2−x1) between the two peaks is calculated as the degree of overlap.

Further, the moving direction of the object is to compare the X-axis projection pattern of the frame currently being processed with that of the immediately preceding frame (both stored in the X-axis projection memory unit 14). Is required. That is,
For example, assuming that the image data and the X-axis projection at the time t1 and the time t2 are as shown in FIGS.
It can be obtained from the coordinate variation (absolute value) (Δxn = xn2−xn1). If the value exceeds a predetermined threshold value th (X), it can be determined that the object is moving in the horizontal direction.

Similarly, the vertical movement amount is the peak value h.
Is obtained from the fluctuation amount (absolute value) (.DELTA.hn = hn2-hn1), and if the value exceeds a predetermined threshold value th (Y), it can be determined that the object is moving in the vertical direction. If the movement is detected in both the vertical and horizontal directions, it is determined that the movement is in an oblique direction. Further, the moving direction can be determined by the sign of each variation.

In the examples of FIGS. 5A and 5B, the X-coordinate variation Δx 1 (= x 12
Since −x11) is large, it can be determined that the object is moving in the horizontal direction. For the object M2, the peak value variation Δh2 (= h
Since 22-h21) is large, it can be determined that it is moving in the vertical direction. In this example, the reason why the moving direction is detected and specified in this manner is to determine the moving direction more accurately because the aspect ratio of a reference (imaged in an image) is different depending on the moving direction.

Then, each feature value extracted in this way is sent to the next stage Y-axis projection comparison pattern generation unit 16, where it is detected using the model data stored in the Y-axis projection pattern data memory unit 17. A Y-axis projection comparison pattern to be obtained when the obtained X-axis projection pattern is a human is generated. Then, specifically, the following procedure is performed.

That is, first, in the Y-axis projection pattern data memory unit 17, three model patterns corresponding to the above-described movement direction, that is, the vertical movement, the horizontal movement, and the oblique movement are stored. The size of each model pattern to be stored is such that the actual human height is standard (for example, 17
0 cm), which is the height when it is assumed to be at the center of the image. However, this is to make the size as small as possible so that the fluctuation rate at the time of performing the enlarging / reducing process thereafter is as small as possible and to perform high-speed processing. The shape (size) of the model pattern may be arbitrary. .

Then, as shown in FIG. 6, a corresponding model pattern P0 is determined from the feature amount data in the given moving direction by a pattern selector 16a constituting the Y-axis projection comparison pattern generator 16, and the Y-axis A predetermined model pattern is read from the projection pattern data memory unit 17. Next, the model pattern is appropriately enlarged or reduced by the enlargement / reduction unit 16b so that the size (height) of the corrected pattern becomes equal to the size information h (peak value) given from the feature amount. Is set to

When the correction patterns Pn for all the objects are generated in this way, the Y-axis projection synthesizing unit 16c performs each correction according to the position information (overlapping degree d, Y-axis coordinate (y)). Actually synthesize the pattern,
A Y-axis projection pattern at that time is generated. The pattern generated in this manner becomes a Y-axis projection comparison pattern Py, which is sent to the Y-axis projection pattern comparison unit 18.

The Y-axis projection pattern comparison unit 18 also receives a Y-axis projection pattern generated from actually picked-up image data stored in the Y-axis projection memory unit 13.
Then, those two patterns are compared. In practice, the difference between the two patterns is calculated. Then, the processing result is sent to the determination unit 19, and is compared with a predetermined reference value.

That is, since the Y-axis projection comparison pattern is created based on the data of the X-axis projection pattern on the assumption that the object is a human, if the object is a human, the actual pattern and the comparison pattern are similar. However, if there are many common parts and the difference is taken, the difference is hardly offset (see FIG. 7). However, in the case of a person such as a dog or a cat whose shape is different from that of a human, the Y-axis projection pattern is greatly different. ,
This is because most of the pattern portions remain without being offset (see FIGS. 8 and 9). The processing procedure from the generation of the projection pattern of each axis, which is a main part of the present invention, to the determination is as shown in a flowchart in FIG.

In the processing of the comparing unit 18 and the judging unit 19, the difference is obtained as described above, and the degree of coincidence between images is compared with each other, instead of detecting the magnitude of the difference with a predetermined reference value. May be.

It should be noted that the present invention is not limited to the above-described embodiment, and that all or some of the prepared model patterns may be determined in a round robin manner, or one type of model pattern may be used. In this case, since the moving direction is not relevant, it is not necessary to extract the feature amount for specifying the moving direction. Humans also change shape (aspect ratio) according to their age and gender, even if the height is the same. Therefore, a plurality of model patterns are prepared in accordance with those, and the judgment result is fed back to determine that the model is not applicable. If so, the determination process may be performed again based on another model pattern.

In the above-described embodiment, the ITV camera fixes the magnification and the photographing direction and always processes the same monitoring area. However, the present invention is not limited to this. The present invention can also be applied to a device having a variable area.
That is, for example, when a zoom function is added, it is possible to easily cope with the case where the magnification of the model pattern also changes according to the magnification.

Further, the method of detecting the existence position coordinate y in the Y-axis direction is not limited to the method performed by the X-axis projected pattern generation unit 12 as described above. The extraction unit 15 may determine the peak height hn of the X-axis projection pattern. That is, as described above, the ITV camera 1 in the present example always captures an image of a certain monitoring area in a fixed manner without zooming or the like. Therefore, the larger the target is, the larger the ITV camera 1 is. That is, as shown in FIG. 11, the distance from the object M to the ITV camera 1 is L, the installation height of the ITV camera 1 is P, the depression angle of the camera is φ, the vertical angle of view of the camera is θH, and the object (human A) is the actual height (height), f is the focal length of the lens, and CCD is mounted on the camera.
Assuming that the vertical size of is CCDv, the height h in the image is
It can be expressed by the following equation.

[0030]

H = Ht ・ (f / (L ・ cos θ + P ・ sin θ)) ・ (480 / CCDv) where Ht = L ・ (A / ((P−A) sin θ + Lscos θ)) If the heights A of the objects are equal, the height h in the image increases as the distance L to the camera increases. The height h in the image substantially matches each peak value hn of the X-axis projection pattern. Therefore, if the above h in the case where it exists at the center of the image is previously calculated (h0) and stored and held, the calculated peak value hn is compared with the reference value h0 to obtain the position on the Y axis coordinate ( The location of y) can be determined. Here, a smoothing process may be added to the projection pattern, and in this case, it is possible to prevent the influence of a case where a plurality of peaks are generated nearby due to noise or the like.

In the above-described embodiment and modified examples,
In each case, the case where the detection target is an intruder (human) is described, but the present invention is not limited to this, and can be used for detection of a desired object by appropriately changing the model pattern.

[0032]

As described above, the image processing apparatus according to the present invention detects the position of an object (overlapping state, etc.) based on the X-axis projection pattern, and generates a pattern generated for each object based on the information. Are combined to generate a Y-axis projection comparison pattern to be compared, so that even if multiple intruders come in and overlap, a predetermined feature amount is extracted and detected reliably.・ Can be determined.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an image monitoring apparatus according to an embodiment of the present invention;

FIG. 2 is a diagram illustrating an operation of a difference processing unit.

FIG. 3 is a block diagram showing a determination processing unit which is a main part of the present invention.

FIG. 4 is a diagram illustrating an operation of a determination processing unit.

FIG. 5 is a diagram illustrating the operation of a determination processing unit.

FIG. 6 is a diagram illustrating an operation of a Y-axis projection comparison pattern generation unit in a determination processing unit.

FIG. 7 is a diagram illustrating the operation of a Y-axis projected pattern comparison unit and a determination unit.

FIG. 8 is a diagram illustrating the operation of a Y-axis projected pattern comparison unit and a determination unit.

FIG. 9 is a diagram illustrating the operation of a Y-axis projected pattern comparison unit and a determination unit.

FIG. 10 is a flowchart illustrating the operation of a determination processing unit.

FIG. 11 is a diagram illustrating a relationship between an actual intruder position and image data.

[Explanation of symbols]

Reference Signs List 1 ITV (imaging device) 4 Difference processing unit (means for extracting predetermined determination target data) 7 Determination processing unit 11 Y-axis projection pattern generation unit 12 X-axis projection pattern generation unit 15 X-axis projection feature amount extraction unit 16 Y-axis Projection comparison pattern generation unit 17 Y-axis projection pattern data memory unit 18 Y-axis projection pattern comparison unit 19 Judgment unit

──────────────────────────────────────────────────の Continuing on the front page (72) Inventor Kenichi Hiramatsu 70, Yanagimachi, Saiwai-ku, Kawasaki-shi, Kanagawa Pref. Toshiba Yanagimachi Plant (56) References JP-A-3-289286 (JP, A) JP-A-4-311187 (JP, A) JP-A-62-147889 (JP, A) JP-A-5-14889 (JP, A) , A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G06T 7/00 G06T 1/00 H04N 7/18

Claims (4)

(57) [Claims] An image data captured by an image capturing apparatus includes:
An image processing apparatus for detecting whether or not there is a predetermined object to be detected; a means for removing unnecessary portions from captured image data and extracting predetermined data to be determined; Means for generating a projection pattern, means for receiving the determination target data and generating a Y-axis projection pattern, means for extracting a predetermined feature amount such as the number of objects constituting the determination target data from the X-axis projection pattern, Means for generating a Y-axis projection comparison pattern when the object constituting the X-axis projection pattern is a detection target from the feature quantity obtained by the means for extracting the feature quantity; and the generated Y An image including an axis projection comparison pattern and the Y-axis projection pattern, and a determination unit configured to determine whether the determination target data is the detection target based on the degree of coincidence between the two patterns. Management apparatus. 2. The method according to claim 1, wherein the means for extracting the feature quantity includes at least a number of peaks of the X-axis projection pattern, a distance between the peaks when a plurality of peaks exist, a height of each peak, The image processing apparatus according to claim 1, wherein the image processing apparatus extracts an existing position of the object corresponding to (a). 3. The method according to claim 1, wherein the means for generating the Y-axis projection comparison pattern enlarges or reduces a model pattern stored in advance as a standard shape of the detection target, and extracts the feature amount. Creating a corrected model pattern substantially coincident with the height of the peak, and, when there are a plurality of peaks, creating the corrected model pattern based on each peak; the distance between the peaks and the position of the object; 3. The image processing apparatus according to claim 2, wherein the Y-axis projection comparison pattern is generated by synthesizing the plurality of correction model patterns from the first model and performing Y-axis projection on the corrected model pattern or the synthesized pattern. . 4. A plurality of model patterns are prepared, a predetermined model pattern is selected from the extracted feature amounts, and a Y-axis projection comparison pattern is generated based on the selected model pattern. The image processing apparatus according to any one of claims 1 to 3.