JP3613301B2 - Video surveillance and tracking device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a video monitoring and tracking apparatus, which digitally processes an image from a television camera, monitors abnormalities such as an intruder, and moves the viewing angle of the television camera in accordance with the movement of the monitored object. Regarding what to track.
[0002]
[Prior art]
In an apparatus that detects a change in a monitoring image captured by a television camera (hereinafter abbreviated as a camera) and reports an abnormality such as an intruder, a background image is captured in the absence of an abnormal object, and this image is sequentially captured. The image of the difference from the image (reference image) is calculated, the presence / absence of an abnormality is determined based on the difference image, and if it is determined to be abnormal, an alarm sound or warning display is issued and an abnormal object is displayed on the monitor screen. However, when an abnormal object passes through the monitoring screen, it is difficult to confirm on the screen and an appropriate measure cannot be taken.
[0003]
[Problems to be solved by the invention]
In view of these points, the present invention turns the direction of the camera so that the abnormal object moves to the center of the viewing angle of the camera when an abnormality such as the intrusion of a suspicious person occurs in the monitoring visual field, and the abnormal object moves. If so, the movement amount (vector amount) is calculated, and the abnormal object is tracked with a camera to facilitate monitoring.
[0004]
[Means for Solving the Problems]
In order to solve the above-described problem, the present invention monitors an abnormality based on an image of a difference between a background image when no abnormal object is present from the camera and a reference image sequentially output from the camera. A background image memory for storing a background image, a reference image memory for storing a reference image, a correlation calculation unit for calculating a correlation between data read from the background image memory and data read from the reference image memory, and a correlation calculation unit An abnormality discriminating unit that discriminates an anomaly based on the data from the data, a first edge extracting unit that extracts an edge of a reference image from data read from the reference image memory based on a signal from the anomaly discriminating unit, and the anomaly discriminating unit Subtractor for calculating the difference between the data read from the background image memory and the data read from the reference image memory based on the signal from A second edge extracting unit for extracting the edge of the difference image based on the data of the above, an AND circuit for calculating the data from the first edge extracting unit and the data from the second edge extracting unit, and the data from the AND circuit An edge circumscribing coordinate calculation unit that calculates the coordinates of a rectangle circumscribing the edge based on the image, and an image cutout unit that extracts an image in a range corresponding to the data from the edge circumscribing coordinate calculation unit from the image read from the reference image memory Edge circumscribing coordinates / image memory for storing the coordinates from the edge circumscribing coordinate calculation unit and the image from the image cutout unit, and the next image data is read from the reference image memory, and from the edge circumscribing coordinates / image memory. Compared with image data, tracking edge calculation unit that calculates edge coordinates and image coordinates that match the image from the image memory, and stores the center of the viewing angle of the camera A turning amount calculation unit for calculating a turning amount for directing the center of the viewing angle of the camera to the center of a rectangle circumscribing the edge based on data from the memory unit and data from the tracking difference calculation unit; The present invention provides a video surveillance and tracking device that includes a turning control unit that is attached to a camera and that turns the camera based on data from a turning amount calculation unit.
[0005]
DETAILED DESCRIPTION OF THE INVENTION
In the video monitoring and tracking apparatus according to the present invention, the background image when there is no abnormal object from the camera is stored in the background image memory, the reference image sequentially output from the camera is stored in the reference image memory, and is read from the background image memory. The correlation between the data and the data read from the reference image memory is calculated by the correlation calculation unit, the abnormality is determined by the abnormality determination unit based on the data from the correlation calculation unit, and the signal from the abnormality determination unit is used to output the correlation from the reference image memory. An edge of the read data is extracted by a first edge extraction unit, a difference between the data read from the background image memory and the data read from the reference image memory is calculated by a subtractor, and the second edge is calculated based on the data from the subtractor. The edge of the difference image is extracted by the extraction unit, and the data from the first edge extraction unit and the data from the second edge extraction unit are expressed by an AND circuit. Then, the edge circumscribing coordinate calculation unit calculates the coordinates of the rectangle circumscribing the edge based on the data from the AND circuit, and the range of the range corresponding to the data from the edge circumscribing coordinate calculation unit from the image read from the reference image memory. The image is cut out by the image cutout unit, the coordinates from the edge circumscribing coordinate calculation unit and the image from the image cutout unit are stored in the edge circumscribing coordinate / image memory, and the tracking difference calculation unit stores the next image from the reference image memory. The image of the camera is read out and compared with the image data from the edge circumscribing coordinates and image memory, the coordinates of the image that matches the image from the edge circumscribing coordinates and image memory are calculated, and the viewing angle of the camera stored in the memory unit The amount of rotation for directing the center of the viewing angle of the camera to the center of the rectangle circumscribing the edge based on this data and the data from the tracking difference calculation unit Calculated at times amount calculation unit, and turning the direction of the camera via the swivel control unit which is attached to the camera to track the abnormal object.
[0006]
【Example】
Embodiments of a video surveillance / tracking apparatus according to the present invention will be described below in detail with reference to the drawings. FIG. 1 is a block diagram of a main part of an embodiment of a video surveillance / tracking apparatus according to the present invention. In the figure, reference numeral 1 denotes a camera, which is installed with a viewing angle in a preset direction. An A / D converter 2 converts the video signal from the camera 1 into a digital signal. A background image memory 3 records a background image when no abnormal object exists. Reference numeral 4 denotes a reference image memory, which records images that are sequentially output from the camera 1. A correlation calculation unit 5 calculates the correlation (similarity) between the image data read from the background image memory 3 and the image data read from the reference image memory 4. Reference numeral 6 denotes an abnormality determination unit that determines the presence or absence of an abnormality based on data from the correlation calculation unit 5. Reference numeral 7 denotes a control unit that controls the identification and tracking of an abnormal object when the abnormality determination unit 6 determines an abnormal state. Reference numeral 8 denotes an edge extraction unit that extracts the edge of the image read from the reference image memory 4 (differentiates the level difference between the shades of the image). A subtractor 9 calculates the difference between the image data read from the reference image memory 4 and the image data read from the background image memory 3. Reference numeral 10 denotes an edge extraction unit that extracts an edge of a difference image from the subtractor 9. An AND circuit (AND gate) 11 calculates data from the edge extraction unit 8 and the edge extraction unit 10.
[0007]
Reference numeral 12 denotes an edge circumscribing coordinate calculation unit that calculates the coordinates of the upper side, the lower side, the left side, and the right side of the quadrangle circumscribing the edge image based on the data from the logical product circuit 11. Reference numeral 13 denotes an image cutout unit that cuts out image data of a rectangular area calculated by the edge circumscribed coordinate calculation unit 12 from the next image data read from the reference image memory 4. Reference numeral 14 denotes an edge circumscribing coordinate / image memory, which stores the rectangular coordinate data from the edge circumscribing coordinate calculation unit 12 and the image data of the rectangular region cut out by the image cutout unit 13. Reference numeral 15 denotes a tracking difference calculation unit, which sequentially compares the image data from the edge circumscribing coordinates / image memory 14 with the image data read from the reference image memory 4, and matches the image from the edge circumscribing coordinates / image memory 14. The coordinates of the image to be calculated are calculated. Reference numeral 16 denotes a turning amount calculation unit. Based on data from the tracking difference calculation unit 15, a turning amount (vector amount) for directing the center of the viewing angle of the camera 1 to the center of the rectangle calculated by the edge circumscribing coordinate calculation unit 12 is shown. calculate. Reference numeral 17 denotes a turning control unit, which turns the camera 1 based on data from the turning amount calculation unit 16. Reference numeral 18 denotes a memory unit that integrates and records preset orientation data of the camera 1 and turning amount data from the turning control unit 17.
[0008]
FIG. 2 shows an example in which data processing units 21 and 22 are provided after the edge extraction units 8 and 10. The data processing units 21 and 22 round up all the bit data of each pixel from the edge extraction unit 8 or 10 from the most significant bit of “1” to the least significant bit to “1” data. For example, if the input data is 8 bits and “00100010”, the fourth digit from the top is rounded up to “1” to be “00111111”. As a result, the edge is emphasized, the edge of the abnormal object becomes clear, and the abnormal object is easily identified. Alternatively, as shown in FIG. 3, binarization processing units 31 and 32 are provided at the subsequent stage of the edge extraction units 8 and 10, and the data of each pixel from the edge extraction unit 8 or 10 is binarized by a preset threshold value. Convert to data, erase unclear data including noise, etc., and make it possible to extract abnormal objects clearly.
[0009]
Next, the operation of the video monitoring and tracking apparatus according to the present invention will be described with reference to the image diagram of FIG. First, prior to the start of monitoring, the video signal of the background image (A) from the camera 1 is converted into a digital signal by the A / D converter 2 and input to the background image memory 3 to store one screen. This background image (A) is a case where there is a door 41 and a staircase 42 near the entrance of the building, and there is no abnormal object in the screen. Thereafter, the video signal of the reference image sequentially output from the camera 1 is digitally converted by the A / D converter 2 and input to the reference image memory 4 to record one screen.
[0010]
Then, the image data is read from the background image memory 3 and the reference image memory 4 and input to the correlation calculation unit 5. For example, the correlation calculation unit 5 divides the two screens into multiple sections of the same size, calculates the correlation (similarity) of the two images for each section, and inputs this data to the abnormality determination unit 6. The abnormality determination unit 6 compares the data from the correlation calculation unit 5 with a required threshold value, and if it is larger than the threshold value, outputs a signal that an abnormality has occurred. Based on this signal, the control described below is performed via the control unit 7.
[0011]
Each time image data is recorded in the reference image memory 4, the image data is read from the reference image memory 4 and input to the edge extraction unit 8 and the subtracter 9. Then, the edge image generated by the level difference in the reference image is extracted by the edge extraction unit 8 by differentiation or the like, and the background image read out from the background image memory 3 from the input reference image (b) by the subtractor 9. Subtract (a). By this subtraction, when the abnormal object 43 is present in the reference image (b), difference image data is output, and this image data is input to the edge extraction unit 10 to extract the edge of the difference image. Then, the edge image from the edge extraction unit 8 and the edge image from the edge extraction unit 10 are input to the logical product circuit 11, and an edge image (C43) of the abnormal object is generated by calculation. According to this method, as described in Japanese Patent Application No. 7-172231 filed by the present applicant, there is a difference in brightness in the background (the left half of the abnormal object 43 in FIG. Is bright and the right half is in front of the door 41, so the background is dark.) The edge of the background brightness step (vertical line on the left side of the door 41, etc. = passes substantially through the center of the abnormal object 43) is the edge. It is possible to prevent the abnormal object from being divided at the edge of the step and being mistaken as if there were two objects.
[0012]
The data of the edge image 43 from the AND circuit 11 is input to the edge circumscribing coordinate calculation unit 12, and as shown in (d), the coordinates (x1,. x2, y1, y2) are calculated. Then, the image cutout unit 13 cuts out the image data of the rectangular region surrounded by the coordinates x1, x2, y1, and y2 from the next image data read from the reference image memory 4, and the edge circumscribed coordinates / image together with the coordinate data. Record in memory 14. Then, these data are read out, input to the tracking difference calculation unit 15, and compared with the reference image data read out from the reference image memory 4, and the coordinates of the matching positions of the images are calculated. That is, as shown in (e), the image data (d. 43) from the edge circumscribing coordinates / image memory 14 is converted into the required number of pixels in the x-axis direction and the y-axis direction on the image data from the reference image memory 4 ( Move (scan) one pixel at a time or several pixels), find the correlation with the reference image each time, find the coordinate (x11, x12, y11, y12) that minimizes the difference value, and calculate this amount of turning Input to part 16. Accumulated data of the turning amount of the camera 1 up to the previous time is recorded in the memory unit 18, and this data is read out through the turning control unit 17, and the data from the memory unit 18 and the tracking difference calculation are calculated by the turning amount calculation unit 16. Calculation is performed based on the data from the unit 15, and the turning amount (vector amount) for directing the center of the viewing angle of the camera 1 to the center of the rectangle surrounded by the coordinates (x11, x12, y11, y12) is calculated. Then, the turning control unit 17 turns the camera 1 so that the abnormal object 43 is substantially at the center of the screen as shown in FIG.
[0013]
Since the turning control unit 17 turns the camera 1 in the direction of the vector amount from the turning amount calculating unit 16, no turning is wasted and it is possible to cope with fast movement of an abnormal object. In addition, when the abnormal object is moving in the same direction, it may be applied to turn the camera 1 ahead by the same vector amount so as to anticipate the movement of the abnormal object. Incidentally, abnormal object from the monitoring field is extinguished, i.e., the field image data that matches the rectangle surrounding the abnormal object in the reference image is no longer obtained if, initially set stored in advance in the memory unit 18 orientation Return to .
[0014]
【The invention's effect】
As described above, according to the video monitoring and tracking apparatus of the present invention, when an abnormal object is determined, an abnormal object is identified by extracting an edge. ) And based on this data, the camera turns to track abnormal objects, so there is no waste in turning the camera, and abnormal objects can be captured accurately and quickly on the monitoring screen. .
[Brief description of the drawings]
FIG. 1 is a block diagram of a main part of an embodiment of a video surveillance / tracking apparatus according to the present invention;
FIG. 2 is a partial block diagram of another embodiment of the video surveillance / tracking apparatus according to the present invention;
FIG. 3 is a partial block diagram of another embodiment of the video surveillance / tracking apparatus according to the present invention;
FIG. 4 is an image diagram for explaining the operation of the video monitoring and tracking apparatus according to the present invention.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Television camera 2 A / D conversion part 3 Background image memory 4 Reference image memory 5 Correlation calculation part 6 Abnormality determination part 7 Control part 8, 10 Edge extraction part 9 Subtractor
11 AND circuit (AND gate)
12 Edge circumscribed coordinate calculation unit
13 Image cutout
14 Edge circumscribed coordinates and image memory
15 Tracking difference calculator
16 Turning amount calculation unit
17 Turning control unit
18 Memory section
21, 22 Data processing section
31, 32 Binary processing section

Claims (8)

For monitoring an abnormality based on a difference image between a background image when there is no abnormal object from the television camera and a reference image sequentially output from the television camera, the background image from the television camera is stored. Based on the data from the correlation calculation unit, the reference image memory for storing the reference image, the correlation calculation unit for calculating the correlation between the data read from the background image memory and the data read from the reference image memory Based on the signal from the abnormality determination unit, the first edge extraction unit that extracts the edge of the reference image from the data read from the reference image memory based on the signal from the abnormality determination unit, and the signal from the abnormality determination unit A subtractor for calculating a difference between data read from the background image memory and data read from the reference image memory; A second edge extraction unit that extracts edges of the difference image based on the data of the difference, an AND circuit that calculates the data from the first edge extraction unit and the data from the second edge extraction unit, and an AND circuit An edge circumscribing coordinate calculation unit that calculates the coordinates of a rectangle circumscribing the edge based on the data, and an image cutout that cuts out an image in a range corresponding to the data from the edge circumscribing coordinate calculation unit from the image read from the reference image memory An edge circumscribing coordinate / image memory that stores the coordinates from the edge circumscribing coordinate calculation unit and the image from the image cutout unit, and the next image is read from the reference image memory, and the image from the edge circumscribing coordinate / image memory is read out. A tracking difference calculation unit that calculates the coordinates of an image that matches the data, a memory unit that stores the center of the viewing angle of the television camera, and data and tracking differences from the memory unit Based on data from the calculation unit, a turning amount calculation unit for calculating a turning amount for directing the center of the viewing angle of the television camera to the center of the rectangle circumscribing the edge, and a turning amount calculation unit attached to the television camera A video monitoring and tracking device comprising a turning control unit for turning a television camera based on the data of. 2. The video surveillance according to claim 1, wherein an A / D conversion unit is provided at a subsequent stage of the television camera, wherein a video signal from the television camera is converted into a digital signal and input to the background image memory or the reference image memory. Tracking device. A data processing unit is provided after each of the first edge extracting unit and the second edge extracting unit, and bit data of each pixel from each edge extracting unit is converted from the most significant bit of data “1” to the least significant bit. The video surveillance / tracking apparatus according to claim 1, wherein all of the video data are converted into “1” data and input to the AND circuit, respectively. A binarization processing unit is provided after each of the first edge extraction unit and the second edge extraction unit, and each pixel data from each edge extraction unit is converted into binary data by a preset threshold value, 3. The video surveillance / tracking apparatus according to claim 1, wherein the video surveillance / tracking apparatus inputs to the logical product circuit. 4. The edge circumscribing coordinate / image memory rewrites stored data to new data each time a coordinate of a rectangle circumscribing an edge is calculated by the edge circumscribing coordinate calculation unit. Or the video surveillance tracking apparatus of Claim 4. The tracking difference calculation unit sequentially moves the image data from the edge circumscribing coordinates / image memory and compares the image data with the image data read from the reference image memory, and calculates the coordinates of the matching positions of the images. 6. A video surveillance / tracking apparatus according to claim 2, claim 3, claim 4 or claim 5. The tracking difference calculation unit sequentially moves the image data from the edge circumscribing coordinates / image memory by the required number of pixels in the horizontal direction and the vertical direction, and compares the image data with the image data read from the reference image memory, thereby matching the images. 6. The video monitoring and tracking device according to claim 1, wherein the coordinates of the position to be calculated are calculated. The direction of the center of the viewing angle of the preset television camera is stored in the memory unit, and the image read from the reference image memory by the tracking difference calculation unit matches the image from the edge circumscribed coordinate / image memory. 3. The display device according to claim 1, wherein when the coordinates are not calculated, the viewing angle of the television camera is turned in a direction preset in the memory unit based on the data read from the memory unit by the turning control unit. The video surveillance tracking device according to claim 3, claim 4, claim 5, claim 6, or claim 7.

Images