JPH0998412A - Television camera monitoring system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately detect the distance to an intruding object. SOLUTION: Image information obtained by a television camera 101 for picking up the image of an object and background image information obtained beforehand by the television camera 101 are compared and the presence/absence of the intruding object are detected in an object extraction part 104. When it is detected that the intruding object is present, control is performed to the television camera 101 so as to make a visual field depth shallow and to focus in a lens control part 106. Every time focusing is performed, based on the information of a focus position obtained from the television camera 101, the distance to the intruding object is detected in an edge/distance detection part 107 and the obtained distance to the intruding object is displayed on a monitor TV 112.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a television camera monitoring system used in security systems and the like.

[0002]

2. Description of the Related Art Conventionally, in a television camera monitoring system, an intruding object is detected by comparing a current image captured by the television camera with a previously captured image when there is no intruder (background image). Was. When measuring the distance to an intruding object, the pixel position on the screen is used, so an error may occur depending on the height of the intruding object. Also, when monitoring from a distant to a short distance In particular, there is a problem that an error occurs in a distance to a far intruding object.

Further, when a plurality of objects overlap each other, it is difficult to identify them and there is a problem that two or more objects are detected as one or less than the original number. Therefore, even when trying to identify an intruding object by pattern matching, there has been a problem that the matching rate decreases when a plurality of objects overlap. Therefore, there is a system in which a surveillance TV camera is installed directly above and the direction is monitored directly below, but in this system, unless the TV camera is installed at a very high position, the field of view becomes narrow and There is a problem in that it is necessary to widen the field of view and the cost becomes high.

[0004]

In monitoring railroad crossings and tunnels, for example, it is not possible to determine whether an automobile or the like is merely stopped or stopped due to an accident, and an alarm cannot be output accurately. There was a problem.

The present invention has been made to solve the above-mentioned problems of the conventional TV camera monitoring system, and an object thereof is to provide a TV camera monitoring system capable of accurately detecting the distance to an intruding object. Is to provide. Another object of the present invention is to provide a television camera monitoring system capable of accurately detecting the number of intruders even when the intruders overlap. Furthermore, it is another object of the present invention to provide a television camera monitoring system capable of appropriately detecting an event that is occurring based on the image captured by the television camera and the collected sound.

[0006]

A television camera monitoring system according to claim 1 of the present application obtains an image of an object, a television camera, image information obtained by the television camera, and the television camera previously obtained. An object extraction unit that detects the presence or absence of an intruder by comparing it with the background image information, and when the presence of an intruder is detected by the object extraction unit, the depth of field is reduced with respect to the television camera. And a lens control unit that controls to perform focusing,
Every time the TV camera is focused by the control of the lens control unit, an edge / distance detection unit that detects the distance to the intruder based on the focus position information obtained from the TV camera and a display regarding the intruder are displayed. It is characterized by comprising a display unit for performing and a display control unit for displaying the distance to the intruding object obtained by the edge / distance detection unit on the display unit.

According to a second aspect of the present invention, there is provided a television camera monitoring system in which a television camera for picking up an object, an image memory for storing image information, image information obtained by the television camera, and the television camera are used in advance. An object extraction unit that detects the presence or absence of an intruder by comparing the obtained background image information and stores difference image information related to the difference between the two image information in the image memory, and an object extraction unit. When it is detected that there is an intruding object, a lens control unit that controls the television camera so that the depth of field is shallow and focus is performed, and the television is controlled by the lens control unit. Each time the camera is in focus, the image information obtained by the television camera is processed to obtain the image information of an individual intruder, which is stored in the image memory. Another object detection unit, an overlap identification unit that detects the number of intruders based on the image information stored in the image memory, a display unit that displays information about the intruders, and a display unit that displays the number of intruders obtained by the overlap identification unit. And a display control means for displaying on the display.

According to a third aspect of the present invention, there is provided a television camera monitoring system in which a television camera for picking up an image of an object, an image memory for storing image information of the object picked up by the television camera, and the image memory are stored in the image memory. Object recognition means for recognizing an object based on the accumulated image information, a microphone for collecting voice, and presence / absence of special sound generation based on the sound information obtained by this microphone and special sound information stored in advance. Special sound detecting means for detecting, and event determining means for determining an event that has occurred based on the detection result of the special sound detecting means and the recognition result by the object recognizing means,
And an output unit for outputting the determination result of the event determination unit.

The television camera monitoring system according to claim 1 of the present application compares the image information obtained by the television camera for picking up an object with the background image information obtained in advance by the television camera and compares the intruding object. Detecting the presence or absence, when it is detected that there is an intruder, the television camera is controlled so that the depth of field is shallow and the focus is adjusted, and each time the focus is achieved, The distance to the intruder is detected based on the focus position information obtained from the television camera, and the obtained distance to the intruder is displayed on the display unit.

The television camera monitoring system according to claim 2 of the present application compares the image information obtained by the television camera for picking up an object with the background image information obtained in advance by the television camera and compares the intruding object. Detecting the presence or absence, when it is detected that there is an intruder, the television camera is controlled so that the depth of field is shallow and the focus is adjusted, and each time the focus is achieved, Image information obtained by the television camera is processed to obtain image information of an individual intruder and stored in the image memory, and the number of intruders is detected based on the image information stored in the image memory. The number of intruders that have been removed will be displayed on the display unit.

In the television camera monitoring system according to claim 3 of the present application, the image information of the object imaged by the television camera is accumulated in the image memory, the object is recognized based on the image information, and at the same time, the microphone is used. The presence or absence of the occurrence of special sound is detected based on the voice information obtained in advance and the special sound information stored in advance, and the occurrence of the special sound is determined based on the detection result and the recognition result described above. Will be output.

[0012]

DETAILED DESCRIPTION OF THE INVENTION A television camera monitoring system according to an embodiment of the present invention will be described below with reference to the accompanying drawings. In the description of each drawing, the same components are denoted by the same reference numerals, and redundant description will be omitted. FIG. 1 shows a block diagram of a television camera monitoring system according to an embodiment of the present invention. As shown in the figure, the television camera 1
The target object is imaged by 01, and the obtained image signal is digitized by the A / D conversion unit 103. TV camera 10
The movable part such as the first lens system and the diaphragm is operated by the lens operating part 102, and the lens operating part 102 is operated by being controlled by the lens control part 106.

The image information digitized by the A / D converter 103 is stored in each frame or field.
It is given to the object extracting unit 104 and the edge / distance detecting unit 107. The object extracting unit 104 compares the image information obtained by the television camera 101 with the background image information obtained in advance by the television camera 101 to detect the presence or absence of an intruding object, and to detect the difference between the two image information. The difference image information (contour image information) is related to the first page (105-1) of the image memory 105, and the image information as it is obtained by the television camera 101 is referred to as the nth page (1) of the image memory 105.
05-n). Each page of the image memory 105 has a capacity for storing one frame or one field of image information. When the object extraction unit 104 detects that there is an intruder as a result of the comparison, the lens control unit 1
A control signal is sent to 06. When the object extracting unit 104 detects that there is an intruding object, the lens control unit 106 adjusts the lens operating unit 102 of the television camera 101 by reducing the depth of field and focusing. Take control. The edge / distance detecting unit 107 controls the lens operating unit 10 of the TV camera 101 every time the TV camera 101 is in focus under the control of the lens control unit 106.
The distance to the intruder is detected based on the focus position information obtained from 2. The detected distances overlap and the identification unit 110
Sent to

In the above, when there is a focus,
The output of the A / D converter 103 is the edge / distance detector 10
7 is supplied to the individual object detection unit 108, and the individual object detection unit 108 processes the image information obtained by the television camera 101 to obtain the image information of the individual intruder to obtain the image memory 105 (105- 2)). The individual object detection unit 108 notifies the comparison unit 109 of the accumulation of image information. The comparison unit 109 compares the difference image information of the first page 105-1 of the image memory 105 with the image information accumulated by the individual object detection unit 108 to detect whether there is a difference, and if there is a difference (that is, , If there are a plurality of intruding objects), the overlapping identifying section 110 is notified of this. The overlap identifying unit 110 gives an instruction to the edge / distance detecting unit 107 to focus at another position and obtain image information. In addition, the overlap identifying unit 110
An intruder is recognized by pattern recognition or the like using the image information of each page of the image memory 105 via the comparison unit 109. The image information of the nth page of the image memory 105 is given from the comparison unit 109 to the D / A conversion unit 111, converted into an analog signal and sent to the monitor TV (for example, CRT display device) 112, and the monitor image is displayed. Has been. On the other hand, the overlap identifying unit 110 provides the D / A converting unit 111 with image information created to superimpose the distance to the intruding object, the number of intruding objects, the recognition result, and the like on the image on the monitor TV 112. As a result, in the image on the monitor TV 112, the distance to the intruder, the number of intruders, the recognition result, etc. are superimposed and displayed. In this way, the comparison unit 109 and the overlap identification unit 1
10, the D / A converter 111 includes an edge / distance detector 10
Distance to the intruder obtained by 7 and overlap identification unit 11
It functions as display control means 120 for displaying the number of intruders obtained by 0 on the display unit (monitor TV).

The television camera monitoring device configured as described above operates as follows. The object extraction unit 104
The presence or absence of an intruder is detected by obtaining the difference between the image information obtained by the television camera 101 and the background image information held in advance. That is, the background image information showing the existing object 201 as shown in FIG. 3 and, for example, as shown in FIG.
When the image information displayed together with 1 is compared, it is determined that there is an intruder, the lens control unit 106 is notified of that fact, and there is a difference between the background image information and the information of the currently captured image. A certain difference image information is obtained, and the image memory 1
No. 05 is stored in the first page (105-1).

FIG. 4 shows the difference image information stored in the first page (105-1) of the image memory 105. In the imaging range 302 of the intruder shown in FIG. 4, the contour image 301 of the intruder is multivalued (for example, 64 gradations), and for example, 3 shown by 303-1 to 303-3.
Edge detection is performed by the edge detection sample line of the book.

In the above description, the lens control unit 106 that has received the notification that there is an intruder controls the lens operating unit 102 to control the aperture and shutter speed so that the depth of field becomes shallow. That is, the aperture is opened to the maximum and the shutter speed is increased to the maximum. Then, control is performed so that the imaging range 302 shown in FIG. 4 is gradually focused from the shortest distance. In the case of focus control, the lens controller 106 notifies the edge / distance detector 107. The edge / distance detection unit 107 includes an A / D conversion unit 103.
Image information is obtained from the edge detection sample line 303
At edge detection. That is, taking the edge detection sample line 303-1 as an example, as shown in FIG.
When the brightness change (brightness difference) of the multi-valued pixel is detected in the x direction on the edge detection sample line 303-1 with respect to the contour image information 300, the change shown in the brightness change information 400 is obtained. At this time, when the brightness change information 400 is sliced with the focus matching threshold value 401, focus positions 402 and 403 beyond this are detected. Focus position 404
Is less than or equal to the threshold value and is not the target when detecting whether or not focus is achieved. Focus position 402, 40
When 3 is detected, the edge / distance detecting unit 107 further gives an instruction to the lens control unit 106 to instruct the lens operating unit 102.
Let the camera perform the focusing operation. By this operation, the lens state in which the focus positions 402 and 403 are maximized is detected, and it is determined that the focus is achieved, and the lens control unit 106 is instructed to interrupt the control. At this time, the edge / distance detection unit 107 causes the lens operating unit 10 to operate via the lens control unit 106.
The information of the focus position, that is, the focus information P _I of the digital value in consideration of the aperture of the diaphragm, the shutter speed, the position of the lens, etc. is obtained from 2 and the distance to the intruding object is obtained correspondingly. That is, the edge / distance detector 107
As shown in FIG. 10, it has a comparison table of the distance x ₁ ~x _N corresponding to focus information P ₁ to P _N, seeking distance with this, sends the overlap identifying unit 110.

In the above-mentioned focused state, the output of the A / D converter 103 is given to the individual object detector 108 through the edge / distance detector 107, and the image information of each intruder is detected. That is, contour detection processing, hole filling processing (connection of discontinuous portions), and the like are performed on image information having a shallow depth of field to generate a binary image of the individually extracted intruder 501 as shown in FIG. Then, the image is stored in the second page and thereafter (from 105-2) of the image memory 105, and the comparison unit 109 is notified of the storage. The comparison unit 109 uses the image memory 105.
The difference between the image information of the first page (FIG. 4) and the image information of the second page (FIG. 6) is detected to detect whether there is a remaining image (whether there is another intruder). In the case of this example, the remaining image information shown in FIG. 8 is detected. The detection result is given to the overlap identifying unit 110, and the overlap identifying unit 110 detects the edge
The distance detection unit 107 is instructed to focus again.

The edge / distance detection unit 107 gives an instruction to the lens control unit 106 so that focusing is performed as described above at a distance from the present, and an intruder 701 shown in FIG.
10 to obtain the focus information, the distance to the intruder 701 is obtained from the contrast table of FIG.
Send to In this way, the distances 601 and 602 from the television camera 101 to the intruders 501 and 701 are obtained. FIG. 7 shows a state in which these relationships are viewed from directly above.

When the intruder 701 is in focus, the output of the A / D converter 103 is the edge / distance detector 1
It is given to the individual object detection unit 108 via 07, and the image information of each intruding object is detected. That is, contour detection processing, hole filling processing (connection of discontinuous portions), and the like are performed on image information with a shallow depth of field to generate a binary image as shown in FIG. 8 in which intruders are individually extracted. , Image memory 10
The data is accumulated on the third and subsequent pages (105-3 to 5) of No. 5, and the comparison unit 109 is notified of the accumulation. The intruding object 701 in FIG.
It overlaps with 01. The comparison unit 109 takes the difference between the image information of the first page (FIG. 4), the image information of the second page (FIG. 6) and the image information of the third page (FIG. 8) of the image memory 105, and there is a remaining image. Detect if there is another intruder. In the case of this example, the remaining image information is not detected.

As described above, when the signal corresponding to the absence of the remaining image is transmitted from the comparison unit 109, the overlap identification unit 110 is detected by the individual object detection unit 108 and accumulated in the image memory 105. Pattern recognition such as template matching is performed on the image information of each intruder to recognize the intruder. In this case, in the case of image information having the joint surface 703 as shown in FIG. 8, the collation rate is obtained on the portion other than the joint surface and the intruder is recognized.

The above recognition result and the distance to the intruder,
The number of intruders is replaced with the character image data for superimposing and is sent to the D / A conversion unit 111. Comparison section 1
09, the image information of the nth page of the image memory 105 is given to the D / A conversion unit 111, converted into an analog signal, and then the monitor TV (for example, CRT display device) 1
It is sent to 12, and the surveillance image is displayed. Therefore,
On the monitor TV 112, the recognition result, the distance to the intruder, and the number of intruders are superimposed in the superimpose area 210 as shown in FIG. 9, and the image initially captured by the television camera 101 is displayed. The state of intrusion and the degree of intrusion can be recognized at a glance. When detecting an intruder, an alarm (not shown) may be activated.

FIG. 11 shows a television camera monitoring system according to another embodiment. This system
A television camera 101 that captures an image of an object, an image memory 105 that stores image information of the object captured by the television camera 101, and an object that recognizes an object based on the image information stored in the image memory 105. A recognition unit 150, a microphone (hereinafter referred to as a microphone) 140 that collects a voice, and special sound detection that detects the presence or absence of a special sound based on the sound information obtained by the microphone 140 and the special sound information stored in advance. Means 160, the detection result of the special sound detecting means 160 and the object recognizing means 150
The event determining means 170 for determining an event that has occurred based on the recognition result of the above, and the output means 180 for outputting the determination result of this event determining means 170 are provided.

The object recognition means 150 includes an image memory 105.
A feature extraction unit 151 that performs processing such as binarization and difference image information for each image information of one frame or one field stored in, and the image information of the processing result obtained by the feature extraction unit 151 and itself. An object recognition unit 1 that recognizes an object by performing pattern recognition by using a template matching method or the like based on pattern data or the like included in
52. The recognition result of the object recognition unit 152 is
It is sent to the tracking unit 141 and the object is tracked.

The special sound detecting means 160 includes a microphone 140.
Of the A / D conversion unit 161, which digitizes the audio signal obtained by the above, an audio memory 162 which stores digital audio data for processing, a special sound pattern memory 164 in which a special sound pattern is stored in advance, and an audio memory 162. A special sound detector 163 is included to detect the occurrence of a special sound by comparing the contents or the output of the A / D converter 161 with the voice pattern of the special sound pattern memory 164. The detection result by the special sound detection unit 163 is sent to the event determination unit 171, the event that has occurred is determined together with the tracking result by the tracking unit 141, and the result is output to the output means 180. The output unit 180 includes a D / A conversion unit 111 and a monitor TV 112.

A case in which an accident in a tunnel is detected by the television camera monitoring system configured as described above will be described. The object recognition unit 152 stores, for example, interval information regarding the tail lamp of the vehicle in advance, and can recognize one automobile or motorcycle, and the special pattern memory 164 stores the brake sound of the vehicle. , Voice patterns such as collision sounds are stored.

FIG. 1 taken by the television camera 101.
Images such as 2 are digitized by the A / D conversion unit 103, accumulated in the image memory 105, and binarized by the feature extraction unit 151, and, for example, a tail lamp is detected. Reference numeral 192 indicates the position of the tail lamp that changes for each frame, and reference numeral 191 indicates the wall surface of the tunnel.
The image information of the tail lamp is sent to the object recognition unit 152, is recognized as a passenger car, and the recognition result is sent to the tracking unit 141, and it is detected that the passenger car has moved along the locus of the tail lamp position indicated by 192. It

On the other hand, in the special sound detecting means 160, as shown in FIG.
Brake noise is detected between time 193 and time 194 shown in (1), and a collision noise is detected at time 195. The event determination unit 171 uses the above times 193, 194, and 195,
From the tail lamp positions F1 to Fn corresponding to the time obtained from the tracking unit 141, along with the generation of the brake sound,
It is determined that the vehicle 190 has moved to the right and a collision sound has been generated at the position Fn, and it is determined that the collision occurred on the wall surface 191 and the accident occurred.
The output of 1 is given, and the event judging section 1
A message such as “collision accident occurrence”, which is the determination result by 71, is superimposed and displayed on the monitor TV 112.

Further, FIG. 13 shows image information when the railroad crossing is monitored according to the present embodiment. Track 19
An intruder 197 is standing inside the railroad crossing where 6 and the road 198 intersect. Based on the image obtained by the television camera 101, the object recognition unit 150 detects the presence of an intruder inside the railroad crossing. The object recognition unit 152 has pattern information and the like for recognizing whether the intruded content is a person or a car. The tracking unit 141 tracks the position of the intruder 197 (inside the railroad crossing).

On the other hand, in the special sound pattern memory 164,
A voice pattern of a level crossing sound as shown in FIG. 14 is stored. In the special sound detecting means 160, it is detected that the level crossing sound is ringing, and the event determination unit 171 causes the tracking unit 14 to perform.
1 indicates that there is an intruder 197 in the railroad crossing, and the railroad crossing sound is generated, so the output of the TV camera 101 is given and a message such as “There is a person inside the railroad crossing, danger” along with the image in FIG. Is superimposed and displayed on the monitor TV 112.

[0031]

As described above, according to the television camera monitoring system of claim 1 of the present application, the distance to the intruder is detected, and the obtained distance to the intruder is displayed on the display unit. Therefore, the distance can be accurately detected.

As described above, according to the television camera monitoring system of claim 2 of the present application, the number of intruders is detected based on the image information, and the obtained number of intruders is displayed on the display unit. It is possible to properly detect whether there is an intruder.

As described above, according to the television camera monitoring system of claim 3 of the present application, it is possible to detect the occurrence of the special sound based on the sound information obtained by the microphone and the special sound information stored in advance. Then, the event that has occurred is determined based on the detection result and the recognition result described above, and the determination result is output. Therefore, the event can be appropriately determined and an error-free determination can be performed.

[Brief description of drawings]

FIG. 1 is a block diagram of a television camera monitoring system according to an embodiment of the present invention.

FIG. 2 is a diagram showing an example of an image captured by the television camera monitoring system according to the embodiment of the invention.

FIG. 3 is a diagram showing an example of a background image captured in advance by the television camera monitoring system according to the embodiment of the present invention.

FIG. 4 is a diagram showing an example of a difference image captured by the television camera monitoring system according to the embodiment of the present invention.

FIG. 5 is a diagram for explaining an edge detection operation by the television camera monitoring system according to the embodiment of the present invention.

FIG. 6 is a diagram showing an example of an individual object image captured by the television camera monitoring system according to the embodiment of the invention.

FIG. 7 is a diagram illustrating an example of a distance detected by the television camera monitoring system according to the embodiment of the present invention.

FIG. 8 is a diagram showing an example of an individual object image captured by the television camera monitoring system according to the embodiment of the invention.

FIG. 9 is a diagram showing an example of an image obtained by superimposing a monitoring result on an image captured by the television camera monitoring system according to the embodiment of the present invention.

FIG. 10 is a diagram showing a reference table used in the television camera monitoring system according to the embodiment of the present invention.

FIG. 11 is a block diagram of a television camera monitoring system according to another embodiment of the present invention.

FIG. 12 is a diagram showing images and audio when a tunnel camera is monitored by a television camera monitoring system according to another embodiment of the present invention.

FIG. 13 is a diagram showing an image when a railroad crossing is monitored by a television camera monitoring system according to another embodiment of the present invention.

FIG. 14 is a diagram showing a level crossing sound pattern when a level crossing is monitored by a television camera monitoring system according to another embodiment of the present invention.

[Explanation of symbols]

101 TV camera 102 Lens operating unit 103 A / D conversion unit 104 Object extraction unit 105 Image memory 106 Lens control unit 107 Edge / distance detection unit 108 Individual object detection unit 109 Comparison unit 110 Overlap identification unit 111 D / A conversion unit 112 Monitor TV 120 display control means

Claims (3)

[Claims] 1. A television camera for picking up an image of an object, and an object extraction unit for detecting the presence or absence of an intruder by comparing image information obtained by the television camera with background image information obtained in advance by the television camera. And a lens control unit that controls the television camera so as to reduce the depth of field and focus when the presence of an intruding object is detected by the object extraction unit, Every time the TV camera is focused by the control of the control unit, an edge / distance detection unit that detects the distance to the intruder based on the focus position information obtained from the TV camera, and a display that displays information about the intruder And a display control unit for displaying the distance to the intruding object obtained by the edge / distance detection unit on the display unit. La monitoring system. 2. A television camera for picking up an image of an object, an image memory in which image information is stored, image information obtained by the television camera and background image information obtained in advance by the television camera are compared. An object extraction unit that detects the presence or absence of an intruder and stores difference image information relating to the difference between the two image information items in the image memory; and if the object extraction unit detects the presence of an intruder, A lens control unit that controls the television camera so that the depth of field is reduced to perform focusing, and by the control of the lens control unit, each time the television camera focuses, the television camera causes the television camera to focus. An individual object detection unit that processes the obtained image information to obtain image information of an individual intruder and stores the image information in the image memory, and an image stored in the image memory. An overlap identification unit that detects the number of intruders based on information, a display unit that displays information about the intruders, and a display control unit that displays the number of intruders obtained by the overlap identification unit on the display unit. TV camera surveillance system. 3. A television camera for picking up an object, an image memory for storing image information of the object picked up by the television camera, and an object recognition based on the image information stored in the image memory. An object recognition means, a microphone for collecting voice, a special sound detection means for detecting the presence or absence of a special sound based on the sound information obtained by this microphone and the special sound information stored in advance, and the special sound detection And an output unit for outputting the determination result of the event determination unit based on the detection result of the unit and the recognition result of the object recognition unit. TV camera monitoring system.