JPH06225197A - Monitor equipment by video camera - Google Patents

Abstract

PURPOSE:To save man-hours by realizing automatic recognition of an object existent in a monitor area and a resulting notice by voice so as to eliminate the need for monitoring a monitor screen continuously. CONSTITUTION:A picture of the entire monitor area is obtained by a stationary camera 11 with a wide visual field and the picture is inputted to a 1st picture processing section 12. The 1st picture processing section 12 performs processing of the picture to output a position and a size of an object to be noticed within a monitor area. The output of the 1st picture processing section is fed to a moving camera 13 whose visual field is variable. A closeup picture of an object is obtained based on information of the position and the size in the output of the processing section 12 in the camera 13. The picture is sent to a 2nd picture processing section 14, in which the object is recognized. The processing result of the 2nd picture processing section 14 is informed to a monitor in voice by a voice synthesis section 15 and a speaker 16.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a surveillance device, and more particularly to a technique effectively applied to a surveillance device using a video camera.

[0002]

2. Description of the Related Art Conventionally, in monitoring by a video camera, a person looks at an image from the video camera to monitor the condition of the area, and the monitoring method by the video camera is as follows. There was a formula. (1) A method of overlooking the entire surveillance area at once with a wide-angle fixed camera. (2) A method of overlooking the entire surveillance area over a certain period of time by periodically moving the line of sight of a camera with a standard viewing angle. (3) A method of monitoring a target area by remotely operating a camera whose view angle is variable and whose line of sight can be moved.

[0003]

However, in any of these methods, it is the basis of surveillance that a person looks at an image of a video camera. Therefore, unless another sensor system is used in combination to detect a change or abnormality, it is necessary to continue to see an image that does not change for most of the time, which is a waste of labor.

Further, even if the image is viewed only when a change occurs due to the combined use of other sensor systems, the conventional methods of (1), (2) and (3) have the following problems. there were. In the method (1), since a wide-angle camera is used, the image of each object is small, and even when viewed by a person, the situation may not be judged. In the method (2), a certain period of time usually elapses before the line of sight of the camera is directed at the changed object. In addition, the line of sight of the camera is immediately removed from the target. In the method (3), as in the method (2), a certain amount of time elapses until the line of sight of the camera is adjusted to the changed object. It also requires time to adjust the camera to a viewing angle suitable for the size of the target.

An object of the present invention is to eliminate the waste of labor for monitoring the monitor screen without interruption and to automatically track a notable target in the monitored area, thereby reducing the time for searching for an abnormal object in an emergency. It is to provide a technology that can be disused.

The above and other objects and novel features of the present invention will become apparent from the description of this specification and the accompanying drawings.

[0007]

In order to achieve the above-mentioned object, the present invention is a surveillance device using a video camera, a means for obtaining a wide-angle image of the entire surveillance area, and processing the wide-angle image to focus attention. Means for determining the target and its position and size, means for moving the line of sight of the camera to the target of interest to obtain a close-up image of the target, and processing the close-up image A means of recognizing
The most main feature is to have a means for notifying the recognition result by voice.

[0008]

According to the above-mentioned means, a wide-angle image of the entire surveillance area is always obtained by using a fixed camera having a wide viewing angle.
The wide-angle image is processed to extract a target of interest, and its position and size are obtained. Using a camera with a movable line of sight and a variable viewing angle, the line of sight is adjusted to the target and the viewing angle is adjusted to obtain a close-up image of the target. The close-up image is processed to recognize what the object is. The result is notified to the observer by voice.

With the above operation, it is possible to automatically recognize an object in the surveillance area, notify the observer by voice, and automatically track the object of interest.

The structure of the present invention will be described below together with embodiments.

In all the drawings for explaining the embodiments, parts having the same functions are designated by the same reference numerals, and the repeated description thereof will be omitted.

[0012]

FIG. 1 is a block diagram of an embodiment of the present invention, FIG. 2 is a diagram showing an example of an output image of a video synthesizing section of FIG. 1, and FIG. 3 is a diagram of a first image processing section of FIG. 4A to 4F are diagrams showing the flow of processing, and FIG. 5A to FIG. 4F are diagrams showing the contour line extraction result of the input image for every 6 consecutive frames in time.
FIG. 5F is a diagram showing, for each frame, a logarithmic polar coordinate conversion result corresponding to each frame of FIGS. 4A to 4F, and FIGS. 6A to 6F.
8A and 8B are diagrams showing, for each frame, an inspection image corresponding to each frame in FIGS. 4A to 4F. FIG. 7 is a diagram showing pixel frequencies in the vertical and horizontal directions for the inspection image of the first frame in FIG. 6A. FIG. 3 is a diagram showing a flow of processing in a second image processing section of FIG. 1.

As shown in FIG. 1, the surveillance apparatus using the video camera of this embodiment first obtains an image of the entire surveillance area by the fixed camera 11 having a wide viewing angle. As the fixed camera 11 having a wide viewing angle, a normal video camera equipped with a wide angle lens can be used. The image is input to the first image processing unit 12. The first image processing unit 12 processes the image and outputs the position and size of the object to be noted of the object in the monitoring area. The processing of the first image processing unit 12 will be described later. It should be noted that the notable targets are those that are unknown in what they are, such as those that first appeared in the surveillance area, and those that are moving.

The output of the first image processing section 12 is sent to a movable camera 13 with a variable viewing angle. Movable camera 13 with variable viewing angle
Uses a video camera equipped with an electric zoom lens that is mounted on a pan head that can be electrically controlled to rotate in both vertical and horizontal directions. The movable camera 13 with a variable viewing angle obtains a close-up image of the object by controlling the platform and the zoom lens based on the position and size information in the output of the first image processing unit 12. The image is sent to the second image processing unit 14, where the object is recognized. The processing of the second image processing unit 14 will be described later.

The processing result of the second image processing unit 14 is audibly notified to the monitor by the voice synthesizing unit 15 and the speaker 16. Further, the image from the fixed wide viewing angle camera 11, the image from the variable viewing angle movable camera 13, and the second image processing unit 1
The object recognition result, which is the output of No. 4, is input to the video synthesizing unit 17, and the monitoring image is synthesized there. The image is displayed on the video monitor 18 for monitoring by a monitor. The video composition unit 17 and the video monitor 18 are provided as necessary and are not essential to the present invention.

An example of the monitoring image is shown in FIG. On the screen 21 of the video monitor 18, the images 22 to 26 of the object in the surveillance area and the line-of-sight position 27 of the close-up image are displayed. The line-of-sight position 27 is located on the image 22 of the target object to be noticed, and the close-up image 28 of the image 22 of the target object is displayed on a part 29 of the screen. The close-up image 28 is not displayed when there is no target object to be noticed. In that case, the line-of-sight position 27 is located at the center of the screen.

Next, the flow of processing in the first image processing section 12 will be described with reference to FIGS. 3 to 7.

First, according to the flow chart of FIG.
The flow of processing in the first image processing unit 12 will be described.

The following processing is repeated for each frame of the input image.

In step 301, the wide viewing angle fixed camera 1 is used.
Input the wide-angle image from 1.

In step 302, the contour line of the object is extracted from the input wide-angle image. Although many methods have been proposed, for example, “Setsuyuki Hongo, Mitsuo Kawahito, Toshiro Inui, Makoto Miyake: Contour line extraction by local parallel stochastic algorithm with energy learning function, theory D -II, V
ol.J74-D-II, No.3, pp.348-356 (1991) ”can be used. As a result of the contour line extraction, the contour line images 4a to 4f shown in FIGS. 4A to 4F are obtained. In the contour line image,
○ Figure 41, △ Figure 42, □ Figure 43 and line-of-sight position 27
Is displayed.

In step 303, a labeling process is performed in which each line figure representing the contour of the object is numbered. this is,
“Junichi Hasegawa, Yamato Koshimizu, Akira Nakayama, Shigeki Yokoi: Basic Techniques for Image Processing <Introduction to Techniques>, Technical Review, pp.45-49 (19
86) ”can be used. As a result of this processing, FIG.
1 for ○ figure 41 of A, 2 for △ figure 42, □
The number 3 is given to the graphic 43.

In step 304, logarithmic polar coordinate conversion is performed. The logarithmic polar coordinate transformation is the distance from the center of the image to r,
When the rotation angle from the horizontal axis is θ, the horizontal axis is log
(R) is to transform the vertical axis into a space of θ. Note that this conversion is not essential in the present invention. However, “Yamasazawa Isamu: Image Sampling Method, Japanese Patent Application No. 3-97032
(1991) ”, by applying this conversion,
The calculation amount of the subsequent image processing can be greatly reduced. In the monitoring system, real-time property is important, and it is necessary to reduce the calculation amount as much as possible. Therefore, in this embodiment, this conversion is adopted. Converted images 5a to 5f obtained by the conversion result are shown in FIGS. 5A to 5F. The converted images include the converted figures 51, 52, 53.
Is displayed.

In step 305, the difference image is obtained by performing the operation of (converted image)-(pre-converted image) and setting the negative pixel value to zero. Here, the pre-conversion image is
It is a converted image one frame before. The pre-transformed image in the first frame is 0 (0 means that all pixel values are 0.
It is said that).

At step 306, it is determined whether or not the difference is zero. The difference of 0 means that the image has not changed unless the object disappears from the screen. If the difference is not 0, the process proceeds to step 307, and if the difference is 0, the process proceeds to step 308.

In step 307, the difference image is used as the inspection image. The inspection images 6a to 6f obtained here are shown in FIGS.
It is shown in FIG. 6F. The inspection image includes figures 61, 62, 63.
Is displayed.

In step 308, the accumulated image is used as the inspection image. Here, the accumulated image is from the first converted image,
It is an image in which the object that is the target of attention is sequentially erased in each frame. The accumulated image in the first frame is the converted image 5a.

In step 309, the pixel frequency in the vertical and horizontal directions of the inspection image obtained in step 307 or step 308 is counted. The result for the first frame is shown in FIG.

In step 310, it is determined whether the pixel frequency is 0 in both the vertical and horizontal directions. If the pixel frequency is not 0, the process proceeds to step 311, and if the pixel frequency is 0, the process proceeds to step 314.

In step 311, the maximum vertical and horizontal frequency positions are extracted.

In step 312, the center in the input image of the object contour line having the label of the pixel existing at the maximum frequency position in the vertical and horizontal directions or the position closest to that position is extracted as the line-of-sight position.

At step 313, the size of the target object is measured.

In step 314, the line-of-sight position 27 is set to the center of the screen and the size of the target is set to zero.

At step 315, the line-of-sight position and the size of the target object are output. With this output, the movable camera 13 with a variable viewing angle controls the platform and the zoom lens to obtain a close-up image of the object.

In step 316, (accumulated image)-(target object contour line) is accumulated as a new accumulated image.

The above processing is repeated for each frame of the input image.

Next, how the above-described processing of FIG. 3 is performed for each frame of the input image will be specifically described.

FIGS. 4A to 4F show the contour line extraction result of the input image in step 302 for every 6 frames which are temporally continuous. Similarly, FIGS. 5A to 5F show the logarithmic polar coordinate conversion result of step 304 for each frame, and FIGS. 6A to 6F show the inspection image of step 307 or 308 for each frame. The size of the wide-angle image is 64 × 64 pixels.

The following describes how the frames in the contour extraction results shown in FIGS. 4A to 4F change.

The first frame shown in FIG. 4A is an initial frame, and its image 4a has an object contour line.
1, a Δ graphic 42, and a □ graphic 43 are displayed. The image 4b of the second frame is exactly the same as the image 4a of the first frame. In the image 4c of the third frame, the position of the □ figure 43 has changed. The image 4d of the fourth frame is the third
It is exactly the same as the frame image 4c. The image 4e of the fifth frame is exactly the same as the image 4d of the fourth frame. In the image 4f of the sixth frame, the position of the □ figure 43 has changed.

First, the processing in the first frame will be described. Since the first frame is the first frame to appear, the converted image 5a is used as the accumulated image and the previous converted image is set to 0. Therefore, the difference image in step 305 becomes the converted image 5a itself. As a result, the difference is 0
Therefore, the process proceeds from step 306 to step 307, and the difference image is set as the inspection image 6a. In the inspection image 6a, figures 61, 62, 63 are displayed as targets to be noticed. As shown in FIG. 7, the vertical and horizontal pixel frequencies of the inspection image 6a are maximum at vertical = 2 and horizontal = 44.
The pixel label at this position is 3, and the center of the object outline □ figure 43 having that label on the input image 4a is at the position of vertical = 27, horizontal = 52. Also, the size of the object outline □ figure 43 (length of one side of the circumscribed square)
Is twelve. By outputting this position and size, the □ graphic 43 becomes a close-up image as an object to be noticed. Then, from the accumulated image, the label 3
The pixels of are erased to make a new accumulated image.

Next, in the second frame, the converted image 5b
Is exactly the same as the pre-converted image 5a, so step 3
The difference image of 06 becomes 0. As a result, the accumulated image is used as the inspection image 6b. In the inspection image 6b, the graphic 61 corresponding to the ◯ graphic 41 and the Δ graphic 42, which are notable objects to be noticed but are notable targets,
62 is displayed. The vertical and horizontal pixel frequencies of the inspection image 6b are maximum at vertical = 38 and horizontal = 54. The label of the pixel at this position is 1, and the center of the object contour line ◯ figure 41 having the label on the input image is vertical =
The position is 14 and the width = 17. The size of the object contour line is 18. By outputting this position and size, the O graphic 41 becomes a close-up image as an object to be noticed. Then, the pixel of the label 1 is erased from the accumulated image to obtain a new accumulated image.

Next, in the third frame, since the position of the □ figure 43 has changed in the image 4c, the converted image 5c is different from the previous converted image 5b, and the difference image is not zero. As a result, the difference image is used as the inspection image 6c. In the inspection image 6c, the figure 63 corresponding to the □ figure whose position has changed
Is displayed. The vertical and horizontal pixel frequencies of the inspection image 6c are maximum at vertical = 64 and horizontal = 54. The label of the pixel at this position is 3, and the center of the object outline □ figure 43 having that label on the input image is at the position of vertical = 25, horizontal = 50. The size of the object contour line is 12. By outputting this position and size, the □ graphic 43 becomes a close-up image as an object to be noticed. Then, the pixel of the label 3 is erased from the accumulated image to obtain a new accumulated image. Since the pixel of label 3 has been erased in the processing of the first frame, the accumulated image is not effectively updated.

Next, in the fourth frame, the converted image 4d
Is exactly the same as the pre-conversion image 4c, the difference image is 0. As a result, the accumulated image is used as the inspection image 6d. In the inspection image 6d, a graphic 62 corresponding to a Δ graphic 42 which is an unknown object but has not yet been targeted for close-up is displayed. The vertical and horizontal pixel frequencies of the inspection image 6d are maximum when vertical = 26 and horizontal = 52. The pixel label at this position is 2, and the center of the object contour line Δ graphic 42 having that label on the input image is at the position of vertical = 46, horizontal = 15. The size of the object contour line is 13. By outputting this position and size, the Δ graphic 42 becomes a close-up image as an object to be noticed. Then, the pixel of the label 2 is erased from the accumulated image to obtain a new accumulated image. At this point, the accumulated image becomes 0.

Next, in the fifth frame, the converted image 5e
Is exactly the same as the pre-conversion image 5d, and the difference image becomes 0. In that case, the accumulated image is used as the inspection image 6e. The vertical and horizontal pixel frequencies of the inspection image 6e are both 0. Therefore, the position of the line of sight is the center of the screen (vertical =
32, width = 32), the size of the object is 0, and these are output. This output means that there is no target object for the close-up image. And the accumulated image is not effectively updated.

Next, in the sixth frame, since the position of the □ figure 43 has changed in the image 4f, the converted image 5f is different from the previous converted image 5e, and the difference image is not zero. In that case, a difference image is used as the inspection image 6f. In the inspection image 6f, the graphic 63 corresponding to the □ graphic 43 whose position has changed is displayed. The vertical and horizontal pixel frequencies of the inspection image are maximum when vertical = 2 and horizontal = 44. The label of the pixel at this position is 3, and the center of the object outline □ figure 43 having that label on the input image is at the position of vertical = 27, horizontal = 52. The size of the object contour line is 12. By outputting this position and size, the □ graphic 43 becomes a close-up image as an object to be noticed. The accumulated image is not effectively updated.

Thereafter, processing is similarly performed for each frame,
If there is an unknown object or moving object that first appears in the surveillance area, an output for obtaining a close-up image is output.

Next, the flow of processing in the second image processing unit 14 of FIG. 1 will be described with reference to FIG. The second image processing unit 14 operates when the size of the target is not 0 for each frame output from the movable camera 13 having a variable viewing angle. Most of the processing here is the same as that used in “Yamasazawa: Drawing Interpretation Processing Method, Japanese Patent Application No. 3-21623 (1991)”.

In step 801, the close-up image 28 is input from the movable camera 13 with a variable viewing angle in which the direction of the line of sight and the viewing angle are controlled based on the output of the first image processing unit 12.

In step 802, the contour line of the object is extracted from the input close-up image. As the method, the same method as in step 302 of FIG. 3 may be used.

In step 803, logarithmic polar coordinate conversion is performed to convert changes in apparent size and rotation of the target object into parallel movement. This method is also the same as in step 304 of FIG.

In step 804, R conversion is performed to obtain an image that does not change due to parallel movement. For R conversion, refer to "Reitbo
eck HJ and Brody, TP: A transformation with inva
riance under cyclic permutation for applications i
n pattern recognition, Information and Control, Vol
.15, pp.130-154 (1969) h ”can be used.

In step 805, the R conversion result is collated with the dictionary image.

At step 806, the difference (distance of vector)
The name of the dictionary image with the smallest number is identified as the name of the target object. This achieves recognition.

Finally, in step 807, the recognition result is output. As described above, the result of recognizing the object is notified to the monitor by voice.

Although the present invention has been specifically described based on the above embodiments, the present invention is not limited to the above embodiments, and various modifications can be made without departing from the scope of the invention. Of course.

[0057]

As described above, according to the present invention,
It is possible to realize automatic recognition of an object in the surveillance area and voice notification of the result. As a result, it is not necessary to monitor the monitor screen without interruption, and labor can be saved.

Further, it becomes possible to automatically track an object to be noticed, that is, an object in the surveillance area, which is unknown, and a moving object, and a close-up image thereof. Since it is always obtained, the time to search for an abnormal object in an emergency becomes unnecessary.

INDUSTRIAL APPLICABILITY The present invention is effective for labor saving in industries where monitoring is required, such as security for buildings and parks, exploration of the deep sea and radioactive contamination areas.

[Brief description of drawings]

FIG. 1 is a block configuration diagram in an embodiment of the present invention.

FIG. 2 is a diagram showing an output image of the video composition unit of FIG.

FIG. 3 is a flowchart showing a processing flow in a first image processing unit in FIG.

FIG. 4A is a diagram showing a first frame of a contour line extraction result of the input image of FIG. 3;

FIG. 4B is a diagram showing a second frame of the contour line extraction result of the input image of FIG. 3;

FIG. 4C is a diagram showing a third frame of the contour line extraction result of the input image of FIG. 3;

FIG. 4D is a diagram showing a fourth frame of the contour line extraction result of the input image of FIG. 3;

FIG. 4E is a diagram showing a fifth frame of the contour line extraction result of the input image of FIG. 3;

FIG. 4F is a diagram showing a sixth frame of the contour line extraction result of the input image of FIG. 3;

FIG. 5A is a diagram showing a first frame of the logarithmic polar coordinate conversion result of FIG. 3;

FIG. 5B is a diagram showing a second frame of the logarithmic polar coordinate conversion result of FIG. 3;

FIG. 5C is a diagram showing a third frame of the logarithmic polar coordinate conversion result of FIG. 3;

FIG. 5D is a diagram showing a fourth frame of the logarithmic polar coordinate conversion result of FIG. 3;

FIG. 5E is a diagram showing a fifth frame of the logarithmic polar coordinate conversion result of FIG. 3;

FIG. 5F is a diagram showing a sixth frame of the logarithmic polar coordinate conversion result of FIG. 3;

6A is a diagram showing a first frame of the inspection image of FIG. 3. FIG.

6B is a diagram showing a second frame of the inspection image shown in FIG. 3;

6C is a diagram showing a third frame of the inspection image of FIG. 3. FIG.

6D is a diagram showing a fourth frame of the inspection image of FIG. 3. FIG.

FIG. 6E is a diagram showing a fifth frame of the inspection image shown in FIG. 3;

6F is a diagram showing a sixth frame of the inspection image of FIG. 3. FIG.

FIG. 7 is a diagram showing pixel frequencies in the vertical and horizontal directions of FIG. 6A.

FIG. 8 is a diagram showing the flow of processing in the second image processing unit in FIG.

[Explanation of symbols]

11 ... Wide view angle fixed camera, 12 ... First image processing unit,
13 ... Movable camera with variable viewing angle, 14 ... Second image processing section, 15 ... Audio synthesis section, 16 ... Speaker, 17 ... Video synthesis section, 18 ... Video monitor, 21 ... Video monitor screen, 27 ... Close-up image Eye position, 28 ... close-up image.

Claims (1)

[Claims] 1. A means for obtaining a wide-angle image of the entire surveillance area,
A means for processing the wide-angle image to determine a target to be noticed, a means for obtaining its position and size, a means for moving the line of sight of the camera to the target to obtain a close-up image of the target, A surveillance device using a video camera, comprising: a means for recognizing a target by processing a close-up image and a means for notifying a recognition result by voice.