JP2000078564A - System for tracing person on television camera monitor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To display an image of a person photographed by a television camera with high accuracy in the middle of a screen of a television monitor. SOLUTION: A television camera of a doorphone slave set photographs a person by using at least three consecutive images, that is, an image Sn of a current frame, an image Sn-1 of a preceding frame, and an image Sn-2 anterior to the preceding frame, a person tracing processing unit 22 of a doorphone master set applies difference processing 34 and binary processing 35 to the image of the current frame and the image of the preceding frame, and the image of the preceding frame and the image anterior to the preceding frame respectively to obtain two difference images S1, S2, applies expansion processing 36, contraction processing 37 and AND processing 39 to the two difference images and then applies labeling processing 40 to them, applies person discrimination processing 42 to the images at a maximum label region after the labeling processing and applies segmentation processing 44 to the image around the person based on the center of gravity of the maximum label region so as to display an image within a prescribed range around the person in the middle of a screen of a television monitor with high accuracy.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a person tracking system for a television camera monitor, and more particularly, to an image of a person taken by a television camera of a television doorphone, in a certain range (person) centered on the center of the screen of the television monitor. The present invention relates to a television camera monitor person tracking method for projecting images with high precision as (nearby) images.

[0002]

2. Description of the Related Art Conventionally, an image of a person captured by a television camera is projected on the center of the screen of a display device such as a television monitor, thereby achieving
Tokuho 7 that automatically and precisely adjusts the visual field of a person's image
An automatic visual field adjustment television doorphone of the automatic visual field adjustment method and apparatus described in JP-A-71288 has been proposed. According to the technique of the person tracking algorithm disclosed in this publication, the difference information in the image is searched in order from the upper line, and it is determined that the face is located at the position of the difference pixel equal to or larger than the first found threshold value. It was something to recognize.

[0003] However, in the automatic person tracking system based on the conventional person tracking algorithm, when a moving object other than a person exists at the top of the screen, it does not operate at all, and there are two or more people in front of the door phone camera. In this case, since the image is cut out only for a tall person, it is considered that the person near the doorphone is more important in the doorphone, and it is not preferable to apply the method to the doorphone. There were difficulties.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned difficulties, and automatically adjusts the field of view of a person's image picked up by a TV camera under various environments where video equipment such as a TV door phone is installed. Then, an object of the present invention is to provide a television camera monitor person tracking system capable of projecting an image in a certain range (near a person) centering on a person at the center of the screen of the television monitor with high accuracy.

[0005]

In order to achieve the above object, a television camera monitor person tracking method according to the present invention is applied to displaying an image of a person captured by a television camera on a television monitor. The current frame image, the previous frame image, the previous frame image, and the previous frame image are included in the current frame image, the previous frame image, and the previous frame image that are at least three continuous images among the images. , The absolute value of the difference is taken, two binarized difference images are obtained, the logical product of the two difference images is taken, and the dilation process and the erosion process are performed. Perform labeling processing to divide, determine whether or not a person is in the maximum label area after labeling processing, cut out an image near the person from the center of gravity of the maximum label area, and perform telephoto. It is those that reflect on the monitor.

According to such a television camera monitor person tracking system, at least three continuous images of a current frame image, a previous frame image, Each frame image is captured, and a difference process and a binarization process are performed on the current frame image and the previous frame image, and the previous frame image and the two-before frame image by the person tracking processing device of the intercom master unit. Two difference images are obtained respectively, and after performing expansion, contraction processing, and logical product processing of the two difference images, labeling processing is performed, and person determination processing is performed on the largest label area after labeling processing, and the center of gravity of the maximum label area is performed. The image of a certain area (near a person) centered on the person is displayed with high precision in the center of the screen of the TV monitor by extracting the image near the person from It is possible.

[0007]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A preferred embodiment to which a television camera monitor person tracking system according to the present invention is applied will be described below with reference to the drawings. FIG. 2 is a block diagram showing a configuration of a television doorphone to which an embodiment of a television camera monitor person tracking system according to the present invention is applied. And a door phone master unit connected to the door phone slave unit 1 via a transmission line L1 and installed in a living room of a dwelling unit and having a function of reproducing an image captured by the door phone slave unit 1 and a function of talking. And 2.

[0008] The door phone slave unit 1 captures an image in which a person in front of the television camera and a moving object other than the person are combined, and outputs the image as a color image signal in 640 × 480 frame units. A television camera 10, a slave unit microphone 11 and a slave unit speaker 12 used by a person using the intercom slave 1 to establish a call with a person using the intercom master 2, and a multiplexing circuit 13 are provided. The transmission line L1 is connected to the multiplexing circuit 13 via the slave unit line connection terminal T1.

The intercom master unit 2 sequentially outputs a multiplexing circuit 20 and a color image signal for each frame image picked up by the television camera 10 of the intercom slave unit 1 to a person tracking processing unit 22 and a person tracking process. A video signal control circuit 21 that outputs human coordinates of an image that has been subjected to signal processing (described later) by the device 22 as a video signal that has been appropriately processed (not described in detail), and a video output from the video signal control circuit 21 A television monitor 23 for displaying, at the center of the screen, an image of a person captured by the television camera 10 of the intercom slave unit 1 by a signal
And a handset 24 used by the person using the intercom base unit 2 to establish a call with the person using the intercom slave unit 1, and appropriately processing the audio signals transmitted and received by the handset 24 (detailed description). An audio signal control circuit 25 is provided, and the transmission line L1 is connected to the multiplexing circuit 20 via a parent device line connection terminal T2.

As shown in the block diagram of FIG. 1, the person tracking processing device 22 converts a plurality of color image signals of a plurality of frames of 640.times.480 imaged by the television camera 10 of the intercom slave unit 1 into frame units. Image memory 30 for storing
And a black-and-white conversion processing unit 31 for converting a color image signal for each frame unit into a monochrome black-and-white image signal, and reducing the resolution of a plurality of frames of black-and-white image signals via the black-and-white conversion processing unit 31 to 80 × 60 An image reduction processing unit 32 to perform
An image memory 33 for storing, on a frame-by-frame basis, a plurality of black-and-white image reduced signals of a plurality of frames whose images have been reduced via the image reduction processing unit 32;
3, and a binarization processing unit 35 that performs signal processing on a differential image signal obtained by binarizing the absolute value of the pixel difference into a binary image.
And a dilation processing unit 36 that performs dilation processing on the difference image signal passed through the binarization processing unit 35, a contraction processing unit 37 that similarly performs contraction processing, and a differential image signal that passes through the contraction processing unit 37. An image memory 38 for storing a binarized image,
A logical product processing unit 39 that performs a logical product process on the digitized image, a labeling processing unit 40 that divides the connected components of the binary image via the logical product processing unit 39 into regions, and a labeling processing unit 4
0, a feature extraction processing unit 41 for extracting a feature (region feature) of the region divided, a person determination processing unit 42 for determining whether or not the extracted region feature is a person, and a centroid feature of the region. And an image extraction processing unit 44 for extracting an image from an arbitrary position.

The signal processing operation of the person tracking processor shown in the block diagram of FIG. 1 applied to the TV door phone shown in the block diagram of FIG. 3 is a flowchart showing the processing operation, FIG. 4 is a schematic diagram of an image in the middle of processing, FIG. 5 is a processing operation explanatory diagram, FIG. 6 is an image reduction processing explanatory diagram, FIG. 7 is a difference processing explanatory diagram, and FIG. The description will be made with reference to the explanatory diagram and the logical product processing explanatory diagram of FIG.

A color image picked up by the television camera 10 of the intercom slave unit 1 shown in the block diagram of FIG. 2 has a resolution of 640 × 480 and contains RGB luminance information as shown in the schematic diagram of the image of FIG. The color image S101 of a person in front of the television camera 10 and the color image S1 of a moving object other than the person
02 is synthesized. The television camera 10 sequentially outputs this color image (color image signal) for each frame unit. The color image signals of the plurality of frames are supplied to the multiplexing circuit 1.
3. The information is sequentially transmitted to the person tracking processor 22 via the slave unit line connection terminal T1, the transmission path L1, the master unit line connection terminal T2 of the intercom master unit 2, the multiplexing circuit 20, and the video signal control circuit 21. Here, the signal processing of the color image signals of a plurality of frames in the person tracking processing device 22 of the intercom master unit 2 to be described later is performed by the color image signal Sn of the current frame, which is at least three continuous images, and the color image signal of the previous frame. The processing is performed for three frames of the signal Sn-1 and the color image signal Sn-2 of the frame two frames before the previous frame.

The color image signals Sn, Sn-1 and Sn-2 of the three frames input to the person tracking processor 22 are sequentially stored in an image memory 30 shown in the block diagram of FIG. 31 respectively. The black-and-white conversion processing unit 31 reduces the noise of the color image signals Sn, Sn-1, and Sn-2 between the three input frames, fills the area, and suppresses the consumption of the image memory 33 described later. The converted black-and-white image signals Sn ', Sn-1', and Sn-2 'for the three frames are sequentially output to the image reduction processing unit 32, and at the same time, the luminance value of each pixel ｛(R + 2
(G + B) / 4} (steps ST1 and ST2).

The image reduction processing section 32 converts the input black-and-white image signals Sn ', Sn-1' and Sn-2 'between the three frames into an image shown in FIG. Is reduced from 640 × 480 to １ ／ of 80 × 60 to reduce the screen, and the black-and-white image reduced signals Sn ″ and Sn−1 ″ between the three reduced frames shown in the processing operation explanatory diagram of FIG. , Sn-
2 '' is sequentially stored in the image memory 33 (step S
T3, ST4). The change in resolution at this time is performed by setting the average of 8 × 8 blocks as pixels after image reduction.

The difference processing unit 34 calculates the difference between the black and white image reduced signals Sn ″, Sn−1 ″ and Sn−2 ″ of three frames sequentially stored in the image memory 33 for each frame unit as shown in FIG. In order to detect a pixel that has moved due to a change in the position or shape of a target from two images temporally adjacent (between adjacent frames) in the image shown in the processing explanatory diagram, a black-and-white image reduced signal Sn of the current frame is detected. '' And the black-and-white image reduced signal Sn of the previous frame
-1 '' pixel difference, black and white image reduced signal S of the previous frame
The difference signal S11 of the current frame and the difference signal S12 of the previous frame, which are two difference signals respectively calculated from the pixel difference between the black-and-white image reduced signal Sn-2 '' of the previous frame and the difference signal S12 of the previous frame, are calculated as 2
The data is output to the value processing unit 35 (step ST5).

The binarization processing unit 35 has a series of images shown in FIG. 7 for explaining the difference processing in order to determine the shape of the moving object from the arrangement of the differences between the pixels of the two input difference signals S11 and S12. The absolute value of the positive or negative area in the pixel difference is taken and binarized and shown in FIG.
Differential image signals S of the current frame,
1. The difference image signal S2 of the previous frame is calculated and output to the dilation processing unit 36 (step ST6).

The dilation processing section 36 performs the dilation / shrinkage processing shown in FIG. 8 on the input two difference image signals S1 and S2 so as to further increase the boundary points of the connected components given to the difference image signals. While expanding several times with the image shown in the illustration,
The current frame which is two differential image expansion / contraction signals shown in the processing operation explanatory diagram of FIG. 5 in which the image is contracted several times to return to the original size via the contraction processing unit 37 and the noise is reduced and the difference area is filled in. Are sequentially stored in the image memory 38 as the differential image expansion / contraction signal S1 ′ of the previous frame and the differential image expansion / contraction signal S2 ′ of the previous frame (step ST7).

The logical product processing unit 39 applies the logical product processing shown in FIG. 9 to the differential image expansion / contraction signal S1 'of the current frame and the differential image expansion / contraction signal S2' of the previous frame read from the image memory 38. By calculating a logical product using the image shown, a labeling process of the logical product image signal S3 shown in the schematic diagram of the image of FIG. 4 in which the accurate moving object region of the image of the current frame is extracted and the process operation explanatory diagram of FIG. Output to the unit 40 (step ST8).

The labeling processing unit 40 assigns the same label (number) to all pixels belonging to the same connected component to the input logical product image signal S3, and assigns different labels to different connected portions. In order to extract the maximum area of the component, the area of the connected component of the logical product image signal S3 is divided, and the square of the area area and the perimeter of the area, which is the number of pixels of the area, is used.
A schematic diagram of the image in FIG. 4 including the complexity of the figure calculated from the area of the region and a region feature S shown in the processing operation explanatory diagram of FIG.
4 is extracted and output to the feature extraction processing unit 41 (step S
T9).

The feature extraction processing unit 41 extracts the complexity, area, image, etc., which are the features of the maximum label area of the input area feature S4, as shown in the processing operation explanatory diagram of FIG. Output to the person determination processing unit 42. still,
By cutting out the maximum label area, a small area that is not a person is removed, so that the image is resistant to noise. The person determination processing section 42 determines whether or not the input person candidate area S5 is a person area. That is, the area of the person candidate area S5 is, for example, 4 at a resolution of 80 × 60.
It is equal to or greater than the threshold value of 00 pixels, and the complexity is equal to or less than the threshold value of, for example, 35 (this value is 1 in a small area). When all of these four conditions of about 90 ° (for example, −45 ° to + 45 °) are satisfied, the person is determined as shown in the schematic diagram of the image in FIG. 4 and the processing operation explanatory diagram in FIG. The processing unit 42 determines the input person candidate area S5 as a person and outputs the person candidate area S5 to the center-of-gravity extraction processing unit 43 as a person confirmed area S6. When even one of the above four conditions is insufficient, the person determination processing unit 42 outputs the person uncertainty area S7 shown in the schematic diagram of the image in FIG. 4 to the centroid extraction processing unit 43 (step ST11). , ST12). The above-mentioned threshold value is determined by conducting an experiment in a door phone environment, and the threshold value can be appropriately adjusted.

The center-of-gravity extraction processing unit 43 extracts a center-of-gravity region obtained by calculating an average value from the X and Y coordinates of the maximum label region of the image from the input person defined region S6, and converts the center-of-gravity region into the current frame. Is output to the image cutout processing unit 44 as the barycenter coordinate S8 of the person's face position. If the center-of-gravity extraction processing unit 43 determines that the object is not a person, the center-of-gravity coordinates of the previous frame are returned (steps ST13 and ST1).
Four).

The image cut-out processing section 44 cuts out an image around a person, which is an image in a certain range centered on the center of gravity, from the input center-of-gravity coordinates S8 and outputs the cut-out image to the video signal control circuit 21 as person coordinates S9. The control circuit 21 outputs the video signal S10 to the television monitor 23 based on the input person coordinates S9.
Therefore, as shown in the schematic diagram of the image in FIG. 4, an image of a certain range (near a person) centered on a person is displayed in the center of the screen of the television monitor 23.

In this case, the slave unit microphone 11 used by the person using the door phone slave unit 1 shown in the block diagram of FIG.
The description of the operation at the time of establishing a call by a voice signal transmitted and received between the handset 24 used by the person using the intercom master unit 2 via the slave unit speaker 12 and the transmission line L1 via the transmission path L1 is omitted. In the above embodiment, the video doorphone is provided with the person tracking processing device. However, the present invention is not limited to this. The present invention may be applied to a videophone and the person tracking can be performed as in the above embodiment. Further, by providing, for example, an abnormality detection processing unit for detecting (determining) a crime prevention abnormality in place of the person determination processing unit included in the person tracking processing device, it can be applied not only to a person but also to abnormality detection. Furthermore,
In the above embodiment, the image having the resolution of 640 × 480 is reduced to 80 × 60, but the same effect can be obtained by changing the resolution.

[0024]

As is clear from the above description, according to the television camera monitor person tracking system of the present invention, at least a color image composed of a person and a moving object other than the person captured by the television camera is obtained. The image of the current frame, the image of the previous frame, and the image of the frame before the last, which are three continuous images, are captured, and the image of the current frame, the image of the previous frame, and the image of the previous frame are taken by the person tracking processing device of the intercom master unit. Then, the difference processing and the binarization processing are performed on the image of the frame two frames before, two difference images are obtained, and the expansion processing, the contraction processing, and the logical product processing of the two difference images are performed, and then the labeling processing is performed. In the maximum label area after the labeling processing, the person determination processing is performed, and an image near the person is extracted from the center of gravity of the maximum label area to track the position of the person, and the It can be displayed an image of a radius around a person in the center of the screen of the monitor (near person) with high accuracy.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a person tracking processing device used in an embodiment of a television camera monitor person tracking method according to the present invention.

FIG. 2 is a block diagram showing a configuration of a television doorphone to which an embodiment of a television camera monitor person tracking system according to the present invention is applied.

FIG. 3 is a flowchart showing a processing operation of the person tracking processing device of FIG. 1;

FIG. 4 is a schematic diagram showing an image in the process of the person tracking processing device of FIG. 1;

FIG. 5 is a processing operation explanatory diagram showing the processing operation of the person tracking processing device of FIG. 1;

FIG. 6 is an explanatory diagram of image reduction processing showing the operation of an image reduction processing unit provided in the person tracking processing device of FIG. 1;

FIG. 7 is an explanatory diagram of a difference process showing an operation of a difference processing unit provided in the person tracking processing device of FIG. 1;

FIG. 8 is an explanatory view of expansion / contraction processing showing operations of an expansion processing unit and a contraction processing unit provided in the person tracking processing device of FIG. 1;

9 is an explanatory diagram of a logical product process showing an operation of a logical product processing unit provided in the person tracking processing device of FIG. 1;

[Explanation of symbols]

 10 TV camera 23 TV monitor Sn ... current frame image (color image signal) Sn-1 ... previous frame image (color image signal) Sn- 2... Image of two frames before (color image signal) S1, S2... Two difference images (difference image signal)

Claims (1)

[Claims] When displaying an image of a person captured by a television camera (10) on a television monitor (23), an image of a current frame (Sn), which is a continuous image of at least three of the input images, is displayed. ), Previous frame image (Sn
-1) The absolute value of the difference between the image of the current frame and the image of the previous frame, and the absolute value of the difference between the image of the previous frame and the image of the previous frame is determined for the image (Sn-2) of the previous two frames. And binarized two difference images (S1, S2)
After performing logical expansion of the two difference images and performing dilation processing and erosion processing, labeling processing for dividing each image area is performed, and whether or not a person is in the maximum label area after the labeling processing is determined. A television camera monitor person tracking system, wherein a determination is made, and an image near a person is cut out from the center of gravity of the maximum label area and projected on the TV monitor.