JPH11113022A - Method and device for detecting mobile object in each picture type in different coding mode - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent a mobile object from being lost sight of even when a macro block exists in which a motion vector can not be sought and to always detect a correct mobile object by correcting a motion vector in each different picture type image frame by a coding mode of every macro block. SOLUTION: This device correct a motion vector by a coded mode of each macro block of every different picture type image frame in detecting a group of motion vectors as a mobile object. A motion vector correcting part 4 and a motion vector information storing part 5 are added to the device. Coding mode class information which is conventionally decoded by a variable length decoder 3 but is not used and motion vectors are given to the part 4. Next, the device checks the picture type of the current image frame and performs P picture motion vector correction processing in the case of a P picture, B picture motion vector correction processing in the case of a B picture and I picture correction processing in the case of an I picture.

Description

DETAILED DESCRIPTION OF THE INVENTION

Conventionally, in order to monitor crime prevention and disaster prevention using a monitoring camera, an observer must be stationed at the monitoring center and check the monitor monitor all the time, so that the burden on the observer is large. Was. For this reason, automatic detection of a moving object has come to be performed. To detect moving objects,
It compares continuous image frames (screens) and recognizes that a moving object is present at a location where a difference occurs. As a means for doing so, for example, taking an international standard MPEG as an example, an image frame is divided into macroblocks of 16 × 16 pixels by 15 × 22 pixels, and each of the previous image frame and the current image frame is divided into macroblocks. Are compared, and a motion vector is obtained based on the difference between the two images. For example, the example shown in FIG. 10 is obtained by converting an image frame of a P picture into a macroblock. FIG.
Numeral 28 is the minimum unit of the macroblock. The arrow shown in the round frame 29 of the image frame indicates the size and direction of the motion vector, and a blank portion around the arrow indicates that no motion vector has been generated because there is no difference as a result of comparison with the previous image frame. That is, the data in the image of the previous image frame and the data in the image of the current image frame are compared, and a block of the macroblock having a difference is detected as a moving object. The method of obtaining a motion vector based on a macroblock of an image frame is disclosed in detail in "Basics of Digital Image Compression" (author Hiroshi Yasuda, Hiroshi Watanabe) published by Nikkei BP Publishing Center, and the description is omitted here. I do.

By using a means for automatically detecting a moving object by a surveillance camera as described above, when a moving object is detected by a remote monitoring system, if a monitoring person is notified by an alarm or the like,
Based on this, the observer can easily check whether or not a suspicious person has entered by seeing the image displayed on the monitor, and the burden on the observer is greatly reduced. FIG. 6 is a block diagram showing a form example of a conventional remote monitoring system.
In FIG. 6, reference numeral 6 denotes a monitoring area-side device, which converts an image captured by the monitoring camera 7 into image-encoded data by an image encoder 8, and sends the image-encoded data from a transmission unit 9 to a transmission path 10. It is configured as follows. Reference numeral 11 denotes a monitoring center side device, which distributes the encoded image data received via the receiving unit 12 to two parts, an image decoder 14 and a moving object detecting unit 15 for obtaining a motion vector by a distributor 13, and The output of the image decoder 14 and the output of the moving object detector 15 are combined by a combiner 16 and displayed on a monitor 17. The moving object detection unit 15 has a block configuration as shown in FIG. The decoding processing unit 19 obtains a motion vector based on the encoded image data and sends the motion vector to the image processing unit 20. The image processing unit 20 detects the monitoring target moving object, and determines whether or not the monitoring target moving object exists and the center coordinates, size, Information on the direction, shape, etc. is stored in the state change detection unit 21.
The state change display signal and the moving object information for displaying an alarm and an arrow indicating the moving object on the monitoring monitor to notify the monitor if there is a state change compared with the moving object information held in advance. The image is output to the synthesizer 16 and the image reproduced by the image decoder 14 is output to the synthesizer 16.
As shown in the block configuration of FIG. 8, the decoding processing unit 19 stores the encoded image data in the buffer 23, and the quantization information, the quantized DCT coefficient, the encoding mode type information, the motion vector, To decrypt. At this time, since only a motion vector was required to detect a moving object, other quantization information, quantized DCT coefficients, and coding mode type information were not particularly used.

[0003] MPEG1 is briefly described as an example of encoded image data for displaying an image. As shown in FIG. 9, image encoding data for displaying an image is a GOP (Gr.
(Out of Pictures). That is, 15 image frames are created in 0.5 seconds. The configuration of the fifteen image frames is such that the first image 25 holds complete image encoded data capable of displaying all images, and subsequent image data is based on the first image encoded complete data. The latest image is displayed from the motion vector, or the latest image is displayed from the completed image encoded data inserted at an appropriate cycle. An encoded image frame (I picture) that does not use motion prediction of a moving object (holds complete image encoded data) unlike the first image frame
And an inter-frame forward prediction coded image frame (hereinafter, referred to as a P picture) including a motion vector obtained based on an image frame of the immediately preceding I picture or P picture with respect to the current image frame, and one of the current image frames There is a bidirectionally predictive coded image frame (hereinafter, referred to as a B picture) including a motion vector obtained based on a previous image frame and a next I or P picture image frame. Normally, the insertion ratio of pictures in the encoded image data is “IPBBPBB”. By macroblocking each picture,
Since the I-picture is only complete encoded data, the mode is an intra-frame prediction encoding (hereinafter referred to as intra-encoding) mode in which data is encoded only in the image frame without using motion prediction. If it is determined that it is more efficient to predict and display the motion using the motion vector of the existing macroblock from the immediately preceding image to display the current image, the P picture is used as the forward frame. The coding mode is set, and the mode determined to be better to display a new image is set to the intra coding mode. The B picture is similar to the P picture in addition to the forward inter-frame coding mode and the intra-coding mode, and also includes a temporally succeeding I picture.
When it is determined that the prediction efficiency is higher when the motion is predicted from the picture or the P picture, the reverse interframe coding mode is set, and the I picture or the P
If it is determined that the prediction efficiency of the motion based on the picture is higher in compression efficiency, the mode is set to the interpolative inter-frame encoding mode. The interpolative interframe coding mode has two motion vectors, a motion vector of a forward interframe code and a motion vector of a backward interframe code. Therefore,
In the intra-coding mode, motion is not predicted and displayed using the motion vector of an existing macroblock, so that it does not have a motion vector.

[0004]

However, the conventional moving object detection method and the conventional apparatus have the following drawbacks. For example, when a sample image frame in which a B picture in the image encoded data transmitted from the monitoring area side device 6 shown in FIG. 6 is macroblocked is shown in FIG. 10, one moving object is actually detected. When the motion vector as shown in FIG. 10 is obtained from the state and the next image frame is obtained as the motion vector of the sample image frame No. 2 in FIG. 11, if the intra-encoding mode 30 in FIG. It is not obtained, and it is recognized that two moving objects, the round frame 31 and the round frame 32, are newly found. Further, when the next image frame is an I-picture, a motion vector cannot be obtained by the preceding description, so that if a state in which a motion vector that is recognized as one moving object is obtained is repeated, erroneous recognition occurs. That is, the state change detection unit 21 determines the center coordinates, size, shape, and the like of the moving object for arbitrary image frames,
Since the existence of a moving object is determined only when the moving object in the image frame has continuity, it was recognized when the originally recognized moving object repeatedly became two or disappeared An erroneous recognition that one moving object cannot be followed and is lost can occur. In summary, in the conventional image coding technology, since the image frame cannot obtain the motion vectors of all the macroblocks by the coding mode for each picture, the decoding processing unit 19 only uses the image coding data from the image coding data. Then, when a moving object is detected, erroneous recognition occurs as described above. According to the present invention, the method of correcting a motion vector is changed by an image encoding technique in the image encoder 8 of the monitoring area side device 6 according to an encoding mode for each picture,
It is an object of the present invention to provide a moving object detection method and apparatus for each encoding mode for each picture type, which prevent a moving object from being lost.

[0005]

According to the present invention, a motion vector of each macroblock of image-encoded data obtained by dividing an image frame divided by a picture type into macroblocks for each required pixel unit by an image encoding means is provided. In the method of detecting the presence of a moving object in a monitoring image by using the method, a method of correcting a motion vector is changed depending on the coding mode of the macroblock. 2. An image monitoring method according to claim 1, wherein a method of correcting a motion vector is changed according to a coding mode of each macroblock for each image frame of each picture type. 3. An image monitoring apparatus, comprising: means for changing a method of correcting a motion vector according to a coding mode of each macroblock for each picture type-specific image frame according to claim 2.

[0006]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below in detail with reference to the accompanying drawings. FIG. 1 is a block diagram of an embodiment of the present invention. The difference from the conventional decoding processing unit 22 shown in FIG. 8 is that a motion vector correction unit 4 and a motion vector information storage unit 5 are added. is there. Conventionally, the coding mode type information and the motion vector that have been decoded but not used by the variable length decoder 3 are passed to the motion vector correction unit 4, and the schematic flowchart shown in FIG. First,
The picture type of the current image frame is checked (ST1).
If the picture is a P picture motion vector correction process (ST
2) is performed, and if it is a B picture, a B picture motion vector correction process (ST3) is performed; if it is an I picture,
An I picture motion vector correction process (ST4) is performed. Thereafter, in the case of a P picture or a B picture, the motion vector information is stored in the motion vector information storage unit so that it can be corrected even if a situation where a motion vector cannot be obtained in the next image frame occurs (ST5). Thereafter, the corrected motion vector information is sent to the image processing unit (ST6).

P picture motion vector correction processing (ST
In 2), as shown in FIG. 3, the coding mode of each macroblock is checked (ST1), the motion vector is used as it is for forward inter-frame prediction coding (ST2), and the motion vector is stored for intra coding. The motion vector information is read from the section and used (ST3). This process is repeated for all macroblocks (ST4).

[0008] B picture motion vector correction processing (ST
3) checks the coding mode of each macroblock as shown in FIG. 4 (ST1).
1) The motion vector information is read from the motion vector storage unit and used (ST2), and if it is forward inter-frame predictive coding (ST3), the motion vector is used as it is (ST3).
4) In the case of backward inter-frame prediction coding (ST5), the components of the motion vector are inverted and used (ST6), and in the case of interpolation inter-frame prediction coding, the motion vector of the forward inter-frame prediction code is used. The average value of the motion vector of the backward inter-frame prediction code is obtained, and the average value is used as the motion vector of the interpolative inter-frame prediction code (ST7). This process is repeated for all macroblocks (ST8).

I picture motion vector correction processing (ST
4) reads and uses the motion vector information of all macroblocks from the motion vector storage unit for intra coding as shown in FIG. 5 (ST1).

The above description is not limited to MPEG, and if an image coding mode for performing motion compensation is used, MPEG2 or H.264 is used.
261 or the like can be handled by the above processing.

[0011]

As described above, the present invention uses the motion vector of each macroblock of image coded data obtained by dividing an image frame divided for each picture type into macroblocks for each required pixel unit. When detecting a block of a vector as a moving object, by correcting the motion vector by the coding mode of each macroblock for each picture type-specific image frame, there is a case where there is a macroblock for which a motion vector cannot be obtained. This also prevents the moving object from being lost, thereby always enabling correct detection of the moving object.

[Brief description of the drawings]

FIG. 1 is a block diagram of a main part of a moving object detection device according to the present invention.

FIG. 2 is a flowchart of a main part of the moving object detection device according to the present invention.

FIG. 3 is a flowchart of a P picture motion vector correction process according to the present invention.

FIG. 4 is a flowchart of a B picture motion vector correction process according to the present invention.

FIG. 5 is a flowchart of an I-picture motion vector correction process according to the present invention.

FIG. 6 is a block diagram of a conventional remote monitoring system.

FIG. 7 is a block diagram of a conventional moving object detection unit.

FIG. 8 is a block diagram of a conventional decoding processing unit.

FIG. 9 shows a GOP (Group Of Picture).
s) Configuration diagram.

FIG. 10 is an image frame sample 1;

FIG. 11 is an image frame sample No. 2;

[Explanation of symbols]

 1, 15, 18, 22: Moving object detection unit 2, 23: Buffer 3, 24: Variable length decoder 4: Motion vector correction unit 5: Motion vector information storage unit 6. ··· Monitoring area side device 7 ··· Monitoring camera 8 ··· Image encoder 9 ···· Transmission unit 10 ··· Transmission line 11 ··· Monitoring center side device 12 ··· Reception unit 13 ··· Distributor 14 ... Image decoder 16 ... Synthesizer 17 ... Monitoring monitor 1,19,22 ... Decoding processor 20 ... Image processor 21 ... Status change detector 25 ... · I picture 26 ··· P picture 27 ··· B picture 28 ··· macrobook 29 ···································· Separated into two

Claims (2)

[Claims] An image frame divided for each picture type by a moving picture coding means is monitored by using a motion vector obtained in a coding mode of each macroblock obtained by converting the picture frame into macroblocks for each required pixel unit. An image monitoring method for detecting the presence of a moving object in an image, wherein a method of correcting a motion vector is changed according to an encoding mode of each macroblock for each image frame of each picture type. 2. A monitoring method using a motion vector obtained by an encoding mode of each macroblock in which an image frame divided for each picture type is divided into macroblocks for each required pixel unit by a moving image encoding means. An apparatus comprising means for detecting the presence of a moving object in an image, comprising means for changing a method of correcting a motion vector according to an encoding mode in each macroblock for each image frame of each picture type. Image monitoring device.