JP3948596B2 - Moving object detection and tracking device in moving images - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a moving object detection / tracking apparatus in a moving image, and in particular, to detect a moving object in a moving image that can extract a moving region at high speed and with high accuracy from information obtained by decoding compressed encoded data itself or only a part thereof. It relates to a tracking device.
[0002]
[Prior art]
As a conventional method for detecting a moving region (hereinafter referred to as a first detection method), particularly a method for detecting a moving object, the differential or difference of a certain scanning line in a target frame with respect to a background image held in advance. There is a method that uses a sum of values for detection detection of a moving object. In addition, a method for obtaining a moving object from an optical flow, a method for detecting a moving region in consideration of a luminance distribution correlation, distance information, and velocity information in a small region have been proposed. These methods are methods for extracting a moving region by applying various algorithms to the luminance values of each pixel of a moving image.
[0003]
In order to apply this first detection method to compression-encoded moving image data, it is necessary to once decode the encoded data. For example, this is realized by the configuration shown in the block diagram of FIG. The variable length decoding unit 51 in FIG. 9 receives compressed image encoded data and performs decoding processing such as variable length decoding, inverse quantization, and coefficient range restriction. The variable length decoding unit 51 outputs motion prediction error information a, motion prediction information b, encoding information, and the like of the decoded frame or block. The motion prediction error information a is input to the image generation unit 52 and is inversely converted by the image generation unit 52 to generate an image of one frame. Further, the motion compensation unit 53 extracts the reference image c from the image memory 54 holding the motion prediction reference frame using the motion prediction information b. The adder 55 adds the output image from the image generation unit 52 and the motion compensation image from the motion compensation unit 53 to generate a complete decoded image d of one frame. The decoded image d is sent to the moving area detecting unit 56, and the moving area detecting process of the first detection method described above is performed. Further, if the decoded image d is a frame that becomes a reference image after the next frame, it is stored in the image memory 54. The image subjected to the movement area detection process by the movement area detection unit 56 is displayed on the image display unit 57.
[0004]
In addition, as another conventional moving region detection method (hereinafter referred to as a second detection method), a method has been proposed in which encoded data of a compressed moving image is used instead of using the pixel itself. Yes. This method achieves a moving region detection process by directly manipulating various parameters and encoded data required for compression. In the detection process, the movement area is extracted by selecting only necessary information, centering on the motion prediction information of the code unit block in the image. For this reason, the extracted resolution is set to the size of the encoding unit block of each compression method having motion prediction information.
[0005]
Further, as a conventional moving area tracking method (hereinafter, referred to as a third detection method), there is a technique of using past detection results in detection processing of future frames along with detection processing. This technique is effective for both the first and second detection methods. In the first detection method, the moving area is estimated from the area of the target object in the previous frame and the velocity of the optical flow. In the second detection method, the moving area is estimated based on the motion prediction information. In these methods, a predicted future position is estimated for each region from the trajectory of the moving region of the past several frames, and the estimation result is used for the detection processing determination of the next frame.
[0006]
[Problems to be solved by the invention]
However, the above first to third detection methods have the following problems. Since a moving image has a very wide signal band and requires an enormous amount of data, the moving image is generally widely used for recording and transmission in a compressed form. Therefore, in the first detection method in which the moving area detection process in the pixel area is performed for the first time after the decoding process of one frame, in addition to the cost of the original moving area detection process, the large cost of the moving picture decoding process There is a problem that requires.
[0007]
On the other hand, the second detection method uses the presence / absence of motion prediction information for motion compensation in the determination of the moving region. Therefore, the decoding process can be omitted and the detection process can be executed at high speed. However, due to factors such as ambient light scattering, scattering, and atmospheric fluctuations, an actual moving image may be assigned a motion vector to the change even in an originally stationary background region in the compression process. In particular, this influence is large in a region where pixels are uniform, and there is a problem that detection accuracy is inferior, for example, a still region is erroneously recognized as a moving region.
[0008]
Further, in the third detection method, there is a method of obtaining the trajectory of the moving object from the trajectory of the center of gravity and the average of the intra-region motion vectors in the tracking process in the encoded data region. However, in the locus of the center of gravity, if the moving object is small, the center of gravity may fluctuate greatly with a slight increase or decrease in the number of blocks. In addition, in the case of a simple motion vector average, there is a problem that it is also affected by information with low reliability such as motion prediction information in a flat part.
[0009]
An object of the present invention is to solve the above-described problems of the prior art and to detect and track a moving object at high speed and with high accuracy from compressed encoded data itself or information obtained by decoding only a part thereof. It is to provide a detection tracking device.
[0010]
[Means for Solving the Problems]
In order to achieve the above-described object, the present invention provides a moving object detection / tracking apparatus in a moving image which receives compressed moving image data as an input, adds moving region information to the moving image data, and outputs the moving image data. A variable length decoding unit that partially decodes the compressed moving image data and outputs encoding mode information, prediction error information, and motion prediction information; and the encoding mode information; Position information of motion-predicted area When A detection target setting unit that sets a detection processing target block of a moving object, a moving region detection unit that detects whether or not the detection processing target block set by the detection target setting unit belongs to the movement region, An area motion prediction unit that predicts the movement of the entire movement area from the movement area detected by the movement area detection unit; The moving region detecting unit detects a block including both the prediction error information and the motion prediction information or a block encoded only with intra-frame information as a moving region, and the region motion predicting unit detects the moving region. A weighted average corresponding to the power of the prediction error information is taken with respect to the motion compensation information, and the weighted average is used as the motion prediction of the entire moving region, and the motion prediction is provided to the detection target setting unit. There is a feature in the point.
[0011]
According to the present invention, in addition to partially decoding the compressed and encoded moving image data, the moving object detection processing target is set based on the position information of the motion-predicted region, and the set Since whether or not a block belongs to a moving region is detected, it goes without saying that the conventional pixel region detection method (the first detection method) is a detection method in the code data region (the first one). Compared with the detection method (2), the processing cost can be greatly reduced.
[0012]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an embodiment of the present invention.
[0013]
The variable length decoding unit 1 partially decodes the compressed encoded data of the input moving image, and outputs encoding mode information p, prediction error information a, and motion prediction information b of the current frame. The encoding mode information p is sent to the detection target setting unit 2 and the moving region detection processing unit 3, and the prediction error information a and the motion prediction information b are sent to the moving region detection processing unit 3.
[0014]
The coding mode information p includes a frame coding mode and a block coding mode. The frame coding mode includes an intra-frame coded frame (I picture), a forward prediction frame (P picture), and bidirectional prediction. There are three types of frames (B pictures). On the other hand, the block coding mode is classified based on the presence or absence of motion compensation and coding, intra-frame coding block (Intra), motion compensation coding block (MC coded), frame differential coding block (no MC coded). There are four types of motion compensation blocks (MC no coded). There are three types of motion prediction: forward direction, reverse direction, and both directions.
[0015]
Next, the detection target setting unit 2 receives the encoding mode information p from the variable length decoding unit 1 and the motion prediction position information q of the region from the region motion prediction unit 4, and detects the detection target block position information r and the detection. Output the result. The function of the detection target setting unit 2 will be described with reference to the flowchart of FIG. 2 and FIG. The detection target setting unit 2 performs processing using the block coding mode in the coding mode information p.
[0016]
Although the detailed function of the area motion prediction unit 4 will be described later, if the moving area C provided from the detection result memory 5 is an area composed of the blocks B1 to B5 in FIG. 4 compares the area C with the current block data, uses the motion prediction MV of the area C, and outputs the motion estimation position information q of the area as shown in FIG.
[0017]
In step S2 of FIG. 2, for example, it is determined whether or not each block in FIG. 2B is completely covered by the inner edge of the motion prediction position q of the region, that is, the prediction position q. . When this determination is affirmative, the block is determined to be a moving object in view of the temporal continuity of the moving area (step S10), and the process proceeds to step S8. In step S8, a process that does not set the detection target flag is performed. For example, the block Bm in FIG. 5B is determined as a moving object. On the other hand, when this determination is negative, the process proceeds to step S3, where it is determined whether or not the block is partially covered by the outer peripheral portion of the motion predicted position q of the region, that is, the predicted position q. For example, it is determined that the blocks Bn, Bn + 1, etc. in FIG. 5B are at the outer periphery, and the block Bs is not at the outer periphery.
[0018]
If this determination is affirmative, the process proceeds to step S4. In step S4, the possibility of the moving area is determined using the encoding mode information, and it is determined whether or not the detection target is appropriate. Intra blocks existing in P and B pictures often appear when motion compensation cannot maintain sufficient image quality, such as when a background hidden behind a moving area appears, and are therefore subject to detection.
[0019]
On the other hand, since no MC coded has no motion prediction information, it is unlikely that it is a moving region, and it is excluded from the detection determination processing target and determined as a still region. MC no coded has motion prediction information representing motion but no prediction error information. The absence of prediction error information indicates that the corresponding block is a flat region with low vector reliability or does not require prediction error information in a completely stationary region. Therefore, MC no coded which does not have prediction error information and has poor reliability of motion prediction information is excluded from the detection determination processing target and determined as a still region. MC coded has both motion prediction information and prediction error information.
[0020]
When the determination in step S4 is affirmative, the process proceeds to step S5, and a detection target flag is set in the block. On the other hand, when the determination at step S3 is negative or when the determination at step S4 is negative, that is, when the block coding mode of the block is MC no coded or no MC coded, the process proceeds to step S6 and Determined. In step S7, processing for not setting a detection target flag in the block is performed.
[0021]
In step S9, it is determined whether or not the detection target process has been completed for all the blocks of one frame. If this determination is negative, the process returns to step S1 to perform the process for the next block.
[0022]
Therefore, in FIG. 5B, it is determined that the blocks Bn, Bn + 1, etc. are in the outer periphery of the motion prediction position q of the region, and if the block coding mode of these blocks is MC coded or Intra, A detection target flag is set in these blocks. These blocks are sent to the moving area detection processing unit 3 as moving area detection targets in the current frame. Specifically, the position information r of the block for which the detection target flag is set in step S5 is sent to the moving area detection processing unit 3. Further, the block determined as the moving area or the stationary area in the detection target setting section is sent to the detection result display section and the detection result memory.
[0023]
Next, the function of the moving area detection processing unit 3 will be described with reference to FIG. The moving region detection processing unit 3 receives the encoding mode information p, the prediction error information a, and the motion prediction information b from the variable length decoding unit 1, and the detection target block position information r from the detection target setting unit 2. Is done.
[0024]
In step S11, it is determined whether or not the detection target block position information r is input. If this determination is affirmative, the process proceeds to step S12, and the block is moved using DCT coefficient determination, that is, using the DCT coefficient. A determination is made as to whether the region is a candidate. If it is determined in this determination that the block is a moving area candidate, the process proceeds to step S13. If it is determined negative, the process proceeds to step S17. In step S13, it is determined whether or not the macroblock type of the block is MC coded. If this determination is affirmative, the process proceeds to step S14, and if not, the process proceeds to step S15.
[0025]
In step S15, it is determined whether the block corresponds to a moving area candidate based on the motion vector. If it is determined in this determination that the block is a moving area candidate, the process proceeds to step S15. If it is determined that the block is a still area, the process proceeds to step S17. Next, in step S15, it is determined whether or not the block corresponds to a moving region candidate by spatial correlation. If it is determined in this determination that the block is a moving area candidate, the process proceeds to step S16, and it is finally determined that the block is a moving area candidate. On the other hand, if the determination is negative, the process proceeds to step S17 and is determined to be a still area.
[0026]
In step S18, it is determined whether or not the above-described processing has been completed for all the moving area detection target blocks. If this determination is negative, the processing returns to step S11 and the same processing is repeated.
[0027]
The processing result obtained by the above processing is shaped in the area as necessary and output to the detection result display unit 6 and at the same time is output and stored in the detection result memory 5.
[0028]
Next, details of step S12 based on the motion prediction error information will be described. Here, DCT coefficients are used as motion prediction error information. A DCT coefficient can be used as information representing the shape or pattern of a moving object. However, when the DCT coefficient of the block itself is used, all regions having a complex texture are extracted regardless of whether or not there is movement. The DCT coefficients of motion prediction errors have only a few DCT coefficients when the background with a complex pattern is stationary because motion prediction compensates for most of the information. Therefore, when the motion prediction error amount is large, it can be set as a moving region candidate.
[0029]
If this determination is satisfied, the process proceeds to the motion vector correlation determination unit. Otherwise, it is a static area. As a specific determination method using DCT coefficients, threshold processing such as the number of DCT coefficients equal to or greater than the threshold, the sum of absolute values of DCT coefficients, and the sum of squares can be used. Moreover, the following methods can also be used. As shown in FIG. 5A, DCT coefficient partial sum determination uses a partial sum of low-frequency AC components excluding DC components as a criterion, and narrows down areas having significant error information as candidate areas. The AC component closer to the low frequency here refers to DCT coefficient information from the first to the thr first in the coding scan order such as zigzag scanning in the prediction error information. This AC component is a difference value Ci, j, z (0 ≦ z ≦ 63) by motion compensation in block j of frame i. At this time, it is determined that the expression (1) in FIG. 11 is satisfied. The threshold value thr 2 used for determination is set for each frame encoding mode.
[0030]
When this determination is affirmative, since the block is an area having significant error information, the process proceeds to step S32 to be a moving area candidate. On the other hand, when the determination is negative, the process proceeds to step S33 to determine that the block is a still area. The AC component is a difference value by motion compensation, and the threshold is set for each frame coding mode.
[0031]
Next, details of the motion vector determination in step S14 will be described with reference to FIGS. 6 (a) to 6 (d). In step S41, it is determined whether or not the length of the motion vector (MV) of the block is greater than or equal to a predetermined threshold (see FIG. 5B). Since a motion vector with a short length is considered as a noise vector due to various disturbances, a block having the motion vector with a short length is a still region.
[0032]
If the motion vector length is greater than or equal to the threshold, the process proceeds to the spatial determination unit. Here, the motion vector of the moving region is roughly proportional to the distance between the reference frame and the input frame. Therefore, the moving region candidate can be determined by threshold processing according to the interframe distance. For example, the threshold for determining the vector length is changed according to the temporal distance to the reference frame, and is reset to the initial threshold when an I picture or P picture is output. The forward and backward motion vectors of block j in frame i are fmvi, j and bmvi, j, respectively, the distance from the input frame to the forward reference frame is ui, the P picture interval is m, and the initial threshold is thr 3 Then, for example, discriminants such as the following formulas (2) and (3) can be used to determine the moving region candidates. However, || represents Euclidean norm.
| Fmvi, j | ≧ ui · thr 3 (2)
| Bmvi, j | ≧ (m−ui) · thr 3 (3)
[0033]
When the input block is a bidirectional prediction vector, both vector lengths of the forward prediction vector and the backward prediction vector are verified. If both expressions are satisfied, the region is set as a moving area candidate. Otherwise, it is determined as the background area. In the case of unidirectional prediction, only the forward prediction or the backward prediction vector is verified. When the condition is satisfied, it is determined that the motion vector has a sufficient length, and the process proceeds to the next determination process. Otherwise, it is determined as a still area.
[0034]
Next, in step S42, it is determined whether or not the block is a bidirectional prediction block. If this determination is affirmative, the process proceeds to step S43, and if negative, the process proceeds to step S44. The bi-predictive block contains two motion prediction vectors. Assuming that the motion of the moving region in a short time is linear, the directions indicated by the two motion prediction vectors are opposite to each other. Therefore, in decision S43, the angle of the vector indicating the forward direction and the reverse direction is investigated. As a method for examining the directions of the two vectors, a method using an inner product value as shown in the following equation (4) is used. thr 4 sets the threshold of the angle created by the two vectors. If Expression 4 is satisfied, it is determined that the correlation between the motion vectors is high, and the process proceeds to temporal correlation determination. Otherwise, it is determined as a still area.
fmvi, j · bmvi, j / | fmvi, j | · | bmvi, j | ≦ cos (thr 4) (4)
[0035]
In step S43, in bi-directional prediction, it is determined whether or not the angle θ of the vector indicating the forward direction and the reverse direction is equal to or greater than a predetermined threshold (see FIG. 5C), and blocks exceeding the threshold are moved. A region candidate is determined, and if it is smaller than the threshold, it is determined that the region is a still region.
[0036]
Next, in step S44, it is determined whether or not the temporal change rate of the motion vector MV is proportional to the inter-frame distance (see FIG. 4D). Assuming that the movement of the object is locally a linear movement, a block having a temporal correlation between the motion prediction information of the region referred to by the current block and the motion prediction information of the current block is determined as a moving region candidate. select. For the bi-predictive block, the temporal correlation between the forward prediction and backward prediction vectors and the motion vector between the reference frames is investigated, and a block with temporal correlation is selected as a moving region candidate.
[0037]
If there is no temporal correlation in the motion vector, the block is determined as a still area. Otherwise, it is determined as a moving area with respect to the motion vector information. An example of determining temporal correlation is shown in FIG. FIG. 6D uses the fact that the motion vectors pointing to the same moving area in the reference frame are proportional to the distance between the frames. For the block j of the current frame i, the comparison target block in the frame k to be compared is l Threshold thr Equations (5) and (6) suppressed by 5 are used as discriminants for time correlation. Where block l Will be described later, but block j and l Corresponds to the same moving object in each frame. When the input block is a bidirectional prediction vector, the temporal correlation is verified for the forward prediction vector and the backward prediction vector. If both expressions are satisfied, the region is set as a moving area candidate. Otherwise, it is a static area. In the case of unidirectional prediction, the determination is made only with either the forward prediction vector or the backward prediction vector.
| Uk · fmvi, j −ui · fmvk, l | ≦ thr 5 (5)
| (M−uk) · bmvi, j − (m−ui) · bmvk, j | ≦ thr 5 (6)
[0038]
There is a degree of freedom in selecting the frame k to be compared. For example, the frame k is a frame (i−1) immediately before the frame i. However, if the frame k is an I picture or a P picture, the reference frame of the frame i and the reference frame of the frame k are different, so the future reference frame (i−1 + m) is set as the comparison target frame k. Block to be compared in frame k l Is a block occupying a place where the current motion vector is internally divided by the ratio of the distance to the reference frame as shown in FIG. If the discriminant is satisfied, block j, block l The motion vector length ratio is close to the distance ratio to the reference frame, and the motion prediction information is determined to be highly reliable. Otherwise, it is determined that the motion prediction information has low reliability, and the block is a still area.
[0039]
Also, in a coding scheme in which no B picture exists, each motion vector in a continuous P picture is similar to the amount of motion prediction information in the immediately preceding reference frame. In such a case, the correlation with the motion vector of the immediately preceding P picture can be investigated. For example, the correlation is investigated using the motion vector of the previous P picture and the absolute difference value of the current block as shown in the following equation (7).
| Fmvi, j -fmvi-1, l | ≦ thr 6
[0040]
Next, details of the spatial correlation determination in step S15 will be described with reference to FIG. FIG. 7A is a flowchart showing the same region generation unit, and the spatial correlation is examined in the flowchart. The input information includes coding mode information, motion prediction information, prediction error information, and moving area candidate information. First, the input information moves to the same area generation unit, and forms the same area.
[0041]
In coding mode determination (step S51) in FIG. 7 (a), MC coded and Intra are examined. In the case of MC coded, the process proceeds to vector length determination (step S52), and in the case of Intra, the process proceeds to DC determination (step S54). In motion vector length determination, the similarity of motion vector lengths is determined by Equation (8) and Equation (9) for neighboring moving region candidate blocks. When the input block is a bidirectional prediction vector, both vector lengths of the forward prediction vector and the backward prediction vector are verified. If both equations are satisfied, the process proceeds to motion vector angle determination (step S53). Otherwise, it is determined that they are not in the same area (step S56). In the case of unidirectional prediction, only the forward prediction or the backward prediction vector is verified. If the difference between the vector lengths of the block j and the block n is equal to or smaller than the threshold value, the process proceeds to the motion vector angle determination (step S53). Otherwise, it is determined that they are not in the same area (step S56). For example, the neighboring block n indicates eight surrounding blocks centered on the input block j.
| Fmvi, j | − | fmvi, n | ≦ thr 7 (8)
│bmvi, j │-│bmvi, n │≤thr 7 (9)
[0042]
In the motion vector angle determination (step S53), the similarity of the angle is determined by the equations (10) and (11) with respect to the neighboring moving region candidate blocks. When the input block is a bidirectional prediction vector, both vector angles are verified for the forward prediction vector and the backward prediction vector. In the case of unidirectional prediction, only the forward prediction or the backward prediction vector is verified. If the angle formed by the vector of block j and block n is equal to or smaller than the threshold value, the process proceeds to DC component determination (step S54). Otherwise, it is determined that they are not in the same area.
fmvi, j · fmvi, n / | fmvi, j | · | fmvi, n | ≦ cos (thr8) (10)
bmvi, j · bmvi, n / | bmvi, j | · | bmvi, n | ≦ cos (thr8) (11)
[0043]
In the DC component determination (step S54), the similarity of the DCT coefficient DC component is determined. However, the DCT coefficient similarity determination is performed using the original DCT coefficient, not the difference value after motion compensation. The DC component is the reference block in reference frame k l Is obtained by adding the DC component Ck, j, 0 and the prediction error DC component Ci, j, 0. If the difference of the DC component from the neighboring block n shown in the equation (12) is equal to or smaller than the threshold value thr 9, it is determined as the same region (step S55), otherwise it is determined as not the same region. Here, the block s is a neighboring block of the block n.
| (Ci, j, 0 + Ck, l , 0) − (Ci, n, 0 + Ck, s, 0) | ≦ thr 9 (12)
[0044]
In all the moving area candidate blocks, blocks having sufficient similarity are determined to be the same moving area and combined with each other (step S57). At the same time, the combined information is output as the same movement area information.
[0045]
The same area information obtained from FIG. 7A is input to the spatial correlation determination of FIG. 10 (step S61). This determination verifies the spatial correlation of the moving region. A region where the number of blocks forming the same region is smaller than a threshold is an isolated block in which no similar block exists in the vicinity. Therefore, if the number of area blocks is less than the threshold and the spatial correlation is poor (No in step S62), the area is set as a still area (step S64). Otherwise (judgment in step S62 is affirmative), it is determined as a moving area candidate (step S63).
[0046]
When the processing for all the moving area detection target blocks is completed (step S65), the detection processing result is output to the detection result display unit 6, and at the same time, this result is used for detection in a future frame. Output and record.
[0047]
For example, when there is a motion vector for each block as indicated by an arrow in FIG. 7B, the length and angle of the motion vector of the adjacent block, the DCT coefficient DC, with respect to the moving region candidate blocks B1 to B5. The correlation between components is examined, and the degree of connection with neighboring blocks is calculated. Based on this, blocks having similar angle and length motion vectors are integrated to form the same region, and a moving object region is determined based on this.
[0048]
Next, the function of the region motion prediction unit 4 (see FIG. 1) will be described. The region motion prediction unit 4 performs motion prediction for each region, and predicts the appearance position of the region in a future frame. Assuming that the shape of the moving object is almost the same in local time, the moving object determination result in the immediately preceding frame is used to track the moving object.
[0049]
FIGS. 8A, 8B, and 8C show a method for predicting the motion of the entire area obtained by the detection process. The input information to the region motion prediction unit 4 (see FIG. 1) is the configuration information of the moving region that is the output of the moving region detection processing unit 3 as shown in FIG.
[0050]
The estimation of motion prediction for each region is performed as follows. The motion prediction for the entire area reflects the power of the DCT coefficient representing the texture. Since it is difficult for the motion vector to accurately capture the motion of the object in a flat region having no texture, the power of the DCT coefficient can be used as a reference for the reliability of the motion vector. Therefore, based on the prediction error information of the input information, the motion vector is weighted and averaged according to the power of the DCT coefficient to obtain the motion of the entire region (step S71). Here, the weight calculation unit can use information such as the sum of absolute values of DCT coefficients, the sum of squares, and the number of thresholds or more, etc., for the power representation of the weights wi, j. For example, the formula (13) in FIG. 11 shows a specific formula using the sum of absolute values of DCT coefficients as a weighting coefficient. Here, thr 1 is a threshold value used for separating the low frequency and high frequency of the DCT coefficient in equation (1).
[0051]
The weight information obtained by the weight calculator proceeds to the motion vector calculator. For the weight information, the motion vector calculation unit obtains a weighted average of the motion vectors using the weighting coefficient w i, j obtained by the weight calculation unit (step S72). When motion prediction vectors for an entire moving region R are set to fmvR and bmvR, they are obtained by equations (14) and (15) in FIG. 11, respectively. The calculation result moves to the scaling unit.
[0052]
In the scaling unit, the motion prediction vector of the entire region is subjected to scaling correction with the inter-frame distance information, and is used as motion prediction position information indicating the location in the next frame (step S73). The prediction vector mvR in the next frame is obtained from equation (16) when the weighted average bmvR of the backward motion prediction information exists in the moving region R. On the other hand, when there is no backward motion prediction information in the moving region R or in the case of a P picture, the weighted average fmvR of the forward motion prediction information is inverted and obtained from Equation (17). In the I picture, the motion in the immediately preceding frame is retained. The motion prediction information for each region is output to the aforementioned detection target setting unit.
mvR = bmvR / (m-ui) (16)
mvR = -fmvR / ui (17)
[0053]
【The invention's effect】
As is apparent from the above description, according to the present invention, in addition to partially decoding the compression-coded moving image data, both the motion prediction information and the prediction error information are targeted for the moving region detection processing. By limiting to the provided block, it goes without saying that the conventional pixel area detection method (the first detection method) is compared with the detection method in the code data area (the second detection method). Even processing costs can be greatly reduced.
[0054]
Further, since the detection processing targets are limited based on the past detection processing results, it is possible to effectively reduce the processing cost. Further, the processing result information from the past maintains temporal continuity, and a natural shape can be ensured when the detection result is confirmed by overlapping with the reproduced image.
[0055]
Further, according to the present invention, since the motion prediction of the moving region in the past is obtained from the weighted motion prediction vector considering the reliability of the motion prediction information, the moving region can be reliably tracked. As a result, it is possible to detect a moving object without degrading the detection accuracy even if the number of blocks and frames to be investigated are greatly reduced. In other words, since the method of the present invention can minimize the application range of detection determination, it is possible to further utilize the high speed of moving region extraction on compressed and encoded data.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a schematic configuration of an embodiment of the present invention.
FIG. 2 is a flowchart showing an operation of a detection target setting unit in FIG. 1;
FIG. 3 is an explanatory diagram of an operation of a detection target setting unit in FIG. 1;
4 is a flowchart showing an operation of a moving area detection processing unit in FIG. 1;
FIG. 5 is a detailed explanatory diagram of step S12 of FIG.
FIG. 6 is a detailed explanatory diagram of step S14 in FIG.
FIG. 7 is a detailed explanatory diagram of step S15 in FIG.
FIG. 8 is a detailed explanatory diagram of a region motion prediction unit in FIG. 1;
FIG. 9 is a schematic block diagram showing a configuration of a conventional apparatus.
FIG. 10 is another detailed explanatory diagram of step S15 of FIG.
FIG. 11 is a diagram illustrating a mathematical expression.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Variable length decoding part, 2 ... Detection object setting part, 3 ... Moving region detection process part, 4 ... Area motion estimation part, 5 ... Detection result memory, 6 ... Detection result display part

Claims (6)

In a moving object detection and tracking device in a moving image that receives compressed moving image data as an input, adds moving area information to the moving image data, and outputs it.
A variable length decoding unit that partially decodes the compressed moving image data and outputs encoding mode information, prediction error information, and motion prediction information;
Based on the position information of the encoding mode information and the motion predicted region, a detection target setting unit for setting a detection target block of the moving object,
A moving region detecting unit for detecting whether or not the detection processing target block set by the detection target setting unit belongs to the moving region;
An area motion prediction unit that predicts the movement of the entire movement area from the movement area detected by the movement area detection unit;
The moving region detection unit detects a block including both the prediction error information and motion prediction information, or a block encoded only with intra-frame information as a moving region,
The region motion prediction unit takes a weighted average corresponding to the power of the prediction error information for motion compensation information in the moving region, sets the weighted average as a motion prediction of the entire moving region, and detects the motion prediction as the detection A moving object detection and tracking device in a moving image, which is provided to a target setting unit . The moving object detection and tracking device in a moving image according to claim 1 ,
For the power expression of the prediction error information, any one of the number of DCT coefficients having a value equal to or larger than a threshold value in the moving region, an absolute value sum of DCT coefficients, a square sum, and a partial sum is selectively used. A moving object detection and tracking device in a moving image. In the moving region detection tracking apparatus in the moving image according to claim 1,
The position information of the motion-predicted region is motion prediction position information of the region predicted by the region motion prediction unit, and the detection target setting unit determines an inner edge portion of the motion prediction position of the region as a moving object. A moving object detection and tracking device in a moving image.   The moving object detection and tracking device in a moving image according to claim 1, wherein the position information of the motion-predicted region is motion prediction position information of the region predicted by the region motion prediction unit, and the detection target setting The moving object detection and tracking device in a moving image is characterized in that an outer peripheral portion of the motion prediction position of the region is a detection target of the moving region. 2. The moving region detection and tracking device in a moving image according to claim 1, wherein the position information of the motion predicted region is motion predicted position information of the region predicted by the region motion prediction unit, and the detection target setting is set. Is a peripheral part of the motion prediction position of the region predicted by the region motion prediction unit, and a predicted position corresponding to predetermined block coding information in the coding mode information is set as a detection target of the moving region A moving object detection and tracking device in a moving image. In the moving region detection tracking apparatus in the moving image according to any one of claims 1 to 5 ,
The region motion prediction unit performs tracking based on temporal distance between frames based on motion information of the entire region predicted for motion, and then tracks the entire region. Detection tracking device.

Images