JP6436846B2 - Moving object detection device, video decoding device, and moving object detection method - Google Patents

Description

  The present invention relates to a moving object detection device, a video decoding device, and a moving object detection method for detecting a moving object from an encoded video.

  Moving object detection from video is an important technology in various applications such as video surveillance. In order to detect a moving object from an image, it is necessary to first obtain motion information of the image, but this generally requires a large amount of computation. For this reason, for example, high-performance hardware is required to perform real-time processing, which is a cause of narrowing the application range of applications.

  In addition, since the amount of information is enormous if the video is as it is, generally MPEG or ITU-TH. An encoding process is performed using an international standard video encoding method such as 26x, and the compressed data is transmitted and stored in a compressed state. Therefore, in order to detect a moving object with respect to an encoded video, it is necessary to first decode the video before performing a process for obtaining motion information, which further increases the amount of calculation.

  By the way, MPEG and ITU-TH. In the international standard video coding scheme such as 26x, first, a video is divided into small blocks, a prediction process is performed for each block, and only the prediction residual is transmitted to reduce the amount of information. Here, the prediction of a block to be encoded includes intra-frame prediction performed using only the frame to which the block belongs, and motion used by searching for a block similar to the encoding target from other encoded frames. It is divided into compensation prediction. When performing motion compensation prediction, a vector representing a movement from a current block to a similar block (so-called “reference block”) in another frame is encoded and output as a part of a bitstream. This vector is called a motion vector. When the encoding target block includes a moving object, it corresponds to the motion information of the moving object. That is, if the motion vector is decoded from the encoded video, the motion information of the video can be obtained with a small amount of calculation. For example, Patent Document 1 and Patent Document 2 propose a video analysis method using a motion vector of an encoded video.

JP-A-6-292203 Japanese Patent Laid-Open No. 10-75457

  Motion vectors calculated in the process of motion compensation prediction in video coding do not always represent subject motion with high precision, and bitstreams may contain motion vectors with low reliability. Many. For this reason, as in the prior art, when a moving object is detected by simply decoding a motion vector from an encoded video, an unreliable motion vector acts as noise and frequent false detections occur. There was a problem that accurate moving object detection could not be performed.

  The present invention has been made to solve the above-described problems, and a moving object detection apparatus and a moving object detection method that improve the accuracy of moving object detection by reducing the influence of motion vectors with low reliability. The purpose is to provide. Another object of the present invention is to provide a video decoding device having such a moving object detection device.

A moving object detection apparatus according to the present invention includes a variable length decoding unit that decodes a motion vector between an encoding target frame of a coding block and a reference frame from a video bitstream encoded by motion compensation prediction, and a plurality of motion vectors A voting value calculation unit that calculates a voting value for each of the voting value calculated by the voting value calculation unit, and a voting unit that votes the coordinates indicated by the motion vector of the voting surface corresponding to the reference frame of each motion vector; A voting value voted by the voting unit is accumulated for each coordinate of the voting plane, and a cumulative voting value storage unit that stores a cumulative voting value, and a moving object included in the video bitstream is detected using the cumulative voting value. It includes a moving object detection unit which, a voting value calculation unit, a variable length decoding unit a vote value for the motion vector using the index decoded from the video bitstream Been made to output, and calculates a large voting value as the block size of the encoded block is less with the motion vector.

  A video decoding apparatus according to the present invention includes the above video decoding apparatus and a decoded image generation unit that generates a decoded image from a video bitstream.

In the moving object detection method of the present invention, the variable length decoding unit decodes the motion vector between the encoding target frame of the encoding block and the reference frame from the video bitstream encoded by the motion compensation prediction, and calculates a voting value. A voting value for each of a plurality of motion vectors, and a voting unit sets the voting value calculated by the voting value calculation unit to a coordinate indicated by a motion vector of a voting surface corresponding to a reference frame of each motion vector. The accumulated vote value storage unit stores the accumulated vote value that is a value obtained by accumulating the vote value voted by the vote unit for each coordinate of the voting plane, and the moving object detection unit uses the accumulated vote value to store the video bit. in the moving object detection method for detecting a moving object included in the video stream, voting value calculating unit includes a motion variable length decoding unit by using the index decoded from the video bit stream vector A calculates a voting value for, and calculates the larger voting value block size of the encoded block is less with the motion vector.

  The moving object detection device, the video decoding device, and the moving object detection method of the present invention reduce the influence of a motion vector with low reliability by integrating and using a plurality of motion vectors in a plurality of frames. Accuracy can be improved.

It is a block diagram which shows the principal part of the moving object detection apparatus which concerns on Embodiment 1 of this invention. It is explanatory drawing which shows the reference structure of the flame | frame in motion compensation prediction. It is explanatory drawing which shows the moving object in each flame | frame. It is explanatory drawing which shows the moving object and encoding block in each flame | frame, and the coordinate which a motion vector points out. It is explanatory drawing which shows the voting surface of the initial state memorize | stored in the accumulation vote value memory | storage part. H. 3 is an explanatory diagram illustrating the size of a coding block in H.265. FIG. It is explanatory drawing which shows the mode of correction | amendment of the vote value using a Gaussian function. It is explanatory drawing which shows the voting result which made the block size 8x8, 16x16, 32x32, and 64x64. It is explanatory drawing which shows the mode of the vote by a voting part. It is explanatory drawing which shows the mode of the vote by a voting part. It is explanatory drawing which shows the mode of the vote by a voting part. It is explanatory drawing which shows the reference structure of the flame | frame in motion compensation prediction. It is explanatory drawing which shows the mode of the vote of the reverse direction by a voting part. It is explanatory drawing which shows the voting surface which the moving object detection part read from the accumulation value memory | storage part. It is explanatory drawing which shows the result of the threshold value process with respect to the voting surface of FIG. It is explanatory drawing which shows the moving object detection result which a moving object detection part outputs. It is a flowchart which shows operation | movement of the moving object detection apparatus which concerns on Embodiment 1 of this invention. It is a block diagram which shows the principal part of the video decoding apparatus which concerns on Embodiment 2 of this invention. It is explanatory drawing which shows the decoded image which the video decoding apparatus concerning Embodiment 2 of this invention decoded. It is explanatory drawing which shows the image which synthesize | combined the detection result of the moving object with the decoded image of FIG. It is explanatory drawing which shows the image which synthesize | combined the detection result of the moving object with the decoded image of FIG.

Embodiment 1 FIG.
FIG. 1 is a block diagram illustrating a main part of the moving object detection device according to the first embodiment.
The variable length decoding unit 11 is a processing circuit that receives, as an input, a video bitstream including a frame encoded using motion compensated prediction, and decodes it according to a method defined by a corresponding encoding method. The variable length decoding unit 11 decodes and outputs information used for detecting a moving object including at least a motion vector among information included in the video bitstream.

  The voting value calculation unit 12 is a processing circuit that receives the output of the variable length decoding unit 11 and calculates a predetermined value corresponding to the reliability in moving object detection of the motion vector for each motion vector. Hereinafter, this value is referred to as “voting value”.

  The cumulative vote value storage unit 13 is a storage element that stores a two-dimensional array. For example, this two-dimensional array is the same size as each frame included in the video bitstream. Hereinafter, this two-dimensional array is referred to as a “voting surface”. The cumulative vote value storage unit 13 stores the cumulative value of the vote value for the motion vector indicating the coordinates for each coordinate of the voting plane. Hereinafter, this cumulative value is referred to as “cumulative vote value”.

  The voting unit 14 includes the motion vector output from the variable length decoding unit 11, the voting value of the motion vector calculated by the voting value calculating unit 12, and the motion vector stored in the cumulative voting value storage unit 13. This is a processing circuit that receives the cumulative vote value of the indicated coordinate, adds the vote value to the cumulative vote value, and stores it again in the cumulative vote value storage unit 13. Hereinafter, the process in which the voting unit 14 adds the vote values is referred to as “voting”.

  The moving object detection unit 15 is a processing circuit that receives a cumulative vote value from the cumulative vote value storage unit 13, detects a moving object in a moving object detection target frame by performing threshold processing, and outputs the result.

  The variable length decoding unit 11, the vote value calculation unit 12, the cumulative vote value storage unit 13, the vote unit 14, and the moving object detection unit 15 constitute a moving object detection device 100.

Here, the international standard video encoding method ITU-T H.264. H.264 and H.H. A frame reference structure in motion compensation prediction generally used in H.265 and the like will be described.
Consider encoding the nine frames shown in FIG. The numbers shown in each frame in FIG. 2 are frame numbers, and frames 1 to 8 are frames for motion compensation prediction. Frame 0 is the very first frame of the video, and only intra prediction is performed without performing motion compensation prediction. H. H.264 and H.H. In the case of H.265, etc., 0, 1, 2, etc. can be encoded in the order of frame numbers, but the compression performance is improved by changing the order of the frames and hierarchizing the reference structure as shown in FIG. In the example of FIG. 2, first, frame 0 is encoded, then frame 8 is encoded, and then encoded in the order of 4, 2, 6, 1, 3, 5, and 7. In addition, the arrow shown in FIG. 2 represents a reference relationship. That is, when encoding frame 8, a motion vector from frame 8 to frame 0 is obtained for each block to be encoded with reference to frame 0. Further, when encoding frame 4, reference is made to frame 0 and frame 8. Whether to refer to frame 0, frame 8, or both can be switched in units of blocks at the time of encoding. In FIG. 2, the reference relationship is always from the top to the bottom of the hierarchy. That is, a frame higher than the encoding target frame is not referred to.

  In the reference structure shown in FIG. 2, for example, in frame 4, the motion vector from frame 2, the motion vector from frame 3, the motion vector from frame 5, and the motion vector from frame 6 are obtained. Motion vectors from a plurality of frames can be used for one frame. In the first embodiment, highly accurate moving object detection is performed by using motion vectors from a plurality of frames integrated by voting.

  It should be noted that the reference structure shown in FIG. 2 is similarly repeated in subsequent frames with this as a unit. This is shown in FIG. Therefore, although description will be given here using frames 0 to 8 as an example, the same operation may be simply repeated after frame 8. Further, the first embodiment is not limited to the reference structure shown in FIG. 2, and can be applied to any reference structure.

  In frames 2 to 6 shown in FIG. 2, it is assumed that the moving object (showing the subject car in the example of FIG. 3) moves from left to right as shown in FIG. At this time, if there is a coding block at the position of the moving object as shown in FIG. 4, the motion vectors from each of the frames 2, 3, 5 and 6 to the frame 4 are consistently frames when the reliability is high. 4 points to almost the same coordinates. On the other hand, since the motion vector with low reliability generated in noise is random and inconsistent, it indicates various coordinates of the frame 4. Therefore, in the first embodiment, a reference based on a motion vector is regarded as a vote for the coordinates, and a region having a high vote value as a result of voting based on a plurality of motion vectors is detected as a moving object.

Next, the operation of the moving object detection device 100 will be described with reference to the flowchart of FIG.
First, the variable length decoding unit 11 receives a video bitstream including a frame encoded using motion compensated prediction as an input, and performs decoding according to a method defined by a corresponding encoding method (FIG. 17). Step ST100). At this time, the variable length decoding unit 11 may decode all of the bit stream or only a part thereof. When only a part is decoded, only the information used in the vote value calculation unit 12 and the vote unit 14 is decoded.

  Voting is performed by preparing a voting surface having the same size as each frame as shown in FIG. The accumulated voting value of the coordinates (x, y) of the frame i is stored in the coordinates (x, y) of the voting plane i. Here, i is a frame number. The voting plane is stored in the cumulative voting value storage unit 13 in a state in which all coordinate values are initialized with a value such as 0 in advance. In the example of FIG. 5, since all coordinate values of all frames are 0, all frames are expressed in black.

  Here, the voting surface is described as having the same size as each frame, but the voting surface is not necessarily the same size. For example, by making the size of the voting surface smaller than the size of the frame, it is possible to save the memory size required for the cumulative voting value storage unit 13 and shorten the processing time required for voting. is there.

  The variable length decoding unit 11 first decodes the frame 0, but here the frame 0 is the first encoded frame, and there is no other frame that can be referred to in motion compensation prediction. Is encoded in the screen and no motion vector exists. That is, frame 0 is an I picture. Therefore, the voting value calculation unit 12 first receives the motion vectors of all the encoded blocks included in the frame 8 decoded next to the frame 0 from the variable length decoding unit 11, and calculates the voting value for each motion vector ( Step ST101 in FIG. 17).

  The vote value may always be a fixed constant value, or may be changed according to each motion vector based on another index. When a different voting value is set for each motion vector, the voting value is determined according to, for example, the size of the encoded block having the motion vector. International standard video encoding system ITU-T H.264. In H.265, the encoded block can be recursively divided into quadtrees at the time of encoding, and the block size can be changed according to the content of the frame from 64 × 64 pixels to 8 × 8 pixels. At this time, as shown in FIG. 6, in the moving object region, the block size is reduced in order to increase the accuracy of the motion vector, and in the background region where there is no motion, the block size is likely to be increased. Therefore, it can be said that the smaller the size of a coding block having a motion vector, the higher the possibility that the motion vector belongs to a moving object (high reliability in moving object detection).

式（１）において、符号化ブロックの横幅の最大値をＬｘ、縦幅の最大値をＬｙとしている。ＬｘとＬｙの値は例えば６４画素である。 Therefore, the voting value calculation unit 12 also receives the block size from the variable length decoding unit 11 together with the motion vector, and uses this to determine the voting value of the motion vector. Here, assuming that the coordinates of the voting plane corresponding to the coordinates indicated by a certain motion vector are (m, n), the horizontal width of the block is Bx, and the vertical width is By, the motion vector with respect to the voting plane coordinates (x, y) The vote value v is determined as in the following formula (1).

In equation (1), the maximum horizontal width of the encoded block is Lx, and the maximum vertical width is Ly. The value of Lx and Ly is, for example, 64 pixels.

  By determining the motion vector voting value based on the block size in this way, the motion vector voting value of a small size block that is likely to be a moving object is large, and the motion vector voting value of a large block is large. Therefore, the accuracy of moving object detection based on motion vector voting can be increased.

  Since the variable length decoding unit 11 can decode various parameters in addition to the block size from the bitstream, the voting value calculation unit 12 receives these parameters to receive parameters other than the block size and those parameters. The voting value may be determined based on the combination.

式（２）において、ｖ´は補正後の投票値、σ_ｘ ^２とσ_ｙ ^２はそれぞれ水平方向と垂直方向の分散であり、例えば次式（３）のようにしてブロックサイズが小さいほど分散を小さく、ブロックサイズが大きいほど分散を大きくする。

式（３）において、αとβは例えば０．２５などの定数である。 Furthermore, it is considered that the closer to the center of the block in the coding block, the higher the possibility that the moving object is, and the vote value v is corrected using a predetermined weighting function. A Gaussian function is well known as a weighting function whose value decreases from the center toward the outside. The correction of the vote value v by a Gaussian function is given by the following equation (2).

In equation (2), v ′ is the voted value after correction, and σ _x ² and σ _y ² are the variances in the horizontal and vertical directions, respectively. For example, as shown in the following equation (3), the smaller the block size, the more the variance And increase the variance as the block size increases.

In Expression (3), α and β are constants such as 0.25.

  FIG. 7 shows how the vote value is corrected with a Gaussian function according to the block size. In addition, FIG. 8 shows the results of actual voting with block sizes of 8 × 8, 16 × 16, 32 × 32, and 64 × 64.

  By correcting the voting value of the motion vector with the weighting function in this way, the voting value at the block center that is likely to be a moving object can be increased, and the voting value can be decreased as the distance from the center increases. The accuracy of moving object detection based on vector voting can be improved.

  For example, the voting may be performed only for one point such as the center of the block, but the voting reliability can be improved by voting on a region having a certain area instead of the point. Since the motion vector may be shifted by several pixels due to an error, voting on the area as shown in FIG. 8 can reduce the influence of the motion vector shift and obtain a more stable voting result. Can do.

  In this way, the voting value calculation unit 12 calculates voting values for the motion vectors of all the encoded blocks included in the frame 8 and outputs the result to the voting unit 14.

  The voting unit 14 receives the motion vector decoded by the variable length decoding unit 11 and the voting value of the motion vector calculated by the voting value calculating unit 12, reads the voting surface from the cumulative voting value storage unit 13, and performs voting. . Since each motion vector of the frame 8 is calculated using the frame 0 as a reference frame, the voting unit 14 reads the voting plane 0 corresponding to the frame 0 from the cumulative vote value storage unit 13 (step ST102 in FIG. 17), and is variable. The voting value received from the voting value calculation unit 12 is voted to the coordinates indicated by the motion vector received from the long decoding unit 11. That is, the current voting value is added to the cumulative voting value of the coordinates on the voting plane 0. This is shown in FIG. FIG. 9 shows the result of voting on voting plane 0 based on each of the motion vectors of the three encoded blocks in frame 8. After the voting is completed, the voting unit 14 stores the voting surface 0 with the accumulated voting value updated in the accumulated voting value storage unit 13 (step ST103 in FIG. 17).

  Here, when voting, motion vectors that can be determined to have low reliability may be excluded from voting. The reliability of the motion vector can be determined based on the interval between the encoding target frame and the reference frame, for example. When the interval between frames for voting is large, the accuracy of the motion vector is lowered. Therefore, a threshold (hereinafter referred to as “interval threshold”) is set for the frame interval, and voting is performed when the frame interval exceeds the interval threshold Do not do. The frame interval may be obtained simply as the difference between the frame numbers of the two frames, or may be obtained as an actual time interval using the frame rate when the frame rate of the video is known.

  Further, the reliability of the motion vector may be determined using parameters obtained from the variable length decoding unit 11. For example, when a DCT (Discrete Cosine Transform) coefficient in the block is received from the variable length decoding unit 11 and the energy exceeds a threshold (hereinafter referred to as “energy threshold”), it is determined that the reliability of the motion vector is low. You may choose not to vote. Here, since the energy of the DCT coefficient corresponds to the magnitude of the prediction error, it is considered that the block with a large DCT coefficient energy has low motion compensated prediction accuracy and low motion vector reliability.

  By determining the reliability of motion vectors in this way and not voting using motion vectors with low reliability, the effect of motion vectors with low reliability can be reduced, and more accurate moving object detection can be performed. Is possible.

なお、グローバルモーションに関する情報がビットストリームに含まれている場合は、これを可変長復号部１１から受け取って利用してもよい。カメラ運動およびカメラパラメータの変化によるグローバルモーションを補正して投票を行うことで、より高精度な移動物体検出を行うことが可能となる。 Also, the video is not necessarily captured with the camera fixed and without changing camera parameters such as the focal length. Changes in camera motion and camera parameters during filming produce motion vectors throughout the video (so-called “global motion”). Therefore, in order to detect a moving object with high accuracy, it is necessary to estimate a global motion and correct a motion vector as necessary. If the motion vector of a certain encoded block obtained from the variable length decoding unit 11 is (MVx, MVy) and the global motion estimated in the encoded block is (GMVx, GMVy), the corrected motion vector (MVx ′, MVy ′) is given by the following equation (4).

In addition, when the information regarding global motion is included in the bit stream, it may be received from the variable length decoding unit 11 and used. By correcting the global motion due to camera motion and camera parameter changes and voting, it becomes possible to detect moving objects with higher accuracy.

  Next, when the frame 8 is not a moving object detection target frame (when NO is determined in step ST104 in FIG. 17), the voting value calculation unit 12 determines all the codes included in the frame 4 to be decoded next to the frame 8. The motion vector of the generalized block is received from the variable length decoding unit 11, and the voting value of each motion vector is obtained in the same manner as in the case of frame 8 (steps ST105 and ST101 in FIG. 17).

  Since each motion vector of frame 4 is calculated by using either one or both of frame 0 and frame 8 as a reference frame, voting unit 14 votes from frame voting value storage unit 13 corresponding to frame 0 and frame 8. The plane 0 and the voting plane 8 are read (step ST102 in FIG. 17), and the voting value received from the voting value calculation unit 12 is voted to the coordinates indicated by the motion vector received from the variable length decoding unit 11. At this time, the voting value of the motion vector referring to the frame 0 is to the voting plane 0, the voting value of the motion vector referring to the frame 8 is to the voting plane 8, and the voting value of the motion vector referring to both Vote for both voting planes 0 and 8. This is shown in FIG. After the voting is completed, the voting unit 14 stores the voting surface 0 and the voting surface 8 with the accumulated voting value updated in the cumulative voting value storage unit 13 (step ST103 in FIG. 17).

  Thereafter, when the voting for the moving object detection target frame is incomplete (NO in step ST104 of FIG. 17), the same processing as that of frames 2, 6, 1, 3, 5, and 7 is repeated. (Step ST105 in FIG. 17). FIG. 11 shows how voting is performed in frame 2. In addition, although operation | movement which performs voting in the encoding order, ie, the decoding order, was demonstrated here, it does not necessarily need to be performed in this order. For example, all the frames from frame 0 to frame 8 may be decoded by the variable length decoding unit 11, and then voting may be performed according to the display order of frames 1, 2, 3, and the like.

  Further, the voting process in the reverse direction may be repeated by inverting the sign of each motion vector. That is, after the voting for the frames 0 to 8 is completed, the voting unit 14 reads the voting plane 0 from the cumulative vote value storage unit 13. At this time, since the frames referring to the frame 0 are the frames 1, 2, 4, and 8, the voting unit 14 receives the motion vectors of the frames 1, 2, 4, and 8 from the variable length decoding unit 11. Further, the voting planes 0, 1, 2, 4, and 8 are read from the cumulative voting value storage unit 13, and the signs of the motion vectors of the frames 1, 2, 4, and 8 are inverted to obtain the cumulative voting values of the voting plane 0. Vote on voting surfaces 1, 2, 4, and 8, respectively. FIG. 13 shows a state in which voting in the reverse direction is performed from frame 0 to frame 4. In FIG. 13, the coordinates of the motion vector from frame 4 to frame 0 are reversed, and the vote on voting plane 4 is performed using the cumulative voting value of voting plane 0. After the voting in the reverse direction is completed, the voting unit 14 stores the voting surfaces 1, 2, 4, and 8 in which the cumulative voting value is updated in the cumulative voting value storage unit 13.

  Thus, by voting in the reverse direction once the voting is completed, the reliability of the frames 1, 2, 3, 5, 6, 7, etc. High voting value can be given.

  When the voting of the moving object detection target frame is completed (when YES in step ST104 in FIG. 17), the moving object detection unit 15 corresponds to the moving object detection target frame from the cumulative vote value storage unit 13. The voting surface is read out and threshold processing for moving object detection is performed (step ST106 in FIG. 17). For example, it is assumed that the voting surface read from the cumulative vote value storage unit 13 is as shown in FIG. Here, a region having a higher final vote value is more likely to be a moving object region. Therefore, the moving object detection unit 15 compares the cumulative vote value stored in each coordinate of the voting surface with a threshold (hereinafter referred to as “voting threshold”), and selects only a region where the cumulative vote value is larger than the vote threshold. In this way, moving object detection is performed. FIG. 15 shows the result of the threshold processing for FIG. In FIG. 15, the area shown in white is a moving object detected.

The voting threshold value may be stored in advance in a memory inside the moving object detection unit 15, or may be calculated and obtained inside or outside the moving object detection unit 15. In the hierarchical reference structure shown in FIG. 2, the voting threshold value may be changed according to the hierarchy because the cumulative vote value differs depending on the hierarchy. For example, if the voting threshold value held in the memory inside the moving object detection unit 15 is Th and the hierarchy to which the moving object detection target frame belongs is H (H = 0, 1, 2, 3), it is conceivable to use a voting threshold as Th / ^{2 H.} By adaptively changing the voting threshold according to the hierarchy or video characteristics in the reference structure, it becomes possible to detect a moving object with higher accuracy.

  For example, the moving object detection result may be output as a binary mask image in which the value of the moving object area is 1 and the values of the other areas are 0 as shown in FIG. 15, or as shown in FIG. Label the moving object area to distinguish each area, define a rectangle (so-called “bounding box”) that encloses each area, and set the coordinates (for example, upper left coordinates) and size (for example, the length of two sides) of the rectangle You may output (step ST107 of FIG. 17).

  As described above, the moving object detection apparatus 100 according to Embodiment 1 has a variable length that decodes a motion vector between an encoding target frame of a coding block and a reference frame from a video bitstream encoded by motion compensation prediction. The voting value calculation unit 12 that calculates the voting value for each of the plurality of motion vectors, the voting value calculated by the voting value calculation unit 12 is used as the motion of the voting plane corresponding to the reference frame of each motion vector. The voting unit 14 that votes for the coordinates pointed to by the vector, the cumulative voting value storage unit 13 that stores the cumulative voting value that is a value obtained by accumulating the voting values voted by the voting unit 14 for each coordinate of the voting plane, and the cumulative voting value And a moving object detection unit 15 that detects a moving object included in the video of the video bitstream. In moving object detection using motion vectors obtained from encoded video, moving object detection is performed by using a plurality of motion vectors in a plurality of frames in an integrated manner by voting processing using a reference relation of motion vectors. Since it is configured, it is possible to perform highly accurate moving object detection by reducing the influence of motion vectors with low reliability.

  The voting value calculation unit 12 calculates a voting value for the motion vector using the index decoded by the variable length decoding unit 11 from the video bitstream. Specifically, for example, the voting value calculation unit 12 calculates a larger voting value as the block size of an encoded block having a motion vector is smaller. By determining the motion vector voting value based on the block size, the motion vector voting value of a small-sized block that is likely to be a moving object is large, and the motion vector voting value of a large block is small. Therefore, the accuracy of moving object detection based on motion vector voting can be improved.

  The voting value calculation unit 12 corrects the voting value for the calculated motion vector using a weighting function. Specifically, for example, the voting value calculation unit 12 uses a Gaussian function to increase the voting value for a motion vector indicating a coordinate closer to the center of the encoded block. By correcting the voting value of the motion vector with a weighting function, the voting value at the center of the block that is likely to be a moving object can be increased, and the voting value can be decreased as the distance from the center increases. The accuracy of moving object detection based on the above can be improved.

  Moreover, the coordinate of a voting surface is a coordinate which shows the area | region containing a some pixel. Since the motion vector may be shifted by several pixels due to an error, voting on the area as shown in FIG. 8 can reduce the influence of the motion vector shift and obtain a more stable voting result. Can do.

  The voting unit 14 determines the reliability of each motion vector, and excludes some motion vectors from the vote according to the reliability. Specifically, for example, the voting unit 14 compares the interval between the encoding target frame of each motion vector and the reference frame with the interval threshold value, and excludes the motion vector whose interval exceeds the interval threshold value from the vote. Alternatively, the voting unit 14 compares the energy of the discrete cosine transform coefficient of the coding block having each motion vector with the energy threshold, and excludes the motion vector whose energy exceeds the energy threshold from the vote. By judging the reliability of motion vectors and not voting using motion vectors with low reliability, it is possible to reduce the influence of motion vectors with low reliability and perform more accurate moving object detection Become.

  The voting unit 14 performs voting by correcting the global motion included in the motion vector. By correcting the global motion due to camera motion and camera parameter changes and voting, it becomes possible to detect moving objects with higher accuracy.

  The voting unit 14 votes the voting value calculated by the voting value calculating unit 12 to the coordinates indicated by the motion vector of the voting surface corresponding to the reference frame of each motion vector, and then the voting surface corresponding to the reference frame. The cumulative voting value of each coordinate is voted to the coordinates of the voting plane pointed to by the vector obtained by inverting the sign of the motion vector. By voting in the reverse direction once voting is completed, highly reliable voting values are given to frames 1, 2, 3, 5, 6, 7, etc., which are rarely referred to by other frames. Can be given.

  In addition, the moving object detection unit 15 determines the region where the moving object exists in the moving object detection target frame by comparing the cumulative vote value of each coordinate with the vote threshold. The frames included in the video bitstream have a hierarchical reference structure, and the moving object detection unit 15 sets a voting threshold according to the hierarchy of the moving object detection target frame. By adaptively changing the voting threshold according to the hierarchy in the reference structure, it is possible to detect a moving object with higher accuracy.

Embodiment 2. FIG.
FIG. 18 is a block diagram showing a main part of the video decoding apparatus according to the second embodiment. With reference to FIG. 18, video decoding apparatus 300 having moving object detection apparatus 100 similar to that in the first embodiment will be described.

  The variable length decoding unit 11 is a processing circuit that receives, as an input, a video bitstream including a frame encoded using motion compensated prediction, and decodes it according to a method defined by a corresponding encoding method.

  The inverse quantization / inverse transform unit 21 is a processing circuit that receives the DCT coefficient decoded by the variable length decoding unit 11, dequantizes and inverse transforms the DCT coefficient, and outputs a prediction residual.

  The addition unit 22 receives the prediction residual output from the inverse quantization / inverse conversion unit 21 and the prediction image output from the intra prediction unit 23 or the motion compensation prediction unit 24, and adds them to obtain a decoded image. An arithmetic unit that generates and outputs.

  The intra prediction memory 25 is a storage element that stores the decoded image output from the adding unit 22.

  The loop filter unit 26 is a processing circuit that receives the decoded image output from the adder unit 22, performs a loop filter process, and outputs the result.

  The motion compensation prediction memory 27 is a storage element that receives the decoded image that has been subjected to the loop filter processing by the loop filter unit 26 and stores it as a reference image.

  The intra prediction unit 23 is a processing circuit that reads pixel data of a decoded image stored in the intra prediction memory 25, performs intra prediction, generates an intra predicted image, and outputs the intra predicted image.

  The motion compensation prediction unit 24 is a processing circuit that reads a reference image stored in the motion compensation prediction memory 27, performs motion compensation prediction, and generates and outputs a motion compensation prediction image.

  The voting value calculation unit 12, the cumulative voting value storage unit 13, the voting unit 14, and the moving object detection unit 15 are the same blocks as those in the first embodiment, and thus description thereof is omitted.

  The variable length decoding unit 11, the vote value calculation unit 12, the cumulative vote value storage unit 13, the vote unit 14, and the moving object detection unit 15 constitute a moving object detection device 100. A variable length decoding unit 11, an inverse quantization / inverse transform unit 21, an adding unit 22, an intra prediction unit 23, a motion compensation prediction unit 24, an intra prediction memory 25, a loop filter unit 26, and a motion compensation prediction memory 27 generate a decoded image. The unit 200 is configured. The moving object detection apparatus 100 and the decoded image generation unit 200 constitute a video decoding apparatus 300.

Next, the operation of the video decoding device 300 will be described.
Note that the decoded image generation unit 200 operates in the same manner as a general video decoding device, and thus description thereof is omitted. In the moving object detection apparatus 100, the variable length decoding unit 11, the vote value calculation unit 12, the cumulative vote value storage unit 13, and the vote unit 14 operate in the same manner as in the first embodiment, and thus the description thereof is omitted.

  When the voting of the moving object detection target frame is completed, the moving object detection unit 15 reads out the voting plane corresponding to the moving object detection target frame from the cumulative vote value storage unit 13 and performs threshold processing for moving object detection. I do. The moving object detection unit 15 compares the accumulated voting value stored in each coordinate of the voting surface with the voting threshold, and performs moving object detection by selecting only a region where the accumulated voting value is larger than the voting threshold.

The voting threshold value may be stored in advance in a memory inside the moving object detection unit 15, or may be calculated and obtained inside or outside the moving object detection unit 15. In the hierarchical reference structure shown in FIG. 2, the voting threshold value may be changed according to the hierarchy because the cumulative vote value differs depending on the hierarchy. For example, if the voting threshold value held in the memory inside the moving object detection unit 15 is Th and the hierarchy to which the moving object detection target frame belongs is H (H = 0, 1, 2, 3), it is conceivable to use a voting threshold as Th / ^{2 H.} By adaptively changing the voting threshold according to the hierarchy or video characteristics in the reference structure, it becomes possible to detect a moving object with higher accuracy.

  For example, the moving object detection result may be output as a binary mask image in which the value of the moving object area is 1 and the values of the other areas are 0 as shown in FIG. 15, or as shown in FIG. The moving object area may be labeled and distinguished for each area, a bounding box that includes each area may be defined, and output as rectangular coordinates (for example, upper left coordinates) and sizes (for example, the length of two sides).

  Further, the moving object detection unit 15 may receive the decoded image from the decoded image generation unit 200, and may synthesize and output the decoded image and the moving object detection result. For example, if the input decoded image is as shown in FIG. 19, the moving object detection unit 15 paints black and detects the pixel value of the area other than the moving object area detected in FIG. 19. The image shown in FIG. 20 may be generated and output without changing the pixel value of the moving object region.

  Alternatively, as shown in FIG. 21, moving object regions may be labeled and distinguished for each region, and bounding boxes containing each region may be defined and combined with a decoded image for output.

  As described above, the video decoding device 300 according to the second embodiment includes the moving object detection device 100 similar to that of the first embodiment and the decoded image generation unit 200 that generates a decoded image from the video bitstream. In moving object detection using motion vectors obtained from encoded video, moving object detection is performed by using a plurality of motion vectors in a plurality of frames in an integrated manner by voting processing using a reference relation of motion vectors. Since it is configured, it is possible to perform highly accurate moving object detection by reducing the influence of motion vectors with low reliability.

  Further, the moving object detection unit 15 synthesizes the detection result of the moving object with the decoded image generated by the decoded image generation unit 200 and outputs the result to the outside. By synthesizing and outputting the decoded image obtained by decoding the bitstream and the moving object detection result, the moving object detection result can be presented in a format that is visually understandable to the user.

  In the present invention, within the scope of the invention, any combination of the embodiments, or any modification of any component in each embodiment, or omission of any component in each embodiment is possible. .

  DESCRIPTION OF SYMBOLS 11 Variable length decoding part, 12 Vote value calculation part, 13 Cumulative vote value storage part, 14 Voting part, 15 Moving object detection part, 21 Inverse quantization / inverse conversion part, 22 Adder part, 23 Intra prediction part, 24 Motion compensation Prediction unit, 25 intra prediction memory, 26 loop filter unit, 27 motion compensation prediction memory, 100 moving object detection device, 200 decoded image generation unit, 300 video decoding device.

Claims (16)

A variable length decoding unit that decodes a motion vector between an encoding target frame of a coding block and a reference frame from a video bitstream encoded by motion compensation prediction;
A vote value calculator for calculating a vote value for each of the plurality of motion vectors;
A voting unit for voting the voting value calculated by the voting value calculating unit to coordinates indicated by the motion vector of a voting surface corresponding to a reference frame of each of the motion vectors;
An accumulated voting value storage unit that stores an accumulated voting value that is a value obtained by accumulating the voting value voted by the voting unit for each coordinate of the voting surface;
A moving object detection unit that detects a moving object included in the video of the video bitstream using the cumulative vote value ;
The voting value calculation unit calculates the voting value for the motion vector using an index decoded from the video bitstream by the variable length decoding unit, and a block size of an encoded block having the motion vector The smaller the is, the larger the vote value is calculated.
A moving object detection device characterized by that .   The moving object detection device according to claim 1, wherein the vote value calculation unit corrects the vote value for the calculated motion vector using a weighting function. 3. The moving object detection device according to claim 2, wherein the voting value calculation unit increases the voting value by using a Gaussian function as the motion vector indicating a coordinate closer to the center of the encoded block.   The moving object detection apparatus according to claim 1, wherein the coordinates of the voting surface are coordinates indicating a region including a plurality of pixels.   The moving object detection device according to claim 1, wherein the voting unit determines the reliability of each of the motion vectors and excludes some of the motion vectors from the voting according to the reliability. The voting unit compares an interval between an encoding target frame of each of the motion vectors and a reference frame with an interval threshold, and excludes the motion vector whose interval exceeds the interval threshold from voting. Item 6. The moving object detection device according to Item 5 . The voting unit compares the energy of a discrete cosine transform coefficient of an encoded block having each of the motion vectors with an energy threshold, and excludes the motion vector whose energy exceeds the energy threshold from voting. The moving object detection device according to claim 5 .   The moving object detection device according to claim 1, wherein the voting unit performs voting by correcting global motion included in the motion vector.   The voting unit, after voting the voting value calculated by the voting value calculating unit to the coordinates indicated by the motion vector of the voting surface corresponding to the reference frame of each motion vector, 2. The moving object detection apparatus according to claim 1, wherein the cumulative voting value of each coordinate of the voting surface is voted to the coordinate of the voting surface indicated by a vector obtained by inverting the sign of the motion vector.   The said moving object detection part determines the area | region where the said moving object exists in a moving object detection object flame | frame by comparing the said accumulation vote value of each coordinate with a vote threshold value. Moving object detection device. Frames included in the video bitstream have a hierarchical reference structure;
The moving object detection device according to claim 10, wherein the moving object detection unit sets the voting threshold according to a hierarchy of the moving object detection target frame. The moving object detection device according to claim 10, wherein the moving object detection unit outputs a binary mask image indicating a region where the moving object exists to the outside. The moving object detection device according to claim 10 , wherein the moving object detection unit outputs the coordinates and size of a bounding box including an area where the moving object exists to the outside. A moving object detection device according to claim 1;
A decoded image generation unit for generating a decoded image from the video bitstream;
A video decoding device comprising: 15. The video decoding device according to claim 14, wherein the moving object detection unit synthesizes the detection result of the moving object with the decoded image generated by the decoded image generation unit and outputs the result to the outside. The variable length decoding unit decodes the motion vector between the encoding target frame of the encoding block and the reference frame from the video bitstream encoded by the motion compensation prediction,
A voting value calculation unit calculates a voting value for each of the plurality of motion vectors,
The voting unit votes the voting value calculated by the voting value calculating unit to the coordinates indicated by the motion vector of the voting surface corresponding to the reference frame of each of the motion vectors,
A cumulative voting value storage unit stores a cumulative voting value that is a value obtained by accumulating the voting value voted by the voting unit for each coordinate of the voting plane;
In the moving object detection method, the moving object detection unit detects the moving object included in the video of the video bitstream using the cumulative vote value .
The voting value calculation unit calculates the voting value for the motion vector using an index decoded from the video bitstream by the variable length decoding unit, and a block size of an encoded block having the motion vector The moving object detection method is characterized in that the smaller the value is, the larger the vote value is calculated .

Images