JP2001283227A - Moving image analysis processor and method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving image analysis processor capable of efficiently analyzing a moving image by a filter processing by small amount of calculation. SOLUTION: The moving image analysis processor is provided with a moving image decoding part 100 at least to generate a reproduced image signal and encoding mode information by every unit area by decoding encoded data formed by compressing and encoding the moving image, a filter 113 to perform the filter processing of the reproduced image signal, an output image memory 114 to store the filter processed image signal as an analysis processing result and an area rewrite deciding part 111 to decide whether or not the contents of the output image memory by every unit area from the encoding mode information and the filter processing is performed only to the image signal of the unit area in which the contents of the output image memory 114 is judged to be rewritten by the area rewrite deciding part 111 in the filter 113.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image analyzing method and apparatus for analyzing moving images by filtering.

[0002]

2. Description of the Related Art Generally, filter processing such as edge extraction and binarization is used for image analysis processing, for example, processing for analyzing a specific object in an image. Conventionally, such a filtering process is realized by a pixel-by-pixel process.

On the other hand, since a moving image has a very large data amount, when storing and transmitting moving image data, an MP
It is often handled as data compressed by a moving picture coding method such as EG-1, MPEG-2, or MPEG-4. Therefore, when analyzing a moving image, after performing the moving image decoding, the filtering process is performed on the reproduced image signal.

[0004] However, even in this filter processing, since it is necessary to perform processing for each pixel, there is a problem that a large amount of calculation is required as compared with a still image. For example, the commonly used CCIR601 format (NTSC)
In the case of, a filter process must be performed on a total of 345,600 pixels of 720 horizontal pixels and 480 vertical pixels only with the luminance signal. Such a filtering process requires a very large amount of calculation as compared with the video decoding process. Therefore, when the filter processing is realized by software, it is difficult to realize the processing in real time, and even when realized by hardware, the circuit scale becomes large and the cost is large.

[0005]

As described above, in the moving image analysis processing according to the prior art, a large amount of calculation is required for the filter processing performed for each pixel. However, there is a problem in that the cost is increased when hardware is used.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a moving image analysis processing method and apparatus capable of effectively analyzing a moving image by a filtering process with a small amount of calculation.

[0007]

In order to solve the above-mentioned problems, according to one aspect of the present invention, moving image analysis processing is combined with moving image decoding to decode coded data obtained by compressing and coding a moving image. When the reproduced image signal obtained by the filtering is filtered, and the filtered image signal is stored in the output image memory as the analysis processing result, the encoding mode for each unit area obtained by decoding the encoded data It is characterized in that it is determined from the information whether or not the contents of the output image memory are to be rewritten for each unit area, and that only the image signal of the unit area determined to be rewritten among the reproduced image signals is subjected to filter processing.

In another aspect of the present invention, a video decoding process is combined with video decoding to filter a locally decoded video signal generated in the process of compression-coding a video,
When storing the filtered image signal as an analysis result in the output image memory, whether to rewrite the contents of the output image memory for each unit area from the encoding mode information for each unit area generated in the compression encoding process It is characterized in that it is determined whether or not the filter processing is performed only on the image signal of the unit area determined to be rewritten among the locally decoded image signals.

If the analysis processing of a moving image is performed by such a filter processing, in an area where the contents of the output image memory are not rewritten, the image signal obtained by the previous filter processing is directly used as the analysis processing result. It will be output to the output image memory. Therefore, the result is substantially the same as that obtained by performing a filtering process on the whole reproduced image signal obtained by moving image decoding or the locally decoded image signal generated at the time of moving image encoding over one frame. , The amount of calculation required for the filtering process can be reduced.

The filter according to the present invention performs, for example, edge extraction processing and binarization processing as filter processing. By performing such a filtering process in combination with video decoding or decoding, according to the present invention, it is possible to efficiently execute a video edge extraction process and a binarization process.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS [First Embodiment] FIG. 1 shows a configuration of a moving picture analyzing apparatus according to a first embodiment of the present invention. This moving picture analyzing apparatus is largely composed of a moving picture decoding unit 10
0 and the analysis filter unit 110. Hereinafter, the moving image decoding unit 100 and the analysis filter unit 1
10 will be described sequentially. The following description is based on MPE.
This is a case where the present invention is applied to a moving picture decoding apparatus based on the G scheme, and a unit area corresponds to a macroblock in the MPEG scheme.

(Regarding the moving picture decoding unit 100) The moving picture decoding unit 100 is, for example, a moving picture decoding apparatus based on the MPEG system, a so-called MPEG decoder. The encoded data obtained by compressing and encoding the moving image signal by the moving image encoding unit as described above is input via a transmission path or a storage system.

The input encoded data is once stored in the input buffer 101. The encoded data read from the input buffer 101 is demultiplexed by the demultiplexing unit 102 into one.
After being separated based on the syntax for each frame, it is input to the variable length decoding unit 103. Variable length decoding unit 103
Then, the quantized DCT coefficient information that has been subjected to the variable length coding,
Information of each syntax such as coding mode information and motion vector information is decoded for each macroblock. Note that a macroblock that is a unit area to be processed in the following description is referred to as a target macroblock.

If the coding mode of the macroblock of interest is INTRA (intra-frame coding) in the variable length decoding unit 103, the coding mode changeover switch according to the coding mode information output from the variable length decoding unit 103. 1
09 is turned off. In this case, the variable length coding unit 1
The quantized DCT coefficient information decoded in step 03 is inversely quantized in the inverse quantization unit 104 and inverse discrete cosine transformed in the IDCT unit 105 to generate a reproduced image signal. The reproduced image signal is stored as a reference image signal in the frame memory 107 via the adder 106, and is input to the switch 112 in the analysis filter 110.

On the other hand, if the coding mode of the macroblock of interest is INTER (inter-frame coding) and NOT_CODED (non-coding) in the variable-length decoding unit 103, the coding output from the variable-length decoding unit 103 The encoding mode changeover switch 109 is turned on according to the mode information. In this case, the quantized DCT coefficient information about the prediction error signal decoded by the variable length decoding unit 103 is inversely quantized by the inverse quantization unit 104, and further inversely discrete cosine transformed by the IDCT unit 105, A prediction error signal is generated.

Based on the motion vector information decoded by the variable-length decoding unit 103, the motion compensation unit 108 performs motion compensation on the reference image signal from the frame memory 107. And the prediction error signal from the IDCT unit 105 are added by the adder 106 to generate a reproduced image signal. This reproduced image signal is stored in the frame memory 107 as a reference image signal, and is input to the switch 112 in the analysis filter 110.

(Regarding the analysis filter unit 110)
In the analysis filter unit 110, the video decoding unit 10
The coding mode information output from the variable length coding unit 103 within 0 is input to the area rewriting determination unit 111, and the adder 1
The reproduced image signal output from 06 is input to the filter 113 via the switch 112.

The area rewriting judging section 111 judges whether each macroblock is coded (Coded) or not coded (Not Coded) based on the coding mode information from the variable length decoding section 103. . The switch 112 is controlled based on the determination result of the area rewriting determination unit 111. If the coding mode is Coded, the switch 112 is turned on to input a reproduced image signal to the filter 113. The image signal filtered by the filter 113 is written to the output image memory 114. If the encoding mode is Not Coded, the switch 112 is turned off, and the filter processing of the reproduced image signal and the rewriting of the output image memory 114 are not performed.
Is output to the next stage.

If the contents of the output image memory 114 cannot be rewritten by performing such analysis filter processing, the image signal that has been previously subjected to the filter processing by the filter 113 is output as it is. Therefore, the operation is equivalent to performing a filtering process on the reproduced image signal output from the video decoding unit 100 over the entire frame, and the amount of calculation for the filtering process can be reduced by the amount of the Not Coded macroblock. it can.

[Second Embodiment] FIG. 2 shows the configuration of a moving picture analysis processing apparatus according to a second embodiment of the present invention.
This moving image analysis processing device is largely composed of a moving image
0 and the analysis filter unit 220.

In FIG. 2, an input video signal input to a video encoding unit 200 is first divided into macroblocks by a blocking unit 201. The input moving image signal of each macroblock is input to a subtractor 202, where a difference from the predicted image signal is calculated to generate a prediction residual signal. Either the prediction residual signal or the input video signal from the blocking unit 201 is
DCT (discrete cosine transform) unit 20
4 performs a discrete cosine transform. The DCT coefficient data obtained by the DCT unit 204 is quantized by the quantization unit 205. The DCT coefficient data quantized by the quantization unit 205 is split into two, and one of the DCT coefficient data is subjected to variable-length encoding by the variable-length encoding unit 214.

The other of the DCT coefficient data quantized and bifurcated by the quantization unit 205 is supplied to an inverse quantization unit 206 and an ID
Quantization unit 2 by CT (inverse discrete cosine transformation) unit 207
05 and the processing reverse to the processing of the DCT unit 204 are sequentially performed, and then added to the predicted image signal input via the switch 211 by the adder 208 to generate a locally decoded image signal. The local decoded image signal is stored in the frame memory 2
09 is input to the motion compensation unit 210 from the frame memory 209. The motion compensation unit 210 generates a predicted image signal and sends necessary information to the encoding mode selection unit 212. Further, the locally decoded image signal is also input to a switch 222 in the analysis filter unit 210.

The coding mode selection unit 212 selects a macroblock for inter-frame coding and a macroblock for intra-frame coding based on the prediction information P from the motion compensation unit 210 for each macroblock. When performing intra-frame encoding (Indra encoding), the encoding mode selection switch information M is A, and the switch information S is A. When performing inter-frame coding (inter coding), the coding mode selection switch information M is set to B, and the switch information S is set to B. Encoding mode selection switch 203
Are switched based on the encoding mode selection switch information P, and the switch 211 is switched based on the switch information S.

Here, as the encoding mode, an intra encoding mode (INTRA), an inter encoding mode (IN
TER) and a non-coding mode (NOT_CODED), and these coding modes are associated with each macroblock. The INTRA macroblock is an image area to be intra-coded, the INTER macroblock is an image area to be inter-coded, NOT_CODE
The D macro block is an image area that does not require encoding.

The encoding control section 213 includes an encoding section 217
Based on the coding information in the (moving image coding unit 200 and before the multiplexing unit 215) and the buffer amount of the output buffer 216, the quantization step size and the like in the quantization unit 205 in the coding unit 217 are determined. Control parameters. The coded data coded by the variable-length coding unit 214 is multiplexed by the multiplexing unit 216, then the transmission rate is smoothed by the output buffer 217, and is transmitted to the transmission system or the storage system.

On the other hand, in the analysis filter unit 220,
The region rewriting determination unit 221 determines whether each macroblock is coded (Coded) or not coded (Not Coded) based on the coding mode information input from the variable length coding unit 214. . The switch 222 is controlled based on the determination result of the area rewrite determination unit 221. If the coding mode is Coded, the switch 222 is turned on and inputs the locally decoded image signal to the filter 223. The image signal filtered by the filter 223 is written to the output image memory 224. If the encoding mode is Not Coded, the switch 222 is turned off, the filtering process of the locally decoded image signal and the rewriting of the output image memory 224 are not performed, and the image signal held in the output image memory 224 is transferred to the next stage. Is output.

If the contents of the output image memory 224 are not rewritten by performing such an analysis filter process, the image signal that has been subjected to the filter process by the filter 223 is output as it is. Therefore, the operation is equivalent to performing the filtering process on the entire locally decoded image signal generated by the video decoding 200 over one frame, and the calculation amount of the filtering process can be reduced by the amount of the Not Coded macroblock. it can.

[Filter Process] Next, a specific example of the filter process by the filter 113 of FIG. 1 and the filter 223 of FIG. 2 will be described. FIG. 3 shows a specific processing flow of the filter processing. Here, m and n represent the vertical and horizontal macroblock addresses in the frame, respectively, and V_NMB and HNMB represent the number of vertical and horizontal macroblocks in the frame.

First, the coding mode information MODE is determined for each macroblock (step S103). Here, if MODE is INTRA (intra-frame coding) or INTER (inter-frame coding), it is determined as Coded, and a reproduced image signal or a locally decoded image signal of the macro block is filtered, and the filtered image signal is processed. Is written into the output image memory (step S10).
4). If the MODE is Not Coded, nothing is performed, and the process proceeds to the next macroblock.

Here, the filter processing in step S104 is a filter processing performed as an image processing such as a noise removal processing, an edge extraction processing, a binarization processing, etc. Then, any filtering process may be used.

In this embodiment, an example is shown in which an edge extraction process is performed as shown in FIG. 4 (step S201), and then a binarization process is performed (step S202).

For the edge extraction processing, for example, a spatial differential filter is used. i, j is the address of the pixel on the two-dimensional memory, P (j, j) is the reproduced image signal or the locally decoded image signal,
When B (i, j) is an image signal in the output image memory,
The horizontal and vertical edge extraction is as follows.

First, a 2 × 2 vertical primary spatial differential filter of the following equation (1) is used for horizontal edge extraction. F (i, j) =-P (i, j) + P (i, j + 1) -P (i + 1, j) + P (i + 1, j + 1) (1) FIG. This is a diagram of the filter coefficients of equation (1), and the upper left pixel is a reference pixel P (i, j).

For the vertical edge extraction, a 2 × 2 horizontal primary spatial differential filter of the following equation (2) is used. F (i, j) =-P (i, j) -P (i, j + 1) + P (i + 1, j) + P (i + 1, j + 1) (2) FIG. , (2), and the upper left pixel is the reference pixel P (i, j).

A Laplacian filter represented by the following equation (3) may be used as a filter for simultaneously performing horizontal, vertical, and diagonal edge extraction. F (i, j) =-P (i-1, j-1) -P (i-1, j) + P (i-1, j + 1) -P (i, j-1) + 8 * P ( i, j) −P (i, j + 1) −P (i + 1, j−1) −P (i + 1, j) −P (i + 1, j + 1) (3) FIG. , (3) in which the coefficients of the filter are represented in a diagram, and the central pixel is the reference pixel P (i, j).

The value of F (i, j) as a result of the above edge extraction filter processing is such that a region where the absolute value of F (i, j) is large is an edge portion, and a region near zero is a flat portion. It has the property of being. Therefore, in the binarization processing, a simple threshold function as shown in the following equation (4) is used to express 1 if the absolute value of the edge extraction result F (i, j) is larger than the threshold T and 0 if the absolute value is equal to or smaller than the threshold T. Can be.

[0037]

(Equation 1)

(Display Example) Next, an example of the result of the analysis processing according to the present embodiment on an actual image will be described with reference to FIGS. FIG. 7 is a display screen of a reproduced image signal. FIG. 8 shows coding mode information M for each macroblock.
ODE is INTRA or INTER, Coded
FIG. 3 is a diagram in which a macro block is represented in white and a Not Coded macro block is represented in black. Therefore, the portion where the contents of the output image memory are rewritten is only the white portion on the screen.

FIG. 9 is a flowchart showing the operation of Cod by the processing as shown in FIG.
An analysis result obtained by performing edge extraction processing in the horizontal direction and the vertical direction only on the portion of the ed macro block and binarizing the result is shown.

FIG. 10 shows the contents of the output image memory.
Since the previously processed result is stored at the position of the Not Coded macroblock in the output image memory, the result of binarizing the entire screen is obtained.

By effectively using the encoding mode information as described above, the processing can be partially performed without processing the reproduced image signal or the entire locally decoded image, so that the filter processing can be performed with a small amount of calculation.

In this embodiment, an example in which edge extraction processing is performed as filter processing and binarization processing is performed has been described. However, the present invention is also applicable to the case where filter processing generally used in other image processing is used. The invention is valid.

[0043]

As described above, according to the present invention, it is possible to effectively perform a filtering process with a small amount of calculation and perform a moving image analysis process. In the case of realization by hardware, the required circuit scale can be reduced to reduce the cost.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a moving image analysis processing device according to a first embodiment of the present invention.

FIG. 2 is a block diagram illustrating a configuration of a moving image analysis processing device according to a second embodiment of the present invention.

FIG. 3 is a flowchart illustrating a flow of a filtering process according to the present invention.

FIG. 4 shows 2 × 2 used for filtering in the present invention.
Showing the coefficients of the first-order spatial differential filter in the vertical direction

FIG. 5 shows 2 × 2 used for filtering in the present invention.
Showing the coefficients of the first-order spatial differential filter in the horizontal direction

FIG. 6 is a diagram showing coefficients of a Laplacian filter used for filter processing in the present invention.

FIG. 7 is a diagram showing a display example of a reproduced image signal or a decoded image signal according to the present invention.

FIG. 8 is a diagram showing a display example of an image of a portion of the output image memory in which rewriting has been performed in the present invention.

FIG. 9 is a diagram showing a portion for performing a filtering process on a screen according to the present invention.

FIG. 10 is a diagram showing a display example of the contents of an output image memory according to the present invention.

[Explanation of symbols]

 Reference Signs List 100: moving image decoding unit 101: input buffer 102: demultiplexing unit 103: variable length decoding unit 104: inverse quantization unit 105: IDCT unit 106: adder 107: frame memory 108: motion compensation unit 109: mode Changeover switch 110: Analysis filter unit 111: Area rewriting determination unit 112: Switch 113: Filter 114: Output image memory 201: Blocking unit 202: Subtractor 203: Mode selection switch 204: DCT unit

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5C059 KK00 KK15 LA00 MA00 MA05 MA23 MC11 MC38 ME01 NN01 TA68 TB07 TC03 UA05 UA11 UA31 5L096 AA02 AA06 BA18 EA43 FA06 GA06 GA55 HA03

Claims (6)

[Claims] 1. A moving image in which a reproduced image signal obtained by decoding encoded data obtained by compressing and encoding a moving image is filtered, and the filtered image signal is stored in an output image memory as an analysis result. In the analysis processing method, it is determined whether or not the content of the output image memory is to be rewritten for each unit area from encoding mode information for each unit area obtained by decoding the encoded data, A moving image analysis processing method, characterized in that filter processing is performed only on an image signal of a unit area determined to be rewritten. 2. A moving image analyzing method for filtering a locally decoded image signal generated in a process of compressing and encoding a moving image, and storing the filtered image signal as an analysis result in an output image memory. Judging whether or not to rewrite the contents of the output image memory for each unit area from the encoding mode information for each unit area generated in the compression encoding process, and judging to rewrite the local decoded image signal A moving image analysis processing method, wherein a filtering process is performed only on an image signal of a selected unit area. 3. The moving image analysis processing method according to claim 1, wherein at least one of an edge extraction process and a binarization process is performed as the filtering process. 4. A moving image decoding unit for decoding encoded data obtained by compressing and encoding a moving image to generate at least a reproduced image signal and encoding mode information for each unit area, and filtering the reproduced image signal. A filter, an output image memory for accumulating the filtered image signal as an analysis result, and an area rewriting determination unit for determining whether or not to rewrite the contents of the output image memory for each unit area from the encoding mode information Wherein the filter performs a filtering process only on an image signal of a unit area of the reproduced image signal which is determined by the area rewriting determination unit to rewrite the content of the output image memory. Analysis processing device. 5. A moving image encoding unit for compressing and encoding a moving image to output encoded data, and generating a locally decoded image signal and encoding mode information for each unit area in the compression encoding process. A filter for filtering the decoded image signal, an output image memory for storing the filtered image signal as an analysis result, and whether or not to rewrite the contents of the output image memory for each unit area from the encoding mode information And a filter that performs filter processing only on a signal of a unit area of the decoded image signal that is determined to rewrite the contents of the output image memory in the decoded image signal. A moving image analysis processing device characterized by the following. 6. The moving image analysis processing apparatus according to claim 3, wherein the filter performs at least one of an edge extraction process and a binarization process as the filter process.