JPH10304385A - Moving picture converter - Google Patents

Abstract

PROBLEM TO BE SOLVED: To attain high speed processing of an MPEG encoder by performing classification according to the magnitude of the movement of a macroblock before decoding DVC data, and thus reducing the processing of the MPEG encoder. SOLUTION: In a process for decoding inputted DVC data, and encoding the data into MPEG data, an area dividing part 113 is provided before encoding. A movement detecting means 106 checks movement for each macroblock, and outputs a rough motion vector. A macroblock classifying means 107 provides three labelings, that is, large movement, medium movement, and small movement to each macroblock according to the magnitude of the rough motion vector. Thus, the area dividing part 113 preliminarily performs classification at a high speed for each macroblock, and an encoding part 111 performs the detection of the motion vector only for the necessary macroblock from the classified result so that the processing can be reduced.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture conversion apparatus for performing a moving picture compression format conversion, and more particularly, to a moving picture conversion apparatus capable of reducing a calculation amount of motion vector detection to speed up a moving picture format conversion.

[0002]

2. Description of the Related Art In recent years, digitalization of telecommunications technology has been advanced, and plans for digitizing the current analog broadcasting system such as NTSC / PAL / SECAM are also in progress. In addition, as new broadcasting services applying digital technology, there are satellite broadcasting that provides multi-channel services, terrestrial broadcasting (ATV: Advanced Television) of high-definition broadcasting, and video-on-demand. However, when such moving images are communicated by digital signals, a common problem is the enormous amount of information. Moving image compression technology is indispensable to increase the frame rate for playback in real time and to suppress network traffic.

[0003] In order to efficiently compress moving image data having a large amount of information, a method of reducing the redundancy by using a correlation in a time axis direction or a space axis direction is used. A typical example is MPEG (Moving Picture Experts
Group). This is ISO-IEC / JTC1 / SC2 / W
Discussed at G11 and proposed as a standard draft, motion compensation coding, DCT (Discrete Cosine Transform)
A hybrid system combining coding and variable length (VLC) coding is employed. FIG. 15 shows a configuration diagram of a typical MPEG encoding apparatus.

The current frame is input to a motion vector detecting section and a subtractor. The motion vector detection unit divides the current frame into rectangular blocks, and detects a motion vector for each block with reference to past frames already stored in the frame memory. Then, a predicted image generated by shifting the past frame by a motion vector from the block of the current frame is input to the subtractor. The subtractor calculates the difference image between the current frame and the predicted image, and calculates the DCT
DCT and quantization are sequentially performed by means and quantization means. Then, the data is subjected to variable-length encoding by the variable-length encoding means, and then output to the outside. At this time, the motion vector detected by the motion vector detecting unit is also coded by the motion information coding unit and then output to the outside. The quantized data is input to the variable-length coding means and also to the inverse quantization means,
The difference image is restored by means. In the adder, the predicted image of the current frame and the difference image are added to reconstruct the current frame, and are stored in the frame memory in preparation for encoding of the next frame.

[0005] In this series of encoding processes, most of the time is spent for the operation by the motion vector detecting unit.
Research has been done to reduce or simplify this process. However, since this is performed equally for all macroblocks, the accuracy of motion vector detection is reduced. The motion vector is a macro block (16 × 1
(6 pixels) is the processing unit. A macroblock has three types (intra:
Use that block because the movement is intense. Non-MC coded: Simple inter-frame difference is performed because there is little movement. MC
(Coded: normal motion compensation). Therefore, a method described in Japanese Patent Application Laid-Open No. 4-181888 is known in which a macroblock is classified by placing a block determination circuit before detecting a motion vector, and processing is changed according to the result. However, in that method, when judging the movement,
Only the block at the position of the macroblock itself (corresponding to the motion vector 0) in the past frame is compared, the block is simply classified as having no motion, and motion detection is performed for those determined to have no motion. It is not done.

On the other hand, one of the important axes of the digital broadcasting communication technology described above is a digital VTR. Conventionally, uncompressed digital VTRs have been used as broadcasting station digital VTRs in broadcasting stations and the like. Digital VTRs have the advantages of high image quality and little degradation during dubbing. However, digitalization has a problem that recorded information is increased, and is not widely used because of its large size and high cost. However, recently, with the advancement of the band compression technology and the improvement of the recording density, a small home digital VTR has been developed. Here, DVC (Digital) standardized by the HD Digital VTR Council
Video Cassette) format is used. DVC
The data uses DCT for spatial compression, and then performs variable-length coding by Huffman coding, and the amount of information is very large because information is compressed without using temporal correlation. It has the features of high image quality and easy editing for each frame.

[0007] Computers are also becoming available for use in computers, and it is expected that they will be used more and more in the future. In systems such as video-on-demand, which exchange moving images via a network, the DVC can be used in real time, depending on the transmission capacity of the network.
It is difficult to send and receive data. At that time, the moving image is DV
It is effective to accumulate C data and convert the image to MPEG or the like when necessary from the viewpoint of effective use of disk resources. Conventionally, image conversion was also performed using dedicated hardware.However, due to image quality, cost, flexibility in various formats, installation time on computers, compatibility with computers, ease of distribution, etc., software It has been desired to do so using.

[0008]

However, there is a problem that a large amount of processing time is required to perform DVC data decoding and MPEG encoding on one side by software. The present invention has been made in view of the above circumstances, and has as its object to reduce the load involved in motion vector detection in MPEG encoding and improve the conversion speed of moving image conversion.
As a conventional example of the motion vector detection processing, a method of classifying macroblocks by placing a block determination circuit before motion vector detection in MPEG encoding and changing the processing according to the result is disclosed in JP-A-4-181888. Is known for. However, 1. There are only two types of classification by the block determination circuit (non-MC coded and other types), and among three types of blocks classified by MPEG, after performing motion prediction, an intra with no motion vector is not classified. 2. The determination of the presence or absence of the motion by the block determination circuit is based on the difference between the simple blocks, and has a problem in the accuracy of detecting the motion 0 vector.

As described above, since the classification of macroblocks is insufficient and the method of classification is not effective, there is little reduction in the processing and the problem of a decrease in the accuracy of motion vector detection.

[0010]

According to the present invention, in order to solve the above-mentioned problems, an input is performed by applying a DCT to each block.
The VC data and output are limited to MPEG data, and before decoding the DVC data, classification is performed according to the degree of motion of the macroblock to reduce MPEG encoding processing.

First, input means for receiving a digital image signal subjected to orthogonal transformation processing from the outside, a frame memory for storing an image, and block reading means for reading a block or macroblock from the frame memory as required. An area dividing unit controlling means for issuing a read request to the block reading means, a component extracting means for extracting only a DC component of a captured image, two low-resolution frame memories for storing extracted information, An area dividing unit comprising: motion detecting means for performing motion detection based on information; macroblock classifying means for labeling macroblocks according to the motion detection result by the motion detecting means; and label storing means for storing macroblock labels. , Decoding means, and a label of the label storage means An encoding processing unit for changing the encoding processing of the macro block in response, to provide a moving picture conversion apparatus and an output means for outputting the encoded signal to the outside. As described above, three types of labels are assigned to the macroblocks before the encoding process according to the degree of the motion, and the encoding process is changed according to the label, thereby reducing the load on the motion vector detection and performing high-speed moving image conversion.

Secondly, when the label storage means in the first step outputs a label indicating that the motion of the macroblock is small, the encoding processing unit takes a small motion vector search area to obtain a small motion vector search area. Provided is a moving image conversion device characterized by performing a detection process. Thereby, the accuracy of the motion vector detection in the encoding processing unit can be improved.

[0013] Thirdly, a moving picture conversion apparatus characterized in that the first area dividing section further comprises a resolution conversion means for performing a resolution conversion by interpolating an input image. I do. This allows
If the resolution of the low-resolution search area is too low for motion detection only by extraction by the component extraction means, the resolution of the search area can be increased by converting the data read from the low-resolution frame memory.

Fourthly, the component extracting means in the first aspect is adapted to extract a DC component and a low frequency component of the fetched image, and further, into the area dividing section, the low frequency component extracted by the component extracting means. Inverse DCT for frequency components
And an inverse DCT means for performing a moving image conversion. As a result, pattern matching can be performed using the pixel values of the actual image instead of the DCT data, so that the detection accuracy can be increased.

Fifthly, input means for receiving a digital image signal subjected to orthogonal transformation processing from the outside, a frame memory for storing an image, and a block for reading out a block or a macroblock from the frame memory as required. Reading means, area dividing section control means for issuing a read request to the block reading means, component extracting means for extracting only specific components decomposed by a basis orthogonal to each other from a captured image, and performing inverse orthogonal transformation on the extracted information. Inverse orthogonal transform means, two frame memories for storing information obtained by the inverse orthogonal transform means, a motion vector detecting means for performing motion detection based on information obtained by the inverse orthogonal transform means, Label macroblocks according to the size of the output motion vector An area dividing unit including a macro block classification unit and a label storage unit for storing a label of the macro block; a decoding unit; an encoding processing unit for changing an encoding process of the macro block according to the label of the label storage unit; And an output means for outputting the output signal to the outside.

By doing so, motion detection is performed without performing high-load processing such as full search, macroblocks are labeled, and encoding processing is changed according to labels to reduce the load on motion vector detection and reduce the speed. Is subjected to moving image conversion.

Sixth, an input means for inputting a digital image signal composed of macroblocks, which is a set of the blocks, which is subjected to an orthogonal transformation process for each block, and a frame memory for storing the digital image signals A block reading means for reading the block or the macroblock from the frame memory in response to a request, an area dividing unit controlling means for issuing a read request to the block reading means, a component extraction for extracting only a DC component from a captured image Means, two low-resolution frame memories for storing extracted information, motion detection means for performing motion detection based on the information in the low-resolution frame memory,
Macroblock classification means for labeling macroblocks based on the output of the motion detection means; motion vector restoration means for restoring a motion vector of a macroblock having motion based on the result of the motion estimation means; An area dividing unit including a label storage unit for storing a label, a decoding unit, a DCT unit, a quantization unit, a variable length encoding unit, an inverse quantization unit, an inverse DCT unit, a frame memory unit, an image differentiator, and an image addition unit. And an encoding unit comprising a motion vector detecting unit and a motion information encoding unit in which a search area changing unit is located, and an output unit for outputting an encoded signal to the outside. A vector detecting unit that limits a motion vector search area based on a motion detection result by a motion detecting unit in the area dividing unit; And to provide a video conversion apparatus for performing motion vector detection. As a result, classification is performed according to the degree of motion of the macroblock, a motion vector is detected only for a necessary macroblock from the classification result, and the load on the encoding process is reduced to perform high-speed video conversion.

Seventhly, an area dividing unit according to the fifth aspect is provided in place of the area dividing unit in the sixth aspect, and a motion vector detecting unit in the encoding processing unit is provided with a motion vector in the area dividing unit. There is provided a moving image conversion apparatus characterized in that a motion vector search area is limited by using a motion vector detected by a detecting means and a motion vector is detected. Thereby, the motion vector is detected at high speed.

Eighth, the encoding processing unit in the seventh embodiment does not perform any motion vector detection by omitting the motion vector detection unit in the encoding processing unit. Provided is a moving image conversion device characterized by using a detected motion vector. This eliminates the load associated with motion vector detection in MPEG encoding.

[0020]

Embodiments of the present invention will be described below with reference to the drawings.

(First Embodiment) FIG. 1 is a diagram showing a configuration of a moving picture conversion apparatus according to a first embodiment of the present invention.

First, an outline of the apparatus according to the first embodiment will be described. This device is a device that converts an input image in the DVC format into an image in the MPEG format and outputs the image. At this time, a feature is that a coarse motion vector is detected in macroblock units, macroblocks are classified according to the magnitude of the vector, and processing related to image coding is reduced. The DVC format is described below with reference to FIGS.
7 will be described briefly. FIG. 16A shows a frame obtained as a result of sampling the luminance and color difference signals in the A / D converter. Brightness frame is 72
It has a size of 0x480 pixels, and each chrominance frame has a size of 180x480 pixels. To apply DCT, the frame is divided into small blocks (DCT blocks). However, as shown in FIG.
Divide into blocks of equal shape. As for the color difference frame, only the rightmost column is a block having a different shape. FIG. 16B is a diagram in which blocks are extracted from the frame of FIG. 16A and shows the size of each block. Luminance blocks are 8x8 in size, chrominance blocks are 8x8 and 4
There are two sizes of × 16. Several DCT blocks are combined to form a macroblock. FIG. 16C shows the state of the configuration of a macroblock. A macroblock is composed of four luminance blocks adjacent to one line and two chrominance blocks corresponding to the four luminance blocks. However, the macroblock in the rightmost column is composed of four luminance blocks adjacent to each other in two directions and two color difference blocks corresponding to the four luminance blocks. FIG. 17D shows the arrangement of macro blocks in a frame. A group of 27 macroblocks is called a superblock, and one frame has 5 rows and 10 columns of superblocks. FIG. 17E shows the arrangement of super blocks in one frame, and FIG. 17F shows the configuration of a super block composed of 27 macro blocks. Before performing data compression, the order of data is changed in order to equalize the image quality in the screen after compression and to disperse the influence of code errors in consideration of dropout and trick play. This process is called shuffling. Shuffling selects one superblock from each column, extracts one macroblock from each superblock, and forms one video segment with five macroblocks. Then, at the time of compression, DCT, quantization, and variable length coding are controlled so that the data is always within a predetermined amount in video segment units. Thus, in the DVC format,
DCT is applied to each block to remove spatial correlation.

Next, the configuration of the device according to the first embodiment will be described. Input means 101 for taking in images from outside
And a frame memory 102 for storing an image, and a block reading means for reading a block or a macroblock.
103, the component extracting means 104 for extracting only the DC component of the image, two low-resolution frame memories 105a and 105b for storing the extracted information, and moving based on the information of the low-resolution frame memories 105a and 105b. Motion detection means 106 for performing detection, macroblock classification means 107 for labeling macroblocks based on the output of the motion detection means 106,
An area dividing unit 113 including a label storing unit 114 and an area dividing unit controlling unit 109; a decoding unit 110;
111 and the output means 112 are sequentially connected.

Next, the overall operation of the apparatus according to the first embodiment will be described with reference to the flowchart of FIG.

Step 101: First, the input means 101 writes an input image in the DVC format into the frame memory 102.

Step 102: The area dividing unit control means 109 sends a block read request to the block read means 103.
Block reading means 103 which has received the read request
Sends the image (the 0th block to the Nth block) to the component extracting means 104.

Step 103: The component extracting means 104 extracts only the DC component from each block, and step 104: Writes the extracted data to the low-resolution frame memory 105a in order. Step 105: When the image written in the low-resolution frame memory 105a is the first frame of the moving image to be processed, step 106: The area dividing unit control means 109 regards all macroblocks in the frame as intra blocks and performs labeling. The label “large motion vector of macroblock” is sent to the storage unit 114.

Step 107: If the frame is not the first frame, the motion detecting means 106 reads the current image from the low-resolution frame memory 105a and the past image from the low-resolution frame memory 105b. Output a vector.

Step 109: The macroblock classification means 107
Based on the magnitude of the coarse motion vector, each macroblock is labeled with three types of motion, large motion, and small motion. Then, the label of each macro block is sent to the label storage unit 114 of the FIFO system.

Step 110: The area dividing unit control means 109 checks the status of the queue in the label storage means 114. step 111: When the label of the macroblock for one frame is accumulated in the queue, a macroblock read request is sent to the block reading means 103. send. Upon receiving the read request, the block reading unit 103 sends the image (macroblock with an ID of 0 to M) to the decoding unit 110.

Step 112: When the decoding means 110 receives the macroblock with the ID m from the block reading means 103, it extracts the first label from the queue of the label storage means 114.

Step 113: When the label is small or has a motion, the inverse DCT is applied to the macroblock with ID m.
And returns it to the original image, and sends it to the encoding processing unit 111 together with the label. If the label is any other label, the macro block with ID m is sent to the encoding processing unit 111 together with the label without doing anything.

Step 114: The encoding processing section 111 encodes the macroblock input from the decoding means 110 according to the label.

Step 115: The output unit 112 outputs the data encoded by the encoding processing unit 111 to the outside of the apparatus.

Step 116: The area dividing unit control means 109 checks the status of the queue in the label storage means 114. step 117: When there are no more labels stored in the queue,
The image stored in the low resolution frame memory 105a is written to the low resolution frame memory 105b.

Step 118: If the current frame is not the last frame of the moving image to be processed, then the area dividing section control means 109 sends a block read request to the block read means 103 for processing the next frame (step 102). .

Next, a method of reading a macroblock using the block reading means 103 (step 102) will be described. Each unit of the apparatus according to the present embodiment uses a block ID which is a serial number from the upper left to the lower right in a frame in order to process the frame as a block or a macroblock. For one frame, the block IDs are from 0 to N, and the macroblock IDs are from 0 to M. The block reading unit 103 cuts out an image from the frame memory for each block or macroblock in response to a block or macroblock read request from the region dividing unit control unit 109, and selects a block from a block with a block ID 0 to an N block or a macro. The macroblocks whose block IDs are 0 to M are sequentially output.

Next, a method of reading data by the motion detecting means 106 (step 107) will be described. The motion detection unit 106 reads a template (having a size of 2 × 2 pixels) corresponding to the macroblock with the ID m from the low-resolution frame memory 105a, and searches the m-th macroblock from the low-resolution frame memory 105b. Is read out from the low-resolution search area corresponding to. Then, the motion detection means 106 performs motion detection in order from the macroblock ID 0 to M as described below.
(step 108) is performed. When the template and the low-resolution search area are aligned, the motion detection means 106 performs pattern matching between the two and detects a block having the same size as the template having the highest correlation with the pattern of the template from the reduced search area. I do. As the pattern matching, an absolute difference sum, a ratio of the luminance value between the template and the search area, or the like can be used. In any case, if the correlation degree is given a threshold value and the correlation degree does not reach the threshold value, the motion detection unit 106 determines that there is no block with a high correlation to the macroblock with ID m, that is, there is no motion vector. Output. When a block having a high correlation is detected, the center of the search area is set as the origin, and the position of the detected block is output as a coarse motion vector. The motion detecting means 106 sends the macroblock ID of the macroblock for which the motion detection has been performed and its motion information (no motion vector or coarse motion vector) to the macroblock classifying means 107 as a packet.

Next, the macroblock classification method (step 109) by the macroblock classification means 107 will be described.
The macroblock classification means 107 reads out the motion information from the packet input from the motion detection means 106, and when the magnitude of the coarse motion vector is 0, the motion is small; when there is no coarse motion vector, the motion is large; At that time, a label is added to the packet indicating that there is movement. After that, the macroblock classification means 107 sends the packet to the label storage means 114. The label storage unit 114 is of a FIFO type, and sequentially adds transmitted packets to a queue.

Next, the encoding process (step 114) of the macroblock input from the decoding means 110 based on the difference in label in the encoding processing unit 111 will be described with reference to the flowchart of FIG.

Step 121: When the label of the macro block has a motion, step 122: The encoding processing unit 111 performs a motion vector detection on the macro block.

Step 123: If the label is small in motion, the encoding processing unit 111 omits the motion vector detection (step 122), and the macro block and the block having the same size as the macro block at the position of the motion vector in the search area. And calculate the difference. The difference signal is divided into blocks (8 × 8), and step 124: DCT is performed on each block.

Step 125: When the label is large in motion, the encoding processing unit 111 omits motion vector detection (step 122), difference calculation (step 123), and DCT processing (step 124), and performs quantization,
Only variable length coding is performed.

As described above, in the apparatus according to the present embodiment, before the encoding process, the macroblock is classified according to the degree of the motion by providing the area dividing unit 113, and the process of the subsequent macroblock is changed, so that the motion of the macroblock is changed. The load on vector detection can be reduced.

As a first modification of the apparatus of the first embodiment, the encoding unit 111 detects a motion vector in a small motion vector search area for a macroblock labeled as small motion. May be performed. In this way, the load required for encoding a macroblock classified as small motion increases, but the accuracy of motion vector detection increases accordingly, and is equal to the macroblock at the position of the motion vector in the search area. The difference from the block having the size becomes smaller, and the quantization error decreases.

FIG. 4 is a diagram showing a configuration of a second modification of the area dividing unit 113 of the moving picture conversion apparatus according to the first embodiment. In addition to the configuration of the area dividing unit in FIG. 1, a resolution conversion unit 406 is provided between the low resolution frame memory 402b and the motion detection unit 403. The resolution conversion unit 406 performs resolution conversion by interpolating the input image. Thus, if the resolution of the low-resolution search area is too low for motion detection by just extracting with the component extraction means 401, the resolution of the data read from the low-resolution frame memory 402b is converted to increase the resolution of the search area. it can.

As a third modification of the region dividing unit of the moving image converter of the first embodiment, the component extracting means 10
Inverse DCT means is provided between 4 and the low resolution frame memory 105a, and the component extraction means 104 extracts the low frequency component together with the extraction of the DC component, before inputting to the low resolution frame memory 105a, by the inverse DCT means. By applying inverse DCT to the low-frequency component, it is possible to create a convenient low-resolution image for motion detection by the motion detection means 106. By doing so, the calculation load by the inverse DCT means increases, but pattern matching can be performed using the pixel values of the real image instead of the DCT data, and the resolution of the DCT extracted by the component extraction means 104 can be improved.
Since it can be changed according to the low frequency component, the detection accuracy is increased.

Next, as a specific example, consider a case where a moving image as shown in FIG. 14A is input to the device of the third modification. FIG. 14A shows a block A having the same size as a macroblock in a search area of a certain macroblock.
including. The macro block and the block A are each composed of four blocks having the same pattern. Four minutes
An example of one macroblock and a quarter block A is shown in FIG. Pixels in a quarter macroblock have a pixel value of 10 and a pixel value of 0 alternately in a grid pattern. In the quarter block A, all pixels have a pixel value of 5. FIG. 14B also shows a DCT of this quarter macroblock. When such a moving image is input to the device of the third modification,
The component extraction means 104 extracts a DCT low frequency component, that is, a component almost equal to 0 except for DC, from the macroblock. Then, an inverse DCT is performed by the inverse DCT means to obtain an image having flat pixel values. Therefore, the motion detecting means 106 detects a coarse motion vector for the position of the block A.

As is apparent from the above description, in the apparatus of the present embodiment, in the process of decoding the input DVC data and encoding it into MPEG data, the area dividing section 113 is provided before the encoding process, and the motion of the macroblock is Classify by degree. In the motion detection, only the low frequency component and / or the DC component of the DCT coefficient is used, so that the load on the pattern matching is very small. In addition, since most of the energy of the image is concentrated on the low-frequency components, the accuracy is less deteriorated compared to the matching using the real image. As described above, the area dividing unit 113 performs high-speed classification for each macroblock in advance, and
Performs motion vector detection only for a necessary macroblock based on the classification result, so that processing can be reduced. The apparatus of the present embodiment realizes the moving image format conversion by software by reducing the encoding time in the moving image format conversion.

(Second Embodiment) FIG. 5 is a diagram showing a configuration of a moving image conversion device according to a second embodiment of the present invention.

First, an outline of the apparatus according to the present embodiment will be described. The device according to the present embodiment is a device that converts an input image in the DVC format into an image in the MPEG format and outputs the image. The configuration of the area dividing unit 508 is different from that of the moving image conversion device according to the first embodiment. . The region dividing unit 508
The motion of the image is detected and each macroblock is labeled according to the magnitude of the motion.

Next, the configuration of the apparatus according to the present embodiment will be described. The components other than the area dividing unit 508 are the same as those of the moving image converter of the first embodiment. The region dividing unit 508 includes a component extracting unit 50 that extracts only a specific component of the image subjected to DCT.
1 and inverse orthogonal transform means 502 for applying inverse DCT to the extracted information
And two frame memories 503a and 503b for storing images obtained by the inverse orthogonal transform means 502;
02, a motion vector detecting means 504 for performing motion detection based on the image obtained by the
And a label storage unit 506 and a region division unit control unit 507. Macroblock classification means 505 and label storage means 5
06 and the area dividing unit control means 507 are the same as the macroblock classifying means 107, the label storage means 114, and the area dividing unit control means 109 of the first embodiment, respectively.

Next, the operation of the apparatus according to the present embodiment will be described. Image input by input means 509, block reading by block reading means 512, image storage by frame memory 510, decoding by decoding means 513, encoding by encoding processing section 514, output means 5
The operation of output to the outside by 14 is the same as that of the moving image conversion device of the first embodiment. The operation of the area dividing unit 508 is different from that of the moving image conversion device according to the first embodiment. The operation of the area dividing unit 508 will be described with reference to the flowchart of FIG. This is an operation corresponding to the component extraction (step 103) to the labeling (step 109) in the flowchart of FIG. 2 for explaining the overall operation of the moving picture conversion apparatus according to the first embodiment.

Step 201: The component extracting means 501 extracts only a specific component of the image received from the block reading means 512.

Step 202: The inverse orthogonal transform means 502 performs an inverse DCT on the extracted data, and step 203: Writes the extracted data to the frame memory 503a sequentially for each block.

Step 204: The motion vector detecting means 504 reads the current image from the frame memory 503a and obtains the spatial gradient of the image. Next, a past image is read from the frame memory 503b, and a time-direction gradient of the image is obtained.
From these two results, the motion vector detection means 504 obtains a specific direction component of the motion vector for each block. The specific direction is the direction of the maximum image gradient in the block.
Based on the obtained motion vector maximum gradient direction component, the motion vector detection means 504 performs pattern matching on a candidate block within a certain one-dimensional range to obtain a motion vector. The motion vector detection means 504 sends the motion vector to the macroblock classification means 505.

Step 205: The macroblock classification means 505
Label the macro block based on the magnitude of the motion vector. The label is classified into a large motion if the motion vector exceeds the normal search area of the macroblock, a small motion if the motion vector is within a small area defined by the motion vector, and a motion other than the small area. Then, the label of each macro block is output to the label storage unit 506.

Next, the operation of the component extracting means 501 (step 201) and the operation of the inverse orthogonal transform means 502 (step 202) will be described in detail. The component extracting means 501 is provided with a block reading means 5
12 from the block whose block ID read by 12 is 0
Blocks in order. Figure 7 shows component extraction and reverse
It is a figure showing a situation of DCT. The origin is at the upper left of the block, and the positive x-axis direction is to the right and the positive y-axis direction is below, as shown in the figure.
The component extracting unit 501 applies a mask in which the component on the x-axis is 1 and the others are 0 to the block with the ID of n. Next, the inverse orthogonal transform unit 502 performs a two-dimensional inverse DCT on the masked block. As a result, as shown in the figure, an image without gradient in the y direction, that is, an image whose cross section cut along a plane parallel to the x axis is equal to an arbitrary y is output and written to the frame memory 503a (step 203). However, in FIG. 7, I represents the intensity of the image. The component extraction means 501 continues to the same block
A mask is applied to (ID is n) such that the component on the y-axis is 1 and the others are 0. Then, the inverse orthogonal transform unit 502 performs a two-dimensional inverse DCT on the masked block.
As a result, as shown in the figure, an image without gradient in the x direction, that is, y
An image whose cross section cut by a plane parallel to the axis is equal to an arbitrary x is output and written to the frame memory 503a (step 203).
The frame memory 503a simultaneously holds an image having no y-direction frequency component and an image having no x-direction frequency component.

Since an image having no gradient in the y direction and no gradient in the x direction has only one-dimensional information, writing all of the two-dimensional image into the frame memory 503a is useless, and there is no gradient in the y direction. , One-dimensional information in the x-direction and one-dimensional information in the y-direction may be extracted from the image having no gradient in the x direction and written into the frame memory 503a. Also, instead of applying a two-dimensional mask to each DCT block in the component extraction means 501, the inverse orthogonal transform means 502 applies a one-dimensional inverse DC
By performing T, one-dimensional information in each of the x direction and the y direction may be obtained.

Next, the motion detection processing (step 204) in the motion vector detection means 504 will be described with reference to the flowchart of FIG. 8 and FIG.

Step 211: First, the motion vector detecting means 504 reads an image without a gradient in the y-direction corresponding to the m-th macroblock from the frame memory 503a.

Step 212: Next, in the image without gradient in the y direction, focus on one point A at the center in the macroblock, and gradient in the x direction from a pixel on a straight line parallel to the x axis passing through that point.
Calculate I_x. Here, I represents the intensity of the image, I_x is the inclination of the luminance in the x direction at that point, and is obtained by dividing the difference in the luminance value by the distance between pixels. The calculation result is temporarily stored in the motion vector detecting means 504.

Step 213: The motion vector detecting means 504 determines the gradient in the x-axis direction from the images without gradient in the y-direction of the past several frames stored in the frame memory 503b (step 212).
The pixel at the position equal to the point A of interest is read out.

Step 214: The motion vector detecting means 504 calculates a temporal gradient I_t from the read pixel (pixel). Here, I_t is a change in pixel value per frame.

Step 215: The motion vector detecting means 504 calculates -I_t / I_x from the gradient I_x in the x-axis direction and the gradient I_t in the time direction which have been held (step 212). However, if I_x is
When the value is 0, the image speed cannot be determined in principle, so the x-direction component is undefined, and an undefined signal is output. The value of -I_t / I_x is the x-direction component of the motion vector maximum image gradient direction component. Also, it can be said that the motion vector to be obtained in the maximum image gradient direction at the point of interest A in the macroblock is orthogonally projected, and further, it is orthogonally projected on the x-axis.

Step 216: Next, the motion vector detecting means 504
Reads an image without an x-direction gradient corresponding to the m-th macroblock from the frame memory 503a.

Step 217: Hereinafter, calculation of the x-direction component (step 21)
2 to step 213), the motion vector detecting means 5
04 finds the gradient I_y in the y-axis direction, step 218: reads out pixels from the past several frames of the image without gradient in the x direction stored in the frame memory 503b, step 219: calculates the gradient in the time direction I_t, step 220: y direction Is calculated as -I_t / I_y. That is the ID
Is the y-direction component of the maximum image gradient direction component of the motion vector of the macroblock of m.

Step 221: By the above process, the motion vector component in the maximum image gradient direction (-I_t / I_x, -I_t / I_y)
Is obtained. Next, in order to detect the motion vector of the macroblock with the ID m, the motion vector detection means 504 performs pattern matching. At this time, the search area is one-dimensionally determined from the already obtained motion vector maximum image gradient direction component. Limited to. The one-dimensional search area is a point B obtained by adding a motion vector component in the maximum image gradient direction to the position vector of the point A.
, And is a line segment having a fixed length centered on the point B on a straight line in a direction perpendicular to the maximum image gradient direction.

Step 222: The motion vector detecting means 504 performs pattern matching on the image in the one-dimensional search area to obtain a motion vector.

Although the pixel at the center of the macro block is selected as the target pixel A, any pixel within the macro block may be used. Further, a simple inter-pixel difference may be calculated to obtain a gradient in the time or space direction, or may be obtained by fitting a straight line using the least squares method.

As described above, in the apparatus of the present embodiment, in the process of decoding the input DVC data and encoding it into MPEG data, the area dividing unit 508 performs the encoding process before the encoding process.
Are provided, and classification is performed according to the degree of movement of the macroblock. At the time of motion detection, an image having no y-direction frequency component and an image having no x-direction frequency component are created using only specific frequency components of DCT coefficients. Then, the motion vector detecting means 504 calculates a motion vector component in the maximum image gradient direction from the spatial and temporal gradients of those images, and furthermore, performs a motion vector of a macroblock by pattern matching from a search area limited to one dimension. Get.
The magnitude of the motion vector obtained in this way is used for macroblock classification.

As described above, the region dividing unit 508 classifies the degree of motion for each macroblock at high speed in advance, and the encoding processing unit 514 detects a motion vector with higher accuracy only for a necessary macroblock from the classification result. Since it is performed, the processing can be reduced. The apparatus of the present embodiment realizes the moving image format conversion by software by reducing the encoding time in the moving image format conversion.

(Third Embodiment) FIG. 9 shows a third embodiment of the present invention, in which a search area for a motion vector is narrowed based on the output of a motion detection means, and FIG. 2 is a configuration diagram illustrating a configuration of a moving image conversion device that performs detection.

Before starting the description of the apparatus according to the present embodiment, an outline of the motion vector detection method will be described first. Motion vector detection unit 924 in encoding processing unit 925
Is to determine the motion of a macroblock labeled as having motion by referring to a past frame. The motion vector detecting unit 924 is characterized in that the search range of the motion vector is limited based on the output of the area dividing unit 911, and the motion vector is detected.

Next, the configuration of the moving picture converter of the third embodiment will be further described. An input unit 901 for capturing an image from outside, a frame memory 902 for storing the image,
The block reading means 903 is connected to the component extracting means 904 for extracting only the DC component of the image, the two low-resolution frame memories 905a and 905b for storing the extracted information, and the information from the low-resolution frame memories 905a and 905b. Motion detecting means 906 for performing motion detection, macroblock classifying means 907 for labeling macroblocks based on the output of the motion detecting means 906, and a motion vector of a macroblock having motion based on the result of the motion detecting means 906. A read request is issued to the motion vector generating means 909, the label storing means 908 and the block reading means 903 for generating
An area dividing unit 911 composed of an area dividing unit controlling means 910 for controlling the components within, a decoding means 912, a DCT means 914,
Quantizing means 915, variable length coding means 91, and inverse quantization means 9
17, inverse DCT means 918, frame memory 921, image differentiator 91
3. An encoding unit 925 including an image adder 920, a motion vector detecting unit 924, and a motion information encoding unit 922, and an output unit 926 are connected. Motion vector detector 924
Inside, a search area changing means 923 is provided.

Next, the operation of the moving picture converter of the third embodiment will be described. The basic operation flow is the same as that of the flowchart of FIG. 2 for explaining the operation of the apparatus of the first embodiment, except that the packet output from the macroblock classification unit 907 is received by the motion vector generation unit 909, and the coarse motion The difference is that the vector is converted into a motion vector at the actual image resolution and the packet is transmitted to the label storage unit 908. The motion detecting means 906 in the area dividing unit 911 detects a coarse motion vector of a macroblock in the low resolution search area. Then, based on the coarse motion vector, the macroblock classification means 907 performs macroblock classification. In the moving picture conversion apparatus according to the first embodiment, a motion vector is calculated from the macroblock classified as having motion by a motion vector detection unit in the encoding processing unit 111. The apparatus is characterized in that a search area is limited and a search is performed based on a coarse motion vector obtained by the motion detection means 906 in the area division unit 911. A high-speed motion vector detection process in which the motion vector search area is narrowed will be described with reference to the flowchart in FIG.

Step 301: The motion vector generation means 909 reads a label from the packet input from the macroblock classification means 907.

Step 302: When the label is small or large, the packet is transmitted to the label storage unit 908 as it is (step 305). If the read label has a motion, step 303: read out further motion information (coarse motion vector) from the packet, and step 304: return the coarse motion vector obtained in the low resolution search area to the motion vector at the original resolution, and Update the motion information of.

Step 305: The motion vector generation means 909 transmits the packet to the label storage means 908.

Step 306: When the decoding means 912 receives the m-th macroblock from the block reading means 903, it extracts the first packet from the queue of the label storage means 908. After performing the decoding process using the label (step 113) in the apparatus of the first embodiment, step 307: If the label is small or large in motion, the macroblock of number m and the label are transmitted to the encoding unit 925. If the label has motion, step 308: fetch the motion vector from the packet, and add the m-th macroblock and the label to the encoding processing unit 92.
Send to 5.

Step 309: The search area changing means 923 placed in the motion vector detecting section 924 is smaller than the original search area around the received motion vector and is included in the original search area. New search area is determined.

Step 310: The motion vector detecting section 924 obtains a motion vector from the newly determined search area.

As described above, the apparatus according to the present embodiment decodes the input DVC data and encodes the data into MPEG data before the encoding process.
Are provided, and classification is performed according to the degree of movement of the macroblock. Since the encoding processing unit 925 performs motion vector detection only on a necessary macroblock based on the classification result, the processing can be reduced. Then, at the time of motion vector detection, the motion vector detection unit 924 is used based on the coarse motion vector obtained by the motion detection unit 906 in the region division unit 911.
Since the search is performed by narrowing the search area, the motion vector can be obtained at high speed. The device of the present embodiment is
The encoding time is further reduced from the moving picture conversion apparatus of the first embodiment, and the moving picture format conversion is realized by software.

Next, as an example, consider a case where a moving image as shown in FIG. 11A is input to the apparatus of the present embodiment. FIG. 11A shows a block A having a size equal to the size of a macroblock in a search area of a certain macroblock such that the sum of all pixel values in the macroblock is equal to the sum of all pixel values in block A. , And includes a block B having a size equal to the size of the macroblock, such that each pixel value in the block B is smaller by 1 than each pixel value in the macroblock.
In addition, block A and block B are located apart from each other in the search area. The macro block, block A and block B are each composed of four blocks having the same pattern. FIG. 11B shows an example of a quarter macro block, a quarter block A, and a quarter block B. Quarter
The macro block has pixel values from 127 to 127 in order from the upper left to the lower right pixel so as to form an arithmetic progression having a tolerance of 2. In the quarter block A, all pixels have a pixel value of 64. Then, the quarter block B has 12 pixel values such that each pixel value is 1 smaller than the macroblock, that is, an arithmetic progression having a tolerance of 2 from 0 in order from the upper left pixel to the lower right pixel.
It has pixel values up to 6.

When such a moving image is input to the apparatus of this embodiment, the component extracting means 904 extracts a DC component having exactly the same value from the macroblock and the block A. The DC component of block B is smaller. Therefore, the motion detection means 906 detects a coarse motion vector for the position of the block A. As a result, the motion vector detection unit 924 calculates
A motion vector near the coarse motion vector is output or no motion vector is detected and no motion vector is output.

As a first modification of the moving picture conversion apparatus according to the third embodiment, the second division is replaced with the second division.
Using the region dividing unit 508 in the moving image converter of the embodiment, the motion vector is output from the motion vector detecting unit 504 in the region dividing unit 508, and the motion vector is detected in the motion vector detecting unit 924 in the encoding unit 925. It is possible to narrow down the search area and detect a motion vector at high speed.

Next, as a specific example, the moving image 1201 shown in FIG.
It is assumed that the data encoded into the C data is input to the device of the first modification. The moving image 1201 has a macroblock composed of four blocks A, and a block having the same size as the macroblock composed of four blocks B in the past frame is separated by several pixels from the position of the macroblock in the current frame. That is what exists. In block A, the pixel values in the block 0 column are all 100, and the value in the column 0 decreases by 10 as the column increases, such as 90 in the first column. Here, the column of the block means that the leftmost pixel of the block is 0 column, the pixel on the right side is 1 column, and the column numbers are sequentially added toward the right. In the row of a block, the pixel at the top of the block is set to 0 row, the pixel immediately below the block is set to 1 row, and row numbers are sequentially added downward. In the block B, the same values are stored in the pixels in the 0th, 2nd, 4th and 6th rows. As the value of the row, the value of block A is 2
It is doubled. That is, column 0 is 200, column 1 is 18.
The value decreases by 20 as the column increases, such as 0. The pixel values of the blocks 1, 3, 5, and 7 are 0.

The right side of FIG. 12 shows the result of applying DCT to blocks A and B, applying a mask such that the 0th row of the block is 1 and the other rows are 0, and applying a DCT. In pattern matching for data at the actual image level, there is not much correlation between the two, but the data after processing as in the present modified example is equal. The motion vector detecting means 504 of the present modified example device determines the image speed in the x-axis direction as shown in FIG.
Use the data to the right of Since both data have the same gradient in the x-axis direction, the motion vector detection means 504 calculates the x-value of the macroblock and the block consisting of four blocks B.
The same value as the difference between the positions on the axis is output as the x component of the motion vector maximum image gradient direction component. The y component of the motion vector maximum image gradient direction component outputs an indefinite signal because I_y is 0. However, since the magnitude of the motion vector is equal to or larger than the value output as the x component of the motion vector maximum image gradient direction component, the label of this macroblock is classified as having motion.

Further, as a second modification of the moving picture conversion apparatus of the third embodiment, the encoding processing section 925 omits the motion vector detecting section 924 and detects the motion vector by the motion vector detecting means 504 in the area dividing section 508. The motion vector obtained can be used as it is. By doing so, the load of motion vector detection in the encoding processing unit 925 is eliminated,
Processing is reduced.

Next, as a specific example, consider a case where the moving image 1302 in FIG. 13 is input to the device of the second modification. The moving image 1302 has, for each DCT block of the DVC data 1301, the origin is the upper left of the block, the x-axis positive direction to the right, and the y
When the axis is taken in the positive direction, the components on the x-axis and y-axis are 1 and the other components are 0. When the moving image 1302 is input to the present modified example device, the motion vector detecting unit 504 outputs a motion vector that is the same as when the DVC data is input to the motion vector detecting unit 504. It can be used as a motion vector without being obtained.

As is apparent from the above description, in the apparatus of the present embodiment, in the process of decoding the input DVC data and encoding it into MPEG data, the area dividing unit 911 is provided before the encoding process, and the motion of the macroblock is determined. Classify by degree. Since the encoding processing unit 925 performs motion vector detection only on a necessary macroblock based on the classification result, the processing can be reduced. At the time of motion vector detection, based on the motion vector obtained by the motion vector detection means 504 in the region division unit 508, the motion vector detection unit 924 narrows the search area and searches for the motion vector. By using the motion vector obtained by the motion vector detecting means 504 in the dividing unit 508 as the motion vector without obtaining the motion vector, the motion vector detecting unit 924 can obtain the motion vector at high speed. The apparatus according to the present embodiment realizes a moving image format conversion by software by further reducing the encoding time from the moving image conversion apparatus according to the second embodiment.

[0092]

As described above, before performing the image compression processing, the macroblock is classified by the area dividing unit provided by the present invention, and the encoding processing is changed according to the classification result, thereby calculating the motion vector detection. The amount can be reduced and the speed of the image compression processing can be increased.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a moving image conversion device according to a first embodiment of the present invention;

FIG. 2 is a flowchart showing an operation of the entire moving picture conversion apparatus according to the first embodiment of the present invention;

FIG. 3 is a flowchart showing the flow of processing performed by a difference in label in an encoding processing unit;

FIG. 4 shows a second example of the moving image conversion device according to the first embodiment.
Diagram showing a configuration of a modification of the

FIG. 5 is a diagram illustrating a configuration of a moving image conversion device according to a second embodiment of the present invention;

FIG. 6 is a flowchart showing an operation of a region dividing unit of the moving image conversion device according to the second embodiment of the present invention;

FIG. 7 is a diagram illustrating a method of component extraction and inverse DCT from an input image,

FIG. 8 is a flowchart showing a flow of a motion detection process of the moving image conversion device according to the second embodiment of the present invention;

FIG. 9 is a diagram illustrating a configuration of a moving image conversion device according to a third embodiment of the present invention;

FIG. 10 is a flowchart showing a flow of processing for narrowing a search area and detecting a motion vector;

FIG. 11 is a diagram illustrating an example of an input moving image to the moving image conversion device according to the third embodiment;

FIG. 12 is a diagram illustrating an example of an input moving image for a first modification of the moving image conversion device according to the third embodiment;

FIG. 13 is a diagram showing an example of an input moving image for a second modification of the moving image conversion device according to the third embodiment;

FIG. 14 is a diagram showing an example of an input moving image for a second modification of the moving image conversion device according to the first embodiment;

FIG. 15 is a diagram showing a configuration of a typical MPEG encoding apparatus;

FIG. 16 is a diagram for explaining a DVC format in a home digital VTR,

FIG. 17 is a diagram for explaining a DVC format in a home digital VTR,

FIG. 18 is a diagram illustrating a method of obtaining a motion vector from a component of a motion vector in the maximum gradient direction of an image.

[Description of Signs] 101, 509, 901 Input means 102, 510, 503a, 503b, 902, 921 Frame memory 103, 512, 903 Block reading means 104, 401, 501, 904 Component extraction means 105a, 105b, 402a, 402b , 905a, 905b Low-resolution frame memories 106, 403, 906 Motion detection means 107, 404, 505, 907 Macroblock classification means 109, 407, 507, 910 Area division section control means 110, 513, 912 Decoding means 111, 514, 925 encoding processing unit 112, 515, 926 output unit 113, 408, 508, 911 region division unit 114, 405, 506, 908 label storage unit 406 resolution conversion (interpolation) unit 502 inverse orthogonal conversion unit 504 motion vector detection unit 909 motion Vector generation means 913 Image subtractor 914 DCT means 915 Quantization means 916 Variable length coding means 917 Inverse quantization means 918 Inverse DCT means 920 Image adder 922 Motion information coding means 923 Search area change means 924 Motion vector detection section 1201 , 1302 video 1301 DVC Data

Claims (8)

[Claims] An input means for inputting a digital image signal composed of macroblocks, which is a set of blocks, which is subjected to orthogonal transformation processing for each block, a frame memory for storing the digital image signal, A block reading unit for reading the block or the macroblock from the frame memory, a region dividing unit controlling unit for issuing a reading request to the block reading unit, a component extracting unit for extracting only a DC component of a captured image, Two low-resolution frame memories for storing information, motion detection means for performing motion detection based on the information in the low-resolution frame memory, and macroblocks for labeling the macroblocks according to the result of motion detection by the motion detection means Stores classification means and labels of the macroblocks An area dividing unit including a label storage unit, a decoding unit, an encoding unit that changes an encoding process of the macroblock according to a label of the label storage unit, and an output unit that outputs an encoded signal to the outside. A video conversion device comprising: 2. The method according to claim 1, wherein when the label storage unit outputs a label indicating that the motion of the macroblock is small, the encoding processing unit performs a motion vector detection process by setting a small motion vector search area. The moving image conversion device according to claim 1, wherein: 3. The moving image conversion apparatus according to claim 1, further comprising: a resolution conversion unit that performs resolution conversion by interpolating an input image in the area dividing unit. 4. The component extracting means extracts a DC component and a low-frequency component of a captured image, and further performs an inverse DCT on the low-frequency component extracted by the component extracting means in the area dividing unit. 2. The moving picture conversion apparatus according to claim 1, further comprising an inverse DCT means for performing the conversion. 5. An input means for inputting a digital image signal composed of macroblocks, which is a set of blocks, which is subjected to orthogonal transformation processing for each block, a frame memory for storing the digital image signal, and A block reading unit that reads the block or the macro block from the frame memory, a region dividing unit control unit that issues a read request to the block reading unit, and only a specific component decomposed by a basis orthogonal to each other from a captured image. Component extraction means for extracting the information, inverse orthogonal transformation means for performing inverse orthogonal transformation of the extracted information, two frame memories for storing information obtained by the inverse orthogonal transformation means, and information obtained by the inverse orthogonal transformation means Motion vector detecting means for performing motion detection; An area dividing unit including a macroblock classifying unit that labels a macroblock according to the size of a motion vector output by the stage and a label storage unit that stores a label of the macroblock; a decoding unit;
A moving picture conversion device comprising: an encoding processing unit that changes an encoding process of a macroblock according to a label of the label storage unit; and an output unit that outputs an encoded signal to the outside. 6. An input means for inputting a digital image signal composed of macroblocks, which is a set of blocks, which is subjected to orthogonal transformation processing for each block, a frame memory for storing the digital image signal, and A block reading unit for reading the block or the macroblock from the frame memory, a region dividing unit control unit for issuing a read request to the block reading unit, a component extracting unit for extracting only a DC component from a captured image, Two low-resolution frame memories for storing information, motion detection means for detecting motion based on the information in the low-resolution frame memory, and macroblock classification for labeling macroblocks based on the output of the motion detection means Means, based on the result of the motion detecting means, An area dividing unit including a motion vector restoring unit for restoring a lock motion vector and a label storage unit for storing a macroblock label, a decoding unit, a DCT unit, a quantization unit, a variable length encoding unit,
An encoding processing unit including a motion vector detecting unit and a motion information encoding unit in which an inverse quantization unit, an inverse DCT unit, a frame memory unit, an image differentiator, an image adder, and a search area changing unit are provided; Output means for outputting the extracted signal to the outside, wherein the motion vector detection unit in the encoding processing unit limits the motion vector search area based on the motion detection result by the motion detection means in the area division unit, A video converter that performs vector detection. 7. The apparatus according to claim 5, further comprising a region dividing unit according to claim 5 in place of said region dividing unit, wherein a motion vector detecting unit in said encoding processing unit detects a motion vector detected by a motion vector detecting unit in said region dividing unit. 7. The moving image conversion device according to claim 6, wherein a motion vector search area is limited by using the motion vector detection, and a motion vector is detected. 8. The encoding processing unit performs no motion vector detection by omitting a motion vector detection unit in the encoding processing unit, and uses a motion vector detected by a motion vector detection unit in the region division unit. The moving image conversion device according to claim 7, wherein