JP2004215049A - Encoding device and method, decoding device and method, and program - Google Patents

Encoding device and method, decoding device and method, and program Download PDF

Info

Publication number
JP2004215049A
JP2004215049A JP2003000819A JP2003000819A JP2004215049A JP 2004215049 A JP2004215049 A JP 2004215049A JP 2003000819 A JP2003000819 A JP 2003000819A JP 2003000819 A JP2003000819 A JP 2003000819A JP 2004215049 A JP2004215049 A JP 2004215049A
Authority
JP
Japan
Prior art keywords
image information
unit
storage unit
search
motion compensation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
JP2003000819A
Other languages
Japanese (ja)
Other versions
JP2004215049A5 (en
Inventor
Yukio Yanagida
幸雄 柳田
Original Assignee
Sony Corp
ソニー株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Corp, ソニー株式会社 filed Critical Sony Corp
Priority to JP2003000819A priority Critical patent/JP2004215049A/en
Publication of JP2004215049A publication Critical patent/JP2004215049A/en
Publication of JP2004215049A5 publication Critical patent/JP2004215049A5/ja
Granted legal-status Critical Current

Links

Images

Abstract

Image processing using motion compensation is performed more efficiently.
A memory controller stores local decoded image data supplied from an IDCT processing unit in an external memory. Further, the memory controller 101 searches the internal memory 102 for local decoded image data including the search image data requested by the motion prediction unit 12 or the motion compensation unit 13, acquires, and holds the data. If the necessary local image data does not exist in the internal memory 102, the memory controller 101 acquires the local decoded image data from the external memory 21, holds the local decoded image data, and stores the local decoded image data in the internal memory 102. Then, the memory controller 101 generates the requested search image data using the held local image data, and supplies the generated search image data to the request source. The present invention can be applied to an image information encoding device or an image information decoding device.
[Selection diagram] FIG.

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an encoding apparatus and method, a decoding apparatus and method, and a program, and more particularly to an encoding apparatus and method, a decoding apparatus, and a method capable of more efficiently executing image processing using motion compensation. , And the program.
[0002]
[Prior art]
In recent years, various compression encoding methods have been proposed as methods for reducing the amount of information by compressing video data and audio data, and a representative one is MPEG2 (Moving Picture Experts Group Phase 2).
[0003]
With reference to FIG. 1, a description will be given of a process in a case where video data is compression-encoded by the MPEG2 system.
[0004]
In FIG. 1, an image information encoding device 10 encodes input image data by an MPEG method and outputs a bit stream. The input image data input to the image information encoding device 10 is supplied to the image rearrangement processing unit 11, rearranged for each frame image, and supplied to the motion prediction unit 12.
[0005]
The motion prediction unit 12 calculates a motion vector using the input image data that has been input and the search image data that is reference image data supplied from the memory controller 19, and the motion compensation unit 13 uses the memory controller According to the search image data supplied from the motion prediction unit 12 and the motion vector supplied from the motion prediction unit 12, the motion compensation of the input image data is performed, and the DCT processing unit 14 calculates the prediction residual supplied from the motion compensation unit 13. Then, the quantizing unit 15 performs a quantizing process on the transform coefficient supplied from the DCT process 14, and a VLC (Variable Length Code: variable length coding) unit 16 The quantized transform coefficients are variable-length coded and output from the image information coding device 10 as a bit stream.
[0006]
The quantization processing unit 15 also supplies the quantized transform coefficients to the inverse quantization processing unit 17. The inverse quantization processing unit 17 performs an inverse quantization process on the quantized transform coefficient, and an IDCT (Inverse Discrete Cosine Transform: inverse discrete cosine transform) processing unit 18 is supplied from the inverse quantization processing unit 17. The transform coefficients are subjected to an inverse discrete cosine transform process, and supplied to the memory controller 19 as locally decoded image data.
[0007]
The memory controller 19 stores the supplied locally decoded image data in the external memory 21, reads the locally decoded image data from the external memory 21 for a subsequent encoding process, and converts the search image data into the motion prediction unit 12 or the motion compensation To the unit 13.
[0008]
In the image information encoding device 10 as described above, the memory controller 19 converts the search image data from the locally decoded image data stored in the external memory 21 based on the request of the motion prediction unit 12 or the motion compensation unit 13. It is generated and supplied to the motion prediction unit 12 or the motion compensation unit 13 that is the request source.
[0009]
In the external memory 21, data transfer is usually performed in a burst manner to improve the access efficiency. Therefore, the memory controller 19 can control the data transfer of the external memory 21 only in a certain unit such as a macro block unit, for example, and can control the external memory 21 in a small unit such as a pixel unit. It is not possible to control data transfer and transfer only the minimum necessary search image data.
[0010]
For example, as shown in FIG. 2A, when requesting search image data corresponding to a macroblock 32 in a frame image 31 composed of macroblocks arranged in 4 × 4, the motion prediction unit 12 2B, a motion vector 37 indicating the motion of the object 33 is calculated based on the frame image 35 which is the previous frame image of the frame image 31 of FIG. 2A, and the motion compensation unit 13 calculates the motion vector. Based on 37, a request is made to the memory controller 19 for search image data 36 corresponding to the macroblock 32.
[0011]
Based on the request, the memory controller 19 reads the macro blocks 41 to 44 including the requested search image data 36 from the external memory 21 as shown in FIG. The search image data 36 is generated using the read macroblocks 41 to 44 and supplied to the motion compensation unit 13.
[0012]
In this case, as shown in FIG. 3, the memory controller 19 sends a macro image from the external memory 21 to generate search image data (image data having a size of one macro block) corresponding to one macro block. Blocks 41 to 44, that is, image data for four macroblocks must be read.
[0013]
At this time, data that is not used as search image data among the read image data of four macroblocks is discarded as it is. Therefore, in the above case, the data transfer between the memory controller 19 and the external memory 21 becomes inefficient, which may have a great effect on the processing time.
[0014]
On the other hand, the reference image data (search image data) used in the motion detection (motion prediction) processing is reused in the motion compensation processing, so that the number of times of reading the reference image data from the external memory is one. (See, for example, Patent Document 1).
[0015]
[Patent Document 1]
JP-A-9-055940 (page 6-8, FIG. 1)
[0016]
[Problems to be solved by the invention]
Generally, in image data, similar motion vectors are likely to be selected in adjacent macroblocks, and in many cases, unnecessary data in a certain macroblock is required as search image data of a neighboring macroblock.
[0017]
However, in the above-mentioned method, even if data is acquired in macroblock units from the external memory 21 for processing of a certain macroblock, an unnecessary portion is discarded as unused. Therefore, there is a problem that the memory controller 19 has to read the same data from the external memory 21 again in the process for the neighboring macro block.
[0018]
In the above method, the reference image data (search image data, that is, locally decoded image data) used in the motion detection (motion prediction) process is reused in the motion compensation process. However, in the motion prediction process, using the input image data may improve the image quality of the obtained image rather than using the locally decoded image data. In the motion prediction process and the motion compensation process, It is desirable to be able to use different image data as search image data.
[0019]
Further, the above method has a problem that the motion detection process is not included and cannot be applied to the decoding process.
[0020]
FIG. 4 is a block diagram illustrating a configuration example of a conventional image information decoding device that decodes image data encoded by the image encoding device of FIG.
[0021]
In FIG. 4, an image information decoding device 50 decodes an input bit stream and outputs decoded image data. The bit stream input to the image information decoding device 50 is supplied to a frame memory 51, and is supplied to a VLD (Variable Length Decode: variable length decoding) unit 52 for each frame to perform a decoding process.
[0022]
The bit stream decoded by the VLD unit 52 is subjected to an inverse quantization process by an inverse quantization unit 53, and then subjected to an inverse discrete cosine transform process by an IDCT unit 54 and supplied to a motion compensation unit 55. You.
[0023]
The motion compensation section 55 performs motion compensation based on the search image data supplied from the memory controller 57, and the image rearrangement processing section 56 rearranges the supplied image data in the original order and outputs the decoded image data. Is output to the outside of the image information decoding device 50.
[0024]
The memory controller 57 stores the decoded image data supplied from the motion compensation unit 55 in the external memory 61, reads the decoded image data from the external memory 61 for the subsequent decoding process, and converts the search image data into the motion compensation unit. 55.
[0025]
As described above, in the image information decoding device 50 in FIG. 7 corresponding to the image information encoding device 10 in FIG. 1, the memory controller 57 is different from the motion compensation unit 55 in that the memory controller 19 in FIG. The same processing as that performed for the section 13 is performed, and the requested search image data is supplied.
[0026]
That is, similarly to the case of the image information encoding device 10, the image information decoding device 50 has a problem that the efficiency of data transfer from the external memory 61 is reduced. Therefore, it is desirable to solve the above-described problems in both the image information encoding device 10 and the image information decoding device 50.
[0027]
The present invention has been made in view of such a situation, and it is an object of the present invention to more efficiently execute image processing using motion compensation.
[0028]
[Means for Solving the Problems]
In the coding apparatus according to the present invention, the motion compensation is performed by the motion compensation unit that performs motion compensation on the first image information using the search image information corresponding to the input first image information. Encoding means for encoding the generated first image information, first generating means for generating second image information for generating search image information, and second generating means for generating the second image information. A first storage unit for storing the image information, a first acquisition unit for acquiring the second image information stored in the first storage unit, and a first operation unit that operates faster than the first storage unit; A second storage unit configured to store the second image information acquired by the acquisition unit, a second acquisition unit configured to acquire the second image information stored by the second storage unit, Second image information acquired by the acquisition unit or the second acquisition unit A second generation unit for generating search image information by using the first storage unit and a second storage unit for searching for second image information necessary for generation of the search image information by the second generation unit. And a second acquisition unit that, when the second image information is stored in the second storage unit based on a search result by the search unit, stores the second image information in the second storage unit. Information is acquired, and the first acquiring means, based on a search result by the searching means, stores the second image information in the first storage means instead of being stored in the second storage means And acquiring the second image information from the first storage means.
[0029]
The motion compensator may use the search image information generated by the second generator to perform motion compensation on the first image information.
[0030]
The image processing apparatus further includes a calculation unit that calculates a motion compensation vector of the attention area based on the additional information added to the input first image information, and the calculation unit includes a search image generated by the second generation unit. Using the information, the motion compensation vector can be calculated.
[0031]
The encoding unit may orthogonally transform the first image information and encode the quantized data.
[0032]
The first generation unit generates the second image information by performing an orthogonal transformation on the first image information, performing an inverse quantization process on the quantized data, and performing an inverse orthogonal transformation process. You can do so.
[0033]
The first storage means can store a larger amount of information than the second storage means.
[0034]
The first acquisition unit may acquire the second image information at a lower speed than the second acquisition unit.
[0035]
A compression unit configured to compress the second image information acquired by the first acquisition unit; and a decompression unit configured to decompress the second image information compressed by the compression unit. The second image information compressed by the compression unit is stored, and the expansion unit expands the second image information acquired from the second storage unit and compressed by the compression unit by the second acquisition unit. You can do so.
[0036]
A plurality of motion compensators operating in parallel and performing motion compensation for different first image information using different search image information, and a plurality of motion compensators operating in parallel and performing motion compensation by the plurality of motion compensators. A plurality of encoding means for encoding a plurality of first image information different from each other, and a plurality of second image information for generating second image information for generating search image information used by the corresponding motion compensation means. First generating means, wherein the first storing means stores the second image information generated by the plurality of first generating means, and the second generating means stores the first image information or the second obtaining information. The search image information used by the plurality of motion compensation units can be generated using the second image information acquired by the acquisition unit.
[0037]
A compression unit configured to compress the second image information acquired by the first acquisition unit; and a decompression unit configured to decompress the second image information compressed by the compression unit. The second image information compressed by the compression unit is stored, and the expansion unit expands the second image information acquired from the second storage unit and compressed by the compression unit by the second acquisition unit. You can do so.
[0038]
The encoding method according to the present invention searches the first storage unit and the second storage unit for second image information for generating search image information used for motion compensation of the input first image information. When the target second image information is stored in the second storage unit based on the search step performed by the search step and the search result obtained by the processing of the search step, the target second image information is stored in the second storage unit. If the acquired second image information is not stored in the second storage unit but is stored in the first storage unit, the target second image information is obtained from the first storage unit The second control unit controls the second storage unit when the second image information is obtained from the first storage unit. Storage control step of storing the second image information It is controlled by the process control steps, characterized in that it comprises a generation step of generating search image data by using the second image information acquired.
[0039]
According to a first program of the present invention, second image information for generating search image information used for motion compensation of input first image information is stored in a first storage unit and a second storage unit. When the target second image information is stored in the second storage unit based on the search step of searching and the search result of the processing of the search step, the target second image information is stored in the second storage unit. And if the target second image information is not stored in the second storage unit but is stored in the first storage unit, the target second image information is stored in the first storage unit. An acquisition control step of controlling acquisition, and a control of the acquisition control step. When the target second image information is acquired from the first storage section, the second storage section is controlled and acquired. Storage control step of storing the obtained second image information Is controlled by the processing of the acquisition control step, characterized in that to realize a generation step of generating a search image information to a computer using the second image information acquired.
[0040]
A decoding apparatus according to the present invention includes: decoding means for decoding an input bit stream; and motion compensation of first image information, which is a bit stream decoded by the decoding means, by using search image information corresponding to the first image information. , A first storage unit that stores second image information that is the first image information on which motion compensation has been performed by the motion compensation unit, and a first storage unit that stores the second image information. A first acquisition unit for acquiring second image information, a second storage unit that operates faster than the first storage unit, and stores the second image information acquired by the first acquisition unit. A second obtaining unit for obtaining the second image information stored by the second storage unit, and a search using the second image information obtained by the first obtaining unit or the second obtaining unit. Generation means for generating image information Search means for searching the first storage means and the second storage means for second image information required for generation of search image information by the generation means, When the second image information is stored in the second storage unit based on the result, the second image information is obtained from the second storage unit, and the first obtaining unit obtains the search result by the search unit. If the second image information is not stored in the second storage means but is stored in the first storage means, the second image information is obtained from the first storage means based on Features.
[0041]
The motion compensator may perform motion compensation on the first image information using the search image information generated by the generator.
[0042]
The decoding means may decode the bit stream, perform an inverse orthogonal transform, and perform an inverse quantization process.
[0043]
The first storage means can store a larger amount of information than the second storage means.
[0044]
The first acquisition unit may acquire the second image information at a lower speed than the second acquisition unit.
[0045]
A compression unit configured to compress the second image information acquired by the first acquisition unit; and a decompression unit configured to decompress the second image information compressed by the compression unit. The second image information compressed by the compression unit is stored, and the expansion unit expands the second image information compressed by the compression unit and acquired from the second storage unit by the second acquisition unit. Can be
[0046]
A plurality of decoding means operating in parallel to decode a plurality of different bit streams; and a plurality of motion compensation means operating in parallel to perform motion compensation of different first image information using different search image information Means, wherein the first storage means stores the second image information on which motion compensation has been performed by the plurality of motion compensation means, and the generation means obtains the second image information by the first acquisition means or the second acquisition means. Using the obtained second image information, search image information used by a plurality of motion compensation units can be generated.
[0047]
A compression unit configured to compress the second image information acquired by the first acquisition unit; and a decompression unit configured to decompress the second image information compressed by the compression unit. The second image information compressed by the compression unit is stored, and the expansion unit expands the second image information compressed by the compression unit and acquired from the second storage unit by the second acquisition unit. Can be
[0048]
The decoding method according to the present invention includes a search step of searching the first storage unit and the second storage unit for second image information in which motion compensation has been performed on the first image information, which is a decoded bit stream, and When the target second image information is stored in the second storage unit based on the search result obtained by the processing of the search step, the target second image information is acquired from the second storage unit, When the second image information is not stored in the second storage unit but is stored in the first storage unit, control is performed such that the target second image information is obtained from the first storage unit. When the target second image information is acquired from the first storage unit, the second storage unit is controlled by the acquisition control step and the acquisition control step. Second storage control step of storing image information and acquisition control It is controlled by the processing of step, with the second image information acquired, characterized in that it comprises a generation step of generating search image data used for motion compensation of the first image information.
[0049]
According to a second program of the present invention, a search is performed for searching, from a first storage unit and a second storage unit, for second image information obtained by performing motion compensation on first image information that is a decoded bit stream. When the target second image information is stored in the second storage unit based on the search result obtained by the processing of the step and the search step, the target second image information is acquired from the second storage unit. In a case where the target second image information is not stored in the second storage unit but is stored in the first storage unit, the target second image information is acquired from the first storage unit. The second control unit controls the second storage unit when the target second image information is obtained from the first storage unit. A second storage control step of storing image information of the second type; And generating the search image data used for motion compensation of the first image information using the acquired second image information controlled by the processing of the acquisition control step. I do.
[0050]
In the encoding device and method and the first program according to the present invention, motion compensation of the first image information is performed using search image information corresponding to the input first image information, and the motion compensation is performed. The performed first image information is encoded, second image information for generating search image information is generated and stored in the first storage unit, and the second image information stored in the first storage unit is generated. The second image information is obtained and stored in the second storage unit, the second image information stored in the second storage unit is obtained, and the second image information is obtained from the first storage unit or the second storage unit. Search image information is generated using the obtained second image information, and second image information necessary for generating the search image information is searched from the first storage unit or the second storage unit, and the search result is obtained. , The second image information is stored in the second storage unit. In this case, the second image information stored in the second storage unit is obtained, and the second image information is not stored in the second storage unit but is stored in the first storage unit. If there is, the second image information is obtained from the first storage unit.
[0051]
In the decoding device and method and the second program according to the present invention, the input bit stream is decoded, and the motion compensation of the first image information that is the decoded bit stream corresponds to the first image information. The second image information, which is the first image information subjected to the motion compensation and performed using the search image information, is stored in the first storage unit, and the second image information stored in the first storage unit is stored. The image information is acquired, the acquired second image information is stored in the second storage unit, the second image information stored in the second storage unit is acquired, and the first storage unit or the second Search image information is generated using the second image information acquired from the second storage unit, and the second image information necessary for generating the search image information is obtained from the first storage unit and the second storage unit. A second image is searched based on the search result. When the information is stored in the second storage unit, the second image information is obtained from the second storage unit, and the second image information is not stored in the second storage unit. When stored in one storage unit, the second image information is obtained from the first storage unit.
[0052]
BEST MODE FOR CARRYING OUT THE INVENTION
FIG. 5 is a block diagram illustrating a configuration example of an image information encoding device to which the present invention has been applied.
[0053]
In FIG. 5, an image information encoding device 100 is a device that encodes input image data that has been input and outputs it as a bit stream. The image information encoding device 100 compresses input image data by orthogonal transform such as discrete cosine transform and motion compensation using the redundancy specific to image information by the MPEG2 compression method.
[0054]
The MPEG2 (ISO / IEC 13818-2) compression method is a standard defined as a versatile image compression method, and covers both interlaced scan images and progressive scan images, as well as standard resolution images and high definition images. For example, as represented by the DVD (Digital Versatile Disk) standard, it is currently widely used in a wide range of applications for professional use and consumer use.
[0055]
By using the MPEG2 compression method, for example, 4 to 8 Mbps for a standard resolution interlaced scan image having 720 × 480 pixels, and 18 to 22 Mbps for a high resolution interlace scan image having 1920 × 1088 pixels. By assigning a code amount (bit rate), it is possible to realize a high compression rate and good image quality.
[0056]
As in the case of the image information encoding device 10 in FIG. 1, uncompressed input image data input to the image information encoding device 100 is supplied to the image rearrangement processing unit 11.
[0057]
The image rearrangement processing unit 11 converts the input image data into an intra-frame coded image (hereinafter, referred to as an I-picture (Intra-Picture)) and an inter-frame forward prediction coded image (hereinafter, a P-picture (Predictive-Picture)). ) Or a bidirectionally predictive coded image (hereinafter, referred to as a B-picture (Bidirectionally-predictive-Picture)), and designates which image type is to be processed. According to the image type (I picture, P picture, or B picture) of the image, the I picture and the P picture are rearranged in the encoding order of the frame images so as to give priority to the B picture between them. Execute the ordering and move ahead in that order. It is supplied to the measuring unit 12.
[0058]
The motion prediction unit 12 calculates a motion vector for each macro block based on the input image data supplied for each macro block from the image rearrangement processing unit 11 and the search image data supplied from the memory controller 101, and The image data and the motion vector information relating to the calculated motion vector are supplied to the motion compensation unit 13.
[0059]
Note that the motion prediction unit 12 is capable of storing a plurality of frames of image data supplied from the image rearrangement processing unit 11 for each frame, and based on only the input image data supplied from the image rearrangement processing unit 11. Then, the motion vector may be calculated.
[0060]
Based on the input image data and the motion vector information supplied from the motion prediction unit 12 and the search image data supplied from the memory controller 101, the motion compensation unit 13 converts each macroblock , The motion compensation is performed in the intra mode for the I picture and in the forward prediction mode for the P picture.
[0061]
That is, in the case of the intra mode, the motion compensating unit 13 sends the input image data of the input macroblock unit as it is to the DCT processing unit 14 as operation data, and in the case of the forward prediction mode, the input macroblock unit Then, the search image data is subtracted from the input image data, and the calculated difference data as the prediction residual is sent to the DCT processing unit 14.
[0062]
The DCT processing unit 14 performs a two-dimensional discrete cosine transform on the input image data in block units of 8 pixels × 8 pixels, and supplies the obtained DCT coefficients to the quantization unit 15.
[0063]
The quantization unit 15 converts a DCT coefficient supplied from the DCT processing unit 14 into a DC (Direct Current) component and an AC (Alternate Current) component based on a predetermined quantization table stored therein. Are supplied to the VLC unit 16 for further compression, and to the inverse quantization unit 17 for restoring the input image data.
[0064]
The VLC unit 16 performs entropy coding such as Huffman coding or arithmetic coding on the quantized DCT coefficient, compresses the amount of information, and outputs it as a bit stream to the outside of the image information coding apparatus 100. I do.
[0065]
The inverse quantization unit 17 inversely quantizes the quantized DCT coefficient and supplies the approximately restored DCT coefficient to the IDCT processing unit 18, and the IDCT processing unit 18 performs inverse two-dimensional conversion on the restored DCT coefficient. The input image data restored by performing the discrete cosine transform process is supplied to the memory controller 101 as locally decoded image data.
[0066]
The memory controller 101 supplies the supplied local decoded image data to the external memory 21 and stores it. When the memory controller 101 obtains a request for search image data from the motion prediction unit 12 or the motion compensation unit 13, the memory controller 101 accesses the internal memory 102 based on the request, and obtains a locally decoded image including the requested search image data. If the data does not exist, the external memory 21 is accessed, the locally decoded image data including the requested search image data is transferred in units of macroblocks, and stored in a built-in cache (not shown) or the like. To memorize.
[0067]
The external memory 21 is a semiconductor memory composed of a DRAM (Dynamic Random Access Memory) or the like, is connected to the memory controller 101 by a predetermined memory bus, and is controlled by the memory controller 101 to supply the locally decoded image supplied thereto. It accumulates data and supplies requested local decoded image data.
[0068]
Since the external memory 21 is provided outside the image information encoding device 100, the structure of the memory bus connecting the external memory 21 and the memory controller 101 is limited to some extent. Therefore, since the bandwidth of the memory bus is limited, the data transfer rate is also limited.
[0069]
Also, due to the structure of the external memory 21, when transferring the locally decoded image data stored in the external memory 21, the memory controller 101 performs burst transfer of the restored image data in macroblock units to increase the data transfer speed. There is a need.
[0070]
On the other hand, the internal memory 102 is a semiconductor memory provided in the image information encoding device 100 and configured by an SRAM (Static Random Access Memory) or the like, and operates at a higher speed than the external memory 21. However, the storage capacity is smaller than that of the external memory 21 in terms of integration degree and cost.
[0071]
Further, since the bandwidth is provided in the image information encoding device 100, the bandwidth of the memory bus connecting the internal memory 102 and the memory controller 101 can be freely set, and the transfer speed can be easily increased. .
[0072]
The memory controller 101 that has read all the necessary local decoded image data from the external memory 21 or the internal memory 102 as described above generates search image data using the local decoded image data, It is supplied to the section 12 or the motion compensating section 13.
[0073]
If local decoded image data including the requested search image data exists in the internal memory 102, the memory controller 101 reads the local decoded image data from the internal memory 102 without accessing the external memory 21, and Generate and supply the requested search image data.
[0074]
As described above, the memory controller 101 stores the local decoded image data in the external memory 21 having a larger storage capacity than the internal memory 102, transfers the local decoded image data from the external memory 21, and uses the local decoded image data to generate the search image data. The decoded image data is stored in the internal memory 102 which operates faster than the external memory 21.
[0075]
By doing so, the memory controller 101 stores the locally decoded image data in units of macroblocks once transferred from the external memory 21 in the internal memory 102 without discarding them, and uses the data for processing on nearby macroblocks. And the efficiency of the transfer process with the external memory 21 can be improved.
[0076]
The memory control processing in the image information encoding device 100 as described above will be described with reference to the flowcharts of FIGS.
[0077]
First, in step S1, the memory controller 101 of the image information encoding device 100 determines whether or not to end the memory control process, and if not, proceeds to step S2.
[0078]
In step S2, the memory controller 101 determines whether or not to reset the internal memory 102. If it is determined that the internal memory 102 should be reset, the process proceeds to step S3, where the internal controller 102 stores the internal memory 102 in a cache or the like. All internal memory tags, which are a list of data held in the memory 102, are initialized, and the process proceeds to step S4.
[0079]
If it is determined in step S2 that the internal memory 102 is not reset, the memory controller 101 advances the process to step S4.
[0080]
In step S4, the memory controller 101 determines whether or not a transfer request for search image data has been obtained from the motion prediction unit 12 or the motion compensation unit 13. If it is determined that the request has not been obtained, the process proceeds to step S1. And repeat the subsequent steps.
[0081]
If it is determined in step S4 that a transfer request for search image data has been obtained, the memory controller 101 proceeds to step S5, and obtains the external memory 21 so that local decoded image data can be requested in macroblock units. The content of the transfer request is divided into macroblock units, and the process proceeds to step S7 in FIG.
[0082]
In step S7 of FIG. 7, the memory controller 101 determines whether or not all the divided transfer requests in units of macroblocks have been processed. If it is determined that there is an unprocessed transfer request, the process proceeds to step S8. .
[0083]
In step S8, the memory controller 101 selects a transfer request in units of macroblocks to be processed, and in step S9, the memory controller 101 holds local decoded image data of the macroblock corresponding to the selected transfer request. Search from the list of internal memory tags.
[0084]
In step S10, it is determined from the search result whether or not the local decoded image data corresponding to the transfer request exists in the internal memory 102. If it is determined that the local decoded image data does not exist, the process proceeds to step S11 and the external memory From 21, the local decoded image data of the target macroblock is read and held.
[0085]
The memory controller 101 that has acquired the locally decoded image data determines in step S12 whether or not there is free space in the internal memory 102. If it determines that there is no free space, in step S13, the information of the internal memory tag is deleted. One is deleted as necessary, and the free space is secured, and then the process proceeds to step S14.
[0086]
If it is determined in step S12 that there is free space in the internal memory 102, the memory controller 101 skips step S13 and proceeds to step S14.
[0087]
In step S14, the memory controller 101 writes the held locally decoded image data into the internal memory 102. In step S15, after updating the internal memory tag, the process returns to step S7 and repeats the subsequent processes.
[0088]
If it is determined in step S10 that the local decoded image data corresponding to the transfer request exists in the internal memory 102, the memory controller 101 proceeds to step S17, reads the local decoded image data from the internal memory 102, and stores the read data. Then, the process returns to step S7, and the subsequent processes are repeated.
[0089]
If it is determined in step S7 that all transfer requests in units of macroblocks have been processed, the memory controller 101 proceeds to step S16, and converts the requested search image data using the held local decoded image data. It is generated and supplied to the motion prediction unit 12 or the motion compensation unit 13 that is the request source, the process returns to step S1 in FIG. 6, and the subsequent processes are repeated.
[0090]
If it is determined in step S1 in FIG. 6 that the memory control process is to be ended, the memory controller 101 ends the memory control process after performing the end process in step S6.
[0091]
As described above, by using the internal memory 102 to generate and supply the search image data, the memory controller 101 can reduce the amount of data for transferring the local decoded image data from the external memory 21 and can reduce the motion compensation. The used image processing can be executed more efficiently.
[0092]
FIG. 8 is a diagram showing how the memory controller 101 generates the requested search image data by the above-described memory control processing.
[0093]
8, a frame 110 is image data for one frame composed of locally decoded image data stored in the external memory 21 or the internal memory 102. In the first stage, it is assumed that all the image data of the frame 110 is stored in the external memory 21.
[0094]
When the search image data 121 for the object 33 is requested by the motion prediction unit 12 or the motion compensation unit 13, the memory controller 101 transmits the local decoded image data 111 and 112 including the search image data 121 and the local decoded image The data 114 and 115 are transferred and stored in the internal memory 102, and the requested search image data 121 is generated from the data 114 and 115 and supplied to the motion prediction unit 12 or the motion compensation unit 13 of the request source.
[0095]
Next, when the search image data 122 for the object 33 is requested by the motion prediction unit 12 or the motion compensation unit 13, the memory controller 101 sends the local decoded image data 112 including the search image data 122 from the internal memory 102. And 115, and the local decoded image data 113 and 116 including the search image data 122 from the external memory 21.
[0096]
Then, the memory controller 101 stores the local decoded image data 113 and 116 in the internal memory 102, generates the requested search image data 122, and supplies the generated search image data 122 to the motion prediction unit 12 or the motion compensation unit 13 of the request source.
[0097]
Further, when the motion prediction unit 12 or the motion compensation unit 13 requests the search image data 123 for the object 33, the memory controller 101 sends, from the internal memory 102, the local decoded image data 115 including the search image data 123 and At the same time as acquiring 116, the local decoded image data 117 and 118 including the search image data 123 are acquired from the external memory 21.
[0098]
Then, the memory controller 101 stores the local decoded image data 117 and 118 in the internal memory 102, generates the requested search image data 123, and supplies the generated search image data 123 to the motion prediction unit 12 or the motion compensation unit 13 of the request source.
[0099]
As described above, by using the internal memory 102, the memory controller 101 can transfer the local decoded image data including the plurality of search image data such as the local decoded image data 112, 115, and 116 from the external memory 21. Of the search image data can be improved, and the processing time from when the search image data is requested to when the search image data is supplied can be shortened. Accordingly, the image information decoding apparatus 100 can more efficiently execute the image processing using the motion compensation.
[0100]
The case where the image data is encoded has been described above. However, the present invention is not limited to this, and the present invention is also applicable to the case where the encoded image data is decoded, for example.
[0101]
FIG. 9 is a block diagram illustrating an example of the internal configuration of the image information decoding device to which the present invention has been applied.
[0102]
9, an image information decoding device 150 is a decoding device corresponding to the image information encoding device 100 in FIG. 5, and is a bit stream generated by compressing image data by the image information encoding device 100 using the MPEG2 compression method. And restore the image data.
[0103]
The bit stream input to the image information decoding device 150 is supplied to the frame memory 51, and after being stored, is supplied to the VLD unit 52 for each frame.
[0104]
The VLD unit 52 performs lossless decoding (variable-length decoding or arithmetic decoding) on the image compression information based on a predetermined format of the image compression information, and converts the encoding mode information stored in the header unit. Obtained and supplied to the inverse quantization unit 53. Similarly, the VLD unit 52 acquires the quantized transform coefficient and supplies it to the inverse quantization unit 53. Further, when the frame to be decoded is inter-coded, the VLD unit 52 also decodes the motion vector information stored in the header part of the image compression information, and supplies the information to the motion compensation unit 55. I do.
[0105]
The inverse quantization unit 53 inversely quantizes the quantized transform coefficient supplied from the VLD unit 52, and supplies the obtained transform coefficient to the IDCT unit 54. IDCT section 54 performs an inverse discrete cosine transform on the transform coefficient.
[0106]
Here, when the frame of the target image data is coded in the intra mode, the image data on which the inverse discrete cosine transform has been performed is passed through the motion compensation unit 55 as it is to the screen rearrangement processing unit. 56 and supplied to the memory controller 151 as decoded image data.
[0107]
When the target frame is encoded in the inter mode, the motion compensation unit 55 acquires the search image data from the memory controller 151 and supplies the search image data and the search image data from the VLD unit 52. Reference image data is generated based on the motion vector information. Then, the motion compensation unit 55 combines the reference image data with the image data (difference data) supplied from the IDCT processing unit 54, generates image data similar to the intra-mode image data, and rearranges the screen. The data is stored in the processing unit 56 and supplied to the memory controller 151 as decoded image data.
[0108]
The screen rearrangement processing unit 56 rearranges the order of the stored image data for each frame to the original order, and outputs the image data to the outside of the image information decoding device 150 as decoded image data.
[0109]
The memory controller 151 supplied with the decoded image data supplies the decoded image data to the external memory 61 to be stored. When the memory controller 151 obtains a request for search image data from the motion compensation unit 55, the memory controller 151 accesses the internal memory 152 based on the request and finds a macroblock of decoded image data including the requested search image data. If not, the external memory 61 is accessed, the macroblock of the decoded image data including the requested search image data is transferred, stored in a built-in cache (not shown), and stored in the internal memory 152.
[0110]
The external memory 61 is a semiconductor memory composed of a DRAM or the like, is connected to the memory controller 151 via a predetermined memory bus, and is controlled by the memory controller 151 to store decoded image data supplied in macroblock units. Or the requested decoded image data is supplied in macroblock units.
[0111]
Since the external memory 61 is provided outside the image information decoding device 150, the structure of the memory bus connecting the external memory 61 and the memory controller 151 is limited to some extent. Therefore, since the bandwidth of the memory bus is limited, the data transfer rate is also limited.
[0112]
Further, due to the structure of the external memory 61, when transferring the decoded image data stored in the external memory 61, the memory controller 151 needs to burst-decode the decoded image data in macroblock units to increase the speed of data transfer. There is.
[0113]
On the other hand, the internal memory 152 is a semiconductor memory such as an SRAM provided in the image information decoding device 150, and can operate at a higher speed than the external memory 61. In terms of cost and the like, the storage capacity is smaller than that of the external memory 61.
[0114]
Further, since it is provided in the image information decoding device 150, the bandwidth of the memory bus connecting the internal memory 152 and the memory controller 151 can be freely set, and the transfer speed can be easily increased.
[0115]
The memory controller 151 that has read all the necessary macroblocks of the decoded image data from the external memory 61 or the internal memory 152 as described above generates search image data using the decoded image data, and generates the motion compensation request source. To the unit 55.
[0116]
When a macroblock of decoded image data including the requested search image data exists in the internal memory 152, the memory controller 151 reads the decoded image data from the internal memory 152 without accessing the external memory 61. , Generates and supplies the requested search image data.
[0117]
As described above, the memory controller 151 stores the decoded image data in the external memory 61 having a larger storage capacity than the internal memory 152, similarly to the case of the memory controller 101 of the image information encoding device 100 shown in FIG. The decoded image data stored and transferred from the external memory 61 and used for generating the search image data is stored in the internal memory 152 which operates at a higher speed than the external memory 61.
[0118]
In this manner, the memory controller 151 stores the decoded image data in macroblock units once transferred from the external memory 61 in the internal memory 152 without discarding the decoded image data, and uses the decoded image data for processing of nearby macroblocks. The efficiency of the transfer process with the external memory 61 can be improved.
[0119]
The memory control process by the image information decoding device 150 as described above is basically the same as the process in the case of the memory control process by the image information encoding device of FIG. 5 described with reference to the flowcharts of FIGS. Is performed, and the flowcharts of FIGS. 6 and 7 can be applied, and thus the description thereof is omitted.
[0120]
In the above, it has been described that the memory controller 101 of the image information encoding device 100 in FIG. 5 stores the decoded image data read from the external memory 21 in the internal memory 102 as it is. Similarly, the description has been given such that the memory controller 151 of the image information decoding device 150 in FIG. 9 also stores the decoded image data read from the external memory 61 in the internal memory 152 as it is.
[0121]
However, the present invention is not limited to this, and the decoded image data read from the external memory 21 (the external memory 61) is compressed in the memory controller 101 (the memory controller 151) and then stored in the internal memory 102 (the internal memory 152). You may make it so.
[0122]
As described above, the internal memory 102 (the internal memory 152) has a lower degree of integration and a higher manufacturing cost than the external memory 21 (the external memory 61). ) Less storage capacity. Therefore, for example, even if the restored image data is read out once from the external memory 21 (the external memory 61) and stored in the internal memory 102 (the internal memory 152), it is updated by the subsequent update of the internal memory 102 (the internal memory 152). In some cases, the restored image data is erased.
[0123]
In such a case, if the restored image data is needed again, the memory controller 101 (memory controller 151) must read the restored image data again from the external memory 21 (external memory 61).
[0124]
However, since the internal memory 102 (the internal memory 152) operates at a higher speed and has a higher transfer efficiency than the external memory 21 (the external memory 61), the restored image stored in the internal memory 102 (the internal memory 152) is as much as possible. The data amount of the data is increased, and the memory controller 101 (memory controller 151) uses the restored image data stored in the internal memory 102 (internal memory 152). It is desirable to reduce the amount of data) and to minimize the occurrence frequency of the cases described above.
[0125]
To obtain such an effect, the main controller 101 (main controller 151) reversibly compresses the restored image data read from the external memory 21 (external memory 61), and then compresses the restored image data into the internal memory 102 (internal memory 152). To memorize.
[0126]
The memory control processing in such a case by the image information encoding device 100 in FIG. 5 will be described with reference to the flowcharts in FIGS. Description will be made with reference to FIG. 12 as necessary.
[0127]
The memory controller 101 of the image information encoding device 100 executes the processing of steps S31 to S36 as in the case of steps S1 to S6 in FIG.
[0128]
That is, in step S31, the memory controller 101 determines whether or not to end the memory control process, and when not, in step S32, determines whether or not to reset the internal memory 102, and when it is determined to reset. In step S33, all internal memory tags are initialized, and in step S34, it is determined whether or not a transfer request for image data has been obtained from the motion prediction unit 12 or the motion compensation unit 13, and if it is determined that the transfer request has been obtained, In step S35, the transfer request is divided into macroblock units, and the process proceeds to step S37 in FIG.
[0129]
If it is determined in step S32 that the internal memory 102 is not to be reset, the memory controller 101 skips step S33 and proceeds to step S34.
[0130]
If it is determined in step S31 that the process is to be ended, the memory controller 101 proceeds to step S36, performs an end process, and ends the memory control process.
[0131]
The memory controller 101 that has proceeded to step S37 of FIG. 11 executes the processing of steps S37 to S41, similarly to the case of steps S7 to S11 of FIG.
[0132]
That is, the memory controller 101 determines in step S37 whether or not all transfer requests in units of macroblocks have been processed. If it is determined that there is an unprocessed transfer request, the memory controller 101 determines in step S38 that the transfer request to be processed is In step S39, the local decoded image data corresponding to the selected transfer request is searched from the internal memory tag. In step S40, it is determined whether or not the local decoded image data corresponding to the transfer request exists in the internal memory 102. If it is determined that it does not exist, in step S41, the external memory 21 is accessed to read and hold the target local decoded image data.
[0133]
Then, in step S42, the memory controller 101 compresses the obtained locally decoded image data.
[0134]
FIG. 12 is a diagram illustrating a state in which the locally decoded image data obtained from the external memory 21 is compressed.
[0135]
In FIG. 12A, a block 161 is read from the external memory 21 and constitutes locally decoded image data held by the microcomputer 101, and is composed of 8 pixels × 8 pixels (P (0,0) to P (7,7) )).
[0136]
The microcomputer 101 performs the following compression processing on the block 161 described above.
[0137]
(Equation 1)
[0138]
That is, the memory controller 101 leaves the pixel value of the pixel P (0,0) at the upper left end of the block 161 as it is (Equation (1)), and the pixels P (0,1) to P (0,0) at the left end of the block 161. The pixel values of the pixels P (0,0) to P (0,6) one above are subtracted from the pixel values of (7) (Equation (2)), and the other pixels P (1,0) to P (1,0) P (7,0), P (1,1) to P (7,1), P (1,2) to P (7,2), P (1,3) to P (7,3), P (1,4) to P (7,4), P (1,5) to P (7,5), P (1,6) to P (7,6), P (1,7) to P ( 7, 7), the pixels P (0,0) to P (6,0), P (0,1) to P (6,1), P (0,2) one pixel to the left To P (6,2), P (0,3) to P (6,3) P (0,4) to P (6,4), P (0,5) to P (6,5), P (0,6) to P (6,6), P (0,7) to P The pixel value of (6, 7) is subtracted (formula (3)) to generate a block 162 shown in FIG. 12B.
[0139]
As described above, the block 162 calculates the pixel Q (0,0) having the same pixel value as the pixel P (0,0) at the upper left corner of the original block 161 and the difference value between the pixel values of the adjacent pixels. Pixels Q (1,0) to Q (7,0), Q (0,1) to Q (7,1), Q (0,2) to Q (7,2), Q (0, 3) to Q (7,3), Q (0,4) to Q (7,4), Q (0,5) to Q (7,5), Q (0,6) to Q (7,6) ), Q (0,7) to Q (7,7).
[0140]
As described above, the memory controller 101 determines that the pixel value of each pixel P (i, j) of the block 161 is 8 bits and the data amount of the block 161 is 512 bits in step S42 of FIG. If so, the block 161 is compressed with the goal of reducing the data amount of the compressed block 162 to half or less of 256 bits.
[0141]
Then, in step S43, the memory controller 101 determines whether or not the locally decoded image data has been losslessly compressed at the target compression ratio by the above-described compression processing.
[0142]
As a result of the compression processing in step S42, when it is determined that the data amount after compression has become smaller than the predetermined data amount and the local decoded image data has been losslessly compressed at the target compression ratio, the memory controller 101 proceeds to step S44. In the same manner as in step S12 in FIG. 7, it is determined whether or not there is free space in the internal memory 102. If it is determined that there is no free space, in step S45, the process proceeds to step S13 in FIG. Similarly to the above, the information of the internal memory tag is deleted from the oldest one, and in step S46, the locally decoded image data compressed by the processing in step S42 is written in the internal memory 102, and in step S47, the case of step S15 in FIG. Similarly, after updating the internal memory tag, the process returns to step S37, and the subsequent processes are repeated. Return.
[0143]
If it is determined in step S44 that there is free space in the internal memory 102, the memory controller 101 omits the processing in step S45, proceeds to step S46, and stores the compressed local decoded image data in the internal memory 102. Write.
[0144]
If it is determined in step S43 that the lossless compression has not been performed at the target compression ratio, the memory controller 101 skips the processing of steps S44 to S47, returns to step S37, and repeats the subsequent processing.
[0145]
If it is determined in step S40 that the local decoded image data corresponding to the transfer request exists in the internal memory 102, the memory controller 101 advances the processing to step S49, and performs the processing similar to the case of step S17 in FIG. From the memory 102, the compressed local decoded image data corresponding to the transfer request is read.
[0146]
Then, in step S50, the memory controller 101 expands the compressed local decoded image data obtained from the internal memory 102 by performing the reverse of the method described with reference to FIG. Restore and retain.
[0147]
That is, the memory controller 102 performs the following operation on each block of the locally decoded image data (the block 162 in FIG. 12B) acquired from the internal memory 102, and performs each of the blocks of the original locally decoded image data (FIG. 12A). Block 161) is restored.
[0148]
(Equation 2)
[0149]
The memory controller 101 having performed the operations of the above equations (4) to (6) and restoring the block 161 from the block 162 returns the process to step S37, and repeats the subsequent processes.
[0150]
If it is determined in step S37 that all transfer requests in units of macroblocks have been processed, the memory controller 101 proceeds to step S48, generates the requested search image data, and outputs the requested motion prediction unit. 12 or to the motion compensator 13.
[0151]
By performing the memory control process as described above, the memory controller 101 can store more locally decoded image data in the internal memory 102, and thus can use the internal memory 102 more effectively. .
[0152]
In the above, one example of the compression method of the locally decoded image data by the memory controller 101 has been described. However, the compression method is not limited to this, and may be any method. In this case, in order to suppress the image quality deterioration of the image data, it is better to use a reversible compression method.
[0153]
Further, as described above, the method of compressing the decoded image data obtained from the external memory and storing it in the internal memory can be applied to the image information decoding device 150 in FIG.
[0154]
The memory controller 151 of the image information decoding device 150 in FIG. 9 performs basically the same process as the memory control process by the memory controller 101 of the image information decoding device 101 in FIG. 5 described above. The flowcharts of FIGS. 10 and 11 can be applied to the control processing. Therefore, the description is omitted.
[0155]
In the above, a case has been described in which one motion prediction unit or motion compensation unit requests search image data for one memory controller. However, the present invention is not limited to this, and the present invention is illustrated in FIGS. As described above, the present invention can be applied to a case where a plurality of motion prediction units or motion compensation units request search image data for one memory controller.
[0156]
FIG. 13 is a block diagram illustrating a configuration example of an image information encoding device including two encoding processing units each encoding input image data.
[0157]
13, the image information coding apparatus 200 has the same configuration as the motion prediction section 12 of the image information coding apparatus 100 described with reference to FIG. 5, and operates in the same manner as the motion prediction sections 212A and 212B. , Have the same configuration as the motion compensation unit 13 and operate similarly, and have the same configuration as the motion compensation units 213A and 213B, the DCT processing unit 14, and the DCT processing units 214A and 214B Quantizing sections 215A and 215B, which operate in the same manner as quantization section 15, and have the same configuration as VLC section 16, and operate in the same manner as VLC sections 216A and 216B, and inverse quantization section 17 And the same configuration as the inverse quantization units 217A and 217B and the IDCT processing unit 18 that operate similarly. And a IDCT processor 218A and 218B operate similarly.
[0158]
That is, the image information encoding device 200 includes an encoding processing unit including a motion prediction unit 212A, a motion compensation unit 213A, a DCT processing unit 214A, a quantization unit 215A, a VLC unit 216A, an inverse quantization unit 217A, and an IDCT unit 218A. 203A and an encoding processing unit 203B including a motion prediction unit 212B, a motion compensation unit 213B, a DCT processing unit 214B, a quantization unit 215B, a VLC unit 216B, an inverse quantization unit 217B, and an IDCT unit 218B. The image rearrangement processing unit 211 rearranges input image data input from outside the image information encoding device 200 in a predetermined order for each frame, and converts the input image data into blocks, macroblocks, and frames. , Slice, or GOP (Group of Pictures) etc. Supplies distributed to the 3A or 203B.
[0159]
The encoding units 203A and 203B operate in parallel with each other, respectively, compress the supplied input image data by the MPEG system, and perform image information encoding as a bit stream, as in the case described with reference to FIG. Output to the outside of the device 200.
[0160]
In addition, the image information encoding device 200 has a memory controller 201 and an internal memory 202. The memory controller 201 acquires the local decoded image data from the IDCT processing unit 218A for the encoding processing of the encoding processing unit 203A, accumulates the locally decoded image data in the external memory 21, and performs the motion prediction Alternatively, based on a request from the motion compensating unit 213A, it obtains decoded image data from the external memory 21 or the internal memory 202, generates search image data, and supplies the search image data to the motion predicting unit 212A or the motion compensating unit 213A of the request source.
[0161]
When acquiring the decoded image data from the external memory 21, the memory controller 201 stores the decoded image data in the internal memory 202.
[0162]
Similarly, the memory controller 201 acquires the local decoded image data from the IDCT processing unit 218B and stores it in the external memory 21 for the encoding processing of the encoding processing unit 203B, and also performs the motion prediction unit 212B or the motion compensation. Based on the request of the unit 213B, the image data acquisition unit acquires the decoded image data from the external memory 21 or the internal memory 202, generates the search image data, and supplies the search image data to the motion prediction unit 212B or the motion compensation unit 213B of the request source. Then, when the decoded image data is obtained from the external memory 21, the memory controller 201 stores the decoded image data in the internal memory 202.
[0163]
That is, the memory controller 201 executes the same memory control processing as described with reference to the flowcharts of FIGS. 6 and 7 for each of the encoding processing units 203A and 203B. Therefore, the description of the details of the memory control processing by the memory controller 201 is omitted.
[0164]
As described above, since the image information encoding device 200 includes a plurality of encoding processing units that operate in parallel, the image information encoding device 200 can perform the compression processing of the input image data at higher speed. For example, the image information encoding device 200 may include an HDTV (High Definition). Even when input image data having a large data amount, such as image data for TeleVision, is processed in real time, compression can be performed without causing an error such as dropped frames.
[0165]
In the case of such an image information encoding device 200 having a plurality of encoding processing units (when there are a plurality of motion prediction units, motion compensation units, or IDCT processing units), the image information encoding device 100 of FIG. As compared with the case, the number of times of acquiring the local decoded image data or the number of times of requesting the search image data increases, so that the transfer amount of the local decoded image data performed by the memory controller 201 with the external memory 21 also increases, Overflow is likely to occur.
[0166]
Therefore, in such a case, by using the internal memory 202 as described above, the memory controller 201 suppresses an increase in the amount of image data transferred to and from the external memory 21 and suppresses the frequency of occurrence of overflow and the like. The effect is greater than in the case of the image information encoding device 100 of FIG. 5 having one encoding processing unit.
[0167]
In the above description, the image information encoding apparatus having two encoding processing units has been described. However, the present invention is not limited to this. May be any number.
[0168]
In addition, the main controller 201 of the image information encoding device 200 executes the memory control process described with reference to the flowcharts of FIGS. May be stored in the internal memory 202 after compression.
[0169]
In this case, the memory controller 201 executes the same memory control processing as described with reference to the flowcharts of FIGS. 10 and 11 for each of the encoding processing units 203A and 203B. Therefore, a detailed description of the memory control process performed by the memory controller 201 when compressing the locally decoded image data acquired from the external memory 21 and storing the compressed data in the internal memory 202 will be omitted.
[0170]
FIG. 14 is a block diagram illustrating a configuration example of an image information decoding device including two decoding processing units that decode input bit streams.
[0171]
14, the image information decoding device 250 has the same configuration as the VLD unit 52 of the image information decoding device 150 described with reference to FIG. 9, and operates in the same manner as the VLD units 262A and 262B, The inverse quantization units 263A and 263B, which have the same configuration as the unit 53 and operate similarly, and the IDCT processing units 264A and 264B which have the same configuration as the IDCT processing unit 54 and operate similarly, It has the same configuration as the compensator 55, and includes motion compensators 265A and 265B that operate similarly.
[0172]
That is, the image information decoding device 250 includes a decoding processing unit 253A including a VLD unit 262A, an inverse quantization unit 263A, an IDCT processing unit 264A, and a motion compensation unit 265A, a VLD unit 262B, an inverse quantization unit 263B, and an IDCT processing. And a decoding processing unit 253B including a decoding unit 253B including a motion compensation unit 265B and a decoding processing unit 253B. The input bit stream is distributed and supplied to the decoding processing unit 253A or 253B for each frame.
[0173]
The decoding processing units 253A and 253B operate in parallel with each other, and respectively perform the image information encoding device 200 supplied from the image information encoding device 200 in FIG. 13 in the same manner as described with reference to FIG. Expands the compressed bit stream according to the MPEG method and supplies it to the image rearrangement processing unit 266. The supplied image rearrangement processing unit 266 rearranges the supplied image data in a predetermined order for each frame, and outputs the rearranged image data to the outside of the image information decoding device 250 as decoded image data. .
[0174]
The image information decoding device 250 has a memory controller 251 and an internal memory 252. The memory controller 251 acquires the local decoded image data from the motion compensating unit 265A for the encoding processing of the decoding processing unit 253A, accumulates the locally decoded image data in the external memory 61, and stores the data in the motion compensating unit 265A. Based on the request, it obtains decoded image data from the external memory 61 or the internal memory 252, generates search image data, and supplies the search image data to the motion compensating unit 265A of the request source.
[0175]
When acquiring the decoded image data from the external memory 61, the memory controller 251 stores the decoded image data in the internal memory 252.
[0176]
Similarly, the memory controller 251 acquires the local decoded image data from the motion compensating unit 265B and stores it in the external memory 61 based on the request of the motion compensating unit 265B for the encoding process of the decoding processing unit 253B. Then, it obtains the decoded image data from the external memory 61 or the internal memory 252, generates the search image data, and supplies the search image data to the motion compensating unit 265B of the request source. Then, when the decoded image data is obtained from the external memory 61, the memory controller 251 stores the decoded image data in the internal memory 252.
[0177]
That is, the memory controller 251 executes the same memory control processing as described with reference to the flowcharts of FIGS. 6 and 7 for each of the decoding processing units 253A and 253B. Therefore, the description of the details of the memory control processing by the memory controller 251 is omitted.
[0178]
As in the case of the image information encoding device 200, the image information decoding device 250 includes a plurality of encoding processing units that operate in parallel, so that it is possible to perform compression processing of input image data at higher speed. For example, even when input image data having a large data amount, such as image data for HDTV, is processed in real time, it can be expanded without generating an error such as dropped frames.
[0179]
In the case of such an image information decoding device 250 having a plurality of decoding processing units (when there are a plurality of motion compensation units), local decoded image data is obtained as compared with the case of the image information decoding device 150 in FIG. Since the number of times the search image data is requested or the number of times the search image data is requested increases, the transfer amount of the decoded image data performed by the memory controller 251 with the external memory 61 also increases, and overflow easily occurs.
[0180]
Therefore, in such a case, by using the internal memory 252 as described above, the memory controller 251 suppresses an increase in the amount of image data transferred to and from the external memory 61 and suppresses the frequency of occurrence of overflow and the like. The effect is greater than in the case of the image information decoding device 150 of FIG. 9 having one decoding processing unit.
[0181]
In the above description, an image information decoding device having two decoding processing units has been described. However, the present invention is not limited to this. How many decoding processing units of the image information decoding device execute image processing in parallel with each other are described. It may be.
[0182]
Further, the main controller 251 of the image information decoding device 250 compresses the decoded image data acquired from the external memory 61 by a predetermined method by executing the memory control process described with reference to the flowcharts of FIGS. After that, it may be stored in the internal memory 252.
[0183]
In that case, the memory controller 251 executes the same memory control processing as described with reference to the flowcharts of FIGS. 10 and 11 for each of the decoding processing units 253A and 253B. Therefore, a detailed description of the memory control process performed by the memory controller 251 when compressing the decoded image data acquired from the external memory 61 and storing the compressed image data in the internal memory 252 will be omitted.
[0184]
In the above, the image information encoding devices 100 and 200 and the image information decoding devices 150 and 250 have been described as being configured as one device, respectively. However, the present invention is not limited to this, and has all the functions described above. It may be configured as a single device, or may be realized as a system configured with a plurality of devices.
[0185]
The image information encoding devices 100 and 200 and the image information decoding devices 150 and 250 are each configured as a module, and configured as one device including the external memory 21 or 61. Is also good.
[0186]
Note that the series of processes described above can be executed by hardware or can be executed by software. In this case, for example, the image information encoding device 100 in FIG. 5, the image information decoding device 150 in FIG. 9, the image information encoding device 200 in FIG. 13, or the image information decoding device 250 in FIG. The personal computer 300 is configured as described above.
[0187]
15, a CPU (Central Processing Unit) 301 performs various processes in accordance with a program stored in a ROM (Read Only Memory) 302 or a program loaded from a storage unit 313 into a RAM (Random Access Memory) 303. Execute. The RAM 303 also appropriately stores data necessary for the CPU 301 to execute various processes.
[0188]
The CPU 301, the ROM 302, and the RAM 303 are mutually connected via a bus 304. The input / output interface 310 is also connected to the bus 304.
[0189]
The input / output interface 310 includes an input unit 311 including a keyboard and a mouse, a display including a CRT (Cathode Ray Tube), an LCD (Liquid Crystal Display), an output unit 312 including a speaker, a hard disk, and the like. A communication unit 314 including a storage unit 313, a modem, a terminal adapter, and the like is connected. The communication unit 314 performs communication processing via a network (not shown).
[0190]
A drive 315 is connected to the input / output interface 310 as necessary, and a removable disk 316 composed of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately mounted, and a computer program read therefrom is read. Is installed in the storage unit 313 as necessary.
[0191]
When a series of processing is executed by software, a program constituting the software may execute various functions by installing a computer built into dedicated hardware or installing various programs. It is installed in a possible, for example, a general-purpose personal computer from a network or a recording medium.
[0192]
As shown in FIG. 15, the recording medium is distributed separately from the apparatus main body to provide the user with the program, and is recorded on a magnetic disk (including a flexible disk) on which the program is recorded, or on an optical disk (CD-ROM). Not only is it configured by a removable disk 316 including a ROM (Compact Disk-Read Only Memory), a DVD (including a Digital Versatile Disk), a magneto-optical disk (including an MD (Mini-Disk)), or a semiconductor memory, but also The storage unit 313 includes a ROM 302 in which a program is recorded, which is provided to a user in a state where the program is incorporated in the apparatus main body in advance, and a hard disk included in the storage unit 313.
[0193]
In this specification, a step of describing a program recorded on a recording medium may be performed in chronological order according to the described order, or may be performed in parallel or not necessarily in chronological order. This also includes processes executed individually.
[0194]
【The invention's effect】
As described above, according to the present invention, image processing can be performed. In particular, image processing using motion compensation can be executed more efficiently.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a configuration example of a conventional image information encoding device.
FIG. 2 is a schematic diagram showing how a motion vector is calculated.
FIG. 3 is a schematic diagram showing how to generate search image data.
FIG. 4 is a block diagram illustrating a configuration example of a conventional image information decoding device.
FIG. 5 is a block diagram illustrating a configuration example of an image information encoding device to which the present invention has been applied.
FIG. 6 is a flowchart illustrating an example of a memory control process.
FIG. 7 is a flowchart illustrating an example of a memory control process, continued from FIG. 6;
FIG. 8 is a schematic diagram showing how a search image data is generated by the memory controller of FIG. 5;
FIG. 9 is a block diagram illustrating a configuration example of an image information decoding device to which the present invention has been applied.
FIG. 10 is a flowchart illustrating another example of the memory control process.
FIG. 11 is a flowchart illustrating another example of the memory control process.
FIG. 12 is a schematic diagram illustrating a state of blocks of decoded image data to be compressed.
FIG. 13 is a block diagram illustrating another configuration example of an image information encoding device to which the present invention has been applied.
FIG. 14 is a block diagram illustrating another configuration example of an image information decoding device to which the present invention has been applied.
FIG. 15 is a block diagram illustrating a configuration example of a personal computer to which the present invention has been applied.
[Explanation of symbols]
Reference Signs List 21 external memory, 61 external memory, 100 image information encoding device, 101 memory controller, 102 internal memory, 110 frames, 111 to 118 locally decoded image data, 121 to 123 search image data, 150 image information decoding device, 151 memory controller , 152 internal memory, 161 and 162 blocks, 200 image information encoding device, 201 memory controller, 202 internal memory, 203A and 203B encoding processing section, 250 image information decoding device, 251 memory controller, 252 internal memory, 253A and 253B Decryption processing unit

Claims (22)

  1. In an encoding device that encodes input first image information using motion compensation,
    A motion compensation unit that performs motion compensation on the first image information using search image information corresponding to the input first image information;
    Encoding means for encoding the first image information on which the motion compensation has been performed by the motion compensation means;
    First generating means for generating second image information for generating the search image information;
    First storage means for storing the second image information generated by the first generation means,
    First acquisition means for acquiring the second image information stored by the first storage means;
    A second storage unit that operates faster than the first storage unit and stores the second image information acquired by the first acquisition unit;
    Second acquisition means for acquiring the second image information stored by the second storage means;
    A second generating unit configured to generate the search image information using the second image information obtained by the first obtaining unit or the second obtaining unit;
    A search unit for searching the second storage unit and the second storage unit for the second image information necessary for generating the search image information by the second generation unit,
    The second acquisition unit, when the second image information is stored in the second storage unit based on a search result by the search unit, stores the second image information in the second storage unit. Get information,
    The first acquisition unit is configured to store the second image information based on a search result obtained by the search unit and not to be stored in the second storage unit but to be stored in the first storage unit; Encoding apparatus for acquiring the second image information from the first storage means.
  2. The encoding apparatus according to claim 1, wherein the motion compensation unit performs motion compensation on the first image information using the search image information generated by the second generation unit.
  3. A calculating unit configured to calculate a motion compensation vector of the attention area based on the additional information added to the input first image information;
    The encoding device according to claim 1, wherein the calculation unit calculates the motion compensation vector using the search image information generated by the second generation unit.
  4. 2. The encoding apparatus according to claim 1, wherein the encoding unit orthogonally transforms the first image information and encodes the data subjected to the quantization processing.
  5. The first generation unit performs an orthogonal transform on the first image information, performs an inverse quantization process on the data subjected to the quantization process, performs an inverse orthogonal transform process on the data, and converts the second image information into The encoding device according to claim 1, wherein the encoding device generates the information.
  6. 2. The encoding apparatus according to claim 1, wherein the first storage unit is capable of storing a larger amount of information than the second storage unit.
  7. 2. The encoding apparatus according to claim 1, wherein the first acquiring unit acquires the second image information at a lower speed than the second acquiring unit. 3.
  8. Compression means for compressing the second image information acquired by the first acquisition means;
    A decompression unit for decompressing the second image information compressed by the compression unit,
    The second storage unit stores the second image information compressed by the compression unit,
    2. The apparatus according to claim 1, wherein the expansion unit expands the second image information acquired by the second acquisition unit from the second storage unit and compressed by the compression unit. 3. Encoding device.
  9. A plurality of motion compensation units that operate in parallel and perform motion compensation of the first image information different from each other using the different search image information;
    A plurality of the encoding means operating in parallel, the plurality of the first image information different from each other, the plurality of the first image information is subjected to the motion compensation by the plurality of the motion compensation means,
    A plurality of first generating means for generating second image information for generating the search image information used by the corresponding motion compensating means,
    The first storage unit stores the second image information generated by the plurality of first generation units,
    The second generating unit generates the search image information used by the plurality of motion compensating units using the second image information obtained by the first obtaining unit or the second obtaining unit. The encoding device according to claim 1, wherein the encoding is performed.
  10. Compression means for compressing the second image information acquired by the first acquisition means;
    A decompression unit for decompressing the second image information compressed by the compression unit,
    The second storage unit stores the second image information compressed by the compression unit,
    10. The apparatus according to claim 9, wherein the expansion unit expands the second image information acquired by the second acquisition unit from the second storage unit and compressed by the compression unit. 11. Encoding device.
  11. A first storage unit that stores second image information for generating search image information used for motion compensation of the input first image information; and a second storage unit that has a higher transfer speed than the first storage unit. 2. An encoding method for an encoding device, comprising: a storage unit for encoding the first image information using motion compensation
    A search step of searching the first storage unit and the second storage unit for second image information for generating search image information used for motion compensation of the input first image information;
    When the target second image information is stored in the second storage unit based on a search result obtained by the processing of the search step, the target second image information is stored in the second storage unit. If the acquired second image information is not stored in the second storage unit but is stored in the first storage unit, the target second image information is stored in the first storage unit. An acquisition control step of controlling to acquire image information of
    When the target second image information is acquired from the first storage unit, the second image unit is controlled by the process of the acquisition control step to control the second storage unit to acquire the acquired second image information. A storage control step of storing information;
    A generating step of generating the search image information using the acquired second image information, which is controlled by the processing of the acquisition control step.
  12. A process of encoding the input first image information using motion compensation is performed by a computer having a first storage unit and a second storage unit having a higher transfer speed than the first storage unit. In the program
    A search step of searching the first storage unit and the second storage unit for second image information for generating search image information used for motion compensation of the input first image information;
    When the target second image information is stored in the second storage unit based on a search result obtained by the processing of the search step, the target second image information is stored in the second storage unit. If the acquired second image information is not stored in the second storage unit but is stored in the first storage unit, the target second image information is stored in the first storage unit. An acquisition control step of controlling to acquire image information of
    When the target second image information is acquired from the first storage unit, the second image unit is controlled by the process of the acquisition control step to control the second storage unit to acquire the acquired second image information. A storage control step of storing information;
    A generation step of generating the search image information using the acquired second image information, which is controlled by the processing of the acquisition control step.
  13. In a decoding device that decodes a bit stream that is encoded image information using motion compensation,
    Decoding means for decoding the input bit stream;
    Motion compensation means for performing motion compensation of the first image information which is the bit stream decoded by the decoding means, using search image information corresponding to the first image information;
    A first storage unit that stores second image information that is the first image information subjected to motion compensation by the motion compensation unit;
    First acquisition means for acquiring the second image information stored by the first storage means;
    A second storage unit that operates faster than the first storage unit and stores the second image information acquired by the first acquisition unit;
    Second acquisition means for acquiring the second image information stored by the second storage means;
    Generating means for generating the search image information using the second image information obtained by the first obtaining means or the second obtaining means;
    A search unit that searches the second storage unit for the second image information necessary for generation of the search image information by the generation unit,
    The second acquisition unit, when the second image information is stored in the second storage unit based on a search result by the search unit, stores the second image information in the second storage unit. Get information,
    The first acquisition unit is configured to store the second image information based on a search result obtained by the search unit, without storing the second image information in the second storage unit; And decoding the second image information from the first storage means.
  14. 14. The decoding apparatus according to claim 13, wherein the motion compensation unit performs motion compensation on the first image information using the search image information generated by the generation unit.
  15. 14. The decoding device according to claim 13, wherein the decoding unit decodes the bit stream, performs an inverse orthogonal transform, and performs an inverse quantization process.
  16. 14. The decoding device according to claim 13, wherein the first storage unit can store a larger amount of information than the second storage unit.
  17. 14. The decoding device according to claim 13, wherein the first obtaining unit obtains the second image information at a lower speed than the second obtaining unit.
  18. Compression means for compressing the second image information acquired by the first acquisition means;
    A decompression unit for decompressing the second image information compressed by the compression unit,
    The second storage unit stores the second image information compressed by the compression unit,
    14. The apparatus according to claim 13, wherein the expansion unit expands the second image information acquired by the second acquisition unit from the second storage unit and compressed by the compression unit. 15. Decoding device.
  19. A plurality of decoding means that operate in parallel and decode the plurality of different bit streams,
    Operating in parallel, using a plurality of different search image information, a plurality of the motion compensation means for performing motion compensation of the first image information different from each other,
    The first storage unit stores the second image information on which motion compensation has been performed by the plurality of motion compensation units,
    The generating unit may use the second image information obtained by the first obtaining unit or the second obtaining unit to generate the search image information used by a plurality of the motion compensation units. The decoding device according to claim 13, wherein
  20. Compression means for compressing the second image information acquired by the first acquisition means;
    A decompression unit for decompressing the second image information compressed by the compression unit,
    The second storage unit stores the second image information compressed by the compression unit,
    20. The expansion device according to claim 19, wherein the expansion unit expands the second image information acquired by the second acquisition unit from the second storage unit and compressed by the compression unit. Decoding device.
  21. A decoding device that has a first storage unit and a second storage unit that has a higher transfer speed than the first storage unit, and decodes a bit stream that is encoded image information using motion compensation. A decoding method,
    A retrieval step of retrieving, from the first storage unit and the second storage unit, second image information obtained by performing motion compensation on the first image information that is the decoded bit stream;
    When the target second image information is stored in the second storage unit based on a search result obtained by the processing of the search step, the target second image information is stored in the second storage unit. If the acquired second image information is not stored in the second storage unit but is stored in the first storage unit, the target second image information is stored in the first storage unit. An acquisition control step of controlling to acquire image information of
    When the target second image information is acquired from the first storage unit, the second image unit is controlled by the process of the acquisition control step to control the second storage unit to acquire the acquired second image information. A second storage control step of storing information;
    A generating step of generating search image data used for motion compensation of the first image information, using the obtained second image information, which is controlled by the processing of the obtaining control step. The decryption method to use.
  22. A process of decoding a bit stream that is encoded image information using motion compensation is performed by a computer having a first storage unit and a second storage unit that has a higher transfer speed than the first storage unit. In the program to be performed,
    A retrieval step of retrieving, from the first storage unit and the second storage unit, second image information obtained by performing motion compensation on the first image information that is the decoded bit stream;
    When the target second image information is stored in the second storage unit based on a search result obtained by the processing of the search step, the target second image information is stored in the second storage unit. If the acquired second image information is not stored in the second storage unit but is stored in the first storage unit, the target second image information is stored in the first storage unit. An acquisition control step of controlling to acquire image information of
    When the target second image information is acquired from the first storage unit, the second image unit is controlled by the process of the acquisition control step to control the second storage unit to acquire the acquired second image information. A second storage control step of storing information;
    A generating step of generating search image data used for motion compensation of the first image information, using the obtained second image information, which is controlled by the processing of the obtaining control step. Program to do.
JP2003000819A 2003-01-07 2003-01-07 Encoding device and method, decoding device and method, and program Granted JP2004215049A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2003000819A JP2004215049A (en) 2003-01-07 2003-01-07 Encoding device and method, decoding device and method, and program

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2003000819A JP2004215049A (en) 2003-01-07 2003-01-07 Encoding device and method, decoding device and method, and program

Publications (2)

Publication Number Publication Date
JP2004215049A true JP2004215049A (en) 2004-07-29
JP2004215049A5 JP2004215049A5 (en) 2006-02-09

Family

ID=32818998

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2003000819A Granted JP2004215049A (en) 2003-01-07 2003-01-07 Encoding device and method, decoding device and method, and program

Country Status (1)

Country Link
JP (1) JP2004215049A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007043421A (en) * 2005-08-02 2007-02-15 Matsushita Electric Ind Co Ltd Motion detector
WO2007055013A1 (en) * 2005-11-11 2007-05-18 Fujitsu Limited Image decoding apparatus and method, and image encoding apparatus
US8681862B2 (en) 2007-05-22 2014-03-25 Panasonic Corporation Moving picture decoding apparatus and moving picture decoding method

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007043421A (en) * 2005-08-02 2007-02-15 Matsushita Electric Ind Co Ltd Motion detector
JP4570532B2 (en) * 2005-08-02 2010-10-27 パナソニック株式会社 Motion detection device, motion detection method, integrated circuit, and program
WO2007055013A1 (en) * 2005-11-11 2007-05-18 Fujitsu Limited Image decoding apparatus and method, and image encoding apparatus
JPWO2007055013A1 (en) * 2005-11-11 2009-04-30 富士通株式会社 Image decoding apparatus and method, and image encoding apparatus
JP4675383B2 (en) * 2005-11-11 2011-04-20 富士通株式会社 Image decoding apparatus and method, and image encoding apparatus
US8681862B2 (en) 2007-05-22 2014-03-25 Panasonic Corporation Moving picture decoding apparatus and moving picture decoding method

Similar Documents

Publication Publication Date Title
US9357222B2 (en) Video device finishing encoding within the desired length of time
US20200099929A1 (en) Moving image coding device, moving image decoding device, moving image coding/decoding system, moving image coding method and moving image decoding method
ES2389718T3 (en) Decoding of a block of quantified transform coefficients, where an even parity of the sum of the inverse quantized transform coefficients is adjusted to an odd parity
JP4440640B2 (en) Image block decoding method
KR101227667B1 (en) Piecewise processing of overlap smoothing and in-loop deblocking
US8767819B2 (en) Moving picture encoding apparatus
JP4082525B2 (en) System and method for performing motion estimation with improved efficiency in the DCT domain
JP3149418B2 (en) Image prediction decoding method and apparatus
JP4384130B2 (en) Video decoding method and apparatus
KR100376607B1 (en) Mpeg video decoder with integrated scaling and display functions
JP5017825B2 (en) Decoding device and decoding method
TWI382764B (en) Scratch pad for storing intermediate loop filter data
US7266148B2 (en) Video transcoding apparatus
US7489729B2 (en) Method and apparatus for implementing reduced memory mode for high-definition television
US7792193B2 (en) Image encoding/decoding method and apparatus therefor
JP3732867B2 (en) Image expansion device
EP0843485B1 (en) Video decoder with unified memory
US8503521B2 (en) Method of digital video reference frame compression
JP4261630B2 (en) Image encoding apparatus and method, and computer-readable recording medium on which an image encoding program is recorded
US10652554B2 (en) Image coding apparatus, image coding method, image decoding apparatus, image decoding method, and storage medium
US6571017B1 (en) Method and apparatus for digital image coding and decoding, and data recording media
KR100971833B1 (en) Image coding method and apparatus
JP4393875B2 (en) System and method for providing a single layer video encoded bitstream suitable for reduced complexity decoding
US20200195926A1 (en) Image coding apparatus image coding method image decoding apparatus image decoding method and program
DE69834959T2 (en) Predictive image decoding method

Legal Events

Date Code Title Description
A521 Written amendment

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20051212

A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20051212

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20080222

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20080226

A521 Written amendment

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20080428

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20090115

A02 Decision of refusal

Free format text: JAPANESE INTERMEDIATE CODE: A02

Effective date: 20090512