JP2004135007A - Apparatus and method for processing image data and camera system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image data processing apparatus and an image data processing method capable of converting the number of image data by a plurality of different conversion ratios and to provide a camera system. <P>SOLUTION: The image data S1 of a chroma signal is outputted for each 8×8 pixels from an image data decoding section 10 for performing JPEG decoding processing. At a first image data generation section 21, the image data are synthesized for each 1×2 pixels in the 8×8 pixels and the 8×8 pixels are converted to 8×4 pixels. At a second image data generation section 22, image data S7 at the boundary region of the adjacent 8×4 pixels are used and the image data are synthesized for each 1×3 pixels in the 8×4 pixels and the 8×4 pixels are converted to 8×2 pixels, thus converting (4:4:4) to (4:1:1) in a YCrCb ratio. The conversion from (4:4:4) to (4:2:2) in the YCrCb ratio is executed by voiding data number conversion processing at the first image data generation section 21. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image data processing apparatus and method for converting the number of image data, and a camera system having the image data processing apparatus. For example, a set of 8 × 8 pixel blocks is input / output as a processing unit. Data processing apparatus and method for performing a process of converting the ratio of the number of image data between a luminance signal and a color difference signal in an interface section of a JPEG encoder / decoder to be performed, and a camera system having the image data processing apparatus It is about.
[0002]
[Prior art]
The format of image data handled in digital video or the like is generally a format (YCrCb format) using a luminance signal Y and color difference signals (chroma signals) Cr and Cb. In the YCrCb format, the number of chroma signal components with respect to the luminance signal component can be reduced, and the amount of image data can be reduced. Since the natural image has a property that the band of the chroma signal is narrower than that of the luminance signal, the resolution of the original image can be maintained to some extent even if the chroma signal component is somewhat thinned out in the YCrCb format.
[0003]
The ratio of the sampling frequency of the luminance signal Y to the chroma signal Cr and the chroma signal Cb (hereinafter referred to as YCrCb ratio) is Y: Cr: Cb = (4: 4: 4), (4: 2: 2), (4 : 1: 1) are often used. If no thinning is performed, Y: Cr: Cb = (4: 4: 4), and the information amount is equal to that of the RGB format.
[0004]
In a digital image processing apparatus such as a digital video that handles both still images and moving images, a YCrCb ratio (4: 2: 2) is generally used in a portion that handles still images, and a YCrCb ratio (4: 1: 1) is used in a portion that handles moving images. Can be On the other hand, it is desirable to write image data at a YCrCb ratio (4: 1: 1) in the frame memory inside the digital image processing apparatus in order to reduce the storage capacity. As described above, since the YCrCb ratio of the image data to be processed may be different between the frame memory and another unit accessing the frame memory, the interface unit is often provided with a function of performing the conversion process of the YCrCb ratio. .
[0005]
FIG. 19 is a block diagram illustrating an example of a configuration of an interface unit of a general digital video image processing apparatus.
The image processing apparatus shown in FIG. 19 includes YCrCb ratio conversion units 1 and 2, a frame memory 3, a frame memory controller 4, and an image processing unit 5.
[0006]
The YCrCb ratio conversion unit 1 receives YCrCb format image data Sin having a predetermined YCrCb ratio, and converts the YCrCb ratio of the input image data Sin to (4: 1: 1). For example, the YCrCb ratio of still image data decoded by the JPEG (Joint Photographic Experts Group) method is converted from (4: 2: 2) to (4: 1: 1).
[0007]
The YCrCb ratio conversion unit 2 converts the image data of the YCrCb ratio (4: 1: 1) input from the frame memory controller 4 into a predetermined ratio according to the specification of the output destination and outputs it. For example, still image data having a YCrCb ratio (4: 1: 1) is converted into still image data having a YCrCb ratio (4: 2: 2) and output to a JPEG encoder.
[0008]
The frame memory 3 stores image data of the YCrCb ratio (4: 1: 1) processed by the image processing unit 5 in frame units. Writing and reading of image data are performed under the control of the frame memory controller 4.
The frame memory controller 4 controls a writing process or a reading process of image data to the frame memory 3 in response to a request from the YCrCb ratio conversion unit 1, the YCrCb ratio conversion unit 2, and the image processing unit 3.
The image processing unit 5 performs predetermined image processing on the image data having the YCrCb ratio (4: 1: 1), such as a filtering process of the image data and a conversion process of the aspect ratio.
[0009]
According to the image processing apparatus shown in FIG. 19, image data having a predetermined YCrCb ratio is converted into image data having a YCrCb ratio (4: 1: 1) in a YCrCb ratio conversion unit 1, and the converted image data is converted into an image. Predetermined image processing is performed in the processing unit 5. In this process, the YCrCb ratio-converted image data is temporarily written to the frame memory 3 and read out from the frame memory 3 at any time in accordance with the processing of the image processing unit 5. After the processing by the image processing unit 5 is completed, the image data having the YCrCb ratio (4: 1: 1) is converted by the YCrCb ratio conversion unit 2 into a YCrCb ratio according to the specification of the output destination.
[0010]
By the way, in the image data encoding / decoding system of the JPEG system, the processing of image data is performed in units of an MCU (Minimum Coded Unit) which is an aggregate of 64 pixel blocks of 8 pixels in length × 8 pixels in width. .
[0011]
FIGS. 20, 21, and 22 show one MCU in the YCrCb ratio (4: 4: 4) format, the YCrCb ratio (4: 2: 2) format, and the YCrCb ratio (4: 1: 1) format, respectively. I have.
In the MCU of the YCrCb ratio (4: 4: 4) format, one luminance signal Y corresponds to one chroma signal (Cr, Cb) as shown in FIG.
In the MCU of the YCrCb ratio (4: 2: 2) format, as shown in FIG. 21, one chroma signal (Cr, Cb) corresponds to two luminance signals Y arranged on a horizontal line.
In the MCU of the YCrCb ratio (4: 1: 1) format, one chroma signal (Cr, Cb) corresponds to four luminance signals Y arranged on a horizontal line as shown in FIG.
[0012]
In the JPEG encoding and decoding units, data is input and output in pixel block units as shown in FIGS. Therefore, in order to exchange image data with the JPEG encoding and decoding units in the image processing apparatus shown in FIG. 19, it is necessary to appropriately perform interpolation processing on chroma signals input and output in pixel block units. There is a need to do.
[0013]
FIG. 23 is a diagram schematically illustrating a chroma signal interpolation process for converting image data having a YCrCb ratio (4: 2: 2) output from a JPEG decoding unit into image data having a YCrCb ratio (4: 1: 1). It is.
As shown in FIG. 23A, the image data of the chroma signal having the YCrCb ratio (4: 2: 2) decoded by the JPEG decoding unit is transmitted from the JPEG decoding unit in the order of the pixel blocks B1, B2,. Is output. The image data of each pixel block is output from the JPEG decoding unit in the order of image data c000, c001, ..., c007, c008, ... c063, c064, ....
[0014]
The image data for one pixel in the YCrCb ratio (4: 1: 1) format is the even-numbered image data (c000, c002, c004,...) In such an image data stream output from the JPEG decoding unit. The image data is generated by synthesizing image data of three pixels including the image data before and after the image data. That is, the data obtained by multiplying the even-numbered image data by the weighting factor １ ／ and the data obtained by multiplying the preceding and following image data by the weighting factor １ ／ are added to obtain the YCrCb ratio (4: 1: 1). ) Format image data for one pixel is generated. Since one image data is generated corresponding to the even-numbered image data, the number of image data of the chroma signal is reduced by half.
[0015]
Pixel blocks B3 and B4 shown in FIG. 23 (B) are YCrCb ratio-converted pixel blocks corresponding to pixel blocks B1 and B2 shown in FIG. 23 (A). As shown in FIG. 23, pixel blocks B1 and B2 of 8 × 8 pixels are converted into pixel blocks B3 and B4 of 8 × 4 pixels. The image data c'0, c'1,..., C'31, c'32,... Of the pixel blocks B3 and B4 are the even-numbered image data c000, c002,. are generated in correspondence with c064,.
[0016]
Focusing on the pixels of the horizontal line positioned at the top of the pixel blocks B1 and B2, pixels of the image data c064, c065,... Continue immediately to the right of the pixels of the image data c007 in the original image, and the image data c ′ 32 is generated by synthesizing image data c007, c064 and c065 of three adjacent pixels. However, since the three-pixel block (small pixel block) extends over different MCUs, the image data of the small pixel block is not continuous on the image data stream output from the JPEG decoding unit. That is, since the JPEG decoder outputs the image data in the order of the image data c007, c008, c009,..., The image data c007 and the image data c064 and c065 are not continuous on the image data stream. Therefore, in the method of simply combining continuous image data on the image data stream output from the JPEG decoder, conversion from the YCrCb ratio (4: 2: 2) to the YCrCb ratio (4: 1: 1) is performed. I can't do that.
[0017]
Therefore, as a method of converting the YCrCb ratio (4: 2: 2) to the YCrCb ratio (4: 1: 1), one frame of image data output from the JPEG decoder is temporarily stored in a frame memory. A method of reading the data into a line memory and processing the read data is generally used.
[0018]
FIG. 24 is a block diagram showing an example of the configuration of a general JPEG interface circuit that performs conversion from the YCrCb ratio (4: 2: 2) to the YCrCb ratio (4: 1: 1).
The JPEG interface circuit shown in FIG. 24 includes a YCrCb ratio conversion circuit 1, a frame memory 3, an MCU memory 6, and a line memory 7, and the same reference numerals in FIGS. 19 and 24 indicate the same components.
[0019]
The MCU memory 6 is an image data having a YCrCb ratio (4: 2: 2) which is divided for each MCU and sequentially inputted from a circuit block (hereinafter, referred to as a JPEG circuit) for performing a JPEG encoding or decoding process. Is stored for each MCU. Usually, it has a two-bank configuration in which a memory for writing and a memory for reading are separately provided. By alternately switching between the memory for writing and the memory for reading, the writing of image data from the JPEG circuit and the frame Reading of image data from the memory 3 is performed simultaneously.
The line memory 7 reads out and stores one horizontal line of image data from one frame of the image data stored in the frame memory 3.
[0020]
The image data of the YCrCb ratio (4: 2: 2) decoded by the JPEG circuit and input for each MCU is stored in the MCU memory 6 for each MCU. Then, the image data divided into 8 × 8 pixel blocks is rearranged into line data and written into the frame memory 3. This process is repeated for all MCUs of one frame of image data, so that one frame of image data is stored in the frame memory 3.
[0021]
Next, one line of image data is sequentially read from the frame memory 3 and stored in the line memory 7. The image data stored in the line memory 7 is converted from the YCrCb ratio (4: 2: 2) to the YCrCb ratio (4: 1: 1) by chroma interpolation in the YCrCb ratio conversion unit 1 and stored again in the line memory 7. You. After the conversion for one line is completed, the converted image data is written back to the frame memory again. This process is repeated for all lines, and the image data for one frame converted to the YCrCb ratio (4: 1: 1) is stored in the frame memory 3.
[0022]
In such a processing mode, since the chroma interpolation is performed on the data arranged in one line, an interpolation process for synthesizing a wider range of data becomes possible, and an improvement in image quality can be expected. However, since the image data once stored in the frame memory 3 is once read into the line memory 7 and then written back to the frame memory again, the number of writing processes to the memory increases, and the processing speed decreases. There is a benefit. Further, the line memory 7 for chroma interpolation must be provided, and there is a disadvantage that the circuit scale is increased.
[0023]
Next, chroma signal interpolation processing for reading out image data having a YCrCb ratio (4: 1: 1) from the frame memory, converting the image data into image data having a YCrCb ratio (4: 2: 2), and outputting the image data to a JPEG encoding unit. This will be described with reference to FIG.
The frame memory holds image data for one frame in the YCrCb ratio (4: 1: 1) format, and as shown in FIG. 25A, the pixel blocks B5, B6,. Image data is sequentially read from the frame memory. The image data of each pixel block is read from the frame memory in the order of the image data c0, c1,..., C31, c32,.
[0024]
The image data for two pixels in the YCrCb ratio (4: 2: 2) format is centered on each image data (c0, c1, c2,...) Having the YCrCb ratio (4: 1: 1) and the pixels of each image data. In this case, the image data of two pixels composed of the image data of the pixel adjacent to the right side is synthesized by using two types of weighting coefficients. That is, by adding the data obtained by multiplying the image data of the center pixel by the weighting factor 1 and the data obtained by multiplying the image data of the pixel on the right by the weighting factor 0, the YCrCb ratio (4: 2: 2) is obtained. ) Format image data for one pixel is generated. Also, by adding the data obtained by multiplying the image data of the center pixel by a weighting factor of 1/2 and the data obtained by multiplying the image data of the pixel on the right by the same weighting factor of 1/2, the YCrCb ratio ( Image data for one pixel in the 4: 2: 2) format is generated. Since two image data are generated corresponding to each image data, the number of image data of the chroma signal is doubled.
[0025]
Pixel blocks B7 and B8 shown in FIG. 25B are pixel blocks subjected to YCrCb ratio conversion corresponding to pixel blocks B5 and B6 shown in FIG. As shown in FIG. 25, pixel blocks B5 and B6 of 8 × 4 pixels are converted into pixel blocks B7 and B8 of 8 × 8 pixels.
[0026]
Focusing on the pixels of the horizontal line located at the top of the pixel blocks B5 and B6, the pixels of the image data c32, c33,... Continue immediately to the right of the pixels of the image data c3 in the original image, and the image data c ′ 007 is generated by combining the image data c3 and c32 of two adjacent pixels. However, since these two pixels are different in the 8.times.4 pixel block when read from the frame memory, the image data of the two pixels is not continuous on the image data stream read from the frame memory. That is, since the image data is read from the frame memory in the order of the image data c3, c4, c5,..., The image data c3 and the image data c32 are not continuous on the image data stream. Therefore, in the method of simply combining continuous image data on the image data stream read from the frame memory, it is not possible to convert the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 2: 2). Can not.
[0027]
Therefore, as a method of converting the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 2: 2), a method of reading image data necessary for interpolation from a frame memory as needed is generally used.
[0028]
FIG. 26 shows an example of the configuration of a general JPEG interface circuit that performs conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 2: 2).
The JPEG interface circuit shown in FIG. 26 has a YCrCb ratio conversion circuit 2, a frame memory 3 and an MCU memory 6, and the same reference numerals in FIGS. 19, 24 and 26 denote the same components.
[0029]
In the example of FIG. 26, image data for one frame has already been stored in the frame memory 3. Image data for one MCU is sequentially read from the image data stored in the frame memory 3 and temporarily stored in the MCU memory 6. From the image data of one MCU stored in the MCU memory 6, image data for each 8 × 4 pixel block is further read out to the YCrCb ratio conversion unit 2, and the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 2: 2) is performed. Thus, the YCrCb ratio converter 2 sequentially outputs image data for each 8 × 8 pixel block. At the time of this conversion, if image data that is not included in the image data of the 8 × 4 pixel block read from the MCU memory 6 is used for synthesis, the image data is directly read from the frame memory 3 and synthesized. Used for
[0030]
In general, a memory device such as a DRAM or an SRAM is often used in a large amount for the frame memory 3 which needs to store a large amount of data. However, a circuit for reading necessary data from such a memory device is often used. It is complicated and has the disadvantage of reducing the processing speed. Further, since a circuit for accessing the frame memory 3 is added for the YCrCb ratio converter 2, there is a disadvantage that the circuit scale is increased.
[0031]
As an invention for improving the above disadvantages in the JPEG interface circuit shown in FIGS. 24 and 26, for example, there is an invention described in Patent Document 1.
According to the present invention, it is possible to directly perform YCrCb ratio conversion on an image data stream by providing a means for storing image data used for interpolation of chroma signals in advance.
[0032]
[Patent Document 1]
JP-A-2002-237953
[0033]
[Problems to be solved by the invention]
However, the invention described in Patent Literature 1 does not disclose a technique for implementing YCrCb ratio conversion at a plurality of different conversion ratios. For example, in addition to the conversion from the YCrCb ratio (4: 2: 2) format to the YCrCb ratio (4: 1: 1) format, the YCrCb ratio (4: 4: 4) format is converted to the YCrCb ratio (4: 1: 1). It does not describe a technique that also allows conversion to a format. For example, in addition to conversion from the YCrCb ratio (4: 1: 1) format to the YCrCb ratio (4: 2: 2) format, the YCrCb ratio (4: 1: 1) format is converted to the YCrCb ratio (4: 4: 4). ) It does not describe a method that also enables conversion to a format.
[0034]
The present invention has been made in view of such circumstances, and a first object of the present invention is to provide an image data processing apparatus capable of converting the number of image data with a plurality of different conversion ratios.
A second object is to provide an image data processing method capable of converting the number of image data with a plurality of different conversion ratios.
A third object is to provide a camera system capable of converting the number of image data with a plurality of different conversion ratios for image data of a captured image.
[0035]
[Means for Solving the Problems]
In order to achieve the above object, an image data processing apparatus according to a first aspect of the present invention is an image data processing apparatus that performs conversion of the number of image data. And the input image data is synthesized for each first small pixel block smaller than the first pixel block, and one or a plurality of pixels corresponding to the first small pixel blocks are combined. First image data generating means for generating a plurality of image data; and, among image data generated by the first image data generating means, the first pixel block on the image composed of the image data. A boundary data storage unit that stores image data of pixels included in a predetermined boundary region of the corresponding second pixel block; and an image generated by the first image data generation unit. The data is synthesized for each second small pixel block smaller than the second pixel block to generate one or a plurality of image data corresponding to each of the second small pixel blocks. If some pixels of the small pixel block are included in the boundary area of another second pixel block adjacent to the second pixel block being generated by the first image data generation means, the boundary data The image data of the partial pixels stored in the storage unit and the image data of the remaining pixels of the second small pixel block generated by the first image data generation unit are combined, and Second image data generating means for generating one or a plurality of image data corresponding to the two small pixel blocks.
Preferably, when the conversion ratio of the number of image data is set to the first conversion ratio, the first image data generation unit executes generation of image data corresponding to the first small pixel block, When the conversion ratio is set to the second conversion ratio, the image data input for each third pixel block is output as the image data of the second pixel block.
[0036]
According to the image data processing device of the first aspect of the present invention, in the first image data generating means, image data corresponding to each pixel of the original image is input for each first pixel block, and the input is performed. Image data is synthesized for each first small pixel block. Thereby, one or a plurality of image data corresponding to each first small pixel block is generated. The boundary data storage means stores a predetermined boundary of a second pixel block corresponding to the first pixel block on an image formed of the image data among the image data generated by the first image data generation means. The image data of the pixels included in the area is stored. The image data generated by the first image data generating means is synthesized by the second image data generating means for each second small pixel block, and one or a plurality of image data corresponding to each of the second small pixel blocks are combined. Image data is generated. If a part of the pixels of the second small pixel block is included in the boundary area of another second pixel block adjacent to the second pixel block being generated by the first image data generating means, the second The image data of the partial pixels stored in the boundary data storage means and the image data of the remaining pixels of the second small pixel block generated by the first image data generation means Are synthesized to generate one or a plurality of image data corresponding to the second small pixel block.
When the conversion of the number of image data is set to the first conversion ratio and the generation of the image data corresponding to the first small pixel block is executed by the first image data generation means, the first In the image data generation means, the conversion processing of the number of image data is executed. On the other hand, in response to the conversion ratio of the number of image data being set to the second conversion ratio, the image data input for each third pixel block is converted into the first pixel data as the image data of the second pixel block. When the image data is output from the image data generating means, the first image data generating means does not execute the conversion processing of the number of image data. That is, the conversion ratio of the number of image data is changed by enabling or disabling the conversion process of the number of image data in the first image data generating means according to the set conversion ratio of the number of image data. .
[0037]
The first image data generating means sequentially inputs image data included in the first pixel block, and converts the input series of image data into a predetermined number corresponding to the number of pixels of the first small pixel block. A first data holding unit that holds the image data up to and including the image data held by the first data holding unit and the image data that is input subsequently thereto by giving a predetermined weight according to the input order and combining them. And a first data synthesizing unit. A control circuit for generating a first enable signal indicating whether or not the image data synthesized by the first data synthesizing means is image data synthesized corresponding to the first small pixel block; But it's fine.
[0038]
The second image data generation means sequentially inputs image data synthesized corresponding to the first small pixel block from the first data synthesis means based on the first enable signal, A second data holding unit for holding the series of image data thus obtained up to a predetermined number corresponding to the number of pixels of the second small image block; an image data held by the second data holding unit; Subsequently, a predetermined weighting is given to the input image data in accordance with the input order, and the image data is synthesized. The pixels of a part of the image data in the series of image data of the synthesis source and the boundary data stored in the boundary data storage unit. When the pixels of the image data form the second small pixel block, a second data synthesizing means for giving a predetermined weight to the partial image data and the stored image data and synthesizing them It may include a. The control circuit generates a second enable signal indicating whether or not the image data synthesized by the first data synthesizing means is image data synthesized corresponding to the second small pixel block. You may.
[0039]
An image data processing device according to a second aspect of the present invention is an image data processing device that converts the number of image data, and in the first stage, converts image data corresponding to each pixel of an original image for each first pixel block. In each of the stages following the first stage, the image data generated in the previous stage is input for each pixel block corresponding to the first pixel block, and the input image data is converted into a small pixel block determined in each stage. A plurality of image data generating means connected in cascade to generate one or a plurality of image data corresponding to each of the small pixel blocks, and a part of the plurality of image data generating means Or boundary data storage means corresponding to all of the image data input to the corresponding image data generation means, Storing image data of pixels included in a predetermined boundary region of the pixel block corresponding to the serial first pixel block, and a one or more boundary data storage means. The image data generation unit associated with the boundary data storage unit is configured to handle a case where some pixels of the small pixel block are included in the boundary region of another pixel block adjacent to the pixel block being input. The image data of the partial pixels stored in the boundary data storage unit to be combined with the input image data of the remaining pixels of the small pixel block are combined to form one or a plurality of pixels corresponding to the small pixel block. Is generated.
Preferably, the plurality of image data generation units each determine whether or not to execute a process of generating image data corresponding to the small pixel block according to a set conversion ratio of the number of image data.
[0040]
According to the image data processing apparatus of the second aspect of the present invention, image data corresponding to each pixel of the original image is input for each first pixel block at the first stage of the plurality of cascade-connected image data generating means. Is done. In the image data generating means of each stage following the first stage, the image data generated in the previous stage is input for each pixel block corresponding to the first pixel block, and the input image data is determined in each stage. One or a plurality of image data corresponding to each of the small pixel blocks is generated. Further, the boundary data storage means is provided corresponding to a part or all of the plurality of image data generation means, and the boundary data storage means stores the image data inputted to the corresponding image data generation means. Among them, the image data of the pixels included in the predetermined boundary area of the pixel block corresponding to the first pixel block on the image composed of the image data is stored. In the image data generation unit associated with the boundary data storage unit, if some pixels of the small pixel block are included in the boundary region of another pixel block adjacent to the input pixel block, the corresponding boundary data The image data of some of the pixels stored in the storage unit and the input image data of the remaining pixels of the small pixel block are combined, and one or more image data corresponding to the small pixel block is formed. Generated.
Further, according to the set conversion ratio of the number of image data, whether or not to execute the generation processing of the image data corresponding to the small pixel block is determined in each of the plurality of image data generation means, so that the cascade connection is performed. The conversion ratio of the number of image data from the first stage to the last stage of the plurality of image data generating means is changed.
[0041]
An image data processing apparatus according to a third aspect of the present invention is an image data processing apparatus that converts the number of image data, and converts image data corresponding to each pixel of an original image into a set conversion ratio of the number of image data. Pixel block storage means for inputting and storing for each predetermined pixel block according to the above, and image data stored in the pixel block storage means are read, and the read image data is synthesized for each predetermined small pixel block. A plurality of image data generating means for generating one or a plurality of image data for each of the small pixel blocks; and another pixel adjacent to a pixel block next read from the pixel block storage means by the image data generating means. Block or another pixel block adjacent to the pixel block being read from the pixel block storage means. Boundary data storage means for inputting and storing image data included in the predetermined boundary area, and outputting image data of the image data generation means selected according to the conversion ratio among the plurality of image data generation means. Selecting means for performing the setting. The image data generating means stores the boundary data when some pixels of the small pixel block are included in the boundary area of another pixel block adjacent to the pixel block being read from the pixel block storage means. The image data of the partial pixels stored in the unit and the image data of the remaining pixels of the small pixel block read from the pixel block storage unit are combined, and one or a pixel corresponding to the small pixel block is combined. Generate a plurality of image data.
[0042]
According to the image data processing apparatus of the third aspect of the present invention, the image data corresponding to each pixel of the original image is stored in the pixel block storage means for each predetermined pixel block corresponding to the set conversion ratio of the number of image data. And stored in it. The image data stored in the pixel block storage unit is read by the image data generation unit, and the read image data is combined for each predetermined small pixel block. Thereby, one or a plurality of image data is generated for each of the small pixel blocks. A plurality of such image data generation means are provided for the pixel block storage means. In the boundary data storage unit, another pixel block adjacent to the pixel block next read from the pixel block storage unit by the image data generation unit, or another pixel block adjacent to the pixel block being read from the pixel block storage unit. The image data included in the predetermined boundary area according to the conversion ratio is input and stored. The image data of the image data generation unit selected according to the conversion ratio among the plurality of image data generation units is output from the selection unit. When some pixels of the small pixel block are included in the boundary area of another pixel block adjacent to the pixel block being read from the pixel block storage unit, the image data is stored in the boundary data storage unit in the image data generation unit. The image data of the partial pixels and the image data of the remaining pixels of the small pixel block read from the pixel block storage unit are combined. Thereby, one or a plurality of image data corresponding to the small pixel block is generated.
[0043]
The plurality of image data generating means sequentially reads the image data stored in the pixel block storage means, and performs sampling of the read image data once or a plurality of times each time one pixel of image data is read. A data holding unit for holding a series of image data obtained up to a predetermined number, and a series of image data held in the data holding unit, and a pixel of a part of image data in the series of When the pixels of the image data stored in the boundary data storage unit form the small pixel block, the pixel data unit may include a data combining unit that combines the partial image data and the stored image data. good.
[0044]
The data synthesizing unit outputs a series of image data held in the data holding unit, and a pixel of a part of the plurality of image data to be output and an image stored in the boundary data storage unit. When the pixels of the data constitute the small pixel block, the remaining image data of the plurality of image data to be output, excluding the partial image data, is stored in the boundary data storage unit. And an adder for adding the values of the plurality of image data output from the data output unit.
[0045]
An image data processing apparatus according to a fourth aspect of the present invention is an image data processing apparatus that converts the number of image data, and inputs image data corresponding to each pixel of an original image for each first pixel block. A pixel block storing means for reading and storing the image data stored in the pixel block storing means for each first pixel block, and reading the read image data into first small pixels smaller than the first pixel block. First image data generating means for generating one or a plurality of image data corresponding to each of the first small pixel blocks by synthesizing each block, and the first image data generating means Another pixel block adjacent to the pixel block to be read next from the storage means, or adjacent to the pixel block being read from the pixel block storage means. A boundary data storage unit for inputting and storing image data included in a predetermined boundary region of another pixel block to be processed, and image data generated by the first image data generation unit. One or a plurality of images corresponding to each of the second small pixel blocks by synthesizing for each second small pixel block smaller than the second pixel block corresponding to the first pixel block Second image data generating means for generating data. The first image data generation unit may include a part of the pixels of the first small pixel block, the pixels of another first pixel block adjacent to the first pixel block being read from the pixel block storage unit. When included in the boundary area, the image data of the partial pixels stored in the boundary data storage unit and the image data of the remaining pixels of the first small pixel block read from the pixel block storage unit To generate one or a plurality of image data corresponding to the first small pixel block. The second image data generating means is configured so that a part of the pixels of the second small pixel block includes another second pixel adjacent to the second pixel block being generated by the first image data generating means. When included in the block, image data of a pixel equivalent to the partial pixel in the boundary area is read from the boundary data storage unit, and the read image data and the image data generated by the first image data generation unit are generated. The image data of the remaining small pixels of the second small pixel block are combined with one another to generate one or a plurality of image data corresponding to the second small pixel block.
Preferably, when the conversion ratio of the number of image data is set to the first conversion ratio, the pixel block storage means inputs and stores the image data for each of the first pixel blocks, and stores the conversion ratio. When the second conversion ratio is set, image data is input and stored for each third pixel block. Further, when the conversion ratio is set to the first conversion ratio, the first image data generation unit executes generation of image data corresponding to the first small pixel block, and the conversion ratio is When the second conversion ratio is set, the image data of the third pixel block is read from the pixel block storage means and output as the image data of the second pixel block. The boundary data storage means inputs and stores image data included in a predetermined boundary region according to the set conversion ratio of the number of image data.
[0046]
According to the image data processing apparatus of the fourth aspect of the present invention, the first pixel block storage means inputs and stores the image data corresponding to each pixel of the original image for each first pixel block. The image data stored in the pixel block storage unit is read by the first image data generation unit for each first pixel block, and the read image data is combined for each first small pixel block. You. Thereby, one or a plurality of image data corresponding to each first small pixel block is generated. In the boundary data storage unit, another pixel block adjacent to a pixel block next read from the pixel block storage unit by the first image data generation unit, or another pixel block adjacent to a pixel block being read from the pixel block storage unit. Image data included in a predetermined boundary area of the pixel block is input and stored. The image data generated by the first image data generating means is, for each second small pixel block smaller than the second pixel block corresponding to the first pixel block on the image constituted by the image data, 2 are combined in the image data generating means. Thereby, one or a plurality of image data corresponding to each second small pixel block is generated. When some pixels of the first small pixel block are included in the boundary area of another first pixel block adjacent to the first pixel block being read from the pixel block storage means, In the image data generation means, the image data of the partial pixels stored in the boundary data storage means and the image data of the remaining pixels of the first small pixel block read from the pixel block storage means are combined. , One or more image data corresponding to the first small pixel block is generated. If some pixels of the second small pixel block are included in another second pixel block adjacent to the second pixel block being generated by the first image data generation means, the second image data In the generation unit, image data of a pixel equivalent to the partial pixel in the boundary area is read from the boundary data storage unit, and the read image data and the image data generated by the first image data generation unit are generated. The image data of the remaining pixels of the second small pixel block are combined with each other to generate one or a plurality of image data corresponding to the second small pixel block.
By setting the conversion ratio of the number of image data to the first conversion ratio, image data is input for each first pixel block and stored in the pixel block storage means. When the process of generating image data corresponding to the first small pixel block is executed by the first image data generating unit, the first image data generating unit executes a process of converting the number of image data. On the other hand, in response to the conversion ratio of the number of image data being set to the second conversion ratio, image data is input for each third pixel block and stored in the pixel block storage means. When the image data of the pixel block is read out from the pixel block storage unit and output from the first image data generation unit as the image data of the second pixel block, the first image data generation unit Conversion process is not performed. That is, the conversion ratio of the number of image data is changed by enabling or disabling the conversion process of the number of image data in the first image data generating means according to the set conversion ratio of the number of image data. .
[0047]
The first image data generation unit sequentially reads the image data stored in the pixel block storage unit, and performs sampling of the read image data one or more times each time one pixel of image data is read, A first data holding unit for holding a series of sampled image data up to a predetermined number, and a series of image data held by the first data holding unit; When the pixels of the image data and the pixels of the image data stored in the boundary data storage unit constitute the first small pixel block, the partial image data and the stored image data are When the first data synthesizing unit to be synthesized and the conversion ratio of the number of image data are set to the first conversion ratio, the first data synthesizing unit performs synthesis First selecting means for selecting and outputting image data and selecting and outputting the image data sampled in the first data holding means when the conversion ratio is set to the second conversion ratio; And may be included.
The second image data generating means performs the sampling of the image data output from the first selecting means one or more times each time the image data is read and sampled by the first data holding means. A second data holding means for holding a series of sampled image data up to a predetermined number, a series of image data held in the second data holding means, and the first selection to be subsequently sampled; The output image data of the means is combined with a pixel of a part of image data in a series of image data of the combination source and a pixel on the second pixel block equivalent to a pixel included in the boundary area. Constitutes the second small pixel block, reads image data corresponding to the equivalent pixel from the boundary data storage means, It may include a second data synthesis means for synthesizing the image data and the part of the image data.
[0048]
An image data processing method according to a fifth aspect of the present invention is an image data processing method for converting the number of image data, wherein the image data of each pixel included in the first pixel block is converted from the first pixel block. The process of synthesizing each of the first small pixel blocks and generating one or a plurality of image data corresponding to each of the first small pixel blocks is performed by a plurality of the first pixel blocks included in the original image. A first image data generating step for sequentially executing the first image data generating step, and among the image data generated in the first image data generating step, the first image data generating step corresponds to the first pixel block on the image composed of the image data. A boundary data storing step of storing, in a boundary data storage device, image data of pixels included in a predetermined boundary region of the second pixel block to be processed; The image data generated in the data generation step is synthesized for each second small pixel block smaller than the second pixel block, and one or more images corresponding to the respective second small pixel blocks are combined. Generating data, wherein a part of the pixels of the second small pixel block is in the boundary area of another second pixel block adjacent to the second pixel block being generated in the first image data generating step. , The image data of the partial pixels stored in the boundary data storage device and the image of the remaining pixels of the second small pixel block generated in the first image data generating step A second image data generating step of generating one or a plurality of image data corresponding to the second small pixel block by combining the data with the second small pixel block.
Preferably, when the conversion ratio of the number of image data is set to the first conversion ratio, in the first image data generation step, generation of image data corresponding to the first small pixel block is performed. When the conversion ratio is set to the second conversion ratio, in the first image data generation step, the same data as the image data of the third pixel block included in the original image is replaced by the second image data. It is sequentially generated as image data of a pixel block.
[0049]
An image data processing method according to a sixth aspect of the present invention is an image data processing method for converting the number of image data, wherein the image data processing method comprises the steps of: A pixel block storage step of reading image data of each pixel for each predetermined pixel block corresponding to the conversion ratio and storing the read image data in the pixel block storage device, and reading the image data stored in the pixel block storage device; An image data generating step of combining image data for each predetermined small pixel block corresponding to the conversion ratio to generate one or a plurality of image data for each of the small pixel blocks; When reading of image data from the frame image storage device is performed, in the image data generating step, It is included in the predetermined boundary area of another pixel block adjacent to a pixel block read next from the pixel block storage device or another pixel block adjacent to a pixel block being read from the pixel block storage device. Reading the image data from the frame image storage device and storing the read image data in the boundary data storage device. In the image data generating step, when some pixels of the small pixel block are included in the boundary area of another pixel block adjacent to the pixel block being read from the pixel block storage device, the boundary data storage is performed. The image data of the partial pixels stored in the device is combined with the image data of the remaining pixels of the small pixel block read from the pixel block storage device, and one or Generate a plurality of image data.
[0050]
An image data processing method according to a seventh aspect of the present invention is an image data processing method for converting the number of image data, wherein the image data processing method comprises the steps of: A pixel block storage step of reading out image data of each pixel and storing the image data in the pixel block storage device, and reading out the image data stored in the pixel block storage device for each first pixel block; A first image data generating step of synthesizing each first small pixel block smaller than the first pixel block to generate one or a plurality of image data corresponding to each of the first small pixel blocks; When reading of image data from the frame image storage device is performed in the pixel block storing step, the first A predetermined boundary between another pixel block adjacent to a pixel block to be read next from the pixel block storage device in the image data generation step or another pixel block adjacent to a pixel block being read from the pixel block storage device A boundary data storing step of reading image data included in the area from the frame image storage device and storing the read image data in the boundary data storage device; and image data generated in the first image data generation step. One or a plurality of second small pixel blocks corresponding to the second small pixel blocks are synthesized on the image to be synthesized for each second small pixel block smaller than the second pixel block corresponding to the first pixel block. A second image data generating step of generating image data. In the first image data generation step, a part of the pixels of the first small pixel block is a pixel of another first pixel block adjacent to the first pixel block being read from the pixel block storage device. When included in the boundary area, the image data of the partial pixels stored in the boundary data storage device and the image of the remaining pixels of the first small pixel block read from the pixel block storage device By combining the data with the data, one or more image data corresponding to the first small pixel block is generated. In the second image data generating step, a part of the pixels of the second small pixel block is replaced with another second pixel adjacent to the second pixel block being generated in the first image data generating step. When included in the block, image data of pixels equivalent to the partial pixels in the boundary area is read from the boundary data storage device, and the read image data and the image data generated in the first image data generation step are generated. The image data of the remaining small pixels of the second small pixel block are combined with one another to generate one or a plurality of image data corresponding to the second small pixel block.
Preferably, when the conversion ratio of the number of image data is set to a first conversion ratio, in the pixel block storage step, image data is read out from the frame image storage device for each of the first pixel blocks, and The image data is stored in the block storage device, and the image data corresponding to the first small pixel block is generated in the first image data generating step. When the conversion ratio is set to the second conversion ratio, in the pixel block storage step, image data is read out from the frame image storage device for each third pixel block and stored in the pixel block storage device; In the first image data generation step, the same data as the image data of the third pixel block read from the pixel block storage device is generated as the image data of the second pixel block. In the boundary data storing step, image data included in a predetermined boundary area according to the set conversion ratio of the number of image data is stored in the boundary data storage device.
[0051]
A camera system according to an eighth aspect of the present invention captures an image, generates image data corresponding to each pixel of the captured image, and stores image data of the captured image generated by the capturing unit. A frame image storing means for storing the image data of the photographed image stored in the frame image storing means for each predetermined pixel block corresponding to a set conversion ratio of the number of image data to convert the number of image data; Perform image data processing means for sequentially outputting a block of image data corresponding to the pixel block, and a predetermined number of image data blocks including image data blocks output from the image data processing device as a unit, Image data encoding means for encoding the image data. A pixel block storage unit that reads and stores the image data of the captured image stored in the frame image storage unit for each predetermined pixel block corresponding to the conversion ratio; A plurality of image data generating means for reading image data stored in the means, synthesizing the read image data for each predetermined small pixel block, and generating one or a plurality of image data for each small pixel block; And another pixel block adjacent to a pixel block next read from the pixel block storage unit by the image data generation unit when image data is read from the frame image storage unit in the pixel block storage unit. Or the pixel block being read from the pixel block storage means Boundary data storage means for reading and storing, from the frame image storage means, image data included in the predetermined boundary area of another pixel block in contact with the image data, and the plurality of image data generation means, Selecting means for outputting image data of the selected image data generating means. The image data generating means stores the boundary data when some pixels of the small pixel block are included in the boundary area of another pixel block adjacent to the pixel block being read from the pixel block storage means. The image data of the partial pixels stored in the unit and the image data of the remaining pixels of the small pixel block read from the pixel block storage unit are combined, and one or a pixel corresponding to the small pixel block is combined. Generate a plurality of image data.
[0052]
A camera system according to a ninth aspect of the present invention captures an image, generates image data corresponding to each pixel of the captured image, and stores image data of the captured image generated by the capturing unit. A frame image storing means for storing the image data of the photographed image stored in the frame image storing means for each predetermined pixel block corresponding to a set conversion ratio of the number of image data to convert the number of image data; Perform image data processing means for sequentially outputting a block of image data corresponding to the pixel block, and a predetermined number of image data blocks including image data blocks output from the image data processing device as a unit, Image data encoding means for encoding the image data. The image data processing means reads out the image data of the photographed image stored in the frame image storage means for each first pixel block and stores the image data, and the image data processing means stores the image data in the pixel block storage means. The image data is read out for each first pixel block, and the read out image data is combined for each first small pixel block smaller than the first pixel block to correspond to each of the first small pixel blocks. First image data generating means for generating one or a plurality of pieces of image data, and generating the first image data when reading out the image data from the frame image storing means in the pixel block storing means. A pixel block to be read next from the pixel block storage means by means, or the pixel block storage means A boundary data storage unit for reading image data included in the predetermined boundary region of another pixel block adjacent to the pixel block being read from the frame image storage unit and storing the read image data; The image data generated by the means is synthesized for each second small pixel block smaller than the second pixel block corresponding to the first pixel block on an image constituted by the image data, and Second image data generating means for generating one or more image data corresponding to the second small pixel block. The first image data generation unit may include a part of the pixels of the first small pixel block, the pixels of another first pixel block adjacent to the first pixel block being read from the pixel block storage unit. When included in the boundary area, the image data of the partial pixels stored in the boundary data storage unit and the image data of the remaining pixels of the first small pixel block read from the pixel block storage unit To generate one or a plurality of image data corresponding to the first small pixel block. The second image data generating means is configured so that a part of the pixels of the second small pixel block includes another second pixel adjacent to the second pixel block being generated by the first image data generating means. When included in the block, image data of a pixel equivalent to the partial pixel in the boundary area is read from the boundary data storage unit, and the read image data and the image data generated by the first image data generation unit are generated. The image data of the remaining small pixels of the second small pixel block are combined with one another to generate one or a plurality of image data corresponding to the second small pixel block.
Preferably, the pixel block storage means sets the conversion ratio to the second conversion ratio for each of the first pixel blocks when the conversion ratio of the number of image data is set to the first conversion ratio. In this case, the image data is read from the frame image storage means and stored for each third pixel block. Further, when the conversion ratio is set to the first conversion ratio, the first image data generation unit executes generation of image data corresponding to the first small pixel block, and the conversion ratio is When the second conversion ratio is set, the image data of the third pixel block is read from the pixel block storage means and output as the image data of the second pixel block.
[0053]
BEST MODE FOR CARRYING OUT THE INVENTION
Seven embodiments of the present invention will be described with reference to the drawings.
<First embodiment>
FIG. 1 is a block diagram illustrating an example of a configuration of the image data processing device according to the first embodiment of the present invention.
The image data processing device illustrated in FIG. 1 includes an image data decoding unit 10, a first image data generation unit 21, a second image data generation unit 22, a boundary data storage unit 30, and a pixel block storage unit 40. And a frame memory 50.
The image data decoding unit 10 is an embodiment of an image data decoding unit according to the first aspect of the present invention.
The first image data generator 21 is an embodiment of a first image data generator according to the first aspect of the present invention.
The second image data generator 22 is an embodiment of a second image data generator according to the first aspect of the present invention.
The boundary data storage unit 30 is an embodiment of a boundary data storage unit according to the first aspect of the present invention.
[0054]
(Image data decoding unit 10)
The image data decoding unit 10 performs a predetermined decoding process on the image data of the original image encoded by a predetermined method, and outputs the image data decoded by the decoding process for each predetermined pixel block. .
For example, a decoding process is performed on the image data of the original image encoded by the JPEG method, and the decoded image data is output for each MCU as shown in FIGS.
[0055]
(First Image Data Generation Unit 21)
The first image data generation unit 21 inputs image data S1 corresponding to each pixel of the original image for each predetermined pixel block (this pixel block is referred to as a first pixel block in the description of the present embodiment). The input image data S1 is synthesized for each predetermined small pixel block smaller than the first pixel block (this small pixel block is referred to as a first small pixel block in the description of the present embodiment). One or a plurality of image data corresponding to the small pixel blocks are generated.
[0056]
For example, of the image data of 1 MCU output from the image data decoding unit 10, the 8 × 8 pixel block of the chroma signals Cr and Cb is input as the first pixel block. Then, a 1 × 2 pixel block adjacent on the horizontal line is set as a first small pixel block, and image data of an 8 × 8 pixel block is synthesized for each of the first small pixel blocks. One pixel image data corresponding to the small pixel block is generated. As a result, the 8 × 8 pixel block is converted into an 8 × 4 pixel block, and the number of image data is reduced by half.
[0057]
Further, the first image data generation unit 21 may enable or disable the conversion process of the number of image data according to the set conversion ratio of the number of image data. That is, when the conversion ratio of the number of image data is set to the first conversion ratio, generation of the above-described image data corresponding to each first small pixel block is performed. Thereby, the conversion of the number of image data is executed. When the conversion ratio of the number of image data is set to the second conversion ratio, the image data input for each predetermined third pixel block equal to or different from the first pixel block is output as it is. Thereby, the conversion processing of the number of image data in the first image data generation unit 21 is invalidated.
[0058]
For example, when the conversion from the YCrCb ratio (4: 4: 4) to the YCrCb ratio (4: 1: 1) is set as the first conversion ratio for the image data processing device, the first image data generation unit Reference numeral 21 executes the above-described processing of synthesizing the image data for each small pixel block of 1 × 2 pixels to generate image data of one pixel corresponding to each small pixel block. As a result, the 8 × 8 pixel block of the chroma signal is converted into an 8 × 4 pixel block, and the number of image data is reduced to １ ／. When the conversion from the YCrCb ratio (4: 2: 2) to the YCrCb ratio (4: 1: 1) is set as the second conversion ratio for the image data processing device, every 8 × 8 pixel block The input image data is output as it is. Thereby, the conversion processing of the number of image data in the first image data generation unit 21 is invalidated.
In this example, regardless of which conversion ratio is set, in the second image data generation unit 22 described below, processing for converting an 8 × 8 pixel block of a chroma signal into an 8 × 4 pixel block is executed. Is done. Therefore, as a whole, the image data processing device converts the YCrCb ratio (4: 4: 4) to the YCrCb ratio (4: 1: 1) or converts the YCrCb ratio (4: 2: 2) to the YCrCb ratio (4: 1). 1) is performed.
[0059]
(Second image data generation unit 22)
The second image data generation unit 22 combines the image data generated by the first image data generation unit 21 for each second small pixel block smaller than the second pixel block, and One or more image data corresponding to the small pixel block is generated. The second pixel block is a pixel block corresponding to the first pixel block or the third pixel block of the original image on the image composed of the image data generated by the first image data generation unit 21. Is shown.
[0060]
For example, when a first pixel block of a chroma signal having a size of 8 × 8 pixels is converted into an 8 × 4 pixel block in the first image data generation unit 21, the converted 8 × 4 pixel block is This corresponds to a second pixel block. In contrast to the 8 × 4 pixel block, the second small pixel block is, for example, a block of 1 × 3 pixels arranged adjacently on a horizontal line such that pixels at both ends except for a center pixel overlap each other. Is set as Therefore, when the second image data generating unit 22 combines the image data of the small pixel blocks of 1 × 3 pixels to generate image data of one pixel corresponding to each small pixel block, An 8 × 4 pixel block is converted to an 8 × 2 pixel block. As a result, the number of image data is further reduced by half.
[0061]
However, the pixels of the second small pixel block are not necessarily all included in the second pixel block being generated by the first image data generation unit 21. That is, some pixels of the second small pixel block are included in a predetermined boundary area of another second pixel block adjacent to the second pixel block being generated in the first image data generation unit 21. Sometimes.
In this case, the second image data generation unit 22 reads out the image data of the partial pixels stored in the boundary data storage unit 30 and generates the read-out image data and the first image data generation unit 21. The image data of the remaining small pixels of the second small pixel block are combined with one another to generate one or a plurality of image data corresponding to the second small pixel block.
[0062]
For example, when the above-described block of small pixels of 1 × 3 pixels is arranged on the right side of the 8 × 4 pixel block, two small pixel blocks are arranged on the horizontal line of the 8 × 4 pixel block. , The leftmost pixel of the small pixel block located on the left side is not included in the pixel block being generated by the first image data generation unit 21, and the rightmost 8 × 1 pixel block of the 8 × 4 pixel block on the left is It is included in the pixel area. In this case, the above-described boundary region corresponds to the rightmost 8 × 1 pixel region.
When synthesizing a 1 × 3 pixel small pixel block located on the left side of the 8 × 4 pixel block, the second image data generation unit 22 stores the image data of the leftmost pixel of the small pixel block in the boundary data storage unit. Read from 30. Then, the read image data is combined with the image data of the remaining two pixels of the small pixel block generated by the first image data generation unit 21 to generate image data of one pixel.
[0063]
(Boundary data storage unit 30)
The boundary data storage unit 30 stores image data of pixels included in a predetermined boundary region of the second pixel block, out of the image data generated by the first image data generation unit 21.
This boundary area is an area of another second pixel block adjacent to the second pixel block that includes image data necessary for performing interpolation processing of one second pixel block. This boundary area is generated when a second small pixel block is set to extend between adjacent second pixel blocks. For example, regions L1 and L2 in FIG. 23 and regions L5 and L6 in FIG. 25 correspond to this boundary region.
[0064]
(Pixel block storage unit 40)
The pixel block storage unit 40 stores the image data generated by the image data generation unit 22 for each predetermined pixel block corresponding to the second pixel block.
For example, the image data of the chroma signal of the 8 × 2 pixel block output from the second image data generation unit 22 and the image data of the luminance signal of the 8 × 8 pixel block output from the image data decoding unit 10 are Remember.
The pixel block storage unit 40 can have, for example, a two-bank configuration in which a memory for writing and a memory for reading are separately provided. With the two-bank configuration, the memory for writing and the memory for reading are alternately switched, so that the writing of the image data generated by the image data generating unit 22 and the reading of the image data to the frame memory 50 are simultaneously performed. It can be carried out.
[0065]
(Frame memory 50)
The frame memory 50 sequentially reads out the image data stored in the pixel block storage unit 40 in pixel block units and sequentially stores the read image data, thereby storing the image data corresponding to the original image in frame units. The stored image data is used by other blocks that perform various image processing.
As the frame memory 50, for example, a high-speed, large-capacity storage device such as an SDRAM (Synchronous DRAM) is used.
[0066]
Here, the operation of the image data processing apparatus shown in FIG. 1 having the above configuration will be described.
The image data of the original image encoded by the predetermined method is decoded by the image data decoding unit 10, and the decoded image data is output for each predetermined pixel block.
For example, the image data of the original image encoded by the JPEG method is decoded by the image data decoding unit 10, and the decoded image data is output for each MCU.
[0067]
In the first image data generation unit 21, image data S1 corresponding to each pixel of the original image is input for each first pixel block, and the input image data S1 is synthesized for each first small pixel block. You. Thereby, one or a plurality of image data corresponding to each small pixel block is generated.
For example, among the 1 MCU image data output from the image data decoding unit 10, the 8 × 8 pixel block of the chroma signals Cr and Cb is input to the first image data generation unit 21. Then, the input image data is synthesized for each 1 × 2 pixel block adjacent to the horizontal line. As a result, image data of one pixel corresponding to each first small pixel block is generated. The 8 × 8 pixel blocks of the chroma signals Cr and Cb are converted into 8 × 4 pixel blocks, and the number of image data of the chroma signals is reduced by half.
[0068]
In the second image data generation unit 22, the image data generated in the first image data generation unit 21 is synthesized for each second small pixel block, and one or two corresponding to the second small pixel blocks are combined. A plurality of image data are generated.
For example, the image data converted from the 8 × 8 pixel block into the 8 × 4 pixel block in the first image data generation unit 21 is arranged adjacently on a horizontal line such that the pixels at both ends overlap each other. It is synthesized for each small pixel block of × 3 pixels. As a result, image data for one pixel corresponding to each small pixel block is generated. The 8 × 4 pixel blocks of the chroma signals Cr and Cb are converted into 8 × 2 pixel blocks, and the number of image data is further reduced to half.
[0069]
In addition, some pixels of the second small pixel block are included in a predetermined boundary region of another second pixel block adjacent to the second pixel block being generated in the first image data generation unit 21. In this case, the second image data generation unit 22 reads out the image data of some of the pixels stored in the boundary data storage unit 30. Then, the read image data and the image data of the remaining pixels of the second small pixel block generated by the first image data generation unit 21 are combined. Thereby, one or a plurality of image data corresponding to the second small pixel block is generated.
For example, in a case where blocks of small pixels of 1 × 3 pixels are arranged on the right side of the 8 × 4 pixel block, the second image data generation unit 22 sets the 1 × 3 pixel block located on the left side of the 8 × 4 pixel block. When the three-pixel small pixel block is synthesized, the image data of the leftmost pixel of the small pixel block is read from the boundary data storage unit 30. Then, the read one-pixel image data and the image data of the remaining two pixels of the small pixel block generated by the first image data generation unit 21 are combined, and the image data of one pixel is obtained. Is generated.
[0070]
The boundary data storage unit 30 stores image data of pixels included in a predetermined boundary region of the second pixel block among the image data generated by the first image data generation unit 21.
For example, when blocks of small pixels of 1 × 3 pixels are arranged on the right side of an 8 × 4 pixel block, the rightmost 8 × 1 pixel region in the 8 × 4 pixel block corresponds to a boundary region. The image data of the area is stored in the boundary data storage unit 30. When the stored image data is combined with the 1 × 3 pixel small pixel block located on the left side of the 8 × 4 pixel block in the 8 × 4 pixel block on the right side of the boundary area, the second The image data is read out by the image data generation unit 22 and used for synthesizing the small pixel block.
[0071]
The image data generated by the image data generation unit 22 is stored in the pixel block storage unit 40 for each predetermined pixel block corresponding to the second pixel block.
For example, the image data of the chroma signal of the 8 × 2 pixel block output from the second image data generation unit 22 and the image data of the luminance signal of the 8 × 8 pixel block output from the image data decoding unit 10 are It is temporarily stored in the pixel block storage unit 40.
[0072]
The image data stored in the pixel block storage unit 40 in pixel block units is sequentially read out and stored in the frame memory 50. As a result, image data corresponding to the original image is stored in frame units.
[0073]
As described above, according to the image data processing device illustrated in FIG. 1, the number of image data output from the image data decoding unit 10 in pixel block units is determined by the first image data generation unit 21 and the second image data The image data is converted in the image data generation unit 22 in two stages. Therefore, the conversion ratio of the number of image data at each stage is reduced, and the circuit configuration can be simplified. In addition, since a small pixel block as a unit for synthesizing image data can be reduced, it is easy to set a small pixel block so as not to span adjacent pixel blocks. This makes it possible to simplify the storage device for storing the image data of the boundary area.
[0074]
Further, according to the image data processing device shown in FIG. 1, the conversion of the number of image data is performed in two stages, so one of the stages, for example, the conversion process of the number of image data in the first image data generation unit 21 is performed. Is enabled or disabled, it is possible to perform a conversion process of the number of image data based on two types of conversion ratios. In this case, since the conversion process of the number of image data in the second image data generation unit 22 is shared to realize two conversion ratios, two image data processing devices that perform the conversion of the number of image data at different conversion ratios are used. The circuit scale can be reduced as compared with the method of providing individually.
[0075]
In the image data processing apparatus shown in FIG. 1, the conversion process of the number of image data is performed in two stages in the two image data generation units. The conversion process of the number of data can be performed in an arbitrary number of stages.
That is, a plurality of cascade-connected image data generating units are provided, and in the first stage, image data corresponding to each pixel of the original image is input for each first pixel block, and in each stage following the first stage, the image data is generated in the preceding stage. The input image data is input for each pixel block corresponding to the first pixel block. The image data generation unit of each stage combines the input image data for each small pixel block determined in each stage, and generates one or a plurality of image data corresponding to each of the small pixel blocks.
In addition, the boundary data storage unit is provided corresponding to a part or all of the plurality of cascade-connected image data generation units. The boundary data storage unit includes, among image data input to the corresponding image data generation unit, a predetermined boundary area of a pixel block corresponding to the first pixel block on an image formed by the input image data. The image data of the selected pixel is stored.
In the image data generation unit associated with the boundary data storage unit, when some pixels of the small pixel block to be synthesized are included in the boundary region of another pixel block adjacent to the input pixel block, By combining the image data of the partial pixels stored in the corresponding boundary data storage unit and the input image data of the remaining pixels of the small pixel block, one or the Generate a plurality of image data.
With such a configuration, it is possible to perform image data number conversion processing with a higher conversion ratio.
Further, in each of the plurality of cascade-connected image data generation units, it is determined whether or not to execute the above-described generation processing of the image data corresponding to the small pixel block according to the set conversion ratio of the number of image data. May be. Thereby, the conversion processing of the number of image data in the image data generation unit of each stage is arbitrarily enabled or disabled, so that the conversion processing of the number of image data with a plurality of conversion ratios can be easily realized.
[0076]
<Second embodiment>
Next, a second embodiment of the present invention will be described.
The image data processing device according to the second embodiment has, for example, a configuration equivalent to the image data processing device shown in FIG. However, the first image data generation unit 21, the second image data generation unit 22, and the boundary data storage unit 30 shown in FIG. 1 are replaced with a first image data generation unit 21A and a second image data generation unit described below. 22A and the boundary data storage unit 30A.
[0077]
FIG. 2 shows the first image data generation unit 21A, the second image data generation unit 22A, the boundary data storage unit 30A, and the control unit CONT1 included in the image data processing device according to the second embodiment of the present invention. It is a block diagram showing an example of a composition.
The first image data generation unit 21A illustrated in FIG. 2 includes flip-flops FF1 and FF2, an addition unit AD1, a multiplication unit ML1, and a selector SL1. The second image data generation unit 22A illustrated in FIG. 2 includes flip-flops FF3 to FF5, multiplication units ML2 to ML4, a selector SL3, and an addition unit AD2. The boundary data storage unit 30A shown in FIG. 2 includes registers RG1 and RG2 and a selector SL2.
The flip-flop FF2 is an embodiment of a first data holding unit according to the first aspect of the present invention.
The unit including the addition unit AD1 and the multiplication unit ML1 is an embodiment of the first data synthesizing unit according to the first aspect of the present invention.
The unit including the flip-flops FF3 to FF5 is an embodiment of the second data holding unit according to the first aspect of the present invention.
The unit including the multiplication units ML2 to ML4, the addition unit AD2, and the selector SL3 is an embodiment of the second data synthesizing unit according to the first aspect of the present invention.
[0078]
(First Image Data Generation Unit 21A)
The flip-flop FF1 holds the image data S1 of the 8 × 8 pixel chroma signal output from the image data decoding unit 10 in synchronization with the clock signal CLK.
The flip-flop FF2 holds the image data S2 of the chroma signal held in the flip-flop FF1 in synchronization with the clock signal CLK.
The adder AD1 adds the image data of the chroma signal held in the flip-flops FF1 and F2.
The multiplication unit ML1 multiplies the image data added by the addition unit AD1 by a weighting factor of １ ／.
[0079]
The selector SL1 selects and outputs the image data S3 held in the flip-flop FF2 or the image data multiplied by the weight coefficient in the multiplier ML1, according to the set conversion ratio of the number of image data.
That is, when the conversion process from the YCrCb ratio (4: 4: 4) to the YCrCb ratio (4: 1: 1) is performed, the image data of the multiplication unit ML1 is selected, and the YCrCb ratio (4: 4: 4) is selected. In the case where the conversion processing is performed to the YCrCb ratio (4: 1: 1), the image data S3 of the flip-flop FF2 is selected.
[0080]
(Second image data generation unit 22A)
The flip-flop FF3 holds the image data S30 output from the first image data generation unit 21 in synchronization with the clock signal CLK when the enable signal in-en1 output from the control unit CONT1 is at a low level.
In the case where the conversion process from the YCrCb ratio (4: 4: 4) to the YCrCb ratio (4: 1: 1) is performed, the enable signal in-en1 alternates between high level and low level every one cycle of the clock signal CLK. The level is set in the control unit CONT1 so as to be repeated. Thus, the flip-flop FF3 holds the image data S30 every two cycles of the clock signal CLK.
When the conversion process from the YCrCb ratio (4: 2: 2) to the YCrCb ratio (4: 1: 1) is performed, the enable signal in-en1 is set to a constant low level.
[0081]
The flip-flop FF4 holds the image data S4 held in the flip-flop FF3 in synchronization with the clock signal CLK when the enable signal in-en1 is at the low level.
The flip-flop FF5 holds the image data S5 held in the flip-flop FF4 in synchronization with the clock signal CLK when the enable signal in-en1 is at a low level.
[0082]
The selector SL3 selects and outputs the image data S6 held in the flip-flop FF5 when the selection signal blk-left output from the control unit CONT1 is at a low level, and outputs the image data S6 when the selection signal blk-left is at a high level. , And selects and outputs the image data S7 output from the boundary data storage unit 30.
The selection signal blk-left indicates that two pixels corresponding to the image data S4 and S5 and one pixel corresponding to the image data S7 of the predetermined boundary area output from the boundary data storage unit 30A are on the same horizontal line. When a small pixel block of 1 × 3 pixels is formed (corresponding to the second small pixel block described in the first embodiment), the high level is set by the control unit CONT1.
[0083]
The multiplication unit ML2 multiplies the image data S4 held in the flip-flop FF3 by a weighting factor of 1/4.
The multiplying unit ML3 multiplies the image data S5 held in the flip-flop FF4 by a weighting factor of 1/2.
The multiplier ML4 multiplies the image data output from the selector SL3 by a weighting factor of 1/4.
In the example of FIG. 2, since the weighting coefficients of the multiplication units ML1 to ML4 all have values of powers of 2, these multiplication units can be realized by a circuit that performs a simple bit shift. There is no need to use.
[0084]
The addition unit AD2 adds the multiplication results of the multiplication units ML2 to ML4, and outputs the addition result to the pixel block storage unit 40 as image data S8.
The image data S8 output from the addition unit AD2 is stored in the pixel block storage unit 40 as valid image data when the enable signal out-en output from the control unit CONT1 is at a high level.
In the case where the conversion process from the YCrCb ratio (4: 4: 4) to the YCrCb ratio (4: 1: 1) is performed, the enable signal out-en alternates between a high level and a low level every two cycles of the clock signal CLK. The level is set in the control unit CONT1 so as to be repeated.
When the conversion process from the YCrCb ratio (4: 2: 2) to the YCrCb ratio (4: 1: 1) is performed, the enable signal out-en alternates between a high level and a low level every cycle of the clock signal CLK. Are set in the control unit CONT1 so as to be repeated.
[0085]
(Boundary data storage unit 30A)
When the enable signal xcb-en output from the control unit CONT1 is at a low level, the register RG1 holds the image data S30 output from the first image data generation unit 21A in synchronization with the clock signal CLK. In this case, the image data that is already held is shifted in synchronization with the clock signal CLK, and is output to the selector SL2 in the order in which the image data is held. The register RG1 holds up to eight image data.
The enable signal xcb-en is a predetermined boundary of the image data S30 of the 8 × 4 pixel block output from the first image data generation unit 21A according to the image data S1 of the chroma signal Cb of the 8 × 8 pixel block. At the timing when the image data included in the area is output from the first image data generation 21A, it is set to the low level.
[0086]
The register RG2 holds the image data S30 output from the first image data generation unit 21A in synchronization with the clock signal CLK when the enable signal xcr-en output from the control unit CONT1 is at a low level. In this case, the image data that is already held is shifted in synchronization with the clock signal CLK, and is output to the selector SL2 in the order in which the image data is held. The register RG2 also holds up to eight image data.
The enable signal xcr-en is a predetermined boundary of the 8 × 4 pixel block image data S30 output from the first image data generation unit 21A in accordance with the 8 × 8 pixel block chroma signal Cr image data S1. At the timing when the image data included in the area is output from the first image data generation 21A, it is set to the low level.
[0087]
The selector SL2 selects image data output from either the register RG1 or the register RG2 according to the selection signal area-cr output from the control unit CONT1, and outputs the selected image data as image data S7. When the image data S1 is the chroma signal Cb, the register RG3 is selected, and when the image data S1 is the chroma signal Cr, the register RG4 is selected.
[0088]
Here, regarding the operation of the image data processing device having the above-described configuration, the conversion process from the YCrCb ratio (4: 4: 4) to the YCrCb ratio (4: 1: 1) is performed, and the YCrCb ratio (4: 4: 4) is performed. (2: 2) to the YCrCb ratio (4: 1: 1).
[0089]
// Conversion process from YCrCb ratio (4: 4: 4) to (4: 1: 1)
FIG. 3 shows an outline of a chroma signal interpolation process for converting image data S1 having a YCrCb ratio (4: 4: 4) output from the image data decoding unit 10 into image data S8 having a YCrCb ratio (4: 1: 1). FIG.
FIG. 4 is an example of the relationship between the value of each signal and the timing shown in the block diagram of FIG. 2 when the conversion process from the YCrCb ratio (4: 4: 4) to the YCrCb ratio (4: 1: 1) is performed. FIG.
[0090]
Pixel blocks B0 to B2 illustrated in FIG. 3A indicate image data of a chroma signal of an 8 × 8 pixel block output from the image data decoding unit 10. The pixel blocks of the chroma signal are output from the image data decoding unit 10 in the order of the pixel blocks B0, B1, B2,. The image data S1 of this pixel block (FIG. 4B) is the image data C1. ₀ , C1 ₁ , ..., C1 ₆₃ , C2 ₀ , C2 ₁ ,... Are output in synchronization with the clock signal CLK (FIG. 4A).
[0091]
The image data S1 output from the image data decoding unit 10 is sequentially input to the first image data generation unit 21A via the flip-flop FF1. The input image data S2 (FIG. 4C) is held in the flip-flop FF2 in synchronization with the clock signal CLK. The image data S3 (FIG. 4D) held in the flip-flop FF2 and the image data S2 (FIG. 4C) subsequently input to the first image data generation unit 21A are added to the addition unit. AD1 is added, and the addition result is multiplied by a coefficient 係数 in a multiplication unit ML. As a result, the multiplying unit ML1 outputs image data that is obtained by adding two successive image data (S3, S2) output from the image data decoding unit 10 to each other with weighting of 1/2. The synthesized image data output from the multiplication unit ML1 is selected by the selector SL1, and output from the first image data generation unit 21A as the image data S30.
[0092]
The image data S30 output from the first image data generation unit 21A is output when the enable signal in-en1 output from the control unit CONT1 goes low, that is, every two cycles of the clock signal CLK (FIG. 4A). The data is input to the second image data generation unit 22A via the flip-flop FF3. Therefore, the image data S4 (FIG. 4E) input to the second image data generation unit 22A is adjacent to the same horizontal line of the 8 × 8 pixel block output from the image data decoding unit 10. The image data is generated corresponding to a 1 × 2 pixel small pixel block (corresponding to the first small pixel block described in the first embodiment).
[0093]
The pixel blocks B3 to B5 illustrated in FIG. 3B are pixel blocks obtained by converting the 8 × 8 pixel pixel blocks B0 to B2 in FIG. 3A into 8 × 4 pixels in the first image data generation unit 21A. It is. Each image data of the pixel blocks B3 to B5 corresponds to the image data S4 input to the second image data generation unit 22A.
For example, as shown in FIG. 3A, the image data C1 of the pixel block B1 is displayed. ₀ And C1 ₁ Is a small pixel block MB1 of 1 × 2 pixels ₀ Is composed. Small pixel block MB1 ₀ Are combined in the first image data generation unit 21A, so that the image data C4 of the pixel block B4 is ₀ (FIG. 3B) is generated. This image data C4 ₀ Is held in the flip-flop FF3 at the time t11 shown in the timing chart of FIG. 4 (FIG. 4E).
Further, as shown in FIG. 3A, the image data C1 of the pixel block B1 is displayed. ₂ And C1 ₃ Is the small pixel block MB1 ₁ Is composed. Small pixel block MB1 ₁ Are combined in the first image data generation unit 21A, so that the image data C4 of the pixel block B4 is ₁ (FIG. 3B) is generated. This image data C4 ₁ Is held in the flip-flop FF3 at the time t12 shown in the timing chart of FIG. 4 (FIG. 4E).
[0094]
The image data S4 (FIG. 4E) input to the second image data generation unit 22A is held in the flip-flop FF4 as image data S5 (FIG. 4F) every two cycles of the clock signal CLK. . The image data S5 is held in the flip-flop FF5 as image data S6 (FIG. 4G) every two cycles of the clock signal CLK. The image data S6 and S5 held in the flip-flops FF5 and FF4 and the image data S4 subsequently input to the second image data generator 22A are given predetermined weights according to the input order. . That is, the image data S6 is multiplied by the weight coefficient １ ／ in the multiplication unit ML4, the image data S5 is multiplied by the weight coefficient １ ／ in the multiplication unit ML3, and the image data S4 is multiplied by the weight coefficient 1 in the multiplication unit ML2. / 4 is multiplied. The weighted image data is added in the adder AD2 and output from the second image data generator 22A as image data S8 (FIG. 4K). As a result, three consecutive image data input to the second image data generation unit 22A are given a predetermined weight according to the input order and are synthesized. The synthesized image data is output as image data S8 from the second image data generation unit 22A every two cycles of the clock signal CLK.
[0095]
When the enable signal out-en (FIG. 4 (L)) output from the control unit CONT1 is at a high level, that is, every four cycles of the clock signal CLK, the image data S8 is used as valid image data in the pixel block storage unit. 40. Therefore, the image data S8 (FIG. 4 (K)) stored in the pixel block storage unit 40 is such that one pixel at both ends overlaps with each other in the 8 × 4 pixel block input from the first image data generation unit 21A. Image data is synthesized for each 1 × 3 pixel small pixel block (second small pixel block) arranged adjacently on the horizontal line.
[0096]
The pixel blocks B6 and B7 shown in FIG. 3C are pixel blocks obtained by converting the 8 × 4 pixel pixel blocks B4 and B5 in FIG. 3B into 8 × 2 pixels in the second image data generation unit 22A. It is. Each image data of the pixel blocks B6 and B7 corresponds to the image data S8 stored in the pixel block storage unit 40.
For example, as shown in FIG. 3B, the image data C4 in the pixel block B4 ₁ ~ C4 ₃ Is a small pixel block MB4 of 1 × 3 pixels ₁ Is composed. This small pixel block MB4 ₁ Are combined in the second image data generation unit 22A, so that the image data C6 of the pixel block B6 is ₁ (FIG. 3C) is generated. This image data C6 ₁ Is output from the second image data generation unit 22A at time t14 shown in the timing chart of FIG. 4 (FIG. 4K).
[0097]
On the other hand, among the image data S30 output from the first image data generation unit 21A, the 8 × 4 pixel block input to the second image data generation unit 22A is stored in the registers REG1 and REG2 of the boundary data storage unit 30A. , The image data included in the predetermined boundary area is held for eight pixels while being sequentially shifted.
[0098]
In FIG. 3B, regions L0 to L2 of the 8 × 1 pixel block at the right end of the 8 × 4 pixel block correspond to the above-described boundary region. When the image data S30 output from the first image data generation unit 21A is included in this boundary region, the control unit CONT1 generates an enable signal xcb-en or xcr-en corresponding to the type of the chroma signal (Cr, Cb). The image data S30 set to the low level and output from the first image data generation unit 21A is held in the register REG1 or REG2.
[0099]
For example, at time t12 in the timing chart shown in FIG. 4, the register REG2 of the boundary data storage unit 30A stores the image data C3 of the pixel block B3. ₃ , C3 ₇ , C3 ₁₁ , C3 _Fifteen , C3 ₁₉ , C3 ₂₃ , C3 ₂₇ And C3 ₃₁ Is held. Of the eight pieces of image data, the first held image data C3 ₃ Are output to the second image data generation unit 22A as image data S7 (FIG. 4H).
At time t13, the image data C1 of the pixel block B1 ₆ And C1 ₇ Are synthesized, and the image data C4 included in the boundary area L1 is synthesized by the synthesis. ₃ Is generated, the generated image data C4 ₃ Is output from the first image data generation unit 21A as image data S30. In this clock cycle, the enable signal xcr-en is set to the low level by the control unit CONT1.
[0100]
When the enable signal xcr-en is set to the low level, the image data C4 output as the image data S30 at the time t14 in the next clock cycle. ₃ Is image data C3 ₃ Is newly stored in the register REG2 (FIG. 4E). At the same time, the data is shifted in the register REG2, and the image data C3 ₃ Image data C3 held next to ₇ Is newly output from the register REG2. As a result, the register REG2 stores the image data C3 of the pixel block B3. ₇ , C3 ₁₁ , C3 _Fifteen , C3 ₁₉ , C3 ₂₃ , C3 ₂₇ And C3 ₃₁ And the image data C4 of the pixel block B4 ₃ And are held.
In this way, the registers RG1 and REG2 hold the image data of the boundary area while being sequentially shifted.
[0101]
The 8 × 4 pixel block adjacent to the 8 × 4 pixel block input to the second image data generation unit 22A, that is, the 8 × 4 pixel block input immediately before the 8 × 4 pixel block being input When the above-described boundary area includes the leftmost pixel in the 1 × 3 pixel small pixel block to be synthesized, the image output from the register REG1 or REG2 as the image data corresponding to the leftmost pixel The data S7 is used for synthesis with the image data S4 and S5 instead of the image data S6.
That is, in this case, since two pixels corresponding to the image data S4 and S5 and one pixel corresponding to the image data S6 are located on different horizontal lines, it is impossible to combine these image data. Appropriate image data S8 is generated. On the other hand, the two pixels corresponding to the image data S4 and S5 and the pixel in the boundary area corresponding to the image data S7 are located on the same horizontal line, and form a small pixel block of 1 × 3 pixels. I do. The enable signal blk-left is set to the high level by the control unit CONT1 at this timing, the image data S7 is selected by the selector SL3, and the selected image data S7 is used to generate the image data S8.
[0102]
For example, at time t12 in FIG. 4, image data C4 is used as image data S4 and S5. ₁ And C4 ₀ Are stored in the flip-flops FF3 and FF4, and the image data C3 ₃₁ Are held in the flip-flop FF5. As shown in FIG. 3B, the image data C4 ₁ And C4 ₀ And the image data C3 ₃₁ Are located on a different horizontal line from one pixel corresponding to. On the other hand, the image data C3 output from the boundary data storage unit 30A as the image data S7 ₃ Of the image data C4 ₀ And C4 ₁ Are located on the same horizontal line as the pixel corresponding to the small pixel block MB4 of 1 × 3 pixels. ₀ Is composed. At this timing, the enable signal blk-left is set to the high level. As a result, the image data C3 held in the flip-flop FF5 ₃₁ Image data C3 output from the boundary data storage unit 30A instead of ₃ Is selected in the selector SL3, multiplied by a weighting factor of 1/4 in the multiplication unit ML4, and input to the addition unit AD2. As a result, the small pixel block MB4 ₀ Image data C4 constituting ₁ , C4 ₀ , And C3 ₃ Are synthesized, and the image data C6 ₀ Is output from the second image data generation unit 22A.
[0103]
// Conversion processing from YCrCb ratio (4: 2: 2) to (4: 1: 1)
FIG. 5 shows an outline of a chroma signal interpolation process for converting image data S1 having a YCrCb ratio (4: 2: 2) output from the image data decoding unit 10 into image data S8 having a YCrCb ratio (4: 1: 1). FIG.
FIG. 6 is an example of the relationship between the value of each signal and the timing shown in the block diagram of FIG. 2 when the conversion process from the YCrCb ratio (4: 2: 2) to the YCrCb ratio (4: 1: 1) is performed. FIG.
[0104]
Pixel blocks B8 and B9 shown in FIG. 5A indicate image data of a chroma signal of an 8 × 8 pixel block output from the image data decoding unit 10. The pixel blocks of the chroma signal are output from the image data decoding unit 10 in the order of the pixel blocks B8, B9,. Each image data S1 of this pixel block is represented by image data C8 ₀ , C8 ₁ , ..., C8 ₆₃ , C9 ₀ , C9 ₁ ,... Are output in synchronization with the clock signal CLK (FIG. 6A).
[0105]
The image data S1 output from the image data decoding unit 10 is held in the order of the flip-flop FF1 and the flip-flop FF2 in synchronization with the clock signal CLK, and then, as it is, via the selector SL1, the image data S30 (FIG. 6 ( B)). That is, the first image data generation unit 21A does not perform the image data combining process, and the input image data S1 is output as it is as the image data S30.
[0106]
When the conversion process from the YCrCb ratio (4: 2: 2) to the YCrCb ratio (4: 1: 1) is performed, the enable signal in-en1 is set to a low level, so that in the flip-flops FF3 to FF5, The input data holding operation is performed in each cycle of the clock signal CLK. That is, the image data S30 output from the first image data generation unit 21A is sequentially held in the flip-flops FF3 to FF5 of the second image data generation unit 22A in synchronization with the clock signal CLK.
[0107]
The image data S6 (FIG. 6E) and the image data S5 (FIG. 6D) held in the flip-flops FF5 and FF4, and the image data subsequently input to the second image data generation unit 22A S4 (FIG. 6C) is given a predetermined weight according to the input order. That is, the image data S6, S5, and S4 are multiplied by the weighting factors 1/4, 1/2, and 1/4 in the multipliers ML4, ML3, and ML2. The weighted image data is added in the adder AD2 and output from the second image data generator 22A as image data S8 (FIG. 6 (I)). As a result, three consecutive image data output from the image data decoding unit 10 are given a predetermined weight according to the input order and are synthesized. The combined image data is output from the second image data generation unit 22A as image data S8.
[0108]
When the enable signal out-en (FIG. 6 (J)) output from the control unit CONT1 is at a high level, that is, every two cycles of the clock signal CLK, the image data S8 is used as valid image data in the pixel block storage unit. 40. Therefore, the image data S8 (FIG. 6 (I)) stored in the pixel block storage unit 40 is horizontal so that one pixel at both ends overlaps with each other in the 8 × 8 pixel block output from the image data decoding unit 10. The image data is synthesized for each small pixel block of 1 × 3 pixels arranged adjacently on the line.
[0109]
The pixel block B10 shown in FIG. 5B is a pixel block obtained by converting the pixel block B9 of 8 × 8 pixels in FIG. 5A into 8 × 4 pixels in the second image data generation unit 22A. Each image data of the pixel block B10 corresponds to the image data S8 stored in the pixel block storage unit 40.
For example, as shown in FIG. 5A, the image data C9 in the pixel block B9 ₁ ~ C9 ₃ Is a small pixel block MB9 of 1 × 3 pixels. ₁ Is composed. This small pixel block MB9 ₁ Are combined in the second image data generation unit 22A, so that the image data C10 of the pixel block B10 is ₁ (FIG. 5B) is generated. This image data C10 ₁ Is output from the second image data generation unit 22A at time t22 shown in the timing chart of FIG. 6 (FIG. 6 (I)).
[0110]
On the other hand, in the registers REG1 and REG2 of the boundary data storage unit 30A, a predetermined boundary in the 8 × 8 pixel block of the image data S30 of the 8 × 8 pixel block output from the first image data generation unit 21A is stored. The image data included in the area is held for eight pixels while being sequentially shifted.
[0111]
In FIG. 5A, regions L8 and L9 of the 8 × 1 pixel block at the right end of the 8 × 8 pixel block correspond to the above-described boundary region. When the image data S30 output from the first image data generation unit 21A is included in this boundary region, the control unit CONT1 generates an enable signal xcb-en or xcr-en corresponding to the type of the chroma signal (Cr, Cb). The image data S30 set to the low level and output from the first image data generation unit 21A is held in the register REG1 or REG2.
[0112]
For example, at time t23 in the timing chart shown in FIG. 4, the register REG2 of the boundary data storage unit 30A stores the image data C8 of the pixel block B8. ₇ , C8 _Fifteen , C8 ₂₃ , C8 ₃₁ , C8 ₃₉ , C8 ₄₇ , C8 ₅₅ And C8 ₆₃ Is held. Of the eight pieces of image data, the first held image data C8 ₇ Are output as image data S7 to the second image data generation unit 22A.
At time t23, the image data C9 included in the boundary area L9 ₇ Is output from the first image data generation unit 21A as image data S30 (FIG. 6B), the enable signal xcr-en is set to a low level by the control unit CONT1 (FIG. 6H).
[0113]
When the enable signal xcr-en is set to low level, the image data C9 output as the image data S30 at the time t24 in the next clock cycle. ₇ Is the image data C8 ₇ Is newly held in the register REG2. At the same time, the data is shifted in the register REG2, and the image data C8 ₇ Image data C8 held next to _Fifteen Is newly output from the register REG2. As a result, the register REG2 stores the image data C8 of the pixel block B8. _Fifteen , C8 ₂₃ , C8 ₃₁ , C8 ₃₉ , C8 ₄₇ , C8 ₅₅ And C8 ₆₃ And the image data C9 of the pixel block B9 ₇ And are held.
In this way, the registers RG1 and REG2 hold the image data of the boundary area while being sequentially shifted.
[0114]
The image data S7 output by the shift operation from the registers REG1 and REG2 is an 8 × 8 pixel block adjacent to the 8 × 8 pixel block input to the second image data generation unit 22A, that is, the input image data S7. When the above-described boundary region of the 8 × 8 pixel block input immediately before the 8 × 8 pixel block includes the left end pixel of the 1 × 3 pixel small pixel block to be synthesized, The image data corresponding to the pixel is used for synthesis with the image data S4 and S5 instead of the image data S6.
[0115]
For example, at time t21 in FIG. 6, image data C9 is used as image data S4 and S5. ₁ And C9 ₀ Are stored in the flip-flops FF3 and FF4, and the image data C8 ₆₃ Are held in the flip-flop FF5. As shown in FIG. 5A, the image data C9 ₁ And C9 ₀ And the image data C8 ₆₃ Are located on a different horizontal line from one pixel corresponding to. In contrast, image data C8 output from boundary data storage unit 30A as image data S7 ₇ Is the image data C9 ₀ And C9 ₁ And a small pixel block MB9 of 1 × 3 pixels. ₀ Is composed. At this timing, the enable signal blk-left is set to the high level. As a result, the image data C8 held in the flip-flop FF5 ₆₃ Image data C8 output from the boundary data storage unit 30 in place of ₇ Is selected in the selector SL3, multiplied by a weighting factor of 1/4 in the multiplication unit ML4, and input to the addition unit AD2. As a result, the small pixel block MB9 ₀ Image data C8 constituting ₇ , C9 ₀ , And C9 ₁ Are synthesized and the image data C10 ₀ Is output from the second image data generation unit 22A.
[0116]
As described above, according to the image data processing apparatus including the first image data generation unit 21A, the second image data generation unit 22A, and the boundary data storage unit 30A illustrated in FIG. 2, the image data illustrated in FIG. Similarly to the processing device, the number of image data output from the image data decoding unit 10 in pixel block units is converted in two stages in the first image data generation unit 21A and the second image data generation unit 22A. Is done. Therefore, the conversion ratio of the number of image data at each stage is reduced, and the circuit configuration can be simplified. In addition, since a small pixel block as a unit for synthesizing image data can be reduced, it is easy to set a small pixel block so as not to span adjacent pixel blocks. This makes it possible to simplify the storage device for storing the image data of the boundary area.
[0117]
By enabling or disabling the conversion processing of the number of image data in the first image data generation unit 21A, the number of image data can be converted by two types of conversion ratios. In this case, since the conversion processing of the number of image data in the second image data generation unit 22A is shared in order to realize two conversion ratios, the two image data processing devices that convert the number of image data with different conversion ratios Can be simplified as compared with the method of individually providing.
[0118]
A register that performs a shift operation is used in the boundary data storage unit 30A. Image data included in the boundary area is sequentially stored in this register, and the stored image data is sequentially shifted in the register. Since the image data is supplied to the second image data generation unit 22A and used for synthesizing the image data, no special processing for reading the image data from the boundary data storage unit 30A is required, and the circuit can be simplified.
[0119]
The selection signal (blk-left, area-cr) and the enable signal (xcb-en, xcr-en, in-en1, out-en) generated in the control unit CONT1 are output from the image data decoding unit 10. The control unit CONT1 has a certain timing within a block of one MCU of image data, and each time the image data of one MCU is repeatedly output, a selection signal and an enable signal at the same timing are generated in the control unit CONT1.
Therefore, for example, a counter which is initialized at the head of the image data of the MCU output from the image data decoding unit 10 and counts each time image data is output, and a selection signal ( By providing the control unit CONT1 with a decoding circuit for generating the blk-left, area-cr) and the enable signal (xcb-en, xcr-en, in-en1, out-en), these signals can be easily generated. can do.
[0120]
<Third embodiment>
Next, a third embodiment of the present invention will be described.
In the image data processing apparatuses according to the above-described first and second embodiments, a conversion process of the number of image data is performed on image data input in units of predetermined pixel blocks. On the other hand, in the image data processing device according to the third embodiment, the image data input from the frame memory or the like is subjected to a conversion process of the number of image data, and the converted image data is converted into a predetermined pixel block unit. Is output.
[0121]
FIG. 7 is a block diagram illustrating an example of a configuration of an image data processing device according to the third embodiment of the present invention.
The image data processing device shown in FIG. 7 includes a frame memory 50, a pixel block storage unit 60, a boundary data storage unit 70, an image data generation unit 81, an image data generation unit 82, a selector SL4, an image data code And a conversion unit 90. However, the same reference numerals in FIGS. 7 and 1 indicate the same components.
The frame memory 50 is an embodiment of a frame image storage unit according to the third aspect of the present invention.
The pixel block storage unit 60 is an embodiment of a pixel block storage unit according to the third aspect of the present invention.
The boundary data storage unit 70 is an embodiment of a boundary data storage unit according to the third aspect of the present invention.
The image data generation unit 81 and the image data generation unit 82 are an embodiment of a plurality of image data generation units according to the third aspect of the present invention.
The selector SL4 is an embodiment of the selecting means according to the third aspect of the present invention.
The image data encoding unit 90 is an embodiment of an image data encoding unit according to the third aspect of the present invention.
[0122]
(Pixel block storage unit 60)
The pixel block storage unit 60 reads and stores the image data corresponding to each pixel of the original image stored in the frame memory 50 for each predetermined pixel block according to the set conversion ratio of the number of image data.
[0123]
For example, when the conversion ratio is set so as to perform conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 4: 4), the image data of the chroma signals Cr and Cb is divided into 8 × 2 pixel blocks. Is read from the frame memory 50 and stored.
When the conversion ratio is set so as to perform conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 2: 2), the chroma signal Cr and the image data of Cb are divided into 8 × 4 pixel blocks. Is read from the frame memory 50 and stored.
[0124]
The pixel block storage unit 60 can have, for example, a two-bank configuration in which a memory for writing and a memory for reading are separately provided. By using a two-bank configuration, the memory for writing and the memory for reading are alternately switched to write the image data read from the frame memory 50 and to write the image data to the image data generator 81 or the image data generator 82. And reading of data can be performed simultaneously.
[0125]
(Image data generation units 81 and 82)
The image data generation units 81 and 82 read out the image data stored in the pixel block storage unit 60, synthesize the read out image data for each predetermined small pixel block, and generate one or more image data for each small pixel block. Generate image data.
[0126]
For example, when the conversion ratio is set so as to perform the conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 4: 4), the image data generation unit 81 stores in the pixel block storage unit 60 The stored image data of the 8 × 2 pixel block is sequentially read. In this case, for example, the small pixel block is set as a 1 × 2 pixel block arranged on the horizontal line of the original image such that one pixel at each end overlaps each other. The image data generation unit 81 combines the sequentially read image data for each of the small pixel blocks by four types of predetermined weighting methods, and generates image data for four pixels corresponding to each small pixel block. Thereby, the image data of the 8 × 2 pixel block stored in the pixel block storage unit 60 is converted into the image data of the 8 × 8 pixel block, and the number of image data is expanded four times.
[0127]
Further, for example, when the conversion ratio is set so as to perform conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 2: 2), the image data generation unit 82 operates in the pixel block storage unit. The image data of the 8 × 4 pixel block stored in the memory 60 is sequentially read. In this case, for example, the small pixel block is set as a 1 × 2 pixel block arranged on the horizontal line of the original image such that one pixel at each end overlaps each other. The image data generation unit 82 combines the sequentially read image data for each of the small pixel blocks by two types of predetermined weighting methods, and generates image data for two pixels corresponding to each small pixel block. Thus, the image data of the 8 × 4 pixel block stored in the pixel block storage unit 60 is converted into the image data of the 8 × 8 pixel block, and the number of image data is doubled.
[0128]
However, when some of the pixels of the small pixel block are included in a predetermined boundary area of another pixel block adjacent to the pixel block being read from the pixel block storage unit 60, The image data of the partial pixels stored in the boundary data storage unit 70 to be described later is combined with the image data of the remaining pixels of the small pixel block read from the pixel block storage unit 60 to form the small pixel block. Is generated.
[0129]
For example, when the conversion ratio is set to perform conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 4: 4), the 8 × 2 pixels stored in the pixel block storage unit 60 are used. Assuming that a small pixel block of 1 × 2 pixels is arranged on the left side of the block, among the two small pixel blocks arranged on the horizontal line of the 8 × 2 pixel block, the small pixel block located on the right side Is not included in the 8 × 2 pixel block whose right end is being read from the pixel block storage unit 60, but is included in the left end 8 × 1 pixel region in the 8 × 2 pixel block on the right. . In this case, the above-described boundary region corresponds to the leftmost 8 × 1 pixel region.
When synthesizing a small pixel block of 1 × 2 pixels located on the right side of the 8 × 2 pixel block, the image data generation unit 81 transfers the image data of the rightmost one pixel of the small pixel block from the boundary data storage unit 70. read out. Then, the image data read from the boundary data storage unit 70 and the image data of the remaining one pixel of the small pixel block read from the pixel block storage unit 60 are combined, and four pixels corresponding to the small pixel block are combined. Generate image data for each minute.
[0130]
Further, for example, when the conversion ratio is set so as to perform conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 2: 2), the 8 × 4 stored in the pixel block storage unit 60 is used. Assuming that a small pixel block of 1 × 2 pixels is arranged on the left side of the four pixel block, among the four small pixel blocks arranged on the horizontal line of the 8 × 4 pixel block, The rightmost one pixel of the pixel block is not included in the 8 × 4 pixel block being read from the pixel block storage unit 60, but is included in the leftmost 8 × 1 pixel region of the right neighboring 8 × 4 pixel block. It is. In this example, the above-described boundary region corresponds to the region of the leftmost 8 × 1 pixel in the 8 × 4 pixel block.
When synthesizing a 1 × 2 pixel small pixel block located on the rightmost side of the 8 × 4 pixel block, the image data generating unit 82 stores the rightmost one pixel image data of the small pixel block in the boundary data storage unit 70. Read from Then, the image data read from the boundary data storage unit 70 and the image data of the remaining one pixel of the small pixel block read from the pixel block storage unit 60 are combined, and four pixels corresponding to the small pixel block are combined. Generate image data for each minute.
[0131]
(Boundary data storage unit 70)
The boundary data storage unit 70 may be a different pixel block adjacent to a pixel block next read from the pixel block storage unit 60 by the image data generation unit 81 or 82, or a pixel block generated by these image data generation units. The image data which is another pixel block adjacent to the pixel block being read from the block storage unit 60 and which is included in a predetermined boundary area according to the set conversion ratio of the number of image data is read out from the frame memory 50. Remember.
The reading of the image data from the frame memory 50 by the boundary data storage unit 70 is performed when the image data is read from the frame memory 50 in the pixel block storage unit 60.
[0132]
For example, when the pixel block storage unit 60 has the above-described two-bank configuration, the pixel block storage unit 60 reads image data from the frame memory 50 while reading image data to the image data generation unit 81 or 82. It is possible to memorize. In this case, the image data read from the frame memory 50 by the pixel block storage unit 60 is the image data to be supplied from the pixel block storage unit 60 to the image data generation unit 81 or 82 next. The boundary data storage unit 70 can fetch and store the image data included in the predetermined boundary region from the image data read from the frame memory 50 as needed.
However, in this case, before the image data in the boundary data storage unit 70 is read by the image data generation unit 81 or 82, the pixel block storage unit 60 is configured to store the read image data in the boundary data storage unit 70 in advance. It is necessary to appropriately set the timings of writing and reading in.
[0133]
On the other hand, when the pixel block storage unit 60 does not have the above-described two-bank configuration, the boundary data storage unit 70 reads the image data from the frame memory 50 when the pixel block storage unit 60 reads out the image data. The image data of the boundary area adjacent to the pixel block of the image data to be read is also read out.
[0134]
(Selector SL4)
The selector SL4 outputs the image data of the image data generation unit selected from the image data generation unit 81 or the image data generation unit 82 according to the set conversion ratio of the number of image data.
For example, when the conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 4: 4) is performed, the image data generated by the image data generation unit 81 is selected and output, and the YCrCb ratio ( 4: 1: 1) to the YCrCb ratio (4: 2: 2), the image data generated by the image data generator 82 is selected and output.
[0135]
(Image Data Decoding Unit 90)
The image data decoding unit 90 uses a predetermined number of image data blocks including the image data blocks output from the selector SL4 corresponding to the image data of the pixel blocks stored in the pixel block storage unit 60 as a unit. Encoding processing of image data such as JPEG method is performed.
[0136]
For example, when the conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 4: 4) is performed, the conversion is performed in accordance with the chroma signal of the 8 × 2 pixel block stored in the pixel block storage unit 60. 1 MCU in YCrCb ratio (4: 4: 4) format combining the chroma signal of the 8 × 8 pixel block output from the selector SL4 and the luminance signal of one 8 × 8 pixel block separately read from the frame memory 50 The JPEG encoding process is performed in units of image data (see FIG. 20).
For example, when the conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 2: 2) is performed, the conversion corresponds to the chroma signal of the 8 × 4 pixel block stored in the pixel block storage unit 60. YCrCb ratio (4: 2: 2) format in which the chroma signal of the 8 × 8 pixel block output from the selector SL4 and the luminance signals of the two 8 × 8 pixel blocks separately read from the frame memory 50 are combined. The encoding process of the JPEG method is performed in units of the image data of one MCU (see FIG. 21).
[0137]
Here, the operation of the image data processing device shown in FIG. 7 having the above configuration will be described.
[0138]
Image data corresponding to each pixel of the original image stored in the frame memory 50 is read out and stored in the pixel block storage unit 60 for each predetermined pixel block corresponding to the set conversion ratio of the number of image data. .
For example, when the conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 4: 4) is performed, the YCrCb ratio (4: 1: 1) to the YCrCb ratio ( When the conversion to 4: 2: 2) is performed, the image data of the chroma signals Cr and Cb is read from the frame memory 50 and stored in the pixel block storage unit 60 for each 8 × 4 pixel block.
[0139]
The image data stored in the pixel block storage unit 60 is sequentially read out by the image data generation unit 81 or the image data generation unit 82, and the read image data is synthesized for each predetermined small pixel block, and One or more image data is generated for each pixel block.
[0140]
For example, when the conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 4: 4) is performed, the image data of the 8 × 2 pixel block stored in the pixel block storage unit 60 is used to generate image data. The data is sequentially read by the unit 81. The read image data is synthesized by four types of predetermined weighting methods for each 1 × 2 pixel small pixel block arranged on the horizontal line of the original image such that the pixels at both ends overlap each other. . Thereby, image data for four pixels corresponding to each small pixel block is generated. As a result, the image data of the 8 × 2 pixel block stored in the pixel block storage unit 60 is converted into the image data of the 8 × 8 pixel block, and the number of image data is expanded four times.
[0141]
For example, when the conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 2: 2) is performed, the image data of the 8 × 4 pixel block stored in the pixel block storage unit 60 is used to generate image data. The data is sequentially read out by the unit 82. The read image data is synthesized by two types of predetermined weighting methods for each 1 × 2 pixel small pixel block arranged on the horizontal line of the original image such that the pixels at both ends overlap each other. . Thereby, image data for two pixels corresponding to each small pixel block is generated. As a result, the image data of the 8 × 4 pixel block stored in the pixel block storage unit 60 is converted into the image data of the 8 × 8 pixel block, and the number of image data is doubled.
[0142]
However, when some pixels of the small pixel block are included in a predetermined boundary area of another pixel block adjacent to the pixel block being read from the pixel block storage unit 60, the image data generation unit 81 and the image data generation unit At 82, the image data of some of the pixels stored in the boundary data storage unit 70 is read. Then, the image data read from the boundary data storage unit 70 and the image data of the remaining pixels of the small pixel block read from the pixel block storage unit 60 are combined. Thereby, one or a plurality of image data corresponding to the small pixel block is generated.
[0143]
For example, when the conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 4: 4) is performed, the conversion is performed on the left side of the 8 × 2 pixel block stored in the pixel block storage unit 60 and 1 × Assuming that a small pixel block of two pixels is arranged, the image data generation unit 81, when combining a small pixel block located on the right side of the 8 × 2 pixel block, one pixel at the right end of the small pixel block Are read from the boundary data storage unit 70. Then, the one-pixel image data read from the boundary data storage unit 70 and the remaining one-pixel image data of the small pixel block read from the pixel block storage unit 60 are obtained by four types of weighting methods. The images are combined to generate image data for four pixels corresponding to the small pixel block.
[0144]
For example, when the conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 2: 2) is performed, the left side of the 8 × 4 pixel block stored in the pixel block storage unit 60 is packed into 1 × Assuming that small pixel blocks of two pixels are arranged, the image data generating unit 82 combines the small pixel block located on the rightmost side of the 8 × 4 pixel block with the rightmost one of the small pixel blocks. The image data of the pixel is read from the boundary data storage unit 70. Then, the image data of one pixel read from the boundary data storage unit 70 and the image data of the remaining one pixel of the small pixel block read from the pixel block storage unit 60 are obtained by two types of weighting methods. The images are combined to generate image data for two pixels corresponding to the small pixel block.
[0145]
In the boundary data storage unit 70, another pixel block adjacent to a pixel block next read from the pixel block storage unit 60 by the image data generation unit 81 or 82 or a pixel block by these image data generation units. Image data that is another pixel block adjacent to the pixel block being read from the block storage unit 60 and that is included in a predetermined boundary area according to the set conversion ratio of the number of image data is read from the frame memory 50. Is stored. The reading process from the frame memory 50 is performed together with the reading process from the frame memory 50 in the pixel block storage unit 60.
[0146]
For example, when the pixel block storage unit 60 has the above-described two-bank configuration, the image data is read from the pixel block storage unit 60 to the image data generation unit 81 or 82 and the image data is read from the frame memory 50. It is output and written into the pixel block storage unit 60. Of the image data read from the frame memory 50, image data included in a predetermined boundary area is fetched and stored in the boundary data storage unit 70 as needed.
This eliminates the need to provide a special circuit for reading image data from the frame memory 50 in the boundary data storage unit 70. Further, since the process of reading image data from the frame memory 50 is shared with the pixel block storage unit 60, the number of accesses to the frame memory 50 is reduced.
[0147]
For example, in a case where the pixel block storage unit 60 does not have the above-described two-bank configuration, when image data is read from the frame memory 50 to the pixel block storage unit 60, a boundary adjacent to a pixel block of the image data to be read is used. The image data of the area is also read out together with this and stored in the boundary data storage unit 70.
In this case, the addresses of the image data read from the frame memory 50 are adjacent to each other in the data written in the pixel block storage unit 60 and the data written in the boundary data storage unit 70. The read access to the frame memory 50 is shared by the pixel block storage unit 60 and the boundary data storage unit 70. Therefore, a circuit for reading image data from the frame memory 50 can be easily shared between the pixel block storage unit 60 and the boundary data storage unit 70.
[0148]
The selector SL4 outputs the image data of the image data generator selected from the image data generator 81 or the image data generator 82 in accordance with the set conversion ratio of the number of image data.
[0149]
In the image data decoding unit 90, a predetermined number of image data blocks including the image data blocks output from the selector SL4 corresponding to the image data of the pixel blocks stored in the pixel block storage unit 60 are used as units. An encoding process of the image data is performed.
For example, a unit of image data for one MCU, which is a combination of a chroma signal of an 8 × 8 pixel block output from the selector SL4 and a luminance signal of a predetermined number of 8 × 8 pixel blocks separately read from the frame memory 50, is used. , JPEG encoding process is performed.
[0150]
As described above, according to the image data processing device shown in FIG. 7, two types of conversion ratios are used by using two image data generation units that generate image data whose number of image data is converted at a predetermined conversion ratio. Can convert the number of image data.
By providing the image data generation unit that generates the image data whose number of image data has been converted at different conversion ratios in parallel, it is also possible to perform the image data number conversion with two or more types of conversion ratios.
[0151]
In addition, since the pixel block storage unit 60 and the boundary data storage unit 70 can be shared by a plurality of image data generation units, the circuit configuration is different from the method in which these configurations are individually provided for each image data generation unit. The scale can be reduced.
[0152]
<Fourth embodiment>
Next, a fourth embodiment of the present invention will be described.
The image data processing device according to the fourth embodiment has, for example, a configuration equivalent to the image data processing device shown in FIG. However, the boundary data storage unit 70, the image data generation unit 81, and the image data generation unit 82 illustrated in FIG. 7 are replaced with a boundary data storage unit 70A, an image data generation unit 81A, and an image data generation unit 82A described below. .
[0153]
FIG. 8 is a block diagram illustrating an example of a configuration of the image data generation unit 81A and the boundary data storage unit 70A included in the image data processing device according to the fourth embodiment of the present invention.
[0154]
As illustrated in FIG. 8, the image data generation unit 81A includes flip-flops FF6 to FF9, selectors SL6 to SL8, addition units AD3 to AD5, and a multiplication unit ML5. The boundary data storage unit 70A has registers RG3 and RG4 and a selector SL5.
The flip-flops FF6 to FF9 are an embodiment of a data holding unit according to the third aspect of the present invention.
The unit including the selectors SL6 to SL8, the addition units AD3 to AD5, and the multiplication unit ML5 is an embodiment of a data synthesizing unit according to the third aspect of the present invention.
The unit including the selectors SL6 to SL8 is an embodiment of the data output unit according to the third aspect of the present invention.
The unit including the adders AD3 to AD5 is an embodiment of the adder according to the third aspect of the present invention.
[0155]
(Image data generation unit 81A)
The flip-flop FF6 holds the image data S9 read from the pixel block storage unit 60 by the control unit (not shown) in synchronization with the clock signal CLK.
The read cycle of the image data by the control unit is set to be four times the cycle of the clock signal CLK for the image data generation unit 81A. That is, each time one image data S9 is read from the pixel block storage unit 60, the flip-flop FF6 samples the read image data four times.
[0156]
The flip-flop FF7 holds the image data held in the flip-flop FF6 in synchronization with the clock signal CLK.
The flip-flop FF8 holds the image data held in the flip-flop FF7 in synchronization with the clock signal CLK.
The flip-flop FF9 holds the image data held in the flip-flop FF8 in synchronization with the clock signal CLK.
[0157]
The selector SL6 selects the image data S11 output from the boundary data storage unit 70A when the selection signal csel3 output from the control unit (not shown) is at a high level, and selects the flip-flop when the selection signal csel3 is at a low level. The image data held in the flip-flop FF6 is selected, and the selected image data is output as image data S12.
On the horizontal line of the original image, the pixels of the image data held in the flip-flops FF7 to FF9 may not be adjacent to the pixels of the image data held in the flip-flop FF6. In this case, the boundary data storage unit 70A outputs the image data S11 of the pixels included in the predetermined boundary area adjacent to the pixels of the image data held in the flip-flops FF7 to FF9. The pixels form a predetermined small pixel block of 1 × 2 pixels. In this case, the selection signal csel3 is set to a high level. In other cases, the selection signal csel3 is set to a low level.
[0158]
The selector SL7 selects the image data S11 output from the boundary data storage unit 70A when the selection signal csel2 output from the control unit (not shown) is at a high level, and selects the flip-flop when the selection signal csel2 is at a low level. The image data held in the flip-flop FF7 is selected, and the selected image data is output as image data S13.
On the horizontal line of the original image, the pixel of the image data held in the flip-flops FF8 and FF9 may not be adjacent to the pixel of the image data held in the flip-flops FF6 and FF7. In this case, the boundary data storage unit 70A outputs image data S11 of pixels included in a predetermined boundary area adjacent to the pixels of the image data held in the flip-flops FF8 and FF9. The pixels form a predetermined small pixel block of 1 × 2 pixels. In this case, the selection signal csel2 is set to a high level. In other cases, the selection signal csel2 is set to a low level.
[0159]
The selector SL8 selects the image data S11 output from the boundary data storage unit 70A when the selection signal csel1 output from the control unit (not shown) is at the high level, and selects the flip-flop when the selection signal csel1 is at the low level. The image data held in the flip-flop FF8 is selected, and the selected image data is output as image data S14.
On the horizontal line of the original image, the pixel of the image data held in the flip-flop FF9 may not be adjacent to the pixel of the image data held in the flip-flops FF6 to FF8. In this case, the boundary data storage unit 70A outputs the image data S11 of the pixels included in the predetermined boundary area adjacent to the pixels of the image data held in the flip-flop FF9. A predetermined small pixel block of 1 × 2 pixels is formed. In this case, the selection signal csel1 is set to a high level. In other cases, the selection signal csel1 is set to a low level.
[0160]
The adder AD3 adds the image data S15 held in the flip-flop FF9 and the image data S14 output from the selector SL8.
The addition unit AD4 adds the addition result of the addition unit AD3 and the image data S13 output from the selector SL7.
The addition unit AD5 adds the addition result of the addition unit AD4 and the image data S12 output from the selector SL6.
[0161]
The multiplication unit ML5 multiplies the addition result of the addition unit AD5 by a weighting factor of 1/4, and outputs the multiplication result as image data S17.
[0162]
(Boundary data storage unit 70A)
The register RG3 holds the image data S10 of the chroma signal Cb read from the frame memory 50 when the enable signal xcb-en output from the control unit (not shown) is at a low level. Further, the stored image data is sequentially shifted at a predetermined timing and output to the selector SL5.
The register RG4 holds the image data S10 of the chroma signal Cr read from the frame memory 50 when the enable signal xcr-en output from the control unit (not shown) is at a low level. Further, the stored image data is sequentially shifted at a predetermined timing and output to the selector SL5.
[0163]
When the image data of the chroma signal (Cb, Cr) is read from the frame memory 50 to the pixel block storage unit 60, the image data is next read from the pixel block storage unit 60 to the image data generation units (81A, 82A). An image included in a predetermined boundary region of another pixel block adjacent to the pixel block or another pixel block adjacent to the pixel block being read from the pixel block storage unit 60 to the image data generation unit (81A, 82A). Data is read from the frame memory 50 by a control unit (not shown). The enable signals (xcb-en, xcr-en) are set to the low level at the timing when the image data in the boundary area is read from the frame memory 50. Thus, the image data of the boundary area among the image data read from the frame memory 50 is held in the registers (REG3, REG4) of the boundary data storage unit 70A.
[0164]
The selector SL5 selects image data output from either the register RG3 or the register RG4 according to the input selection signal area-cr, and outputs the selected image data as the image data S11. When the image data S9 is the chroma signal Cb, the register RG3 is selected, and when the image data S9 is the chroma signal Cr, the register RG4 is selected.
[0165]
FIG. 9 is a block diagram illustrating an example of a configuration of the image data generation unit 82A and the boundary data storage unit 70A included in the image data processing device according to the fourth embodiment of the present invention.
As shown in FIG. 9, the image data generation unit 82A includes flip-flops FF10 and FF11, a selector SL9, an addition unit AD6, and a multiplication unit ML6.
The flip-flops FF10 and F11 are an embodiment of a data holding unit according to the third aspect of the present invention.
The unit including the selector SL9, the adder AD6, and the multiplier ML6 is an embodiment of a data synthesizing unit according to the third aspect of the present invention.
[0166]
(Image data generation unit 82A)
The flip-flop FF10 holds the image data S9 read from the pixel block storage unit 60 by the control unit (not shown) in synchronization with the clock signal CLK.
Note that the read cycle of the image data by the control unit is set to be twice as long as the clock signal CLK for the image data generation unit 82A. Therefore, in the flip-flop FF10, each time one image data S9 is read from the pixel block storage unit 60, the read image data is sampled twice.
[0167]
The flip-flop FF11 holds the image data S18 held in the flip-flop FF10 in synchronization with the clock signal CLK.
[0168]
The selector SL9 selects the image data S11 output from the boundary data storage unit 70A or the image data S18 held in the flip-flop FF10 according to a selection signal blk-right output from a control unit (not shown). The output data is output as image data S20.
[0169]
The adder AD6 adds the image data S19 held in the flip-flop FF11 and the image data S20 output from the selector SL9.
[0170]
The multiplication unit ML6 multiplies the addition result of the addition unit AD6 by a weight coefficient １ ／, and outputs the multiplication result as image data S21.
[0171]
Here, regarding the operation of the image data processing apparatus having the above-described configuration, the case where the conversion process from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 4: 4) is performed, and the case where the YCrCb ratio (4: 1: 1) and a case where conversion processing to a YCrCb ratio (4: 2: 2) is performed.
[0172]
// Conversion process from YCrCb ratio (4: 1: 1) to (4: 4: 4) //
FIG. 10 illustrates the outline of the chroma signal interpolation processing for converting the image data of the YCrCb ratio (4: 1: 1) read from the frame memory 50 into the image data S17 of the YCrCb ratio (4: 4: 4). FIG.
FIG. 11 is an example of the relationship between the value of each signal and the timing shown in the block diagram of FIG. 8 when the conversion process from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 4: 4) is performed. FIG.
[0173]
Pixel blocks B11 and B12 shown in FIG. 10A indicate image data of chroma signals of 8 × 2 pixel blocks read from the frame memory 50 to the pixel block storage unit 60. The pixel blocks of the chroma signal are read from the frame memory 50 in the order of the pixel blocks B11, B12,... And stored in the pixel block storage unit 60. From this pixel block storage unit 60, the image data C11 is output every four cycles of the clock signal CLK (FIG. 11A). ₀ , C11 ₁ , ..., C11 _Fifteen , C12 ₀ , C12 ₁ ,... Are read out in the order of the chroma signal (FIG. 11B) and input to the flip-flop FF6.
[0174]
As shown in FIG. 11B, the read cycle of the image data from the pixel block storage unit 60 is set to be four times the cycle of the clock signal CLK (FIG. 11A). In the flip-flop FF6, each time the image data S9 of one pixel is read from the pixel block storage unit 60, the read image data is sampled four times. The image data sampled by the flip-flop FF6 is shifted in the order of the flip-flops FF7, FF8 and FF9 in synchronization with the clock signal CLK. Thus, the flip-flops FF6 to FF9 hold four sampled series of image data.
[0175]
The image data S9 of one pixel is sampled four times, and up to four of the sampled image data are stored in the flip-flops FF6 to FF9. Image data is one pixel or two pixels of the original image.
[0176]
First, a case where image data of two pixels adjacent on the horizontal line of the original image are sequentially input to the image data generation unit 81A will be described.
When the image data held in the flip-flops FF6 to FF9 is one-pixel image data and two-pixel image data adjacent on the horizontal line of the original image, the selectors SL6 to SL8 output the flip-flop FF6. To FF8 are selected and output.
In the adders AD3 to AD5, the four image data held in the flip-flops FF6 to FF9 are summed, and the sum S16 is multiplied by a coefficient 1/4 in the multiplying unit ML5 to generate image data S17. The image data S17 is obtained as a result of summing the four image data held in the flip-flops FF6 to FF9 and dividing the total by four.
[0177]
When the image data of one of the two adjacent pixels is input to the image data generation unit 81A and the image data of the one pixel is held in all of the flip-flops FF6 to FF9, the image data S17 becomes Is obtained.
Next, the data held in the flip-flops FF6 to FF9 are shifted in synchronization with the clock signal CLK, and the image data of the other two adjacent pixels are sequentially transferred to the flip-flops FF6 to FF8 following the image data of the one pixel. When held, the flip-flops FF6 to FF8 hold image data of two pixels at a ratio of 1: 3, 2: 2, and 3: 1, respectively. Therefore, as the image data S17, image data of three pixels synthesized by three types of weighting methods corresponding to these ratios is obtained.
Therefore, when image data of two adjacent pixels is sequentially input to the image data generation unit 81A, the addition units AD3 to AD5 output two pixels at a ratio of 1: 0, 1: 3, 2: 2, and 3: 1. Are added. As the image data S17, image data of four pixels synthesized by four types of weighting methods corresponding to these ratios is obtained. Since the image data of four pixels is obtained with respect to the image data of two adjacent pixels, the number of image data is expanded four times, and the image data of the 8 × 2 pixel block held in the pixel block storage unit 60 is 8 It is converted to image data of × 8 pixels.
[0178]
Pixel blocks B13 and B14 shown in FIG. 10B are pixel blocks obtained by converting the 8 × 2 pixel pixel blocks B11 and B12 in FIG. 10A into 8 × 8 pixels in the image data generation unit 81A. Each image data of the pixel blocks B13 and B14 corresponds to the image data S17 generated by the image data generation unit 81A.
For example, as shown in FIG. ₂ And C11 ₃ Are adjacent on the original image, and a 1 × 2 pixel small pixel block MB11 ₂ Is composed. This small pixel block MB11 ₂ Image data (C11 ₂ , C11 ₃ ) Are sequentially input to the image data generation unit 81A and are held in the flip-flops FF6 to FF9 (time t31 to t34 in the timing chart shown in FIG. 11), and the selection signal csel3 (FIG. 11D) and the selection signal The signal csel2 (FIG. 11F) and the selection signal csel1 (FIG. 11H) go low, and the data held in the flip-flops FF6 to FF8 is output from the selectors SL6 to SL8. During this period, the small pixel block MB11 ₂ Image data (C11 ₂ , C11 ₃ ) Are synthesized by four types of weighting methods, whereby the image data C13 of four pixels is obtained. ₈ ~ C13 ₁₁ (FIG. 10B) are sequentially generated as image data S17.
[0179]
On the other hand, when the image data held in the flip-flops FF6 to FF9 is image data of two pixels that are not adjacent on the horizontal line of the original image, one of the two pixels and the data output from the boundary data storage unit 70A are output. The pixel of the image data S11 is adjacent on the horizontal line of the original image, and these two pixels constitute a small pixel block of 1 × 2 pixels.
In this case, the image data of the other pixel of the two pixels is replaced by the image data S11 output from the boundary data storage unit 70A by the selectors SL6 to SL8, and output to the addition units AD3 to AD5. The image data S17 is obtained as a synthesis result of the image data of one of the two pixels held in the flip-flops FF6 to FF9 and the image data of a predetermined boundary area adjacent to the pixel.
[0180]
The image data of one of the two non-adjacent pixels on the horizontal line is input to the image data generation unit 81A, and the image data of the one pixel is held in all of the flip-flops FF6 to FF9. Obtains image data equal to the image data of the one pixel.
Next, the data held in the flip-flops FF6 to FF9 are shifted in synchronization with the clock signal CLK, and the image data of the other two non-adjacent pixels are sequentially transferred to the flip-flops FF6 to FF8 following the image data of the one pixel. When held, the selectors SL6 to SL7 select the image data S11 output from the boundary data storage unit 70A instead of the image data of the other pixel. In accordance with the data shift of the flip-flops FF6 to FF9, the adders AD3 to AD5 add image data of two pixels at a ratio of 1: 3, 2: 2, and 3: 1, so that the image data S17 is obtained. Obtains image data of three pixels synthesized by three types of weighting methods corresponding to these ratios.
Therefore, when image data of two pixels that are not adjacent to each other are sequentially input to the image data generation unit 81A, the image data of one pixel of the two pixels and the boundary area adjacent to the one pixel are used as the image data S17. Image data of 4 pixels is obtained by weighting the image data at a ratio of 1: 0, 1: 3, 2: 2, and 3: 1, and combining them.
[0181]
For example, as shown in FIG. ₃ And C12 ₂ Are adjacent on the original image, and a 1 × 2 pixel small pixel block MB11 ₃ Is composed. On the other hand, the image data generation unit 81A includes the image data C11 ₃ And the image data C11 ₄ Are not adjacent to each other on the horizontal line of the original image.
This image data C11 ₃ And C11 ₄ Are sequentially input to the image data generation unit 81A, first, the image data C11 is stored in all of the flip-flops FF6 to FF9. ₃ Are held (time t35 in the timing chart shown in FIG. 11), the selection signals csel1 to csel3 all go low, and the data held in the flip-flops FF6 to FF8 are output from the selectors SL6 to SL8. As the image data S17, the image data C11 ₃ Is output.
Next, the data held in the flip-flops FF6 to FF9 is shifted in synchronization with the clock signal CLK, and the image data C11 ₄ Is the image data C11 ₃ Subsequently, when the signals are sequentially held in the flip-flops FF6 to FF8 (time t36 to t38 in the timing chart shown in FIG. 11), the signal levels become high in the order of the selection signals csel3, csel2, and csel1. Thus, the selectors SL6 to SL7 output the image data C11 ₄ Instead of the image data C12 output as image data S11 from the boundary data storage unit 70A. ₂ Is selected and output to the adders AD3 to AD5. The adders AD3 to AD5 output the image data C12 at a ratio of 1: 3, 2: 2, and 3: 1, according to the data shift of the flip-flops FF6 to FF9. ₂ And C11 ₃₂ Is added, image data of three pixels synthesized by three types of weighting methods corresponding to these ratios is obtained as the image data S17.
As a result, from time t35 to time t38, the small pixel block MB11 over two different pixel blocks B11 and B12 ₃ Is synthesized with the small pixel block MB11. ₃ Image data of four pixels corresponding to (C13 ₁₂ ~ C13 _Fifteen ) Is generated.
[0182]
Further, when the image data of the chroma signal is read from the frame memory 50 to the pixel block storage unit 60, the image data of the chroma signal is read out from the pixel block storage unit 60 to the image data generation unit 81A. It is included in a predetermined boundary area of another adjacent 8 × 2 pixel block or another 8 × 2 pixel block adjacent to the 8 × 2 pixel block being read from the pixel block storage unit 60 to the image data generation unit 81A. The image data to be read is read from the frame memory 50 by a control unit (not shown).
[0183]
In FIG. 10, regions L11 and L12 of 8 × 1 pixels located at the left ends of the 8 × 2 pixel blocks B11 and B12 correspond to the boundary region.
For example, when the pixel block storage unit 60 has a two-bank configuration, the image data of the pixel block B11 is read to the image data generation unit 81A, and at the same time, the image data of the pixel block B12 to be processed next to the pixel block B11 is The data is read from the frame memory 50 to the pixel block storage unit 60. In this case, when the image data of the pixel block B12 is read from the frame memory 50 to the pixel block storage unit 60, the pixel block B12 adjacent to the pixel block B11 being read from the pixel block storage unit 60 to the image data generation unit 81A is read. The image data of the boundary area L12 is read from the frame memory 50.
When the pixel block storage unit 60 does not have the two-bank configuration, when the image data of the pixel block B11 is read out to the image data generation unit 81A, the pixel block storage unit 60 transmits the image data to the image data generation unit 81A. Is not performed. In this case, when the image data of the pixel block B11 is read from the frame memory 50 to the pixel block storage unit 60, the pixel block B12 adjacent to the pixel block B11 next read from the pixel block storage unit 60 to the image data generation unit 81A Is read from the frame memory 50 together with the image data of the pixel block B11.
[0184]
The enable signal (xcb-en, xcr-en) is set to a low level at the timing when the image data in the boundary area is read from the frame memory 50 in the process of reading from the frame memory 50 described above. Thus, the image data of the boundary area is held in the registers (REG3, REG4) of the boundary data storage unit 70A. The held image data of the boundary area is used in the process of generating the image data of the 8 × 8 pixel block from the image data of the 8 × 2 pixel block in the image data generation unit 81A.
[0185]
// Conversion process from YCrCb ratio (4: 1: 1) to (4: 2: 2) //
FIG. 12 illustrates an outline of a chroma signal interpolation process for converting image data having a YCrCb ratio (4: 1: 1) read from the frame memory 50 into image data S21 having a YCrCb ratio (4: 2: 2). FIG.
FIG. 13 shows an example of the relationship between the value of each signal and the timing shown in the block diagram of FIG. 9 when the conversion process from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 2: 2) is performed. FIG.
[0186]
Pixel blocks B16 and B17 shown in FIG. 12A show image data of a chroma signal of an 8 × 4 pixel block read from the frame memory 50 to the pixel block storage unit 60. The pixel blocks of the chroma signal are read from the frame memory 50 in the order of the pixel blocks B16, B17,... And stored in the pixel block storage unit 60. From the pixel block storage unit 60, the image data C16 is output every two cycles of the clock signal CLK (FIG. 13A). ₀ , C16 ₁ , ..., C16 ₃₁ , C17 ₀ , C17 ₁ ,... Are read out and stored in the flip-flop FF10 as image data S18 (FIG. 13B).
[0187]
As shown in FIG. 13B, the cycle of reading image data from the pixel block storage unit 60 is set to be twice the cycle of the clock signal CLK (FIG. 11A). In the flip-flop FF10, every time the image data S9 of one pixel is read from the pixel block storage unit 60, the read image data is sampled twice. The image data sampled by the flip-flop FF10 is shifted to the flip-flop FF11 in synchronization with the clock signal CLK. Thus, the flip-flops FF10 and F11 hold two sampled series of image data.
[0188]
The image data S9 of one pixel is sampled twice, and up to two of a series of sampled image data are stored in the flip-flops FF10 and FF11. Image data is one pixel or two pixels of the original image.
[0189]
First, a case where image data of two pixels adjacent on the horizontal line of the original image are sequentially input to the image data generation unit 82A will be described.
When the image data held in the flip-flops FF10 and FF11 is one-pixel image data and when the image data is two-pixel image data adjacent on the horizontal line of the original image, the selector SL9 holds the image data in the flip-flop FF10. The selected image data S18 is selected and output.
In the adder AD6, the two image data held in the flip-flops FF10 and F11 are summed, and the sum S16 is multiplied by a coefficient に お い て in the multiplier ML6 and output as image data S21. This image data S21 is obtained as a result of summing the two image data held in the flip-flops FF10 and F11 and dividing this by the value 2.
[0190]
When the image data of one of the two adjacent pixels is input to the image data generation unit 82A, and the image data of the one pixel is held in both of the flip-flops FF10 and FF11, the image data S21 becomes Is obtained.
Next, the data held in the flip-flop FF10 is shifted to the flip-flop FF11 in synchronization with the clock signal CLK, and the image data of the other two adjacent pixels is held in the flip-flop FF10 following the image data of the one pixel. Then, the flip-flops FF10 and FF11 hold image data of two pixels in a one-to-one ratio, and obtain, as the image data S21, image data of one pixel weighted and synthesized in this ratio.
Therefore, when the image data of two adjacent pixels is sequentially input to the image data generation unit 82A, the addition unit AD6 adds the image data of two pixels at a ratio of 1 to 0 and 1 to 1. As the image data S21, image data of two pixels synthesized by two types of weighting methods corresponding to these ratios is obtained. Since image data of two pixels is obtained with respect to image data of two adjacent pixels, the number of image data is doubled, and the image data of the 8 × 4 pixel block held in the pixel block storage unit 60 is 8 It is converted into image data of a × 8 pixel block.
[0191]
Pixel blocks B18 and B19 shown in FIG. 12B are pixel blocks obtained by converting the 8 × 4 pixel pixel blocks B16 and B17 in FIG. 12A into 8 × 8 pixels in the image data generation unit 82A. Each image data of the pixel blocks B18 and B19 corresponds to the image data S21 generated by the image data generation unit 82A.
For example, as shown in FIG. ₄ And C16 ₅ Are adjacent on the original image, and a 1 × 2 pixel small pixel block MB16 ₄ Is composed. This small pixel block MB16 ₄ Of two image data (C16 ₄ , C16 ₅ ) Are sequentially input to the image data generation unit 82A and held in the flip-flops FF10 and F11 (time t41 and t42 in the timing chart shown in FIG. 13), the selection signal blk-right (FIG. 13E) It goes low. As a result, the data held in the flip-flop FF10 is output from the selector SL9 as image data S19 (FIG. 13C). During this period, the small pixel block MB16 ₄ Of two image data (C16 ₄ , C16 ₅ ) Are combined with one-to-one and one-to-one weighting, whereby two-pixel image data C18 ₈ And C18 ₉ (FIG. 12B) are sequentially generated as image data S21.
[0192]
On the other hand, when the image data held in the flip-flops FF10 and FF11 is image data of two pixels that are not adjacent on the horizontal line of the original image, one of the two pixels and the data output from the boundary data storage unit 70A are output. The pixel of the image data S11 is adjacent on the horizontal line of the original image, and these two pixels constitute a small pixel block of 1 × 2 pixels.
In this case, the held data S18 of the flip-flop FF10, which is the image data of the other pixel of the two pixels, is replaced by the image data S11 output from the boundary data storage unit 70A by the selector SL9, and output to the addition unit AD6. You. The image data S21 is obtained as a result of synthesizing the held data S19 of the flip-flop FF11, which is the image data of one of the two pixels, with the image data S11 of a predetermined boundary area adjacent to the pixel.
[0193]
The image data of one of the two non-adjacent pixels on the horizontal line is input to the image data generation unit 82A, and the image data of the one pixel is held in both the flip-flops FF10 and F11. Obtains image data equal to the image data of the one pixel.
Next, the data held in the flip-flop FF10 is shifted to the flip-flop FF11 in synchronization with the clock signal CLK, and the image data of the other non-adjacent two pixels is held in the flip-flop FF10 following the image data of the one pixel. Then, the selector SL9 selects the image data S11 output from the boundary data storage unit 70A instead of the image data of the other pixel (data S18 held in the flip-flop FF10). In the adder AD6, image data of two pixels is added at a ratio of 1: 1. Therefore, as the image data S21, image data of one pixel which is weighted and synthesized at this ratio is obtained.
Therefore, when image data of two pixels that are not adjacent to each other are sequentially input to the image data generation unit 82A, the image data S21 includes the image data of one of the two pixels and the boundary area adjacent to the one pixel. Image data of two pixels synthesized by weighting the image data with the ratio of 1 to 0 and 1 to 1 is obtained.
[0194]
For example, as shown in FIG. ₇ And C17 ₄ Are adjacent on the original image, and a 1 × 2 pixel small pixel block MB16 ₇ Is composed. On the other hand, the image data generation unit 82A includes the image data C16 ₇ And the image data C16 ₈ Are not adjacent to each other on the horizontal line of the original image.
This image data C16 ₇ And C16 ₈ Are sequentially input to the image data generation unit 82A, first, the image data C16 is supplied to both the flip-flops FF10 and FF11. ₇ Is held (time t47 in the timing chart shown in FIG. 13), the selection signal blk-right goes low, and the data held in the flip-flop FF10 is output from the selector SL9. As the image data S21, the image data C16 ₇ Is output.
Next, the data held in the flip-flop FF10 is shifted by the flip-flop FF11 in synchronization with the clock signal CLK, and the image data C16 ₈ Is the image data C16 ₇ Subsequently, when the data is held in the flip-flop FF10 (time t48 in the timing chart shown in FIG. 13), the signal level of the selection signal blk-right (FIG. 13E) becomes high. Thereby, the selector SL9 outputs the image data C16 ₈ In place of the image data C17 output from the boundary data storage unit 70A as the image data S11. ₄ Is selected and output to the adder AD6 as the image data S20 (FIG. 13D). In the adder AD6, the image data C16 held in the flip-flop F11 ₇ And the image data C17 output from the boundary data storage unit 70A ₄ Are added at a ratio of 1: 1. As the image data S21, image data of one pixel which is weighted at this ratio and synthesized is obtained.
As a result, at times t47 and t48, the small pixel block MB16 over two different pixel blocks B16 and B17 ₇ Are synthesized by two types of weighting methods, and image data (C18) of two pixels corresponding to this small pixel block is synthesized. ₁₄ , C18 _Fifteen ) Is generated.
[0195]
Further, when the image data of the chroma signal is read from the frame memory 50 to the pixel block storage unit 60, the 8 × 4 pixel block to be read next from the pixel block storage unit 60 to the image data generation unit 82 </ b> A is also generated. It is included in a predetermined boundary area of another adjacent 8 × 4 pixel block or another 8 × 4 pixel block adjacent to the 8 × 4 pixel block being read from the pixel block storage unit 60 to the image data generation unit 82A. The image data to be read is read from the frame memory 50 by a control unit (not shown).
[0196]
In FIG. 12, regions L16 and L17 of 8 × 1 pixels located at the left ends of the 8 × 4 pixel blocks B16 and B17 correspond to the boundary region.
For example, when the pixel block storage unit 60 has a two-bank configuration, the image data of the pixel block B16 is read out to the image data generation unit 82A, and at the same time, the image data of the pixel block B17 to be processed next to the pixel block B16 is The data is read from the frame memory 50 to the pixel block storage unit 60. In this case, when the image data of the pixel block B17 is read from the frame memory 50 to the pixel block storage unit 60, the pixel block B17 adjacent to the pixel block B16 being read from the pixel block storage unit 60 to the image data generation unit 82A is read. The image data of the boundary area L17 is read from the frame memory 50.
When the pixel block storage unit 60 does not have the two-bank configuration, when the image data of the pixel block B16 is read out to the image data generation unit 82A, the pixel block storage unit 60 transmits the image data to the image data generation unit 82A. Is not performed. In this case, when the image data of the pixel block B16 is read from the frame memory 50 to the pixel block storage unit 60, the pixel block B17 adjacent to the pixel block B16 to be read next from the pixel block storage unit 60 to the image data generation unit 82A Is read from the frame memory 50 together with the image data of the pixel block B16.
[0197]
The enable signal (xcb-en, xcr-en) is set to a low level at the timing when the image data in the boundary area is read from the frame memory 50 in the process of reading from the frame memory 50 described above. Thus, the image data of the boundary area is held in the registers (REG3, REG4) of the boundary data storage unit 70A. The held image data of the boundary area is used in the process of generating image data of an 8 × 8 pixel block from image data of an 8 × 4 pixel block in the image data generation unit 82A.
[0198]
As described above, according to the image data processing device including the image data generating units 81A and 82A shown in FIGS. 8 and 9, similarly to the image data processing device shown in FIG. Data number conversion can be performed.
The image data generation units 81A and 82A have the same configuration, and the image data generation units that generate the image data whose number of image data is converted at different conversion ratios are further provided in parallel. It is also possible to perform data number conversion.
In addition, since the pixel block storage unit 60 and the boundary data storage unit 70 can be shared by a plurality of image data generation units, the circuit configuration is different from the method in which these configurations are individually provided for each image data generation unit. The scale can be reduced.
[0199]
As the selection signals (csel1 to csel3, blk-right, area-cr) and the enable signals (xcb-en, xcr-en), image data for one MCU processed in the image data encoding unit 90 is stored in the pixel block storage unit 60. Is generated at the same timing by a control unit (not shown) each time the data is read from the.
Therefore, for example, every time one MCU of image data processed in the image data encoding unit 90 is read from the pixel block storage unit 60, the image data is initialized when the first image data is read, and the image data is read. A counter that counts each time the counter is operated, and a decoding circuit that generates selection signals (csel1 to csel3, blk-right, area-cr) and enable signals (xcb-en, xcr-en) according to the count value of the counter. Are provided in this control unit, these signals can be easily generated.
[0200]
<Fifth embodiment>
Next, a fifth embodiment of the present invention will be described. In the image data processing apparatuses according to the third and fourth embodiments, the conversion ratio is switched by selecting an image data generation unit corresponding to the conversion ratio from the plurality of image data generation units. In the image data processing device according to the fifth embodiment, the conversion ratio is switched by enabling or disabling the image data number conversion processing in each stage of the cascade-connected image data generation unit.
[0201]
FIG. 14 is a block diagram illustrating an example of a configuration of an image data processing device according to the fifth embodiment of the present invention.
The image data processing device illustrated in FIG. 14 includes a frame memory 50, a pixel block storage unit 60B, a boundary data storage unit 70B, a first image data generation unit 81B, a second image data generation unit 82B, And a data encoding unit 90. However, the same reference numerals in FIGS. 14 and 7 indicate the same components.
The pixel block storage unit 60B is an embodiment of a pixel block storage unit according to the fourth aspect of the present invention.
The first image data generation unit 81B is an embodiment of a first image data generation unit according to the fourth aspect of the present invention.
The second image data generator 82B is an embodiment of a second image data generator according to the fourth aspect of the present invention.
The boundary data storage unit 70B is an embodiment of a boundary data storage unit according to the fourth aspect of the present invention.
[0202]
(Pixel block storage unit 60B)
The pixel block storage unit 60B stores image data corresponding to each pixel of the original image stored in the frame memory 50 in a predetermined pixel block (this predetermined pixel block is referred to as a first pixel block in the description of the present embodiment). ), For example, input and stored for each 8 × 2 pixel block of chroma signal.
[0203]
When the conversion ratio of the number of image data is set, the pixel block storage unit 60B may read and store the image data from the frame memory 50 for each predetermined pixel block corresponding to the set conversion ratio. .
For example, if the conversion ratio is set so as to perform conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 4: 4), the pixel block is assumed to be a first conversion ratio. The storage unit 60B reads and stores the image data of the chroma signal from the frame memory 50 for each 8 × 2 pixel block as a first pixel block.
Further, for example, when the conversion ratio is set so as to perform conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 2: 2), assuming that this conversion ratio is a second conversion ratio, The pixel block storage unit 60B reads out and stores the image data of the chroma signal from the frame memory 50 for each 8 × 4 pixel block as a third pixel block.
[0204]
(First Image Data Generation Unit 81B)
The first image data generation unit 81B reads the image data stored in the pixel block storage unit 60B for each first pixel block, and converts the read image data into a predetermined small pixel block smaller than the first pixel block. (In the description of the present embodiment, this small pixel block is referred to as a first small pixel block), and one or a plurality of image data corresponding to each first small pixel block is generated.
[0205]
For example, the image data is read from the pixel block storage unit 60B for each 8 × 2 pixel block as the first pixel block. Then, a pixel block of 1 × 2 pixels arranged adjacently on the horizontal line of the original image so that pixels at both ends overlap each other is set as a first small pixel block, and 8 × The image data of the two-pixel block is synthesized by two types of weighting methods, and image data of two pixels corresponding to each first small-pixel block is generated. As a result, the 8 × 2 pixel block is converted into an 8 × 4 pixel block, and the number of image data is doubled.
[0206]
However, the pixels of the first small pixel block are not necessarily all included in the first pixel block being read from the pixel block storage unit 60B. That is, some pixels of the first small pixel block may be included in a predetermined boundary area of another first pixel block adjacent to the first pixel block being read from the pixel block storage unit 60B. .
In this case, the first data generation unit 81B reads the image data of the partial pixels stored in the boundary data storage unit 70B, and reads the read image data and the first data read from the pixel block storage unit 60B. Is combined with the image data of the remaining pixels of the small pixel block to generate one or a plurality of image data corresponding to the first small pixel block.
[0207]
For example, when the above-described block of small pixels of 1 × 2 pixels is arranged to be arranged on the left side of the block of 8 × 2 pixels, among the two small pixel blocks arranged on the horizontal line of the block of 8 × 2 pixels, In the small pixel block located on the right side, the rightmost pixel is not included in the pixel block being read from the pixel block storage unit 60B, and the leftmost 8x1 pixel region in the 8x2 pixel block on the right is include. In this case, the above-described boundary region corresponds to the leftmost 8 × 1 pixel region.
When synthesizing a 1 × 2 pixel small pixel block located on the right side of the 8 × 2 pixel block, the first image data generation unit 81B stores the image data of the rightmost pixel of the small pixel block in the boundary data storage unit. Read from 70B. Then, the image data read from the boundary data storage unit 70B and the image data of the remaining one pixel of the small pixel block read from the pixel block storage unit 60 are combined by two types of weighting methods, and two pixels are combined. Is generated.
[0208]
In addition, the first image data generation unit 81B may enable or disable the conversion processing of the number of image data according to the set conversion ratio of the number of image data. That is, when the conversion ratio of the number of image data is set to the first conversion ratio, generation of the above-described image data corresponding to each first small pixel block is performed. Thus, the first image data generation unit 81B executes conversion of the number of image data. When the conversion ratio of the number of image data is set to the second conversion ratio, the input image data for each third pixel block is output as it is. Thereby, the conversion processing of the number of image data in the first image data generation unit 81B is invalidated.
[0209]
For example, when the conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 4: 4) is set as the first conversion ratio for the image data processing device, the first image data generation unit 81B executes the above-described processing of generating image data of two pixels corresponding to each small pixel block by synthesizing image data for each small pixel block of 1 × 2 pixels by two types of weighting methods. As a result, the 8 × 2 pixel block of the chroma signal is converted into an 8 × 4 pixel block, and the number of image data is doubled. When the conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 2: 2) is set as the second conversion ratio for the image data processing device, every 8 × 4 pixel block The stored image data of the pixel block storage unit 60B is output as it is. Thereby, the conversion processing of the number of image data in the first image data generation unit 81B is invalidated.
In this example, regardless of which conversion ratio is set, in the second image data generation unit 82B described later, processing for converting an 8 × 4 pixel block of a chroma signal into an 8 × 8 pixel block is executed. Is done. Therefore, as a whole, the image data processing device converts the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 4: 4) or converts the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 2). : 2) is performed.
[0210]
(Second image data generation unit 82B)
The image data generated by the first image data generation unit 81B is synthesized for each second small pixel block smaller than the second pixel block, and one or a plurality of image data corresponding to each second small pixel block are combined. Is generated. The second pixel block indicates a pixel block corresponding to the first pixel block or the third pixel block of the original image on the image formed by the image data generated by the first image data generation unit 81B. .
[0211]
For example, when a first pixel block of a chroma signal having a size of 8 × 2 pixels is converted into an 8 × 4 pixel block in the first image data generation unit 81B, the converted 8 × 4 pixel block is This corresponds to a second pixel block. In contrast to the 8 × 4 pixel block, the second small pixel block is set, for example, as a block of 1 × 2 pixels arranged adjacently on a horizontal line such that the pixels at both ends overlap each other. The second image data generation unit 82B combines the image data of the small pixel blocks of 1 × 2 pixels by two types of weighting methods, and generates image data of two pixels corresponding to each small pixel block. . As a result, the 8 × 4 pixel block is converted into an 8 × 8 pixel block, and the number of image data is further doubled.
[0212]
However, the pixels of the second small pixel block are not necessarily all included in the second pixel block being generated by the first image data generation unit 81B. That is, some pixels of the second small pixel block are included in a predetermined boundary area of another second pixel block adjacent to the second pixel block being generated by the first image data generation unit 81B. Sometimes.
In this case, the second image data generating unit 82B is a pixel equivalent to the partial pixel in the above-described boundary area, that is, a pixel having image data equal to the partial pixel, and Is read from the boundary data storage unit 70B. Then, the read image data is combined with the image data of the remaining pixels of the second small pixel block generated by the first image data generation unit 81B to correspond to the second small pixel block. One or more pieces of image data are generated.
[0213]
For example, when the above-described block of small pixels of 1 × 2 pixels is arranged to be arranged on the left side of the block of 8 × 4 pixels, among the four small pixel blocks arranged on the horizontal line of the block of 8 × 4 pixels, The rightmost pixel of the small pixel block located on the rightmost side is not included in the pixel block being generated in the first image data generation unit 81B, and the leftmost 8 × 4 pixel block of the 8 × 4 pixel block on the right is not included. It is included in the area of one pixel. In this case, the above-described boundary region corresponds to the leftmost 8 × 1 pixel region.
When synthesizing a 1 × 2 pixel small pixel block located on the rightmost side of the 8 × 4 pixel block, the second image data generation unit 82B generates image data of pixels equivalent to the rightmost pixel of the small pixel block. From the boundary data storage unit 70B. Then, the read image data and the image data of the remaining one pixel of the small pixel block generated in the first image data generation unit 81B are combined by two types of weighting methods, and an image of two pixels is formed. Generate data.
[0214]
(Boundary data storage unit 70B)
The boundary data storage unit 70B is either a different pixel block adjacent to a pixel block next read from the pixel block storage unit 60B by the first image data generation unit 81B, or a pixel block by the first image data generation unit 81B. The image data which is another pixel block adjacent to the pixel block being read from the storage unit 60B and which is included in a predetermined boundary area according to the set conversion ratio of the number of image data is read from the frame memory 50 and stored. I do.
Reading of image data from the frame memory 50 by the boundary data storage unit 70B is performed when reading of image data from the frame memory 50 is performed in the pixel block storage unit 60B.
[0215]
For example, when the pixel block storage unit 60B has a two-bank configuration, the pixel block storage unit 60B reads image data from the frame memory 50 while reading image data to the first image data generation unit 81B. It is possible to memorize. In this case, the image data read from the frame memory 50 by the pixel block storage unit 60B is image data that is next supplied from the pixel block storage unit 60B to the first image data generation unit 81B. The boundary data storage unit 70B can fetch and store the image data included in the predetermined boundary region from the image data read from the frame memory 50 as needed.
However, in this case, before the image data of the boundary data storage unit 70B is read by the first image data generation unit 81B, the pixel block storage unit is configured to store the read image data in the boundary data storage unit 70B in advance. It is necessary to appropriately set the timing of writing and reading in 60B.
[0216]
On the other hand, when the pixel block storage unit 60B does not have the two-bank configuration, the boundary data storage unit 70B stores the read image data when the pixel block storage unit 60B reads the image data from the frame memory 50. The image data of the boundary area adjacent to the pixel block of the data is also read out.
[0217]
(Image Data Decoding Unit 90)
The image data decoding unit 90 includes a predetermined number of image data including image data blocks output from the second image data generation unit 82B corresponding to the image data of the pixel blocks stored in the pixel block storage unit 60B. The encoding process of the image data of the JPEG system or the like is performed for each block.
[0218]
Here, the operation of the image data processing apparatus shown in FIG. 14 having the above configuration will be described.
[0219]
Image data corresponding to each pixel of the original image stored in the frame memory 50 is read as a first pixel block, for example, for each 8 × 2 pixel block, and stored in the pixel block storage unit 60B.
[0220]
The image data stored in the pixel block storage unit 60B is read by the first image data generation unit 81B for each first pixel block, and the read image data is combined for each first small pixel block. Thus, one or a plurality of image data is generated for each first small pixel block.
[0221]
For example, image data is read from the pixel block storage unit 60B for each 8 × 2 pixel block as the first pixel block. The read image data is obtained by assuming that a pixel block of 1 × 2 pixels arranged adjacently on a horizontal line of the original image so that pixels at both ends overlap each other as a first small pixel block. Image data of an 8 × 2 pixel block is synthesized for each pixel block by two types of weighting methods. As a result, the image data of the 8 × 2 pixel block stored in the pixel block storage unit 60B is converted into the image data of the 8 × 4 pixel block, and the number of image data is doubled.
[0222]
Further, when some pixels of the first small pixel block are included in a predetermined boundary area of another first pixel block adjacent to the first pixel block being read from the pixel block storage unit 60B, In one data generation unit 81B, the image data of some of the pixels stored in the boundary data storage unit 70B is read. Then, the read image data and the image data of the remaining pixels of the first small pixel block read from the pixel block storage unit 60B are combined. Thereby, one or a plurality of image data corresponding to the first small pixel block is generated.
[0223]
For example, assuming that a block of small pixels of 1 × 2 pixels is arranged on the left side of the 8 × 2 pixel block, the first image data generation unit 81B is located on the right side of the 8 × 2 pixel block. When a 1 × 2 pixel small pixel block is synthesized, the image data of the rightmost pixel of the small pixel block is read from the boundary data storage unit 70B. Then, the image data read from the boundary data storage unit 70B and the image data of the remaining one pixel of the small pixel block read from the pixel block storage unit 60B are combined by two types of weighting methods. . Thereby, image data for two pixels corresponding to the small pixel block is generated.
[0224]
The image data generated in the first image data generation unit 81B is synthesized for each second small pixel block in the second image data generation unit 82B, and one or two corresponding to the second small pixel blocks are combined. A plurality of image data are generated.
For example, the image data converted from the 8 × 2 pixel block into the 8 × 4 pixel block in the first image data generation unit 81B is arranged in a 1 × array in which the pixels at both ends are adjacently arranged on a horizontal line so as to overlap each other. The two small pixel blocks are combined by two types of weighting methods, and image data for two pixels corresponding to each small pixel block is generated. As a result, the 8 × 4 pixel block is converted into an 8 × 8 pixel block, and the number of image data is further doubled.
[0225]
In addition, some pixels of the second small pixel block are included in a predetermined boundary region of another second pixel block adjacent to the second pixel block being generated in the first image data generation unit 81B. In this case, in the second image data generation unit 82B, image data of pixels equivalent to the partial pixels in the above-described boundary area is read from the boundary data storage unit 70B. Then, the read image data is combined with the image data of the remaining pixels of the second small pixel block generated by the first image data generation unit 81B. Thereby, one or a plurality of image data corresponding to the second small pixel block is generated.
For example, assuming that a block of small pixels of 1 × 2 pixels is arranged on the left side of an 8 × 4 pixel block, the second image data generation unit 82B positions the block at the rightmost side of the 8 × 4 pixel block. When the 1 × 2 small pixel block is synthesized, image data of a pixel equivalent to the right end pixel of the small pixel block is read from the boundary data storage unit 70B. Then, the read image data and the image data of the remaining one pixel of the small pixel block generated by the first image data generation unit 81B are combined by two types of weighting methods, and two pixels are combined. Is generated.
[0226]
In the boundary data storage unit 70B, another pixel block adjacent to the pixel block next read from the pixel block storage unit 60B by the image data generation unit 81B, or the pixel block storage unit 60B by the first image data generation unit 81B. Image data that is another pixel block adjacent to the pixel block being read from and that is included in a predetermined boundary area according to the set conversion ratio of the number of image data is read from the frame memory 50 and stored. You. The reading process from the frame memory 50 is performed together with the reading process from the frame memory 50 in the pixel block storage unit 60B.
[0227]
For example, when the pixel block storage unit 60B has the above-described two-bank configuration, image data is read from the pixel block storage unit 60B to the first image data generation unit 81B, and image data is read from the frame memory 50. It is read and written to the pixel block storage unit 60B. Of the image data read from the frame memory 50, image data included in a predetermined boundary area is fetched and stored in the boundary data storage unit 70B as needed.
This eliminates the need for a special circuit for reading image data from the frame memory 50 in the boundary data storage unit 70B. Further, since the process of reading image data from the frame memory 50 is shared with the pixel block storage unit 60B, the number of accesses to the frame memory 50 is reduced.
[0228]
For example, in a case where the pixel block storage unit 60B does not have the two-bank configuration described above, when image data is read from the frame memory 50 to the pixel block storage unit 60B, the boundary adjacent to the pixel block of the read image data is used. The image data of the area is also read out together with this, and stored in the boundary data storage unit 70B.
In this case, the addresses of the image data read from the frame memory 50 are adjacent to each other in the data written in the pixel block storage unit 60B and the data written in the boundary data storage unit 70B. The read access to the frame memory 50 is shared by the pixel block storage unit 60B and the boundary data storage unit 70B. Therefore, a circuit for reading image data from the frame memory 50 can be easily shared between the pixel block storage unit 60B and the boundary data storage unit 70B.
[0229]
In the image data decoding unit 90, a predetermined number of image data including image data blocks output from the second image data generation unit 82B corresponding to the image data of the pixel blocks stored in the pixel block storage unit 60B The encoding process of the image data is performed for each block.
For example, one MCU of image data obtained by combining the chroma signal of the 8 × 8 pixel block from the second image data generating unit 82B and the luminance signal of a predetermined number of 8 × 8 pixel blocks separately read from the frame memory 50 Is subjected to JPEG encoding processing.
[0230]
As described above, according to the image data processing device shown in FIG. 14, the number of image data read out from the pixel block storage unit 60B in pixel block units is determined by the first image data generation unit 81B and the second image data The conversion is performed in two stages in the generation unit 82B. Therefore, the conversion ratio of the number of image data at each stage is reduced, and the circuit configuration can be simplified. In addition, since a small pixel block as a unit for synthesizing image data can be reduced, it is easy to set a small pixel block so as not to extend between adjacent pixel blocks. This makes it possible to simplify the storage device for storing the image data of the boundary area.
[0231]
Further, according to the image data processing apparatus shown in FIG. 14, since the conversion of the number of image data is performed in two stages, the conversion of the number of image data in one of the stages, for example, the first image data generation unit 81B Is enabled or disabled, it is possible to perform a conversion process of the number of image data based on two types of conversion ratios.
[0232]
That is, when the conversion ratio of the number of image data is set as the first conversion ratio or the second conversion ratio, the image data is read from the frame memory 50 for each predetermined pixel block corresponding to the set conversion ratio. Then, it is stored in the pixel block storage unit 60B.
In the first image data generation unit 81B, the conversion processing of the number of image data is enabled or disabled according to the set conversion ratio of the number of image data. That is, when the conversion ratio of the number of image data is set to the first conversion ratio, generation of the above-described image data corresponding to each of the first small pixel blocks is performed. Be executed. When the conversion ratio of the number of image data is set to the second conversion ratio, the image data input for each third pixel block is output as it is, thereby invalidating the conversion process of the number of image data. .
In the second image data generation unit 82B, the same processing is performed regardless of which conversion ratio is set.
This makes it possible to perform image data number conversion using two different conversion ratios.
[0233]
For example, when the conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 4: 4) is performed as the first conversion ratio, every 8 × 2 pixel block is used as the first pixel block. The image data of the chroma signal is read from the frame memory 50 and stored in the pixel block storage unit 60. In the first image data generation unit 81B, the image data is synthesized by two types of weighting methods for each small pixel block of 1 × 2 pixels, and image data of two pixels corresponding to each small pixel block is generated. As a result, the 8 × 2 pixel block of the chroma signal is converted into an 8 × 4 pixel block, and the number of image data is doubled. In the second image data generation unit 82B, a process of converting an 8 × 4 pixel block into an 8 × 8 pixel block is performed. As a result, the conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 4: 4) is performed on the chroma signal.
On the other hand, when the conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 2: 2) is performed as the second conversion ratio, as the third pixel block, every 8 × 4 pixel block: The image data of the chroma signal is read from the frame memory 50 and stored in the pixel block storage unit 60. The first image data generation unit 81B outputs the image data of the pixel block storage unit 60B stored for each 8 × 4 pixel block as it is, and the second image data generation unit 82B outputs the 8 × 4 pixel block. A process of converting into an 8 × 8 pixel block is performed. As a result, the conversion from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 2: 2) is performed on the chroma signal.
[0234]
In this case, since the conversion processing of the number of image data in the second image data generation unit 82B is shared in order to realize two conversion ratios, for example, as shown in the image data processing apparatus of FIG. The circuit scale can be further reduced as compared with a method in which two image data processing devices for performing data number conversion are separately provided.
[0235]
<Sixth embodiment>
Next, a sixth embodiment of the present invention will be described.
The image data processing device according to the sixth embodiment has, for example, a configuration equivalent to the image data processing device shown in FIG. However, a boundary data storage unit 70B, a first image data generation unit 81B, and a second image data generation unit 82B shown in FIG. And the second image data generation unit 82C.
[0236]
FIG. 15 shows an example of a configuration of a first image data generation unit 81C, a second image data generation unit 82C, and a boundary data storage unit 70C included in the image data processing device according to the sixth embodiment of the present invention. It is a block diagram.
As shown in FIG. 15, the first image data generation unit 81C includes flip-flops FF12 and FF13, selectors SL10 and SL11, an addition unit AD7, and a multiplication unit ML7.
The second image data generation unit 82C includes a flip-flop FF14, a selector SL12, an addition unit AD8, and a multiplication unit ML8.
The boundary data storage unit 70C has a register RG5, a register RG6, and a selector SL13.
The flip-flops FF12 and FF13 are an embodiment of a first data holding unit according to the fourth aspect of the present invention.
The unit including the selector SL10, the adder AD7, and the multiplier ML7 is an embodiment of the first data synthesizing unit according to the fourth aspect of the present invention.
The selector SL11 is an embodiment of the first selecting means according to the fourth aspect of the present invention.
The flip-flop FF14 is an embodiment of the second data holding unit according to the fourth aspect of the present invention.
The unit including the selector SL12, the adder AD8, and the multiplier ML8 is an embodiment of the second data synthesizing unit according to the fourth aspect of the present invention.
[0237]
(First Image Data Generation Unit 81C)
The flip-flop FF12 holds the image data S9 read from the pixel block storage unit 60B in synchronization with the clock signal CLK when the enable signal in-en2 set by the control unit (not shown) is at a low level.
The read cycle of the image data S9 is set to four times or twice the clock signal CLK by a control unit (not shown) according to the set conversion ratio of the number of image data. When the conversion ratio is set to the first conversion ratio, the read cycle of the image data S9 is set to four times the clock signal CLK, and when the conversion ratio is set to the second conversion ratio, the image data S9 is read. The cycle is set to twice the clock signal CLK.
When the conversion ratio is set to the first conversion ratio and the read cycle of the image data S9 is set to four times the clock signal CLK, the cycle of the level change at which the enable signal in-en2 changes to the high level or the low level is , Is set to twice the clock signal CLK. The level change timing is set so as to coincide with the read timing of the image data S9. Therefore, the flip-flop FF12 holds data obtained by delaying the image data S9 by two cycles of the clock signal CLK. Further, in this case, every time one image data S9 is read from the pixel block storage unit 60, sampling is performed twice on the read image data.
When the conversion ratio is set to the second conversion ratio and the read cycle of the image data S9 is set to twice the clock signal CLK, the enable signal in-en2 is always set to the low level. Therefore, the flip-flop FF12 holds the data obtained by delaying the image data S9 by one cycle of the clock signal CLK. Further, in this case, every time one image data S9 is read from the pixel block storage unit 60, sampling is performed twice on the read image data.
[0238]
The flip-flop FF13 holds the image data S22 held in the flip-flop FF12 in synchronization with the clock signal CLK when the enable signal in-en2 is at a low level.
[0239]
The selector SL10 selects the image data S11 output from the boundary data storage unit 70C when the selection signal blk-right1 output from the control unit (not shown) is at the high level, and selects the image data S11 when the selection signal blk-right1 is at the low level. The image data S22 held in the flip-flop FF12 is selected, and the selected image data is output as the image data S23.
On the horizontal line of the original image, the pixel of the image data S22 held by the flip-flop FF12 may not be adjacent to the pixel of the image data S24 held by the flip-flop FF13. In this case, the boundary data storage unit 70C outputs the image data S11 of the pixel included in the predetermined boundary area adjacent to the pixel of the image data S24 held in the flip-flop FF13. Thus, a predetermined small pixel block of 1 × 2 pixels (corresponding to the first small pixel block in the fifth embodiment) is formed. In this case, the selection signal blk-right1 is set to a high level. In other cases, the selection signal blk-right1 is set to a low level.
[0240]
The adder AD7 adds the image data S24 held in the flip-flop FF13 and the image data S23 output from the selector SL10.
The multiplication unit ML7 multiplies the addition result of the addition unit AD6 by a weight coefficient １ ／.
[0241]
When the first conversion ratio is set as the conversion ratio of the number of image data, the selector SL11 selects the image data resulting from the multiplication in the multiplying unit ML7, and when the second conversion ratio is set, the selector SL11 supplies the data to the flip-flop FF12. The stored image data S22 is selected, and the selected image data is output as image data S25.
[0242]
(Second image data generation unit 82C)
The flip-flop FF14 holds the image data S25 generated by the first image data generation unit 81C in synchronization with the clock signal CLK.
[0243]
The selector SL12 selects the image data S11 output from the boundary data storage unit 70C when the selection signal blk-right2 output from the control unit (not shown) is at a high level, and selects the image data S11 output from the boundary data storage unit 70C when the selection signal blk-right2 is at a low level. The image data S25 output from the first image data generation unit 81C is selected, and the selected image data is output as the image data S26.
On the horizontal line of the image constituted by the image data S25, the pixel of the image data S27 held in the flip-flop FF14 may not be adjacent to the pixel of the image data S25 held next in the flip-flop FF14. In this case, from the boundary data storage unit 70C, a pixel equivalent to a pixel adjacent to a pixel of the image data S27 held in the flip-flop FF14, that is, an element having image data equal to a pixel adjacent to a pixel of the image data S27 is obtained. Image data S11 of pixels of the image and included in a predetermined boundary area of the original image is output. In the image composed of the image data S25, the two pixels of the image data S27 and the image data S11 form a small 1 × 2 pixel small pixel block (corresponding to the second small pixel block in the fifth embodiment). Constitute. In this case, the selection signal blk-right2 is set to a high level. In other cases, the selection signal blk-right2 is set to a low level.
[0244]
The adder AD8 adds the image data S27 held in the flip-flop FF14 and the image data S26 output from the selector SL12.
The multiplication unit ML8 multiplies the addition result S28 of the addition unit AD8 by a weighting factor of ２, and outputs the multiplication result as image data S29.
[0245]
(Boundary data storage unit 70C)
The register RG5 holds the image data S10 of the chroma signal Cb read from the frame memory 50 when the enable signal xcb-en output from the control unit (not shown) is at a low level. Further, the stored image data is sequentially shifted at a predetermined timing and output to the selector SL13.
The register RG6 holds the image data S10 of the chroma signal Cr read from the frame memory 50 when the enable signal xcr-en output from the control unit (not shown) is at a low level. Further, the stored image data is sequentially shifted at a predetermined timing and output to the selector SL13.
[0246]
When the image data of the chroma signal (Cb, Cr) is read from the frame memory 50 to the pixel block storage unit 60B, the pixel to be read next from the pixel block storage unit 60B to the first image data generation unit 81C is also read. According to the set image data number conversion ratio of another pixel block adjacent to the block or another pixel block adjacent to the pixel block being read from the pixel block storage unit 60B to the first image data generation unit 81C. The image data included in the predetermined boundary area is read from the frame memory 50 by a control unit (not shown). The enable signals (xcb-en, xcr-en) are set to the low level at the timing when the image data in the boundary area is read from the frame memory 50. Accordingly, the image data of the boundary area among the image data read from the frame memory 50 is held in the registers (REG5, REG6) of the boundary data storage unit 70C.
[0247]
The selector SL13 selects image data output from either the register RG5 or the register RG6 according to the input selection signal area-cr, and outputs the selected image data as the image data S11. When the image data S9 is the chroma signal Cb, the register RG5 is selected, and when the image data S9 is the chroma signal Cr, the register RG6 is selected.
[0248]
Here, regarding the operation of the image data processing apparatus having the above-described configuration, the conversion process from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 4: 4) is performed as the conversion process using the first conversion ratio. The conversion process from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 2: 2) is performed as a conversion process using the second conversion ratio.
[0249]
// Conversion process from YCrCb ratio (4: 1: 1) to (4: 4: 4) //
FIG. 16 illustrates the outline of a chroma signal interpolation process for converting image data having a YCrCb ratio (4: 1: 1) read from the frame memory 50 into image data S29 having a YCrCb ratio (4: 4: 4). FIG.
FIG. 17 is an example of the relationship between the value of each signal and the timing shown in the block diagram of FIG. 15 when the conversion process from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 4: 4) is performed. FIG.
[0250]
Pixel blocks B20 and B21 shown in FIG. 16A indicate image data of chroma signals of 8 × 2 pixel blocks read from the frame memory 50 to the pixel block storage unit 60B. The pixel blocks of the chroma signal are read from the frame memory 50 in the order of the pixel blocks B20, B21,... And stored in the pixel block storage unit 60B. From the pixel block storage unit 60B, the image data C20 is output every four cycles of the clock signal CLK (FIG. 17A). ₀ , C20 ₁ , ..., C20 _Fifteen , C21 ₀ , C21 ₁ ,... Are read in the order of the chroma signals (FIG. 17B) and input to the flip-flop FF12.
[0251]
As shown in FIG. 17B, the read cycle of the image data from the pixel block storage unit 60B is set to be four times as long as the cycle of the clock signal CLK (FIG. 16A). The image data S9 of one pixel is read out every four cycles. The read image data S9 is held in the flip-flop FF12 every two cycles of the clock signal CLK. That is, every time the image data S9 of one pixel is read from the pixel block storage unit 60, the read image data is sampled twice. The image data sampled by the flip-flop FF12 is shifted to the flip-flop FF13 every two cycles of the clock signal CLK. That is, every time sampling is performed in the flip-flop FF12, the image data held in the flip-flop FF12 in the previous sampling is shifted to the flip-flop FF13. As a result, the flip-flops FF12 and FF13 hold two series of sampled image data.
[0252]
The image data S9 of one pixel is sampled twice, and up to two of a series of sampled image data are stored in the flip-flops FF12 and FF13. Image data is one pixel or two pixels of the original image.
[0253]
First, a case in which image data of two pixels adjacent on the horizontal line of the original image are sequentially input to the first image data generation unit 81C will be described.
When the image data held in the flip-flops FF12 and FF13 is one-pixel image data, and when the image data is two-pixel image data adjacent on the horizontal line of the original image, the selector SL10 holds the image data in the flip-flop FF12. The selected image data S22 is selected and output.
In the adder AD7, the two image data held in the flip-flops FF12 and FF13 are summed, and the sum is multiplied by a coefficient に お い て in the multiplier ML7. This multiplication result is output as image data S25 via the selector SL11. That is, the image data S25 is obtained as a result of adding the two image data held in the flip-flops FF12 and FF13 and dividing this by the value 2.
[0254]
In a case where image data of one of two adjacent pixels is input to the first image data generation unit 81C, first, when the image data of the one pixel is held in both the flip-flops FF12 and FF13, As the image data S25, image data equal to the image data of the one pixel is obtained.
Next, the data held in the flip-flop FF12 is shifted to the flip-flop FF13, and the image data of the other pixel of the two adjacent pixels is held in the flip-flop FF12 following the image data of the one pixel. And F13 hold image data of two pixels in a one-to-one ratio. Therefore, as the image data S25, image data of one pixel which is weighted and synthesized at the ratio of 1: 1 is obtained.
Therefore, when the image data of two adjacent pixels is sequentially input to the first image data generation unit 81C, the addition unit AD7 adds the image data of two pixels at a ratio of 1: 0 and 1: 1. As the image data S25, image data of two pixels synthesized by two types of weighting methods corresponding to these ratios is obtained. Since the image data of two pixels is obtained with respect to the image data of two adjacent pixels, the number of image data is doubled, and the image data of the 8 × 2 pixel block held in the pixel block storage unit 60 is 8 It is converted into image data of a × 4 pixel block.
[0255]
The pixel blocks B22 and B23 shown in FIG. 16B are obtained by converting the pixel blocks B20 and B21 of 8 × 2 pixels in FIG. 16A into 8 × 4 pixels in the first image data generation unit 81C. It is. Each image data of the pixel blocks B22 and B23 corresponds to the image data S25 generated by the first image data generation unit 81C.
For example, as shown in FIG. ₂ And C20 ₃ Are adjacent on the original image, and a small pixel block MB20 of 1 × 2 pixels ₂ Is composed. This small pixel block MB20 ₂ Of two image data (C20 ₂ , C20 ₃ ) Are sequentially input to the first image data generation unit 81C and are held in the flip-flops FF12 and FF13 (time t51 to t54 in the timing chart shown in FIG. 17), and the selection signal blk-right1 (FIG. 17G )) Becomes low level, and the data held in the flip-flop FF12 is output from the selector SL10. During this period, the small pixel block MB20 ₂ Of two image data (C20 ₂ , C20 ₃ ) Are synthesized by two types of weighting methods, whereby two-pixel image data C22 ₄ And C22 ₅ (FIG. 16B) are sequentially generated as image data S25 (FIG. 17H).
[0256]
On the other hand, when the image data held in the flip-flops FF12 and FF13 is image data of two pixels that are not adjacent on the horizontal line of the original image, one of the two pixels and the image data output from the boundary data storage unit 70C are output. The pixel of the image data S11 is adjacent on the horizontal line of the original image, and these two pixels constitute a small pixel block of 1 × 2 pixels.
In this case, the held data S22 of the flip-flop FF12, which is the image data of the other pixel of the two pixels, is replaced by the image data S11 output from the boundary data storage unit 70C by the selector SL10 and output to the addition unit AD7. You. The image data S25 is obtained as a synthesis result of the data S24 held by the flip-flop FF13, which is the image data of one of the two pixels, and the image data S11 of a predetermined boundary area adjacent to the pixel.
[0257]
In a case where image data of one of two non-adjacent pixels on the horizontal line is input to the image data generation unit 81C, first, the image data of the one pixel is held in both the flip-flops FF12 and F13. As image data S25, image data equal to the image data of the one pixel is obtained.
Next, the data held in the flip-flop FF12 is shifted to the flip-flop FF13, and the image data of the other non-adjacent two pixels is held in the flip-flop FF12 following the image data of the one pixel. The image data S11 output from the boundary data storage unit 70C is selected instead of the image data of the other pixel. In the adder AD7, image data of two pixels is added at a ratio of 1: 1. Therefore, as the image data S25, image data of one pixel synthesized by a weighting method corresponding to the ratio is obtained.
Therefore, when image data of two pixels that are not adjacent to each other are sequentially input to the image data generation unit 82C, the image data of one pixel of the two pixels and the boundary area adjacent to the one pixel are used as the image data S25. Image data of two pixels synthesized by weighting the image data with the ratio of 1 to 0 and 1 to 1 is obtained.
[0258]
For example, as shown in FIG. ₃ And C21 ₂ Are adjacent on the original image, and a small pixel block MB20 of 1 × 2 pixels ₃ Is composed. In contrast, the image data generation unit 82C includes the image data C20 ₃ And the image data C20 ₄ Are not adjacent to each other on the horizontal line of the original image.
This image data C20 ₃ And C20 ₄ Are sequentially input to the image data generation unit 82C, first, the image data C20 is supplied to both the flip-flops FF12 and FF13. ₃ Is held (at times t55 and t56 in the timing chart shown in FIG. 13), the selection signal blk-right1 goes low, and the data held in the flip-flop FF12 is output from the selector SL10. As the image data S25, the image data C20 ₃ Is output.
Next, the data held in the flip-flop FF12 is shifted to the flip-flop FF13, and the image data C20 ₄ Is the image data C20 ₃ Then, when the data is held in the flip-flop FF12 (time t57 and time t58 in the timing chart shown in FIG. 13), the signal level of the selection signal blk-right1 (FIG. 17 (G)) becomes high. Thereby, the selector SL10 outputs the image data C20 ₄ Instead of the image data C21 output from the boundary data storage unit 70A as the image data S11 ₂ Is selected and output to the adder AD7 as image data S23 (FIG. 17E). In the adder AD7, the image data C20 stored in the flip-flop F11 ₃ And the image data C21 output from the boundary data storage unit 70C ₂ Are added in a one-to-one ratio, so that, as the image data S25, image data of one pixel synthesized by a weighting method corresponding to this ratio is obtained.
As a result, at time t55 to t58, the small pixel block MB20 extending over the two different pixel blocks B20 and B21 ₃ Of the two pixels corresponding to the small pixel block (C22). ₆ , C22 ₇ ) Is generated.
[0259]
In this manner, the image data S25 converted from 8 × 2 pixels to 8 × 4 pixels in the first image data generation unit 81C is transmitted to the second image data generation unit 82C every two cycles of the clock signal CLK2. Is output.
[0260]
The image data S25 output every two cycles of the clock signal CLK2 is held in the flip-flop FF14 every cycle of the clock signal CLK. That is, every time the one-pixel image data S25 is output from the first image data generation unit 81C, the output image data is sampled twice.
Therefore, the image data S27 held in the flip-flop FF14 and the image data S25 sampled subsequent to the image data S27 are the same as the image data for one or two pixels in the image constituted by the image data S25. Become.
[0261]
First, a case where image data of two pixels adjacent on a horizontal line of an image constituted by the image data S25 are sequentially output from the first image data generation unit 81C will be described.
When the image data S27 held in the flip-flop FF14 and the image data S25 to be sampled next are one-pixel image data, and two pixels adjacent to each other on the horizontal line of the image constituted by the image data S25. If it is image data, the selector SL12 selects and outputs the image data S25 to be sampled next.
In the adder AD8, the image data S25 and the image data S27 are added, and the added value is multiplied by a coefficient に お い て in the multiplier ML8. This multiplication result S28 is output as image data S29. That is, the image data S29 is obtained as a result of adding the image data S25 and the image data S27 and dividing this by the value 2.
[0262]
In the case where image data of one of two adjacent pixels is output from the first image data generation unit 81C, first, when the image data of the one pixel is sampled only once in the flip-flop FF14, the sampling is performed. Both the subsequent image data S27 and the next sampled image data S25 are the image data of the one pixel. Therefore, as the image data S29, image data equal to the image data of the one pixel is obtained.
Next, the sampling of the image data S25 is performed once more in the flip-flop FF14, and the image data of the other pixel of the two adjacent pixels is output from the first image data generation unit 81C following the image data of the one pixel. When output, the sampled image data S27 becomes the image data of the one pixel, and the next sampled image data S25 becomes the image data of the other pixel. For this reason, the image data of two adjacent pixels is input to the addition unit AD8 at a ratio of 1: 1. As the image data S29, the image data of the two pixels is weighted at a ratio of 1: 1 and synthesized. Thus, image data of one pixel is obtained.
Therefore, when the image data of two adjacent pixels are sequentially output from the first image data generation unit 81C, the image data of two pixels is weighted at a ratio of 1: 0 and 1: 1, as the image data S29. Thus, image data of two pixels combined is obtained. Since the image data of two pixels is obtained with respect to the image data of two adjacent pixels, the number of image data is doubled and the image data of the 8 × 4 pixel block output from the first image data generation unit 81C Is converted into image data of an 8 × 8 pixel block.
[0263]
The pixel blocks B24 and B25 shown in FIG. 16C are pixel blocks obtained by converting the 8 × 4 pixel pixel blocks B22 and B23 in FIG. 16B into 8 × 8 pixels in the second image data generation unit 82C. It is. Each image data of the pixel blocks B24 and B25 corresponds to the image data S29 generated by the second image data generation unit 82C.
For example, as shown in FIG. ₄ And C20 ₅ Are adjacent on the image constituted by the image data S25, and the 1 × 2 pixel small pixel block MB22 ₄ Is composed. This small pixel block MB22 ₄ Of two image data (C22 ₄ , C22 ₅ ) Are sequentially output from the first image data generating unit 81C and are held in the flip-flop FF14 (time t52 and t53 in the timing chart shown in FIG. 17), and the selection signal blk-right2 (FIG. 17 (K)) Becomes low level, and the selector SL12 outputs image data S25 to be sampled next in the flip-flop FF14. During this period, the small pixel block MB22 ₄ Of two image data (C22 ₄ , C22 ₅ ) Are synthesized by two types of weighting methods, whereby two-pixel image data C24 ₈ And C24 ₉ (FIG. 16C) are sequentially generated as image data S29 (FIG. 17M).
[0264]
On the other hand, when the two-pixel image data output from the first image data generation unit 81C is not adjacent to the horizontal line of the image constituted by the image data S25, the image data S27 held in the flip-flop FF14 And the pixel of the image data S25 that is next held in the flip-flop FF14 become image data of two pixels that are not adjacent to each other.
In this case, from the boundary data storage unit 70C, a pixel equivalent to a pixel adjacent to a pixel of the image data S27 held in the flip-flop FF14, that is, an element having image data equal to a pixel adjacent to a pixel of the image data S27 is obtained. Image data S11 of pixels of the image and included in a predetermined boundary area of the original image is output. The two pixels of the image data S11 and the image data S27 form a small 1 × 2 pixel small pixel block on the horizontal line of the image constituted by the image data S25.
In this case, the image data S25 is replaced by the selector SL12 with the image data S11 output from the boundary data storage unit 70C, and output to the addition unit AD8. The image data S29 is obtained as a result of synthesizing the held data S27 of the flip-flop FF14 and the image data S11 of a predetermined boundary area.
[0265]
When the image data of one of the two non-adjacent pixels is output from the first image data generation unit 81C, first, if the image data of the one pixel is sampled only once in the flip-flop FF14, the sampling is performed. Both the subsequent image data S27 and the next sampled image data S25 are the image data of the one pixel. Therefore, as the image data S29, image data equal to the image data of the one pixel is obtained.
Next, the sampling of the image data S25 is performed once more in the flip-flop FF14, and the image data of the other non-adjacent two pixels is output from the first image data generation unit 81C following the image data of the one pixel. When output, the sampled image data S27 becomes the image data of the one pixel, and the next sampled image data S25 becomes the image data of the other pixel. At this time, the selector SL12 selects the image data S11 output from the boundary data storage unit 70C instead of the next sampled image data S25, and outputs the selected image data S11 to the addition unit AD8. Image data S11 and image data S27 forming a small pixel block of 1 × 2 pixels are input to the adder AD8 at a ratio of 1: 1. As the image data S29, image data of one pixel is obtained by weighting the two-pixel image data at a one-to-one ratio and combining the two.
Therefore, when image data of two pixels that are not adjacent to each other are sequentially output from the first image data generation unit 81C, the image data of one pixel of the two pixels and the image data of the boundary data storage unit 70C are output as the image data S29. The image data S11 to be obtained is weighted at a ratio of 1: 0 and 1: 1, to obtain image data of two pixels synthesized.
[0266]
For example, as shown in FIG. ₇ And C23 ₄ Are adjacent on the image constituted by the image data S25, and the 1 × 2 pixel small pixel block MB22 ₇ Is composed. On the other hand, the first image data generation unit 81C sends the image data C22 ₇ And the image data C22 ₈ Is entered. These two image data are not adjacent on the horizontal line of the image constituted by the image data S25.
This image data C22 ₇ And C22 ₈ Are sequentially output from the first image data generation unit 81C, first, the image data C22 ₇ Is sampled only once in the flip-flop FF14 (time t58 in the timing chart shown in FIG. 17), the selection signal blk-right2 (FIG. 17 (K)) becomes low level, and the flip-flop The FF 14 outputs the next sampled image data S25. The adder AD8 stores the image data C22 as the image data S26 and S27. ₇ Is entered. As the image data S29, the image data C22 ₇ Is output.
Next, the sampling of the image data S25 is performed once more in the flip-flop FF14, and the image data C22 is sampled. ₈ Is the image data C22 ₇ Is output from the first image data generation unit 81C, the sampled image data S27 becomes the image data C22. ₇ And the next sampled image data S25 is image data C22 ₈ It becomes. At this time, in the selector SL12, the next sampled image data C22 ₈ Instead of the image data C21 output from the boundary data storage unit 70C. ₂ Is selected and output to the addition unit AD8.
The pixels included in the boundary region L21 of the original image are equivalent to the pixels included in the boundary region L23 on the image obtained by converting the original image in the first image data generation unit 81C. That is, since each pixel of the boundary region L23 is generated by combining each pixel of the boundary region L21 with a weight of 100%, both pixels have the same image data. That is, the image data C21 output from the boundary data storage unit 70C ₂ Is the image data C23 ₄ Has a value equal to
Therefore, the adder AD8 stores the image data C22 constituting the small pixel block of 1 × 2 pixels. ₇ And C23 ₄ Are input at a ratio of 1: 1. As the image data S29, image data of one pixel is obtained by weighting the two-pixel image data at a one-to-one ratio and combining the two.
As a result, the small pixel block MB22 ₇ Of two image data (C22 ₇ , C23 ₄ ) Are synthesized by two types of weighting methods, whereby two-pixel image data C24 ₁₄ And C24 _Fifteen (FIG. 16C) are sequentially generated as image data S29.
[0267]
Further, when the image data of the chroma signal is read from the frame memory 50 to the pixel block storage unit 60B, the 8 × 2 pixel data to be read next from the pixel block storage unit 60B to the first image data generation unit 81C is also performed. Another 8 × 2 pixel block adjacent to the pixel block, or another 8 × 2 pixel block adjacent to the 8 × 2 pixel block being read from the pixel block storage unit 60B to the first image data generation unit 81C, The image data included in the predetermined boundary area is read from the frame memory 50 by a control unit (not shown).
[0268]
In FIG. 16A, regions L20 and L21 of 8 × 1 pixels located at the left ends of the 8 × 2 pixel blocks B20 and B21 correspond to this boundary region.
For example, when the pixel block storage unit 60B has a two-bank configuration, the image data of the pixel block B20 is read out to the first image data generation unit 81C and the image of the pixel block B21 to be processed next to the pixel block B20 at the same time. Data is read from the frame memory 50 to the pixel block storage unit 60B. In this case, when the image data of the pixel block B21 is read from the frame memory 50 to the pixel block storage unit 60B, the pixel adjacent to the pixel block B20 being read from the pixel block storage unit 60B to the first image data generation unit 81C is read. The image data of the boundary area L21 of the block B21 is read from the frame memory 50.
Further, when the pixel block storage unit 60B does not have the two-bank configuration, when the image data of the pixel block B20 is read out to the first image data generation unit 81C, the first block is output from the pixel block storage unit 60B. The reading process to the image data generation unit 81C is not performed. In this case, when the image data of the pixel block B20 is read from the frame memory 50 to the pixel block storage unit 60B, it is adjacent to the pixel block B20 to be read next from the pixel block storage unit 60B to the first image data generation unit 81C. The image data of the boundary area L21 of the pixel block B21 is read from the frame memory 50 together with the image data of the pixel block B20.
[0269]
The enable signal (xcb-en, xcr-en) is set to a low level at the timing when the image data in the boundary area is read from the frame memory 50 in the process of reading from the frame memory 50 described above. As a result, the image data of the boundary area is held in the registers (REG5, REG6) of the boundary data storage unit 70C. The held image data of the boundary region is used when the first image data generation unit 81C and the second image data generation unit 82C generate image data.
[0270]
// Conversion process from YCrCb ratio (4: 1: 1) to (4: 2: 2) //
In this case, in the pixel block storage unit 60B, the image data of the chroma signal of the frame memory 50 is read and stored for each 8 × 4 pixel block.
The read cycle from the pixel block storage unit 60B to the first image data generation unit 81C is set to twice the cycle of the clock signal CLK.
Since the enable signal in-en2 input to the flip-flop FF12 is always set to the low level, the flip-flop FF12 holds the read data S9 from the pixel block storage unit 60B every cycle of the clock signal CLK. That is, in the flip-flop FF12, each time the image data S9 of one pixel is read from the pixel block storage unit 60B, the read image data is sampled twice.
In the selector SL11 in the first image data generation unit 81C, the data held in the flip-flop FF12 is selected as the image data S25, and is directly input to the flip-flop FF14.
[0271]
When the image data of the chroma signal is read from the frame memory 50 to the pixel block storage unit 60B, the 8 × 4 pixel data to be read next from the pixel block storage unit 60B to the first image data generation unit 81C is also read. Another 8 × 4 pixel block adjacent to the pixel block, or another 8 × 4 pixel block adjacent to the 8 × 4 pixel block being read from the pixel block storage unit 60B to the first image data generation unit 81C, The image data included in the predetermined boundary area is read from the frame memory 50 by a control unit (not shown). The image data included in the read boundary area is held in the register RG5 or RG6 of the boundary data storage unit 70C.
[0272]
From the above, the configuration and operation of the second image data generation unit 82C and the flip-flop FF12 are equivalent to the configuration and operation of the image data generation unit 82A shown in FIG. Therefore, by the same operation as that described in the description of the image data generation unit 82A, the image data of the 8 × 4 pixel block read from the pixel block storage unit 60B is converted into the image data of the 8 × 8 pixel block, A conversion process from the YCrCb ratio (4: 1: 1) to (4: 2: 2) is realized.
[0273]
As described above, according to the image data processing apparatus including the first image data generation unit 81C, the second image data generation unit 82C, and the boundary data storage unit 70C illustrated in FIG. 15, the image data illustrated in FIG. Similar to the processing device, the conversion processing of the number of image data is executed in two stages in the first image data generation unit 81C and the second image data generation unit 82C. Therefore, the conversion ratio of the number of image data at each stage is reduced, and the circuit configuration can be simplified.
In addition, according to the image data processing device shown in FIG. 15, by enabling or disabling the conversion process of the number of image data in the first image data generation unit 81C, the conversion of the number of image data by the two conversion ratios Processing can be performed. In this case, since the conversion processing of the number of image data in the second image data generation unit 82C is shared in order to realize two conversion ratios, for example, as shown in the image data processing apparatus of FIG. The circuit scale can be further reduced as compared with a method in which two image data processing devices for performing data number conversion are separately provided.
[0274]
The selection signal (blk-right1, blk-right2, area-cr) and the enable signal (xcb-en, xcr-en, in-en2) correspond to one MCU of image data processed by the image data encoding unit 90. Each time it is read from the block storage unit 60, it is generated at the same timing by a control unit (not shown).
Therefore, for example, each time one MCU of image data processed by the image data encoding unit 90 is read from the pixel block storage unit 60B, the image data is initialized at the time of reading the first image data, and the image data is read. And a selection signal (blk-right1, blk-right2, area-cr) and an enable signal (xcb-en, xcr-en, in-en2) corresponding to the count value of the counter. By providing such a decoding circuit in the control unit, these signals can be easily generated.
[0275]
<Seventh embodiment>
Next, a seventh embodiment of the present invention will be described.
FIG. 18 is a block diagram illustrating an example of a configuration of a camera system 100 according to the third embodiment of the present invention.
The camera system 100 shown in FIG. 18 includes an optical system 101, a CCD 102, an A / D converter 103, an image compressor 104, an SDRAM 114, and a CPU 115.
The image compression unit 104 includes a CCD signal processing unit 105, a bus 106, a buffer 107, an SDRAM interface (SDRAM I / F) 108, a JPEG processing unit 109a, a JPEG interface unit 109b, and a clock generation unit 110. , A system controller 111, a CPU interface (CPU I / F) 112, and a memory controller 113.
The unit including the optical system 101, the CCD 102, and the A / D conversion unit 103 is an embodiment of the photographing unit according to the eighth and ninth aspects of the present invention.
The SDRAM 114 is an embodiment of a frame image storage unit according to the eighth and ninth aspects of the present invention.
The JPEG interface unit 109a is an embodiment of the image data encoding unit according to the eighth and ninth aspects of the present invention.
The unit including the JPEG interface 109b and the buffer 107 is one embodiment of the image data processing means according to the eighth and ninth aspects of the present invention.
[0276]
The optical system 101 captures a desired image by a user's operation, and forms an optical signal of the captured image on an imaging surface of the CCD 102.
The CCD 102 converts an optical signal on the imaging surface formed by the optical system 101 into an electric signal, and outputs the electric signal to the A / D converter 103 as an analog image signal.
The A / D converter 103 converts the analog image signal input from the CCD 102 into a digital image signal having a predetermined gradation, and outputs the digital image signal to the CCD signal processing unit 105 of the image compression unit 104.
[0277]
The CCD signal processing unit 105 of the image compression unit 104 separates the input digital image signal into R (red), G (green), and B (blue) color signals under the control of the system controller 111, and Gamma correction is performed on the signal for color reproducibility, and a luminance signal and a chroma signal are generated. The generated image data including the luminance signal and the chroma signal is output to the buffer 107 via the bus 106.
[0278]
The buffer 107 sequentially stores the image data input from the CCD signal processing unit 105 via the bus 106, and outputs the image data to the SDRAM I / F 108 under the control of the memory controller 113 when a certain amount is stored. Further, it temporarily stores the image data read from the SDRAM 114, which is input from the SDRAM I / F 108, and outputs it to the JPEG processing unit 109a via the bus 106 and the JPEG interface unit 109b.
The SDRAM I / F 108 is an external memory of the image compression unit 104, and stores image data of each predetermined unit input from the buffer 107 in the SDRAM 114 under the control of the memory controller 113. Also, the image data stored in the SDRAM 114 is read out for each 8 × 8 pixel block and output to the buffer 107.
[0279]
The JPEG processing unit 109a JPEG-encodes image data input for each MCU via the JPEG interface unit 109b to generate an encoded bit stream. Then, the generated encoded bit stream is output to the CPU 115 via the JPEG interface unit, the bus 106, and the CPU I / F 112.
[0280]
The JPEG interface unit 109b performs YCrCb ratio conversion on image data read from the SDRAM 114 under the control of the system controller 111 and input for each predetermined image block via the buffer 107 and the bus 106. Then, the image data of 1 MCU including the 8 × 8 pixel block of the chroma signal whose data number has been converted by the YCrCb ratio conversion is sequentially output to the JPEG processing unit 109a. The coded bit stream generated by the JPEG processing unit 109a is output to the CPU 115 via the bus 106 and the CPU I / F 112.
[0281]
Note that the JPEG interface unit 109b includes the image data generation unit and the boundary data storage unit described in the third to sixth embodiments.
For example, the block diagram shown in FIG. 7 includes configurations similar to those of the boundary data storage unit 70, the image data generation unit 81, the image data generation unit 82, and the selector SL4. 8 and 9 include configurations similar to those of the boundary data storage unit 70A, the image data generation unit 81A, and the image data generation unit 82A. The block diagram shown in FIG. 14 includes configurations similar to those of the boundary data storage unit 70B, the first image data generation unit 81B, and the second image data generation unit 82B. The block diagram shown in FIG. 15 includes configurations similar to those of the boundary data storage unit 70C, the first image data generation unit 81C, and the second image data generation unit 82C.
[0282]
The clock generation unit 110 generates a clock to be used in each unit in the image compression unit 104 based on the control of the system controller 111 and provides the clock to each component.
The bus 106 schematically shows a data bus in the image compression unit 104. Transfer of image data from the CCD signal processing unit 105 to the buffer 107 and from the buffer 107 to the JPEG interface unit 109b, and transfer of an encoded bit stream from the JPEG interface unit 109b to the CPU I / F 112 via the bus 106 And so on.
[0283]
The system controller 111 operates under the control of the CPU 115, and operates the image compression unit 104, that is, stores the input image data in the SDRAM 114, transfers the image data stored in the SDRAM 114 to the JPEG processing unit 109a, Each component of the image compression unit 104 is controlled so that operations such as JPEG encoding in the JPEG processing unit 109a and output of the encoded image data to the CPU 115 can be appropriately performed.
The CPU I / F 112 is an interface with the CPU 115 and performs control signals from the CPU 115, input of image signals, control signals to the CPU 115, output of encoded data, and the like.
The memory controller 113 controls the buffer 107 and the SDRAM I / F 108 under the control of the system controller 111, and controls storage of image data in the SDRAM 114, reading of image data stored in the SDRAM 114, and the like.
[0284]
The SDRAM 114 is a memory that temporarily stores image data including a captured luminance signal and a chroma signal. Image data photographed by the optical system 101, the CCD 102, and the A / D converter 103 are temporarily stored in the SDRAM 114, then sequentially supplied to the JPEG processing unit 109a, encoded, output to the CPU 115, and stored, displayed, Used for transmission.
The CPU 115 captures a desired image by the optical system 101, the CCD 102, the A / D conversion unit 103, the image compression unit 104, and the SDRAM 114, performs image processing, stores and reproduces image data, JPEG encodes, stores JPEG encoded data, Each component of the camera system 100 is controlled so that each process such as display and transmission is appropriately performed, and the camera system 100 performs a desired operation as a whole.
[0285]
In the camera system 100 having such a configuration, first, when a desired image is captured by the optical system 101 by a user's operation, the CCD 102 converts the optical signal into an electric signal to generate an image signal. The image signal is converted from an analog signal to a digital signal by an A / D converter 103, further decomposed into respective color signals by a CCD signal processing unit 105 of an image compression unit 104, subjected to gamma correction, and then output as a luminance signal. It is converted into image data composed of chroma signals.
The image data is temporarily stored in the SDRAM 114 via the buffer 107 and the SDRAM I / F 108, and is sequentially read out for each predetermined pixel block, the YCrCb ratio is converted in the JPEG interface unit 109b, and the image data is stored for each 8 × 8 pixel block. Is input to the JPEG processing unit 109a.
The JPEG processing unit 109a JPEG-encodes the sequentially input image data of each 8 × 8 pixel block, and generates a JPEG-encoded data stream of a predetermined format. This JPEG encoded data stream is output to the CPU 115 via the CPU I / F 112, where processing such as storage, display, and transmission is performed.
[0286]
As described above, in the camera system shown in FIG. 18, the number of data can be directly converted into image data input to the JPEG interface unit 109b for each predetermined pixel block. This eliminates the process of reading and writing image data from and to the frame memory and line memory, so that data processing can be speeded up and the circuit scale can be significantly reduced.
In addition, since the YCrCb ratio can be changed in various ways, it is possible to handle image data in various formats, and it is possible to improve convenience.
[0287]
Note that the present invention is not limited to the above-described first to seventh embodiments.
Various specific numerical values and the like shown in each of the above-described embodiments, for example, the size of the pixel block or the small pixel block, the arrangement method of the small pixel block, the weighting factor, the readout cycle of the image data, the number of samplings, etc. This is merely an example for explanation, and can be arbitrarily modified.
[0288]
In the first and second embodiments, only the case where the number of image data is reduced is shown, and in the third to sixth embodiments, only the case where the number of image data is enlarged is shown. The present invention is not limited to this. That is, in the image data generation unit shown in the first and second embodiments, the size of the small pixel block, the arrangement method, the type of weighting when combining image data from one small pixel block, and the like are appropriately determined. This makes it possible to increase the number of image data. Similarly, in the image data generation unit shown in the third to sixth embodiments, the size of the small pixel block, the arrangement method thereof, the type of weighting when combining image data from one small pixel block, and the like are set. By appropriately setting, the number of image data can be reduced.
[0289]
【The invention's effect】
According to the present invention, firstly, it is possible to provide an image data processing apparatus capable of converting the number of image data with a plurality of different conversion ratios.
Second, it is possible to provide an image data processing method capable of converting the number of image data with a plurality of different conversion ratios.
Third, it is possible to provide a camera system capable of converting the number of image data with a plurality of different conversion ratios for image data of a captured image.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating an example of a configuration of an image data processing device according to a first embodiment of the present invention.
FIG. 2 illustrates an example of a configuration of a first image data generation unit, a second image data generation unit, a boundary data storage unit, and a control unit included in an image data processing device according to a second embodiment of the present invention. It is a block diagram shown.
FIG. 3 illustrates an outline of chroma signal interpolation processing for converting image data S1 having a YCrCb ratio (4: 4: 4) output from an image data decoding unit into image data having a YCrCb ratio (4: 1: 1). FIG.
FIG. 4 is an example of the relationship between the value of each signal and the timing shown in the block diagram of FIG. 2 when a conversion process from a YCrCb ratio (4: 4: 4) to a YCrCb ratio (4: 1: 1) is performed. FIG.
FIG. 5 illustrates an outline of a chroma signal interpolation process for converting image data S1 having a YCrCb ratio (4: 2: 2) output from an image data decoding unit into image data having a YCrCb ratio (4: 1: 1). FIG.
FIG. 6 is an example of the relationship between the value of each signal and the timing shown in the block diagram of FIG. 2 when a conversion process from the YCrCb ratio (4: 2: 2) to the YCrCb ratio (4: 1: 1) is performed. FIG.
FIG. 7 is a block diagram illustrating an example of a configuration of an image data processing device according to a third embodiment of the present invention.
FIG. 8 is a first block diagram illustrating an example of a configuration of an image data generation unit and a boundary data storage unit included in an image data processing device according to a fourth embodiment of the present invention.
FIG. 9 is a second block diagram illustrating an example of a configuration of an image data generation unit and a boundary data storage unit included in an image data processing device according to a fourth embodiment of the present invention.
FIG. 10 is a first diagram illustrating an outline of a chroma signal interpolation process for converting image data having a YCrCb ratio (4: 1: 1) read from a frame memory into image data having a YCrCb ratio (4: 4: 4). FIG.
11 shows an example of the relationship between the value of each signal and the timing shown in the block diagram of FIG. 8 when the conversion process from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 4: 4) is performed. FIG.
FIG. 12 is a diagram schematically illustrating a chroma signal interpolation process for converting image data having a YCrCb ratio (4: 1: 1) read from a frame memory into image data having a YCrCb ratio (4: 2: 2). is there.
13 shows an example of the relationship between the value of each signal and the timing shown in the block diagram of FIG. 9 when the conversion process from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 2: 2) is performed. FIG.
FIG. 14 is a block diagram illustrating an example of a configuration of an image data processing device according to a fifth embodiment of the present invention.
FIG. 15 is a block diagram illustrating an example of a configuration of a first image data generation unit, a second image data generation unit, and a boundary data storage unit included in an image data processing device according to a sixth embodiment of the present invention. is there.
FIG. 16 is a second diagram illustrating an outline of a chroma signal interpolation process for converting image data having a YCrCb ratio (4: 1: 1) read from a frame memory into image data having a YCrCb ratio (4: 4: 4). FIG.
FIG. 17 shows an example of the relationship between the value of each signal and the timing shown in the block diagram of FIG. 15 when the conversion process from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 4: 4) is performed. FIG.
FIG. 18 is a block diagram illustrating an example of a configuration of a camera system according to a third embodiment of the present invention.
FIG. 19 is a block diagram illustrating an example of a configuration of an interface unit of a general digital video image processing apparatus.
FIG. 20 is a diagram illustrating one MCU in a YCrCb ratio (4: 4: 4) format.
FIG. 21 is a diagram illustrating one MCU in a YCrCb ratio (4: 2: 2) format.
FIG. 22 is a diagram illustrating one MCU in a YCrCb ratio (4: 1: 1) format.
FIG. 23 is a diagram schematically illustrating a chroma signal interpolation process for converting image data having a YCrCb ratio (4: 2: 2) output from a JPEG decoding unit to image data having a YCrCb ratio (4: 1: 1). It is.
FIG. 24 is a block diagram illustrating an example of a configuration of a general JPEG interface circuit that performs conversion from a YCrCb ratio (4: 2: 2) to a YCrCb ratio (4: 1: 1).
FIG. 25 is a chroma signal interpolation process for reading out image data having a YCrCb ratio (4: 1: 1) from a frame memory, converting the image data into image data having a YCrCb ratio (4: 2: 2), and outputting the image data to a JPEG encoding unit; FIG.
FIG. 26 is a block diagram illustrating an example of a configuration of a general JPEG interface circuit that performs conversion from a YCrCb ratio (4: 1: 1) to a YCrCb ratio (4: 2: 2).
[Explanation of symbols]
10 image data decoding unit, 21, 21A, 81B, 81C first image data generation unit, 22, 22A, 82B, 82C second image data generation unit, 30, 30A, 70, 70A to 70C Boundary data storage unit, 40, 60, 60B: pixel block storage unit, 50: frame memory, 81, 81A, 82, 82A: image data generation unit, 90: image data encoding unit, FF1 to FF14: flip-flop, AD1 ... AD8: adder, ML1 to ML8, multiplier, SL1 to SL13, selector, RG1 to RG6, register, CONT1, controller

Claims (25)

An image data processing device for converting the number of image data,
Image data corresponding to each pixel of the original image is input for each first pixel block, and the input image data is synthesized for each first small pixel block smaller than the first pixel block. First image data generating means for generating one or more image data corresponding to the first small pixel block;
Of the image data generated by the first image data generating means, the image data included in a predetermined boundary area of a second pixel block corresponding to the first pixel block on an image constituted by the image data Boundary data storage means for storing pixel image data;
The image data generated by the first image data generating means is synthesized for each second small pixel block smaller than the second pixel block, and one image data corresponding to each of the second small pixel blocks is synthesized. Alternatively, a plurality of image data is generated, and some of the pixels of the second small pixel block are separated from another second pixel block adjacent to the second pixel block being generated by the first image data generating means. The image data of some of the pixels stored in the boundary data storage means and the remainder of the second small pixel block generated by the first image data generation means. A second image data generating means for generating one or a plurality of image data corresponding to the second small pixel block by combining the image data of . When the conversion ratio of the number of image data is set to the first conversion ratio, the first image data generation unit executes generation of image data corresponding to the first small pixel block, and the conversion ratio is If the second conversion ratio is set, the image data input for each third pixel block is output as the image data of the second pixel block.
The image data processing device according to claim 1. The first image data generating means includes:
First data holding means for sequentially inputting image data included in the first pixel block, and holding a series of input image data up to a predetermined number corresponding to the number of pixels of the first small pixel block; ,
A first data synthesizing unit for giving a predetermined weight according to the input order to synthesize the image data held in the first data holding unit and the image data input subsequently thereto,
A control circuit for generating a first enable signal indicating whether or not the image data synthesized by the first data synthesizing means is image data synthesized corresponding to the first small pixel block; ,
The image data processing device according to claim 2. The second image data generating means includes:
Based on the first enable signal, image data synthesized corresponding to the first small pixel block is sequentially input from the first data synthesizing unit, and a series of input image data is input to the second data synthesizing unit. Second data holding means for holding up to a predetermined number corresponding to the number of pixels of the small image block;
The image data held in the second data holding means and the image data input subsequently thereto are given a predetermined weight in accordance with the input order and synthesized, and one of the series of image data of the synthesis source is synthesized. When the pixel of the image data of the unit and the pixel of the image data stored in the boundary data storage unit constitute the second small pixel block, the partial image data and the stored image data are And a second data synthesizing means for synthesizing by giving a predetermined weight to the
The control circuit generates a second enable signal indicating whether or not the image data synthesized by the first data synthesizing means is image data synthesized corresponding to the second small pixel block. Do
The image data processing device according to claim 3. An image data decoding unit that performs a predetermined decoding process on the image data of the original image encoded in a predetermined method, and outputs the decoded image data for each predetermined pixel block,
The first image data generating unit inputs the image data of the first pixel block from the image data of the predetermined pixel block output from the image data decoding unit,
The image data processing device according to claim 2. An image data processing device for converting the number of image data,
In the first stage, image data corresponding to each pixel of the original image is input for each first pixel block, and in each stage following the first stage, the image data generated in the previous stage is input to the pixel block corresponding to the first pixel block. Cascaded to generate one or more image data corresponding to each of the small pixel blocks by combining the input image data for each of the small pixel blocks determined in each stage. A plurality of image data generating means,
A boundary data storage unit corresponding to a part or all of the plurality of image data generation units, and among image data input to the corresponding image data generation unit, an image formed by the image data; One or more boundary data storage means for storing image data of pixels included in a predetermined boundary area of the pixel block corresponding to the first pixel block,
The image data generation unit associated with the boundary data storage unit is configured to handle a case where some pixels of the small pixel block are included in the boundary region of another pixel block adjacent to the pixel block being input. The image data of the partial pixels stored in the boundary data storage unit to be combined with the input image data of the remaining pixels of the small pixel block are combined to form one or a plurality of pixels corresponding to the small pixel block. Generate image data for
Image data processing device. The plurality of image data generating units each determine whether to execute a process of generating image data corresponding to the small pixel block according to a set conversion ratio of the number of image data.
The image data processing device according to claim 6. An image data processing device for converting the number of image data,
Pixel block storage means for inputting and storing image data corresponding to each pixel of the original image for each predetermined pixel block according to a set conversion ratio of the number of image data;
The image data stored in the pixel block storage unit is read, and the read image data is combined for each predetermined small pixel block to generate one or more image data for each small pixel block. Image data generating means;
The conversion of another pixel block adjacent to a pixel block next read from the pixel block storage unit by the image data generation unit or another pixel block adjacent to a pixel block being read from the pixel block storage unit Boundary data storage means for inputting and storing image data included in a predetermined boundary region according to the ratio,
Among the plurality of image data generating means, having a selecting means for outputting image data of the image data generating means selected according to the conversion ratio,
The image data generating means stores the boundary data when some pixels of the small pixel block are included in the boundary area of another pixel block adjacent to the pixel block being read from the pixel block storage means. The image data of the partial pixels stored in the unit and the image data of the remaining pixels of the small pixel block read from the pixel block storage unit are combined, and one or a pixel corresponding to the small pixel block is combined. Generate multiple image data,
Image data processing device. The plurality of image data generating means include:
The image data stored in the pixel block storage means is sequentially read, and each time the image data of one pixel is read, sampling of the read image data is performed once or plural times, and a series of sampled image data is reduced to a predetermined number. Data holding means for holding;
A series of image data held in the data holding unit is synthesized, and a pixel of a part of the image data in the series of image data of the synthesis source and a pixel of the image data stored in the boundary data storage unit are set to the small size. When configuring a pixel block, each including a part of the image data and a data combining unit that combines the stored image data,
An image data processing device according to claim 8. The data synthesizing means includes:
A series of image data held in the data holding unit is output, and a pixel of a part of the plurality of image data to be output and a pixel of the image data stored in the boundary data storage unit are set as described above. When configuring a small pixel block, data to be output by replacing the remaining image data of the plurality of output image data except for the partial image data with the image data stored in the boundary data storage means. Output means;
Adding means for adding the values of the plurality of image data output from the data output means,
The image data processing device according to claim 9. Including frame image storage means for storing the image data of the original image,
The pixel block storage means reads and stores image data from the frame image storage means for each pixel block,
The above-mentioned boundary data storage means is adjacent to a next pixel block read from the above-mentioned pixel block storage means by the above-mentioned image data generation means when the image data is read from the above-mentioned frame image storage means in the above-mentioned pixel block storage means. The image data included in the predetermined boundary area of another pixel block to be read or another pixel block adjacent to the pixel block being read from the pixel block storage unit is read from the frame image storage unit and stored. ,
An image data processing device according to claim 8. 9. An image data encoding unit that performs an encoding process on image data in units of a predetermined number of image data blocks including image data blocks output from the selection unit corresponding to the pixel blocks. An image data processing device according to claim 1. An image data processing device for converting the number of image data,
Pixel block storage means for inputting and storing image data corresponding to each pixel of the original image for each first pixel block;
The image data stored in the pixel block storage unit is read out for each first pixel block, and the read out image data is synthesized for each first small pixel block smaller than the first pixel block. First image data generating means for generating one or more image data corresponding to the first small pixel block, and a pixel block read next from the pixel block storing means by the first image data generating means A different pixel block adjacent to the pixel block, or another pixel block adjacent to the pixel block being read from the pixel block storage means, a boundary data storage means for inputting and storing image data included in a predetermined boundary area; ,
The image data generated by the first image data generating means is divided into second small pixel blocks smaller than a second pixel block corresponding to the first pixel block on an image composed of the image data. Second image data generating means for generating one or more image data corresponding to each of the second small pixel blocks by combining
The first image data generation unit may include a part of the pixels of the first small pixel block, the pixels of another first pixel block adjacent to the first pixel block being read from the pixel block storage unit. When included in the boundary area, the image data of the partial pixels stored in the boundary data storage unit and the image data of the remaining pixels of the first small pixel block read from the pixel block storage unit To generate one or more image data corresponding to the first small pixel block,
The second image data generating means is configured so that a part of the pixels of the second small pixel block includes another second pixel adjacent to the second pixel block being generated by the first image data generating means. When included in the block, image data of a pixel equivalent to the partial pixel in the boundary area is read from the boundary data storage unit, and the read image data and the image data generated by the first image data generation unit are generated. Combining the image data of the remaining pixels of the second small pixel block with one another to generate one or more image data corresponding to the second small pixel block.
Image data processing device. When the conversion ratio of the number of image data is set to the first conversion ratio, the pixel block storage means inputs and stores the image data for each of the first pixel blocks, and the conversion ratio is the second conversion ratio. When the ratio is set, image data is input and stored for each third pixel block,
When the conversion ratio is set to the first conversion ratio, the first image data generation unit executes generation of image data corresponding to the first small pixel block, and sets the conversion ratio to the second conversion ratio. When the conversion ratio is set to, the image data of the third pixel block is read from the pixel block storage means and output as the image data of the second pixel block,
The boundary data storage means inputs and stores image data included in a predetermined boundary area according to the conversion ratio of the set number of image data.
The image data processing device according to claim 13. The first image data generating means includes:
The image data stored in the pixel block storage means is sequentially read, and each time the image data of one pixel is read, sampling of the read image data is performed once or plural times, and a series of sampled image data is reduced to a predetermined number. First data holding means for holding;
A series of image data held in the first data holding unit is synthesized, and a pixel of some image data in the series of image data of the synthesis source and a pixel of the image data stored in the boundary data storage unit are combined. Comprises the first small pixel block, a first data synthesizing means for synthesizing the partial image data and the stored image data,
When the conversion ratio of the number of image data is set to the first conversion ratio, the image data synthesized by the first data synthesis unit is selected and output, and the conversion ratio is set to the second conversion ratio. If set, includes first selecting means for selecting and outputting the image data sampled in the first data holding means,
The second image data generating means includes:
Each time the reading and sampling of the image data by the first data holding means is performed, the image data output from the first selecting means is sampled once or plural times, and a series of sampled image data is set to a predetermined value. Second data holding means for holding up to a number,
A series of image data held in the second data holding unit is synthesized with output image data of the first selection unit sampled subsequently, and a part of the series of image data of the synthesis source is synthesized. When the pixel of the image data and the pixel on the second pixel block equivalent to the pixel included in the boundary area constitute the second small pixel block, the pixel corresponds to the equivalent pixel. A second data combining unit that reads image data from the boundary data storage unit and combines the read image data and the partial image data.
The image data processing device according to claim 14. Including frame image storage means for storing the image data of the original image,
The pixel block storage means reads and stores image data from the frame image storage means for each pixel block according to the set conversion ratio of the number of image data,
The boundary data storage means includes a pixel which is read next from the pixel block storage means by the first image data generation means when image data is read from the frame image storage means in the pixel block storage means. The image data included in a predetermined boundary area according to the conversion ratio of a block or another pixel block adjacent to the pixel block being read from the pixel block storage unit is read from the frame image storage unit and stored. Do
The image data processing device according to claim 14. Image data to be subjected to image data encoding processing in units of a predetermined number of image data blocks including a plurality of image data blocks generated corresponding to the second pixel block in the second image data generation means 15. The image data processing device according to claim 14, further comprising an encoding unit. An image data processing method for converting the number of image data,
Image data of each pixel included in the first pixel block is synthesized for each first small pixel block smaller than the first pixel block, and one or a plurality of pixels corresponding to each of the first small pixel blocks are combined. A first image data generating step of sequentially executing a process of generating image data for the plurality of first pixel blocks included in the original image;
Of the image data generated in the first image data generation step, the image data included in the predetermined boundary area of the second pixel block corresponding to the first pixel block on the image composed of the image data A boundary data storage step of storing pixel image data in a boundary data storage device;
The image data generated in the first image data generation step is synthesized for each second small pixel block smaller than the second pixel block, and one image data corresponding to each of the second small pixel blocks is synthesized. Alternatively, a plurality of image data is generated, and some of the pixels of the second small pixel block are separated into another second pixel block adjacent to the second pixel block being generated in the first image data generation step. The image data of the partial pixels stored in the boundary data storage device and the rest of the second small pixel block generated in the first image data generating step. A second image data generating step of generating one or a plurality of image data corresponding to the second small pixel block by combining the image data of Image data processing method. When the conversion ratio of the number of image data is set to the first conversion ratio, in the first image data generation step, the generation of image data corresponding to the first small pixel block is performed, and the conversion ratio becomes When the second conversion ratio is set, in the first image data generation step, the same data as the image data of the third pixel block included in the original image is replaced with the image of the second pixel block. Sequentially generated as data,
An image data processing method according to claim 18. An image data processing method for converting the number of image data,
A pixel block that reads out image data of each pixel from a frame image storage device that stores image data of an original image for each predetermined pixel block according to a set conversion ratio of the number of image data, and stores the image data in a pixel block storage device A memory step;
The image data stored in the pixel block storage device is read, and the read image data is synthesized for each predetermined small pixel block corresponding to the conversion ratio, and one or more images are stored for each small pixel block. An image data generating step of generating data;
When reading of image data from the frame image storage device is performed in the pixel block storage step, another pixel block adjacent to a pixel block to be read next from the pixel block storage device in the image data generation step, Alternatively, boundary data to be read from the frame image storage device and stored in the boundary data storage device, the image data included in the predetermined boundary region of another pixel block adjacent to the pixel block being read from the pixel block storage device And a storage step.
In the image data generating step, when some pixels of the small pixel block are included in the boundary area of another pixel block adjacent to the pixel block being read from the pixel block storage device, the boundary data storage is performed. The image data of the partial pixels stored in the device is combined with the image data of the remaining pixels of the small pixel block read from the pixel block storage device, and one or Generate multiple image data,
Image data processing method. An image data processing method for converting the number of image data,
A pixel block storage step of reading image data of each pixel for each first pixel block from a frame image storage device storing image data of an original image, and storing the read image data in the pixel block storage device;
The image data stored in the pixel block storage device is read for each first pixel block, and the read image data is combined for each first small pixel block smaller than the first pixel block, and A first image data generating step of generating one or more image data corresponding to the first small pixel block;
In the case where image data is read from the frame image storage device in the pixel block storage step, another pixel adjacent to a pixel block next read from the pixel block storage device in the first image data generation step Image data included in a predetermined boundary area of a block or another pixel block adjacent to a pixel block being read from the pixel block storage device is read from the frame image storage device and stored in the boundary data storage device. A boundary data storage step;
The image data generated in the first image data generation step is divided into second small pixel blocks smaller than a second pixel block corresponding to the first pixel block on an image composed of the image data. A second image data generating step of generating one or a plurality of image data corresponding to each of the second small pixel blocks by combining
In the first image data generation step, a part of the pixels of the first small pixel block is a pixel of another first pixel block adjacent to the first pixel block being read from the pixel block storage device. When included in the boundary area, the image data of the partial pixels stored in the boundary data storage device and the image of the remaining pixels of the first small pixel block read from the pixel block storage device Combining the data with the data to generate one or more image data corresponding to the first small pixel block;
In the second image data generating step, a part of the pixels of the second small pixel block is replaced with another second pixel adjacent to the second pixel block being generated in the first image data generating step. When included in the block, image data of pixels equivalent to the partial pixels in the boundary area is read from the boundary data storage device, and the read image data and the image data generated in the first image data generation step are generated. Combining the image data of the remaining pixels of the second small pixel block with one another to generate one or more image data corresponding to the second small pixel block.
Image data processing method. When the conversion ratio of the number of image data is set to the first conversion ratio,
In the pixel block storage step, image data is read from the frame image storage device for each of the first pixel blocks and stored in the pixel block storage device,
In the first image data generation step, image data corresponding to the first small pixel block is generated,
When the conversion ratio is set to the second conversion ratio,
In the pixel block storage step, image data is read out for each third pixel block from the frame image storage device and stored in the pixel block storage device,
In the first image data generation step, the same data as the image data of the third pixel block read from the pixel block storage device is generated as the image data of the second pixel block,
In the boundary data storage step, image data included in a predetermined boundary area according to the conversion ratio of the number of set image data is stored in the boundary data storage device.
The image data processing method according to claim 21. Photographing means for photographing an image and generating image data corresponding to each pixel of the photographed image;
Frame image storage means for storing image data of the photographed image generated by the photographing means;
The image data of the photographed image stored in the frame image storage unit is read out for each predetermined pixel block corresponding to the set conversion ratio of the number of image data, and the number of image data is converted. Image data processing means for sequentially outputting blocks of image data to be processed,
Image data encoding means for performing encoding processing of image data in units of a predetermined number of image data blocks including image data blocks output from the image data processing device,
The image data processing means includes:
Pixel block storage means for reading and storing the image data of the captured image stored in the frame image storage means for each predetermined pixel block corresponding to the conversion ratio;
The image data stored in the pixel block storage unit is read, and the read image data is combined for each predetermined small pixel block to generate one or more image data for each small pixel block. Image data generating means;
When reading of image data from the frame image storage means is performed in the pixel block storage means, another pixel block adjacent to a pixel block next read from the pixel block storage means by the image data generation means, Or, boundary data storage means for reading and storing, from the frame image storage means, image data included in the predetermined boundary area of another pixel block adjacent to the pixel block being read from the pixel block storage means,
Among the plurality of image data generating means, having a selecting means for outputting image data of the image data generating means selected according to the conversion ratio,
The image data generating means stores the boundary data when some pixels of the small pixel block are included in the boundary area of another pixel block adjacent to the pixel block being read from the pixel block storage means. The image data of the partial pixels stored in the unit and the image data of the remaining pixels of the small pixel block read from the pixel block storage unit are combined, and one or a pixel corresponding to the small pixel block is combined. Generate multiple image data,
Camera system. Photographing means for photographing an image and generating image data corresponding to each pixel of the photographed image;
Frame image storage means for storing image data of the photographed image generated by the photographing means;
The image data of the photographed image stored in the frame image storage unit is read out for each predetermined pixel block corresponding to the set conversion ratio of the number of image data, and the number of image data is converted. Image data processing means for sequentially outputting blocks of image data to be processed,
Image data encoding means for performing encoding processing of image data in units of a predetermined number of image data blocks including image data blocks output from the image data processing device,
The image data processing means includes:
Pixel block storage means for reading and storing the image data of the captured image stored in the frame image storage means for each first pixel block;
The image data stored in the pixel block storage unit is read out for each first pixel block, and the read out image data is synthesized for each first small pixel block smaller than the first pixel block. First image data generating means for generating one or more image data corresponding to the first small pixel block, and reading of image data from the frame image storing means in the pixel block storing means The predetermined boundary between a pixel block to be read next from the pixel block storage unit by the first image data generation unit or another pixel block adjacent to a pixel block being read from the pixel block storage unit; The boundary data for reading and storing the image data included in the area from the frame image storage means. A storage means,
The image data generated by the first image data generating means is divided into second small pixel blocks smaller than a second pixel block corresponding to the first pixel block on an image composed of the image data. Second image data generating means for generating one or more image data corresponding to each of the second small pixel blocks by combining
The first image data generation unit may include a part of the pixels of the first small pixel block, the pixels of another first pixel block adjacent to the first pixel block being read from the pixel block storage unit. When included in the boundary area, the image data of the partial pixels stored in the boundary data storage unit and the image data of the remaining pixels of the first small pixel block read from the pixel block storage unit To generate one or more image data corresponding to the first small pixel block,
The second image data generating means is configured so that a part of the pixels of the second small pixel block includes another second pixel adjacent to the second pixel block being generated by the first image data generating means. When included in the block, image data of a pixel equivalent to the partial pixel in the boundary area is read from the boundary data storage unit, and the read image data and the image data generated by the first image data generation unit are generated. Combining the image data of the remaining pixels of the second small pixel block with one another to generate one or more image data corresponding to the second small pixel block.
Camera system. The pixel block storage unit is configured to output, for each of the first pixel blocks when the conversion ratio of the number of image data is set to the first conversion ratio, a second conversion ratio when the conversion ratio is set to the second conversion ratio. For each of the three pixel blocks, read and store the image data from the frame image storage means,
When the conversion ratio is set to the first conversion ratio, the first image data generation unit executes generation of image data corresponding to the first small pixel block, and sets the conversion ratio to the second conversion ratio. When the conversion ratio is set to, the image data of the third pixel block is read from the pixel block storage means and output as the image data of the second pixel block.
A camera system according to claim 24.

Images