JP2002237953A - Image data processing unit and its method, and camera system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image data processing unit whose circuit scale can be reduced compared with a conventional processing unit in the processing, where YCrCb ratio of image data received in units of 8×8 pixel blocks is converted, so as to attain high-speed processing, and to provide its method and a camera system. SOLUTION: A data storage section 21 stores image data included in a prescribed area of a pixel block. Furthermore, an image data generating section 22 gives a prescribed weight to image data, received by each pixel block by each prescribed small pixel block, and the resulting data are composited to create at least one image data. When a part of the small pixel blocks is included in the prescribed area of other pixel blocks adjacent to the pixel block of the received image data, the generating section 22 receives the part of the image data from the data storage section 21, and composites the image data of the small pixel blocks including the received image data, and at least one image data are generated.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention has an image data processing apparatus and method for synthesizing image data input for each predetermined pixel block and converting the number of image data, and the image data processing apparatus. According to the camera system, for example, in an interface section of a JPEG encoder / decoder in which a set of 8 × 8 image data is input / output as a processing unit, a process of converting a data ratio of a luminance signal and a color difference signal is performed. The present invention relates to an image data processing device and method, and a camera system having the image data processing device.

[0002]

2. Description of the Related Art Generally, the format of image data handled in digital video is a luminance signal Y that is smaller than that of an RGB format.
And the color difference signal (chroma signal) Cr and Cb (YCrCb format). This is because, in a natural image, the band of the chroma signal tends to be narrower than the luminance signal, so that the sampling frequency of the chroma signal can be made lower than that of the luminance signal, thereby reducing the amount of image data. The ratio of the sampling frequency of the luminance signal Y to the chroma signal Cr and the chroma signal Cb (hereinafter, referred to as
YCrCb ratio) is generally Y: Cr: Cb =
(4: 4: 4), (4: 2: 2), (4: 1: 1)
Type ratios are used. When playing back on a TV monitor with low resolution, YCrCb with a small amount of data
Even with the ratio (4: 1: 1), the deterioration of the image quality is not so large.

[0003] Such a digital image data recording / reproducing apparatus can usually cope with any of the above-mentioned general YCrCb ratios. However, in order to save the memory capacity used for processing the image data, the YCrCb ratio of the image data is set to (4: 2:
2) or (4: 1: 1) such that the ratio of the information amount is small, and all of the internal image processing units have the same YCrCb
In many cases, image data of a ratio is processed.

FIG. 10 is a diagram for explaining a configuration of an interface unit of a general digital video image processing apparatus. The image processing apparatus shown in FIG. 10 includes a YCrCb ratio converter 1, a YCrCb ratio converter 2, a frame memory 3,
It has a frame memory controller 4 and an image processing unit 5.

[0005] The YCrCb ratio conversion unit 1 is provided with a predetermined YCrCb ratio generated in an image pickup device including a CCD or the like.
The YCrCb format image data Sin having a ratio is input, and the input image data Sin is converted to a YCrCb ratio (4:
1: 1). The YCrCb ratio conversion unit 2 converts the image data having the YCrCb ratio (4: 1: 1) input from the frame memory controller 4 into a predetermined ratio corresponding to the output destination and outputs the converted data.

The frame memory 3 stores image data of a YCrCb ratio (4: 1: 1) processed by the image processing unit 5 in image frame units. Writing and reading of image data is performed by the frame memory controller 4
This is performed according to the control from. The frame memory controller 4 controls writing or reading of image data to or from the frame memory 3 in response to a request from the YCrCb ratio conversion unit 1, the YCrCb ratio conversion unit, and the image processing unit 3. The image processing unit 5 performs predetermined image processing on the image data having the YCrCb ratio (4: 1: 1), such as image data filtering and aspect ratio conversion.

A predetermined YCr generated in an imaging device or the like
The image data having the Cb ratio is converted into image data having a YCrCb ratio (4: 1: 1) by the YCrCb ratio conversion unit 1, and then predetermined image processing is performed by the image processing unit 5. In the course of this image processing, the image data is stored in the frame memory 3 under the control of the frame memory controller 4, and is read from the frame memory 3 in accordance with the processing. YCr that has been processed by the image processing unit 5
The image data of the Cb ratio (4: 1: 1) is converted by the YCrCb ratio conversion unit 2 into a YCrCb ratio according to the specification of the output destination.

[0008] In digital video or the like that handles moving images, generally, as in the image processing apparatus shown in FIG.
Image data with an rCb ratio (4: 1: 1) is processed inside the system. In a digital still camera having a relatively small data amount, a YCrCb ratio (4: 2:
The image data of 2) is often processed inside the system. However, even in digital video, still image input / output is often performed at a YCrCb ratio (4: 2: 2). As described above, in order to use image data generated by various devices having different YCrCb ratios of image data to be processed, the conversion technology of the YCrCb ratio is an indispensable technology for a digital video recording / reproducing device. I have.

[0009]

However, JPEG (J
In the image data compression method by Oint Photographic Exparts Group), an MCU (Minimum Coded Unit), which is a set of pixel blocks of 8 × 8 pixels = 64 pixels, is treated as a unit of data processing in image compression or expansion. Have been done.

FIGS. 11, 12 and 13 show the YCrCb ratio (4: 4: 4) and the YCrCb ratio (4: 2:
2) and one MCU at YCrCb ratio (4: 1: 1).

As shown in FIG. 11, the YCrCb ratio (4:
In the 4: 4) MCU, one luminance signal Y corresponds to one chroma signal (Cr, Cb). On the other hand, as shown in FIG. 12, the YCrCb ratio (4:
In the 2: 2) MCU, one chroma signal (Cr, Cb) is provided for two luminance signals Y arranged on a horizontal line.
Is supported. Further, as shown in FIG.
In the MCU having the ratio (4: 1: 1), one chroma signal (C) is provided for four luminance signals Y arranged on a horizontal line.
r, Cb) correspond.

When such conversion of the YCrCb ratio is performed, a process of interpolating a chroma signal (chroma interpolation) is indispensable. However, in order to perform chroma interpolation on data input / output for each MCU, it is necessary to perform interpolation processing using chroma signals divided into two adjacent 8 × 8 pixel blocks. Therefore, in the conventional method,
Image data divided for each MCU is temporarily stored in a line memory for one horizontal line, and chroma interpolation processing is performed using the stored image data.

FIG. 14 shows two chroma signal 8 × 8 pixel blocks in an MCU having a YCrCb ratio (4: 2: 2) and one chroma signal 8 × 8 pixel in an MCU having a YCrCb ratio (4: 1: 1). 9 shows an interpolation process for converting into a block. Pixel block B1 and pixel block B2
Indicates a chroma signal 8 × 8 pixel block before conversion, and a pixel block B3 indicates a chroma signal 8 × 8 pixel block after conversion.

The YCrCb ratio (4: 2:
When chroma interpolation is performed from the image data of 2) to image data having a small data amount and a YCrCb ratio (4: 1: 1),
Usually, a plurality of (three in the example of FIG. 13) image data adjacent in the horizontal line direction are synthesized to generate one image data. Region L1 arranged in the vertical direction in FIG.
The pixels L3 to L3 are located at the boundary between horizontally adjacent 8 × 8 pixel blocks, and are combined with image data belonging to different 8 × 8 pixel blocks in chroma interpolation.

FIG. 15 shows an example of the configuration of a conventional JPEG interface circuit for converting a YCrCb ratio (4: 2: 2) into a YCrCb ratio (4: 1: 1).
The JPEG interface circuit shown in FIG.
Cb ratio conversion circuit 1, frame memory 3, MCU memory 6
And a line memory 7, and the same reference numerals in FIGS. 10 and 15 indicate the same components.

The MCU memory 6 is divided for each MCU from a circuit block (hereinafter, referred to as a JPEG circuit) for performing image compression or decompression in the JPEG system, and is sequentially input as Y.
This storage device stores image data having a CrCb ratio (4: 2: 2) for each MCU. Usually, it has a two-bank configuration in which a memory for writing and a memory for reading are separately provided. By alternately switching between the memory for writing and the memory for reading, the writing of image data from the JPEG circuit and the frame Reading of image data from the memory 3 is performed simultaneously. The line memory 7 reads out and stores one horizontal line of image data from one frame of the image data stored in the frame memory 3.

The MCU is expanded in the JPEG circuit and
The image data of the YCrCb ratio (4: 2: 2) inputted every time is stored in the MCU memory 6 for each MCU. Then, the image data divided into 8 × 8 pixel blocks is rearranged into line data, and
Is written to. This process is repeated for all MCUs of one frame of image data, so that one frame of image data is stored in the frame memory 3.

Next, one line of image data is sequentially read from the frame memory 3 and stored in the line memory 7. The image data stored in the line memory 7 is
Chroma interpolation in the YCrCb ratio converter 1
From the rCb ratio (4: 2: 2) to the YCrCb ratio (4: 1:
1) and stored in the line memory 7 again. 1
After the conversion for the line is completed, the converted image data is written back to the frame memory again. This process is repeated for all lines to obtain a YCrCb ratio (4:
One frame of image data converted to 1: 1) is stored in the frame memory 3.

In such a processing mode, since the chroma interpolation is performed on the data arranged in one line, an interpolation process for synthesizing a wider range of data becomes possible, and an improvement in image quality can be expected. However, since the image data once stored in the frame memory 3 is once read into the line memory 7 and then written back to the frame memory again, there is a problem that the number of writing processes to the memory increases and the processing speed decreases. . Further, the line memory 7 for chroma interpolation must be provided, which causes a problem of increasing the circuit scale.

From the YCrCb ratio (4: 1: 1),
The conventional chroma interpolation method for expanding image data to a CrCb ratio (4: 2: 2) also has the following problems.

FIG. 16 shows one chroma signal 8 × 8 pixel block in an MCU with a YCrCb ratio (4: 1: 1) and two chroma signal 8 × 8 pixels in an MCU with a YCrCb ratio (4: 2: 2). 9 shows an interpolation process for converting into a block. Pixel block B4 and pixel block B5
Is a 4 × 8 pixel block corresponding to the converted chroma signal 8 × 8 pixel block, and is a half-divided 8 × 8 pixel block before conversion. The pixel blocks B6 and B7 correspond to the pixel blocks after the conversion of the pixel blocks B4 and B5, respectively.

The YCrCb ratio (4: 1:
When chroma interpolation is performed from the image data of 1) to image data having a large data amount and a YCrCb ratio (4: 2: 2),
A plurality of (two in the example of FIG. 15) image data adjacent in the horizontal line direction are synthesized, and the synthesized image data is interpolated between the original image data. The pixels in the regions L4 to L6 arranged in the vertical direction in FIG. 15 are at the boundaries between the 4 × 8 pixel blocks adjacent in the horizontal direction, and are combined with image data belonging to different 4 × 8 pixel blocks in chroma interpolation.

FIG. 17 shows an example of the configuration of a conventional JPEG interface circuit for converting a YCrCb ratio (4: 1: 1) to a YCrCb ratio (4: 2: 2).
The JPEG interface circuit shown in FIG.
It has a Cb ratio conversion circuit 2, a frame memory 3 and an MCU memory 6, and the same reference numerals in FIGS. 10, 15 and 17 denote the same components.

In the example shown in FIG. 17, one frame of image data is already stored in the frame memory 3. From the image data stored in the frame memory 3, 1MC
U image data is sequentially read out and temporarily stored in the MCU memory 6. 1 MCU stored in MCU memory 6
Image data for each 4 × 8 pixel block is read out to the YCrCb ratio converter 2 from the image data of
From the rCb ratio (4: 1: 1) to the YCrCb ratio (4: 2:
The conversion to 2) is performed. As a result, the YCrCb ratio converter 2 sequentially outputs image data for each 8 × 8 pixel block. At the time of this conversion, if image data that is not included in the image data of the 4 × 8 pixel block read from the MCU memory 6 is used for synthesis, the image data is directly read from the frame memory 3 and synthesized. Used for

In general, a storage device such as a DRAM or an SRAM is often used in a large amount for the frame memory 3 that needs to store a large amount of data. Is complicated and there is a problem that the processing speed is reduced. Further, since a circuit for accessing the frame memory 3 is added for the YCrCb ratio converter 2, there is a problem that the circuit scale is increased.

The present invention has been made in view of the above circumstances, and an object of the present invention is to combine image data to be input for each predetermined pixel block and convert the number of image data by a conventional method. An object of the present invention is to provide an image data processing apparatus and method capable of reducing the circuit scale and increasing the processing speed as compared with the above, and a camera system having the image data processing apparatus.

[0027]

In order to achieve the above object, an image data processing apparatus according to the present invention synthesizes image data input for each predetermined pixel block for each predetermined small pixel block, and outputs the image data. An image data processing device for converting the number of data, comprising: data holding means for holding image data included in a predetermined area of the pixel block; and image data input for each of the predetermined pixel blocks, Are given weights for each and synthesized.
At least one image data is generated from the composite data, and when a part of the image data of the small pixel block is included in the predetermined region of another pixel block adjacent to the pixel block of the input image data, Partial image data of the pixel block is input from the data holding unit,
Image data generating means for giving the predetermined weight to the image data of the small pixel block including the input image data and synthesizing the image data to generate at least one image data from the synthesized data.

The data holding means holds the image data of the predetermined area among the image data input to the image data generating means.

The image processing apparatus further includes a pixel block holding unit for holding the image data input for each pixel block and outputting the image data of the pixel block already held to the image data generating unit. ,
The image data included in the predetermined area among the image data input to the pixel block holding unit is held.

According to the image data processing device of the present invention, the data holding means holds the image data contained in the predetermined area of the pixel block. In the image data generating means, the image data input for each of the predetermined pixel blocks is synthesized by giving a predetermined weight to each of the small pixel blocks, and at least one image data is generated from the synthesized data. Is done. When a part of the image data of the small pixel block is included in the predetermined area of another pixel block adjacent to the pixel block of the input image data, the image data of a part of the small pixel block is stored in the data holding unit. Is input to the image data generating means, the image data of the small pixel block including the input image data is given the predetermined weight, and is synthesized.
One image data is generated.

In the image data processing apparatus according to the first aspect of the present invention, the data holding means holds the image data of the predetermined area among the image data input to the image data generating means.

In the image data processing apparatus according to the second aspect of the present invention, the pixel block holding means holds the image data input for each pixel block and the image data of the pixel block already held. Is output to the image data generating means. The data holding unit holds image data included in the predetermined area among image data input to the pixel block holding unit.

An image data processing method according to the present invention is a method for processing image data for combining image data input for each predetermined pixel block for each predetermined small pixel block and converting the number of image data. A data holding step of holding image data included in a predetermined area of the pixel block; and image data input for each of the predetermined pixel blocks are synthesized by giving a predetermined weight to each of the small pixel blocks, and And generating at least one image data from the image data, and when the partial image data of the small pixel block is included in the predetermined area of another pixel block adjacent to the pixel block of the input image data, The held partial image data is input, and the small pixel block including the input image data is input into the small pixel block. It was synthesized by applying a constant weighting, and an image data generation step of generating at least one image data from the combined data.

The data holding step holds the image data of the predetermined area among the image data input in the image data generating step.

In addition, the method includes a pixel block holding step of holding the image data input for each pixel block and outputting the image data of the pixel block already held in the image data generating step. And holding the image data included in the predetermined area of the pixel block among the image data input in the pixel block holding step.

According to the image data processing method of the present invention, in the data holding step, image data included in a predetermined area of the pixel block is held. In the image data generating means, the image data input for each of the predetermined pixel blocks is synthesized by giving a predetermined weight to each of the small pixel blocks, and at least one image data is generated from the synthesized data. Is done.
When a part of the image data of the small pixel block is included in the predetermined area of another pixel block adjacent to the pixel block of the input image data, the part of the image data held in the data holding step is Entered,
The predetermined weights are given to the small pixel blocks including the input image data to be synthesized, and at least one image data is generated from the synthesized data.

In the image data processing method according to the first aspect of the present invention, of the image data input in the image data generating step, the image data of the predetermined area is held in the data holding step.

In the image data processing method according to the second aspect of the present invention, in the pixel block holding step, the image data input for each pixel block is held and the image data of the pixel block already held is stored. Are output in the image data generating step. In the data holding step, of the image data input in the pixel block holding step, image data included in the predetermined area of the pixel block is held.

A camera system according to the present invention includes a photographing means for photographing a desired image and generating image data corresponding to each pixel of the photographed image, and a predetermined processing for the image data generated by the photographing means. Processing means for inputting / outputting the processed image data for each predetermined pixel block, and image data input for each predetermined pixel block from the processing means for each predetermined small pixel block, An image data processing device for converting the number of data, wherein the image data processing device inputs data for each upper pixel block, and a data holding unit for holding image data included in a predetermined area of the pixel block The image data is synthesized by giving a predetermined weight to each of the small pixel blocks, and at least one image data is generated from the synthesized data. When a part of the image data of the pixel block is included in the predetermined area of another pixel block adjacent to the pixel block of the input image data, the part of the image data is input from the data holding unit, Image data generating means for giving the predetermined weight to the image data of the small pixel block including the obtained image data and synthesizing the image data to generate at least one image data from the synthesized data.

[0040]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS <First Embodiment> First, a first embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a schematic configuration diagram for explaining an image data processing device according to a first embodiment of the present invention. The image data processing device illustrated in FIG. 1 includes a data number conversion unit 20 and a frame memory 30. The data number conversion unit 20 includes a data holding unit 21, an image data generation unit 22, and a pixel block memory 23. The data holding unit 21 is an embodiment of the data holding unit of the present invention. The image data generator 22 is an embodiment of the image data generator of the present invention. The pixel block memory 23
5 is an embodiment of a pixel block holding unit of the present invention.

First, each component of the image data processing apparatus shown in FIG. 1 will be described.

The frame memory 30 includes a data number converter 2
A storage device that stores the image data whose data number has been converted at 0 in units of frames, and outputs the stored image data to another block that performs image processing. For example, the YCrCb ratio (4: 1:
The image data converted in 1) is output. For the frame memory 30, for example, a high-speed, large-capacity storage device such as an SDRAM (Synchronous DRAM) is used.

The data number conversion unit 20 is a block for converting the data number of the sequentially input image data for each predetermined pixel block which divides an image of one frame. In the example of FIG. 1, image data C422-i input from a JPEG circuit (not shown) for each 8 × 8 pixel block.
n is converted from the YCrCb ratio (4: 2: 2) to the YCrCb ratio (4: 1: 1), and the converted image data is stored in the frame memory 30 for each 8 × 8 pixel block.
To be stored.

The image data generating unit 22 combines image data input for each predetermined pixel block by giving a predetermined weight to each predetermined small pixel block, and generates one or a plurality of image data. In the example of FIG. 1, image data for each 8 × 8 pixel block input from a JPEG circuit (not shown) is synthesized for each predetermined small pixel block, the number of data in the horizontal direction is reduced by half, and two chroma signals are output. One chroma signal 8 for an 8 × 8 pixel block
A × 8 pixel block is output.

For example, when a block of three adjacent pixels arranged on a horizontal line while overlapping one pixel at a time is set as a small pixel block, three pieces of image data of one small pixel block are When the image data is converted into one image data by synthesis, the number of image data is reduced by half. This corresponds to the conversion from the YCrCb ratio (4: 2: 2) shown in FIG. 14 to the YCrCb ratio (4: 1: 1).

Further, for example, when a block of two adjacent pixels arranged on a horizontal line while overlapping each other by one pixel is set as a small pixel block, two image data of one small pixel block are set. Is generated, the number of image data is doubled. This corresponds to the YCrCb shown in FIG.
This corresponds to conversion from the ratio (4: 1: 1) to the YCrCb ratio (4: 2: 2). In the example of FIG. 16, two types of synthesis ratios (50% to 50%, 100
% To 0%), two image data are generated.

However, all the image data in the small pixel block is included in a predetermined pixel block (in the example of FIG. 1, one 8 × 8 pixel block) input to the image data generation unit 22. There is a case where a part is included in another adjacent pixel block. For example, in FIG. 14, the image data included in the area L2 of the pixel block B1 is combined with the image data at the left end of the pixel block B2 adjacent to the right.

As described above, the currently input pixel block does not include a part of the image data of the small pixel block to be synthesized, and is located in the boundary area of another pixel block adjacent to this pixel block. When the image data is included, the image data generation unit 22 inputs the corresponding image data held in the data holding unit 21 in advance, and uses the input image data for synthesis.

The data holding unit 21 includes the image data generation unit 2
2 holds image data included in a predetermined boundary area of the pixel block among image data input for each predetermined pixel block. This boundary area is an area in which a small pixel block is set to extend between adjacent pixel blocks, and is adjacent to another pixel including image data necessary for performing interpolation processing of one pixel block. Block area. For example, regions L1 to L3 in FIG.
Also, the regions L4 to L6 in FIG. 16 correspond to this boundary region.

However, in the data number converter 20 shown in the example of FIG. 1, since the image data C422-in common to the image data generator 22 and the data holder 21 is input, for example, the area L4 in FIG. As in L6, image data of a pixel block that has not yet been input to the image data generation unit 22 cannot be held. That is, the data holding unit 21 shown in the example of FIG.
2, image data of a predetermined boundary area of the pixel block input is held.

The pixel block memory 23 stores the YCrCb ratio (4: 1:
The image data of 1) is stored for each predetermined pixel block. The memory 23 may have a two-bank configuration in which, for example, a memory for writing and a memory for reading are separately provided. With the configuration of two banks, the memory for writing and the memory for reading are alternately switched, and the image data generating unit 22 is switched.
And writing of the image data to the frame memory 3 can be performed simultaneously.

Here, the data number converter 20 shown in FIG.
Data storage unit 21 and image data generation unit 22
The configuration example will be described in detail. FIG. 2 shows a configuration example of the data holding unit 21 and the image data generation unit 22 shown in FIG. The data holding unit 21 illustrated in FIG. 2 includes a register 211, a register 212, and a selector 213. 2 includes a latch unit 221, a latch unit 222, a coefficient multiplying unit 223 to a coefficient multiplying unit 225, an adding unit 226, and a selector 227.
Having.

The register 211 stores the input image data C422 when the signal xcb-en is enabled.
-In is read and held. The signal xcb-en is a signal that is enabled in a predetermined boundary area in the pixel block of the chroma signal Cb, for example, as shown in FIG.
Are enabled at the timing when the image data C422-in of the regions L1 to L3 are input. In this case, the register 211 holds the image data of the boundary area (right end) in the 8 × 8 pixel block of the chroma signal Cb.

Also, in the example of FIG.
Has a simple serial input shift register configuration. Every time image data is held by an input, the already held image data is sequentially shifted and output to the selector 213. The image data output from the register 211 is used when the image data is synthesized at the left end of the next 8 × 8 pixel block.

When the signal xcr-en is in the enable state, the register 212 receives the input image data C422.
-In is read and held. The signal xcr-en is a signal that is enabled in a predetermined boundary region in the pixel block of the chroma signal Cr, for example, as shown in FIG.
Are enabled at the timing when the image data C422-in of the regions L1 to L3 are input. In this case, the register 212 holds the image data of the boundary area (right end) in the 8 × 8 pixel block of the chroma signal Cr.

The register 212 also has a simple serial input shift register configuration similar to the register 211. Every time image data is held by an input, the already held image data is sequentially shifted. Selector 21
3 is output. Then, the image data output from the register 212 is used for synthesizing the image data at the left end of the next 8 × 8 pixel block.

The selector 213 outputs the selection signal area-c.
One of the register 211 and the register 212 is selected according to r, and the image data of the selected register is output to the selector 227. When the image data C422-in is the chroma signal Cb, the register 211 is selected by the selection signal area-cr, and when the image data C422-in is the chroma signal Cr, the register 212 is selected.

The latch section 221 has a clock signal CLK1.
, And outputs the held data to the latch unit 222 and the coefficient multiplication unit 224. The latch unit 222 holds the image data input from the latch unit 221 in synchronization with the clock signal CLK1, and outputs the held data to the selector 227.

The selector 227 outputs the selection signal blk-le
According to ft, either the latch unit 222 or the selector 213 is selected, and image data from the selected block is output. In a predetermined boundary region (for example, regions L1 to L3 in FIG. 14) in the 8 × 8 pixel block, the selector 213 is selected by the selection signal blk-left, and the latch unit 222 is selected in other regions.

The coefficient multiplying unit 223 outputs the image data C422
−in is multiplied by a predetermined weighting coefficient (１ ／ in the example of FIG. 2), and the result of the multiplication is output to the adding section 226. The coefficient multiplication unit 224 multiplies the image data held by the latch unit 221 by a predetermined weighting coefficient (1/2 in the example of FIG. 2), and outputs the multiplication result to the addition unit 226. The coefficient multiplication unit 225
The image data from the selector 227 is multiplied by a predetermined weighting coefficient (1/4 in the example of FIG. 2), and the multiplication result is added to the adder 22.
6 is output. Note that, in the example of FIG.
The coefficient multiplied by the coefficient multiplier 225 is 1/2 or 1 /
Since the coefficients are powers of 2 such as 4, these coefficient multipliers 223 to 225 can be realized by a simple bit shift circuit, and a complicated multiplier is not required.

The adder 226 adds the data output from the coefficient multipliers 223 to 225 and outputs image data C411-out obtained as an addition result to the pixel block memory 23.

Next, the operation of the image data processing apparatus shown in FIGS. 1 and 2 having the above configuration will be described.

In the prior art shown in FIG. 16, once all data for one frame is once written to the frame memory,
Although the chroma interpolation is performed line by line, the image data processing device shown in FIG. 1 uses image data C422-in for each 8 × 8 pixel block sequentially input from the JPEG circuit.
Is directly subjected to YCrCb ratio conversion. Then, the converted image data C411-out is temporarily stored in the pixel block memory 23 and then written into the frame memory 30. The image data of the chroma signal required for the chroma interpolation in the YCrCb ratio conversion is held in the data holding unit 21 at the same time as the chroma interpolation processing, and chroma interpolation is performed on the image data of the next 8 × 8 pixel block to be read. At this time, the image data held by the data holding unit 21 is used.

First, the operation of holding image data in the data holding unit 21 will be described. Image data C42 for each 8 × 8 pixel block sequentially input from the JPEG circuit
In the 2-in, the image data included in the predetermined boundary area of the pixel block includes the chroma signal Cr and the chroma signal C.
The data is stored in the register 211 or the register 212 for each b. This boundary area is, for example, the area L in FIG.
This is the area at the right end of the 8 × 8 pixel block indicated by 1 to L3. When the image data C422-in is the chroma signal Cb, the register 211 is selected by the selector 213. When the image data C422-in is the chroma signal Cr, the image data of the boundary area held in these registers is output to the selector 227. Is done.

The register 211 and the register 212
Has a shift register configuration as described above, and the held image data of the boundary area is sequentially shifted in the shift register and output from the selector 213. For example, in FIG. 14, when image data is sequentially input from left to right on a horizontal line of one 8 × 8 pixel block and from top to bottom on a vertical line, the image data at the left end of the current pixel block is At the time of entry,
The right end image data (for example, image data of the area L1) of the previously input 8 × 8 pixel block adjacent to this image data is output from the register 211 or the register 212. The rightmost image data is input to the image data generation unit 22 via the selector 227, and is used for synthesizing the image data. Thereafter, the image data is sequentially input from left to right on the horizontal line, and the right end image data (image data of the area L2) of the current 8 × 8 pixel block is input as image data C422-in. The signal xcb-en or the signal xcr-en in synchronization with the input
Is enabled, and the rightmost image data is held in the register 211 or the register 212. At the same time, the image data (region L
1 is deleted from the register. As described above, the rightmost image data of the 8 × 8 pixel block is sequentially input to the registers 211 and 212, and is used for synthesizing the image data and then deleted from the registers.

Next, the operation of generating image data in the image data generating section 22 will be described. The latch unit 221 and the latch unit 222 constitute a shift register that holds image data up to two clocks before the image data C422-in input in synchronization with the clock signal CLK1. CLK1
, Three adjacent image data are sequentially output. Normally, the selector 227 selects the latch unit 222, and the three adjacent image data are added to the coefficient multiplying unit 2
23 to the weighting coefficient of the coefficient multiplying unit 225, and the image data C411-out synthesized by the adding unit 226 is generated.

The image data two clocks before the image data C422-in (that is, the image data S25) is 8 × 8.
When the image data is the rightmost image data of the pixel block, the rightmost image data is on the upper horizontal line with respect to the other two image data, and is not included in the adjacent small pixel block of three pixels. On the other hand, the selector 2 of the data holding unit 21
The image data at the right end of the adjacent 8 × 8 pixel block output from 13 is included in this small pixel block.
At this timing, the signal blk-left is switched, the data holding unit 21 is selected by the selector 227, and the rightmost image data of the 8 × 8 pixel block on the left and held by the data holding unit 21 and the other two image data Are combined to generate image data C411-out.

FIG. 3 shows the YCrCb ratio (4: 4) of the data holding unit 21 and the image data generating unit 22 shown in FIG.
FIG. 4 is a diagram for explaining conversion from 2: 2) to a YCrCb ratio (4: 1: 1). In FIG. 3, image data C10
The image data C17 indicates image data for one horizontal line of an 8 × 8 pixel block of a chroma signal having a YCrCb ratio (4: 2: 2), and the image data C07 corresponds to the horizontal line of the image data C10 to image data C17. On the other hand, the image data at the right end of the horizontal line one level higher is shown. The image data C'10
~ Image data C'13 are image data C10 ~ image data C
Reference numeral 17 denotes image data after conversion into the YCrCb ratio (4: 1: 1).

In FIG. 3, the image data C10 to C17 are sequentially input as image data C422-in from the left. When the image data C11 is input, the image data C07 two clocks earlier is on a different horizontal line from the image data C10 and the image data C11, and is not adjacent to the two subsequent image data. Therefore, if the image data C07, the image data C10, and the image data C11 are combined as the same small pixel block, image data that are not completely adjacent to each other will be combined, and appropriate interpolation processing will not be performed. Therefore, in the image data generation unit 22 shown in FIG. 2, the signal blk-left is switched at the timing when the image data C11 is input, and the image data held in the data holding unit 21 is compared with the image data C10 and the image data C11. Are combined to generate image data C411-out. As mentioned above,
The data holding unit 21 holds the image data at the right end of the previously input 8 × 8 pixel block and the image data adjacent to the left of the image data C10 on the image. An appropriate interpolation process is performed by combining the data C10 and the image data C11.

FIG. 4 is a diagram showing the timing of each signal in the data holding unit 21 and the image data generation unit 22 shown in FIG. Note that the reference numerals (C07, C10 to C17) attached to the lower side of FIG. 4B and the reference numerals (C'10 to C'17) attached to the lower side of FIG. It corresponds to.

The image data C422-in (FIG. 4B)
The data is sequentially input to the data number converter 20 in synchronization with the clock signal CLK1 (FIG. 4A). In FIG. 4, eight image data (C10 to C17) correspond to image data for one horizontal line of one 8.times.8 pixel block.

The selection signal area-cr (FIG. 4c) becomes a high level when the input image signal is the chroma signal Cr and selects the register 212, and becomes a low level when the input image signal is the chroma signal Cb and the register 211 becomes the low level. Is a signal for selecting. FIG. 4 shows only an example in which the input image signal is the chroma signal Cr, so that the selection signal area-cr is constant at a high level.

The image data S22 (FIG. 4d) is data in which the image data C422-in is weighted with a weighting factor of "1/4". Further, the image data S23 (FIG. 4)
e) is data in which the image data input one clock before the image data C422-in is weighted by the weighting factor '1/2', and is image data S24 (FIG. 4f).
Is data in which image data input two clocks before the image data C422-in is weighted by the weighting factor '1/4'. This data S22 to data S24
Are combined in the adder 226 to generate image data C411-out (FIG. 4j).

However, the image data C411-out generated in synchronization with the clock signal CLK1 is
422-in, the image data C411-
out needs to be halved. In the timing chart shown in FIG. 4, a signal xwe (FIG. 4k) obtained by dividing the frequency of the clock signal CLK1 by half is supplied to the pixel block memory 2
3 is used as a data write enable signal. Since the image data C411-out is written into the pixel block memory 23 at the timing when the signal xwe becomes low level, the image data C411-o is output.
ut is decimated in half.

The selection signal blk-ref of the selector 227
t (FIG. 4G) is a signal that selects the latch unit 222 at the low level and selects the selector 213 of the data holding unit 21 at the high level. As shown in FIG. 4, at a timing when the horizontal line of the image data two clocks before the image data C422-in and the horizontal line of the two subsequent image data are different, the selection signal blk-left goes high. For example, image data C07 two clocks before image data C11 input as image data C422-in.
And the selection signal blk-left is at the high level at the timing T11 when the horizontal lines of the subsequent two image data C10 and the image data C11 are different. At this timing T11, the selector 21 of the data holding unit 21
3 is a data value (18) in which image data S21 (60h) output from
h), data S22 (12h) and data S23
(22h) and the image data C411-ou
t (4Ch) has been generated.

The signal xcr-en (FIG. 4h) is
12 is a write enable signal for 8 × 8
At the timing when the rightmost image data of the horizontal line of the pixel block is input, the level becomes low, and the input image data C422-in is held in the register 212. For example, in FIG. 4, at timing T17 when the image data signal C17 at the right end of the horizontal line of the 8 × 8 pixel block is input, the signal xcr-en becomes low level, and the image data signal C17 is held in the register 212. In addition, the output value of the register 212 is updated accordingly, and the image data S21 (FIG. 4i) from the selector 213 receiving the update is changed from the value '60h' to the value '70'.

FIG. 4 shows the timing in the case of the chroma signal Cr.
The same applies to the case of b. That is, the selection signal area-
By setting cr to low level and replacing the selection signal xcr-en with the selection signal xcb-en, the above description can be applied to the case of the chroma signal Cb.

The signal xcb-en and the signal x
cr-en, selection signal blk-left, selection signal ar
Each control signal such as ea-cr has a certain timing in one MCU including the chroma signal Cr and the chroma signal Cb. Therefore, for example, a counter which is initialized at the beginning of one MCU of an input pixel block and counts with input of image data is prepared, and each control signal can be easily generated by decoding the count value of the counter. Can be.

As described above, according to the image data processing apparatus shown in FIGS. 1 and 2, for example, a predetermined pixel block, such as an 8 × 8 pixel block of image data input / output to / from a JPEG circuit, is used. When a predetermined number of data conversions (for example, YCrCb ratio conversion) are performed on the input image data for each time, the input image data is stored in a frame memory for one frame as in the conventional method. There is no need to read the image data for each line into the line memory and perform the interpolation processing. The image data necessary for the interpolation is appropriately held in the data holding unit 21 and used for the synthesis, so that the data is input for each pixel block. The number of data can be directly converted for the image data. As a result, the process of reading and writing image data from and to the frame memory and line memory is omitted,
Data processing speed can be increased. Further, it is necessary for the conventional line memory to hold several hundreds of image data for one horizontal line, but the data holding unit 21 of FIG. 2 stores several (for example, one 8 × 8 pixel block). Since only eight (8) image data need be stored, the circuit scale can be significantly reduced.

<Second Embodiment> Next, a second embodiment of the present invention will be described with reference to FIGS. The difference between the first embodiment and the second embodiment is that in the first embodiment, the data holding unit 21 holds a part of the image data already input to the image data generation unit 22. On the other hand, in the second embodiment, the image data before being input to the image data generation unit can be stored in the data storage unit 41 first.

FIG. 5 is a schematic configuration diagram for explaining an image data processing apparatus according to the second embodiment of the present invention.
The image data processing device shown in FIG.
And a frame memory 50. The data number conversion unit 40 includes a data holding unit 41, an image data generation unit 42
And a pixel block memory 43. Note that the data holding unit 41 is an embodiment of the data holding unit of the present invention. The image data generator 42 is an embodiment of the image data generator of the present invention. Pixel block memory 43
Is an embodiment of the pixel block holding means of the present invention.

First, each component of the image data processing apparatus shown in FIG. 5 will be described.

The frame memory 50 is a storage device for storing image data in units of frames. The frame memory 50 outputs the stored image data to the pixel block memory 43 in units of predetermined pixel blocks, and stores a part of the image data. The data is output to the data holding unit 41. In the frame memory 50,
For example, a high-speed, large-capacity storage device such as an SDRAM is used.

The data number conversion unit 40 includes a frame memory 5
This block sequentially reads out image data for each predetermined pixel block from one frame of image data stored in 0 and converts the number of image data. In the example of FIG. 5, the YCrCb ratio of the image data C411-in read for each predetermined pixel block (for example, a 4 × 8 pixel block) from the frame memory 50 is changed from the YCrCb ratio (4: 1: 1) to the YCrCb ratio ( 4: 2: 2), and outputs the converted image data to a JPEG circuit (not shown) for each 8 × 8 pixel block.

The pixel block memory 43 reads out the image data stored in the frame memory 50 by a predetermined pixel block and stores it. In the example of FIG. 5, the image data of the YCrCb ratio (4: 1: 1) stored in the frame memory 50 is read and stored for each MCU. The pixel block memory 43 may have a two-bank configuration in which, for example, a memory for writing and a memory for reading are separately provided.
In this case, the memory for writing and the memory for reading are alternately switched, so that the frame memory 50 is switched.
The image data from the image data generation unit 42
And reading of image data from the data holding unit 41 is performed simultaneously.

The image data generating unit 42 combines the image data read out from the pixel block memory 43 for each predetermined pixel block by giving a predetermined weight to each predetermined small pixel block, and synthesizes one or more image data. Generate In the example of FIG. 5, 4 × 8
The image data read out for each pixel block is combined for each predetermined small pixel block, the number of data in the horizontal direction is doubled, and one 4 × 8 pixel block has one image data.
Two 8 × 8 pixel blocks are generated.

For example, when a block of three adjacent pixels arranged on a horizontal line while overlapping one pixel at a time is set as a small pixel block, three pieces of image data of one small pixel block are When the image data is converted into one image data by synthesis, the number of image data is reduced by half. This corresponds to the conversion from the YCrCb ratio (4: 2: 2) shown in FIG. 14 to the YCrCb ratio (4: 1: 1). Further, for example, when a block of two adjacent pixels arranged on a horizontal line while overlapping each other by one pixel is set as a small pixel block, the two image data of one small pixel block are combined. When two image data are generated, the number of image data is doubled. This corresponds to the YCrC shown in FIG.
This corresponds to conversion from the b ratio (4: 1: 1) to the YCrCb ratio (4: 2: 2).

However, all the image data in the small pixel block is stored in a predetermined pixel block input to the image data generating section 42 (in the example of FIG. 5, within a 4 × 8 pixel block).
Is not always included in the pixel block, and a part may be included in another adjacent pixel block. For example, in FIG. 16, the image data at the right end of the 4 × 8 pixel block B4 is
The image data at the right end of the pixel block B6 is generated by being combined with the image data included in the area L5 of the pixel block B5 adjacent to the right side.

As described above, the currently input pixel block does not include a part of the image data of the small pixel block to be synthesized, and is located in the boundary area of another pixel block adjacent to this pixel block. When the image data is included, the image data generation unit 42 inputs the corresponding image data held in the data holding unit 41 in advance, and uses the input image data for synthesizing the image data.

The data holding unit 41 includes the image data generation unit 4
2 holds image data included in a predetermined boundary area of the pixel block among image data input for each predetermined pixel block. This boundary area is an area in which a small pixel block is set to extend between adjacent pixel blocks, and is adjacent to another pixel including image data necessary for performing interpolation processing of one pixel block. Block area. For example, regions L1 to L3 in FIG.
Also, the regions L4 to L6 in FIG. 16 correspond to this boundary region.

In the data number conversion unit 40 shown in the example of FIG. 5, common image data is input to the image data generation unit and the data holding unit like the data number conversion unit 20 shown in the example of FIG. Instead, the data holding unit 41 holds the image data input to the pixel block memory 43 at a stage preceding the image data generation unit 42. Further, as described above, in the pixel block memory 43, the image data already written can be read out to the image data generation unit 42 while writing the image data from the frame memory 50. However, it is possible to cause the data holding unit 41 to hold image data that has not yet been read by the image data generation unit 42. That is, the data holding unit 41 can hold the image data before being input to the image data generation unit 42.

Therefore, for example, the region L in FIG.
As in 4-L6, image data of a pixel block that has not yet been input to the image data generation unit 42 can be held. Also, for example, by appropriately setting the number of data held in the data holding unit 41, similarly to the data holding unit 21 shown in the example of FIG. Can be stored.

Here, the data number converter 40 shown in FIG.
Data storage unit 41 and image data generation unit 42
The configuration example will be described in detail. FIG. 6 illustrates a configuration example of the data holding unit 41 and the image data generation unit 42 illustrated in FIG. The data holding unit 41 illustrated in FIG. 5 includes a register 411, a register 412, and a selector 413. The image data generation unit 42 illustrated in FIG. 6 includes a latch unit 421, an addition unit 422, a coefficient multiplication unit 423, and a selector 424.

The register 411 stores the input image data C411 when the signal xcb-en is enabled.
-In is read and held. The signal xcb-en is a signal that is enabled in a predetermined boundary area in the pixel block of the chroma signal Cb, for example, as shown in FIG.
Are enabled at the timing when the image data C411-in of the regions L4 to L6 are input. In this case, the register 411 holds the image data of the boundary area (left end) in the 4 × 8 pixel block of the chroma signal Cb.

Further, in the example of FIG.
Has a simple serial input shift register configuration. Each time image data is held from the input, the already held image data is sequentially shifted and the selector 413 is shifted.
Is output to Then, the image data output from the register 411 is used for synthesizing the image data at the right end of the 4 × 8 pixel block.

The register 412 stores the input image data C411 when the signal xcr-en is enabled.
-In is read and held. The signal xcr-en is a signal that is enabled in a predetermined boundary region in a pixel block of the chroma signal Cr, and is, for example, a signal shown in FIG.
Are enabled at the timing when the image data C411-in of the regions L4 to L6 are input. In this case, the register 412 holds the image data of the boundary area (left end) in the 4 × 8 pixel block of the chroma signal Cr.

The register 412 also has a simple serial input shift register configuration like the register 411. Every time image data is held from the input, the already held image data is sequentially shifted. Selector 4
13 is output. The image data output from the register 412 is used when the image data is synthesized at the right end of the 4 × 8 pixel block.

The selector 413 outputs the selection signal area-c
One of the register 411 and the register 412 is selected according to r, and the image data of the selected register is output to the selector 424. When the image data C411-in is the chroma signal Cb, the register 411 is selected by the selection signal area-cr, and when the image data C411-in is the chroma signal Cr, the register 412 is selected.

The latch section 421 receives the clock signal CLK2
In synchronization with the image data C411-in, and outputs the held data to the adder 422.

The selector 424 selects the selection signal blk-ri
In response to the ght, either the image data C411-in or the output image data of the data holding unit 41 is selected, and the selected image data is output. In a predetermined boundary area (for example, areas L4 to L6 in FIG. 16) of the pixel block, the output image data from the data holding unit 41 is selected by the selection signal blk-right, and the image data C411-in is selected in other areas. I do.

The addition section 422 adds the image data S42 held in the latch section 421 and the image data S43 from the selector 424, and outputs the addition result to the coefficient multiplication section 42.
Output to 3.

The coefficient multiplying section 423 adds a predetermined coefficient (1 in the example of FIG. 6) to the image data synthesized by the adding section 422.
/ 2), and outputs the result of the multiplication to a JPEG circuit (not shown). As shown in the example of FIG.
Can be realized by a simple bit shift circuit.

Next, the operation of the image data processing apparatus having the above-described configuration and shown in FIGS. 5 and 6 will be described.

In the conventional method shown in FIG. 17, image data for each MCU is read from the frame memory 3 into the MCU memory 6, and separately from the YCrCb
The ratio conversion circuit 2 also reads image data from the frame memory 3 at its own timing. On the other hand, in the image data processing apparatus shown in the example of FIG. 5, a part of the image data read for each MCU from the frame memory 50 to the pixel block memory 43 is also appropriately held in the data holding unit 41. The unit 41 does not access the frame memory 3 at a unique timing with respect to the pixel block memory 43.

First, the operation of holding image data in the data holding unit 41 will be described. Of the image data C411-in for each 4 × 8 pixel block sequentially input from the pixel block memory, the image data included in a predetermined boundary area of the 4 × 8 pixel block is registered for each of the chroma signal Cr and the chroma signal Cb. 411 or register 4
12 respectively. This boundary area is, for example, the left end area of the 4 × 8 pixel block shown in the areas L4 to L6 in FIG. When the image data C411-in is the chroma signal Cb, the register 411 stores the chroma signal Cb.
In the case of r, the register 412 is selected by the selector 413, and the image data of the boundary area held in these registers is input to the selector 424.

These registers 411 and 412
Has a shift register configuration as described above, and the held image data of the boundary area is sequentially shifted in the shift register and output from the selector 413. For example, in FIG. 16, when image data is sequentially input from left to right on a horizontal line of one 4 × 8 pixel block and from top to bottom on a vertical line, the currently input 4 × 8 pixel block At the time when the rightmost image data of B4 is input, the leftmost image data (image data of the area L5) of the next input scheduled 4 × 8 pixel block B5 adjacent to this image data is output from the register 411 or 412. Have been. The left end image data is input to the image data generation unit 42 via the selector 424, and is used for synthesizing the image data. Further, the left end image data of the 4 × 8 pixel block is sequentially input to the registers 411 and 412 from the frame memory 50, and after being used for synthesizing the image data, is deleted from the registers.

Next, the operation of generating image data in the image data generating section 42 will be described. In the latch unit 421, image data S42 one clock before the clock signal CLK2 is generated with respect to the image data C411-in, whereby two adjacent pixel data (image data C411-in and image data S42) are generated. Generated. Normally, the selector 424 selects the image data C411-i
Since n has been selected, the two adjacent image data are added by the adder 422, and the coefficient
Multiplied by 1/2 '. That is, an average value of two adjacent image data is generated as the image data C422-out.

However, since the conversion is from the YCrCb ratio (4: 1: 1) to the YCrCb ratio (4: 2: 2), the image data C422-out has a sampling frequency twice that of the image data C411-in. Must be.
Therefore, the frequency of the clock signal CLK2 input to the latch unit 421 is twice that of the sampling clock signal of the image data C411-in. Therefore, the image data C411-in and the image data S42 have the same value every other clock of the clock signal CLK2,
The image data C411-in before the conversion and the image data C422-out after the conversion have the same value.

When the image data C411-in is the leftmost image data of the 4 × 8 pixel block, the leftmost image data is on the lower horizontal line with respect to the image data S42, and is a small pixel of two adjacent pixels. Not included in the block. On the other hand, the left end image data of the 4 × 8 pixel block on the right side output from the selector 413 of the data holding unit 41 is included in this small pixel block. At this timing, the signal blk-right switches,
The data holding unit 41 is selected by the selector 424, and the left end image data of the 4 × 8 pixel block on the right held by the data holding unit 41 is combined with the image data S42,
Image data C422-out is generated.

FIG. 7 shows the YCrCb ratio (4: 4) of the data holding unit 41 and the image data generating unit 42 shown in FIG.
FIG. 3 is a diagram for explaining conversion from 1: 1) to a YCrCb ratio (4: 2: 2). In FIG. 7, the image data C10
The image data C13 has a YCrCb ratio (4: 1: 1) of 4
Image data for one horizontal line of a × 8 pixel block, and image data C20 includes image data C10 to image data C
The image data at the left end of the horizontal line one stage below the 13 horizontal lines is shown. The image data C′10 to C′17 are YC, and the image data C10 to C13 are YC.
The image data after conversion into the rCb ratio (4: 2: 2) is shown.

In FIG. 7, the image data C10 to C13 are sequentially input as image data C411-in from the left. When the image data C20 at the left end of the 4 × 8 pixel block is input as the image data C411-in, the image data C13 one clock before is on a different horizontal line from the image data C20 and is not adjacent. Therefore, if the image data C20 and the image data C13 are combined as the same small pixel block, image data that is not adjacent at all will be combined, and appropriate interpolation processing will not be performed. Therefore, in the image data generating unit 42 shown in FIG. 6, the signal blk-rig is input at the timing when the leftmost image data of the 4 × 8 pixel block is input.
ht is switched, the image data held in the data holding unit 41 and the image data C13 are combined, and the image data C4
22-out is generated. As described above, the data holding unit 41 stores the image data at the left end of the next supplied 4 × 8 pixel block and the image data C13 on the image.
Since the image data adjacent to the right of is stored, the image data and the image data C13 are combined to perform an appropriate interpolation process.

FIG. 8 is a diagram showing the timing of each signal in the data holding unit 41 and the image data generating unit 42 shown in FIG. Note that the reference numerals (C10 to C13, C20) attached to the lower side of FIG. 8B and the reference numerals (C10 ′ to C′17) attached to the lower side of FIG. Yes, it is. Also, averaging processing of image data surrounded by dotted lines in FIGS. D and e (a to e)
Correspond to the processes of the same reference numerals in FIG.

The clock signal CLK2 (FIG. 8A) has twice the frequency of the sampling frequency of the image data C411-in (FIG. 8B). Image data C41
1-in is held in the latch unit 421 in synchronization with the clock signal CLK2. This held image data S
42 (FIG. 8D) has a timing delayed by a half cycle with respect to the sampling cycle of the image data C411-in. When the selector 424 selects the image data C411-in, the image data S42 and the image data C4
11-in is averaged by the adding unit 422 and the coefficient multiplying unit 423, and image data C422-out (FIG. 8)
h). For example, the averaging processes a to
In the averaging process d, the image data S43 (FIG. 8e) output from the selector 424 is equal to the image data C411-in, and the image data S43 and the image data S42 are averaged to generate the image data C422-out. Have been. The averaging process a and the averaging process c
Since the image data S42 and the image data C411-in have the same value, the same data as the image data C411-in before the conversion is used as it is.
Output as out.

The selection signal blk-rig of the selector 424
ht (FIG. 8f) indicates that the image data is
411-in is a signal for selecting the data holding unit 41 at a high level. Select signal blk-right
t is normally at a low level, whereby the averaging process of the image data S42 and the image data C411-in is executed. When the image data at the left end of the 4 × 8 pixel block is input as the image data C411-in, the selection signal blk-right goes high, and the image data S41 (FIG. 8g) output from the data holding unit 41 is output. Selected.

For example, in the averaging process e, the selection signal bl
k-right is at a high level, and the image data S41 (84h) output from the data holding unit 41 is input to the adding unit 422 as image data S43. By averaging the image data S43 and the image data S42 (4Ch), the image data C422-out is obtained.
(68h) has been generated.

The selection signal area-cr (FIG. 8c) goes high when the input image signal is the chroma signal Cr to select the register 412, and goes low when the input image signal is the chroma signal Cb to the register 411. Is a signal for selecting. FIG. 8 shows only an example in which the input image signal is the chroma signal Cr, so that the selection signal area-cr is constant at a high level. FIG. 8 shows the timing in the case of the chroma signal Cr, but the same applies to the case of the chroma signal Cb. That is, by setting the selection signal area-cr to low level, the above description
This is also applicable to the case of b.

Further, the signal xcb-en and the signal xcr-e
n, select signal blk-right, select signal area-
Each control signal such as cr has a certain timing in one 4 × 8 pixel block including the chroma signal Cr and the chroma signal Cb. Therefore, for example, a counter which is initialized at the beginning of a 4 × 8 pixel block and counts in response to input of image data is prepared, and each control signal can be easily generated by decoding the count value of the counter.

As described above, according to the image data processing apparatus shown in FIGS. 5 and 6, similarly to the first embodiment, the image data necessary for interpolation is appropriately held in the data holding unit 41 and synthesized. By using this, the number of data can be directly converted to image data input for each pixel block. As a result, the process of reading and writing image data from and to the frame memory and the line memory is omitted, so that the speed of data processing can be increased.
The circuit scale can be significantly reduced.

Further, while writing image data to be input for each predetermined pixel block in the pixel block memory 43, the image data of the already written pixel block is output to the image data generation unit 42, and the data holding unit 41 And image data contained in a predetermined area of a pixel block among image data input to the pixel block memory 43. Therefore, for example, while the image data is sequentially transferred from the pixel block memory 43 to the image data generation unit 42, the image data not yet input to the image data generation unit 42 can be stored in the data storage unit 41. That is, the data holding unit 41 can hold the image data before being input to the image data generation unit 42. Therefore, for example, image data necessary for the interpolation process with the image data of the pixel block not yet input to the image data generation unit 42, such as the regions L4 to L6 in FIG. it can. In this case, the frame memory 5
Since part of the image data read from 0 to the pixel block memory 43 is held in the data holding unit 41, the access processing to the frame memory 50 can be simplified and the processing can be speeded up. A complicated circuit for accessing the 50 is unnecessary, and the circuit scale can be reduced.

<Third Embodiment> FIG. 9 shows a third embodiment of the present invention.
It is a schematic structure figure for explaining camera system 100 of an embodiment. The camera system 100 includes an optical system 101, a CCD 102, an A / D conversion unit 103, an image compression unit 104, an SDRAM 114, and a CPU 115. Further, the image compression unit 104 includes the CCD signal processing unit 10.
5, bus 106, buffer 107, SDRAM interface (SDRAM I / F) 108, JPEG processing unit 109a, JPEG interface unit 109b, clock generation unit 110, system controller 111, CP
It has a U interface (CPU I / F) 112 and a memory controller 113.

The optical system 101 captures a desired image by a user's operation, and forms the optical signal on the image capturing surface of the CCD 102. The CCD 102 converts an optical signal formed on the imaging surface by the optical system 101 into an electric signal, and outputs the electric signal to the A / D converter 103 as an analog image signal. A /
The D conversion unit 103 converts the analog image signal input from the CCD 102 into a digital image signal of a predetermined gradation,
Output to the CCD signal processing unit 105 of the image compression unit 104.

The CCD signal processing unit 10 of the image compression unit 104
5 is based on the control of the system controller 111,
R (red), G (green), B
Each color signal of (blue) is separated, gamma correction is performed on each color signal for color reproducibility, and a luminance signal and a chroma signal are generated. The generated image data including the luminance signal and the chroma signal is output to the buffer 107 via the bus 106.

The buffer 107 is provided for the CCD signal processor 10.
5 sequentially stores the image data inputted via the bus 106, and when a certain amount is stored, the memory controller 11
3 to the SDRAM I / F 108 based on the control. Further, S input from the SDRAM I / F 108
The image data read from the DRAM 114 is temporarily stored and output to the JPEG processing unit 109 via the bus 106. The SDRAM I / F 108 is an image compression unit 104
And stores the image data of each predetermined unit inputted from the buffer 107 in the SDRAM 114 under the control of the memory controller 113. Further, the image data stored in the SDRAM 114 is
The data is read out for each 8 × 8 pixel block and output to the buffer 107.

[0124] The JPEG processing unit 109a performs JPEG encoding on image data input for each 8x8 pixel block via the JPEG interface unit 109b to generate an encoded bit stream. Then, the generated encoded bit stream is transmitted to the CPU 11 via the JPEG interface unit, the bus 106 and the CPU I / F 112.
5 is output.

The JPEG interface unit 109b is
SDRA based on the control of the system controller 111
M114, the buffer 107 and the bus 1
The Y / C converter converts the YCrCb ratio of the image data input for each predetermined image block through the unit 06, and outputs the converted image data to the JPEG processing unit 109a for each 8 × 8 pixel block. Also, the coded bit stream generated in the JPEG processing unit 109a is
Output to CPU 115 via PUI / F112.

Note that the JPEG interface unit 109
b has the image data generation unit and the data holding unit described in the first embodiment or the second embodiment, and causes the data holding unit to appropriately hold the image data necessary for the interpolation to generate the image data. The conversion of the YCrCb ratio is directly performed on the image data input for each predetermined pixel block by using the interpolation of the image data in the section.

The clock generation unit 110 generates a clock to be used in each unit in the image compression unit 104 based on the control of the system controller 111 and provides the clock to each component. The bus 106 schematically shows a data bus in the image compression unit 104. Via this bus 106, the CCD signal processing unit 105 sends the signal to the buffer 107 and the buffer 107 sends the signal to the JPEG interface 10.
The transfer of image data to the CPU 9b and the transfer of an encoded bit stream from the JPEG interface unit 109b to the CPU I / F 112 are performed.

The system controller 111 includes the CPU 1
15, the operation of the image compression unit 104, that is, the SDRAM 11 of the input image data.
4, the transfer of the image data stored in the SDRAM 114 to the JPEG processing unit 109, the JPEG processing unit 10
Each component of the image compression unit 104 is controlled so that operations such as JPEG encoding in 9a and output of encoded image data to the CPU 115 can be appropriately performed. The CPU I / F 112 is an interface with the CPU 115, and performs control signals from the CPU 115, input of image signals, control signals to the CPU 115, output of encoded data, and the like. The memory controller 113 controls the buffer 1 based on the control of the system controller 111.
07 and the SDRAM I / F 108 to store the image data in the SDRAM 114 and the SDRAM 114
And controls the reading of the image data stored in the.

The SDRAM 114 is a memory for temporarily storing image data composed of a photographed luminance signal and chroma signal. Image data photographed by the optical systems 101 to the A / D conversion unit 103 is temporarily stored in the SDRAM 114, then sequentially supplied to the JPEG processing unit 109, encoded, output to the CPU 115, and stored, displayed, transmitted, and the like. Used for The CPU 115 captures a desired image by the optical system 101 to the image compression unit 104 and the SDRAM 114, performs image processing, stores and reproduces image data, and executes JPEG.
The camera system 10 is controlled so that each processing such as encoding, storage, display, and transmission of JPEG encoded data is appropriately performed, and the camera system 100 performs a desired operation as a whole.
0 is controlled.

In the camera system 100 having such a configuration, first, when a desired image is picked up by the optical system 101 by a user's operation, the CCD 102 converts the light signal into an electric signal to generate an image signal. You.
The image signal is converted from an analog signal into a digital signal by the A / D converter 103, and further decomposed into each color signal by the CCD signal processor 105 of the image compressor 104.
After the gamma correction, the image data is converted into image data including a luminance signal and a chroma signal. This image data is temporarily transferred to the SDR via the buffer 107 and the SDRAM I / F 108.
After being stored in the AM 114, it is sequentially read out for each predetermined pixel block and
Is converted into a YCrCb ratio, and input to the JPEG processing unit 109a for each 8 × 8 pixel block. JPEG
In the processing unit 109a, the image data of each sequentially input 8 × 8 pixel block is JPEG encoded, and a JPEG encoded data stream of a predetermined format is generated. This JPEG encoded data stream is CP
The data is output to the CPU 115 via the UI / F 112, and processing such as storage, display, and transmission is performed.

As described above, in the camera system shown in FIG. 9, the conversion of the number of data can be directly performed on the image data input to the JPEG interface unit 109b for each predetermined pixel block. As a result, the process of reading and writing image data from and to the frame memory and line memory is omitted, so that data processing can be speeded up and the circuit scale can be significantly reduced.

The present invention relates to the above-described first to third embodiments.
The invention is not limited to the third embodiment. For example, the first
In the above embodiment, only the case where the number of data is reduced by the conversion of the YCrCb ratio is shown. It is also applicable when the number is expanded. Conversely, the second
In the embodiment, only the case where the number of data is expanded by the conversion of the YCrCb ratio is shown, but the present invention can be applied to the case where the number of data is reduced.

[0133]

According to the present invention, the number of data can be directly converted to image data input for each predetermined pixel block. This eliminates the process of reading and writing image data from and to the frame memory and the line memory, so that the data processing speed can be increased. Further, since the memory capacity required for storing image data is reduced, the circuit scale can be reduced. In addition, since the process of accessing the frame memory is simplified, the data processing speed can be increased. Since the number of complicated circuits for directly accessing the frame memory can be reduced, the circuit scale can be reduced.

[Brief description of the drawings]

FIG. 1 is a schematic configuration diagram for explaining an image data processing device according to a first embodiment of the present invention.

FIG. 2 is a diagram illustrating a configuration example of a data holding unit and an image data generation unit illustrated in FIG. 1;

FIG. 3 shows a YCrCb ratio (4: 2: 2) obtained by the data holding unit and the image data generating unit shown in FIG. 2;
It is a figure for explaining conversion to a ratio (4: 1: 1).

FIG. 4 is a diagram illustrating timings of respective signals in a data holding unit and an image data generation unit illustrated in FIG. 2;

FIG. 5 is a schematic configuration diagram for explaining an image data processing device according to a second embodiment of the present invention.

FIG. 6 illustrates a configuration example of a data holding unit and an image data generation unit illustrated in FIG. 5;

FIG. 7 shows a YCrCb ratio (4: 1: 1) obtained by the data holding unit and the image data generating unit shown in FIG. 6;
It is a figure for explaining conversion to a ratio (4: 2: 2).

FIG. 8 is a diagram illustrating timings of respective signals in a data holding unit and an image data generation unit illustrated in FIG. 6;

FIG. 9 is a schematic configuration diagram illustrating a camera system according to a third embodiment of the present invention.

FIG. 10 is a diagram showing a configuration of an interface unit of a general digital video image processing IC.

FIG. 11 is a diagram showing an MCU of image data having a YCrCb ratio (4: 4: 4).

FIG. 12 is a diagram illustrating an MCU of image data having a YCrCb ratio (4: 2: 2).

FIG. 13 is a diagram illustrating an MCU of image data having a YCrCb ratio (4: 1: 1).

FIG. 14 is a block diagram showing two chroma signal 8 × 8 pixel blocks in an MCU having a YCrCb ratio (4: 2: 2).
One chroma signal 8 in MCU with ratio (4: 1: 1)
FIG. 9 is a diagram for describing an interpolation process of converting into an × 8 pixel block.

FIG. 15 shows a YCrCb ratio based on the YCrCb ratio (4: 2: 2).
FIG. 10 is a diagram illustrating a configuration example of a conventional JPEG interface circuit that performs conversion to a ratio (4: 1: 1).

FIG. 16 shows one chroma signal 8 × 8 pixel block in an MCU having a YCrCb ratio (4: 1: 1).
FIG. 14 is a diagram for describing interpolation processing for converting into chroma signals of two MCUs in an MCU having a ratio (4: 2: 2).

FIG. 17 shows a YCrCb ratio (4: 1: 1)
FIG. 11 is a diagram illustrating a configuration example of a conventional JPEG interface circuit that performs conversion to a ratio (4: 2: 2).

[Explanation of symbols]

20: data number conversion unit, 21: data holding unit, 211,
212 register, 213 selector, 22 image data generator, 221, 222 latch unit, 223-225
... Coefficient multiplying section, 226... Adding section, 227.
3 ... pixel block memory, 30 ... frame memory, 40
... Data number conversion unit, 41 Data holding unit, 411, 41
2. Register 413 Selector 42 Image data generator 421 Latch 422 Adder 423 Coefficient multiplier 424 Selector 43 Pixel block memory 50 Frame memory 100 Camera system ,
101: optical system, 102: CCD, 103: A / D conversion unit, 104: image compression unit, 105: CCD signal processing unit,
106 bus, 107 buffer, 108 SDRAM
Interface, 109a ... JPEG processing unit, 109
b: JPEG interface unit, 110: clock generation unit, 111: system controller, 112: CPU
Interface, 113 ... Memory controller

 ──────────────────────────────────────────────────続 き Continuing on the front page F term (reference) 5C022 AA13 AC69 5C057 AA01 AA06 DC01 EA02 EA07 EH01 EL01 EM00 GF01 GG03 GG04 GJ01 GJ04 5C059 KK06 KK11 LC00 MA00 PP01 PP16 SS15 UA02 UA05 UA38 5C21 DA04 DA04 EA00

Claims (19)

[Claims] 1. An image data processing apparatus for combining image data input for each predetermined pixel block for each predetermined small pixel block and converting the number of image data, comprising: Data holding means for holding included image data, and image data input for each of the predetermined pixel blocks,
At least one image data is generated from the synthesized data by giving a predetermined weight to each of the small pixel blocks, and at least one image data of the small pixel block is adjacent to the pixel block of the input image data. When the image data is included in the predetermined area of another pixel block, a part of the image data of the small pixel block is input from the data holding unit, and the image data of the small pixel block including the input image data is converted to the predetermined image data. An image data processing device comprising: image data generating means for generating images by weighting and synthesizing the data to generate at least one image data from the synthesized data. 2. The image data processing device according to claim 1, wherein the data holding unit holds image data of the predetermined area among image data input to the image data generation unit. 3. A pixel block holding unit that holds image data input for each pixel block and outputs image data of the pixel block already held to the image data generation unit. 2. The image data processing device according to claim 1, wherein the image data included in the predetermined area among image data input to the pixel block holding unit is held. 4. The image data generating means according to claim 1, wherein, of the image data input in synchronization with the first clock signal, the input image data from a most recent input image data to a predetermined number of clocks before the first clock signal. And synthesizes the latest input image data and the held image data by giving a predetermined weight according to the input order, and outputs the synthesized image data in synchronization with a second clock signal. The image data included in the predetermined area of another pixel block adjacent to the pixel block of the input image data and a part of the latest input image data and the stored image data are included. When configuring the small pixel block, image data included in the predetermined area is input from the data holding unit, and the small pixel including the input image data is input. 2. The image data processing device according to claim 1, wherein the image data of the blocks is combined by giving the predetermined weight, and the combined image data is output in synchronization with a second clock signal. 5. The image data generating means according to claim 1, wherein, of the image data input in synchronization with the first clock signal, input image data from a most recent input image data to a predetermined number of clocks before the second clock signal is input. Holding the data in synchronization with the second clock signal, applying a predetermined weighting to the latest input image data and the stored image data in accordance with an input order, and synthesizing the latest input image data and the stored image data. When a part of the stored image data and the image data included in the predetermined area of another pixel block adjacent to the pixel block of the input image data constitutes the small pixel block, The image data included in the predetermined area is input from the data holding unit, and the image data of the small pixel block including the input image data is input to the image data of the predetermined pixel block. The image data processing device according to claim 1, wherein the image data is combined by giving weights. 6. An image decompression unit for decompressing image data compressed by a predetermined method in units of said pixel blocks, and outputting to said data holding unit and said image data generation unit for each of said pixel blocks. An image data processing device according to claim 1. 7. The image data processing device according to claim 1, further comprising image compression means for compressing the image data synthesized by said image data generation means in a unit of said pixel block by a predetermined method. 8. An image data processing method for synthesizing image data input for each predetermined pixel block for each predetermined small pixel block and converting the number of image data, comprising: A data holding step of holding included image data, and image data input for each of the predetermined pixel blocks,
At least one image data is generated from the synthesized data by giving a predetermined weight to each of the small pixel blocks, and at least one image data of the small pixel block is adjacent to the pixel block of the input image data. When included in the predetermined area of another pixel block, the partial image data held in the data holding step is input, and the predetermined weight is given to the small pixel block including the input image data. Synthesizing and generating at least one image data from the synthesized data. 9. The image data processing method according to claim 8, wherein the data holding step holds image data of the predetermined area among the image data input in the image data generating step. 10. A pixel block holding step of holding image data input for each pixel block and outputting image data of the already held pixel block in the image data generation step. 9. The image data processing method according to claim 8, wherein, among the image data input in the pixel block holding step, image data included in the predetermined area of the pixel block is held. 11. The image data generating step according to claim 1, wherein
Of the image data input in synchronization with the clock signal of
The input image data from the latest input image data up to a predetermined number of clocks before the first clock signal is held, and a predetermined weight is assigned to the latest input image data and the held image data according to an input order. Give and combine,
The combined image data is output in synchronization with a second clock signal, and is output to a part of the latest input image data and the held image data and a pixel block of the input image data. When the image data included in the predetermined area of another adjacent pixel block constitutes the small pixel block, the image data included in the predetermined area held in the data holding step is input, and the input image is input. The image data of the small pixel block including the data is combined by giving the predetermined weight to the image data,
The image data processing method according to claim 8, wherein the combined image data is output in synchronization with a second clock signal. 12. The image data generating step according to claim 1, wherein
Of the image data input in synchronization with the clock signal of
The input image data from the latest input image data to a predetermined number of clocks before the second clock signal is held in synchronization with the second clock signal, and the latest input image data and the held image data are stored in the latest input image data and the held image data. A predetermined weighting is given in accordance with the input order, the image data is synthesized, and some of the recent input image data and the held image data are stored in the input image data and the other image data adjacent to the pixel block of the input image data. When the image data included in the predetermined area of the pixel block constitutes the small pixel block, the image data included in the predetermined area held in the data holding step is input, and the image data including the input image data is input. The image data processing method according to claim 8, wherein the image data of the small pixel block is combined by giving the predetermined weight. 13. An image decompression step of decompressing image data input in the data holding step and the image data generation step from the image data compressed in a predetermined format in units of the pixel blocks to generate the image data. Item 9. The image data processing method according to Item 8. 14. The image data processing method according to claim 8, further comprising an image compression step of compressing the image data generated in the image data generation step on a pixel block basis. 15. A photographing means for photographing a desired image and generating image data corresponding to each pixel of the photographed image; and performing predetermined processing on the image data generated by said photographing means. Processing means for inputting and outputting the subsequent image data for each predetermined pixel block; and combining image data input for each predetermined pixel block from the processing means for each predetermined small pixel block, thereby reducing the number of pieces of image data. An image data processing device for converting, the image data processing device includes: a data holding unit that holds image data included in a predetermined area of the pixel block; and an image data input for each upper pixel block. A predetermined weighting is given to each small pixel block to synthesize, and at least one image data is generated from the synthesized data;
When the partial image data of the small pixel block is included in the predetermined area of another pixel block adjacent to the pixel block of the input image data, the partial image data is input from the data holding unit, A camera system comprising: image data of the small pixel block including the input image data; combining the image data by applying the predetermined weight; and generating at least one image data from the combined data. 16. The camera system according to claim 15, wherein said data holding means holds image data of said predetermined area among image data input to said image data generating means. 17. A pixel block holding means for holding image data inputted from the processing means for each of the pixel blocks, and for outputting the already held image data of the pixel block to the image data generating means, 16. The camera system according to claim 15, wherein the data holding unit holds image data included in the predetermined area among image data input to the pixel block holding unit. 18. The image processing apparatus according to claim 1, wherein the processing unit expands the image data compressed by a predetermined method in a unit of the pixel block, and outputs the image data to the data holding unit and the image data generating unit for each pixel block. The camera system according to claim 15, comprising: 19. The camera system according to claim 15, wherein said processing means has image compression means for compressing the image data generated by said image data generation means in units of said pixel blocks by a predetermined method.