JP2003143454A - Image processing device and method therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce a processing time of an image pickup device. SOLUTION: A raster block converting memory 21 uses an SRAM or the like which reads/writes simultaneously, and address control of the raster block converting memory 21 is constituted in a ring buffer configuration, whereby data read out next from a memory can be stored at an address which is finished reading out data in block order, and this can cause to decrease memory capacity. When image data is read out from a memory 4, all the data including data of a neighboring MCU except for data necessary for constituting a MCU of the address accessed to the memory 4 are held in the raster block converting memory 21. An access frequency to the memory 4 is minimized, thereby restricting increase in a processing time.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a digital still camera or a digital video camera capable of recording a video signal photoelectrically converted by the image pickup device.

[0002]

2. Description of the Related Art Image data output from an image sensor is
Temporarily S for each line by separating into luminance signal and color difference signal
Store in a storage device such as DRAM.

When converting the stored image data into raster blocks according to the JPEG standard, a raster block conversion memory having a capacity of several lines is required.

Further, a process of reading image data from a storage device and storing it in a raster block conversion memory,
The process of reading the image data converted by the raster block conversion memory is a sequential process.

[0005]

The conventional image processing apparatus as described above has the drawbacks that the circuit scale is large and the processing speed is slow.

Therefore, the present invention provides an image processing apparatus capable of reducing the capacity of a raster block conversion memory in a raster block conversion circuit without reducing the processing speed.

[0007]

The present invention has taken the following means in order to solve the above problems.

An image pickup device for photoelectrically converting an optical image of a subject, a storage device for temporarily storing various data, and a temporary storage means for temporarily storing image data output from the image pickup device in the storage device. A, image processing means for separating the image data into a component corresponding to a luminance signal and a component corresponding to a color difference signal, and a temporary storage for temporarily storing the luminance signal and the color difference signal in the storage device for recording. A storage unit B, a temporary storage unit C for temporarily storing the luminance signal and the color difference signal in the storage device for display, and a raster block conversion memory are provided to code the luminance signal and the color difference signal. It is characterized in that it has an encoding means for performing encoding processing, and a recording means for recording and retaining the encoded video data obtained by the encoding means in a predetermined recording medium.

By using the raster block conversion memory capable of simultaneous read / write and configuring the address control of the raster block conversion memory in a ring buffer form, data is read from the raster block conversion memory in block order. The data read next from the storage device can be stored in the read-completed address, and the memory capacity can be reduced.

Further, when the image data is read from the storage device, the MCU of the address accessed to the storage device
By holding all the data including the data of adjacent MCUs other than the data required to configure the memory in the raster block conversion memory, the access frequency to the storage device is minimized and the processing time is increased. Can be suppressed.

[0011]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

As shown in FIG. 1, the image processing apparatus 1 includes
Image sensor 2, display device 3, storage device 4, and recording medium 5
Are connected.

The image processing apparatus 1 converts the input image data D1 input from the image pickup device 2 into recording data D2 and display data D3, and the recording data D2 is compressed in accordance with the JPEG system to obtain compressed image data. D4 is recorded on the recording medium 5. The display data D3 is output to the display device 3.

During reproduction, the compressed image data D4 is read from the recording medium 5, expanded according to the JPEG method, converted into display data D3, and output to the display device 3.

The image pickup device 2 is composed of a CCD camera or the like and takes in input image data D1 such as a color still image,
The input image data D1 is output to the image processing device 1.

The display device 3 is composed of a liquid crystal monitor or the like and displays an image. The storage device 4 is composed of a memory device such as an SDRAM having a sufficient capacity to store the recording data D2 for one screen. The recording medium 5 is composed of a non-volatile memory card device or the like.

Next, the structure of the image processing apparatus 1 will be described in detail.

The image processing apparatus 1 includes a temporary storage means A11,
The image processing unit 12, the temporary storage unit B13, the temporary storage unit C14, the encoding unit 15, the recording unit 16, and the arbitration unit 17 are included. The arbitration means 17 includes a temporary storage means A11, a temporary storage means B13, and a temporary storage means C1.
4, storage unit 4 of encoding unit 15 and recording unit 16
Access is arbitrated on a time-sharing basis.

Further, the smoothing means R is stored in the temporary storage means B13.
18 and the smoothing means M19 in the temporary storage means C14,
The encoding means 15 has a JPEG core 20 that performs compression and expansion according to the JPEG algorithm.

The encoding means 15 is for compressing / decompressing image data based on a predetermined method (JPEG method in this embodiment).

The image processing apparatus 1 takes in the input image data D1 input from the image pickup device 2 by the temporary storage means A11 and stores it in the storage device 4. The input image data D1 stored in the storage device 4 is read by the temporary storage means A11, and is output to the image processing means 12 together with the input image data D1 input from the image pickup device 2 next.

In the monitor mode, the input image data D1 input from the image pickup device 2 is output to the image processing means 12 without being stored in the storage device 4.

The image processing means 12 separates the input input image data D1 into a luminance signal and a color difference signal to generate recording data D2 and display data D3. Recording data D
2 is stored by the temporary storage means B, and the display data D3 is stored by the temporary storage means C in the storage device 4 constituted by a memory device such as SDRAM having a sufficient capacity for storing the recording data D2 for one screen. The recording data D2 is converted by the encoding means 15 into raster order data in the raster block conversion memory 21 configured by SRAM or the like and output to the JPEG core 20 in the JPEG core 20.
0 compresses according to the JPEG algorithm.

The compressed compressed image data D4 is stored in the storage device 4 by the encoding means 15, and is recorded in the recording means 16.
The data is output to and recorded on the recording medium 5 composed of a non-volatile memory device or the like.

At the same time, the display data D3 is output as a monitor image or a preview image to the display device 3 such as a liquid crystal monitor, and the display device 3 displays the image.

The image processing apparatus 1 also reads the compressed image data D4 recorded on the recording medium 5 by the recording means 16 and stores it in the storage device 4, and the compressed image data D4 is expanded by the encoding means 15 according to the JPEG algorithm. Then, it is stored in the storage device 4 as the recording data D2.
The recording data D2 stored in the storage device 4 is read by the temporary storage means A11, converted into the display data D3 by the image processing means 12 and output to the display device 3 by the temporary storage means C14, and the display device 3 restores the image. To do.

Further, the expanded recording data D2 is read by the temporary storage means A11, the image size and the angle of view are changed by the image processing means 12, the temporary storage means B stores it in the storage device 4, and the encoding means 15 is used. It has a resizing function and a cutting function for compressing.

In order to reduce the average data rate to the storage device 4, the smoothing means R18 and the smoothing means M19 having a memory such as SRAM are used as the temporary storage means B13 and the temporary storage means C14.

Next, the operation of the image processing apparatus 1 configured as described above will be described.

FIG. 2 shows that the storage device 4 has a format of 4: 2: 0.
Recording data D2 of, 000,000 pixels (2240 x 1680)
It is a conceptual diagram which stores.

Although the storage position of the recording data D2 is described above, the storage position is variable.

The vertical axis of the storage device 4 shows the total number of pages and page addresses of the SDRAM, and the horizontal axis shows one access unit (page) of the storage device 4.

Since one page has 32 bytes and one line has 2240 pixels, it becomes 70 pages / line.

Since the format is 4: 2: 0, the luminance signal has two lines and the color difference signal has one line.

Luminance signals are packed and stored in order from the beginning of the line. As the color difference signal, Cb / Cr interleaved pixel by pixel and output from the image processing unit 12 is stored separately for each 8 pixels.

The details are shown in FIG. One row shows a group of data that can be read and written by one access, and stores 4 bytes from LSB in order from the beginning of the line.

The conversion from the page address output by each means to the actual address of the SDRAM is performed by the arbitration means 17.
I am going to.

In this embodiment, since the access unit of the storage device 4 is 32 Bytes, one access to the storage device 4 makes it possible to form an MCU adjacent to the adjacent MCUs.
Also reads the data.

In order not to reduce the overall processing speed of the image processing apparatus 1, it is necessary to minimize the number of accesses to the storage device 4, so all the read data is stored in the raster block conversion memory 21. 1 in 4: 2: 0 compression
32 bytes x 24 times = 768 to configure MCU
Byte data is read from the storage device 4.

If all the data of the adjacent MCUs are also stored here, the raster block conversion memory 21 has 768B.
However, the memory capacity is reduced by adopting the configuration shown in FIG.

FIG. 4 shows the structure of each JPEG macroblock and storage address of the raster block conversion memory 21 in the encoding means at the time of 4: 2: 0 compression.

Here, the luminance macroblocks Yn and Yn +
1, Yn + 2, Yn + 3 and color difference macroblock Cb
Luminance macroblock Y
Adjacent 1 MCUs are constituted by n + 4, Yn + 5, Yn + 6, Yn + 7 and color difference macroblocks Cbn + 1, Crn + 1 (n = 0, 8, 16 ...).

The luminance signal of the recording data D2 is read from the storage device 4 eight times to obtain luminance macroblocks Yn, Yn +.
Complete 1, Yn + 4 and Yn + 5. While inputting the completed luminance macroblock Yn and luminance macroblock Yn + 1 to the JPEG core 20, the raster block conversion memory 21 has four or more free macroblocks. Therefore, the storage device 4 is accessed eight times to write the recording data D2. The luminance signal for 8 pages is read, and the next luminance macroblock Yn is read.
Complete +2, Yn + 3, Yn + 6, and Yn + 7.

Next, when the luminance macroblocks Yn and Yn + 1 have been input to the JPEG core 20, the raster block conversion memory 21 can have 4 macroblocks or more, so that the color difference signal of the recording data D2 from the storage device 4 is set to 8 times. Color difference macro blocks Cbn, Crn, Cb are read out in batches.
n + 1 and Crn + 1 are completed.

During this period, Yn + is sequentially added to the JPEG core 20.
2, Yn + 3, Cbn, Crn data input, 1MC
After inputting U, the next MCU is set to the luminance macroblock Yn + 4,
Enter Yn + 5.

Luminance macroblocks Yn + 4, Yn + 5
After inputting, the luminance signal of the recording data D2 is read from the storage device 8 eight times and the next luminance macroblock Y is read.
Complete n, Yn + 1, Yn + 4, Yn + 5.

Luminance macroblocks Yn + 6, Yn + 7
After inputting, the luminance signal of the recording data D2 is read from the storage device 8 eight times and the next luminance macroblock Y is read.
Complete n + 2, Yn + 3, Yn + 6, Yn + 7.

Color difference macroblocks Cbn + 1, Crn
Input of +1 is completed, and luminance macroblocks Yn and Yn
After inputting +1 to the JPEG core 20, the storage device 4
Next color difference macroblocks Cbn, Crn, Cbn +
Read 1, Crn + 1.

The subsequent steps are repeated. There are six transitions of the storage address of each macroblock in FIG.

FIG. 5 shows the first 1MC at the time of 4: 2: 0 compression.
Luminance macroblocks Y0, Y1, Y2, which form U
Raster block conversion memory 2 by comparing Y3 and color difference macroblocks Cb0 and Cr0 with each pixel in FIG.
1 shows the detailed configuration stored in 1.

FIG. 6 shows the structure of the JPEG macroblock and the storage address of the raster block conversion memory 21 in the encoding means at the time of 4: 2: 2 compression.

The luminance signal of the recording data D2 is read from the storage device 4 eight times to obtain luminance macroblocks Yn, Yn +.
Complete 1, Yn + 2 and Yn + 3.

While inputting the completed luminance macroblock Yn and luminance macroblock Yn + 1 to the JPEG core 20, the raster block conversion memory 21 has four or more empty macroblocks. Therefore, the color difference signal of the recording data D2 is output from the storage device 4. Are read eight times to complete the color difference macroblocks Cbn, Crn, Cbn + 1, Crn + 1.

Next, the luminance macro blocks Yn, Yn +
1. When the color difference macroblocks Cbn and Crn and the luminance macroblocks Yn + 2 and Yn + 3 have been input to the JPEG core 20, the luminance signals of the recording data D2 are read from the storage device 8 eight times to obtain the luminance macroblocks Yn and Yn +.
Complete 1, Yn + 2 and Yn + 3.

Next, color difference macroblocks Cbn + 1, C
After inputting rn + 1 to the JPEG core 20, the color difference macroblocks Cbn, Crn, Cbn + 1, Crn + 1
To complete.

The subsequent steps are repeated. The transition of the storage address of each macroblock is one in FIG.

As described above, according to this embodiment, the following effects can be obtained.

Temporary storage means A for picking up an image of a subject by the image pickup device 2 and temporarily storing the image data output from the image pickup device in a storage device, and a component of the image data corresponding to a luminance signal and a color difference. Image processing means for separating into components corresponding to signals, temporary storage means B for temporarily storing the luminance signal and color difference signal in the storage device for recording, and the luminance signal and color difference signal for display. Temporary storage means C for temporarily storing in the storage device, encoding means having a raster block conversion memory 21 for performing encoding processing on the luminance signal and color difference signal, and the encoding means A recording means for recording and holding the coded video data thus obtained is stored in a predetermined recording medium.

By using the raster block conversion memory 21 capable of simultaneous read / write and configuring the address control of the raster block conversion memory 21 into a ring buffer, the block order from the raster block conversion memory 21 is changed. Then, the next data read from the storage device can be stored in the address where the data has been read, and the memory capacity can be reduced.

When the image data is read from the storage device, all the data of the accessed address (including the extra data other than the data necessary for configuring the MCU) is read by the raster block conversion memory 21. By holding at, it is possible to reduce the frequency of access to the storage device and suppress an increase in processing time.

The present invention is not limited to the above-mentioned embodiment, but may be carried out as follows. (1) In the present embodiment, the smoothing means R18 and the smoothing means M19 are used to reduce the average data rate, but the smoothing means R18 and the smoothing means M19 may be omitted. (2) In the present embodiment, Cb / Cr is stored in the storage device 4 separately for every 8 pixels, but it may be stored as interleaved. (3) In the present embodiment, the recording medium 5 may be composed of an optical disc device. (4) In the present embodiment, the storage device 4 has a capacity of 6
Although the capacity is 4 Mbits, the capacity may be up to 256 Mbits.

[0062]

As described in detail above, according to the present invention, it is possible to provide an image processing apparatus capable of reducing the capacity of the raster block conversion memory in the raster block conversion circuit without decreasing the processing speed.

[Brief description of drawings]

FIG. 1 is a block circuit diagram of an image processing apparatus 1 according to an embodiment.

FIG. 2 is a conceptual diagram of a recording data D2 memory map of a storage device 4.

FIG. 3 is a conceptual diagram of a Cb / Cr detailed memory map of recording data D2 in a storage device 4.

FIG. 4 is an explanatory diagram showing storage addresses of a JPEG macro block and a raster block conversion memory 21 at 4: 2: 0 compression.

FIG. 5 is an explanatory diagram showing a detailed configuration in which a 4: 2: 0 1 MCU is stored in a raster block conversion memory 21.

FIG. 6 is an explanatory diagram showing storage addresses of a JPEG macro block and a raster block conversion memory 21 at the time of 4: 2: 2 compression.

[Explanation of symbols]

1 Image processing device 2 image sensor 3 display devices 4 storage devices 5 recording media 11 Temporary storage means A 12 Image processing means 13 Temporary storage means B 14 Temporary storage means C 15 Encoding means 16 Recording means 17 Mediation means 18 smoothing means R 19 smoothing means M

Continued front page    F term (reference) 5B047 AB04 BA03 BB04 CA23 CB25                       EA01 EB02                 5C022 AA13 AC03 AC42 AC69                 5C052 AA17 DD02 GA02 GA03 GA07                       GB06 GC05 GE06                 5C053 FA08 GB36 KA04 KA24 LA02                 5C073 AA04 BB01 BB09

Claims (4)

[Claims] 1. An image pickup device for photoelectrically converting an optical image of an object, a storage device for temporarily storing various data, and image data output from the image pickup device for temporarily storing the image data in the storage device. Temporary storage means A, image processing means for separating the image data into a component corresponding to a luminance signal and a component corresponding to a color difference signal, and the luminance signal and the color difference signal are temporarily stored in the storage device for recording. And a temporary storage unit C for temporarily storing the luminance signal and the color difference signal in the storage device for display, and a raster block conversion memory for storing the luminance signal and the color difference signal. And a recording means for recording and holding the encoded video data obtained by the encoding means on a predetermined recording medium. Image processing device. 2. The image pickup device according to claim 1, wherein an arbitration unit that arbitrates access to the temporary storage unit A, the temporary storage unit B, the temporary storage unit C, the encoding unit, or the recording unit to the storage device. An image processing apparatus comprising: 3. The image processing apparatus according to claim 1, wherein the raster block conversion memory is an SRAM. 4. The image processing apparatus according to claim 1, wherein the raster block conversion memory is composed of flip-flops.