JP2000354193A - Camera system and video signal processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video signal processing method that can reduce a circuit scale and the power consumption of the camera system, improve the processing speed and reduce the number of times of access to a memory as its contribution. SOLUTION: In the camera system employing a CCD image pickup element 11 for an image pickup device, a memory 14 such as a DRAM directly stores raw data once that are outputted from the CCD image pickup element 11 and converted into digital data by an A/D converter 13 via a correlation double sampling 12 (CCD Raw data). Then the CCD Raw data are read from the memory 14 in the unit of blocks, a camera signal processing section 21 applies camera signal processing to the data and a reduction processing section 22 conducts reduction processing and a compression processing section 23 applies compression processing to the data after the camera signal processing in the unit of blocks.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a camera system and a video signal processing method, and more particularly to a camera system having a processing function such as image compression / decompression, and performing processing such as compression / decompression processing on a video signal. The present invention relates to a video signal processing method.

[0002]

2. Description of the Related Art FIG. 12 is a block diagram showing a conventional example of a camera system using a solid-state imaging device, for example, a CCD (Charge Coupled Device) imaging device as an imaging device.

[0003] A camera system according to this conventional example is a CC system.
D imaging device 101, correlated double sampling (CDS) circuit 102, AD converter 103, line memory 10
4. It has a digital signal processing (DSP) circuit 105 and a memory 106. As shown in FIG. 13, the digital signal processing circuit 105 includes a camera signal processing unit 111, a reduction processing unit 112, a compression / decompression processing unit 113, and the like.

In the conventional camera system having the above-described configuration, the CCD image sensor 101 outputs signals of respective pixels constituting one screen in the order shown in FIG. This C
The output signal of the CD imaging device 101 is digitized by an AD converter 103 after a reset noise generated at the time of reset at an output portion thereof is removed by a correlated double sampling circuit 102, and is output through a line memory 104. It is supplied to the processing circuit 105.

Here, the raw data output from the CCD image pickup device 101 and digitized by the DA converter 103 through the correlated double sampling circuit 102 is converted into CCD raw data.
Shall be referred to as data.

In the digital signal processing circuit 105, the camera signal processing section 111 converts CCD Raw data into luminance data Y and color difference data CbCr (hereinafter referred to as YCbCr data), R (red), G (green), and B (blue) data. Is performed. In the conversion process, in order to improve the image quality, generally, the signals of the upper, lower, left, and right peripheral pixels are used for one pixel signal. Is required, and therefore the line memory 104 is used. Here, the line refers to a pixel row arranged in the horizontal direction.

FIG. 15 shows a specific configuration example of the line memory 104. In this configuration example, for example, a case where pixel data of two lines above and below the pixel of a certain line is used is taken as an example. Therefore, since a total of five lines of pixel data are required, four horizontal delay line memories 121 to 124 are used. Here, the horizontal delay line memory is a memory that can store pixel data for one horizontal line, and is used when pixel data is required in the vertical direction as described above.

Further, not only in the case of camera signal processing in the camera signal processing unit 111 but also in the case of reduction (or enlargement) processing in the reduction processing unit 112, as shown in FIG.
Processing was performed using a line memory for vertical reduction. Here, for example, a case in which pixel data of three lines above and below the pixel of a certain line is used is taken as an example, and therefore six horizontal delay line memories 131 to 136 are used.

[0009]

As described above, in the conventional camera system, camera signal processing and reduction (or
(Enlargement) When performing the processing, pixel data is temporarily stored in the delay line memory, and the processing is performed using the stored pixel data. In the case of the reduction / enlargement processing, about 1 to 20 lines are required. Therefore, the circuit becomes large-scale and the power consumption increases. Memory capacity is a problem.

The compression / decompression processing unit 113
When performing image compression processing such as G or MPEG, data subjected to signal processing by the camera signal processing unit 111 is temporarily stored in the memory 106, and is read out at an address corresponding to the compression processing, thereby performing compression processing. Therefore, processing takes time and the number of accesses to the memory 106 is large, which is a problem. Similarly, when performing reduction / enlargement processing, data that has undergone JPEG or MPEG decompression processing is temporarily stored in the memory 106, and is read out at an address corresponding to the reduction / enlargement processing. The number of accesses to the memory 106 is also a problem.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and has as its object to realize a circuit with a minimum circuit size, greatly reduce power consumption, and further increase processing speed. It is an object of the present invention to provide a camera system capable of improving and greatly reducing the number of times of accessing a memory.

The present invention further provides a video signal processing method capable of contributing to a reduction in the circuit scale and power consumption of the system, a significant improvement in the processing speed, and a significant reduction in the number of accesses to the memory. The purpose is to:

[0013]

A camera system according to the present invention is a camera system using a solid-state imaging device as an imaging device. The camera system includes a memory for storing raw data obtained by digitizing an output signal of the solid-state imaging device. A signal processing circuit that reads out the raw data stored in the memory in block units and performs camera signal processing, and performs at least image reduction and image compression processing on the processed data in block units. It has a configuration.

In the video signal processing method according to the present invention, a digitized video signal supplied from the outside is temporarily stored in a memory, and then read out from the memory in block units to perform camera signal processing. At least image reduction and image compression processing are performed for each data block.

In the camera system or the video signal processing method having the above-described configuration, raw data obtained by digitizing an output signal of a solid-state imaging device or a digitized video signal provided from the outside is temporarily stored in a memory, and then stored in block units. By executing various kinds of signal processing in (1), it is not necessary to use a delay line memory when performing camera signal processing or reduction (or enlargement) processing. Also, the number of accesses to the memory is greatly reduced.

[0016]

Embodiments of the present invention will be described below in detail with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a camera system according to an embodiment of the present invention, and shows a case where the present invention is applied to, for example, a digital still camera.

In FIG. 1, a camera system 10 according to the present embodiment includes a solid-state image pickup device such as a CCD image pickup device 11 as an image pickup device, and correlated double sampling for removing reset noise of an output signal of the CCD image pickup device 11. (CDS) circuit 12, a DA converter 13 for converting the CCD output signal passed through the correlated double sampling circuit 12 into digital data, a memory 14 for temporarily storing CCD Raw data from the DA converter 13, Address control circuit 15 for specifying an address
And a digital signal processing (DSP) 16 for digitally performing various kinds of signal processing.

In the camera system 10 having the above configuration,
As the CCD image pickup device 11, for example, a PS (all pixel independent readout) -IS (interlace) type device is used. Further, as the memory 14, for example, a DRAM
Is used. As the DRAM, for example, a DRAM used for processing of a microcomputer (not shown) that controls the entire system is also used.

The digital signal processing circuit 16 comprises a camera signal processing section 21, a reduction processing section 22, a compression processing section 23, an expansion processing section 24 and the like. In the digital signal processing circuit 16, the camera signal processing unit 21
The CCD Raw data once stored in the memory 14 is read out in units of a predetermined range of blocks under address designation by the address control circuit 15 to perform camera signal processing.
In this camera signal processing, CCD Raw data is converted to YCbC
A process of converting the data into r data or RGB data is performed. The specific processing will be described later in detail.

In the digital signal processing circuit 16, each of the camera signal processing section 21, the reduction processing section 22, the compression processing section 23 and the decompression processing section 24 has a register functioning as a local memory. Since the CCDRaw data is processed in block units by the camera signal processing unit 21, the camera signal processing unit 21 and the reduction processing unit 22 and the compression processing unit 23, and the decompression processing unit 24 and the reduction processing unit Data can be exchanged with the block 22 directly in block units without passing through the memory 14.

Next, a specific operation of each of the processing units 21 to 24 in the digital signal processing circuit 16 will be described.

First, as shown in FIG. 2, the camera signal processing section 21 reads out CCD Raw data once stored in the memory 14 in blocks of the number of horizontal pixels required for processing × the number of vertical pixels required for processing. For each block, for example, one pixel of Y from one row of CCD Raw data of 5 pixels both horizontally and vertically
A camera signal process for generating CbCr data (or RGB data) is performed.

The reading of CCDRaw data in block units during the camera signal processing is performed by the address control circuit 1.
This is performed by specifying the address from 5. Further, the data after the camera signal processing is output in block units of the number of horizontal pixels required for the next processing × the number of vertical pixels required for the next processing.

More specifically, in performing camera signal processing in units of blocks, in order to generate YCbCr data of one pixel from CCD Raw data of five pixels in both horizontal and vertical directions, the left and right blocks adjacent to the block to be processed are required. Pixel data for two pixels in each line and pixel data for two lines in upper and lower blocks are required.

That is, in FIG. 3, in two blocks adjacent to each other on the left and right sides, two pixels adjacent to each other, a total of 4
Pixel data for a pixel (indicated by a dotted area in the drawing) is required for camera signal processing of two blocks on the left and right, and similarly, for two blocks vertically adjacent to each other, a total of two adjacent lines are calculated. Four lines of pixel data (indicated by scattered areas in the figure) are required for the camera signal processing of the upper and lower two blocks.

Therefore, the CC stored in the memory 14
When reading DRaw data in block units,
In FIG. 3, CCDRaw data of a portion overlapping with the adjacent processing block in the horizontal direction (scattered area in the figure) is read twice in total at the time of the current processing block and at the time of the next processing block. On the other hand, CCDRaw data of a portion (scattered area in the figure) overlapping with the adjacent processing block in the vertical direction is stored in a register which is a local memory.

Then, in the camera signal processing unit 21,
1 pixel Y from 5 pixel CCD Raw data in both horizontal and vertical
Camera signal processing for generating CbCr data (or RGB data) is performed in block units. The YCbCr data (or RGB data) generated by the camera signal processing is sequentially output in block units in the order shown in FIG.

The image data in block units output from the camera signal processing unit 21 is directly supplied to the reduction processing unit 22 or the compression processing unit 23 when the reduction processing or the compression processing is performed following the camera signal processing. You. If these processes are not performed subsequently, the image data in block units output from the camera signal processing unit 21 will be stored in the memory 14.

As an example, assuming that compression processing is performed following camera signal processing, YCbCr 4: 2:
Assuming the JPEG image compression of 2 as shown in FIG. 5, as shown in FIG. 5, input data in units of blocks is horizontally 20 pixels and vertical 12 pixels (hereinafter, horizontal x pixels and vertical y pixels are represented by xx y
The CCDRaw data is directly supplied to the compression processing unit 23 as YCbCr data of 16 × 8 blocks.

Upon receiving the YCbCr data of the 16 × 8 block, the compression processing section 23 executes an image compression process by a well-known JPEG compression technique as shown in FIG.
Outputs JPEG compressed data. The JPEG compressed data output from the compression processing unit 23 is stored in the memory 14.

Next, the processing in the reduction processing section 21 will be described.

FIG. 7 is a block diagram showing an example of a specific configuration of the reduction processing section 21. 7, the reduction processing unit 21 has a configuration including a horizontal LPF (low-pass filter) 211, a horizontal thinning circuit 212, a register 213 having a function as a local memory, a vertical LPF 214, and a vertical thinning circuit 215.

Image reduction processing can be realized by thinning out pixel data in the horizontal and vertical directions. However, since image quality deteriorates if pixel data is simply thinned out, generally, an appropriate coefficient is multiplied to pixel data of peripheral pixels by using an LPF, and one pixel is used by using these pixel data. Image quality is improved by performing a filtering process for generating pixel data.

The horizontal LPF 211 and the vertical LPF 214
Are provided to improve the image quality. Then, the horizontal thinning circuit 212 performs horizontal thinning processing on the data of the block that has been subjected to the filtering processing by the horizontal LPF 211. This horizontal LPF 211
Of the data of the block processed by the horizontal thinning circuit 212, pixel data of a pixel adjacent to the block required for the next block is stored in the register 213.

The image data that has been subjected to the horizontal thinning process is subsequently subjected to a filtering process by the vertical LPF 214. In the filtering process performed by the vertical LPF 214, since the image data of the two blocks above and below the block to be processed are also required, the filtering process is performed in three steps on each of the image data of a total of three blocks. Will be

In the vertical LPF 214, the result of the filtering process performed for each step is stored in the memory 14
And is read out from the memory 14 for use in the filtering process in the next step. Then, the vertical thinning circuit 215 performs vertical thinning processing on the data of the block on which vertical filtering processing has been performed by the vertical LPF 214, and outputs the data as final reduced image data.

As an example, as shown in FIG. 8, it is assumed that YCbCr data of a 16 × 8 block is directly input from the camera signal processing unit 21 to the reduction processing unit 22 and the reduction processing is performed by a factor of two. , 8 × 4 blocks are output as reduced image data. The reduced image data output from the reduction processing unit 22 is stored in the memory 1
4 will be stored.

In the above description, it is assumed that the compression processing and the reduction processing are performed separately after the camera signal processing. However, the compression processing and the reduction processing are performed in parallel after the camera signal processing. Of course, it is also possible.

As an example, YCbCr 4: 2: 0 is set to JP
Assuming the EG, as shown in FIG.
4 is read out to the camera signal processing unit 21 in 20 × 20 blocks, and after performing camera signal processing, the camera signal processing unit 21 converts the data into YCbCr data of 16 × 16 blocks, This is given to the compression processing unit 23 in block units. Then, the reduction processing section 22 performs reduction processing according to the above-described procedure and outputs reduced data, and the compression processing section 23 performs compression processing using a known JPEG compression technique and outputs JPEG compressed data.

In the above-described image compression, the case of JPEG has been described as an example. However, in the case of MPEG, similar block compression is used, so that a similar processing method can be used.

In order to improve the processing speed, YCb
In the case of Cr 4: 2: 2, the CCD Raw data is 36 × 12
And outputs as YCbCr data of a 32 × 8 block after camera signal processing, or YCbC
In the case of r4: 2: 0, there is a method in which CCD Raw data is read out in 36 × 20 blocks, and after camera signal processing, it is output as YCbCr data in 32 × 16 blocks to be processed.

That is, in the camera signal processing, when one pixel is generated from horizontal x pixels and vertical y pixels (x = 5, y = 5 in this example), May be processed. Here, n is a positive integer, and in reality, n = 1, 2, 2,
4,8.

The decompression processing unit 24 also performs decompression processing in units of blocks, similarly to the reduction processing unit 22 and the compression processing unit 23. That is, compressed data is read out from the memory 14 in block units and decompressed. At this time, as shown in FIG. 10, the structure in which the reduction processing unit 22 is accepted, in this case, 16 ×
The expanded data of block 8 is generated.

As described above, the solid-state imaging device, for example, C
In a camera system 10 using a CD imaging device 11 as an imaging device, CCDRaw data is directly transferred to a DRAM.
And the like, and then read CCDRaw data from the memory 14 in units of blocks, perform camera signal processing, and perform reduction processing and compression processing on data after camera signal processing in blocks. Thereby, the following operation and effect can be obtained.

That is, by performing signal processing in block units instead of line units, it is not necessary to use a delay line memory when performing camera signal processing or reduction (or enlargement) processing, so that the circuit scale can be reduced. Therefore, power consumption can be reduced by that much, and even if the number of pixels of the CCD image pickup device 11 increases, it is not affected by the increase, and it is possible to correspond to any number of pixels.

Further, since the memory 14 does not intervene in the transition from the camera signal processing to the next processing, the processing speed from the camera signal processing to the compression processing can be greatly improved. The reduction process can be performed while performing the reduction process or the decompression process, so that the number of accesses to the memory 14 can be significantly reduced, and the power consumption of the entire system can be significantly reduced.

FIG. 11 is a block diagram showing the configuration of a camera system according to another embodiment of the present invention applied to, for example, a digital still camera. In the figure, parts that are the same as those shown in FIG. Is shown. The camera system 10 ′ according to the present embodiment includes an AD converter 13 and a memory 14.
, A pre-processing circuit 17 is newly provided, and the rest is the same as the camera system 10 according to the previous embodiment.

The camera system 1 according to the present embodiment
0 'is a camera system having a monitoring function. Incidentally, the frame rate decreases as the number of pixels of the CCD image sensor 11 increases. Therefore,
At the time of the monitoring mode that displays a movie on the monitor,
Since the frame rate needs to be increased, the CCD image pickup device 11 performs a process of thinning out and reading out signal charges in order to obtain a desired frame rate.

In the thinning-out process, when reading out the signal charge of each pixel of the CCD image sensor 11 in the monitoring mode, the reading out of the signal charge in the vertical direction is thinned out at a predetermined rate in line units. The process of thinning out one line is performed. However, this thinning processing is performed only in the vertical direction. Therefore, if monitoring is performed using the image signal as it is,
The screen is compressed only in the vertical direction, and a desired aspect ratio cannot be secured.

Therefore, in the present embodiment, in the monitoring mode, the signals are output from the CCD image pickup device 11 with the thinning in the vertical direction, and the correlated double sampling circuits 12
CCDRa digitized by AD converter 13
The pre-processing circuit 17 performs a thinning-out (reduction) process in the horizontal direction on the data in accordance with the aspect ratio, and then stores the data in the memory 14. That is, the pre-processing circuit 17 functions only in the monitoring mode, and supplies the CCD Raw data to the memory 14 without performing any processing in the normal imaging mode.

Since the CCD image sensor 11 is provided with a color filter of a predetermined color coding in correspondence with one color per pixel, when the pre-processing circuit 17 performs horizontal reduction (decimation). It is necessary to perform the processing in consideration of the color arrangement so that color mixing does not occur in the subsequent camera signal processing. The processing can be realized by using a well-known processing method, and a specific example thereof is not the gist of the present invention, so that the description thereof is omitted here.

Thus, in the monitoring mode,
Since the pre-processing circuit 17 performs horizontal reduction (thinning) processing on CCD Raw data accompanying vertical thinning and then stores the data in the memory 14, the memory 14 has a predetermined aspect ratio. Since the corresponding CCDRaw data is stored, the number of accesses to the memory 14 at the time of subsequent camera signal processing or the like can be greatly reduced, and accordingly, the processing speed can be increased and the overall system can be improved. It can contribute to lower power consumption.

The camera system according to the present embodiment includes a part of an image screen captured by the CCD image sensor 11,
That is, a so-called partial cut-out function for reading out only pixel information of a part of the effective pixel area of the CCD image sensor 11 is also provided. Also in the case of this partial cutout, the respective processes of thinning out in the vertical direction and reduction in the horizontal direction are performed, and the CCDRaw data obtained by performing these processes is stored in the memory 14. Then, based on the CCD raw data, a process of recording a partially cut out image as a still image on a recording medium is performed.

However, when monitoring a partially cut image, thinning in the vertical direction and reduction in the horizontal direction are performed, and if monitoring is performed based on the CCD Raw data stored in the memory 14, the recording is not performed. The image size recorded on the medium does not match the image size on the monitor. Therefore, it is necessary to perform enlargement processing on the CCD raw data so that the clipped image is displayed over the entire screen of the monitor.

In order to cope with this, the pre-processing circuit 17 has a configuration in which the reduction ratio when performing horizontal reduction is variable. Then, in the pre-processing circuit 17, when monitoring a partially cut-out image, the reduction ratio is set to be smaller than the reduction ratio when recording as a still image. The reduction ratio at this time is determined according to the horizontal (horizontal) screen size of the monitor.

In this way, when monitoring a partially cut-out image, this is equivalent to CCDR data for that image being enlarged in the horizontal direction in the pre-processing circuit 17 and stored in the memory 14. Therefore,
Even when monitoring is performed based on the CCD Raw data stored in the memory 14, a partially cut out image is displayed on the entire screen in the horizontal direction (horizontal direction) on the monitor.

At this time, in the vertical direction, the memory 1
For example, the CCD Raw data stored in No. 4 is repeatedly read out at a rate corresponding to the screen size in the vertical direction (vertical direction) of the monitor, for example, two lines every three lines. This is equivalent to magnifying the cut-out image in the vertical direction, and a partially cut-out image is displayed on the entire screen in the vertical direction on the monitor. As a result, the image size recorded on the recording medium and the image size on the monitor can be matched.

In each of the above embodiments, in a camera system using the CCD image pickup device 11 as an image pickup device, a video signal (C
Although the case of processing (CDRaw data) has been described, the same processing can be applied not only to processing of video signals in a camera system but also to image data stored in a memory or the like in advance. Therefore, the output data may be stored in the memory or output to the outside via the output terminal.

In each of the above embodiments, the system configuration is realized by hardware as shown in FIGS. 1 and 11, but the same function can be realized by software. If implemented by software, CC
Once the raw data is directly stored in the cache memory, the CCD raw data is read out from the memory in units of blocks to perform camera signal processing, and the data after the camera signal processing is subjected to reduction processing and compression processing in units of blocks. Thus, the used capacity of the cache memory can be reduced, the number of memory accesses can be reduced, and the processing speed can be improved.

[0060]

As described above, according to the present invention,
By storing the raw data obtained by digitizing the output signal of the solid-state imaging device or the digitized video signal given from the outside in a memory such as a DRAM, and then executing various processes in block units Since it is not necessary to use a delay line memory when performing camera signal processing or reduction (or enlargement) processing, the circuit scale can be reduced, and power consumption can be reduced accordingly.
Further, the processing speed can be greatly improved, and the number of accesses to the memory can be significantly reduced.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a camera system according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating the size of input / output block data in camera signal processing.

FIG. 3 shows a CC read from a memory during camera signal processing.
It is a figure which shows the state of DRaw data.

FIG. 4 is a diagram illustrating a state of output data after camera signal processing.

FIG. 5 is a diagram illustrating the size of input / output data blocks in camera signal processing when assuming JPEG image compression of YCbCr 4: 2: 2.

FIG. 6 is a diagram showing the size of an input / output data block in a compression process.

FIG. 7 is a block diagram illustrating an example of a specific configuration of a reduction processing unit.

FIG. 8 is a diagram showing the size of an input / output data block in a reduction process.

FIG. 9 is a diagram illustrating the size of input / output data blocks in camera signal processing, compression processing, and reduction processing assuming JPEG of YCbCr 4: 2: 0.

FIG. 10 is a diagram showing the size of an input / output data block in a decompression process.

FIG. 11 is a block diagram showing a configuration of a camera system according to another embodiment of the present invention.

FIG. 12 is a block diagram illustrating a conventional example of a camera system.

FIG. 13 is a block diagram illustrating a configuration of a digital signal processing circuit.

FIG. 14 is a diagram showing the order of CCD output signals for one screen.

FIG. 15 is a block diagram illustrating a configuration example of a camera signal processing line memory.

FIG. 16 is a block diagram illustrating a configuration example of a line memory for reduction (or enlargement) processing.

[Explanation of symbols]

10, 10 ': camera system, 11: CCD image sensor, 12: correlated double sampling (CDS) circuit, 13
... A / D converter, 14 ... Memory, 15 ... Address control circuit, 16 ... Digital signal processing (DSP) circuit, 17 ...
Pre-stage processing circuit, 21: camera signal processing unit, 22: reduction processing unit, 23: compression processing unit, 24: expansion processing unit

Claims (6)

[Claims] 1. A camera system using a solid-state imaging device as an imaging device, comprising: a memory for storing raw data obtained by digitizing an output signal of the solid-state imaging device; and a raw data stored in the memory. A camera system comprising: a signal processing circuit which performs camera signal processing by reading out data in units of blocks and performing at least image reduction and image compression processing on the processed data in units of blocks. 2. The camera system according to claim 1, wherein a memory used for processing of the entire system is used as said memory. 3. The camera according to claim 1, wherein each processing unit in the signal processing circuit has a register, and information necessary for processing of the next block is stored and held in each register. system. 4. A pre-processing circuit for performing a horizontal reduction process on raw data obtained by digitizing an output signal of the solid-state imaging device in a monitoring mode and supplying the raw data to the memory. The camera system according to claim 1, wherein 5. A monitoring method according to claim 1, wherein a reduction ratio of said pre-processing circuit is variable, and when monitoring a partially cut-out image when reading out only pixel information of a part of an effective pixel area of said solid-state imaging device, 5. The camera system according to claim 4, wherein a reduction ratio of the pre-processing circuit is set to be smaller than a reduction ratio when the partially cut-out image is recorded as a still image. 6. A digital video signal supplied from the outside is temporarily stored in a memory, read out in blocks from the memory and subjected to camera signal processing, and the processed data is processed at least in blocks. A video signal processing method characterized by performing image reduction and image compression processing.