JP2006109222A - Signal processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To prevent competition between image data buses without complicating a system. <P>SOLUTION: This apparatus has an imaging device, an image input means for inputting an image from the imaging device, a plurality of signal processing means, a memory for holding processed image data, and a bus arbitrating means for managing access from the imaging device and the plurality of signal processing means other than that to the memory. The apparatus controls the driving of the imaging device, depending on a state of competition between buses. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an imaging apparatus, and more particularly to an imaging apparatus having a control method for improving the transfer efficiency of image data.

  In recent years, the resolution of image data has greatly increased along with the dramatic technological advancement of image sensors. The bus to which the image data processing system belongs becomes congested along with it, and there are problems such as poor CPU response. On the other hand, some image data buses are managed so as to be within the transfer band according to the mode. (For example, refer to Patent Document 1).

In addition, CMOS sensors are also widely used as image sensors. One feature of the CMOS area is that relatively free image data access is possible. Taking advantage of this feature, it has been proposed to synchronize with a coding sequence performed later. (See Patent Document 2)
FIG. 5 shows an example of the system configuration of an imaging apparatus that can be considered at present. Image data imaged by the lens is accumulated as electric charges in the CMOS image sensor (501), and is input to the signal processing unit (505) as digital data via the CDS (502) and AD (503). In the signal processing unit, each process is performed via the SDRAM (504) and output to the outside.

The FIFO (505_09) and its control unit (505_10) that receive the image data at the front stage in this signal processing block part are necessary to match the sensor reading rate and the arbitration processing rate of access to SDRAM. is there.
JP 2000-92365 A Japanese Patent No. 5-326701

  The resolution of the image sensor has increased significantly, and this trend is expected to continue. In a video camera, there is a restriction that processing must be completed within a time of one field or one frame, and it may be impossible to establish a system due to the performance of the image data bus. In order to solve this problem, it would be indispensable to equip multiple buses and alleviate bus contention as a countermeasure, but this is becoming more complex, larger-scale, and more verifiable. It brings about the harmful effect of deterioration.

  There is also a problem with the FIFO (505_09) that first receives the image data in the system configuration given as the conventional example. This FIFO is highly dependent on sensor resolution or processing rate and needs to be reviewed each time the system is modified. If it is necessary to increase the size of the FIFO, the impact on the scale and cost of the IC is significant.

  The present invention is characterized by the following.

  1) Managing access to the memory from the image sensor and the image input means from the image sensor, a plurality of signal processing means, a memory for holding the processed image data, and the image sensor and other signal processing means. An image pickup apparatus having bus arbitration means and controlling the image pickup element drive according to a bus competition state.

  2) A feature in which the memory holding the input from the image sensor selects a timing that does not compete with the access from the plurality of signal processing means, and the image data is read from the image sensor and the image data is written to the memory. 1) The imaging apparatus according to 1).

  3) The image sensor is driven according to the competition state of the memory, except for the period that is prohibited for driving the image sensor, as an image data transferable area from the image sensor to the memory. Imaging device.

  4) In the image transferable area from the image sensor to the memory, the size of the image data held by the image sensor and transferable to the outside is variably controlled as the maximum transfer unit to the memory. The imaging device described in the above.

  According to the present invention, it is possible to improve the data bus transfer efficiency without complicating the use of a plurality of buses as a system for the following reasons.

  * It is possible to avoid the congestion period of other ports by feeding back the traffic condition of the image data bus directly to the sensor drive and controlling the timing.

  * The bus transfer efficiency is improved by variably controlling the amount of data that can be transferred at that time as the maximum burst length to SDRAM during the charge read-out period for sensor driving.

  In addition, with this configuration, sensor and SDRAM rate matching can be realized by simple control, so there is no need for FIFO, and scale and development costs can be expected to increase.

Example 1
The configuration of a system for carrying out the present invention is shown in FIG.

  As a whole, it is composed of a CMOS image sensor (101), a CDS (102), an AD (103), a TG (104), a signal processing unit (105), and a plurality of SDRAMs (106).

  Image data imaged by the lens is stored in a CMOS image sensor. The driving of the CMOS image sensor is controlled by a pulse output from TG. The accumulated charge is sequentially output to the CDS by controlling the pulse from the TG. In this embodiment, the output is described as one output (one channel), but it is a multi-channel output, and the transfer rate can be further improved. Image data that has become a digital signal through CDS and AD conversion is input to a signal processing block. The signal processing unit performs signal processing according to each mode, and finally plays back from the video IF through the SDRAM, and records to the card or tape from the external media IF. Also, image data can be input from the external medium, reprocessed, and recorded and reproduced again.

  Each block of the signal processing unit will be described. The CPU (105_02) realizes high speed by overall sequence control or cache. The camera DSP (105_07) performs such signal processing on the input image. Resizing (105_03) changes the size of the input image from the image sensor according to each application and supplies it to the next processing. NR (105_08) is for removing noise in the input image. The effect (105_04) performs effect processing or composition processing of an image. JPEG (105_09) compresses and decompresses images. The external media IF (105_05) and the video IF (105_06) are for inputting and outputting processed images to the outside.

  The bus arbitration means arbitrates the request of each block when holding and reading data to the SDRAM in accordance with the processing sequence in each block of the signal processing unit, and performs transfer with the SDRAM (106). .

  The bus arbitration may be congested due to an increase in image size and a complicated sequence. The charge readout timing of the CMOS image sensor (101) is controlled by directly feeding back and controlling the traffic situation to the TG (104).

  FIG. 2 illustrates the adverse effects that occur when the bus arbitration to SDRAM of each port is congested. For example, when the sensor is a CCD, the charge readout timing from the CCD is linked with the charge accumulation, and it is necessary to drive with a certain period. In such a case, the retention of image data in the SDRAM also needs to end a certain amount of data during a certain period. FIG. 2-1 shows a normal transfer mode for such a port that needs to transfer a certain amount of data at a certain period. First, the transfer cycle starts when a transfer request is sent to the arbitration unit. From the time when a request is issued until it is granted (that is, when the transfer of the contention port is completed), a wait period is set, and then a command is issued to the SDRAM to start burst transfer. The burst length is preset and varies depending on each port and each mode. The period from the end of burst transfer to the end of the transfer cycle is the margin of the SDRAM bus. Figure 2-2 shows a case where the transfer wait of the other port is long and burst transfer does not end within the transfer period. In this case, a request for the next transfer data starts during the burst transfer, and the transfer operation of this port does not operate correctly or the normal operation is not guaranteed.

  FIG. 3 illustrates driving of the CMOS area sensor. The charge remaining in the photodiode is reset by a reset pulse from TG. In the case of a CMOS image sensor, since the reset pulse is supplied by shifting, a time lag occurs in each scanning line (period C in the figure). After resetting, the shutter is opened and charge accumulation is started. This shutter opening period (D period in the figure) is switched by the imaging mode as the shutter time. After the charge is accumulated, the accumulated charge is read out during period A in the figure. Also at this time, since the drive pulse from the TG is shifted and controlled, a time lag of each scanning line occurs.

  A feature of the CMOS area sensor is that it can be easily mixed with other circuits. For example, if the holding means and its control unit are mounted together, it is possible to hold the charges read in the A period for all pixels until the next field charge reading. In that case, it is possible to transfer all the pixels held in the B and D periods except the A and C periods to the SDRAM. This means that SDRAM transfer from the CMOS area sensor is very flexible. Regarding the burst length, it is possible to set all held pixels as the maximum.

  FIG. 4 shows the case where the transferable burst length is increased. FIG. 4 shows a case where the data transferred in two times is compared with the data that has been completed in one time by doubling the burst length. As can be seen from the figure, the margin increases. Improvement of transfer efficiency can be realized.

  FIG. 6 is a timing chart illustrating the SDRAM bus contention and the CMOS sensor drive sequence. As can be seen from the figure, the image data from the sensor is not competing with the SDRAM bus, and the data is efficiently written to the SDRAM at a timing that avoids the area that is prohibited for sensor driving.

The system configuration of the present invention. About SDRAM burst transfer. About CMOS area sensor drive. About the effect of increasing the burst length. Conventional system configuration. About SDRAM and sensors.

Claims (4)

An image sensor and image input means from the image sensor;
A plurality of signal processing means;
A memory for holding the processed image data;
Has bus arbitration means for managing access to the memory from the image sensor and other signal processing means,
An image pickup apparatus that controls the image pickup element drive in accordance with a bus competition state.   The memory that holds the input from the image sensor selects a timing at which the memory does not compete with the access from the plurality of signal processing means, and the image data is read from the image sensor and the image data is written to the memory. Item 2. The imaging device according to Item 1.   2. The image pickup device is driven according to a race condition of the memory, except for a period prohibited for driving the image pickup device, as an image data transferable area from the image pickup device to the memory. Imaging device.   2. The image transferable area from the image sensor to the memory, wherein the size of the image data held by the image sensor and transferable to the outside is variably controlled as a maximum transfer unit to the memory. The imaging device described.

Images