JP2001022922A - Image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processor which transfers image data, without needing a complicated control means with a simple bus configuration and enabling image processing in real time. SOLUTION: This processor consists of a frame memory 107 for storing the vertical scanning period part of a video signal from an image pickup device 101, a buffer 104 for storing the horizontal scanning period part of the video signal, a timing generator 102 generating a timing signal showing the horizontal and vertical scanning of the video signal, a CPU 106 performing prescribed image processing of the video signal, a data bus 105 connecting the image pickup device, the frame memory and the CPU, and a memory controller 103 transferring the video signal from the buffer to the frame memory and the CPU performing image processing within the blanking period of the video signal.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus in which pixel data is transferred during a scanning retrace period of a video signal.

[0002]

2. Description of the Related Art Conventionally, as an image processing apparatus, for example,
In Japanese Patent Application Laid-Open No. Hei 7-284054, as shown in FIG. 9, a device for image processing is connected to the main bus master 1101, and image data for one frame is stored in the memory 1102. And a memory 1102 that can be switched by a sub bus master 1103 are disclosed. And the one of this configuration is
The sub bus master 1103 is controlled so that the main bus master 1101 can manage the memory 1102 only during the blanking period.
1103 is memory 1102, main bus master 1101 is memory
It manages 1104 and memory 1105. Also,
By switching the bus during the blanking period, it becomes possible to write the image signal to the memory and to output the data of the memory to the monitor or the like, and while the bus is disconnected, the bus of the main bus master is switched to the blanking period. , So that other devices can access it.

For example, Japanese Patent Application Laid-Open No. 8-63136 discloses that when a slew rate of an external device for performing horizontal blanking transfer is set as shown in FIG. The horizontal blanking width is set based on the amount of data.
At the same time, the amount of image data to be transferred in one horizontal blanking period is set in the number-of-transfers-per-blank setting register 1203. The horizontal blanking signal generation circuit 1204 variably controls the blanking period to the set blanking width, and a memory in which the set amount of image data is inserted and transferred during the horizontal blanking period. An image data transfer control device configured to output the control data described above is disclosed. In this configuration, the image data written in the general-purpose memory is read, and the data is transferred to one external device during the effective image period, and the image data is transferred to the other external device during the blanking period. Performs blanking transfer to transfer data, thereby enabling horizontal blanking transfer to improve the response of transfer requests from external devices and buffer memory for time axis adjustment. Can be reduced or reduced.

[0004]

In Japanese Patent Application Laid-Open No. Hei 7-284054, a sub-bus master for transferring image data from an image pickup means to a frame memory, and an image processing device for performing image processing from the frame memory. Means for switching the control of the main bus master for transferring image data or data after image processing to another memory in a blanking period is disclosed, but the method of transferring an image signal from the imaging unit to the frame memory is disclosed. It does not mention how to implement the transfer. Further, although the above-mentioned publication discloses means for configuring two buses for storing image data in a frame memory and performing image processing on the data, the bus configuration is simplified to correspond. No perspective is mentioned.

The present invention has been made in view of this point of view, and the invention according to claims 1 and 2 performs image data transfer with a simple bus configuration without the need for complicated control means and real-time transfer. It is an object of the present invention to provide an image processing apparatus capable of performing image processing on a computer.

Japanese Patent Application Laid-Open No. Hei 8-63136 discloses that image data written in a general-purpose memory is read out, image data is transferred to one external device during a valid image period, and the other external device is read. For this, a blanking transfer unit for transferring image data during a blanking period and a unit for arbitrarily setting a blanking period by a blanking period variable unit based on the transfer amount of the image data are disclosed. However, as described in JP-A-7-28
As in the case of Japanese Patent No. 4054, there is no mention of how to transfer an image signal from an imaging unit to a general-purpose memory. In recent years, with the progress of semiconductor technology, it has become possible to realize the imaging means, the storage means, and the image processing means on the same chip. Not.

The present invention has been made with a focus on this point of view. The invention according to claims 3 to 6 realizes an image pickup means, a storage means and a transfer means on the same chip, and has a simple bus structure and a complicated structure. It is an object of the present invention to provide an image processing apparatus capable of transferring image data without requiring any control means and realizing image processing in real time.

[0008]

According to a first aspect of the present invention, there is provided an image processing apparatus for performing predetermined processing on a video signal forming an image by performing horizontal scanning and vertical scanning. An image sensor that sequentially selects a pixel array in which pixels capable of accumulating charges generated by light irradiation by the scanning and outputs the image signal, and stores the image signal output from the image sensor for at least one vertical scanning period A first storage unit, and a second storage unit that stores a video signal output from the image sensor for at least one horizontal scanning period.
A storage unit, a control unit that generates a timing signal indicating horizontal scanning and vertical scanning of a video signal output from the imaging element, and an image processing unit that performs predetermined image processing on image data obtained by the video signal A data bus connecting the image sensor, the first storage unit, and the image processing unit; and a transfer of a video signal from the second storage unit to the first storage unit and the image processing unit. And a transfer unit that performs the scanning within a blanking period of the scanning signal generated by the unit.

In the image processing apparatus thus configured, during the horizontal retrace period of the video signal, the image from the imaging unit including the imaging element, the second storage unit, the control unit, and the transfer unit to the first storage unit is stored. Signal transfer is performed, and during this time, the data bus is occupied by the imaging unit. On the other hand, during the valid period of the video signal, data transfer of the video signal from the first storage unit to the image processing unit or from the image processing unit to the first storage unit is performed, and during this time, the data bus is occupied by the image processing unit. You. As described above, the data bus occupation period of the imaging unit and the image processing unit is switched according to the blanking period and the valid period of the video signal, so that the image processing apparatus can be configured with a single data bus. Also, a general-purpose C
In a system employing a PU or the like, image data can be transferred by a DMA function or the like mounted as a peripheral circuit of a general-purpose CPU, and an image processing apparatus can be configured without requiring an external special circuit. can do.

According to a second aspect of the present invention, in the image processing apparatus according to the first aspect, a scanning cycle of the image sensor in the scanning period is different from a transfer cycle from the image sensor to the first storage unit. It is characterized by being constituted.

[0011] In the image processing apparatus configured as described above, similarly to the first aspect, the image signal, the second storage unit, the control unit, and the transfer unit are included in the horizontal retrace period of the video signal. The video signal is transferred from the imaging unit to the first storage unit, and during this time, the data bus is occupied by the imaging unit. On the other hand, during the valid period of the video signal, data transfer of the video signal from the first storage unit to the image processing unit or from the image processing unit to the first storage unit is performed, and during this time, the data bus is occupied by the image processing unit. You. Further, the data transfer via the data bus is performed at a cycle corresponding to the bus width and the capabilities of the transfer unit and the control unit, different from the scan cycle of the image sensor determined by the frame rate or the like. As described above, since the data bus occupation period of the imaging unit and the image processing unit is switched depending on the retrace period and the valid period of the video signal, the image processing device can be configured with a single data bus. In addition, the scanning period of the video signal during the valid period of the video signal, the transfer period of the video signal to the first storage unit during the retrace period, the bus width, the number of transfers of reading and writing, and the access speed of the storage unit By making it arbitrarily set correspondingly, it is possible to match a desired frame rate.

According to a third aspect of the present invention, there is provided an image processing apparatus for performing predetermined processing on a video signal forming an image by performing horizontal scanning and vertical scanning. 2 pixels that can be read as
A pixel array arranged and arranged in a dimension, a scanning unit for performing horizontal scanning and vertical scanning on the pixel array, an A / D converter for converting the video signal into a digital signal, and an output from the pixel array. A control unit that generates a timing signal indicating horizontal scanning and vertical scanning of a video signal, a second storage unit that holds an effective image signal during a horizontal scanning period, and a second storage unit based on an output of the control unit. A solid-state imaging unit comprising a transfer unit for writing and outputting the effective image signal and reading and outputting the effective image signal held in the second storage unit during the horizontal retrace period of the video signal on the same chip;
The video signal output from the solid-state imaging unit is at least one
A first storage unit that stores and holds a vertical scanning period, an image processing unit that performs predetermined image processing on an image obtained by the video signal, the solid-state imaging unit, the first storage unit, and the image processing unit And a data bus for transferring a video signal from the solid-state imaging unit to the first storage unit and the image processing unit within a blanking period of a scanning signal generated by the control unit. The present invention is characterized in that it is configured to be implemented via

In the image processing apparatus thus configured, the image signal read from the pixel array during the effective period of the image signal in the horizontal scanning period is temporarily stored in the second storage unit. Next, the image signal stored in the second storage unit is read by an external control circuit connected to the solid-state imaging unit during a horizontal retrace period of the video signal. As described above, the solid-state imaging unit includes the pixel array, the scanning unit, the control unit, the second storage unit, and the transfer unit that performs writing and reading, and has a configuration that can transfer the video signal to the first storage unit during the horizontal retrace period. This eliminates the need for an external circuit conventionally required when transferring the video signal of the imaging unit to the first storage unit. Therefore, the solid-state imaging unit, the first storage unit, and the image processing unit can be connected by the same data bus.

According to a fourth aspect of the present invention, in the image processing apparatus according to the third aspect, the solid-state imaging section performs serial-to-parallel conversion on pixel data of a video signal read from the pixel array during a horizontal scanning period. A conversion unit is provided.

In the image processing apparatus thus configured, the video signal read from the pixel array during the effective period of the video signal in the horizontal scanning period is subjected to serial-parallel conversion by the conversion unit, and then temporarily stored in the second storage unit. Is done. next,
The video signal stored in the second storage unit is read by an external image processing unit connected to the solid-state imaging unit during a horizontal retrace period of the video signal. As described above, by mounting the pixel array, the scanning unit, the control unit, the second storage unit, the conversion unit, and the transfer unit that performs writing and reading as the solid-state imaging unit,
Conventionally, an external circuit, which has been required when transferring the video signal of the imaging unit to the first storage unit, is not required. Therefore,
The solid-state imaging unit, the first storage unit, and the image processing unit can be connected by the same data bus. Further, since the video signal is serial-parallel converted by the conversion unit, the transfer efficiency when transferring data to the external device during the horizontal retrace period of the video signal can be increased, and as a result, the transfer cycle can be increased. Can be.

According to a fifth aspect of the present invention, in the image processing apparatus according to the third or fourth aspect, the solid-state imaging unit has a communication for enabling an upper controller to set an operation mode of the control unit and the transfer unit. A transfer unit that transfers image data of the video signal according to a bit width of a data bus that connects the first storage unit and the image processing unit based on an output of the communication unit. It is characterized by being constituted.

In the image processing apparatus configured as described above, the video signal read from the pixel array during the effective period of the video signal in the horizontal scanning period is subjected to serial-parallel conversion by the conversion unit, and then temporarily stored in the second storage unit. Is done. next,
The video signal stored in the second storage unit is read by an external image processing unit connected to the solid-state imaging unit during a horizontal retrace period of the video signal. When transferring the video signal, the control unit and the transfer unit are controlled in accordance with the operation mode set in advance by the host controller via the communication unit, and the data bus connecting the solid-state imaging unit and the external image processing unit is controlled. Transfer according to the bit width.

As described above, by mounting a pixel array, a scanning unit, a control unit, a second storage unit, a conversion unit, and a transfer unit for performing writing and reading as a solid-state imaging unit,
The external circuit required when transferring the video signal of the imaging unit to the first storage unit is not required. Therefore, the solid-state imaging unit, the first storage unit, and the image processing unit can be connected by the same data bus. Further, since the video signal is serial-parallel converted by the conversion unit, the transfer efficiency when transferring data to the external device during the horizontal retrace period of the video signal can be increased, and as a result, the transfer cycle can be increased. Can be. Further, since the video signal can be transferred in accordance with the bit width of the external data bus, the range of selection of the first storage unit, the image processing unit, and the like is widened.

According to a sixth aspect of the present invention, in the image processing apparatus according to any one of the third to fifth aspects, the solid-state imaging unit can set an operation mode of the control unit and the transfer unit from a host controller. A communication unit for setting the conversion period of the serial-parallel conversion unit and the blanking period of the video signal based on the output of the communication unit. is there.

In the image processing apparatus configured as described above, the video signal read from the pixel array during the effective period of the video signal in the horizontal scanning period is subjected to serial-parallel conversion by the conversion unit, and then temporarily stored in the second storage unit. Is done. next,
The video signal stored in the second storage unit is read by an external image processing unit connected to the solid-state imaging unit during a horizontal retrace period of the video signal. Then, the video signal is transferred according to the bit width of the data bus connecting the solid-state imaging unit and the external image processing unit. Also, the blanking period in the horizontal scanning and vertical scanning periods of the video signal is arbitrarily set.

As described above, by mounting a pixel array, a scanning unit, a control unit, a second storage unit, a conversion unit, and a transfer unit that performs writing and reading as a solid-state imaging unit,
The external circuit required when transferring the video signal of the imaging unit to the first storage unit is not required. Therefore, the solid-state imaging unit, the first storage unit, and the image processing unit can be connected by the same data bus. Further, since the video signal is serial-parallel converted by the conversion unit, the transfer efficiency when transferring data to the external device during the horizontal retrace period of the video signal can be increased, and as a result, the transfer cycle can be increased. Can be. Further, since the video signal can be transferred in accordance with the bit width of the external data bus, the range of selection of the first storage unit, the image processing unit, and the like is widened. In addition, since a communication unit that can arbitrarily set a horizontal retrace period and a vertical retrace period of a video signal is provided, even when the access speed of the first storage unit or the image processing unit is low, the transfer time is adjusted to the speed. Can be secured.

[0022]

DESCRIPTION OF THE PREFERRED EMBODIMENTS (First Embodiment) Next, an embodiment will be described. FIG. 1 is a block diagram showing a first embodiment of the image processing apparatus according to the present invention.
An image processing apparatus according to the present embodiment corresponds to the invention according to claims 1 and 2, and includes an image pickup device and a CP.
U, the frame memory is connected via the same bus, and the transfer of the image data output from the image sensor to the frame memory or the CPU is performed using the horizontal retrace period of the video signal. It is configured as follows.

In FIG. 1, reference numeral 101 denotes a CCD and a CMOS.
This is a solid-state imaging device represented by an image sensor. Ten
Reference numeral 2 denotes a timing generator, which controls the accumulation time of the photoelectric conversion element in the image sensor 101, the scanning timing of the horizontal scanning circuit and the vertical scanning circuit, the correlated double sampling timing of accumulated charges, and the effective image output during each scanning period. It generates a control signal for timing and the like, and supplies its output to the image sensor 101 and the memory controller 103. The memory controller 103 has an image sensor
101 is an SRAM or DRA capable of temporarily storing image data of a video signal output during one horizontal scanning period.
A buffer 104 composed of an M and the like and a system bus 105 in the image processing apparatus are connected, and data is transferred between the buffer 104 and the system bus 105 in accordance with a control signal supplied from the timing generator 102. The system bus 105 is connected to a CPU 106 for performing a desired image processing operation and a frame memory 107 composed of an SRAM or a DRAM. In addition,
An image sensor according to claim 1 or 2, a frame memory 107 as a first storage unit, and a CP as an image processor.
U106, the control unit includes a timing generator 102, a second
A buffer 104 corresponds to the storage unit, a memory controller 103 corresponds to the transfer unit, and a system bus 105 corresponds to the data bus.

Next, the operation of the first embodiment configured as described above will be described with reference to the timing charts shown in FIGS. In FIG. 2, HD is a horizontal scanning synchronizing signal generated by the timing generator 102 or input from the outside in order to perform horizontal scanning of the image sensor 101. HDISP is a horizontal scanning synchronization signal H
This signal is generated by the timing generator 102 in synchronization with D and indicates the valid period and the non-valid period of the image data output from the image sensor 101 during the horizontal scanning period. First, in the valid period of image data (HDISP = 1) in the horizontal scanning period, image data (Imager Out) output from the image sensor 101 is sequentially transferred to the buffer 104 via the memory controller 103, and during one horizontal scanning period. Are all temporarily written into the buffer 104. During this period, the data input (Buffer
Data In) becomes active each time image data is input.

Next, scanning of the effective pixel area of the image sensor 101 is completed, and the non-effective period (HDISP =
0), the effective image data stored in the buffer 104 during one horizontal scanning period is transferred to the frame memory 107 via the bus 105 in parallel with being read by the memory controller 103. When transferring, the CPU
In step 106, a DMA function mounted as a peripheral circuit is used. During this period, the data output (Buffer
er Data Out) becomes active. Therefore, bus 10
The bus master No. 5 has C during the effective period of the horizontal scanning period.
The PU 106 is activated (CPU Bus Enable = 1), and the memory controller 103 is activated during the non-effective period (Memory Cont. B).
us Enable = 1).

FIG. 3 shows more detailed timing of data transfer during one horizontal scanning period. Signals having the same functions as the signals in the timing chart shown in FIG. 2 are given the same signal names, and differences from FIG. 2 will be described. In FIG. 3, CPU Clock is C in FIG.
A clock signal that determines the operation of the PU 106;
Pixel Clock is a clock signal having a period equal to the scanning clock in a horizontal scanning period synchronized with image data output from the image sensor. In general, Pixel Clock is determined by the frame rate and the number of pixels of the image sensor,
It is asynchronous with CPU Clock and has a different cycle, and its cycle is longer than CPU Clock.

By the way, during the effective period of the image data (HDISP = 1) in the horizontal scanning period, the image data (Imager Out) output from the image sensor 101 is a pixel clock.
Therefore, writing to the buffer 104 by the memory controller 103 is performed in synchronization with the Pixel Clock. When all the effective image data during one horizontal scanning period is accumulated in the buffer 104, the memory controller 103
A DMA transfer request signal (XDMA Req.) Is applied to U106. When the CPU 106 receives the DMA transfer request signal, terminates the ongoing processing, and allows the DMA transfer, returns the DMA transfer permission signal (XDMAAck.) To the memory controller 103 and sends the signal from the image sensor 101 to the frame memory 107 or the CPU 106. Start the DMA transfer. Prior to the DMA transfer, it is necessary to set in advance the number of times of transfer of the image data stored in the buffer 104, the transfer source and the transfer destination, and various parameters relating to the DMA transfer mode.

FIG. 4 is a timing chart for explaining a real-time operation executable in the image processing apparatus having the configuration shown in FIG. The operation shown in the timing chart of FIG. 4 can be realized by the system configuration shown in FIG. 1, and signals having the same functions as the signals shown in FIG. 2 are given the same signal names. The difference from the timing chart shown in FIG. 2 will be described.

In FIG. 4, VD and HD are imaging elements 10
These are a vertical scanning synchronization signal and a horizontal scanning synchronization signal generated by the timing generator 102 or input from the outside to perform one vertical scanning and horizontal scanning. Also,
Imager Enable is a drive control signal for the image sensor 101,
In the present embodiment, it is always active (Imager Enable =
1). In the present embodiment, by providing Bank0 and Bank1 as two memory-mapped access areas in the frame memory 107, a frame memory area for two frames is secured.

At time T (N + 1), when the image data (Imager Out) output from the image sensor 101 is Frame (N + 1), the CPU 106 utilizes the image data ineffective period in the horizontal scanning period. And store the image data Frame (N + 1) in the frame memory.
Write to Bank1 in 107. On the other hand, during the image data valid period within the horizontal scanning period, Bank 0 in the frame memory 107 is
Stores the image data Frame (N) written during the image data non-valid period in the horizontal scanning period at time T (N).
0 image data Frame (N) (CPU Acces
s) and perform image processing.

According to the above configuration, the memory controller
103 becomes the bus master and occupies bus 105
In the image data non-valid period during the horizontal scanning period, that is, only during the horizontal retrace period of the video signal, and during the other image data valid periods during the horizontal scanning period and the vertical scanning period, CP
U106 can become the bus master and occupy bus 105. Therefore, it is possible to realize a real-time image processing system that sequentially performs real-time image processing on image data captured by the image sensor for each vertical scan with a relatively simple configuration.

Further, by switching the bus occupation right of the image sensor and the image processing unit (CPU) in accordance with the horizontal blanking period of the video signal, the image data captured by the image sensor for each vertical scan is sequentially and in real time. A real-time image processing system that performs image processing can be realized with a relatively simple configuration.

Furthermore, by realizing the image sensor 101, the timing generator 102, the memory controller 103, and the buffer 104 having the functions described above on the same chip, the number of components can be greatly reduced, and the external bus width can be reduced. By realizing on the same chip including the expanded CPU 106 and the frame memory 107, a high-speed image processing system with an improved bandwidth on the bus 105 can be realized.

(Second Embodiment) Next, a second embodiment of the image processing apparatus according to the present invention will be described with reference to the block diagram of FIG. This embodiment is described in claims 3 to
The solid-state imaging unit 301 corresponds to the first embodiment.
A communication unit (SIO) 108 is arranged on the same chip in addition to the image pickup device 101, the timing generator 102, the memory controller 103, and the buffer 104 in the embodiment, and the CPU 106 and the first frame memory 107 Are connected via the same bus 105. Further, the image pickup device 101 is a pixel (pixel) 201.
Are arranged two-dimensionally, and the number of effective pixel arrays is 38
4 (H) × 288 (V), the output of the ADC (A / D converter) 212 is 8 bits, and the bus width of the bus 105 is 32 bits.

Next, the configuration of the second embodiment will be described in more detail with reference to the timing chart shown in FIG. Pixels 201 constituting the image sensor 101 are photoelectric conversion elements capable of reading out charges generated and accumulated by light irradiation as optical signals, and are arranged two-dimensionally in a matrix. 202 is a vertical scanning shift register (VS)
R), vertical scanning is started by a vertical scanning start signal (V-ST) 204 output from the timing generator 102, and a row selection signal 206 is sequentially output to the pixel 201 every horizontal scanning. Reference numeral 203 denotes a horizontal scanning shift register (H-SR), which starts horizontal scanning by a horizontal scanning start signal (H-ST) 205 output from the timing generator 102, and outputs a column selection signal 207 to the pixel 201.
Are sequentially output each time the charge is read. The column selection signal 207 is connected to the gate of a MOS transistor 208 having a pixel charge readout switch function. The charge readout line 209 is connected to the source of each MOS transistor 208, and is connected to the charge output terminal of the pixel 201 on the same column. On the other hand, a common data output line 210 is connected to the drain of each MOS transistor 208.

With this configuration, the vertical scanning shift register
Each pixel 201 connected to the row selected by 202
Can be sequentially read out to the data output line 210 in accordance with the horizontal scanning of the horizontal scanning shift register 203. When all the charges of the pixels 201 for one row are read, vertical scanning is performed, the next row is selected, and horizontal scanning is started again. By repeating the above operation, it is possible to read out the charges of all the pixels 201 arranged in a matrix.

The data output line 210 is connected to the CDS 211, and the accumulated charges read from each pixel 201 are output from the ADC 212 pixel by pixel after noise removal peculiar to the solid-state imaging device included therein is performed. An operation of collecting the image data into the bit width (n) of the data output terminal of the image pickup device 101 and the pixel width of several pixels of the memory adopted as the buffer 104 is performed. The number of effective pixel arrays configured by two-dimensionally arranging the pixels 201 is 384 (H) × 288.
(V) Assuming that the output of the ADC 212 is 8 bits, the data read in one horizontal scanning period is 384 × 8 bits. If the serial / parallel conversion circuit 213 converts the data of 4 pixels into a serial / parallel format, the buffer 104 The required memory capacity is 96 W x 32 bits.

During the effective period (HDISP = 1) of the image data in the horizontal scanning period, the electric charge accumulated in the pixel 201 is read out in synchronization with the Pixel Clock by the control signal from the timing generator 102, and is subjected to serial / parallel conversion. To the buffer 104 by the memory controller 103
Written as 32bit data. When this operation is repeated 96 times, all the valid image data during the horizontal operation period is temporarily stored in the buffer 104, and the DMA transfer request signal (XDMA Re-request) is sent to the CPU 106 disposed outside the solid-state imaging unit 301.
q.) is output. The CPU 106 sends a DMA transfer request signal (XDM
A Req.), The ongoing image processing ends, and when DMA transfer becomes possible, the DMA transfer permission signal (XDMAAck.) Is returned to the memory controller 103, and the DMA transfer from the image sensor 101 to the frame memory 107 or the CPU 106 is performed. To
Start in synchronization with CPU Clock.

In general, the Pixel Clock is determined by the frame rate and the number of pixels of the image sensor, and is asynchronous with the CPU Clock and has a different cycle. The operation timing of the image sensor 101 is determined by the vertical scanning shift register during the scanning period.
In order to control the horizontal scan shift register 203, CDS 211, etc., a certain period other than the image valid period is required. FIG. 7 shows the basic scanning timing in the solid-state imaging unit 301, and its operation will be described. The solid-state imaging unit 301
To read at 60 frames / s, the required Pi
The cycle F (Pixel Clock) of xel Clock is as follows: F (Pixel Clock) = (400 + 138) x (300 + 10) x 60 = 10
(MHz). Accordingly, the non-effective period T (XHDISP) of the image data in the horizontal scanning period is T (XHDISP) = 1/10 (MHz) × 154 = 15.4 (μs), and the 96 W stored in the buffer 104 during this period.
It is necessary to DMA transfer the image data of × 32 bits to the CPU 106 or the frame memory 107. In the DMA transfer, it is assumed that reading from the buffer 104 requires two cycles as the CPU Clock cycle of the CPU 106 on the bus 105, while writing to the memory in the CPU 106 or writing to the frame memory 107 requires a total of four cycles. Necessary when performing data transfer by DMA transfer 96 times via bus 105 of 32 bit width
The frequency F (CPU Clock) of the CPU Clock must satisfy the following conditions. 1 / F (CPU Clock) × 96 × 4 <15.4 (μs) {F (CPU Clock)> 25 (MHz)} In actuality, a period for establishing a connection for DMA transfer is further required. Time also needs to be taken into account.

CPU Clock of CPU 106 on Bus 105
Is 25 (MHz), there is a possibility that a problem may occur in the actual transfer.
, Or the ineffective period of image data in the DMA transfer period, that is, the horizontal scanning period, needs to be increased. Hereinafter, the latter correspondence will be described. The CPU Clock of the CPU 106 on the bus 105 is 25 (MHz), and the DMA transfer period is 20 (μs) for safety. In the present embodiment, the communication unit 108 is a three-wire serial communication circuit,
It is connected to CPU106. The communication unit 108 incorporates a horizontal blanking register (HBLK-REG) and a vertical blanking register (VBLK-REG) as registers for setting arbitrary additional periods in the horizontal retrace period and the vertical retrace period of the video signal. are doing. A mode register (MO) for controlling the word length of the image signal output terminal (DATA) output from the solid-state imaging unit 301 to 32/16/8 bits.
DE-REG). The timing generator 102 operates in accordance with the contents of these registers.
1. The memory controller 103 is controlled.

Next, a case where an additional period is inserted only in the horizontal blanking period in order to satisfy the above-described condition will be described.
In FIG. 8, the solid-state imaging unit 301 is read at 60 frames / s,
In addition, a basic timing when an ineffective period of image data is provided in a horizontal scanning period of 20 (μs) is shown. The time T (H) allowed for one horizontal scan is T (H) = 1 × {60 × (300 + 10)} = 53.7 (μs), and 20 (μs) must be secured as a DMA transfer period. The image valid period T (HDISP) is as follows: T (HDISP) = T (H) −20 (μs) = 33.7 (μ)
s) Therefore, the frequency F of the Pixel Clock (Pixel C
lock) is F (Pixel Clock) = 1 / {33.7 (μs) / 384} = 11.4
(MHz). Therefore, Pixel Clock during non-effective period
The number N is calculated as follows: N (XHDISP) = 20 (μs) / ｛1 / 11.4 (MH
z) Since｝ = 228, by setting (HBLK-REG) = 228−138 = 90 in the horizontal blanking register (HBLK-REG) as an additional period, while satisfying 60 frames / s,
DMA transfer can be performed during the horizontal retrace period of the video signal.

In this embodiment, the case where the additional period (HBLK-REG) is inserted only in the horizontal retrace period has been described. However, the additional period (VBLK-REG) is also inserted in the vertical retrace period in the same way. Then, it is of course possible to increase the transfer period of the video signal.

In the above description, the bus width of the bus 105 is set to 32 bits. However, the bus width of the system bus is determined in accordance with the CPU 106 and the data bus width of the frame memory 107, and the operation mode corresponding to this is determined. Is set in the mode register (MODE-REG), an operation corresponding to other bus widths is also possible.

According to the above configuration, image data is transferred from the solid-state imaging unit 301 to the CPU 106 and the frame memory during the retrace period while maintaining the frame rate required by the system in accordance with the performance of the CPU 106 and the access speed of the frame memory 107. 107 to the solid-state imaging unit 30
1 and the minimum configuration of the CPU 106 and the frame memory 107.

As described above, the solid-state imaging unit
By realizing the image sensor 101, the timing generator 102, the memory controller 103, and the buffer 104 on the same chip 301, the number of components can be significantly reduced. In particular, if the solid-state imaging unit 301 is composed of CMOS, it is relatively easy to realize other components on the same chip, and the solid-state imaging unit 301 is implemented by an IP (Intellectual Prop.
rty), the CPU 106 and the frame memory 10
7 can be implemented on the same chip. In this case, since the data bus width can be increased without increasing the number of external terminals, a high-speed image processing system with an improved bandwidth can be realized.

[0046]

According to the first and second aspects of the present invention, the transfer of the video signal from the second storage unit to the first storage unit and the image processing unit is scanned as described above. Since it is configured to be performed during the signal retrace period, image data can be transferred in real time and image processing can be performed in a simple bus configuration without the need for complicated control means. A processing device can be realized. According to the third to sixth aspects of the present invention, similarly to the first and second aspects of the present invention, image data is transferred in real time and image processing is performed in a simple bus configuration without requiring complicated control means. And a pixel array, a scanning unit, A
An image processing apparatus in which the / D converter, the control unit, the storage unit, and the transfer unit are configured on the same chip can be realized.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a first embodiment of an image processing apparatus according to the present invention.

FIG. 2 is a schematic timing chart for explaining the operation of the first embodiment shown in FIG. 1;

FIG. 3 is a detailed timing chart for explaining a data transfer operation in the first embodiment shown in FIG. 1;

FIG. 4 is a timing chart for explaining a real-time image processing mode in the first embodiment shown in FIG. 1;

FIG. 5 is a block diagram showing a second embodiment of the present invention.

FIG. 6 is a timing chart for explaining the operation of the second embodiment shown in FIG. 5;

FIG. 7 is a timing chart showing basic scanning timing in the solid-state imaging unit according to the second embodiment shown in FIG.

FIG. 8 is a timing chart showing basic timing when the solid-state imaging unit according to the second embodiment is read at 60 frames / s and an ineffective period of image data is provided in a horizontal scanning period of 20 μs.

FIG. 9 is a block diagram illustrating a configuration example of a conventional image processing apparatus.

FIG. 10 is a block diagram illustrating a configuration example of a conventional image data transfer control device.

[Explanation of symbols]

 101 Image sensor 102 Timing generator 103 Memory controller 104 Buffer 105 System bus 106 CPU 107 Frame memory 108 Communication unit 201 Pixel (pixel) 202 Vertical shift register 203 Horizontal shift register 204 Vertical scanning start signal 205 Horizontal scanning start signal 206 Line selection signal 207 Column selection signal 208 MOS transistor 209 Charge readout line 210 Data output line 211 CDS 212 ADC 213 Series-parallel conversion circuit 301 Solid-state imaging unit

 ──────────────────────────────────────────────────続 き Continued on front page F term (reference) 4M118 AA10 AB01 BA10 BA14 DD09 FA06 5B047 AA11 AB02 EA07 EB04 EB17 5C024 AA01 BA00 CA16 GA31 HA07 HA14 HA15 5C052 GA07 GA09 GB01 GE04

Claims (6)

[Claims] 1. An image processing apparatus for performing predetermined processing on a video signal forming an image by performing horizontal scanning and vertical scanning, wherein the scanning is performed by scanning a pixel array in which pixels capable of accumulating charges generated by light irradiation are arranged. An image sensor that sequentially selects and outputs the image signal as a video signal, a first storage unit that stores the image signal output from the image sensor for at least one vertical scanning period, and at least one horizontal line that stores the image signal output from the image sensor. A second storage unit that stores the scanning period, a control unit that generates a timing signal indicating horizontal scanning and vertical scanning of the video signal output from the image sensor, and a predetermined storage unit that stores a predetermined amount of image data obtained by the video signal. An image processing unit that performs image processing; a data bus that connects the image sensor with the first storage unit and the image processing unit; 1. An image processing apparatus comprising: a storage unit; and a transfer unit that transfers a video signal to the image processing unit during a retrace period of a scan signal generated by the control unit. 2. An image according to claim 1, wherein a scanning cycle of said image sensor in said scanning period is different from a transfer cycle from said image sensor to a first storage unit. Processing equipment. 3. An image processing apparatus for performing predetermined processing on a video signal constituting an image by performing horizontal scanning and vertical scanning, wherein a pixel capable of reading out a charge generated and accumulated as a result of light irradiation as a video signal. A two-dimensionally arranged pixel array, a scanning unit for performing horizontal scanning and vertical scanning on the pixel array, an A / D converter for converting the video signal into a digital signal, and an output from the pixel array A control unit that generates a timing signal indicating horizontal scanning and vertical scanning of a video signal to be output, a second storage unit that holds an effective image signal during a horizontal scanning period, and the second storage unit based on an output of the control unit. And a transfer unit for reading out and outputting the effective image signal stored in the second storage unit during the horizontal retrace period of the video signal and outputting the effective image signal held in the second storage unit on the same chip. A body imaging unit, a first storage unit that stores and holds a video signal output from the solid-state imaging unit for at least one vertical scanning period, and image processing that performs predetermined image processing on an image obtained by the video signal And a data bus that connects the solid-state imaging unit, the first storage unit, and the image processing unit.
Image processing, wherein transfer of a video signal to a storage unit and the image processing unit is performed via the data bus during a retrace period of a scanning signal generated by the control unit. apparatus. 4. The solid-state imaging unit according to claim 3, further comprising: a conversion unit that performs serial-to-parallel conversion on pixel data of a video signal read from the pixel array during a horizontal scanning period. Image processing device. 5. The solid-state imaging unit includes a communication unit for enabling an operation mode of the control unit and the transfer unit to be set by a host controller, and based on an output of the communication unit, image data of the video signal. 5. The image according to claim 3, wherein the image data is transferred according to a bit width of a data bus connecting the first storage unit and the image processing unit. 6. Processing equipment. 6. The solid-state imaging unit includes a communication unit that enables an operation mode of the control unit and the transfer unit to be set by a host controller, and based on an output of the communication unit, the communication unit of the serial-parallel conversion unit. The image processing apparatus according to any one of claims 3 to 5, wherein a conversion cycle and a blanking period of the video signal can be arbitrarily set.