JP2009182993A - Image processing apparatus, image processing system, and imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus capable of processing in parallel a plurality of color component signals output in parallel from an imaging sensor, the apparatus having a reduced cost, a low power consumption and a low circuit scale. <P>SOLUTION: An SPU (image processing apparatus) 12 includes a plurality of defective pixel correction circuits 49, 50, 51 for correcting color component signals corresponding to a defective pixel in the imaging sensor in accordance with a control signal TS, an input control circuit 63 for receiving defect correction data DCD transferred from a memory in synchronization with input of the plurality of color component signals, and a timing generation circuit 52 for generating the control signal TS based on the defect correction data DCD. The defective pixel correction circuits 49, 50, 51 correct color component signals corresponding to the same defective pixel using the same control signal TS. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an apparatus and system for processing an image signal output from a solid-state imaging device such as a CCD or a CMOS.

  In general, a digital camera is equipped with a solid-state image sensor (hereinafter simply referred to as “image sensor”) such as a CCD or a CMOS. In addition to a digital camera equipped with an image sensor for each color of R (red), G (green), and B (blue) (hereinafter referred to as “three-plate digital camera”), a single-plate image sensor is installed. Digital cameras (hereinafter referred to as “single-plate digital cameras”) are also widely used. In the case of a single-plate digital camera, R, G, or B color filters are formed for each pixel according to a known Bayer arrangement on the light receiving portion of the image sensor. A pixel interpolation process is performed on the image signal captured by the image sensor so that each pixel has components of a plurality of colors of R, G, and B. This pixel interpolation process reduces the resolution and false colors. It is known that problems occur.

  On the other hand, in the case of a three-plate type digital camera, since it is necessary to process image signals output in parallel from three image sensors, the circuit configuration of the image processing is complicated, and the manufacturing cost tends to increase. Information relating to image processing of a three-plate digital camera is described in, for example, Patent Document 1 below.

  By the way, in recent years, single-plate full-color imaging devices such as “foveon x3 sensor” having three layers of R, G, and B light-receiving portions in each pixel have become widespread. In this image sensor, since incident light is detected by the light receiving portions of the R, G, and B layers, component signals of three colors R, G, and B are output in parallel from the single-plate image sensor, and pixels Since no interpolation processing is required, there is an advantage that generation of false colors and reduction in resolution are unlikely to occur. Information related to this type of image sensor can be obtained from, for example, the URL shown in Non-Patent Document 1 below.

Japanese Patent Laid-Open No. 10-308901 (FIGS. 1 and 2)

Internet <URL: http://www.foveon.com/X3_better.html>

  Accompanying the widespread use of the single-plate full-color image sensor, it is compatible with digital cameras equipped with this single-plate full-color image sensor, and can be used with conventional single-plate digital cameras at low cost and with low power consumption and small circuit scale. Image processing chips are required.

  In view of the above situation, the object of the present invention is (1) a low-cost, low power consumption, small-circuit scale image processing apparatus and image capable of processing in parallel a plurality of color component signals output in parallel from an image sensor. It is to provide a processing system and an imaging device equipped with these, and (2) to provide an image processing device capable of dealing with a plurality of types of imaging devices and an imaging device equipped with the image processing device.

  To achieve the above object, according to a first aspect of the present invention, there is provided an image processing apparatus for processing any one of a YUV signal composed of a luminance signal and a color difference signal and a plurality of color component signals. An input terminal for inputting the color component signal, a selector for outputting a selection signal obtained by selecting one of the input YUV signal and the plurality of color component signals, and a signal for processing the selection signal A processing unit, wherein the input terminal is shared by the YUV signal and the color component signal.

  According to a second aspect, in the image processing device according to the first aspect, the signal processing unit includes a plurality of defective pixel correction circuits that correct the plurality of selection signals corresponding to defective pixels in the imaging sensor according to a predetermined timing; A defective pixel correction timing generation circuit that generates the predetermined timing for performing defective pixel correction in accordance with the input of the selection signal, and the plurality of defective pixel correction circuits are the same at the same predetermined timing. The selection signals corresponding to the defective pixels are corrected in parallel.

  According to a third invention, in the image processing apparatus of the first or second invention, an oversampling circuit that interpolates a deficient signal using an input signal and outputs the signal to the selector, and the YUV signal into a plurality of component signals A separation circuit for separating, wherein the YUV signal is composed of a luminance signal and a subsampled color difference signal, and the oversampling circuit is configured to output the subsampled color difference signal output from the separation circuit. Is used to interpolate the missing color difference signal.

  According to a fourth invention, in the image processing apparatus of the third invention, there is provided an output control circuit for outputting the YUV signal to the bus as it is.

A fifth invention is an image processing device for processing a multiplexed signal output from an imaging sensor, an input terminal to which the multiplexed signal is input, a sampling circuit for sampling the input multiplexed signal, The multiplexed signal has a color component signal of N ₁ bit width (N ₁ is a positive integer) and a plurality of N ₂ bit widths (N ₂ is a positive integer; N ₁ is twice N ₂ ). The sampling circuit has a function of combining the N ₂ bit width data signal and converting it into the N ₁ bit width color component signal. It is a feature.

A sixth invention is an image processing device for processing a multiplexed signal output from an imaging sensor, an input terminal to which the multiplexed signal is input, a sampling circuit for sampling the input multiplexed signal, The multiplexed signal has a color component signal of N ₁ bit width (N ₁ is a positive integer) and a plurality of N ₂ bit widths (N ₂ is a positive integer; N ₁ is 3 times N ₂ ). The sampling circuit has a function of combining the N ₂ bit width data signal and converting it into the N ₁ bit width color component signal. It is a feature.

A seventh invention is an image processing apparatus for processing a multiplexed signal output from an imaging sensor, an input terminal to which the multiplexed signal is input, a sampling circuit for sampling the input multiplexed signal, The multiplexed signal includes a color component signal of N ₁ bit width (N ₁ is a positive integer) and a plurality of N ₂ bit widths (N ₂ is a positive integer; N ₁ is 4 times N ₂ ). The sampling circuit has a function of combining the N ₂ bit width data signal and converting it into the N ₁ bit width color component signal. It is a feature.

An eighth invention is an image processing apparatus for processing a multiplexed signal of N ₁ bit width (N ₁ is a positive integer) configured by multiplexing a plurality of color component signals in a time division manner. A sampling circuit for sampling the plurality of color component signals from the input multiplexed signal and outputting them in parallel as a signal of N ₂ bit width (N ₂ is a multiple of N ₁ ); A signal processing unit that processes the plurality of color component signals output in parallel from the sampling circuit, wherein the sampling circuit is N ₂ / N of a clock signal synchronized with the output of the N ₂ -bit width signal. _The plurality of color component signals are sampled by using a clock signal having a frequency of ₁ time.

According to a ninth aspect, in the image processing apparatus according to the eighth aspect, the sampling circuit has a frequency four times as high as the clock signal synchronized with the output of the N ₂ bit width signal (N ₂ is four times greater than N ₁ ). The plurality of color component signals are sampled using a clock signal having.

In a tenth aspect, in the image processing apparatus according to the eighth aspect, the sampling circuit has a frequency three times that of the clock signal synchronized with the output of the N ₂ bit width signal (N ₂ is three times N ₁ ). The plurality of color component signals are sampled using a clock signal having.

  An eleventh invention is an image processing system for processing an image signal, wherein the signal processing unit processes in parallel a plurality of image signals read in parallel from a plurality of light receiving units of an image sensor, and the signal processing unit. And a plurality of output control circuits that respectively output the plurality of image signals processed in step 1 to a bus, and a data transfer control unit that transfers the plurality of image signals output to the bus.

  In a twelfth aspect based on the image processing system according to the eleventh aspect, the data transfer control unit includes a DMA (direct memory access) controller.

  In a thirteenth aspect, in the image processing system according to the eleventh or twelfth aspect, the imaging sensor has a function of outputting two image signals read in opposite directions, and the plurality of outputs The control circuit outputs each of the two image signals processed in parallel by the signal processing unit to the bus, and the data transfer control unit sets one of the two image signals as a transfer destination address. The image signal has a function of incrementing in the reverse direction with respect to the other transfer destination address.

  In a fourteenth aspect, in the image processing system according to any one of the first to thirteenth aspects, the signal processing unit also has a function of processing a plurality of color component signals input in parallel from the imaging sensor in parallel.

  A fifteenth aspect of the invention is an image pickup apparatus including any one of the image processing apparatuses of the first to tenth aspects of the invention. A sixteenth aspect of the invention is the image processing system of any of the first to fourteenth aspects of the invention. An imaging apparatus provided.

  As described above, according to the image processing apparatus of the present invention, a plurality of color component signals corresponding to the same defective pixel are corrected in parallel at the same timing, and it is necessary to generate timing for each defective pixel correction circuit. Therefore, the amount of transfer of defect correction data can be reduced, and the use efficiency of the bus can be improved.

  According to another image processing apparatus of the present invention, the number of input pins provided in the input terminal can be reduced, and the circuit configuration can be reduced in scale.

1 is a block diagram schematically showing a configuration of an imaging apparatus (digital camera) according to an embodiment of the present invention. 1 is a diagram schematically showing a circuit configuration of an SPU (image processing apparatus) according to a first embodiment of the present invention. It is a figure which shows schematically the circuit structure of SPU which concerns on the 1st Embodiment of this invention. It is a figure which shows schematically the circuit structure of the sampling circuit of SPU which concerns on the 2nd Embodiment of this invention. It is a timing chart which shows the various signal waveforms in the sampling circuit of a 2nd embodiment. It is a figure which shows schematically the circuit structure of the sampling circuit of SPU which concerns on the 3rd Embodiment of this invention. It is a timing chart which shows the various signal waveforms in the sampling circuit of a 3rd embodiment. It is a figure which shows schematically the circuit structure of the sampling circuit of SPU which concerns on the 4th Embodiment of this invention. It is a timing chart which shows the various signal waveforms in the sampling circuit of a 4th embodiment. It is a figure which shows schematically the circuit structure of the sampling circuit of SPU which concerns on the 5th Embodiment of this invention. It is a timing chart which shows the various signal waveforms in the sampling circuit of a 5th embodiment. It is a block diagram which shows roughly the structure of the image processing system which concerns on the 6th Embodiment of this invention. It is a block diagram which shows roughly the structure of the image processing system which concerns on the 7th Embodiment of this invention. It is a figure which shows typically the scanning direction of the signal read from the light-receiving circuit of an imaging sensor. It is a figure which shows typically the arrangement | sequence of the image signal in 7th Embodiment.

  Hereinafter, various embodiments of the present invention will be described.

<Configuration of Imaging Device 1>
FIG. 1 is a block diagram schematically showing the configuration of an imaging apparatus (digital camera) 1 according to an embodiment of the present invention. The imaging apparatus 1 includes an optical mechanism 10, an imaging sensor 11, an SPU 12, an RPU 13, a CPU (Central Processing Unit) 14, a main memory 18, a ROM 19, a DMA controller (DMAC) 17, a bus 16, and a system clock generation unit 15. It is configured. The bus 16 includes an address bus, a data bus, a DMA transfer bus, and the like, and signal transfer between the modules 12, 13, 14, 17, 18, and 19 via the bus 16 is controlled by the CPU 14 or the DMAC 17. Executed. In this specification, the “data transfer control unit” includes one or both of the CPU 14 and the DMAC 17. Further, the system clock generation unit 15 generates clock signals SCK0 to SCK2 and supplies them to the SPU 12, RPU 13 and DMAC 17.

  Although not shown, the imaging device 1 includes a display device that displays a captured image, various interface circuits corresponding to peripheral devices such as a memory card, a compression encoding circuit, an input device such as an input button and a dial, and flash light. Equipped with a flash device that emits light.

  The optical mechanism 10 has a lens group, a prism, an AF (auto focus) function, an aperture adjustment function, and the like. Incident light IL from the subject passes through the optical mechanism 10 and is imaged. To form an image. The light incident on the image sensor 11 is received by an image sensor such as a CCD or CMOS after passing through an optical LPF (optical low pass filter) (not shown). In the image sensor 11, the light transmitted through the optical LPF is photoelectrically converted into an analog image signal, and then CDS processing (Correlated Double Sampling processing), AGC processing (Automatic Gain Control processing) Processing) and A / D conversion processing in this order, and then output to the SPU 12.

  An SPU (Signal Processing Unit) 12 performs processing such as defective pixel correction on a digital image signal input from the image sensor 11, and then performs processing such as bus 16 or RPU (Real-time Processing Unit; real-time processing).・ Output to unit 13. The RPU 13 has a function of executing various digital image processing such as shading correction processing, pixel interpolation processing, gamma correction processing, color space conversion processing, contour enhancement processing, and resolution conversion processing in real time on the image signal input from the SPU 12. have. Image data output from the SPU 12 or RPU 13 to the bus 16 can be transferred to and stored in the main memory 18 under the control of the CPU 14 or the DMAC 17. Further, the CPU 14 can perform various software processes on the image data read from the main memory 18 by loading and executing a program from the ROM 19.

  The DMAC 17 includes a plurality of DMA channels CH0 to CHn (n is an integer of 1 or more) and a bus controller BC. Upon receiving a DMA transfer request from another module, the control unit (not shown) of the DMAC 17 selects a DMA channel CHk (k is 0 to n) to be assigned to the DMA transfer from among a plurality of installed DMA channels CH0 to CHn. And a bus acquisition request is issued to the bus controller BC. When the bus controller BC succeeds in the bus acquisition request, the bus controller BC executes data transfer between modules using the assigned DMA channel CHk. After completion of the DMA transfer, the bus controller BC releases the used bus. When a plurality of DMA transfers are executed, a plurality of DMA channels are allocated and a plurality of DMA transfers are executed in a time division manner.

  The configuration of each part of the imaging apparatus 1 having the above configuration will be described below.

<First Embodiment>
2 and 3 are diagrams schematically illustrating a circuit configuration of the SPU (image processing apparatus) 12 according to the first embodiment of the present invention. Note that the circuits shown in FIGS. 2 and 3 are continuously connected via a one-dot chain line M. The SPU 12 has a function of processing primary color RGB color component signals, 16-bit YUV signals, or RAW data signals input from the input terminals P1 to P7.

<Processing of primary color RGB color component signals>
When the SPU 12 processes primary color RGB color component signals, the G signal, R signal, and B signal input in parallel from the imaging sensor 11 are input to the input terminals P2, P6, and P7, respectively. A 12-bit wide G signal CCDD [11: 0] input to the input terminal P2 includes a first phase adjustment circuit 21, first selectors 26 and 29, and a black level correction circuit (DLC; dark). level compensation circuit) 35, and further to a selector 43, a white balance compensation circuit (WBC) 46, and a defective pixel correction circuit 49 shown in FIG. Thereafter, the G signal output from the defective pixel correction circuit 49 is supplied to an AF evaluation value calculation circuit (AFC) 53, a pixel averaging circuit 54, a subsampling circuit 55, and a subsampling circuit 59. The G signal transmitted to the pixel averaging circuit 54 or the sub-sampling circuit 55 can be output from the output control circuit 58 to the bus 16 via the selectors 56 and 57. On the other hand, the G signal transmitted to the sub-sampling circuit 59 can be output to the RPU 13 via the selector 62.

  Specifically, the first phase adjustment circuit 21 uses the clocks SPU2CK and SPUCK to sample and adjust the phase of the G signal CCDD [11: 0] transmitted from the input terminal P2, and then the “0” th of the selector 26. Output to the terminal. The selector 26 selects either the “0” -th terminal or the “1” -th terminal in accordance with the value of the selection control signal (not shown) given from the CPU 14, and receives the signal input to the selected terminal. Output to the “0” -th terminal of the selector 29 in the next stage. Now, the selector 26 selects the “0” -th terminal, selects the G signal input from the first phase adjustment circuit 21, and outputs it to the selector 29.

  The selector 29 selects either the “0” -th terminal or the “1” -th terminal in accordance with the value of the selection control signal (not shown) given from the CPU 14, and the signal inputted to the selected terminal. (Selection signal) S0 is output. Now, the selector 29 has selected the “0” -th terminal, and outputs the signal transmitted from the selector 26 in the previous stage as the selection signal S0.

  The upper 2 bits of the selection signal S 0 are input to the “0” terminal of the selector 32, and the lower 10 bits of the selection signal S 0 are input to the black level correction circuit (DLC) 35. The selector 32 selects either the “0” -th terminal or the “1” -th terminal in accordance with the value of the selection control signal (not shown) given from the CPU 14, and the two bits input to the selected terminal Output a signal. When the selector 32 selects the “1” -th terminal, a 2-bit signal having a zero value (= 0x0) is output. The 2-bit signal output from the selector 32 and the lower 10 bits of the selection signal S0 are combined and supplied to the black level correction circuit 35. The reason for providing the selector 32 in this way is to support the imaging sensor 11 with 10-bit output. When the SPU 12 is connected to the imaging sensor 11 that outputs a 10-bit signal, the selector 32 selects the “1” -th terminal and masks the upper 2 bits of the selection signal S0. As a result, the terminal corresponding to the masked upper 2 bits can be used for other purposes.

  The black level correction circuit 35 has a function of correcting the gradation of the input signal so that the black level of the input signal matches the reference level. The black level correction circuit 35 includes a register (not shown) that holds a reference level value. The 12-bit signal output from the black level correction circuit 35 is input to the 2-bit shift calculator (BS) 40 and the “0” -th terminal of the selector 43 shown in FIG.

  The 2-bit shift computing unit 40 has a function of shifting the 12-bit signal transmitted from the black level correction circuit 35 to the left by 2 bits and outputting it to the “1” -th terminal of the selector 43. When the SPU 12 is connected to the imaging sensor 11 with 10-bit output, the lower 10 bits of the 12-bit signal output from the black level correction circuit 35 includes an image component, and the 12-bit signal is shifted left by 2 bits. As a result, the image component is moved to the upper 10 bits. This is to cope with white balance correction in the subsequent stage. The selector 43 selects either the “0” -th terminal or the “1” -th terminal according to the value of the selection control signal (not shown) given from the CPU 14, and receives the signal input to the selected terminal. Output.

  A white balance compensation circuit (WBC) 46 corrects the color tone of the input signal transmitted from the selector 43 in the previous stage according to the reference value, and outputs the corrected signal to the defective pixel correction circuit 49. The white balance correction circuit 46 includes a register (not shown) that stores the reference value.

  The defective pixel correction circuit 49 corrects and outputs an input signal corresponding to the defective pixel in the image sensor 11 at the timing of the control signal TS supplied from the timing generation circuit (TG) 52. When the input signal corresponds to a defective pixel, for example, a process of averaging signals corresponding to normal pixels close to the defective pixel to generate a correction signal and replacing the input signal with the correction signal is executed.

  The position of the defective pixel is detected in advance at the inspection stage of the image sensor 11, and defect correction data DCD including the position information of the defective pixel is recorded in the ROM 19 (FIG. 1). The defect correction data DCD is transferred by the DMAC 17 from the ROM 19 to the SPU 12 in units of pixels in accordance with the input of the G signal to the defective pixel correction circuit 49 and received by the input control circuit 63. Instead of direct DMA transfer from the ROM 19, DMA transfer may be performed after the defect correction data DCD is once copied to the RAM 18. The input control circuit 63 buffers the received defect correction data DCD in the FIFO circuit 63a, and then supplies it to the timing generation circuit 52. The timing generation circuit 52 generates a control signal TS based on the defect correction data DCD and supplies the control signal TS to the defective pixel correction circuit 49 and other defective pixel correction circuits 50 and 51 described later. Therefore, the three defective pixel correction circuits 49, 50, 51 correct signals corresponding to the same pixel at the same timing. In this example, the defect correction data DCD is stored in a memory such as a ROM or a RAM, and DMA transfer is performed from these memories. However, the defect correction data DCD is stored in a register in the timing generation circuit 52, for example. It may be in any form. If the CPU has sufficient processing capacity, the defect correction data stored in a memory such as a ROM may be transferred by the CPU instead of the DMA transfer. That is, the storage location and transfer method of the defect correction data DCD are not particularly limited.

  The G signal output from the defective pixel correction circuit 49 is supplied to an AF evaluation value calculation circuit (AFC) 53, a pixel averaging circuit 54, and subsampling circuits 55 and 59. An AF evaluation value calculation circuit (AFC) 53 calculates an evaluation data signal EV for autofocus control using the G signal output from the defective pixel correction circuit 49. The pixel averaging circuit 54 averages input signals of several tens of pixels to generate an evaluation data signal for white balance adjustment or automatic exposure adjustment, and outputs it to the “1” terminal of the selector 56. The sub-sampling circuit 55 sub-samples the input signal to reduce the resolution of the input signal, and outputs a signal that is the processing result to the “0” -th terminal of the selector 56. The sub-sampling circuit 59 also performs the same processing as that of the sub-sampling circuit 55 and outputs a signal that is the processing result to the “0” -th terminal of the selector 62. These sub-sampling circuits 55 and 59 can output the input signal as it is without sub-sampling.

  The selector 56 selects either the “0” -th terminal or the “1” -th terminal in accordance with the value of the selection control signal (not shown) given from the CPU 14, and receives the signal input to the selected terminal. The data is output to the “0” -th terminal of the subsequent selector 57. The other “1” terminal of the selector 57 is supplied with a signal transmitted from a separation circuit 20 described later, and the selector 57 responds to the value of a selection control signal (not shown) given from the CPU 14. Thus, either the “0” -th terminal or the “1” -th terminal can be selected, and the signal input to the selected terminal can be output to the output control circuit 58. The output control circuit 58 buffers the input signal in the FIFO circuit 58a and then outputs the input signal to the bus 16 for DMA transfer.

  On the other hand, the 12-bit width R signal TCCHR [11: 0] input to the input terminal P6 is sent to the second phase adjustment circuit 22, the selectors 27 and 30, and the black level correction circuit 36 shown in FIG. After the transmission, the signal is further transmitted to the selector 44, the white balance correction circuit 47, the defective pixel correction circuit 50, and the subsampling circuit 60 shown in FIG. 3, and then output to the RPU 13.

  Specifically, the second phase adjustment circuit 22 uses the clocks SPU2CK and SPUCK to sample and adjust the phase of the R signal TCCHR [11: 0] transmitted from the input terminal P6, and then the “0” th of the selector 27. Output to the terminal. The selector 27 selects either the “0” -th terminal or the “1” -th terminal in accordance with the value of the selection control signal (not shown) given from the CPU 14, and receives the signal input to the selected terminal. Output to the “0” -th terminal of the selector 30 in the next stage. Now, the selector 27 selects the “0” -th terminal, selects the R signal input from the second phase adjustment circuit 22, and outputs it to the selector 30.

  The selector 30 selects either the “0” -th terminal or the “1” -th terminal in accordance with the value of the selection control signal (not shown) given from the CPU 14, and the signal inputted to the selected terminal. (Selection signal) S1 is output. Now, the selector 30 selects the “0” -th terminal, and outputs the signal transmitted from the selector 27 in the previous stage as the selection signal S1.

  Further, the upper 2 bits of the selection signal S1 are input to the “0” terminal of the selector 33, and the lower 10 bits of the selection signal S1 are input to the black level correction circuit (DLC) 36. The selector 33 selects either the “0” -th terminal or the “1” -th terminal in accordance with the value of the selection control signal (not shown) given from the CPU 14, and the two bits input to the selected terminal Output a signal. When the selector 33 selects the “1” -th terminal, a 2-bit signal having a zero value (= 0x0) is output. The 2-bit signal output from the selector 33 and the lower 10 bits of the selection signal S1 are combined and supplied to the black level correction circuit 36.

  The black level correction circuit 36 corrects the gradation of the input signal using the reference level held in the built-in register (not shown) so that the black level of the input signal matches the reference level. The signal is output to the 2-bit shift computing unit (BS) 41 and the “0” -th terminal of the selector 44 in the subsequent stage (FIG. 3). The 2-bit shift calculator 41 shifts the 12-bit signal transmitted from the black level correction circuit 36 to the left by 2 bits and outputs it to the “1” -th terminal of the selector 44. The selector 44 selects either the “0” -th terminal or the “1” -th terminal in accordance with the value of a selection control signal (not shown) supplied from the CPU 14, and a signal input to the selected terminal. Is output to the white balance correction circuit 47. The white balance correction circuit 47 uses the reference value held in the built-in register (not shown) to correct the color tone of the input signal transmitted from the previous selector 44 according to the reference value, and the defective pixel correction circuit 50. Output to.

  The defective pixel correction circuit 50 corrects the input signal corresponding to the defective pixel at the timing of the control signal TS supplied from the timing generation circuit 52. The output signal of the defective pixel correction circuit 50 is subsampled by the subsampling circuit 60 and then supplied to the RPU 13. The subsampling circuit 60 can also output the input signal as it is to the RPU 13 without subsampling.

  The output signal of the sub-sampling circuit 60 is also input to the “1” terminal of the selector 62. The selector 62 selects either the “0” -th terminal or the “1” -th terminal in accordance with the value of the selection control signal (not shown) given from the CPU 14, and receives the signal input to the selected terminal. Output to the RPU 13. Therefore, the selector 62 can select and supply either the G signal input to the “0” terminal or the R signal input to the “1” terminal to the RPU 13. Now, the selector 62 selects the G signal and supplies it to the RPU 13.

  On the other hand, the 12-bit width B signal TCCHB [11: 0] input to the input terminal P7 is sent to the third phase adjustment circuit 23, the selectors 28 and 31, and the black level correction circuit 37 shown in FIG. After the transmission, the signal is further transmitted to the selector 45, the white balance correction circuit 48, the defective pixel correction circuit 51, and the sub-sampling circuit 61 shown in FIG.

  Specifically, the third phase adjustment circuit 23 uses the clocks SPU2CK and SPUCK to sample and adjust the phase of the B signal TCCHB [11: 0] transmitted from the input terminal P7, and then the “0” th of the selector 28. Output to the terminal. The selector 28 selects either the “0” -th terminal or the “1” -th terminal according to the value of the selection control signal (not shown) given from the CPU 14, and receives the signal input to the selected terminal. Output to the “0” -th terminal of the selector 31 in the next stage. Now, the selector 31 selects the “0” -th terminal, selects the B signal input from the third phase adjustment circuit 23, and outputs it to the selector 31.

  The selector 31 selects either the “0” -th terminal or the “1” -th terminal in accordance with the value of the selection control signal (not shown) given from the CPU 14, and the signal inputted to the selected terminal. (Selection signal) S2 is output. Now, the selector 31 has selected the “0” -th terminal, and outputs the signal transmitted from the selector 28 in the previous stage as the selection signal S2.

  Further, the upper 2 bits of the selection signal S 2 are input to the “0” -th terminal of the selector 34, and the lower 10 bits of the selection signal S 2 are input to the black level correction circuit (DLC) 37. The selector 34 selects either the “0” -th terminal or the “1” -th terminal in accordance with the value of the selection control signal (not shown) given from the CPU 14, and 2 bits input to the selected terminal Output a signal. When the selector 34 selects the “1” -th terminal, a 2-bit signal having a zero value (= 0x0) is output. The 2-bit signal output from the selector 34 and the lower 10 bits of the selection signal S2 are combined and supplied to the black level correction circuit 37.

  The black level correction circuit 37 corrects the gradation of the input signal using the reference level held in the built-in register (not shown) so that the black level of the input signal matches the reference level. The signal is output to the 2-bit shift computing unit (BS) 42 and the “0” -th terminal of the selector 45 in the subsequent stage (FIG. 3). The 2-bit shift computing unit 42 shifts the 12-bit signal transmitted from the black level correction circuit 37 to the left by 2 bits and outputs it to the “1” -th terminal of the selector 45. The selector 45 selects either the “0” -th terminal or the “1” -th terminal in accordance with the value of a selection control signal (not shown) supplied from the CPU 14, and a signal input to the selected terminal. Is output to the white balance correction circuit 48. The white balance correction circuit 48 uses the reference value held in the built-in register (not shown) to correct the color tone of the input signal transmitted from the selector 45 in the previous stage according to the reference value, and the defective pixel correction circuit 51. Output to.

  The defective pixel correction circuit 51 corrects the input signal corresponding to the defective pixel at the timing of the control signal TS supplied from the timing generation circuit 52. The output signal of the defective pixel correction circuit 51 is subsampled by the subsampling circuit 61 and then supplied to the RPU 13. The subsampling circuit 61 can also output the input signal as it is to the RPU 13 without subsampling.

<Processing of YUV422 signal>
Next, the case where the SPU 12 (FIGS. 2 and 3) according to the first embodiment processes a 16-bit YUV signal will be described below. The 16-bit YUV signal is a YUV422 signal generated so that the sampling ratio of the Y signal (luminance signal), U signal (color difference signal), and V signal (color difference signal) is 4: 2: 2. It is composed of an 8-bit Y signal and an 8-bit UV signal (a signal obtained by multiplexing a U signal and a V signal in a time division manner). Here, the UV signal is obtained by sub-sampling and multiplexing an 8-bit U signal and an 8-bit V signal.

  The upper 4-bit signal YUVI [15:12] of the 16-bit YUV signal is input to the input terminal P1, and the lower 12-bit signal CCDD [11: 0] is input to the input terminal P2. These signals YUVI [15:12] and CCDD [11: 0] are combined and input to the separation circuit 20. Therefore, since the input terminal P2 is shared by the YUV signal and the primary color RGB signal, the number of input pins provided on the input terminal is smaller than when the input terminal is provided independently for the YUV signal and the primary color RGB signal. I'll do it.

  The 16-bit YUV422 signal input to the input terminals P1 and P2 is transmitted to the separation circuit 20. The separation circuit 20 samples the YUV422 signal, separates it into an 8-bit Y signal and an 8-bit UV signal, and outputs them. The Y signal is added with the upper 4 bits of the zero value (= 0b0000), becomes a 12-bit signal, is transmitted to the “1” terminal of the selector 29, and the UV signal is transmitted to the oversampling circuit 25. The oversampling circuit 25 oversamples the input UV signal, and as a result outputs an 8-bit U signal and an 8-bit V signal. These U signal and V signal are each added with the upper 4 bits of the zero value (= 0b0000) to form a 12-bit signal, which is transmitted to the “1” -th terminals of the selectors 30 and 31.

  Each of the selectors 29, 30, and 31 selects and selects either the "0" -th terminal or the "1" -th terminal in accordance with the value of a common selection control signal (not shown) given from the CPU 14. The selection signals S0, S1, S2 input to the selected terminals are output. Now, the selectors 29 to 31 select the “1” -th terminal, the selector 29 outputs the Y signal, the selector 30 outputs the U signal, and the selector 31 outputs the V signal.

  The Y signal (selection signal S0) output from the selector 29 is similar to the processing of the color component signals of the primary colors RGB, the selector 32, the black level correction circuit 35, the 2-bit shift calculator 40, the selector 43 and the defective pixel correction. After being sequentially processed by the circuit 49, it is output to the AF evaluation value calculation circuit (AFC) 53, the pixel averaging circuit 54, and the subsampling circuits 55 and 59 for processing. Now, the selector 62 selects the “0” -th terminal, and outputs the Y signal input from the sub-sampling circuit 59 to the RPU 13.

  Similarly, the U signal (selection signal S1) output from the selector 30 also includes a selector 33, a black level correction circuit 36, a 2-bit shift calculator 41, a selector 44, a white balance correction circuit 47, a defective pixel correction circuit 50, and a sub signal. After being sequentially processed by the sampling circuit 60, it is output to the RPU 13. Further, the V signal (selection signal S2) output from the selector 31 is similarly selected by the selector 34, the black level correction circuit 37, the 2-bit shift calculator 42, the selector 45, the white balance correction circuit 48, the defective pixel correction circuit 51, and the sub signal. After being sequentially processed by the sampling circuit 61, it is output to the RPU 13.

  On the other hand, the Y signal and UV signal output from the separation circuit 20 are combined and transmitted to the “1” terminal of the selector 57 (FIG. 3) as a 16-bit YUV422 signal. The selector 57 outputs either the signal output from the previous selector 56 or the YUV422 signal to the output control circuit 58, and the output control circuit 58 outputs a signal to be DMA transferred to the bus 16. Therefore, by causing the selector 57 to select the “1” -th terminal, the Y signal, the U signal, and the V signal (YUV 444 signal) are output to the RPU 13 in parallel with the process of DMA transfer of the YUV 422 signal to another module as it is. can do. Since the YUV422 signal is not oversampled, the data transfer amount is small compared to the case of transferring the YUV444 signal, and there is an advantage that the use band of the bus 16 can be suppressed.

<RAW data signal processing>
The SPU 12 can process the output signal (RAW data signal) of the single-plate image sensor 11 having a Bayer color filter array. The RAW data signal is a signal in which each pixel has only a single color component.

  The 12-bit width RAW data signal input from the input terminal P2 is sampled and phase-adjusted by the first phase adjustment circuit 21, and then output to the selector 26. Now, the selector 26 has selected the “0” -th terminal, and outputs the RAW data signal output from the first phase adjustment circuit 21 to the selector 29. Now, the selector 29 has selected the “0” -th terminal, and outputs the RAW data signal input from the previous selector 26 as the selection signal S0.

  Thereafter, the upper 2-bit signal of the RAW data signal is transmitted to the “0” -th terminal of the selector 32, and the lower 10-bit signal is transmitted to the black level correction circuit 35. The black level correction circuit 35 is combined with the upper 2-bit signal output from the selector 32 and the lower 10-bit signal of the RAW data to be input as a 12-bit signal.

  The black level correction circuit 35 has four types of operation modes (TC MODE), and the reference level value used for each operation mode is different. The first operation mode is a mode used for processing of the color component signals of the primary colors RGB and processing of the YUV signal. The second to fourth operation modes are modes used for processing the RAW data signal. Since the color of the RAW data signal differs between adjacent pixels, the black level correction circuit 35 switches the operation mode for each pixel in accordance with the color of the input signal. For example, when the R signal and the G signal are alternately input like the R signal, the G signal, the R signal, the G signal,..., The operation mode for the R signal and the operation mode for the G signal are switched alternately. It will be.

  The signal output from the black level correction circuit 35 is transmitted to the 2-bit shift calculator 40 and the selector 43 and then input to the white balance correction circuit 46. Similar to the black level correction circuit 35, the white balance correction circuit 46 has four types of operation modes (TC MODE), and switches the operation mode for each pixel in accordance with the color of the input signal.

  A signal transmitted from the defective pixel correction circuit 49 to the sub-sampling circuit 55 is input to the output control circuit 58 via the sub-sampling circuit 55 and the selectors 56 and 57, and then output to the bus 16 and transferred. On the other hand, the sub-sampling circuit 59 has four types of operation modes (TC MODE) like the black level correction circuit 35, and switches the operation mode for each pixel in accordance with the color of the input signal. A signal transmitted from the defective pixel correction circuit 49 to the sub-sampling circuit 59 is output to the RPU 13 via the sub-sampling circuit 59 and the selector 62.

  As described above, the SPU 12 according to the present embodiment has high versatility because it can process a plurality of types of image signals.

  In addition, the three defective pixel correction circuits 49, 50, 51 correct each signal corresponding to the same defective pixel at the same timing based on the common control signal TS supplied from the timing generation circuit 52. There is no need to generate timing for each pixel correction circuit. Therefore, since the capacity of the defect correction data DCD is also reduced, the data transfer amount can be reduced, and the use efficiency of the bus 16 can be improved. In addition, since it is not necessary to provide a timing generation circuit for each defective pixel correction circuit, it is possible to reduce the circuit scale of the SPU 12 and suppress power consumption. Furthermore, since the defect correction data DCD is transferred by the DMAC 17 by the DMA method, the processing efficiency is improved by reducing the processing load of the CPU 14, and the efficiency is improved because only one DMA channel is allocated to transfer the defect correction data DCD. High data transfer is possible.

<Second Embodiment>
In the first embodiment, the configuration of the SPU 12 that processes three types of signals has been described. However, the SPU 12 can process other types of signals using the sampling circuit 24. The SPU 12 according to the second embodiment is characterized in that the sampling circuit 24 shown in FIG. 2 has the circuit configuration shown in FIG.

  In the present embodiment, the imaging sensor 11 generates a G signal, an R signal, and a B signal of 12 bits for each color, converts each color component signal to 6 bits in series, and outputs to the SPU 12. In other words, a 12-bit color component signal is decomposed into 6-bit data signals, and the multiplexed signals CCDD [5: 0], CCDD [11: 6], obtained by multiplexing these data signals in a time division manner, TCCHR [5: 0] is supplied to the SPU 12.

  The timing chart of FIG. 5 shows a waveform example of the multiplexed signals CCDD [5: 0], CCDD [11: 6], TCCHR [5: 0]. The multiplexed signal CCDD [5: 0] includes a 12-bit G signal G [11: 0], a lower 6-bit data signal G [5: 0], and an upper 6-bit data signal G [11: 6]. The multiplexed signal CCDD [5: 0] is divided into 12 bits R signal R [11: 0] and lower 6 bits data signal R [5: 0]. Are divided into the upper 6-bit data signal R [11: 6] and multiplexed by time division, and the multiplexed signal TCCHR [5: 0] is a 12-bit B signal B [11: 0]. Is divided into a lower 6-bit data signal B [5: 0] and an upper 6-bit data signal B [11: 6] and multiplexed in a time division manner. Each multiplexed data signal is input in synchronization with the clock signal SPU2CK.

  The sampling circuit 24 shown in FIG. 4 uses a clock signal SPUCK and a clock signal SPU2CK having a frequency twice that of the clock signal SPUCK to sample and combine multiplexed signals having a width of 12 bits. It has a function of converting into component signals G [11: 0], R [11: 0], and B [11: 0]. Hereinafter, the configuration and operation of the sampling circuit 24 will be described with reference to FIGS.

  The sampling circuit 24 includes a timing controller 89, latch circuits 90A to 93B, a first selector 94 to a third selector 96, and a first register 97 to a third register 99. The timing controller 89 divides the clock signal SPU2CK by two to generate a low enable signal LEN, supplies the low enable signal LEN to the enable terminals EN of the latch circuits 90A, 91A, 92A, and 93A, and 180 for the low enable signal LEN. A high enable signal HEN that is out of phase is generated and supplied to the enable terminals EN of the latch circuits 90B, 91B, 92B, and 93B.

  The latch circuits 90A to 93B latch the data signal input to the terminal D at the falling edge of the clock signal SPU2CK in the high level period of the input signal to the enable terminal EN, and output this from the terminal Q. Specifically, each of the latch circuits 90A, 91A, 92A, and 93A receives the data signal G [5: 0] (= CCDD [5: at the falling edge of the clock signal SPU2CK in the high level period of the low enable signal LEN). 0]), R [5: 0] (= CCDD [11: 6]), B [5: 0] (= TCCHR [5: 0]), X [5: 0] (= TCCHR [11: 6] Latch). The latch circuits 90B, 91B, 92B, and 93B are arranged at the data signal G [11: 6] (= CCDD [5: 0]), at the falling edge of the clock signal SPU2CK in the high level period of the high enable signal HEN. R [11: 6] (= CCDD [11: 6]), B [11: 6] (= TCCHR [5: 0]), X [11: 6] (= TCCHR [11: 6]) are latched . In the present embodiment, the data signals X [5: 0] and X [11: 6] are not assigned to any color component signal.

  Next, the output signals of the latch circuits 90A and 90B are combined and input to the “0” -th terminal of the first selector 94 to the third selector 96 as a 12-bit signal CH0D [11: 0], and the outputs of the latch circuits 91A and 91B. The signals are combined and input to the “1” terminal of the first selector 94 to the third selector 96 as a 12-bit signal CH1D [11: 0], and the output signals of the latch circuits 92A and 92B are combined to form a 12-bit signal CH2D. [11: 0] is input to the “2” -th terminal of the first selector 94 to the third selector 96, and the output signals of the latch circuits 93A and 93B are combined to form a 12-bit signal CH3D [11: 0]. Input to the “3” -th terminal of the first selector 94 to the third selector 96.

  Each of the first selector 94 to the third selector 96 is “0” according to the values of the 2-bit selection control signals TCPHR [1: 0], TCPHG [1: 0], and TCPHB [1: 0] given from the CPU 14. Any one of the “th terminal to“ 4 ”th terminals is selected, and the signal input to the selected terminal is output.

  In this example, the first selector 94 selects the “1” terminal and outputs an R signal, the second selector 95 selects the “0” terminal and outputs a G signal, and the third selector 96 outputs “ The 2 "th terminal is selected and the B signal is output. Note that data XX shown in FIG. 5 indicates data that is not selected by the first selector 94 to the third selector 96.

  The first register 97 to the third register 99 latch the signals input from the first selector 94 to the third selector 96 at the rising edge of the clock signal SPUCK, respectively, and R signals R [11: 0] (= CHR [ 11: 0]), G signal G [11: 0] (= CHG [11: 0]) and B signal B [11: 0] (= CHB [11: 0]).

  In this way, when the SPU 12 employs the sampling circuit 24 according to the present embodiment, the sampling circuit 24 receives the multiplexed signal transferred in series by 6 bits and converts it into a 12-bit width color component signal. Compared with the case where the component signal is received as it is, the number of input pins provided in the input terminal can be greatly reduced. Therefore, a small circuit scale SPU 12 can be realized.

Although the sampling circuit 24 according to the present embodiment converts a 6-bit width multiplexed signal into a 12-bit color component signal, the present invention is not limited to this, and the N ₂ bit width (N ₂ is a positive integer). It is easy for those skilled in the art to extend the multiplexed signal to a configuration for converting the N ₁ (= 2 × N ₂ ) bit width color component signal.

  The sampling circuit 24 according to the present embodiment has a configuration for converting the multiplexed signal into the three-color component signals G [11: 0], R [11: 0], and B [11: 0]. It is easy for those skilled in the art to extend the configuration to convert the signal into a four-color component signal. Specifically, a selector having the same configuration as the first selector 94 to the third selector 96 and a register having the same configuration as the first register 97 to the third register 99 may be added to the sampling circuit 24.

<Third Embodiment>
Next, a third embodiment of the present invention will be described. The SPU 12 according to the third embodiment is characterized in that the sampling circuit 24 shown in FIG. 2 has the circuit configuration shown in FIG. In the present embodiment, the imaging sensor 11 generates a G signal, an R signal, and a B signal of 12 bits for each color, converts each color component signal into 4 bits in series, and outputs the serial signal to the SPU 12. In other words, a 12-bit width color component signal is decomposed into 4-bit width data signals, and multiplexed signals CCDD [3: 0], CCDD [7: 4], obtained by multiplexing these data signals in a time division manner, CCDD [11: 8] is supplied to the SPU 12.

  The timing chart of FIG. 7 shows a waveform example of the multiplexed signals CCDD [3: 0], CCDD [7: 4], CCDD [11: 8]. The multiplexed signal CCDD [3: 0] is multiplexed by decomposing the 12-bit G signal G [11: 0] into data signals G [3: 0], G [7: 4], G [11: 8]. The multiplexed signal CCDD [7: 4] is composed of the 12-bit R signal R [11: 0] and the data signals R [3: 0], R [7: 4], R [11: 8]. ], And the multiplexed signal CCDD [11: 8] is obtained by converting the 12-bit B signal B [11: 0] into the data signals B [3: 0] and B [7: 4]. , B [11: 8] and multiplexed. Each multiplexed data signal is input in synchronization with the clock signal SPU3CK.

  The sampling circuit 24 shown in FIG. 6 uses the clock signal SPUCK and the clock signal SPU3CK having a frequency three times that of the clock signal SPUCK to sample and combine the multiplexed signal with a 4-bit width. Is converted to 12-bit color component signals G [11: 0], R [11: 0], and B [11: 0]. Hereinafter, the configuration and operation of the sampling circuit 24 will be described with reference to FIGS. 6 and 7.

  The sampling circuit 24 includes a timing controller 106, latch circuits 100A to 102C, and a first register 103 to a third register 105. The timing controller 106 divides the clock signal SPU3CK to generate a first enable signal LEN, a second enable signal MEN, and a third enable signal HEN. The pulse width of the first enable signal LEN corresponds to one period (= T) of the clock signal SPU3CK, and the phase of the second enable signal MEN is only one period (= T) compared to that of the first enable signal LEN. The phase of the third enable signal HEN is delayed by two periods (= 2 × T) compared to that of the first enable signal LEN. The first enable signal LEN is input to the enable terminals EN of the latch circuits 100A, 101A, and 102A, the second enable signal MEN is input to the enable terminals EN of the latch circuits 100B, 101B, and 102B, and the third enable signal HEN is Are input to the enable terminals EN of the latch circuits 100C, 101C, and 102C.

  Each of the latch circuits 100A to 102C latches the data signal input to the terminal D at the falling edge of the clock signal SPU3CK during the high level period of the input signal to the enable terminal EN, and outputs it from the terminal Q. Specifically, each of the latch circuits 100A, 101A, and 102A receives the data signal G [3: 0] (= CCDD [3: 0]) at the falling edge of the clock signal SPU3CK in the high level period of the first enable signal LEN. ), R [3: 0] (= CCDD [7: 4]), B [3: 0] (= CCDD [11: 8]). In addition, the latch circuits 100B, 101B, and 102B respectively receive the data signals G [7: 4] (= CCDD [7: 4]), R [at the falling edge of the clock signal SPU3CK during the high level period of the second enable signal MEN. 7: 4] (= CCDD [7: 4]) and B [7: 4] (= CCDD [11: 8]) are latched. The latch circuits 100C, 101C, and 102C are data signals G [11: 8] (= CCDD [3: 0]), R [at the falling edge of the clock signal SPU3CK in the high level period of the third enable signal HEN, respectively. 11: 8] (= CCDD [7: 4]) and B [11: 8] (= CCDD [11: 8]) are latched.

  Next, the output signals of the latch circuits 100A to 100C are combined and input to the first register 103 as a 12-bit signal, and the output signals of the latch circuits 101A to 101C are combined and input to the second register 104 as a 12-bit signal. The output signals of the latch circuits 102A to 102C are combined and input to the third register 105 as a 12-bit signal.

  Each of the first register 103 to the third register 105 latches the input signal at the rising edge of the clock signal SPUCK, and the G signal G [11: 0] (= CHG [11: 0]) and the R signal R [11: 0] (= CHR [11: 0]) and B signal B [11: 0] (= CHB [11: 0]) are output.

  As described above, when the SPU 12 employs the sampling circuit 24 according to the present embodiment, the multiplexed signal transferred serially by 4 bits is received and converted into a 12-bit width color component signal. Compared with the case of receiving, the number of input pins provided in the input terminal can be further greatly reduced. Therefore, a small circuit scale SPU 12 can be realized.

Although the sampling circuit 24 according to the present embodiment converts the multiplexed signal of 4 bits width into the color component signal of 12 bits width, the present invention is not limited to this, and the N ₂ bit width (N ₂ is a positive integer). It is easy for those skilled in the art to expand the multiplexed signal to a configuration for converting the multiplexed signal into a color component signal having a width of N ₁ (= 3 × N ₂ ).

<Fourth Embodiment>
Next, a fourth embodiment of the present invention will be described. The SPU 12 according to the fourth embodiment is characterized in that the sampling circuit 24 shown in FIG. 2 has the circuit configuration shown in FIG. In the present embodiment, the imaging sensor 11 generates a G signal, an R signal, and a B signal of 12 bits for each color, converts each color component signal to 3 bits in series, and outputs the signal to the SPU 12. In other words, a 12-bit width color component signal is decomposed into 3-bit width data signals, and the multiplexed signals CCDD [2: 0], CCDD [5: 3], obtained by multiplexing these data signals in a time division manner, CCDD [8: 6] is supplied to the SPU 12.

  The timing chart of FIG. 9 shows waveform examples of the multiplexed signals CCDD [2: 0], CCDD [5: 3], and CCDD [8: 6]. The multiplexed signal CCDD [2: 0] is a 12-bit G signal G [11: 0] and data signals G [2: 0], G [5: 3], G [8: 6], G [11: 9], and the multiplexed signal CCDD [5: 3] is a 12-bit R signal R [11: 0] converted into data signals R [2: 0] and R [5: 3]. ], R [8: 6], R [11: 9] and multiplexed, and the multiplexed signal CCDD [8: 6] is a 12-bit B signal B [11: 0] data. The signal B [2: 0], B [5: 3], B [8: 6], and B [11: 9] are decomposed and multiplexed. Each multiplexed data signal is input in synchronization with the clock signal SPU4CK.

  The sampling circuit 24 shown in FIG. 8 uses the clock signal SPUCK and the clock signal SPU4CK having a frequency four times that of the clock signal SPUCK to sample and combine the multiplexed signal with a 3-bit width. Is converted to 12-bit color component signals G [11: 0], R [11: 0], and B [11: 0]. Hereinafter, the configuration and operation of the sampling circuit 24 will be described with reference to FIGS.

The sampling circuit 24 includes a timing controller 116, latch circuits 110A to 112D, and a first register 113 to a third register 115. The timing controller 116 divides the clock signal SPU4CK to generate a first enable signal EN1, a second enable signal EN2, a third enable signal EN3, and a fourth enable signal EN4. The pulse width of the first enable signal EN1 corresponds to one period (= T _K ) of the clock signal SPU4CK, and the phase of the second enable signal EN2 is two periods (= 2 × compared with that of the first enable signal EN1). T _K) delayed, the third enable signal EN3 phase, as compared with that of the first enable signal EN1 3 cycle (= 3 × T _K) delayed, the fourth enable signal EN4 phase, the first enable signal EN1 It is delayed by 4 periods (= 4 × T _K ). The first enable signal EN1 is input to the enable terminals EN of the latch circuits 110A, 111A, and 112A, and the second enable signal EN2 is input to the enable terminals EN of the latch circuits 110B, 111B, and 112B, and the third enable signal EN3. Are input to the enable terminals EN of the latch circuits 110C, 111C, and 112C, and the fourth enable signal EN4 is input to the enable terminals EN of the latch circuits 110D, 111D, and 112D.

  Each of the latch circuits 110A to 112D latches the data signal input to the terminal D at the falling edge of the clock signal SPU4CK during the high level period of the input signal to the enable terminal EN, and outputs it from the terminal Q. Specifically, each of the latch circuits 110A, 111A, and 112A receives the data signal G [2: 0] (= CCDD [2: 0]) at the falling edge of the clock signal SPU4CK in the high level period of the first enable signal EN1. ), R [2: 0] (= CCDD [5: 3]), B [2: 0] (= CCDD [8: 6]). In addition, the latch circuits 110B, 111B, and 112B are connected to the data signals G [5: 3] (= CCDD [2: 0]) and R [at the falling edge of the clock signal SPU4CK in the high level period of the second enable signal EN2, respectively. 5: 3] (= CCDD [5: 3]) and B [5: 3] (= CCDD [8: 6]) are latched. In addition, the latch circuits 110C, 111C, and 112C are connected to the data signals G [8: 6] (= CCDD [2: 0]), R [at the falling edge of the clock signal SPU4CK in the high level period of the third enable signal EN3, respectively. 8: 6] (= CCDD [5: 3]) and B [8: 6] (= CCDD [8: 6]) are latched. The latch circuits 110D, 111D, and 112D respectively receive the data signals G [11: 9] (= CCDD [2: 0]), R [at the falling edge of the clock signal SPU4CK in the high level period of the fourth enable signal EN4. 11: 9] (= CCDD [5: 3]) and B [11: 9] (= CCDD [8: 6]) are latched.

  Next, the output signals of the latch circuits 110A to 110D are combined and input to the first register 113 as a 12-bit signal, and the output signals of the latch circuits 111A to 111D are combined and input to the second register 114 as a 12-bit signal. The output signals of the latch circuits 112A to 112D are combined and input to the third register 115 as a 12-bit signal.

  Each of the first register 113 to the third register 115 latches the input signal at the rising edge of the clock signal SPUCK, and the G signal G [11: 0] (= CHG [11: 0]) and the R signal R [11: 0] (= CHR [11: 0]) and B signal B [11: 0] (= CHB [11: 0]) are output.

  As described above, when the SPU 12 uses the sampling circuit 24 according to the present embodiment, the multiplexed signal transferred serially by 3 bits is received and converted into a 12-bit width color component signal. Compared with the case of receiving, the number of input pins provided in the input terminal can be further greatly reduced. Therefore, a small circuit scale SPU 12 can be realized.

Although the sampling circuit 24 according to the present embodiment converts the multiplexed signal of 3 bit width into the color component signal of 12 bit width, the present invention is not limited to this, and the N ₂ bit width (N ₂ is a positive integer). It is easy for those skilled in the art to expand the multiplexed signal to a configuration for converting the multiplexed signal into a color component signal having a width of N ₁ (= 4 × N ₂ ).

<Fifth Embodiment>
Next, a fifth embodiment of the present invention will be described. The SPU 12 according to the fifth embodiment is characterized in that the sampling circuit 24 shown in FIG. 2 has the circuit configuration shown in FIG. In the present embodiment, the imaging sensor 11 generates a G signal, a R signal, and a B signal of 12 bits for each color, and outputs a multiplexed signal obtained by multiplexing these color component signals in a time division manner to the SPU 12. The timing chart of FIG. 11 shows a waveform example of the multiplexed signal CCDD [11: 0]. In FIG. 11, R, G, and B represent an R signal, a G signal, and a B signal, respectively, and X is not any color component signal (hereinafter referred to as an X signal). The multiplexed signal is input in synchronization with the clock signal SPU4CK.

  The sampling circuit 24 shown in FIG. 10 has a function of sampling the multiplexed signal CCDD [11: 0] using the clock signal SPUCK and the clock signal SPU4CK having a frequency four times that of the clock signal SPUCK and outputting the multiplexed signal CCDD [11: 0] in parallel. Have. Hereinafter, the configuration and operation of the sampling circuit 24 will be described with reference to FIGS. 10 and 11.

The sampling circuit 24 includes a timing controller 120, latch circuits 121 to 124, a first selector 126 to a third selector 128, and a first register 129 to a third register 131. The timing controller 120 divides the clock signal SPU4CK to generate a first enable signal EN1, a second enable signal EN2, a third enable signal EN3, and a fourth enable signal EN4. The pulse width of the first enable signal EN1 corresponds to one period (= T _K ) of the clock signal SPU4CK, and the phase of the second enable signal EN2 is two periods (= 2 × compared with that of the first enable signal EN1). T _K) delayed, the third enable signal EN3 phase, as compared with that of the first enable signal EN1 3 cycle (= 3 × T _K) delayed, the fourth enable signal EN4 phase, the first enable signal EN1 It is delayed by 4 periods (= 4 × T _K ). The first enable signal EN1 to the fourth enable signal EN4 are input to the enable terminals EN of the latch circuits 121 to 124, respectively.

  Each of the latch circuits 121 to 124 latches the signal input to the terminal D at the falling edge of the clock signal SPU4CK during the high level period of the input signal to the enable terminal EN and outputs it from the terminal Q. Specifically, the latch circuit 121 latches the R signal CH0D [11: 0] in the multiplexed signal at the falling edge of the clock signal SPU4CK during the high level period of the first enable signal EN1, and the first selectors 126˜ The latch circuit 122 latches the G signal CH1D [11: 0] at the falling edge of the clock signal SPU4CK during the high level period of the second enable signal EN2 and outputs it to the “0” -th terminal of the third selector 128. The latch circuit 123 outputs the B signal CH2D [11: 0] at the falling edge of the clock signal SPU4CK during the high level period of the third enable signal EN3. The data is latched and output to the “2” -th terminal of the first selector 126 to the third selector 128, and the latch circuit 124 receives the fourth enable signal. Falling edge X signal CH3D clock signal SPU4CK the high level period of EN4 [11: 0] is output to the "3" -th terminal of the first selector 126 to the third selector 128 is latched.

  Each of the first selector 126 to the third selector 128 is “0” according to the values of 2-bit selection control signals TCPHR [1: 0], TCPHG [1: 0], and TCPHB [1: 0] given from the CPU 14. Any one of the “th terminal to“ 3 ”terminal is selected, and the signal input to the selected terminal is output. In this example, the first selector 126 selects the “0” terminal, the second selector 127 selects the “1” terminal, and the third selector 128 selects the “2” terminal. The first register 129 to the third register 131 latch the output signals of the first selector 126 to the third selector 128 at the rising edge of the clock signal SPUKK, and the R signal CHR [11: 0] and the G signal CHG [ 11: 0] and B signal CHB [11: 0].

  As described above, when the SPU 12 employs the sampling circuit 24 according to the present embodiment, the sampling circuit 24 can separate and output a plurality of multiplexed color component signals in parallel. Processing can be performed. In addition, the number of input pins provided in the input terminal can be greatly reduced as compared with the case where the color component signal is received as it is. Therefore, a small circuit scale SPU 12 can be realized.

The sampling circuit 24 according to this embodiment, had converted a multiplexed signal of 12-bit wide parallel signal 36 bits wide, not limited thereto in the present invention, generally N _1-bit width (N ₁ is It is easy for those skilled in the art to expand a configuration in which a multiplexed signal of (positive integer) is converted into a parallel signal of N ₂ bit width (N ₂ is a positive integer multiple of N ₁ ). As a particularly realistic configuration, as in the above embodiment, when N ₂ is four times N ₁ , the four color component signals are separated from the multiplexed signal, and N ₂ is three times N ₁ . In which the three color component signals are separated from the multiplexed signal.

<Sixth Embodiment>
Next, a sixth embodiment of the present invention will be described. FIG. 12 is a block diagram schematically showing a configuration of an image processing system according to the sixth embodiment. This image processing system includes an SPU 12, a bus 16, a main memory 18, and a DMAC 17.

  The SPU 12 includes a signal processing unit 12a that processes in parallel two image signals read out in parallel from the image sensor 11, and two output control circuits 58A and 58B. The signal processing unit 12a has the configuration shown in FIGS. 2 and 3 of the first embodiment, and the SPU 12 according to the present embodiment further includes two output control circuits 58A and 58B. The output control circuit 58 shown in FIG. 3 may be adopted as one output control circuit 58A, and the other output control circuit 58B receiving the output result of the sub-sampling circuit 60 shown in FIG. 3 may be adopted.

The image sensor 11 of this embodiment includes a light receiving circuit 70 made of a CCD, a first analog signal processing circuit 74A, and a second analog signal processing circuit 74B. The light receiving circuit 70 further includes vertical transfer units 71 _{0 to} 71. _n (n is an integer of 3 or more), a first horizontal transfer unit 72A, a first amplifier 73A, a second horizontal transfer unit 72B, and a second amplifier 73B.

A photodiode (not shown) of the light receiving circuit 70 photoelectrically converts incident light to generate carriers (charges or holes). The vertical transfer units 71 _{0 to} 71 _n transfer the generated carrier signal to the first horizontal transfer unit 72A and the second horizontal transfer unit 72B. Among the vertical transfer units 71 _{0 to} 71 _n , the even-numbered vertical transfer units 71 ₀ , 71 ₂ ,... Transfer signals to the first horizontal transfer unit 72A, and the odd-numbered vertical transfer units 71 ₁ , 71 ₃ ,. Transfers the signal to the second horizontal transfer section 72B. As described above, the light receiving circuit 70 includes two light receiving units which are divided into even-numbered vertical transfer units 71 ₀ , 71 ₂ ,... And odd-numbered vertical transfer units 71 ₁ , 71 ₃ ,. The carriers generated in the two light receiving units are transferred to the first horizontal transfer unit 72A and the second horizontal transfer unit 72B, respectively, and then transferred to the first amplifier 73A and the second amplifier 73B in the horizontal direction. Each of the first amplifier 73A and the second amplifier 73B amplifies the transferred signal and outputs the amplified signal to the first analog signal processing circuit 74A and the second analog signal processing circuit 74B. Next, the first analog signal processing circuit 74A and the second analog signal processing circuit 74B perform CDS processing, AGC processing, and A / D conversion processing on the input signal, respectively, and then output a digital image signal to the SPU 12.

  The image signals input from the signal processing unit 12a to the output control circuit 58A and the output control circuit 58B are transferred to the main memory 18 via the bus 16 under the control of the DMAC 17. Here, the DMAC 17 assigns the DMA channel CH0 for data transfer between the output control circuit 58A and the main memory 18, and assigns the DMA channel CH1 for data transfer between the output control circuit 58B and the main memory 18. Then, the image data TD transferred and stored in the main memory 18 is read out, transferred to the RPU 13 and processed.

  As described above, the SPU 12 can process two image signals in parallel and transfer them in parallel to other modules via the bus 16, so that image processing can be performed efficiently. If the SPU 12 has only a single output control circuit 58A, a line memory is required to buffer the other image signal while the output control circuit 58A transfers one of the two image signals. In this embodiment, since two image signals can be DMA-transferred in parallel, there is no need to provide a line memory.

  In the present embodiment, the SPU 12 includes two output control circuits 58A and 58B, but instead of the output control circuit 58B, the one of the RPU 13 may be used. By causing the selector 62 shown in FIG. 3 to select the “1” -th terminal, the signal output from the sub-sampling circuit 60 can be transferred to the RPU 13 and transferred to the main memory 18 via the output control circuit of the RPU 13. Is possible.

<Seventh Embodiment>
Next, a seventh embodiment of the present invention will be described. FIG. 13 is a block diagram schematically illustrating a configuration of an image processing system according to the seventh embodiment. This image processing system includes an SPU 12 and a DMAC 17. The configurations of the SPU 12 and the DMAC 17 are the same as those shown in FIG. The imaging sensor 11 of the present embodiment is substantially the same as the configuration of the imaging sensor 11 illustrated in FIG. 12 except for the configuration of the light receiving circuit 80.

The light receiving circuit 80 of the present embodiment includes a first light receiving unit 81 and a second light receiving unit 82 that are configured independently of each other, and the first light receiving unit 81 includes K + 1 (K is a positive integer) vertical transfer units 82 _0. and ~82 _K, L + 1 present (L is a positive integer) and a vertical transfer unit 81 ₀ to 81 _L of.

The photodiodes (not shown) of the first light receiving unit 81 and the second light receiving unit 82 each generate carriers by photoelectrically converting incident light. The vertical transfer units 82 _{0 to} 82 _K of the first light receiving unit 81 transfer the carrier signal to the first horizontal transfer unit 83B, and the vertical transfer units 81 _{0 to} 81 _L of the second light receiving unit 82 receive the second carrier signal. Transfer to the horizontal transfer unit 83A. Next, the first horizontal transfer unit 83B transfers the transferred signal to the first amplifier 84B, and the first amplifier 84B amplifies the signal and outputs it to the first analog signal processing circuit 74A. On the other hand, the second horizontal transfer unit 83A transfers the transferred signal to the second amplifier 84A, and the second amplifier 84A amplifies the signal and outputs it to the second analog signal processing circuit 74B. As a result, two digital image signals are supplied in parallel to the signal processing unit 12a of the SPU 12.

  Compared with the first horizontal transfer unit 72A and the second horizontal transfer unit 72B of the imaging sensor 11 shown in FIG. 12, the first horizontal transfer unit 83B and the second horizontal transfer unit 83A require substantially half the number of signal transfer stages. Has the advantage. In recent years, high-resolution imaging devices have been provided, but there is a problem that the higher the resolution in the horizontal direction, the greater the number of stages of the horizontal transfer unit, making it difficult to transfer signals at high speed. The configuration of the light receiving circuit 80 of the present embodiment improves this kind of problem. However, since the signal reading directions of the first light receiving portion 81 and the second light receiving portion 82 are opposite to each other, the signal arrangement read from one light receiving portion is matched with the signal arrangement read from the other light receiving portion. Need to be sorted. As schematically shown in FIG. 14, among the image signals 90 read from the light receiving circuit 80, the right half image signal 90 </ b> B is read from the first light receiving unit 81 in the scanning direction indicated by the arrow, and the second The left half image signal 90A is read from the light receiving unit 82 in the scanning direction indicated by the arrow, and it can be seen that the scanning directions are opposite to each other.

  In the present embodiment, one of the two image signals output from the output control circuit 58A and the output control circuit 58B and transferred to the main memory 18 is incremented in the opposite direction with respect to the other transfer destination address. It is a feature. Specifically, the DMAC 17 assigns the DMA channel CH0 to data transfer between the output control circuit 58A and the main memory 18, and assigns the DMA channel CH1 to data transfer between the output control circuit 58B and the main memory 18. The DMA channel CH0 increments the transfer destination address in the opposite direction to the transfer destination address generated by the DMA channel CH1. Thereby, as schematically shown in FIG. 15, the signal arrangement of the left half image signal 91A can be matched with the signal arrangement of the right half image signal 91B.

  As described above, in this embodiment, the step of rearranging one of the left and right image signals of the image data TD transferred from the SPU 12 to the main memory 18 can be omitted, so that the image processing can be speeded up. Become.

DESCRIPTION OF SYMBOLS 1 Imaging device 10 Optical mechanism 11 Imaging sensor 12 SPU
13 RPU
14 CPU
15 System clock generator 16 Bus 17 DMAC (DMA controller)
18 Main memory 19 ROM

Claims (16)

In an image processing apparatus that processes either a YUV signal composed of a luminance signal and a color difference signal and a plurality of color component signals,
An input terminal for inputting the YUV signal and the plurality of color component signals;
A selector that outputs a selection signal obtained by selecting any one of the input YUV signal and the plurality of color component signals;
A signal processing unit for processing the selection signal,
The image processing apparatus, wherein the input terminal is shared by the YUV signal and the color component signal. The image processing apparatus according to claim 1, wherein the signal processing unit includes:
A plurality of defective pixel correction circuits for correcting a plurality of the selection signals corresponding to defective pixels in the imaging sensor according to a predetermined timing;
A defective pixel correction timing generation circuit that generates the predetermined timing for performing defective pixel correction in accordance with the input of the selection signal,
The image processing apparatus, wherein the plurality of defective pixel correction circuits correct the selection signals corresponding to the same defective pixel in parallel at the same predetermined timing. The image processing apparatus according to claim 1 or 2,
An oversampling circuit that interpolates the insufficient signal using the input signal and outputs the signal to the selector;
A separation circuit that separates the YUV signal into a plurality of component signals;
The YUV signal is composed of a luminance signal and a subsampled color difference signal, and the oversampling circuit interpolates an insufficient color difference signal using the subsampled color difference signal output from the separation circuit. , Image processing device.   4. The image processing apparatus according to claim 3, further comprising an output control circuit that outputs the YUV signal to a bus as it is. An image processing apparatus for processing a multiplexed signal output from an imaging sensor,
An input terminal to which the multiplexed signal is input;
A sampling circuit for sampling the multiplexed signal that has been input,
The multiplexed signal decomposes a color component signal of N ₁ bit width (N ₁ is a positive integer) into a plurality of data signals of N ₂ bit width (N ₂ is a positive integer; N ₁ is twice N ₂ ). It consists of multiplexing in time division,
The image processing apparatus, wherein the sampling circuit has a function of combining the N ₂ bit width data signal and converting the data signal into the N ₁ bit width color component signal. An image processing apparatus for processing a multiplexed signal output from an imaging sensor,
An input terminal to which the multiplexed signal is input;
A sampling circuit for sampling the multiplexed signal that has been input,
The multiplexed signal decomposes a color component signal of N ₁ bit width (N ₁ is a positive integer) into a plurality of data signals of N ₂ bit width (N ₂ is a positive integer; N ₁ is 3 times N ₂ ). It consists of multiplexing in time division,
The image processing apparatus, wherein the sampling circuit has a function of combining the N ₂ bit width data signal and converting the data signal into the N ₁ bit width color component signal. An image processing apparatus for processing a multiplexed signal output from an imaging sensor,
An input terminal to which the multiplexed signal is input;
A sampling circuit for sampling the multiplexed signal that has been input,
The multiplexed signal decomposes a color component signal of N ₁ bit width (N ₁ is a positive integer) into a plurality of data signals of N ₂ bit width (N ₂ is a positive integer; N ₁ is 4 times N ₂ ). It consists of multiplexing in time division,
The image processing apparatus, wherein the sampling circuit has a function of combining the N ₂ bit width data signal and converting the data signal into the N ₁ bit width color component signal. An image processing apparatus for processing a multiplexed signal of N ₁ bit width (N ₁ is a positive integer) configured by multiplexing a plurality of color component signals in a time division manner,
An input terminal to which the multiplexed signal is input;
A sampling circuit that samples the plurality of color component signals from the input multiplexed signal and outputs them in parallel as a signal of N ₂ bit width (N ₂ is a multiple of N ₁ );
A signal processing unit that processes the plurality of color component signals output in parallel from the sampling circuit,
The sampling circuit samples the plurality of color component signals using a clock signal having a frequency N ₂ / N ₁ times that of a clock signal synchronized with the output of the N ₂ -bit width signal. Image processing device. 9. The image processing apparatus according to claim 8, wherein the sampling circuit uses a clock signal having a frequency four times that of the clock signal synchronized with the output of the N ₂ bit width signal (N ₂ is four times N ₁ ). An image processing apparatus that samples the plurality of color component signals. 9. The image processing apparatus according to claim 8, wherein the sampling circuit uses a clock signal having a frequency three times that of the clock signal synchronized with the output of the N ₂ bit width signal (N ₂ is three times N ₁ ). An image processing apparatus that samples the plurality of color component signals. An image processing system for processing an image signal,
A signal processing unit for processing in parallel a plurality of image signals read in parallel from a plurality of light receiving units of the image sensor;
A plurality of output control circuits for outputting a plurality of image signals processed by the signal processing unit to a bus;
A data transfer control unit for transferring the plurality of image signals output to the bus;
An image processing system comprising:   12. The image processing system according to claim 11, wherein the data transfer control unit includes a DMA (direct memory access) controller. The image processing system according to claim 11 or 12, wherein the imaging sensor has a function of outputting two image signals read in opposite directions to each other,
The plurality of output control circuits each output two image signals processed in parallel by the signal processing unit to the bus,
The data transfer control unit has a function of incrementing one transfer destination address of the two image signals in a reverse direction with respect to the other transfer destination address of the two image signals.   14. The image processing system according to claim 11, wherein the signal processing unit also has a function of processing a plurality of color component signals input in parallel from an imaging sensor in parallel. system.   The imaging device provided with the image processing apparatus of any one of Claims 1-10.   The imaging device provided with the image processing system of any one of Claims 11-14.

Images