JP2007036774A - Image processing apparatus and imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve flexibility of an image processing apparatus against change of an input sequence of image signals and to make its circuit scale small. <P>SOLUTION: The image processing apparatus is equipped with a storage means having individual storage areas corresponding to specific filter components to be processed by a camera signal processing section 23 in parallel, a rearrangement processing section 21 comprising a writing means of allocating input image signals of a plurality of channels based upon the output signal of a solid-state imaging element by the specified filter components based upon settings of control parameters corresponding to the input sequence of the filter components of the signals and writing them to the corresponding storage areas, and a readout control section which reads the image signals out of the respective storage areas in sequence through individual output channels, and a channel ID generation section 22 which generates channel IDs corresponding to the image signals having been rearranged based upon settings of control parameters for channel ID generation and supplies the channel IDs to a camera signal processing section 23 together with the image signals. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention includes an image processing device that processes a color image signal and an imaging device that has this function, and in particular, a signal processing circuit that has a function of processing image signals of a plurality of specific filter components in parallel. The present invention relates to an image processing apparatus and an imaging apparatus.

  Imaging devices such as digital video cameras and digital still cameras are equipped with CCD (Charge Coupled Devices) or CMOS (Complementary Metal Oxide Semiconductor) type image sensors (hereinafter referred to as CMOS sensors) as imaging devices. In general, these image pickup devices convert signals read from two-dimensionally arranged pixels into one data stream and output the data stream from one output channel. For example, in a general CCD, pixel signals are transferred to a vertical register arranged for each column, and then the signals are transferred to a horizontal register for each row and output to be converted into one data stream. . Further, in the CMOS sensor, for example, pixel signals for one row are read out and accumulated in capacitors in each column, and then these signals are output in order from the end to be converted into one data stream.

FIG. 43 is a block diagram illustrating a configuration example of a main part of an imaging apparatus using such a one-channel output type imaging device.
43, an analog front end (AFE) circuit 912 includes a CDS (Correlated Double Sampling) circuit, an AGC (Auto Gain Control) circuit, an A / D conversion circuit, and the like, and digitally converts an analog image signal from the image sensor 911. And output. The camera signal processing circuit 913 performs various camera signal processing such as digital clamp, noise reduction, defect correction, demosaic (simultaneous processing), white balance adjustment, resolution conversion, and the like on the image signal from the AFE circuit 912, and finally Specifically, an image signal is output to the baseband processing circuit 914 as a luminance signal (Y) and a color difference signal (C). The baseband processing circuit 914 performs baseband processing such as compression encoding of an input image signal and conversion to a monitor display signal.

  Here, a general Bayer arrangement is taken as an example of filter coding of the image sensor, and a color sequence of an image signal input to the camera signal processing circuit 913 is considered. First, FIG. 44 is a diagram for explaining a color sequence when a pixel signal of an image sensor with a Bayer array is output from one output channel.

  As shown in FIG. 44A, the Bayer array image sensor has a pixel array in which R and Gr are alternately arranged in the odd-numbered rows from the top and Gb and B are alternately arranged in the even-numbered rows from the top. When there is one output channel, scanning is sequentially performed in the horizontal direction from the upper left of such a pixel array, and when one horizontal synchronization period ends, the next row is scanned to read out pixel signals. Therefore, the color sequence of the pixel signal output from the image sensor 911 to the output channel Ch1 is R, Gr, R, Gr... In the odd-numbered horizontal synchronization period as shown in FIG. In the even-numbered horizontal synchronization period, Gb, B, Gb, B. Accordingly, in the camera signal processing circuit 913 that performs processing for each color, the color sequence is performed in accordance with the color sequence for each process for R, G, and B with respect to the input signal. It needs to be done appropriately while planning.

  By the way, in recent years, there are an increasing number of image pickup apparatuses that employ a multi-pixel image pickup element that well exceeds one million pixels. In such an image pickup device, since the number of pixels is large, the readout frequency is drastically increased in the method of reading out the pixel signal with one channel as described above. In particular, when the readout time per screen is limited as in a video camera, it is not easy to realize. For example, if the readout frequency becomes too high, not only the power consumption increases but also analog signal processing limitations. This makes it impossible to read out signals properly.

  In order to avoid such a problem, recently, a camera signal processing system has been developed in which pixel signals of an image sensor are read out in multiple channels. If pixel signals can be read out from the image sensor with multiple channels, the readout frequency per channel can be lowered, and the above problems can be avoided.

  Note that when multi-channel reading is employed, there may be a specific problem such as analog signal variation between channels. For this reason, a technique for detecting a level difference between channels and performing level correction (for example, see Patent Document 1), a technique for detecting the level difference from integrated values over a plurality of lines (for example, see Patent Document 2), and the like. Has traditionally been considered.

  Hereinafter, an example of a color sequence in the case where a plurality of output channels are provided in the Bayer array image sensor will be described. 45 to 47 are diagrams illustrating color sequences when pixel signals are read out in 2 channels, 3 channels, and 4 channels, respectively.

  In FIG. 45A, output is alternately performed from the first pixel to the two output channels Ch1 and Ch2. In this case, as shown in FIG. 45B, the readout frequency can be halved by simultaneously reading out two pixels per clock for each output channel. In FIG. 46A, three output channels Ch1 to Ch3 are assigned and output in order from the top pixel. In this case, as shown in FIG. 46B, by simultaneously outputting 3 pixels per clock, the readout frequency can be reduced to 1/3. In FIG. 47A, four output channels Ch1 to Ch4 are assigned and output in order from the top pixel. In this case, as shown in FIG. 47B, by simultaneously outputting 4 pixels per clock, the readout frequency can be reduced to ¼.

  Further, as shown in FIGS. 48 to 50 below, there is a method of simultaneously reading out pixel signals for a plurality of lines to each output channel. 48 to 50 are diagrams for explaining color sequences when pixel signals of 2 lines are read out by 2 channels, 4 channels, and 6 channels, respectively.

  In FIG. 48A, the output channel Ch1 is assigned to the odd-numbered rows on the image sensor, and the output channel Ch2 is assigned to the even-numbered rows for output. In this case, as shown in FIG. 48B, the readout frequency can be halved by simultaneously reading out pixel signals for two rows per clock.

  In FIG. 49A, output channels Ch1 to Ch4 are assigned in units of 2 rows × 2 columns on the image sensor. In this example, the first and second row signals in the first column are output to output channels Ch1 and Ch2, respectively, and the first and second row signals in the second column are output to output channels Ch3 and Ch4, respectively. Yes. In this case, as shown in FIG. 49B, the readout frequency can be reduced to ¼ by simultaneously reading out 4 pixels per clock.

  In FIG. 50A, output channels Ch1 to Ch6 are assigned in units of 2 rows × 3 columns on the image sensor. In this example, the first and second row signals in the first column are output channels Ch1 and Ch2, respectively, and the first and second row signals in the second column are output channels Ch3 and Ch4, respectively. The signals in the first and second rows of the eye are output to the output channels Ch5 and Ch6, respectively. In this case, as shown in FIG. 50B, the readout frequency can be reduced to 1/6 by simultaneously reading out 6 pixels per clock.

  Note that, in an XY address scanning type imaging device such as a CMOS sensor, various readout operations as described above can be realized relatively easily without greatly changing the basic structure.

  As described above, by using a plurality of output channels, the total number of clocks for reading out pixel signals for one screen can be reduced, and the readout frequency can be reduced. On the other hand, depending on the number of output channels and which pixel groups are read in parallel, there are many output sequences as described above. Therefore, it is necessary to develop and install a signal processing circuit suitable for each of these output sequences. There will be.

  For example, in FIG. 49, four filter components R, Gb, Gr, and B repeatedly appear at the time of reading, but the repetition number “4” (or an integer multiple thereof) and the number of output channels are as follows. If they match, as shown in FIG. 49, only one signal of the same filter component can be output from one output channel in any horizontal synchronization period. As a result, in the subsequent circuit (for example, the camera signal processing circuit 913 in FIG. 43), the signal can be processed for each same filter component, so that the circuit configuration can be simplified. However, as in the case of FIG. 50, for example, the above condition often does not hold, and the above condition is always satisfied even when considering the number of output channels and the read frequency based on the circuit scale and manufacturing cost. It is not easy to do.

  Furthermore, there is a system in which a multi-channel signal output from an image sensor is not passed to a subsequent circuit as it is, but is multiplexed and transmitted after reducing the number of channels. FIG. 51 is a block diagram showing a configuration example of a signal processing system having such a multiplexing function.

  The signal processing system of FIG. 51 is different from the configuration of FIG. 43 in that an AFE circuit 912a is provided with an image signal multiplexing (MUX) function. The AFE circuit 912a multiplexes the output signal of the N channel (N is an integer of 2 or more) from the image sensor 911, for example, as an N / 2 channel signal. For example, the number of output channels can be reduced by time-division multiplexing signals of two adjacent output channels from the image sensor 911. Such multiplexing doubles the output frequency, but can be realized by performing multiplexing processing after digital conversion of the pixel signal.

In such a multiplexing process, since there are many possible combinations of channels to be multiplexed, in addition to the number of output channels from the image sensor and the pattern of parallel reading described above, signals corresponding to each of a number of different output sequences. It becomes necessary to develop a processing circuit.
Japanese Patent Laid-Open No. 7-75019 (paragraph numbers [0013] to [0016], FIG. 1) Japanese Patent Laid-Open No. 2002-252808 (paragraph numbers [0020] to [0033], FIG. 1)

  As described above, the output sequence of pixel signals from the image sensor has a very large number of patterns, and when these are multiplexed, a larger number of patterns can be considered. The signal processing circuit in the subsequent stage needs to be configured to perform such an operation in consideration of the output sequence in all the internal blocks, but in order to be able to deal with many patterns as described above, It becomes a complicated and large-scale circuit configuration.

  In recent years, with the aim of reducing development and manufacturing costs when launching various imaging devices, it is possible to support image sensors with different specifications such as the number of pixels, and the basic configuration has been changed over the next year or several years. Therefore, it is required to design a signal processing circuit so that it can be used without being used. However, at the time of development of a new signal processing circuit, it is difficult to determine all pixel signal readout methods to be adopted in the next several years, and there is a limit to the range of specifications that can be handled.

  To solve such a problem, by providing a circuit for rearranging the image signal for each necessary filter component at the input stage of the camera signal processing circuit 913, a specific filter component is always set according to the same rule regardless of the input sequence. It is conceivable to input an image signal to the camera signal processing circuit 913. However, in this case, while the input sequence to the camera signal processing circuit 913 is simplified, the sequence of the source channel indicating from which output channel of the image sensor the input signal is output is also rearranged. .

  In the camera signal processing circuit 913, there are blocks that perform processing for each source channel, such as a digital clamp and a function for correcting the gain between channels. For this reason, even if the rearrangement circuit can cope with a large number of input sequences, various source channel sequences appear according to variations of the input sequence, and eventually a wide part of the camera signal processing circuit 913 is wide. It has been a problem that it becomes impossible to have versatility.

SUMMARY An advantage of some aspects of the invention is that it provides an image processing apparatus that is highly versatile with respect to changes in an input sequence of image signals and has a small circuit scale.
Another object of the present invention is to provide an imaging apparatus having an image processing function that is highly versatile with respect to changes in the input sequence of image signals and has a small circuit scale.

  In the present invention, in order to solve the above-described problem, in an image processing apparatus that processes a color image signal, a channel ID that identifies from which reading channel of the solid-state image sensor the input image signal is read, Signal processing means for processing image signals of a plurality of specific filter components in parallel, storage means having individual storage areas respectively corresponding to the specific filter components, and an input sequence of filter components in the image signals A parameter receiving means for receiving control parameter settings including a rearrangement control parameter and a channel ID generation parameter corresponding to the channel ID sequence corresponding to the image signal read from the storage means; and The input image signal of a plurality of channels based on the output signal is Based on the setting of the control parameter in accordance with the input sequence, writing means for assigning to each specific filter component and writing to the corresponding storage area, and sequentially reading out image signals from the respective storage areas through individual output channels A reading unit that supplies the signal processing unit, and a channel that generates the channel ID corresponding to the image signal read by the reading unit based on the setting of the channel ID generation parameter and supplies the channel ID to the signal processing unit. An image processing apparatus having an ID generation unit is provided.

  In such an image processing apparatus, a channel ID for identifying from which readout channel of the solid-state imaging device the image signal is read is defined. The parameter receiving means includes a rearrangement control parameter corresponding to the filter component input sequence in the image signal, and a channel ID generation parameter corresponding to the channel ID sequence corresponding to the image signal read from the storage means. Accepts control parameter settings. The storage means includes individual storage areas respectively corresponding to specific filter components processed in parallel by the signal processing means.

  When an image signal of a plurality of channels based on the output signal of the solid-state image sensor is input, the writing unit converts the input image signal into a specific filter based on the control parameter setting corresponding to the input sequence in the image signal. Sort by component and write to the corresponding storage area. The reading means sequentially reads image signals from each storage area through individual output channels and supplies them to the signal processing means. As a result, an image signal rearranged for each specific filter component processed in parallel by the signal processing means is output from each output channel of the storage means, and always has a constant rule regardless of the input sequence of the image signal. Thus, the image signal in which the filter components are arranged is supplied to the signal processing means.

  Further, the channel ID generating means generates a channel ID corresponding to the image signal read by the reading means based on the setting of the channel ID generation parameter, and supplies it to the signal processing means. The signal processing means can determine from which reading channel of the image sensor the image signal from the reading means is output by the channel ID from the channel ID generation means, and can execute processing corresponding to the reading channel. .

  Further, according to the present invention, in an image processing apparatus that processes a color image signal, a plurality of specific filters are used by using a channel ID that identifies from which reading channel of the solid-state imaging device the input image signal is read. Signal processing means for processing image signals of components in parallel; channel ID generation means for assigning the channel ID to a plurality of channel image signals based on the output signal of the solid-state imaging device; and outputting together with the image signals; Storage means having individual storage areas respectively corresponding to specific filter components, parameter receiving means for accepting setting of rearrangement control parameters according to the input sequence of filter components in the image signal, and the channel ID generation means Input image signals of a plurality of channels and the channel corresponding to the image signals. The channel ID from the ID ID generating unit is assigned to each specific filter component based on the setting of the rearrangement control parameter according to the input sequence in the image signal, and is written to the corresponding storage area And an image processing apparatus comprising: a reading unit that sequentially reads an image signal and a corresponding channel ID from each storage area through an individual output channel and supplies the image signal to the signal processing unit. Is done.

  In such an image processing apparatus, a channel ID for identifying from which readout channel of the solid-state imaging device the image signal is read is defined. The parameter receiving unit accepts a rearrangement control parameter setting corresponding to the filter component input sequence in the image signal. The storage means includes individual storage areas respectively corresponding to specific filter components processed in parallel by the signal processing means.

  When an image signal of a plurality of channels based on the output signal of the solid-state image sensor is input, the channel ID generation unit assigns a channel ID to the input image signal and outputs it together with the image signal. The writing means specifies the image signal input via the channel ID generation means and the corresponding channel ID from the channel ID generation means based on the control parameter setting according to the input sequence in the image signal. Are sorted for each filter component and written to the corresponding storage area. The reading means sequentially reads the image signal and the corresponding channel ID from each storage area through individual output channels, and supplies them to the signal processing means.

  As a result, an image signal rearranged for each specific filter component processed in parallel by the signal processing means is output from each output channel of the storage means, and always has a constant rule regardless of the input sequence of the image signal. Thus, the image signal in which the filter components are arranged is supplied to the signal processing means. At the same time, the signal processing means can determine from which reading channel of the image sensor the image signal from the reading means is output by the channel ID from the reading means, and can execute processing corresponding to the reading channel. Become.

  According to the image processing apparatus of the present invention, the input image signal of a plurality of channels based on the output signal of the solid-state imaging device is processed in parallel by the signal processing unit according to the setting of the control parameter corresponding to the input sequence by the writing unit. Each specific filter component is sorted and written to the corresponding storage area, and the image signal is sequentially read out from each storage area through an individual output channel by the reading means and supplied to the signal processing means. Thereby, since each image channel rearranged for each specific filter component processed in parallel by the signal processing means is output from each output channel of the storage means, even when the input sequence of the image signals is different, By simply setting the control parameter, an image signal in which filter components are always arranged according to a certain rule can be supplied to the signal processing means, and the signal processing means can perform parallel processing for each filter component by simple processing.

  Further, since the channel ID corresponding to the image signal from the reading unit is generated by the channel ID generating unit based on the setting of the channel ID generation parameter, the signal processing unit outputs the image signal from the reading unit to which of the image sensors. Whether the data is output from the read channel can be easily determined by the channel ID from the channel ID generating means, and processing corresponding to the read channel can be executed by a simple procedure using the channel ID.

  Further, according to the image processing apparatus of the present invention, the channel ID generation unit assigns channel IDs to the input image signals of a plurality of channels based on the output signal of the solid-state imaging device, and outputs them together with the image signals. Then, the image signal and the corresponding channel ID are assigned to the specific filter components processed in parallel by the signal processing unit according to the setting of the control parameter corresponding to the input sequence by the writing unit, and corresponding After being written in the storage area, the image signal and the channel ID are sequentially read out from each storage area through individual output channels by the reading means and supplied to the signal processing means.

  As a result, an image signal rearranged for each specific filter component processed in parallel by the signal processing means is output from each output channel of the storage means, and always has a constant rule regardless of the input sequence of the image signal. Thus, the image signal in which the filter components are arranged is supplied to the signal processing means. At the same time, the channel ID corresponding to the image signal is read by the reading unit, so that the signal processing unit can easily determine from which reading channel of the image sensor the image signal from the reading unit is output by the channel ID. The process corresponding to the read channel can be executed by a simple procedure using the channel ID.

  Therefore, in any of the image processing apparatuses having the above two configurations, for example, the input of filter components based on a combination of specifications such as the number of readout channels of the solid-state imaging device, the method of multiplexing the readout image signal, the number of pixels, and filter coding A wide range of channel ID sequences that appear in response to sequence changes and image signal rearrangements can be handled without changing the circuit configuration, and all of these input sequences and channel ID sequences are supported. The circuit scale can be greatly reduced as compared with the case where the processing function is provided in the signal processing means.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[First Embodiment]
FIG. 1 is a block diagram showing a main configuration of the imaging apparatus according to the first embodiment of the present invention.

  1 includes an optical block 11, a driver 11a, a CMOS image sensor (hereinafter abbreviated as a CMOS sensor) 12, a timing generator (TG) 12a, an analog front end (AFE) circuit 13, and a digital signal processing circuit. 14, a camera control circuit 15, a human I / F (interface) control circuit 16, a user I / F 17, and a camera shake sensor 18.

  The optical block 11 includes a lens for condensing light from the subject on the CMOS sensor 12, a drive mechanism for moving the lens to perform focusing and zooming, a shutter mechanism, an iris mechanism, and the like. The driver 11 a controls driving of each mechanism in the optical block 11 based on a control signal from the camera control circuit 15.

  In the CMOS sensor 12, a plurality of pixels including a photodiode (photogate), a transfer gate (shutter transistor), a switching transistor (address transistor), an amplification transistor, a reset transistor (reset gate), etc. are two-dimensionally formed on a CMOS substrate. And a vertical scanning circuit, a horizontal scanning circuit, an image signal output circuit, and the like are formed. The CMOS sensor 12 is driven based on the timing signal output from the TG 12a, and converts incident light from the subject into an electrical signal. The TG 12 a outputs a timing signal under the control of the camera control circuit 15.

  The CMOS sensor 12 includes a plurality of output channels for outputting pixel signals, and the readout frequency can be reduced by outputting pixel signals in parallel from these output channels. Furthermore, for example, when outputting a pixel signal for one line, by adding and simultaneously outputting the signals of adjacent pixels of the same filter component on the image sensor, the synchronization frequency when reading the pixel signal is set. A function of outputting an image signal at a screen rate faster than usual without increasing the image rate may be provided. In addition, as this image pick-up element, things other than CMOS sensors, such as CCD, may be used, for example.

  The AFE circuit 13 is configured, for example, as one IC (Integrated Circuit), and improves the S / N (Signal / Noise) ratio with respect to the image signal output from the CMOS sensor 12 by CDS (Correlated Double Sampling) processing. The sample is held so as to be maintained, the gain is controlled by AGC (Auto Gain Control) processing, A / D conversion is performed, and a digital image signal is output. Note that the circuit for performing the CDS process may be formed on the same substrate as the CMOS sensor 12.

  The AFE circuit 13 further includes a multiplexing (MUX) function for time-division multiplexing image signals input from the CMOS sensor 12 through a plurality of channels and reducing the number of channels for output. For example, an N channel image signal from the CMOS sensor 12 is received and output as an N / 2 channel signal.

  The digital signal processing circuit 14 is configured as one IC, for example, and performs various camera signal processing such as AF (Auto Focus), AE (Auto Exposure), white balance adjustment, and gamma correction on the image signal from the AFE circuit 13 and its control. Execute detection processing for, or part of those processing. In the present embodiment, in particular, a function for rearranging and inputting a plurality of channels of image signals from the AFE circuit 13 to the various processing blocks described above, and which output channel of the image sensor is output. The function of generating the channel ID shown is provided. In addition, as a camera signal processing function, a function of processing using a channel ID (for example, digital clamp, gain correction between channels, etc.) is provided.

  The camera control circuit 15 is a microcontroller composed of, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like, and executes a program stored in the ROM or the like. The respective units of the imaging apparatus are controlled in an integrated manner. For example, based on the detection data from the digital signal processing circuit 14 and the detection signal from the camera shake sensor 18, the current input image state is grasped, and shooting is performed according to various setting modes designated by the human I / F control circuit 16. Performs operation control and image quality correction control.

  The user I / F 17 includes a display function for notifying the user of a setting state and the like, keys and levers for user operation input, and the like. The human I / F control circuit 16 detects a setting state such as a shooting mode selected by the user and notifies the camera control circuit 15 of the set state. Also, camera control information (subject distance, F value, shutter speed, zoom magnification, etc.) obtained through the camera control circuit 15 is displayed on the user I / F 17 and notified to the user.

The camera shake sensor 18 detects camera shake at the time of photographing by using, for example, an acceleration sensor or an angular velocity sensor with respect to two axes, and supplies a detection signal to the camera control circuit 15.
In this image pickup apparatus, the signals received and photoelectrically converted by the CMOS sensor 12 are sequentially supplied to the AFE circuit 13, subjected to CDS processing and AGC processing, and then converted into digital signals. The digital signal processing circuit 14 performs image quality correction processing on the digital image signal supplied from the AFE circuit 13, and finally converts it into a luminance signal (Y) and a color difference signal (C) for output.

  The image data output from the digital signal processing circuit 14 is supplied to a graphic I / F (not shown) and converted into a display image signal, whereby a camera-through image is displayed on a display unit such as an LCD (Liquid Crystal Display). Is done. When the camera control circuit 15 is instructed to record an image by a user input operation to the user I / F 17 or the like, the image data from the digital signal processing circuit 14 is supplied to an encoder (not shown), and a predetermined compression code Is processed and recorded on a recording medium (not shown). At the time of recording a still image, image data for one frame is supplied from the digital signal processing circuit 14 to the encoder, and at the time of recording a moving image, the image data processed by the digital signal processing circuit 14 is encoded by the encoder. Continuously supplied.

Next, filter coding applied to the CMOS sensor 12 of this imaging apparatus will be described. FIG. 2 is a diagram illustrating an arrangement of color filters in the CMOS sensor 12.
The color filter array shown in FIG. 2 is configured such that a square lattice array is tilted by 45 °, and each of the R and B filters is surrounded by a G filter. With this configuration, the spatial frequency characteristics of the G component, which is more sensitive to humans than these components, are enhanced over the conventional Bayer arrangement, while the spatial frequency characteristics necessary and sufficient for human visibility characteristics are obtained for the R and B components. It becomes possible. The G component is a main component in generating the luminance component, and as a result, the luminance resolution for not only the achromatic subject but also the chromatic subject is increased, and the image quality is improved.

  In this color filter array, as shown by the dotted arrows in FIG. 2, a method of alternately reading pixels in two adjacent rows in one horizontal synchronization period is basically used. That is, when there is only one output channel, scanning is performed in this order and reading is performed. Then, reading is performed by various methods from an image sensor having such a color filter array using a maximum of eight output channels as described below.

FIG. 3 is a diagram showing a configuration for multi-channel readout from the CMOS sensor 12 and multiplexing of the signals.
As described above, the AFE circuit 13 multiplexes the N-channel pixel signals from the CMOS sensor 12 and outputs them as N / 2-channel signals. In the following description of the embodiment, as shown in FIG. 3, it is assumed that pixel signals are read out from the CMOS sensor 12 in parallel through a maximum of eight output channels and supplied to the AFE circuit 13, and the outputs thereof are assumed. The channels are Ch0 to Ch7. The AFE circuit 13 receives these eight channels of input signals, multiplexes them in time, and supplies them to the digital signal processing circuit 14 as multiplexed signals Sig1 to Sig4 of up to four channels.

  With such a configuration, the output frequency of the multiplexed signals Sig1 to Sig4 from the AFE circuit 13 is twice the read frequency of the CMOS sensor 12. In the CMOS sensor 12, even if the number of pixels becomes a large value such as several millions, the readout frequency can be kept within the limits of the analog signal processing by reading out more pixel signals in parallel. On the other hand, since the multiplexed signals Sig 1 to Sig 4 from the AFE circuit 13 to the digital signal processing circuit 14 are transferred as digital signals, the transfer frequency can be easily made higher than the read frequency of the CMOS sensor 12.

FIG. 4 is a diagram illustrating an example of assignment of output channels to pixel positions on the CMOS sensor 12.
In this embodiment, as shown in FIG. 4, the color filters on the CMOS sensor 12 are classified into six types of filter components R, B, Gr, Gb, Ggo, and Gge for convenience. Among these, Gr, Gb, Ggo, and Gge are actually color filters having the same spectral characteristics, but are processed as individual components in the digital signal processing circuit 14, and so on. These six types of pixels will be referred to as individual filter components. Further, the pixels of each filter component are shown with numerical values such as “R1” and “R2” in the horizontal direction in order from the side where reading is first performed on the image sensor (left side in the figure).

  As shown in FIG. 4, on the CMOS sensor 12, R and Gr in the first row, Ggo and Gge in the second row, Gb and B in the third row, Ggo and Gge in the fourth row, They are arranged alternately. As an example of a variation of readout from an image sensor having such filter coding, simultaneous readout of 8 channels, 6 channels, 4 channels, and 2 channels is assumed.

  At the time of simultaneous readout of 8 channels, odd-numbered rows of pixels are assigned to output channels Ch0, Ch1, Ch2, and Ch3 in order from the top, and even-numbered rows of pixels are assigned to output channels Ch4, Ch5, Ch6, and Ch7 in order from the top. At the time of simultaneous reading of 6 channels, odd-numbered rows of pixels are assigned to output channels Ch0, Ch1, and Ch2 in order from the top, and even-numbered rows of pixels are assigned to output channels Ch3, Ch4, and Ch5 in order from the top.

  At the time of simultaneous reading of four channels, odd-numbered rows of pixels are assigned to the output channels Ch0 and Ch1 in order from the top, and even-numbered rows of pixels are assigned to the output channels Ch2 and Ch3 in order from the top. When two channels are read simultaneously, odd-numbered rows of pixels are assigned to the output channel Ch0, and even-numbered rows of pixels are assigned to the output channel Ch1 and output.

FIG. 5 is a diagram illustrating an example of a variation of multiplexing in the AFE circuit 13.
In the present embodiment, as an example, two types of multiplexing methods shown in FIGS. 5A and 5B are assumed. In the MUX type A shown in FIG. 5A, the output channels Ch0 and Ch1 are multiplexed to generate a multiplexed signal Sig1, the output channels Ch2 and Ch3 are multiplexed to generate a multiplexed signal Sig2, and the output channels Ch4 and Ch4 Multiplex Ch5 is multiplexed to generate multiplexed signal Sig3, and output channels Ch6 and Ch7 are multiplexed to generate multiplexed signal Sig4. That is, the signals of pixels adjacent in the horizontal direction on the image sensor are multiplexed.

  5B, the output channels Ch0 and Ch4 are multiplexed to generate a multiplexed signal Sig1, the output channels Ch1 and Ch5 are multiplexed to generate a multiplexed signal Sig2, and the output channel Ch2 and Ch6 are multiplexed to generate a multiplexed signal Sig3, and output channels Ch3 and Ch7 are multiplexed to generate a multiplexed signal Sig4. That is, the signals of pixels adjacent in the vertical direction on the image sensor are multiplexed.

FIG. 6 is a diagram illustrating an example of multiplexing during 8-channel reading. Here, as an example, a color sequence in an odd-numbered horizontal synchronization period (hereinafter referred to as an odd-numbered H period) is shown.
When pixel signals input to the AFE circuit 13 with 8 channels are multiplexed with MUX type A, for example, R1 and Gr1 of the output channels Ch0 and Ch1 read simultaneously from the CMOS sensor 12 with 1 clock are read 2 In synchronization with the double transfer clock, the multiplexed signal Sig1 is time-division multiplexed in order. On the other hand, in the case of MUX type B, R1 and Ggo1 of the output channels Ch0 and Ch4 that are simultaneously read from the CMOS sensor 12 in one clock are time-division multiplexed in order on the multiplexed signal Sig1.

  The color sequence of the image signal input to the digital signal processing circuit 14 due to the above-described variation in the allocation of the output channel from the image sensor and the variation in the multiplexing in the AFE circuit 13 is as described below. Various variations are possible. Furthermore, if the filter coding on the image sensor or the number of pixels is different, a larger number of color sequences can be considered.

  The digital signal processing circuit 14 at the subsequent stage needs to execute internal processing in consideration of the color sequence of the input image signal. The digital signal processing circuit 14 is applied to a number of color sequences as described above. Inefficiently designed and manufactured individually. Therefore, in the digital signal processing circuit 14 according to the present embodiment, a function for rearranging image signals input according to various color sequences is provided in the input stage, and for this rearrangement function, from the camera control circuit 15 for each color sequence. By simply setting parameters, it is possible to support various color sequences without changing the internal circuit configuration. Further, a function for generating a channel ID indicating from which output channel of the image sensor is provided is provided so that signal processing considering the output channel can be easily executed even when rearrangement is performed.

FIG. 7 is a block diagram showing the internal configuration of the digital signal processing circuit 14.
As shown in FIG. 7, the digital signal processing circuit 14 includes a rearrangement processing unit 21, a channel ID generation unit 22, a camera signal processing unit 23, a communication I / F 24, and a signal generator (SG) 25.

  The rearrangement processing unit 21 converts the N / 2 channel (four channels in this case) image signal input from the preceding circuit in various color sequences as described above into an S channel signal from which each filter component is separated. To the camera signal processing unit 23. Here, as an example, four-channel signals of R / Gb channel, Gr / B channel, Ggo channel, and Gge channel are output.

  The channel ID generation unit 22 generates a channel ID corresponding to the image signal after rearrangement by the rearrangement processing unit 21 based on the control parameter from the camera control circuit 15, and matches the timing with the corresponding four-channel image signal. To the camera signal processing unit 23.

  The camera signal processing unit 23 is a block that executes conventional general camera signal processing based on the image signal input via the channel ID generation unit 22. As camera signal processing, for example, digital clamp, noise reduction, defective pixel correction, gain correction between channels, demosaic, white balance adjustment, resolution conversion, and the like are applied. In these processes, the input image signal is processed in parallel for each S (= 4) channel in accordance with the control parameter from the camera control circuit 15, and finally output as Y and C signals to the subsequent baseband processing system. Is done. A block to be processed in consideration of the channel ID, such as a function of digital clamping, gain variation correction between channels, and defective pixel correction, performs processing based on the channel ID from the channel ID generation unit 22.

  The communication I / F 24 is an I / F circuit that controls data input / output between each block in the digital signal processing circuit 14 and the camera control circuit 15. Detection data from the camera signal processing unit 23 is supplied to the camera control circuit 15 via the communication I / F 24, and control parameters for camera signal processing are transmitted from the camera control circuit 15 via the communication I / F 24. 23. Further, the rearrangement processing unit 21 is supplied with rearrangement control control parameters from the camera control circuit 15 via the communication I / F 24.

  SG25 is a block that generates various timing signals necessary for execution of processing in the rearrangement processing unit 21 and the camera signal processing unit 23. For example, the SG25 indicates a horizontal synchronization signal HD of an image signal and an effective period in the horizontal direction. An enable signal H_EN or the like is output.

Next, the rearrangement processing unit 21 will be described in detail first.
FIG. 8 is a diagram illustrating an example of a color sequence before and after rearrangement by the rearrangement processing unit 21. Here, as an example, a color sequence in the case of multiplexing by MUX type A at the time of 8-channel reading is shown.

  In the odd H period, R and Gr filter components appear alternately in the multiplexed signal Sig1 from the AFE circuit 13. Similarly, different filter components appear alternately in the multiplexed signals Sig2 to Sig4. The rearrangement processing unit 21 rearranges such image signals, and distributes and outputs channels for each of R, Gr, Ggo, and Gge filter components as shown on the right side of the drawing. In the case of the even H period, the output channels are allocated to the filter components Gb, B, Ggo, and Gge by rearranging the multiplexed signals Sig1 to Sig4. As a result, the camera signal processing unit 23 can process a combination of spatially adjacent pixels (for example, R1, Ggo1, Gr1, Gge1) in parallel in one clock in the color filter array shown at the lower right in the figure. It becomes.

  Here, in the output signal from the AFE circuit 13, filter components repeatedly appear in a predetermined pattern according to reading. In the present embodiment, the same repeating pattern appears every H period. For example, in the odd-numbered H period, a combination of R, R, Ggo, Ggo, Gr, Gr, Gge, and Gge becomes one repeating pattern. The number of filter components (in this case, four types of R, Ggo, Gr, and Gge, or four types of Gb, Ggo, B, and Gge) included in one of such repeated patterns, or an integer multiple thereof, By matching the number of output channels of the rearrangement processing unit 21, the output channels for the digital signal processing circuit 14 can be distributed for each filter component at least every H period. As a result, the digital signal processing circuit 14 can process in parallel the signals of different filter components present at spatially close positions, and the processing procedure and circuit configuration in the digital signal processing circuit 14 can be simplified.

FIG. 9 is a block diagram showing an internal configuration of the rearrangement processing unit 21.
As shown in FIG. 9, the rearrangement processing unit 21 includes an enable signal generation unit (hereinafter referred to as an EN generation unit) 30, a rearrangement memory 40, a write control unit 50 for this memory, and a read control unit 60. .

  The EN generation unit 30 generates an enable signal indicating the timing at which a signal corresponding to each component of R, Gb, Gr or B, Ggo, Gge appears for each of the multiplexed signals Sig1 to Sig4 from the AFE circuit 13. And output to the write control unit 50. The appearance timing of R or Gb for the multiplexed signals Sig1 to Sig4 is indicated by enable signals R_Gb_EN_Sig1 to R_Gb_EN_Sig4, respectively. Similarly, the appearance timing of Gr or B is indicated by enable signals Gr_B_EN_Sig1 to Gr_B_EN_Sig4, the appearance timing of Ggo is indicated by enable signals Ggo_EN_Sig1 to Ggo_EN_Sig4, and the appearance timing of Gge is indicated by enable signals Gge_EN_Sig1 to Gge_EN_Sig4.

  The EN generation unit 30 sets these enable signals in accordance with control parameters (EN generation parameters) designated by the camera control circuit 15 in the horizontal effective period in which the enable signal H_EN from the SG 25 is at the H level. At the same time, the multiplexed signals Sig 1 to Sig 4 from the AFE circuit 13 are delayed in accordance with the generation of the enable signal and output to the write control unit 50.

  The rearrangement memory 40 is composed of, for example, a dual port SRAM (Static RAM) capable of executing writing / reading in parallel. The storage area of the rearranging memory 40 is divided into areas for each filter component of G / Gb, Gr / B, Ggo, and Gge.

  The write control unit 50 rearranges the multiplexed signals Sig1 to Sig4 input via the EN generation unit 30 based on the enable signal from the EN generation unit 30, and rearranges the signals Sig_R_Gb and Sig_Gr_B for each filter component. , Sig_Ggo, Sig_Gge are generated. Also, write enable signals WEN_R_Gb, WEN_Gr_B, WEN_Ggo, WEN_Gge and write addresses WADRS_R_Gb, WADRS_Gr_B, WADRS_Ggo, WADRS_Gge for each output channel are generated and output to the rearrangement memory 40, and signals Sig_R_g_Gb_gb Write to the corresponding filter component storage area.

  The read control unit 60 outputs the read enable signals REN_R_Gb, REN_Gr_B, REN_Ggo, REN_Gge and the read addresses RADRS_R_Gb, RADRS_Gr_B, RADRS_Ggo, RADRS_Gge to the rearrangement memory 40, and separates the signals Rb_Gb_Gb_Gse , Sig_Ggo, Sig_Gge are read out to the camera signal processing unit 23.

  The rearrangement processing unit 21 sorts the input image signals of a plurality of channels for each filter component, stores them in the rearrangement memory 40, and controls the read / write addresses and timings of the memory, thereby allowing the subsequent stage. It is possible to generate an image signal of a constant sequence that is required by the camera signal processing unit 23.

FIG. 10 is a block diagram illustrating an internal configuration of the EN generation unit 30.
As shown in FIG. 10, the EN generation unit 30 includes a counter 31, a flag generation decoder 32, selector groups 33a to 33d for each filter component, and a delay adjustment unit 34.

  The counter 31 generates a pixel ID and supplies it to the selector groups 33a to 33d in the horizontal effective period in which the enable signal H_EN is at the H level. The pixel ID is an identification number assigned for each input timing of the multiplexed signals Sig1 to Sig4. In this embodiment, four values “0” to “3” are repeated. That is, the counter 31 starts counting up from “0” at the start of the horizontal effective period, and continues from “0” to “0” in synchronization with the pixel clocks of the multiplexed signals Sig1 to Sig4 until the end of the horizontal effective period. Repeat count up to “3”.

  Here, the number of repetitions of the pixel ID is determined from the number of repetitions of the filter component in each of the multiplexed signals Sig1 to Sig4. In the present embodiment, as shown in FIG. 11 and FIG. 12 later, since the number of repetitions of the filter component is “2” or “4”, the number of repetitions of the pixel ID is “4”. In order to be able to deal with color sequences having different numbers of repetitions of filter components, a common multiple of those repetitions may be used as the number of repetitions of the pixel ID. Alternatively, the number of repetitions in the counter 31 may be varied for each color sequence by setting from the camera control circuit 15.

  Based on the EN generation parameter from the camera control circuit 15, the flag generation decoder 32 outputs ON / OFF flags for each channel of the multiplexed signals Sig1 to Sig4 and for each pixel ID to the selector groups 33a to 33d. The ON / OFF flag is set to the H level when the corresponding filter component appears.

  Here, the EN generation parameter is information indicating filter components in the multiplexed signals Sig1 to Sig4. As described above, since the number of repetitions of the filter component in the multiplexed signals Sig1 to Sig4 corresponds to the number of repetitions of the pixel ID, information indicating the filter component of each signal is the number of pixel IDs in the EN generation parameter. included.

  The flag generation decoder 32 decodes the EN generation parameter, generates an ON / OFF flag indicating the presence / absence of one filter component for each pixel ID, and sets the R / Gb, Gr / B, Ggo, and Gge filter components. Are supplied to selector groups 33a to 33d, respectively. If the ON / OFF flag to each selector group 33a to 33d can be set according to the color sequence of the multiplexed signals Sig1 to Sig4 by the EN generation parameter from the camera control circuit 15, the configuration of the flag generation decoder 32 May be anything. For example, the ON / OFF flag may be directly controlled from the camera control circuit 15.

  The selector groups 33a to 33d are each provided with four selectors SEL0 to SEL3. Each selector SEL0 to SEL3 individually receives ON / OFF flags corresponding to the same filter component and the same signal channel (any one of multiplexed signals Sig1 to Sig4) for the pixel ID, and corresponds to the pixel ID from the counter 31. Select the input signal to output.

  Thus, for example, the enable signal R_Gb_EN_Sig1 output from the selector SEL0 of the selector group 33a indicates whether or not an R / Gb component appears at each pixel ID timing in the multiplexed signal Sig1. Similarly, the enable signals R_Gb_EN_Sig2 to R_Gb_EN_Sig4 from the selectors SEL1 to SEL3 indicate the presence / absence of R / Gb components at each pixel ID timing in the multiplexed signals Sig2 to Sig4. Furthermore, enable signals indicating the presence or absence of similar Gr / B, Ggo, and Gge components are output from the selector groups 33b to 33d, respectively.

  The delay adjustment unit 34 receives the multiplexed signals Sig1 to Sig1 from the AFE circuit 13 so that the enable signals output from the selector groups 33a to 33d coincide with the output timings of the corresponding pixels in the multiplexed signals Sig1 to Sig4. Sig4 is delayed.

  FIG. 11 is a diagram showing a color sequence and an enable signal when the number of readout channels of the image sensor is “8” and MUX type A as an example. FIG. 12 is a diagram illustrating a color sequence and an enable signal when the number of readout channels of the image sensor is “6” and the MUX type A is used. In these figures, for the sake of simplicity, the number of pixels in the horizontal effective period is the same, but the number of pixels is not actually limited.

  In FIG. 11, for example, in the multiplexed signal Sig1, R (odd H period) or Gb (even H period) appears when the pixel ID is “0” and “2”, and the pixel IDs are “1” and “3”. "Gr (odd H period) or B (even H period) appears. Therefore, the enable signal R_Gb_EN_Sig1 becomes H level when the pixel ID is “0” and “2”, and the enable signal Gr_B_EN_Sig1 becomes H level when the pixel ID is “1” and “3”. In addition, since the components of Ggo and Gge do not appear in the multiplexed signal Sig1, the enable signals Ggo_EN_Sig1 and Gge_EN_Sig1 are always at the L level.

  Also, as shown in FIG. 12, in the case of 6-channel reading, the multiplexed image signal has 3 channels, and no image signal is transmitted to the multiplexed signal Sig4. In the case of FIG. 12, in the multiplexed signal Sig1, two components R and Gr appear every two pixels in the odd H period, and two components Gb and B appear every two pixels in the even H period. . Further, since four components of R, Ggo, Gr, and Gge (in the case of an odd-numbered H period) repeatedly appear in the multiplexed signal Sig2, the enable signals R_Gb_EN_Sig2, Gr_B_EN_Sig2, Ggo_EN_Sig2, and Gge_EN_Sig2 corresponding to the multiplexed signal Sig2 are All are H level once every four pixels.

  As described above, there are various color sequences of the multiplexed signals Sig1 to Sig4 depending on the combination of the number of readout channels of the image sensor and the multiplexing method. With the above-described circuit configuration, the EN generation unit 30 can set EN generation parameters corresponding to such various color sequences and freely generate a pulse signal (enable signal) indicating the appearance timing of each filter component. It is possible to output.

FIG. 13 is a block diagram showing an internal configuration of the write control unit 50.
As shown in FIG. 13, the write control unit 50 includes delay adjustment units 51a to 51d corresponding to the multiplexed signals Sig1 to Sig4, a signal distribution unit 52 that distributes the multiplexed signals Sig1 to Sig4 for each filter component, A delay adjustment unit 53 that adjusts the delay amount of the input signal to the distribution unit 52 and a decoder 54 that generates a write address for the rearrangement memory 40 are provided.

  The multiplexed signal Sig1 and the enable signals R_Gb_EN_Sig1, Gr_B_EN_Sig1, Ggo_EN_Sig1, Gge_EN_Sig1 corresponding to this signal are input from the EN generator 30 to the delay adjustment unit 51a. Similarly, multiplexed signals Sig2 to Sig4 and corresponding enable signals are input from the EN generation unit 30 to the delay adjustment units 51b to 51d, respectively.

  The delay adjustment units 51 a to 51 d delay the input signals by a uniform time in accordance with the rearrangement parameter that is a control parameter from the camera control circuit 15. These components are configured such that, for example, a plurality of flip-flop (FF) circuits are connected in series, and the output of the FF circuit at each stage can be selected according to the rearrangement parameter. When the same filter component appears in the multiplexed signals Sig1 to Sig4 at the same timing, these delay adjustment units 51a to 51d delay one of the signals and shift the delayed signal to the rearrangement memory 40. The signal can be written continuously.

  The signal distribution unit 52 performs the delay adjustment in each of the delay adjustment units 51 a to 51 d and further uses the multiplexed signals Sig1_adj to Sig4_adj input via the delay adjustment unit 53 according to the selection signal from the decoder 54 as a filter component. The signals Sig_R_Gb, Sig_Gr_B, Sig_Ggo, and Sig_Gge distributed for each are output to the rearrangement memory 40. The delay adjustment unit 53 delays the multiplexed signals Sig1_adj to Sig4_adj by a predetermined amount to correctly synchronize the signal distribution unit 52 and the decoder 54.

  The decoder 54 selects the selection signal for the signal distribution unit 52 based on the delay adjusted enable signal output from the delay adjustment units 51 a to 51 d, the address and enable for writing the selected signal to the rearrangement memory 40. Generate a signal.

FIG. 14 is a block diagram illustrating an internal configuration of the signal distribution unit 52.
As shown in FIG. 14, the signal distribution unit 52 includes selectors 521 to 524 corresponding to R / Gb, Gr / B, Ggo, and Gge filter components. Multiplexed signals Sig1_adj to Sig4_adj are input to each of the selectors 521 to 524, and any one signal is selected and output in accordance with selection signals SEL_R_Gb, SEL_Gr_B, SEL_Ggo, and SEL_Gge from the decoder 54.

  Here, four types of numerical values “1”, “2”, “4”, and “8” described in the input stages of the selectors 521 to 524 in FIG. 14 are numerical values of bit strings based on the enable signal input to the decoder 54 ( Decimal notation). As will be described later, the decoder 54 selects an input channel corresponding to these numerical values based on the input enable signal, so that only the signal of the filter component corresponding to each is output from the selectors 521 to 524. The

  Note that the multiplexed signals Sig1_adj to Sig4_adj that are input are adjusted so that the same filter components are not input at the same timing by the delay adjustment units 51a to 51d, so that the signal distribution unit 52 simply selects the input signal. Thus, the signals of all the filter components can be distributed to the output channels without being missed.

FIG. 15 is a block diagram showing the internal configuration of the decoder 54.
As shown in FIG. 15, the decoder 54 includes selection signal decoders 541a to 541d, OR gates 542a to 542d, inverters 543a to 543d, and counters 544a to 544d corresponding to R / Gb, Gr / B, Ggo, and Gge filter components. 544d.

  The selection signal decoders 541a to 541d consider the enable signals corresponding to the respective filter components from the delay adjustment units 51a to 51d as bit strings, and the signal distribution unit 52 selects the input channel corresponding to the numerical value indicated by the bit string. Selection signals SEL_R_Gb, SEL_Gr_B, SEL_Ggo, and SEL_Gge are output. The selection signal decoding process will be described in detail with reference to FIGS. 16 and 17 later.

  The OR gates 542a to 542d receive the enable signals from the delay adjustment units 51a to 51d corresponding to the filter component signals of R / Gb, Gr / B, Ggo, and Gge, respectively. The output pulse of the OR gate 542a is input to the inverter 543a and the counter 544a, and the output of the inverter 543a is written when the write enable signal XWEN_R_Gb for the R / Gb area of the rearranging memory 40 (in this case, at the L level). It is possible). The counter 544a counts the output pulses of the OR gate 542a, and the count value becomes a write address for the R / Gb area of the rearrangement memory 40. The same applies to the circuits corresponding to the other filter components, and the output pulses of the OR gates 542b to 542d are input to the inverters 543b to 543d and the counters 544b to 544d, respectively, whereby Gr / B and Ggo of the rearrangement memory 40 are used. Write enable signals and addresses for the Gge areas are generated.

  Here, specific examples of the enable signal after delay adjustment are shown in FIGS. 16 and 17 below, and operations of the selection signal decoders 541a to 541d in each case will be described. FIG. 16 is a diagram for explaining the selection signal decoding operation in the case where the number of readout channels of the image sensor is “8” and the MUX type A is used.

  In this case, as shown in FIG. 11, signals having the same filter component appear between the multiplexed signals Sig1 and Sig2 and between the multiplexed signals Sig3 and Sig4. Therefore, in the example of FIG. 16, the multiplexed signals Sig2 and Sig4 are delayed by one clock by the delay adjusting units 51b and 51d.

  Here, a bit string in which the enable signal after delay adjustment is assigned to each filter component from the lower order to the higher order in the direction of the multiplexed signals Sig1 to Sig4 will be considered. For example, at the timing when R, Gr, Ggo, and Gge (or Gb, B, Ggo, and Gge) appear in the multiplexed signals Sig1_adj to Sig4_adj in FIG. 16, “0001” and Gr / B components for the R / Gb component, respectively. Is “0000”, the Ggo component is “0100”, and the Gge component is “0000”. The lower part of FIG. 16 shows the numerical values of these bit strings in decimal notation.

  Since the multiplexed signals Sig1_adj to Sig4_adj do not include the same filter component at the same timing, the values that can be taken by the above bit string are five kinds of decimal notation “0” “1” “2” “4” “8” is there. In addition, since each input channel of the multiplexed signals Sig1_adj to Sig4_adj is assigned in order from the lower order in the bit string, it can be seen that a filter component corresponding to the channel in which the bit is set appears.

  Therefore, the selection signal decoders 541a to 541d select the multiplexed signal Sig1_adj when the numerical value of the bit string based on the input enable signal is “1”, select Sig2_adj when “2”, and select “4”. Sig3_adj is selected, and when “8”, a selection signal instructing to select Sig4_adj is generated and output to the signal distribution unit 52. Further, when the numerical value of the bit string is “0”, no input channel is selected.

  At the timing when R, Gr, Ggo, Gge (or Gb, B, Ggo, Gge) appear in the above example, the bit string for each component of R / Gb, Gr / B, Ggo, Gge is “1” “2”. Since “4” and “8”, the multiplexed signal Sig1_adj is selected for the selector 521 of the signal distribution unit 52, Sig2_adj is selected for the selector 522, Sig3_adj is selected for the selector 523, and Sig4_adj is selected for the selector 524. To control.

  As another example, the case where the readout channel of the image sensor is “6” will be described. FIG. 17 is a diagram for explaining the selection signal decoding operation in the case where the number of readout channels of the image sensor is “6” and the MUX type A is used.

  In this case, in order to prevent the same filter component from appearing at the same time, the delay adjustment unit 51b sets a delay amount for one clock and the delay adjustment unit 51c sets a delay amount for two clocks. In the multiplexed signals Sig1_adj to Sig3_adj after delay adjustment, for example, at the timing when R, Gr, and Ggo components respectively appear, the bit string based on the enable signal corresponding to each filter component is “1000” “0100” “0010” “ 0000 ". Therefore, according to the selection signals from the selection signal decoders 541a to 541d, the selector 521 of the signal distribution unit 52 selects the multiplexed signal Sig1_adj, the selector 522 selects Sig3_adj, the selector 523 selects Sig2_adj, and the selector 524 selects any input signal. do not do.

  As described above, the selection signal decoders 541a to 541d cause the signal distribution unit 52 to output the image signal from the individual channels for each filter component by the simple decoding process based on the input enable signal as described above. Thus, the data can be recorded in a storage area for each filter in the rearrangement memory 40.

  Here, in FIGS. 18 to 24 below, the arrangement of the image signals before and after the rearrangement by the above processing is specifically shown for each color sequence. FIGS. 18 and 19 are diagrams showing image signal arrangements when the number of readout channels of the image sensor is “8” and MUX types A and B, respectively. 20 and 21 are diagrams showing image signal arrangements in the case where the number of readout channels of the image sensor is “6” and MUX types A and B, respectively. FIGS. 22 and 23 are diagrams showing image signal arrangements in the case where the number of readout channels of the image sensor is “4” and MUX types A and B, respectively. FIG. 24 is a diagram showing an image signal arrangement when the number of read channels of the image sensor is “2” and MUX types A and B are used.

  As shown in FIGS. 18 to 24, the write controller 50 having the above configuration sorts the input image signals by filter components regardless of the readout channel of the image sensor and the multiplexing method, and stores them in the rearrangement memory 40. You can write to the corresponding area. For example, the number N of output channels of the image sensor and the number of filter components of pixel signals processed in parallel in the digital signal processing circuit 14 (in this embodiment, four of R, Gr, Ggo, Gge, or Gb, B , Ggo, and Gge), for example, in the case of 6-channel reading as shown in FIGS. 20 and 21, for example, three or more types of filter components in one channel after multiplexing May be transmitted, but in such a case as well, it can be distributed for each filter component.

  By the way, the signal output from the write control unit 50 is output from individual channels for each filter component as described above. However, as it is, the signal of each filter component is subjected to parallel processing in the digital signal processing circuit 14. It is not necessarily output at the appropriate timing. For example, in FIG. 19, there are timings at which R2, Gr2, Ggo1, and Gge1 are simultaneously output in the rearranged signals in the odd-numbered H period. As can be seen from FIG. Since pixels having different phases are included, they are not pixels to be processed simultaneously by the digital signal processing circuit 14. Further, when the number of channels of the multiplexed signal is smaller than the types of filter components (in the case of FIGS. 20 to 24), naturally, the signals of all the filter components necessary for the digital signal processing circuit 14 are output simultaneously. There is no.

  In order to absorb and align such a shift in output timing, the rearranged signals are output via the rearrangement memory 40. The image signals of the respective filter components output from the write control unit 50 are sequentially stored in the rearrangement memory 40 without any gaps regardless of the output timing, and the rearrangement generated by the write control unit 50 is performed. The relationship between the write address of the memory 40 and the spatial phase is the same. For this reason, once the data is stored in the rearrangement memory 40, the output timings of all the channels can be made uniform by reading them in ascending order of addresses.

FIG. 25 is a block diagram showing an internal configuration of the read control unit 60.
The read control unit 60 shown in FIG. 25 includes a counter 61 that outputs a read address, and a delay adjustment unit 62 that matches the output timing of the read enable signal. The counter 61 counts the pixel clocks in ascending order in the horizontal effective period when the enable signal H_EN is at the H level, and resets the count value at the end of the horizontal effective period. The count value is output to the rearrangement memory 40 as read addresses RADRS_R_Gb, RADRS_Gr_B, RADRS_Ggo, and RADRS_Gge for the storage area for each filter component. Further, the delay amount of the delay adjustment unit 62 is set according to the operation delay of the counter 61, and the enable signal H_EN adjusted for delay is used as enable signals REN_R_Gb, REN_Gr_B, REN_Ggo, REN_Gge indicating read permission in each storage area. The data is output to the sorting memory 40.

  By using the read control unit 60 having a simple configuration as described above, the image signals rearranged for each filter component can be output at the correct timing. That is, the signal of each filter component is output so that the spatial position on the image sensor is a correct combination. Therefore, the digital signal processing circuit 14 that has received such an image signal can always process the input image signal in the same procedure regardless of the color sequence of the image signal output from the AFE circuit 13.

  In the color sequence shown in the above embodiment, the pixel signal on the image sensor is output first from the spatially left pixel. However, in practice, such a condition may not always be satisfied. For example, by adding the signals of adjacent pixels of the same filter component on the image sensor and outputting them simultaneously, the number of output pixels is reduced, and the image signal is output at a higher screen rate without increasing the readout frequency. For example, the image signal output to the rearrangement memory 40 may not be in the horizontal order as R2, R1, R4, R3,. In such a case, the signal of the pixel on the left side in the space is not necessarily in the smaller address on the rearrangement memory 40. If the signals are read in the order of addresses, the digital signal processing circuit 14 performs the correct processing. Will not be able to.

  In such a case, for example, the counter 61 of the read control unit 60 is individually provided for each of the read addresses RADRS_R_Gb, RADRS_Gr_B, RADRS_Ggo, and RADRS_Gge. Each counter can be individually controlled by setting from the camera control circuit 15 for each counter. Even in the case of a color sequence that is not output in the order in the horizontal direction, since the spatial position of the output signal is usually regular, each counter is caused to perform a counting operation according to such regularity. For example, when the B component is written in the rearrangement memory 40 as B2, B1, B4, B3..., The counter 63b corresponding to the B component is set to 1, from the start of the horizontal effective period. A count such as 0, 3, 2... Is performed. A counter that performs a regular counting operation can be easily implemented, and by allowing multiple types of counting operations to be switched for each counter, the number of color sequence variations that can be handled is further increased. be able to.

  As described above, by providing the rearrangement processing unit 21 at the input stage of the image signal to the camera signal processing unit 23, it is possible to always input the image signal to the camera signal processing unit 23 according to the same rule. For this reason, it becomes possible to deal with various color sequences without complicating the processing procedure and circuit configuration in the camera signal processing unit 23 and increasing the circuit scale. In particular, the number of output channels of the rearrangement processing unit 21 is set to the number of types of filter components processed in parallel by the camera signal processing unit 23, or an integer multiple thereof, so that the configuration of the camera signal processing unit 23, particularly noise reduction, is reduced. The configuration of a processing block that operates in consideration of the spatial position on the image sensor, such as correction of defective pixels and correction, can be simplified.

  Further, the rearrangement processing unit 21 can cope with various color sequences without changing the circuit configuration only by setting control parameters according to the color sequence of the input image signal. Therefore, the circuit scale can be greatly reduced as compared with the case where the digital signal processing circuit 14 is provided with processing circuits corresponding to each of these various color sequences.

  Therefore, highly versatile digital with a high degree of freedom in combination with the number of readout channels from the image sensor, the method of multiplexing signals read out with multiple channels, the method of thinning out the number of pixels, the number of pixels, filter coding, etc. The signal processing circuit 14 can be realized, and development / manufacturing costs at the time of future specification change or product variation expansion can be suppressed.

Next, the channel ID generation unit 22 will be described in detail.
FIG. 26 is a diagram illustrating an example of a color sequence before and after the rearrangement process. First, problems in processing the image signals after rearrangement by the rearrangement processing unit 21 will be described with reference to FIG.

  In FIG. 26, as an example, the color sequence in the case of the MUX type A when the number of read channels from the image sensor is “8” is shown. As can be seen from this figure, in the signals Sig_R_Gb, Sig_Gr_B, Sig_Ggo, and Sig_Gge output from the rearrangement processing unit 21, the same filter component appears in each channel, and the color sequence is simplified.

  On the other hand, when attention is paid to the origin channel indicating which of the output channels Ch0 to Ch7 of each image sensor is output, the origin channel before and after the rearrangement as shown in the lower part of the figure. The sequence of will also change. Actually, in the signal before rearrangement, two source channel numbers appear alternately for each signal regardless of the number of read channels and the MUX type. There will be various origin channel sequences.

  In the camera signal processing unit 23, there are processing blocks that require processing to be switched depending on the origin channel, that is, a value to be used for processing, such as digital clamp and gain correction between channels. In such a processing block, it is necessary to always perform processing while paying attention to the origin channel regardless of the color sequence.

  In the case of a system without the rearrangement processing unit 21, the signal flows to the subsequent stage with the original channel sequence on the left side in the figure. Therefore, in the block processed for each original channel, in the case of a system without multiplexing, for each internal channel In the case of a system with multiplexing, the values used for processing such as clamp values and gain values should be switched for each internal channel and for each pixel clock. However, the original channel sequence after rearrangement changes depending on the number of output channels, the multiplexing method, the number of pixels, etc., so if all these sequences can be supported, the processing becomes complicated and the circuit scale Will also increase. Conversely, if the number of sequences that can be handled is reduced, the versatility is lowered.

  Therefore, in the present embodiment, a block for generating a channel ID corresponding to the channel from which the image pickup device originates is provided, and an appropriate channel ID sequence corresponding to the change in the color sequence according to the control parameter from the camera control circuit 15 is provided. Can be generated. Then, by operating the camera signal processing unit 23 with reference to the generated channel ID, it is possible to make various color sequences by taking advantage of the rearrangement processing unit 21 without increasing the circuit scale of the camera signal processing unit 23. Make it available.

FIG. 27 is a diagram showing a channel ID definition method in the present embodiment.
In the present embodiment, as an example, channel IDs are defined based on 4-channel signals multiplexed by the AFE circuit 13 (multiplexed signals Sig1 to Sig4). In each of these four-channel image signals, two origin channel numbers appear alternately as described above. Therefore, as shown in FIG. 27, as the channel ID, the input signal at the odd-numbered clock of the first channel after multiplexing is “0”, the input signal at the even-numbered clock is “1”, and 2 channels The input signals at the odd and even clocks of the eye are “2” and “3”, respectively, and the input signals at the odd and even clocks of the third channel are “4” and “5”, respectively. The input signals at the odd-numbered and even-numbered clocks of the eyes are respectively “6” and “7”.

  Note that the channel ID may be defined based on the output channel from the image sensor. However, in that case, the sequence of channel IDs at the time of input to the digital signal processing circuit 14 changes according to the multiplexing method, so that subsequent circuit configuration and control parameter settings are changed to that sequence. It will be necessary to change it accordingly. On the other hand, according to the definition of FIG. 27, it becomes possible to cope with the same circuit configuration and control parameters regardless of the multiplexing method.

  FIG. 28 is a diagram showing a channel ID sequence when the number of channels read from the image sensor is “8” and “6”. FIG. 29 is a diagram showing a channel ID sequence when the number of read channels is “4” and “2”.

  28 and 29 show the origin channel corresponding to each pixel of the signal for each filter component from the rearrangement processing unit 21, that is, the channel ID. As shown in FIG. 28, when the number of read channels from the image sensor is “8”, the channel ID repeatedly appears in units of two pixels in each channel from the rearrangement processing unit 21, and the number of read channels is “6”. In this case, it appears in units of three pixels. As shown in FIG. 29, when the number of read channels is “4” and “2”, the channel IDs of the respective channels are all the same (that is, the repetition unit is “1”).

  The channel ID generation unit 22 plays a role of reproducing such a channel ID sequence on the basis of the channel ID generation parameter from the camera control circuit 15 and transmitting it to the camera signal processing unit 23 at the subsequent stage. Therefore, as shown in FIGS. 28 and 29, the channel ID generation unit 22 assigns a pixel ID for each input timing of the pixel signal in the repetition unit in each sequence, and selectively generates a channel ID for each pixel ID. To do. The pixel ID used in the channel ID generation unit 22 is irrelevant to the pixel ID generated in the EN generation unit 30 described with reference to FIG.

FIG. 30 is a block diagram illustrating an internal configuration of the channel ID generation unit 22.
As shown in FIG. 30, the channel ID generation unit 22 includes counters 221a to 221d, a selector 222, a channel ID generation decoder 223, selectors 224a to 224d, and a delay adjustment unit 225.

  In the counters 221a to 221d, the number of repetitions of the count value is “1”, “2”, “3”, and “4”, respectively, and when the horizontal synchronization signal HD is received from the SG 25, the count value is reset and counted up. By starting, the counts up to the number of repetitions are repeated every horizontal synchronization period. Note that the count period is not the horizontal effective period but the horizontal synchronization period, which is a block that uses the channel ID in the camera signal processing unit 23 and requires an output signal from the OPB (Optical Black) area on the image sensor. This is because the pixels in the OPB region also need to have a channel ID.

  The selector 222 selects the count value of one of the counters 221a to 221d in accordance with the counter switching parameter from the camera control circuit 15, and outputs it to the selectors 224a to 224d as pixel IDs.

  Here, the counters 221a to 221d having different repetition numbers are provided according to the repetition unit of the corresponding channel ID, and a count value corresponding to the number of read channels of the image sensor is selectively selected as the pixel ID by the counter switching parameter. Is output. For example, when the number of read channels of the image sensor is “8”, the count value of the counter 221d having the repetition number “4” is selected as the pixel ID.

  The channel ID generation decoder 223 supplies a channel ID corresponding to the R / Gb filter component for each pixel ID to the selector 224a based on the channel ID generation parameter from the camera control circuit 15, and similarly, Gr / B , Ggo, and Gge, channel IDs for the respective pixel IDs corresponding to the filter component channels are supplied to the selectors 224b to 224d, respectively. The selectors 224a to 224d individually receive channel IDs corresponding to R / Gb, Gr / B, Ggo, and Gge filter components for each pixel ID from the channel ID generation decoder 223, and correspond to the pixel ID from the selector 222. And the channel ID (CHID_R_Gb, CHID_Gr_B, CHID_Ggo, CHID_Gge) corresponding to each component channel is output.

  Here, the channel ID generation parameter is information for specifying a channel ID corresponding to the signal of each channel from the rearrangement processing unit 21. As described above, since the channel ID repetition unit in each channel corresponds to the pixel ID repetition number, the channel ID generation parameter includes information indicating the channel ID of each signal by the number of pixel IDs. The channel ID generation decoder 223 decodes the channel ID generation parameter, generates a channel ID to be output for each pixel ID, and supplies the channel ID to the selectors 224a to 224d for each of R / Gb, Gr / B, Ggo, and Gge channels. To do.

  Accordingly, the selectors 224a to 224d can output the channel ID corresponding to the pixel ID for each channel of each filter component by selecting and outputting the reception value corresponding to the pixel ID designated by the selector 222. Become. In FIG. 30, as an example, the output value from the channel ID generation decoder 223 when the number of read channels of the image sensor is “6” and MUX type A is shown. At this time, the channel ID repeat unit is “3”, and the count value of the counter 221 c is selected by the selector 222. The channel ID generation decoder 223 outputs channel IDs of “0”, “2”, and “1” to the input terminals corresponding to the pixel IDs “0” to “3” of the selector 224a. The selector 224a outputs “0” when the pixel ID from the selector 222 is “0”, “2” when the pixel ID is “1”, and “1” when the pixel ID is “2” as CHID_R_Gb. To do.

  With such a configuration, channel IDs corresponding to various channel ID sequences can be generated. For example, as the number of repetitions of counters that generate pixel IDs (here, counters 221a to 221d) and the number of input terminals of channel ID output stage selectors (herein, selectors 224a to 224d) increase, the more various It becomes possible to generate a channel ID sequence.

  Note that the delay adjustment unit 225 receives the image signal from the rearrangement processing unit 21 so that the channel ID output from the selectors 224a to 224d matches the output timing of the corresponding pixel in the image signal rearranged for each filter component. Delay.

  Next, as examples of processing blocks using channel IDs in the camera signal processing unit 23, processing blocks for digital clamping and gain correction between channels are given, and their configurations and operations will be described.

FIG. 31 is a block diagram illustrating an internal configuration of the camera signal processing unit 23.
As shown in FIG. 31, the camera signal processing unit 23 includes a digital clamp unit 110, an inter-channel gain correction unit 120, and other processing blocks 130.

  The digital clamp unit 110 is a processing block that reproduces a direct current component in an image signal at a constant level to reduce low frequency noise. The digital clamp unit 110 receives clamp control values Clamp_CHID0 to Clamp_CHID7 for each channel ID as control parameters from the camera control circuit 15. Then, using the clamp control value corresponding to the channel ID from the channel ID generation unit 22, the clamp process is performed on the image signal input from the rearrangement unit 21 via the channel ID generation unit 22.

  The inter-channel gain correction unit 120 is a processing block that corrects gain variation between output channels of the image sensor. The inter-channel gain correction unit 120 receives gain control values Gain_CHID0 to Gain_CHID7 for each channel ID as control parameters from the camera control circuit 15. Then, the gain control value corresponding to the channel ID input from the channel ID generation unit 22 via the digital clamp unit 110 is used to adjust the gain of the image signal and output to the other processing block 130.

FIG. 32 is a block diagram showing an internal configuration of the digital clamp unit 110.
As shown in FIG. 32, the digital clamp unit 110 includes selectors 111a to 111d, clamp processing units 112a to 112d, and delay adjustment units 113a to 113d corresponding to R / Gb, Gr / B, Ggo, and Gge channels. ing.

  Each of the selectors 111a to 111d receives the clamp control values Clamp_CHID0 to Clamp_CHID7 from the camera control circuit 15, and receives clamp control values corresponding to the channel IDs (CHID_R_Gb, CHID_Gr_B, CHID_Ggo, CHID_Gge) from the channel ID generation unit 22, respectively. Select and output. The clamp processing units 112a to 112d perform clamp processing on the image signals of the corresponding channels according to the clamp control values output from the selectors 111a to 111d, respectively. The delay adjusting units 113a to 113d adjust the output timing by delaying the channel ID in accordance with the output timing of the image signals from the clamp processing units 112a to 112d.

  Here, detection for digital clamp processing is performed by integrating the output signal of the black region (OPB region) provided on the image sensor for each channel ID. If this detection is performed based on the image signal after the rearrangement by the rearrangement processing unit 21, the sequence of the origin channel becomes complicated as described above, so that it is easier to use the image signal before the rearrangement. Detection can be performed by control.

  In the present embodiment, the camera control circuit 15 generates a clamp control value for each channel ID according to the detection value from the image signal before rearrangement, and the digital clamp unit 110 sets the clamp control value corresponding to the channel ID. Apply. Accordingly, it is possible to accurately perform the clamping process on the rearranged image signal in which the sequence of the origin channel is complicated while performing detection with simple control.

FIG. 33 is a block diagram illustrating an internal configuration of the inter-channel gain correction unit 120.
As shown in FIG. 33, the inter-channel gain correction unit 120 includes selectors 121a to 121d and gain adjustment units 122a to 122d corresponding to R / Gb, Gr / B, Ggo, and Gge channels.

  The selectors 121a to 121d receive the gain control values Gain_CHID0 to Gain_CHID7 from the camera control circuit 15, and select gain control values corresponding to the channel IDs (CHID_R_Gb, CHID_Gr_B, CHID_Ggo, CHID_Gge) from the digital clamp unit 110, respectively. And output. The gain adjustment units 122a to 122d perform gain adjustment on the image signals of the corresponding channels according to the gain control values output from the selectors 121a to 121d, respectively.

  When there is a block that performs processing using the channel ID further after the interchannel gain correction unit 120, the channel ID from the digital clamp unit 110 is used as the image signal from the gain adjustment units 122a to 122d. A function of delaying the output in accordance with the output timing is provided.

  Here, the gain control values Gain_CHID0 to Gain_CHID7 can be obtained by, for example, imaging a uniform subject before product shipment, integrating the effective area for each channel ID, and taking the ratio between channels. The detected value is stored in an EEPROM (Electronically Erasable and Programmable ROM) or the like, and the camera control circuit 15 reads it and outputs it to the inter-channel gain correction unit 120.

  Thus, the detection of the gain control value is performed based on the image signal before rearrangement by the rearrangement processing unit 21. On the other hand, in the rearranged image signal, the sequence of the origin channel is in a complicated state, but the inter-channel gain correction unit 120 applies the gain value corresponding to the channel ID as described above. As a result, gain correction can be performed accurately.

  In the description of FIG. 33, for the sake of simplicity, it is assumed that the gain variation is constant regardless of the image signal level. For example, this variation varies depending on the brightness (image signal level). In this case, a gain control value may be prepared for each brightness and set in the inter-channel gain correction unit 120.

  As described above, the channel ID generation unit 22 can freely generate channel IDs suitable for the rearranged image signals according to the control parameters from the camera control circuit 15. The subsequent camera signal processing unit 23 performs signal processing based on the channel ID, thereby eliminating the need to be aware of various sequences of origin channels that may occur due to rearrangement. Therefore, it is possible to process with the same configuration circuit regardless of the change of the sequence of the origin channel, and to the combination of the image sensor and the multiplexing method without complicating the processing procedure or increasing the circuit scale. The digital signal processing circuit 14 having extremely high versatility can be realized.

[Second Embodiment]
FIG. 34 is a block diagram showing an internal configuration of a digital signal processing circuit according to the second embodiment of the present invention. In FIG. 34, functional blocks corresponding to the configuration shown in FIG. 7 are denoted by the same reference numerals, and description thereof is omitted.

  In the digital signal processing circuit 14 shown in FIG. 34, a channel ID generation unit 22a for generating a channel ID is arranged in the preceding stage of the rearrangement processing unit 21a for rearranging the image signals, and the generated channel ID is combined with the image signal. The point which rearranges by the rearrangement process part 21a differs from the case of FIG.

  In the channel ID generation unit 22a, the generated channel ID is defined as in FIG. The channel IDs (CHID_1 to CHID_4) generated by the channel ID generation unit 22a correspond to the four channel image signals (multiplexed signals Sig1 to Sig4) after being multiplexed by the AFE circuit 13, and are rearranged. The processing unit 21a rearranges the filter components of R / Gb, Gr / B, Ggo, and Gge in the same procedure as those image signals, and outputs CHID_R_Gb, CHID_Gr_B, CHID_Ggo, and CHID_Gge.

FIG. 35 is a block diagram showing an internal configuration of the channel ID generation unit 22a.
As illustrated in FIG. 35, the channel ID generation unit 22a includes a counter 226, selectors 227a to 227d, and a delay adjustment unit 228. The counter 226 performs a 1-bit counting operation for each horizontal synchronization period by resetting the count value and starting counting up when receiving the horizontal synchronization signal HD from the SG 25.

  The selectors 227a to 227d correspond to the output channels of the signals Sig1 to Sig4 from the AFE circuit 13, respectively. Each of the selectors 227a to 227d receives input of two numerical values determined in advance according to the definition of the channel ID, selects an input value corresponding to the count from the counter 226, and outputs it as CHID_1 to CHID_4, respectively.

  As described above, in each of the output signals Sig1 to Sig4 from the AFE circuit 13, two origin channel numbers appear alternately, so that the channel ID generation unit 22a has one value for each of the binary values from the counter 226. What is necessary is just to allocate and output channel ID. If the channel ID is defined similarly to FIG. 27, as shown in FIG. 35, the selector 227a is “0” “1”, the selector 227b is “2” “3”, the selector 227c is “4” “5”, The selector 227d alternately outputs “6” and “7” in accordance with the count value of the counter 226.

  Note that, for example, when three or more source channel numbers repeatedly appear in each of the output signals Sig1 to Sig4, such as when multiplexing three or more channels, the number of inputs of the selectors 227a to 227d according to the number of repetitions, This can be dealt with by increasing the number of repetitions of the count value of the counter 226. Further, input values to the selectors 227a to 227d are changed according to the control parameters from the camera control circuit 15, or counters having different repetition numbers are prepared and those count values are selected for the selectors 227a to 227d. By making it input manually, it becomes possible to cope with more various specifications such as a multiplexing method.

  The delay adjustment unit 228 delays the output signals Sig1 to Sig4 from the AFE circuit 13 in accordance with the output timing of the selectors 227a to 227d, and outputs the delayed signals to the rearrangement processing unit 21a.

FIG. 36 is a block diagram showing an internal configuration of the rearrangement processing unit 21a.
The configuration of the rearrangement processing unit 21a shown in FIG. 36 is basically the same as that of the first embodiment. However, the difference is that the channel IDs (CHID_1 to CHID_4) from the channel ID generation unit 22a are rearranged and output in the same procedure together with the multiplexed signals Sig1 to Sig4.

  Similar to the EN generation unit 30 in FIG. 9, the EN generation unit 30a performs R / Gb, Gr / B, Ggo, and Rg for each of the multiplexed signals Sig1 to Sig4 according to the EN generation parameter from the camera control circuit 15. An enable signal indicating the timing at which a signal corresponding to each filter component of Gge appears is generated and output to the write control unit 50a. Further, the multiplexed signals Sig1 to Sig4 and the channel IDs (CHID_1 to CHID_4) are delayed and output in accordance with the output timing of the enable signal.

  The rearrangement memory 40a includes storage areas for the respective filter components of G / Gb, Gr / B, Ggo, and Gge, but unlike the case of FIG. 9, each storage area includes an image signal for each filter component. In addition, channel IDs (CHID_R_Gb, CHID_Gr_B, CHID_Ggo, CHID_Gge) corresponding to each signal are also stored.

  The write control unit 50a rearranges the multiplexed signals Sig1 to Sig4 after delay adjustment based on the enable signal from the EN generation unit 30a, and generates signals Sig_R_Gb, Sig_Gr_B, Sig_Ggo, and Sig_Gge rearranged for each filter component. To do. Then, write signals WADRS_R_Gb, WADRS_Gr_B, WADRS_Ggo, WADRS_Gge and write enable signals WEN_R_Gb, WEN_Gr_B, WEN_Ggo, WEN_Gge corresponding to the rearranged signals and channel IDs corresponding thereto are generated. Write to the replacement memory 40a.

  The read control unit 60a outputs the read enable signals REN_R_Gb, REN_Gr_B, REN_Ggo, REN_Gge and the read addresses RADRS_R_Gb, RADRS_Gr_B, RADRS_Ggo, RADRS_Gge to the rearrangement memory 40a, and separates R_g and G , Sig_Ggo, Sig_Gge and corresponding channel IDs (CHID_R_Gb, CHID_Gr_B, CHID_Ggo, CHID_Gge) are read out to the camera signal processing unit 23.

  With the above configuration, the channel ID rearranged in accordance with the color sequence in the image signal is output to the camera signal processing unit 23, and the camera signal processing unit 23 performs processing using the input channel ID, thereby As in the first embodiment, processing can always be performed in the same procedure regardless of the input sequence to the digital signal processing circuit 14.

  In the second embodiment described above, the channel ID is generated based on the image signal before rearrangement, thereby simplifying the configuration of the block (channel ID generation unit 22a) for generating the channel ID, and its circuit. Scale can be controlled. Further, since the rearrangement method of the channel ID is the same as that of the image signal, the control parameter from the camera control circuit 15 is not necessary when the channel ID is generated, and the memory capacity for holding the control parameter or the setting Power consumption can be reduced. However, the rearrangement processing unit 21a requires a storage area for holding channel IDs in addition to image signals, and a circuit for rearrangement thereof, which increases the circuit scale.

[Third Embodiment]
In the following third embodiment, a case where a defective pixel correction function is provided as a kind of function to be processed using a channel ID will be described. FIG. 37 is a block diagram illustrating an internal configuration of a camera signal processing unit according to the third embodiment. In FIG. 37, functional blocks corresponding to the configuration shown in FIG. 31 are denoted by the same reference numerals, and description thereof is omitted.

  In the camera signal processing unit 23 shown in FIG. 37, in addition to the digital clamp unit 110 and the inter-channel gain correction unit 120b, a defective pixel correction unit 300 is provided as another processing block 130b. Note that the inter-channel gain correction unit 120b has the same function as the inter-channel gain correction unit 120 shown in FIG. 31, and in addition to the channel from the digital clamp unit 110 according to the output timing of the image signal subjected to gain correction. A function of delaying the ID and outputting it to the other processing block 130b in the subsequent stage (may be directly output to the defective pixel correction unit 300) is provided.

  Note that the channel ID needs to be delayed in accordance with the propagation delay time of the image signal during the time it is propagated from the inter-channel gain correction unit 120b through the other processing block 130b and input to the defective pixel correction unit 300. There is. At this time, the channel ID may be simply delayed by the same time as the propagation delay time of the image signal. However, when there are many processing blocks that do not use the channel ID in the middle, the amount of delay increases, and the delay circuit The scale will increase.

  Therefore, paying attention to the repeatability of the channel ID sequence, a minimum necessary delay circuit (for example, a delay circuit for 4 clocks) corresponding to the number of repetitions is prepared, and the delay is performed by an instruction signal from the camera control circuit 15. The number of stages (number of clocks) may be switched.

  Here, the defective pixel correction will be described. In an image sensor, defective pixels may be generated due to various causes such as generation of dark current and abnormality of a photodiode. Since a defective pixel outputs an abnormal level signal, it is generally performed to interpolate a defective pixel signal using a signal of a peripheral pixel.

  For this defective pixel, the output level may be abnormally high (so-called white defect) or low (so-called black defect). However, in recent image sensors, the output signal from the former defective pixel is huge. In such a case, there has been a problem that a “drag” phenomenon occurs in which the signal affects other pixels that are not defective. There are two types of “drag” phenomena:

  The first phenomenon occurs in the same output channel from the image sensor. In recent image sensors, the readout frequency has become very high due to the increase in the number of pixels. Therefore, in an analog circuit that processes the readout signal, a margin for amplifier performance and a settling time margin for CDS / sample hold are sufficient. It is gone. In such a circuit, when an impulse signal from a defective pixel is input, a subsequent signal passing through the same output channel may be dragged by a previous huge signal, which may affect the signal.

  The second phenomenon occurs between adjacent channels input to the multiplexing circuit. As the analog circuit is highly integrated, the channel input to the multiplexing circuit in the AFE circuit 13 is narrowed. Therefore, the signal of the adjacent channel to be multiplexed is dragged by the huge level signal described above. An abnormal signal may occur.

  When correcting defective pixels, not only the signal interpolation of defective pixels, but also the signals of pixels that are affected by the “drag” as described above (hereinafter referred to as “drag target pixels”) use the signals of surrounding pixels. And have to interpolate. Here, in a system that does not rearrange the image signals as described above, the drag target pixel is always present next to a pixel that generates a huge level (hereinafter referred to as a giant defective pixel). Easy to handle. However, in the rearranged image signal, there is no guarantee that the drag target pixel is adjacent to the giant defective pixel, and their positional relationship changes according to the color sequence of the multiplexed signal. For this reason, like the digital clamp unit and the inter-channel gain correction unit described above, if the defective pixel correction unit 300 is configured to be able to handle all of these variations, the processing becomes complicated and the circuit scale increases. .

  Therefore, in this embodiment, by using the channel ID also in the correction process of the drag target pixel, the position of the drag target pixel can be easily specified from the rearranged image signal, and the interpolation process is always performed with a simple process. Can be executed.

FIG. 38 is a block diagram showing an internal configuration of the defective pixel correction unit 300.
As shown in FIG. 38, the defective pixel correction unit 300 includes a defect detection unit 310, an EEPROM 320, a memory controller 330, a correction signal generation unit 340, and a signal correction unit 350.

  The defect detection unit 310 is an image of each channel of R / Gb, Gr / B, Ggo, and Gge input to the defective pixel correction unit 300 based on the defect level setting value and the giant defect level setting value from the camera control circuit 15. A defective pixel and a giant defective pixel are detected from the signal, and an address indicating the position of the pixel is output to the memory controller 330. The defect level setting value and the giant defect level setting value are signal level thresholds for detecting normal defect pixels and giant defect pixels, respectively.

  When detecting a defective pixel, under the control of the camera control circuit 15, when a large defective pixel (including a giant defective pixel) is detected with a signal level protruding, the iris is closed and the defective pixel has an extremely small signal level. When detecting, the exposure time is lengthened. The defect detection unit 310 compares the detection value in such a state with the defect level setting value and the giant defect level setting value to determine whether or not the pixel is a defect pixel and whether or not the pixel is a giant defect pixel. Determine. If it is determined that the pixel is a defective pixel or a giant defective pixel, the horizontal and vertical positions are latched as defective addresses and output to the memory controller 330 for each channel. At that time, information indicating whether the pixel is a giant defective pixel is also added.

  The EEPROM 320 has separate storage areas for each of R / Gb, Gr / B, Ggo, and Gge channels. The memory controller 330 writes the defect address from the defect detection unit 310 in the corresponding storage area. Then, at the time of actual imaging, a defect address is read from the EEPROM 320 in accordance with a memory control signal from the correction signal generation unit 340 and is output to the correction signal generation unit 340. At this time, a giant defect flag indicating whether or not the pixel is a giant defect is also output.

  Note that the operations of detecting defective pixels and huge defective pixels and writing defective addresses to the EEPROM 320 are performed, for example, at the time of adjustment before product shipment. Alternatively, it may be executed at any time even after shipment, for example, when the power is turned on.

  The correction signal generation unit 340 indicates the appearance timing of the defective pixel and the giant defective pixel according to the input of the image signal of each channel of R / Gb, Gr / B, Ggo, and Gge to the defective pixel correction unit 300 during actual imaging. A correction timing signal is output. The correction signal generation unit 340 counts the address on the screen based on the correction horizontal synchronizing signal and vertical synchronizing signal supplied from the SG 25, and if it matches the defect address read from the EEPROM 320, the correction timing signal is sent to the channel. Output every time. In addition, a drag target pixel correction decoder 400 for generating the drag target pixel appearance timing is provided, and when a huge defect flag is set from the memory controller 330, the channel ID inputted thereafter is scanned to scan the drag target pixel. The appearance timing of the pixel is detected, and a correction timing signal is output as a pixel to be corrected.

  The signal correction unit 350 corrects the image signals of the R / Gb, Gr / B, Ggo, and Gge channels according to the correction timing signal from the correction signal generation unit 340, and outputs the corrected image signal to the subsequent circuit.

FIG. 39 is a block diagram showing an internal configuration of the signal correction unit 350.
As shown in FIG. 39, the signal correction unit 350 includes delay lines 351a to 351d and interpolation processing units 352a to 352d corresponding to R / Gb, Gr / B, Ggo, and Gge channels, respectively.

  The signal correction of the defective pixel and the huge defective pixel is basically performed by linear interpolation using pixel signals of the same filter component existing in the vicinity thereof. However, the captured image signal has various patterns such as a correlation in the vertical direction or a correlation in the horizontal direction depending on the scene. Therefore, the signal correction unit 350 holds the pixels of the same filter component in two upper and lower lines adjacent to the pixel to be interpolated so that the direction of linear interpolation can be changed according to the pattern so that the pattern can be changed even after interpolation. I'm trying not to fail.

  For this purpose, each of the delay lines 351a to 351d includes a line memory for four lines, and can simultaneously output image signals for five adjacent lines in each channel. Here, since the R and B components appear for each line in the input image signal, each delay line 351a to 351d outputs an image signal for five lines instead of three lines. For this reason, the delay lines 351c and 351d of the Ggo and Gge channels may include only a line memory for two lines.

  In the output image signals from the delay lines 351a to 351d, signals delayed by 2H are to be interpolated, and the interpolation processing units 352a to 352d perform the correction timing of the corresponding channel from the correction signal generation unit 340 when the interpolation target pixel is input. When the signal is valid (for example, at the H level), linear interpolation is performed using the signal of the adjacent pixel of the same filter component existing in the direction of strong correlation among the upper, lower, left, and right, and replaced with the signal of the interpolation target pixel.

FIG. 40 is a block diagram showing an internal configuration of the drag target pixel correction decoder 400. As shown in FIG.
As illustrated in FIG. 40, the drag target pixel correction decoder 400 includes correction timing decoders 401 to 404 for each of R / Gb, Gr / B, Ggo, and Gge channels. Each of the correction timing decoders 401 to 404 receives the input of the giant defect flag of the corresponding channel, and when the giant defect flag is set, scans the channel ID that is input thereafter to detect the input timing of the drag target pixel and corrects it. Output timing signal.

  FIG. 41 is a diagram for explaining the detection of the drag target pixel. In FIG. 41, as an example of the channel ID sequence in the rearranged image signal, a case where the number of read channels of the image sensor is “8” and MUX type A is shown.

  As described above, there are two types of “drag” phenomenon of the signal of a giant defective pixel. Among these, in the case of “drag” on the same output channel from the image sensor, the signal of the giant defective pixel affects the next pixel adjacent in the time direction in the output signal of the image sensor, so the pixel to be dragged is , The pixel with the same channel ID that appears next to the giant defective pixel.

  On the other hand, in the case of “dragging” between adjacent channels, the signal of the giant defective pixel affects the next pixel adjacent in the time direction in the output signal of the AFE circuit 13. Such a relationship is based on the definition of the channel ID in FIG. 27, between “0” and “1”, “2” and “3”, “4” and “5”, and “6” and “7”. Since it is established in the sequential number, the pixel of the other channel ID that appears next to the giant defective pixel in each combination is the drag target pixel.

  Accordingly, each of the correction timing decoders 401 to 404 of the drag target pixel correction decoder 400 scans the channel ID input after the appearance of the giant defective pixel, and determines the pixel of the channel ID corresponding to each of the above conditions as the drag target pixel. . For example, in the example of FIG. 41, when a giant defective pixel appears in the R / Gb channel at the third clock, the channel ID of that pixel is “0”. A pixel where “1” appears may be determined as a target pixel by dragging. In the case of FIG. 41, the former appears in the fifth clock R / Gb channel, and the latter appears in the third clock Gr / B channel. In this way, by using the channel ID, it is possible to easily specify the pixel to be dragged even if the channel ID sequence changes due to rearrangement.

FIG. 42 is a block diagram showing the internal configuration of the correction timing decoder 401.
The correction timing decoder 401 is a decoder for detecting the appearance timing of the drag target pixel corresponding to a pixel when a giant defective pixel appears in the R / Gb channel signal. The correction timing decoder 401 includes a latch unit 411, an addition / subtraction unit 412, comparison units 413 and 414, and an OR gate 415.

  When the giant defect flag of the R / Gb channel is set, the latch unit 411 latches the channel ID (CHID_R_Gb) corresponding to the R / Gb channel input at that time. The addition / subtraction unit 412 adds “1” to the value when the output value of the latch unit 411 is an even number, and subtracts “1” when the output value is an odd number.

  The comparison unit 413 is a block for detecting a drag target pixel in the same output channel from the image sensor. The comparison unit 413 starts operation when the R / Gb channel giant defect flag is set, and the output value of the latch unit 411 at that time, that is, the channel ID of the giant defect pixel in the R / Gb channel, is input thereafter. When at least one channel ID matches, it is determined as a pixel to be dragged, a correction timing signal is output, and the comparison operation is stopped.

  Here, the scanning is repeatedly performed in the order of CHID_Gr_B, CHID_Ggo, CHID_Gge, and CHID_R_Gb, and the comparison operation is executed. Then, only the correction timing signal corresponding to the input channel (any of R / Gb, Gr / B, Ggo, and Gge) whose comparison value matches is set to the H level. For example, in the example of FIG. 41, since the channel ID (CHID_R_Gb) corresponding to the R / Gb channel matches, the correction timing signal corresponding to the R / Gb channel is set to the H level, and the comparison operation is stopped. Correction timing signals corresponding to the R / Gb channels are output to the OR gate 415, and correction timing signals corresponding to the Gr / B, Ggo, and Gge channels are OR gates provided in the correction timing decoders 402 to 404 of the corresponding channels. 415 is output.

  On the other hand, the comparison unit 414 is a block for detecting a drag target pixel between adjacent output channels from the image sensor. Such drag target pixels are serial channel IDs based on the combinations described above, and the channel IDs are generated by the adder / subtractor 412. Therefore, the comparison unit 414 starts operation when the giant defect flag of the R / Gb channel is set, and compares the output value of the addition / subtraction unit 412 at that time with the channel ID that is input thereafter, and at least one of them is compared. When they coincide with each other, the pixel is determined as a drag target pixel, a correction timing signal is output, and the comparison operation is stopped.

  Here, similar to the comparison unit 413, the CHID_Gr_B, CHID_Ggo, CHID_Gge, and CHID_R_Gb are repeatedly scanned in this order to perform the comparison operation, and the input channels (R / Gb, Gr / B, Ggo, and Gge with the same comparison value are matched). Only the correction timing signal corresponding to any one) is set to the H level. For example, in the example of FIG. 41, since the channel ID (CHID_Gr_B) corresponding to the Gr / B channel matches, the correction timing signal corresponding to the Gr / B channel is set to H level, and the comparison operation is stopped. Correction timing signals corresponding to the R / Gb channels are output to the OR gate 415, and correction timing signals corresponding to the Gr / B, Ggo, and Gge channels are OR gates provided in the correction timing decoders 402 to 404 of the corresponding channels. 415 is output.

  The OR gate 415 receives the giant defect flag, the correction timing signal from the comparison units 413 and 414, and the correction timing signal corresponding to the R / Gb channel output from the other correction timing decoders 402 to 404. When any signal becomes H level, a final correction timing signal corresponding to the R / Gb channel is output. That is, the output signal from the OR gate 415 represents the appearance timing of the giant defective pixel in the R / Gb channel and the drag target pixel corresponding to the pixel.

  The configuration of the other correction timing decoders 402 to 404 is basically the same as described above. However, the connection order of the channel ID input terminals from the memory controller 330 is different. For example, in association with the configuration of FIG. 42, CHID_Gr_B is input to the latch unit 411 in the correction timing decoder 402 corresponding to the Gr / B channel, and CHID_Ggo, CHID_Gge, CHID_R_Gb, CHID_Gr_B in the comparison units 413 and 414. The channel ID is input so that scanning is performed in this order.

  Further, in the correction timing decoder 403 corresponding to the Ggo channel, CHID_Ggo is input to the latch unit 411, and in the comparison units 413 and 414, the channel ID is input so as to be scanned in the order of CHID_Gge, CHID_R_Gb, CHID_Gr_B, CHID_Ggo. Is done. In the correction timing decoder 404 corresponding to the Gge channel, CHID_Gge is input to the latch unit 411, and in the comparison units 413 and 414, the channel ID is input so as to be scanned in the order of CHID_R_Gb, CHID_Gr_B, CHID_Ggo, and CHID_Gge. . Note that giant defect flags corresponding to Gr / B, Ggo, and Gge channels are input to the correction timing decoders 402 to 404, respectively.

  With such a configuration, the correction timing decoders 401 to 404 output correction timing signals corresponding to the R / Gb, Gr / B, Ggo, and Gge channels, respectively. By operating in response to the signal, not only a normal defective pixel but also a huge defective pixel and its drag target pixel are interpolated. Therefore, even if the channel ID sequence changes variously due to the rearrangement, all defective pixels and drag target pixels can be reliably corrected by the circuit having the same configuration.

  Note that the scan order as described above in the comparison units 413 and 414 is specialized when the reading and multiplexing methods shown in FIGS. 2, 4, and 5 are employed. Therefore, the scan order in the comparison units 413 and 414 may be changed by the setting from the camera control circuit 15 in order to be able to cope with systems with different reading orders and multiplexing methods. Further, when the channel ID is defined based on, for example, the output channel of the image sensor instead of the multiplexed image signal, the pixel to be dragged between adjacent channels is not necessarily located at the position of the consecutive channel ID. The position changes according to the conversion method. However, in this case as well, there is a law in the positional relationship of the drag target pixels in the rearranged image signal. For example, this can be dealt with by making the number of additions / subtractions in the adder / subtractor 412 variable.

  Further, in the defective pixel correction unit 300, the function of the drag target pixel correction decoder 400 is provided in the defect detection unit 310 instead of the correction signal generation unit 340, and when a huge defective pixel is detected, the drag target pixel is detected. The position may be detected and recorded in the EEPROM 320. Thereby, the configuration and processing procedure of the correction signal generation unit 340 can be simplified. However, since the storage capacity required for the EEPROM 320 is increased, in order to save the storage capacity, for example, when the detection timing of the drag target pixel is reached, a defect of a normal defective pixel (that is, a defective pixel excluding a giant defective pixel) is detected. The detection operation may be disabled so that addresses of the same pixel are not recorded in the EEPROM 320 as defective pixels and drag target pixels.

  In each of the above embodiments, the case where the filter coding shown in FIG. 2 is used has been described. However, the present invention can also be applied to an imaging apparatus using other filter coding. For example, in the case of the Bayer array, generally, the camera signal processing unit often processes four types of filter components R, Gr, Gb, and B in parallel in both the odd H period and the even H period. Even in such a case, by using the rearrangement processing unit and the channel ID generation unit having the same configuration as described above, the camera signal processing can be performed regardless of the combination of the number of read channels of the image sensor, the number of pixels, the multiplexing method, and the like. The image signals of the above four types of filter components can always be output from each of the four output channels in accordance with the section, and various sequences of origin channels that can be generated by rearrangement in the subsequent camera signal processing section It becomes possible to process without being conscious of.

  Further, the present invention is applied to an imaging device using a solid-state imaging device such as a digital video camera or a digital still camera, and a device such as a mobile phone or a PDA (Personal Digital Assistants) having such an imaging function. be able to. Further, the present invention is applied not only to a signal processing circuit for an imaging apparatus but also to a processing circuit that accepts and processes a specific data array through a plurality of channels and a large number of input data sequences are assumed. Can do.

  Further, the above processing functions can be realized by a computer. In that case, a program describing the processing contents of the functions that the apparatus should have (functions corresponding to the above-described rearrangement processing unit and channel ID generation unit) is provided. And the said processing function is implement | achieved on a computer by running the program with a computer. The program describing the processing contents can be recorded on a computer-readable recording medium. Examples of the computer-readable recording medium include a magnetic recording device, an optical disk, a magneto-optical disk, and a semiconductor memory.

  In order to distribute the program, for example, portable recording media such as an optical disk and a semiconductor memory on which the program is recorded are sold. It is also possible to store the program in a storage device of a server computer and transfer the program from the server computer to another computer via a network.

  The computer that executes the program stores, for example, the program recorded on the portable recording medium or the program transferred from the server computer in its own storage device. Then, the computer reads the program from its own storage device and executes processing according to the program. The computer can also read the program directly from the portable recording medium and execute processing according to the program. Further, each time the program is transferred from the server computer, the computer can sequentially execute processing according to the received program.

1 is a block diagram illustrating a main configuration of an imaging apparatus according to a first embodiment of the present invention. It is a figure which shows the arrangement | sequence of the color filter in a CMOS sensor. It is a figure which shows the structure for the multichannel read-out from a CMOS sensor, and the multiplexing of the signal. It is a figure which shows the example of allocation of the output channel with respect to the pixel position on a CMOS sensor. It is a figure which shows the example of the variation of the multiplexing in an AFE circuit. It is a figure which shows the example of the multiplexing at the time of 8-channel reading. It is a block diagram which shows the internal structure of a digital signal processing circuit. It is a figure which shows the example of the color sequence before rearrangement by the rearrangement process part, and after rearrangement. It is a block diagram which shows the internal structure of a rearrangement process part. It is a block diagram which shows the internal structure of EN production | generation part. It is a figure which shows a color sequence and an enable signal in case the number of read-out channels of an image sensor is "8" and MUX type A. It is a figure which shows the color sequence and enable signal in case the number of read-out channels of an image sensor is "6" and MUX type A. It is a block diagram which shows the internal structure of a write-control part. It is a block diagram which shows the internal structure of a signal distribution part. It is a block diagram which shows the internal structure of a decoder. It is a figure for demonstrating the decoding operation | movement of the selection signal in case the number of read-out channels of an image pick-up element is "8" and it is MUX type A. It is a figure for demonstrating the decoding operation | movement of the selection signal in case the read-out channel number of an image pick-up element is "6" and it is MUX type A. It is a figure which shows the image signal arrangement | sequence in case the number of read-out channels of an image pick-up element is "8", and is MUX type A. It is a figure which shows the image signal arrangement | sequence in case the number of read-out channels of an image pick-up element is "8", and is MUX type B, respectively. It is a figure which shows the image signal arrangement | sequence in case the number of read-out channels of an image pick-up element is "6", and is MUX type A. It is a figure which shows the image signal arrangement | sequence in case the number of read-out channels of an image pick-up element is "6", and is MUX type B, respectively. It is a figure which shows the image signal arrangement | sequence in case the number of read-out channels of an image pick-up element is "4", and is MUX type A. It is a figure which shows the image signal arrangement | sequence in case the number of read-out channels of an image pick-up element is "4", and is MUX type B, respectively. It is a figure which shows the image signal arrangement | sequence in case the number of read-out channels of an image pick-up element is "2" and MUX type A and B. FIG. It is a block diagram which shows the internal structure of a read control part. It is a figure which shows the example of the color sequence before and behind a rearrangement process. It is a figure which shows the definition method of channel ID. It is a figure which shows the channel ID sequence in case the number of channels read from an image sensor is "8" and "6". It is a figure which shows a channel ID sequence in case the number of read channels is "4" and "2". It is a block diagram which shows the internal structure of a channel ID production | generation part. It is a block diagram which shows the internal structure of a camera signal processing part. It is a block diagram which shows the internal structure of a digital clamp part. It is a block diagram which shows the internal structure of the gain correction part between channels. It is a block diagram which shows the internal structure of the digital signal processing circuit which concerns on the 2nd Embodiment of this invention. It is a block diagram which shows the internal structure of the channel ID production | generation part which concerns on 2nd Embodiment. It is a block diagram which shows the internal structure of the rearrangement process part which concerns on 2nd Embodiment. It is a block diagram which shows the internal structure of the camera signal processing part which concerns on 3rd Embodiment. It is a block diagram which shows the internal structure of a defective pixel correction | amendment part. It is a block diagram which shows the internal structure of a signal correction | amendment part. It is a block diagram which shows the internal structure of the dragging object pixel correction decoder. It is a figure for demonstrating the detection of a dragging object pixel. It is a block diagram which shows the internal structure of a correction timing decoder. It is a block diagram which shows the example of a principal part structure of the conventional imaging device using the 1 channel output type imaging device. It is a figure explaining the color sequence at the time of outputting the pixel signal of the image pick-up element of a Bayer arrangement from one output channel. It is a figure explaining the color sequence at the time of reading a pixel signal by 2 channels. It is a figure explaining the color sequence at the time of reading a pixel signal by 3 channels. It is a figure explaining the color sequence at the time of reading a pixel signal by 4 channels. It is a figure explaining the color sequence at the time of reading the pixel signal of 2 lines by 2 channels. It is a figure explaining the color sequence at the time of reading the pixel signal of 2 lines by 4 channels. It is a figure explaining the color sequence at the time of reading the pixel signal of 2 lines by 6 channels. It is a block diagram which shows the structural example of the signal processing system provided with the multiplexing function of a pixel signal.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Optical block, 11a ... Driver, 12 ... CMOS type image sensor, 12a ... Timing generator, 13 ... Analog front end circuit, 14 ... Digital signal processing circuit, 15 ... Camera control circuit, 16 ... ... Human I / F control circuit, 17 ... User I / F, 18 ... Camera shake sensor, 21 ... Rearrangement processing unit, 22 ... Channel ID generation unit, 23 ... Camera signal processing unit, 24 ... Communication I / F, 25 …… Signal generator

Claims (16)

In an image processing apparatus that processes a color image signal,
Signal processing means for processing the image signals of a plurality of specific filter components in parallel using a channel ID that identifies from which readout channel of the solid-state imaging device the input image signal is read;
Storage means comprising individual storage areas respectively corresponding to the specific filter components;
Setting of control parameters including a rearrangement control parameter corresponding to the filter component input sequence in the image signal and a channel ID generation parameter corresponding to the channel ID sequence corresponding to the image signal read from the storage means Parameter receiving means to accept;
The input image signals of a plurality of channels based on the output signal of the solid-state image sensor are sorted for each specific filter component based on the setting of the control parameter according to the input sequence in the input image signal, and the corresponding memory Writing means for writing to the area;
Read means for sequentially reading out image signals from the respective storage areas through individual output channels and supplying them to the signal processing means;
A channel ID generating unit that generates the channel ID corresponding to the image signal read by the reading unit based on the setting of the channel ID generation parameter and supplies the channel ID to the signal processing unit;
An image processing apparatus comprising: The channel ID generation means includes
Counting means for repeatedly counting values up to the number of repetitions of the channel ID appearing in the channel ID sequence corresponding to the image signal from the reading means in synchronization with the input timing of each pixel signal;
A channel that outputs the value of the channel ID assigned to the image signal for each channel of the image signal from the reading unit and for each count value input timing by the counting unit based on the setting of the channel ID generation parameter ID assigning means;
A plurality of channel ID selections for receiving the input of the channel ID assigned by the channel ID assigning means for the same channel from the reading means, and selecting and outputting the channel ID corresponding to the count value of the counting means Means,
The image processing apparatus according to claim 1, further comprising: The channel ID generation means includes
A plurality of the counting means having different repetition numbers;
A count value selection unit that selects a count value of any of the count units according to the setting of the control parameter and supplies the selected count value to each channel ID selection unit;
The channel ID assigning means assigns the value of the channel ID by the number of repetitions of the counting means selected by the count value selecting means for each channel from the reading means by setting the channel ID generation parameter. Output,
The image processing apparatus according to claim 2.   The image processing apparatus according to claim 1, wherein the signal processing unit performs signal processing on the input image signal of each channel using different signal processing parameters for each corresponding channel ID.   5. The image processing apparatus according to claim 4, further comprising a digital clamp processing unit as the signal processing unit.   The image processing apparatus according to claim 4, further comprising: a gain correction unit that corrects a gain variation between the readout channels of the solid-state imaging device as the signal processing unit. The signal processing means includes a giant defect pixel correction means for correcting an output signal of a giant defect pixel having a relatively large output signal level in a state where there is no incident light on the solid-state imaging device,
The giant defect pixel correcting means is
Position storage means for holding position information of the giant defective pixel;
Based on the position information held by the position storage means, after the giant defective pixel appears in the input image signal of any channel, the channel ID same as that of the giant defective pixel is obtained from the image signals of all channels. Pixel detection means for detecting each appearance timing of the next pixel having the channel ID of the giant defective pixel and the next pixel having another channel ID having a predetermined relationship with the channel ID;
A signal interpolating unit for interpolating the signals of the giant defective pixels and the pixels detected by the pixel detecting means using signals of surrounding pixels corresponding to the same filter component;
The image processing apparatus according to claim 1, further comprising: Multiplexing means for time-division-multiplexing two sets of image signals of the readout channels from the solid-state imaging device and outputting the number of channels to the writing means as half the number of channels is provided. When two adjacent numerical values are alternately defined as the channel ID for the output pixel signal,
The pixel detection unit includes the next pixel having the same channel ID as the giant defective pixel and the adjacent pixel including the channel ID of the giant defective pixel after the giant defective pixel based on the position information appears. Detecting each appearance timing of the next pixel having the other numerical value of the two numerical values as the channel ID;
The image processing apparatus according to claim 7. In an image processing apparatus that processes a color image signal,
Signal processing means for processing the image signals of a plurality of specific filter components in parallel using a channel ID that identifies from which readout channel of the solid-state imaging device the input image signal is read;
Channel ID generation means for assigning the channel ID to an image signal of a plurality of channels based on an output signal of the solid-state imaging device and outputting the image signal together with the image signal;
Storage means comprising individual storage areas respectively corresponding to the specific filter components;
Parameter receiving means for accepting setting of rearrangement control parameters in accordance with the input sequence of filter components in the image signal;
The rearrangement according to the input sequence in the image signal, the image signals of a plurality of channels input through the channel ID generation unit and the channel ID from the channel ID generation unit corresponding to the image signal Based on the setting of the control parameter, writing means for each specific filter component and writing to the corresponding storage area;
Read means for sequentially reading out the image signals and the corresponding channel IDs from the respective storage areas through individual output channels, and supplying them to the signal processing means;
An image processing apparatus comprising: The channel ID generation means includes
Counting means for repeatedly counting values up to the repetition number of the channel ID appearing in the channel ID sequence corresponding to the input image signal in synchronization with the input timing of each pixel signal;
Provided for each channel of the input image signal, the input terminal corresponding to each count value of the count means receives input of the channel ID value unique to all channels, and the output value of the count means A plurality of channel ID selection means for selecting an input value according to and outputting to the writing means;
The image processing apparatus according to claim 9, further comprising: In an imaging device that captures an image using a solid-state imaging device,
Signal processing means for processing the image signals of a plurality of specific filter components in parallel using a channel ID for identifying from which readout channel of the solid-state image sensor the input image signal is read out;
Storage means comprising individual storage areas respectively corresponding to the specific filter components;
Setting of control parameters including a rearrangement control parameter corresponding to the filter component input sequence in the image signal and a channel ID generation parameter corresponding to the channel ID sequence corresponding to the image signal read from the storage means Parameter receiving means to accept;
The input image signals of a plurality of channels based on the output signal of the solid-state image sensor are sorted for each specific filter component based on the setting of the control parameter according to the input sequence in the input image signal, and the corresponding memory Writing means for writing to the area;
Read means for sequentially reading out image signals from the respective storage areas through individual output channels and supplying them to the signal processing means;
A channel ID generating unit that generates the channel ID corresponding to the image signal read by the reading unit based on the setting of the channel ID generation parameter and supplies the channel ID to the signal processing unit;
An imaging device comprising: In an imaging device that captures an image using a solid-state imaging device,
Signal processing means for processing the image signals of a plurality of specific filter components in parallel using a channel ID for identifying from which readout channel of the solid-state image sensor the input image signal is read out;
Channel ID generation means for assigning the channel ID to an image signal of a plurality of channels based on an output signal of the solid-state imaging device and outputting the image signal together with the image signal;
Storage means comprising individual storage areas respectively corresponding to the specific filter components;
Parameter receiving means for accepting setting of rearrangement control parameters in accordance with the input sequence of filter components in the image signal;
The rearrangement according to the input sequence in the image signal, the image signals of a plurality of channels input through the channel ID generation unit and the channel ID from the channel ID generation unit corresponding to the image signal Based on the setting of the control parameter, writing means for each specific filter component and writing to the corresponding storage area;
Read means for sequentially reading out the image signals and the corresponding channel IDs from the respective storage areas through individual output channels, and supplying them to the signal processing means;
An imaging device comprising: In an image processing method for processing an input image signal of a plurality of channels based on an output signal of a solid-state imaging device,
From the storage means, the parameter receiving means receives a rearrangement control parameter corresponding to the input sequence of the filter component in the input image signal, and a channel ID for identifying from which reading channel of the solid-state image sensor the image signal is read. Receiving a setting of control parameters including a channel ID generation parameter according to an appearance sequence in the image signal after reading out;
A writing unit that distributes the input image signal for each specific filter component based on the setting of the rearrangement control parameter, and writes the input image signal to each storage area provided for each specific filter component in the storage unit; ,
Reading means sequentially reading image signals from the respective storage areas through individual output channels;
A step of generating and outputting the channel ID corresponding to the image signal read by the reading unit based on the setting of the channel ID generation parameter;
A signal processing unit that processes the image signals of the output channels in parallel using the channel ID from the channel ID generation unit corresponding to the image signal from the reading unit;
An image processing method comprising: In an image processing method for processing an input image signal of a plurality of channels based on an output signal of a solid-state imaging device,
A step of receiving a setting of a rearrangement control parameter in accordance with an input sequence of filter components in the input image signal;
A step of assigning a channel ID identifying the input image signal from which readout channel of the solid-state image sensor to the input image signal and outputting the channel ID together with the input image signal;
The writing unit distributes the input image signal and the channel ID from the channel ID generation unit corresponding to the input image signal for each specific filter component based on the setting of the rearrangement control parameter, Writing to each of the storage areas provided for each specific filter component;
A step of sequentially reading out an image signal and the corresponding channel ID from each storage area through an individual output channel;
Signal processing means processing the image signals input through the respective output channels in parallel using the corresponding channel ID from the reading means;
An image processing method comprising: An image that causes a computer to execute a process of supplying a plurality of channels of input image signals based on an output signal of a solid-state imaging device to a signal processing circuit having a function of processing image signals of a plurality of specific filter components in parallel In the processing program,
A rearrangement control parameter corresponding to an input sequence of filter components in an image signal, and a channel ID for identifying from which readout channel of the solid-state image sensor the image signal is read out, in the image signal after readout from the storage unit Parameter receiving means for receiving control parameter settings including a channel ID generation parameter corresponding to the appearance sequence;
The input image signals of a plurality of channels based on the output signal of the solid-state imaging device are sorted for each specific filter component based on the setting of the control parameter according to the input sequence in the input image signal, and stored in the storage unit Writing means for writing to each of the storage areas provided for each of the specific filter components;
Read means for sequentially reading image signals from the respective storage areas through individual output channels and supplying them to the signal processing circuit;
Channel ID generation means for generating the channel ID corresponding to the image signal read by the reading means based on the setting of the channel ID generation parameter and supplying the channel ID to the signal processing circuit;
An image processing program for causing the computer to function as: An image that causes a computer to execute a process of supplying a plurality of channels of input image signals based on an output signal of a solid-state imaging device to a signal processing circuit having a function of processing image signals of a plurality of specific filter components in parallel In the processing program,
A channel for assigning a channel ID for identifying from which readout channel of each solid-state image sensor each image signal is read out to a plurality of channels of image signals based on the output signal of the solid-state image sensor, and outputting together with the image signal ID generation means,
Parameter receiving means for accepting setting of rearrangement control parameters according to the input sequence of filter components in the image signal;
The rearrangement according to the input sequence in the image signal, the image signals of a plurality of channels input through the channel ID generation unit and the channel ID from the channel ID generation unit corresponding to the image signal Based on the setting of the control parameter, writing means for each specific filter component, writing means for writing to each storage area provided for each specific filter component,
Read means for sequentially reading out image signals and corresponding channel IDs from the respective storage areas through individual output channels, and supplying them to the signal processing circuit;
An image processing program for causing the computer to function as:

Images