JP2008206030A - Solid-state imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To implement registration correction and format conversion with a small-scale circuit construction by adding some contrivance to construction and coefficients of an interpolation filter. <P>SOLUTION: In a solid-state imaging apparatus using a plurality of solid-state imaging devices, the number of divisions for one pixel in vertical registration correction is determined so as to cause an interpolation phase for vertical format conversion to be included in n interpolation phases in the vertical registration correction, and the number of divisions for one pixel in horizontal registration correction is determined so as to cause an interpolation phase for horizontal format conversion to be included in m interpolation phases in the horizontal registration correction. By so doing, interpolation filter coefficients of horizontal and vertical interpolation filters of a registration correction circuit and interpolation filter coefficients of horizontal and vertical interpolation filters of a format conversion circuit are used in common, and registration correction and format conversion are performed simultaneously at the time of television format conversion output. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

The present invention relates to a television camera having a television format conversion function for outputting video signals of a plurality of television formats.

  The prior art will be described with reference to FIGS. FIG. 2 is a block diagram of a conventional television camera having a TV format conversion function. The HDTV (High Definition Television) signal in 720p (Progressive) format and the HDTV signal in 1080i (Interlace) format. Is a multi-format camera capable of outputting a registration correction function. 3 is a block diagram of a prior art registration correction unit, FIG. 4 is a block diagram of a prior art format conversion unit, FIG. 5 is a block diagram of a general interpolation filter, and FIG. 6 is a prior art. FIG. 7 is a schematic diagram of the interpolation signal generation of the prior art format conversion unit.

  The registration correction function is a function for correcting a shift of each video signal. In the case of a three-plate color camera, three image sensors are used, and red component (R), green component (G), and blue component (B) light separated by a color separation optical system (generally a prism) is used for each. A color image is created by receiving light at the image sensor and combining it by signal processing in the subsequent stage. However, when the image sensor is attached to the prism, if there is a shift in the R, G, B position, the color shift of the video signal It appears. If the image sensor is attached with high accuracy, the color shift can be reduced, but this requires advanced technology. Therefore, if there is a registration correction function, even if there is a slight shift in the mounting position of the image sensor, it can be corrected so that no color shift occurs in the video signal.

  In FIG. 2, which is a block diagram of a television camera having a conventional TV format conversion function, the image sensor 3 photoelectrically converts light input from the lens 2 and registers the video signal converted into an electrical signal. Output to the correction unit 4. The registration correction unit 4 corrects the color shift of the video signal and outputs it to the format conversion unit 5. The format conversion unit 5 converts the video signal into a television format according to the camera settings, and outputs it to the video signal processing unit 6. Thereafter, a video signal in a predetermined television format is output through the video signal processing unit 6 and the video signal output unit 7.

Here, with respect to the registration correction unit 4 and the format conversion unit 5, FIG. 3 which is a block diagram of a conventional registration correction unit, FIG. 4 which is a block diagram of a conventional format conversion unit, and a general interpolation filter Details will be described with reference to FIG. 5 which is a block diagram. As shown in FIG. 3, the registration correction unit 4 is divided into a vertical processing circuit 401 and a horizontal processing circuit 403, each of which is an interpolation filter configured as shown in FIG. By applying necessary coefficients to the coefficients 1 to n, an interpolation signal having an arbitrary phase can be generated from the discrete signals sampled at a predetermined period. (See Patent Document 1)
In this way, an interpolation signal having a phase shift is generated for each of the RGB channels, and the shift amount due to the bonding of the image sensor is corrected by independently adjusting the shift amount for each channel. For example, in the case of registration correction in which the phase can be adjusted in units of 1/5 of one pixel, as shown in FIG. 6, in a video signal sampled at a period T, the period T is divided into 5 and each is divided into 0. .., 2, 0.4, 0.6, and 0.8, the coefficients for generating the interpolated signals of four phases are obtained and applied to the circuit of FIG. Registration correction for correcting deviation in units of 1/5 pixel is possible.

  At this time, n coefficients corresponding to the number of taps of the filter are required for each type of phase to be interpolated, for the coefficients given to the interpolation filter of FIG. In the case of FIG. 6, since there are five types of phases of 0, 0.2, 0.4, 0.6, and 0.8, 5 × n coefficients are required. Since there are two types, vertical and horizontal, if there are five types of phases for both vertical and horizontal, the coefficient is 10 × n.

  As shown in FIG. 4, the format conversion unit 5 is divided into a vertical processing circuit 501 and a horizontal processing circuit 503, each of which is an interpolation filter configured as shown in FIG. The format conversion is also realized by generating an interpolated signal having a phase necessary for the post-conversion format from the discrete signal sampled at a predetermined cycle. For example, in the case of format conversion in which the number of pixels is multiplied by 1.5, as shown in FIG. 7, in a video signal sampled with a period T, 0.3333, 0. A coefficient for generating an interpolated signal having two phases 6667 is obtained and applied to the circuit of FIG. 5, so that a video signal having a format having 1.5 times as many pixels as the video signal sampled at the period T is obtained. Format conversion to be converted is possible.

At this time, n coefficients corresponding to the number of taps of the filter are required for each type of phase to be interpolated, for the coefficients given to the interpolation filter of FIG. In the case of FIG. 7, since there are three types of phases of 0, 0.3333, and 0.6667, 3 × n coefficients are required. Since there are two types, vertical and horizontal, if there are three types of phases for both vertical and horizontal, the coefficient is 6 × n.
JP2003-134524A

  The above-described prior art has dedicated circuits for registration correction and format conversion, but the inside of each is almost the same circuit as described above. That is, the registration correction circuit and the format conversion circuit are both interpolation filters as shown in FIG. The number of filter taps varies depending on the required specifications, but the configuration is the same. In addition, since each interpolation filter is dedicated, the filter coefficient is also increased.

  Therefore, an object of the present invention is to devise the configuration and coefficients of the interpolation filter, to combine the registration correction circuit and the format conversion circuit into one, as shown in FIG. The purpose is to realize format conversion.

  In order to solve the above-described problem, the present invention provides a single circuit that performs registration correction of video signals and TV format conversion for outputting video signals of a plurality of TV formats in a solid-state imaging device using a solid-state imaging device. A solid-state imaging device is provided that is configured to be realized by the above.

  In the above solid-state imaging device, there is provided a signal processing circuit characterized in that an interpolation phase required for format conversion is included in an interpolation phase required for registration correction at the time of television format conversion output.

  Therefore, in the present invention, the number of taps of the vertical interpolation filter of the registration correction circuit and the vertical interpolation filter of the format conversion circuit are made common, and the number of taps of the horizontal interpolation filter of the registration correction circuit and the horizontal interpolation filter of the format conversion circuit are made common. In the vertical interpolation filter, the division number of one pixel of the vertical registration correction is set so that the n interpolation phases necessary for the vertical registration correction include the interpolation phase necessary for vertical format conversion. In a horizontal interpolation filter, division of one pixel for horizontal registration correction is performed so that an interpolation phase necessary for performing horizontal format conversion is included in m interpolation phases necessary for horizontal registration correction. By determining the number, the vertical of the registration correction circuit The vertical interpolation filter of the registration correction circuit and the horizontal interpolation filter of the format conversion circuit are common, and the coefficient of the vertical interpolation filter of the registration correction circuit and the vertical of the format conversion circuit are the same. The interpolation filter coefficient is the same, the horizontal interpolation filter coefficient of the registration correction circuit and the horizontal interpolation filter coefficient of the format conversion circuit are the same, and at the time of TV format conversion output, registration correction and format conversion are performed at the same time. At the time of non-conversion output, only registration correction is performed.

  As described above, according to the present invention, registration correction and format conversion can be realized with a small circuit configuration at the time of TV format conversion output, and only registration correction can be performed at the time of TV format non-conversion output. .

  One embodiment of the present invention will be described with reference to FIGS. 8 to 15 and Table 1 to Table 3 in FIG. FIG. 8 is a detailed block diagram of the registration correction and format conversion unit of one embodiment of the present invention, FIG. 9 is a schematic diagram of the operation of the vertical interpolation filter in one embodiment of the present invention, and FIG. 11 is a schematic diagram of vertical format conversion in one embodiment, FIG. 11 is a timing chart of a vertical processing unit of one embodiment of the present invention, and FIG. 12 is a timing chart of a horizontal processing unit of one embodiment of the present invention. 13 is a schematic diagram of the operation of the vertical interpolation filter in one embodiment of the present invention, FIG. 14 is a schematic diagram of the operation of the horizontal interpolation filter in one embodiment of the present invention, and FIG. FIG. 16 is a schematic diagram of the operation of the horizontal interpolation filter in one embodiment. Table 1 in FIG. 16 shows vertical interpolation coefficients in one embodiment of the present invention, and Table 2 in FIG. 17 shows horizontal interpolation coefficients in one embodiment of the present invention. Table 3 in FIG. 18 is an embodiment of the present invention. Which is a vertical coefficient set count value.

  As shown in FIG. 8, when the number of taps of the vertical interpolation filter is 4, the number of taps of the horizontal interpolation filter is 6, and the video in 720p format (1280 × 720 pixels) is converted into the video in 1080i format (1920 × 1080 pixels). The case where the vertical and horizontal registration corrections are simultaneously performed will be described.

  First, an interpolation phase necessary for format conversion and an interpolation phase necessary for registration correction are considered. The conversion ratio of the vertical format conversion is 720: 1080 = 2: 3. This indicates that 2 pixels in the pre-conversion format become 3 pixels after conversion (frame conversion). This is illustrated in FIG. 9. When converting from the 720p format to the 1080i format, assuming that the pixel pitch before conversion is 1, 0 phase (same phase as before conversion), 0.3333 phase, 0.6667 phase These three phases are required.

Here, the 720p format before conversion is a sequential scanning method, and the 1080i format after conversion is an interlaced scanning method. Therefore, the interpolation phase required for the ODD field and the EVEN field is different. (Fig. 9)
Next, consider the interpolation phase required for vertical registration correction. When only format conversion is considered, the number of interpolation phases is three as described above. However, for the registration correction, a larger number of interpolation phases are required for finer correction. For this reason, the number of interpolation phases necessary for registration correction is set to n times the number of interpolation phases necessary for format conversion (n is a positive number of 2 or more). In this way, since the interpolation phase is determined by further dividing the interpolation phase required for format conversion into n, the interpolation phase required for registration correction includes the interpolation phase required for format conversion. It also leads to simplification.

  In this embodiment, n = 3 and the number of interpolation phases for registration correction is nine. The interpolation phase in this case is shown in FIG. In FIG. 13, in addition to the interpolation phase necessary for format conversion, the phase indicated by the dotted line is the interpolation phase for registration correction. This indicates that 1/9 pixel step registration correction can be performed with respect to the pixel pitch before conversion.

  The conversion ratio of horizontal format conversion is 1280: 1920 = 2: 3. This indicates that 2 pixels in the pre-conversion format become 3 pixels after conversion. This is shown in FIG. 14, and when converting from the 720p format to the 1080i format, if the pixel pitch before conversion is 1, 0 phase (same phase as before conversion), 0.3333 phase, 0.6667 phase These three phases are required.

  Next, consider the interpolation phase required for horizontal registration correction. When only format conversion is considered, the number of interpolation phases is three as described above. However, for the registration correction, a larger number of interpolation phases are required for finer correction. For this reason, the number of interpolation phases necessary for registration correction is set to n times the number of interpolation phases necessary for format conversion (n is a positive number of 2 or more). In this way, since the interpolation phase is determined by further dividing the interpolation phase required for format conversion into n, the interpolation phase required for registration correction includes the interpolation phase required for format conversion. It also leads to simplification.

  In this embodiment, n = 3 and the number of interpolation phases for registration correction is nine. The interpolation phase in this case is shown in FIG. In FIG. 15, in addition to the interpolation phase necessary for format conversion, the phase indicated by the dotted line is the interpolation phase for registration correction. This indicates that 1/9 pixel step registration correction can be performed with respect to the pixel pitch before conversion.

  A vertical coefficient and a horizontal coefficient to be applied to the interpolation filter of FIG. 8 are determined from the format conversion interpolation phase and registration correction interpolation phase determined as described above.

  The coefficients of the vertical interpolation filter required in this embodiment are shown in Table 1 of FIG. 16, and the coefficients of the horizontal interpolation filter are shown in Table 2 of FIG. Although the specific value of the coefficient is not shown in this embodiment, it can be calculated by a general method using a sampling function.

  Hereinafter, the operation of the circuit will be described. First, the CPU sets a vertical registration correction amount (interpolation phase) in the vertical coefficient table. The vertical coefficient table selects a coefficient set from the set correction amount and the vertical coefficient count value from the coefficient set counter, and sends the coefficient of the selected coefficient set to the vertical interpolation filter. Write determination is performed, and the determination result is sent to the write timing control unit.

  Here, the coefficient set is a group of vertical interpolation filter coefficients VK1 to VK4 for generating a predetermined interpolation signal, and there are as many coefficient sets as the number of types of correction amounts. Since the vertical registration correction unit is 1/9 pixels, there are nine interpolation phases and nine coefficient sets as shown in Table 1 of FIG.

  Subsequently, the CPU sets a horizontal registration correction amount (interpolation phase) in the horizontal coefficient table. The horizontal coefficient table selects a coefficient set from the set correction amount and the horizontal coefficient count value from the coefficient set counter, and sends the coefficient of the selected coefficient set to the horizontal interpolation filter.

  Here, the coefficient set is a group of horizontal interpolation filter coefficients HK1 to HK6 for generating a predetermined interpolation signal, and there are as many coefficient sets as the number of types of correction amounts. Since the correction unit of the horizontal registration correction is 1/9 pixel, there are nine types of interpolation phases, and there are nine types of coefficient sets as shown in Table 2 of FIG.

  Input signals are always input to three one-line memories, and signals for four lines are arranged on the same time. An interpolation signal is generated by the vertical interpolation filter and input to the conversion memory.

  FIG. 9 shows the spatial positions of the pixels before and after conversion, and this positional relationship is realized using a conversion memory. FIG. 10 shows the pixel positions before and after conversion including the time relationship before and after the conversion memory. FIG. 11 shows a timing chart of vertical conversion.

  In FIG. 10, the signal after the interpolation filter is the timing of a pixel in the 720p format. Writing to the conversion memory at this timing and reading at the timing of the 1080i format, conversion to the 1080i format is performed.

  The conversion per field is conversion from 720 pixels to 540 pixels which is half of 1080, and the conversion ratio is 720: 540 = 4: 3. Therefore, with respect to the interpolation filter output signal, one pixel is not written to the conversion memory for every four vertical pixels, and only three pixels are written to the conversion memory for every four vertical pixels. The pixels that are not written differ depending on the correction amount set by the CPU, and this control is performed by the vertical coefficient table and the write timing control unit. The vertical coefficient table has a coefficient set selection table as shown in Table 3 of FIG. 18. Usually, a predetermined coefficient set is selected according to the value of the coefficient set counter, and the vertical interpolation filter coefficient of the selected set is selected. However, in any correction amount, the coefficient set is set to 0 once every four times. In this case, a write stop pulse is output to the write timing control unit. In response to this, the write timing control unit stops writing to the conversion memory.

  FIG. 12 shows a timing chart of horizontal conversion.

  The video signal read from the conversion memory is input to the horizontal interpolation filter. The conversion memory stores data of 1280 effective horizontal pixels for which vertical conversion has been completed. In order to convert this into 1920 pixels in the 1080i format, the count of the read address of the conversion memory is stopped only once in 3 times, and the data of 1280 pixels is read out with 1920 clocks. As a result, the output of the conversion memory is 1920 pixels as the number of data. The stop address differs depending on the correction amount set by the CPU, and this control is performed by the read address control unit. The data read from the conversion memory is in the form of the same data being continuous once every three times by the above processing, but this is calculated with a predetermined coefficient set by the horizontal interpolation filter and converted into the 1080i format. Is output as

  The above description is a simultaneous processing of registration correction and format conversion for color shift on the entire screen for correction of parallel movement within the screen of the bonding of the solid-state image pickup device of the image pickup apparatus using the color separation optical system and three solid-state image pickup devices. As described above, simultaneous processing of registration correction and format conversion for rotational color misalignment around the screen due to in-screen rotation of the solid-state image sensor, and vertical / left-right and left-axis axial displacement of the solid-state image sensor (tilting) It can also be applied to the simultaneous processing of registration correction and format conversion for screen size color misalignment in the periphery of the screen, and the simultaneous processing of registration correction and format conversion for screen size color misalignment in the periphery of the screen due to lens color magnification aberration. Further, an image pickup apparatus using a single solid-state image pickup device having a very large number of pixels and an on-chip color filter can also be applied to simultaneous processing of color misregistration registration correction and format conversion.

  Specifically, in registration correction for rotational color misregistration, the registration correction amount is set by reversing the polarity vertically and horizontally in proportion to the distance from the center of the screen to each pixel. In this registration correction, the registration correction amount is set in proportion to the distance from the screen axis to each pixel, and the polarity is reversed on the screen axis. In registration correction, a registration correction amount having the same polarity is set in the vertical and horizontal directions in proportion to the distance from the center of the screen to each pixel.

  In the above description, the registration correction and the format conversion are simultaneously performed at the time of television format conversion output. However, at the time of the television format non-conversion output, the television format conversion operation is stopped and only the registration correction is performed.

  The above description is correction of color misregistration when the visible light is separated into red, green, and blue to create a color image. By the way, the shift due to the wavelength of the lens is much larger in near infrared light and near ultraviolet light than red, green and blue whose correction has been improved. Therefore, a night vision camera that captures visible light and near-infrared light, and a pseudo-color camera that captures green, blue, and near-ultraviolet light equivalent to the visual sensitivity of honeybees to produce a pseudocolor image, Therefore, the registration correction of the present invention can be applied.

The block diagram of the television camera of one Example of this invention Block diagram of a prior art format conversion television camera Block diagram of conventional registration correction unit Block diagram of a conventional format converter Block diagram of a general interpolation filter Schematic diagram of interpolation signal generation of the registration correction unit of the prior art Schematic diagram of interpolated signal generation of the conventional format converter Detailed block diagram of registration correction and format conversion unit of one embodiment of the present invention Schematic diagram of operation of vertical interpolation filter of one embodiment of the present invention Schematic diagram of vertical format conversion of one embodiment of the present invention Timing chart of vertical processing unit of one embodiment of the present invention Timing chart of horizontal processing unit of one embodiment of the present invention Schematic diagram of operation of vertical interpolation filter in one embodiment of the present invention (detail) The schematic diagram of the operation | movement of a horizontal interpolation filter in one Example of this invention. Schematic diagram of operation of horizontal interpolation filter in one embodiment of the present invention (detail) Table 1 Vertical interpolation coefficients of one embodiment of the present invention Table 2 Horizontal interpolation coefficients of one embodiment of the present invention Table 3 Vertical coefficient set count value of one embodiment of the present invention

Explanation of symbols

1: camera, 2: lens unit, 3: image sensor, 4: registration correction unit, 5: format conversion unit, 6: video signal output unit, 7: video signal output unit,
8: CPU

Claims (1)

A solid-state imaging device using a solid-state imaging device is characterized in that registration correction of video signals and television format conversion for outputting video signals of a plurality of television formats are realized by a single circuit. Solid-state imaging device.

Images