JP2007049566A - Imaging apparatus and video signal generation method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus which has a small circuit scale and can reduce power consumption and can generate a video signal of satisfactory image quality and of a desired image size, and a video signal generation method. <P>SOLUTION: Video signals Drb, Dgb, and Dbb are generated in imaging parts 14r, 14g, and 14b using signals of pixels subjected to pixel addition and signal of pixels not subjected to pixel addition. Differences in frequency characteristics due to execution/non-execution of pixel addition, of the video signals Drb, Dgb, and Dbb are compensated by a frequency characteristics compensation means. As regards the frequency characteristic compensation means, for example, a filter part which switches filter characteristics between signals of pixels subjected to pixel addition and those not subjected to pixel addition, to perform filter processing is provided to a camera signal processing part 15. Pixel addition is performed in imaging parts 14r, 14g, and 14b, whereby the circuit scale of the image pickup device can be reduced and the power consumption can be reduced since it is unnecessary to provide, for example, a line memory to the camera signal processing part 15 to perform pixel addition. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and a video signal generation method. Specifically, a video signal is generated using a pixel signal that has undergone pixel addition and a pixel signal that has not undergone pixel addition, and the difference in frequency characteristics due to the presence or absence of pixel addition is corrected for this video signal. By performing this process, a high-quality captured image is obtained.

  In video equipment, with the digitization of video signals, various broadcast transmission systems with different numbers of pixels in the horizontal and vertical directions are standardized. For this reason, image size conversion processing is performed to convert the number of pixels of the original image to the number of pixels corresponding to the desired broadcast transmission system so that it can be applied to devices of broadcast transmission systems with different numbers of pixels in the horizontal and vertical directions. It has been broken.

  For example, as shown in Patent Document 1, an interpolation calculation is performed to generate a video signal having the number of pixels corresponding to a desired broadcast transmission method. In this interpolation calculation, the image size conversion process is performed at an arbitrary conversion ratio by using a filter coefficient closest to the phase of the pixel to be interpolated.

Japanese Patent Laid-Open No. 10-134175

  By the way, in an imaging apparatus, usually, a video signal is basically output with the maximum number of imaging pixels (number of effective pixels). However, a recording device that records a video signal output from the imaging device on a recording medium or a display device that displays a captured image using the video signal output from the imaging device does not support the maximum number of pixels. In this case, the imaging device needs to output a video signal after performing an image size conversion process to obtain the number of pixels of a broadcast transmission method corresponding to a recording device or a display device.

  For example, a progressive scanning system video signal (hereinafter referred to as a “1080p video signal”) Dpa having a maximum imaging pixel of 1920 pixels in the horizontal direction × 1080 lines in the vertical direction, and a progressive scanning system in which the video signal is 1280 pixels in the horizontal direction × 720 lines in the vertical direction. Image signal (hereinafter referred to as “720p video signal”) Dpb, it is necessary to perform image size conversion processing for multiplying the number of pixels in the horizontal and vertical directions by 2/3.

  At this time, if the frame rate of the 1080p video signal Dpa and the 720p video signal Dpb is 60 frames / second (fps), the 1080p video signal Dpa has a sampling frequency of 148.5 MHz and the 720p video signal Dpb. The sampling frequency is 74.25 MHz.

  Here, when the pixel signal is read out from the image sensor with the maximum number of imaging pixels of “1920 × 1080” and the video signal Dpa is generated, and then converted to 2/3 times the number of pixels, before the image size conversion, It is necessary to drive at a frequency twice that of the sampling frequency of 720p, resulting in an increase in power consumption. In addition, the vertical image size conversion requires a line memory for several lines, which increases the circuit scale. Furthermore, if the image size conversion process is performed using interpolation calculation, the circuit configuration becomes complicated.

  Therefore, the present invention provides an imaging apparatus and a video signal generation method capable of generating a video signal having a desired image size with a small circuit scale, low power consumption, and good image quality.

  The imaging apparatus and the video signal generation method according to the present invention generate a video signal using a pixel signal subjected to pixel addition and a pixel signal not subjected to pixel addition, and whether or not the pixel addition is performed in the video signal. Difference in frequency characteristics due to frequency characteristic correction means, for example, a filter unit that performs filter processing by switching filter characteristics between a pixel signal that has undergone pixel addition and a pixel signal that has not undergone pixel addition, or an optical low-pass This is corrected by a filter.

  According to the present invention, the video signal is generated by the imaging unit using the pixel signal for which pixel addition has been performed and the pixel signal for which pixel addition has not been performed. The difference is corrected by the frequency characteristic correcting means.

  For this reason, when performing pixel addition in the vertical direction in image size conversion, it is not necessary to store the video signal output from the imaging means in the line memory and perform pixel addition in the vertical direction, so that the circuit scale can be reduced. Also, low power consumption can be achieved. Furthermore, since the difference in frequency characteristics due to the presence or absence of pixel addition is corrected, it is possible to prevent deterioration in image quality due to the difference in frequency characteristics, and an image with good image quality can be obtained.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows the configuration of the imaging apparatus 10. The subject light that has passed through the imaging lens 11 is reduced by the optical low-pass filter 12 at a high frequency component exceeding a reproducible limit frequency determined by the size of the imaging pixel and the interval between adjacent pixels. The subject light with the reduced high-frequency component is split into the three primary color components by the color separation prism 13 and input to the imaging units 14r, 14g, and 14b.

  The imaging units 14r, 14g, and 14b have the same structure, the imaging unit 14r generates a signal based on the red component, the imaging unit 14g generates a signal based on the green component, and the imaging unit 14b generates a signal based on the blue component. To do. The imaging unit 14g also moves the pixel position by 1/2 pixel in the horizontal direction with respect to the imaging unit 14r and the imaging unit 14b in order to improve the resolution.

  FIG. 2 shows the configuration of the imaging unit. Note that only the imaging unit 14r that generates a signal based on the red component as the imaging unit will be described, and description of the imaging units 14g and 14b will be omitted.

  The pixel unit 141 performs photoelectric conversion and generates a signal corresponding to the input light for each pixel. Further, a video signal Sra of a frame image is generated using a signal for each pixel and supplied to the A / D conversion unit 142. The A / D conversion unit 142 converts the video signal Sra supplied from the pixel unit 141 into a digital video signal Dra and supplies the digital video signal Dra to the addition processing unit 143.

  The addition processing unit 143 performs pixel addition in the horizontal direction and the vertical direction based on the addition control signal CTb supplied from the drive signal generation unit 144 described later, and outputs the video signal Dra having the maximum number of imaging pixels to the imaging device 10. Image size conversion is performed so that the video signal Drb has the number of pixels corresponding to the broadcast transmission method of the connected device.

  Here, when the image size is converted, if the pixel addition or thinning is simply performed, the spatial phase of the pixels may not match. FIG. 3 shows an image size conversion operation by thinning processing. 3A shows a pixel array before image size conversion, FIG. 3B shows a pixel position to be thinned out, and FIG. 3C shows a pixel array after image size conversion. For simplicity of explanation, FIG. 3 shows only 12 pixels in the horizontal direction and 9 pixels in the vertical direction before image size conversion.

  In the thinning process, for example, when 2 pixels are output, the next pixel is not output, and the hatched pixels in FIG. 3B are thinned out to increase the number of pixels by 2/3 as shown in FIG. 3C. . In this case, since the thinned-out portion corresponds to two pixels before the thinning process, in the image after the thinning process, the thinning process separated by two pixels from the pixel before the thinning process separated by one pixel. The previous pixels are used alternately, and the spatial phase is not matched. In FIG. 3C, the positions of the pixel intervals corresponding to two pixels are indicated by double lines.

  FIG. 4 shows an image size conversion operation by pixel addition. 4A shows a pixel array before image size conversion, FIG. 4B shows a pixel position to be added, and FIG. 4C shows a pixel array after image size conversion. For simplicity, FIG. 4 also shows only 12 pixels in the horizontal direction and 9 pixels in the vertical direction before image size conversion, as in FIG.

  In the addition process, for example, two pixels in the horizontal direction and two pixels in the vertical direction are added to form one pixel after image size conversion. Further, the pixel position to be added is moved by one pixel in the horizontal direction, and two pixels in the horizontal direction and two pixels in the vertical direction are added to obtain one pixel after image size conversion. Also in the vertical direction, the pixel position to be added is moved by one pixel in the vertical direction, and two pixels in the horizontal direction and two pixels in the vertical direction are added to obtain one pixel after image size conversion. Do. In this way, processing for generating two horizontal pixels from three horizontal pixels and processing for generating two vertical pixels from three vertical pixels are performed. Furthermore, if such processing is performed in units of three pixels in the horizontal direction and the vertical direction, the number of pixels can be increased to 2/3 as shown in FIG. 4C. In FIG. 4B, the arrow display indicated by a broken line indicates four pixels to be added.

  When the addition processing is performed in this way, for example, the pixel obtained by performing the pixel addition of the first pixel and the second pixel in the horizontal direction and the pixel addition of the second pixel and the third pixel are obtained. The pixel interval with the pixel corresponds to one pixel. The pixel interval between the pixel obtained by adding the second and third pixels in the horizontal direction and the pixel obtained by adding the fourth and fifth pixels is equivalent to two pixels. As a result, the pixel interval is not constant and the spatial phase is not matched. Further, the spatial phase is not matched in the vertical direction as in the horizontal direction. In FIG. 4C, the positions of pixel intervals corresponding to two pixels are indicated by double lines.

  Therefore, when performing pixel addition, the addition processing unit 143 performs pixel addition by adjusting the phase so that the intervals between the pixel centers after the image size conversion processing are equal. Here, the pixel center refers to the spatial center of the pixel. In FIG. 5, the pixel center is indicated by a circle, and the pixel center of one pixel Pa is the center position of the pixel Pa as shown in FIG. 5A. As shown in FIG. 5B, the pixel center when two adjacent pixels Pa and Pb (Pa, Pc) are added at an equal ratio is the center position of the two pixels Pa and Pb (Pa, Pc) to be added. Become. Further, as shown in FIG. 5C, the pixel center when adding four adjacent pixels Pa, Pb, Pc, and Pd at an equal ratio is the center position of the four pixels Pa, Pb, Pc, and Pd to be added. Become.

  Furthermore, the pixel center can be set to a desired position by adjusting the ratio of the pixels to be added. For example, the pixel center when the ratio of the pixel Pa is larger than the pixel Pb (Pc) is a position moved in the direction of the pixel Pa from the center position of the two pixels Pa and Pb (Pc) as shown in FIG. 5D. Become. Further, as shown in FIG. 5E, the pixel center when the ratio of the pixel Pa is smaller than that of the pixel Pb (Pc) is from the center position of the two pixels Pa and Pb (Pc) in the direction of the pixel Pb (Pc). It becomes the moved position.

  The drive signal generation unit 144 in FIG. 2 generates a drive signal CTa for driving the pixel unit 141 and the A / D conversion unit 142 based on a control signal CTm supplied from the control unit 20 described later. In addition, the drive signal generation unit 144 performs pixel addition in the addition processing unit 143 so that the distance between the pixel centers is uniform, and adds control for converting the video signal having the maximum number of pixels into a video signal having a desired image size. The signal CTb is generated based on the pixel addition setting signal CTs supplied from the control unit 20.

  Similarly to the imaging unit 14r, the imaging units 14g and 14b also generate video signals Dgb and Dbb having a resolution corresponding to the broadcast transmission method of the equipment connected to the imaging device 10.

  The camera signal processing unit 15 in FIG. 1 uses the video signals Drb, Dgb, and Dbb generated by the imaging units 14r, 14g, and 14b to perform correction processing, filter processing, process processing, and the like supplied from the control unit 20. Based on the signal CTm, a video output signal DVout having a format corresponding to the connected device is generated.

  FIG. 6 shows the configuration of the camera signal processing unit 15. The preprocessing unit 151 performs defect correction processing, white balance adjustment processing, and the like on the video signals Drb, Dgb, and Dbb, and the processed video signals Drc, Dgc, and Dbc are converted into a horizontal filter unit 152 that is a frequency characteristic correction unit. To supply. Note that the frequency characteristic correcting means includes a horizontal filter unit 152 and a vertical filter unit 153.

  FIG. 7 shows the configuration of the horizontal filter unit 152. The video signals Drc, Dgc, Dbc are supplied to the filter 521 and the filter 522. The filter 521 is composed of a high-pass filter or a band-pass filter, extracts a high-frequency component by a filter process using a signal of a pixel located on the left and right of the pixel subjected to pixel addition, and outputs a signal Hr, Hg and Hb are supplied to the filter output selection unit 523. The filter 522 is formed of a low-pass filter, performs a filter process for reducing a component that is a band of the horizontal sampling frequency, and outputs the video signals Drd, Dgd, and Dbd in which the high-frequency components are attenuated to the filter output selection unit 523. Supply.

  Based on the pixel identification signal CTq indicating whether or not the pixel has been subjected to pixel addition, the filter output selection unit 523 uses the signals Hr, Hg, and Hb output from the filter 521 for the pixel to which pixel addition has been performed. The signals are selected and supplied to the variable ratio mixing unit 524 as selection signals DSr, DSg, and DSb. In addition, for the pixels that have not been subjected to pixel addition, the video signals Drd, Dgd, and Dbd output from the filter 522 are selected and supplied to the variable ratio mixing unit 524 as the selection signals DSr, DSg, and DSb.

  The variable ratio mixing unit 524 sets the mixing ratio of the input video signals Drc, Dgc, Dbc and the selection signals DSr, DSg, DSb supplied from the filter output selection unit 523 to whether or not pixel addition is performed based on the pixel identification signal CTq. The video signals Dre, Dge, and Dbe are generated by performing adjustment and adding according to the correction, and correcting the difference in the frequency characteristics in the horizontal direction depending on the presence or absence of pixel addition.

  The vertical filter unit 153 is also configured in the same manner as the horizontal filter unit 152, and selects a signal after filtering depending on whether or not the pixel has been subjected to pixel addition. Further, the video signal Drf is obtained by adjusting the mixing ratio between the selected signal and the video signals Dre, Dge, Dbe output from the horizontal filter unit 152 and correcting the difference in the vertical frequency characteristics depending on the presence or absence of pixel addition. , Dgf, Dbf are generated. Note that since the vertical filter unit 153 performs vertical filter processing, extraction of high-frequency components by filter processing using signals of pixels located above and below the pixel subjected to pixel addition, and vertical sampling A process of reducing the high frequency component that is the frequency band is performed.

  The process unit 154 in FIG. 6 performs process processing such as gamma correction processing and knee processing on the video signals Drf, Dgf, and Dbf output from the vertical filter unit 153, and the processed video signals Drg, Dgg, Dbg is supplied to the video signal output unit 155. The video signal output unit 155 uses the video signals Drg, Dgg, and Dbg supplied from the process unit 154 as video output signals corresponding to a signal recording device and a display device, for example, video output of components and composites, or luminance signals and color difference signals. Output as signal DVout.

  A user interface unit 21 is connected to the control unit 20 shown in FIG. The user interface unit 21 is operated when setting the imaging device 10 or switching the operation, and generates an operation signal US corresponding to the user operation and supplies the operation signal US to the control unit 20. For example, when an operation for designating an image size corresponding to a device to be connected is performed, an operation signal US indicating the designated image size is supplied to the control unit 20. The designation of the image size according to the connected device may be performed by communicating with the connected device and obtaining information indicating the image size from this device.

  The control unit 20 generates the control signal CTm according to the operation signal US and supplies the control signal CTm to the imaging units 14r, 14g, and 14b and the camera signal processing unit 15, thereby causing the imaging device 10 to operate according to the user operation.

  In addition, the control unit 20 determines which pixel addition operation is to be performed from the maximum number of imaging pixels and the image size of the designated video output signal DVout, and outputs a pixel addition setting signal CTs for setting the pixel addition operation. Generated and supplied to the imaging units 14r, 14g, and 14b.

  Further, a pixel identification signal CTq (not shown) indicating whether the pixel is added or not is sent to the control unit 20 or the drive signal generation unit 144 of the imaging unit. It is generated and supplied to the camera signal processing unit 15.

  Next, an image size conversion operation of the image pickup apparatus, for example, a case where the video output signal DVout in which the number of pixels in the horizontal direction and the vertical direction is 2/3 times the maximum number of image pickup pixels will be described. For the sake of simplicity, in the following drawings, only 12 pixels in the horizontal direction and 9 pixels in the vertical direction before image size conversion are shown.

  The imaging units 14r, 14g, and 14b increase the number of pixels by 2/3 by performing 2-pixel addition every other pixel as shown in FIG. 8 in the horizontal and vertical directions. 8A shows the pixel arrangement of the pixel unit 141, FIG. 8B shows the pixel center when pixel addition is performed, and FIG. 8C shows the image size of the video signal output from the imaging unit.

  For example, by using the pixels on the first line and the second line of the pixel unit 141, the video signal on the first line after the image size conversion is generated. Here, using the first two pixels of the first line and the first two pixels of the second line in the pixel unit 141, four pixels are added at an equal ratio, and the first pixel of the first line after the image size conversion is added. Generate a signal. That is, after adding the signal levels of the four pixels, the addition result is divided by the number of added pixels to generate a signal for the first pixel. Next, using the third pixel on the first line and the third pixel on the second line in the pixel unit 141, two pixels are added at an equal ratio to generate a signal for the second pixel after image size conversion. . Similarly, the signal for the first line after the image size conversion can be generated by repeating the 4-pixel addition and 2-pixel addition.

  Thereafter, a video signal of the second line after the image size conversion is generated using the pixels of the third line of the pixel unit 141. Here, using the first two pixels of the third line in the pixel unit 141, two pixels are added at an equal ratio to generate a signal of the first pixel of the second line after the image size conversion. Next, the third pixel on the third line in the pixel unit 141 is used as the signal of the second pixel after image size conversion. Similarly, the second line signal after the image size conversion can be generated by repeating the addition of two pixels every other pixel in the signal after the image size conversion.

Hereinafter, by repeating the above-described processing, it is possible to obtain video signals Drb, Dgb, and Dbb after image size conversion in which the number of pixels is 2/3 times the maximum number of imaging pixels. Further, since the pixel center when the pixel addition is performed is the center position of the added pixel, the interval between the pixel centers in the video signal after the image size conversion is between each pixel as is apparent from FIG. 8B. Will be equal.
The filter output selection unit 523 of the horizontal filter unit 152 selects a filter output depending on whether or not pixel addition is performed in the horizontal direction. FIG. 9 is a diagram for explaining a filter output selection operation in the filter output selection unit 523. FIG. 9A is a video signal Drc, Dgc, Dbc, and FIG. 9B is a selection signal DSr, output from the filter output selection unit 523. DSg and DSb are shown.

  Here, since the first pixel in the first line of the video signal is the pixel “1 + 2” in which pixel addition is performed in the horizontal direction, the signals Hr, Hg, and Hb output from the filter 521 are selected. Since the second pixel is the pixel “3” that has not been subjected to pixel addition in the horizontal direction, the video signals Drd, Dgd, and Dbd output from the filter 522 are selected. Similarly, the output of the filter 521 is selected when the pixel is subjected to pixel addition in the horizontal direction, and the output of the filter 522 is selected when the pixel is not subjected to pixel addition in the horizontal direction.

  The variable ratio mixing unit 524 is a pixel in which pixel addition is performed in the horizontal direction with respect to the supplied video signals Drc, Dgc, Dbc, and the high frequency output from the video signals Drc, Dgc, Dbc and the filter 521. The mixing ratio of the signals Hr, Hg, and Hb as components is adjusted and added. If the pixel is not subjected to pixel addition in the horizontal direction, the mixing ratio of the video signals Drc, Dgc, Dbc is “0”, and the mixing ratio of the video signals Drd, Dgd, Dbd output from the filter 521 is “ 1 ”is added. That is, instead of the supplied video signals Drc, Dgc, and Dbc, the filtered video signals Drd, Dgd, and Dbd are output. Further, when the video signals Drc, Dgc, Dbc and the signals Hr, Hg, Hb are added, the frequency characteristics of the video signals Dre, Dge, Dbe output from the variable ratio mixing unit 524 are constant regardless of the presence or absence of pixel addition. Adjust the mixing ratio so that

  As described above, when the filtering process is performed by the horizontal filter unit 152, the horizontal frequency characteristics of the video signals Dre, Dge, and Dbe output from the horizontal filter unit 152 are constant regardless of the presence or absence of pixel addition. . Therefore, in the horizontal direction, it is possible to prevent a boundary portion between a pixel on which pixel addition has been performed and a pixel on which pixel addition has not been performed from becoming prominent due to a difference in frequency characteristics.

  Similarly to the filter output selection unit 523 of the horizontal filter unit 152, the filter output selection unit of the vertical filter unit 153 performs high-frequency component signal and vertical pixel addition for pixels in which pixel addition is performed in the vertical direction. For pixels that do not, a video signal with a reduced high-frequency component is selected. Further, the variable ratio mixing unit of the vertical filter unit 153 adjusts and adds the mixing ratio of the video signals Dre, Dge, Dbe and the high-frequency component signal in the pixel where the pixel addition is performed in the vertical direction. Video signals Drf, Dgf, and Dbf are generated. In addition, in a pixel that has not been subjected to pixel addition in the vertical direction, addition is performed by setting the mixing ratio of the video signals Dre, Dge, and Dbe to “0” and the mixing ratio of the video signal in which the high frequency component is reduced to “1”. Video signals Drf, Dgf, and Dbf are generated. Further, when the video signals Dre, Dge, Dbe and the high-frequency component signal are added, the frequency characteristics of the video signals Drf, Dgf, Dbf output from the variable ratio mixing unit are constant regardless of the presence or absence of pixel addition. Adjust the mixing ratio.

  As described above, when the filter processing is performed by the vertical filter unit 153, the video signals Drf, Dgf, and Dbf output from the vertical filter unit 153 have a constant frequency characteristic in the vertical direction regardless of whether or not the pixels are added. . Therefore, in the vertical direction, it is possible to prevent a boundary portion between a pixel on which pixel addition has been performed and a pixel on which pixel addition has not been performed from becoming prominent due to a difference in frequency characteristics.

  Further, for the process unit 154, video signals Drf, Dgf, and Dbf in which the horizontal and vertical boundary portions between the pixel that has undergone pixel addition and the pixel that has not undergone pixel addition are inconspicuous. Is supplied. For this reason, an image based on the video output signal DVout output from the video signal output unit 155 has, for example, a pixel at an edge portion due to a difference in frequency characteristics between a pixel on which pixel addition has been performed and a pixel on which pixel addition has not been performed. The boundary is not displayed as jagged, and an image with good image quality is obtained.

  By the way, when the pixel addition shown in FIG. 8 is repeated for each frame, the boundary between the pixel on which pixel addition has been performed and the pixel on which pixel addition has not been performed becomes a fixed position on the screen. Therefore, if the filter characteristics in the horizontal filter unit 152 and the vertical filter unit 153, the mixing ratio in the variable ratio mixing unit, and the like are not optimized, the boundary between the pixel that has undergone pixel addition and the pixel that has not undergone pixel addition is noticeable. It becomes easy. For this reason, the control unit 20 or the drive signal generation unit 144 is used as a frequency correction unit, and the addition control is performed so that the phase of the pixel that has undergone pixel addition and the pixel that has not undergone pixel addition vary for each frame. A signal CTb is generated and supplied to the addition processing unit 143. As described above, if the phase of the pixel that has been subjected to pixel addition and the pixel that has not been subjected to pixel addition are varied for each frame, the pixel addition and the pixel addition are performed due to the visual integration effect. It is possible to make the boundary of no pixels inconspicuous.

  FIG. 10 shows a pixel addition operation when the phase of a pixel that has undergone pixel addition and a pixel that has not undergone pixel addition are varied for each frame. 10A shows the pixel arrangement of the pixel unit 141, FIGS. 10B to 10E show the pixel addition position, and FIG. 10F shows the image size of the video signal output from the imaging unit.

  For example, in the n frame shown in FIG. 10B, 4-pixel addition is performed using two pixels in the horizontal and vertical directions as the first pixel in the first line after image size conversion, and two pixels are added in the vertical direction for the next pixel. I do. Such processing is repeated thereafter. It is assumed that the first pixel on the second line performs two-pixel addition in the horizontal direction and does not perform pixel addition on the next pixel. Thereafter, the image size conversion of n frames is performed by repeating such processing.

  In the n + 1 frame shown in FIG. 10C, pixel addition is not performed on the first pixel in the first line after image size conversion, and two pixels are added in the horizontal direction at the next pixel. Such processing is repeated thereafter. It is assumed that the first pixel on the second line adds two pixels in the vertical direction, and the next pixel does not perform pixel addition that adds four pixels. Thereafter, the image size conversion of n + 1 frame is performed by repeating such processing.

  In the n + 2 frame shown in FIG. 10D, two pixels are added in the horizontal direction at the first pixel of the first line after image size conversion, and pixel addition is not performed at the next pixel. Such processing is repeated thereafter. The first pixel on the second line adds 4 pixels, and the next pixel adds 2 pixels in the vertical direction. Thereafter, the image size conversion of n + 2 frames is performed by repeating such processing.

  In the n + 3 frame shown in FIG. 10E, two pixels are added in the vertical direction at the first pixel of the first line after image size conversion, and four pixels are added at the next pixel. Such processing is repeated thereafter. It is assumed that pixel addition is not performed for the first pixel on the second line, and two pixels are added in the horizontal direction for the next pixel. Thereafter, the image size conversion of n + 3 frames is performed by repeating such processing.

  In this way, by changing the phase of the pixel that has undergone pixel addition and the pixel that has not undergone pixel addition for each frame, the boundary between the pixel that has undergone pixel addition and the pixel that has not undergone pixel addition can be defined. It can be inconspicuous.

  Here, as shown in FIG. 10, if the phase of the pixel that has been subjected to pixel addition and the pixel that has not been subjected to pixel addition are varied, the pixel center indicated by a circle in FIG. 10 will be different for each frame. End up. For this reason, when the n frame is used as a reference, the horizontal filter unit 152 converts the n-frame into a frame in which the pixel center position is different in the horizontal direction, that is, a frame that performs pixel addition as shown in the n + 1 frame and the n + 3 frame. On the other hand, filter processing is performed so that the pixel center coincides with the pixel center of the n frame. For example, the ratio of the first pixel in the horizontal direction and the ratio of the second pixel that has undergone pixel addition is adjusted by the filter coefficient to perform pixel addition to generate a pixel whose pixel center is equal to the pixel center of the first pixel in n frames To do. Further, the pixel addition is performed by adjusting the ratio of the second pixel where the pixel addition is performed and the third pixel where the pixel addition is not performed by using a filter coefficient, and the pixel center is the second pixel of the n frame. Generate equal pixels. By performing the same processing thereafter, the horizontal position of the pixel center can be matched with n frames.

  The vertical filter unit 153 has a pixel center of n frames with respect to a frame in which the pixel center is different in the vertical direction with respect to n frames, that is, a frame in which pixel addition is performed as shown in n + 1 frame and n + 2 frame. Filter to match. For example, the pixel addition is performed by adjusting the ratio of the first pixel of the first line in the vertical direction where pixel addition is performed to the first pixel of the second line where pixel addition is not performed using a filter coefficient. Produces a pixel equal to the first pixel in the first line of n frames. Also, pixel addition is performed by adjusting the ratio of the first pixel on the second line where pixel addition is not performed to the first pixel on the third line where pixel addition is performed using a filter coefficient, and the pixel center is n frames. A pixel equal to the first pixel in the second line is generated. By performing the same processing thereafter, the vertical position of the pixel center can be matched with n frames.

  As described above, if the phase of the pixel that has been subjected to pixel addition and the pixel that has not been subjected to pixel addition are varied for each frame so that the boundary is not fixed on the screen, the horizontal filter unit 152 or the vertical filter unit By optimizing the characteristics of the filter of 153, the mixing ratio of the variable ratio mixing unit, etc., the frequency characteristics of the pixels that have been subjected to pixel addition and the pixels that have not been subjected to pixel addition are not precisely matched. It is possible to make the boundary between the pixel on which pixel addition has been performed and the pixel on which pixel addition has not been performed inconspicuous.

  Furthermore, the pixel addition and the pixel addition are performed by changing the phase of the pixel on which the pixel addition has been performed and the pixel on which the pixel addition has not been performed for each frame and matching the pixel centers. When the boundary with a non-pixel can be made reasonably inconspicuous, the frequency characteristics can be corrected by adding a high-frequency component to the pixel that has undergone pixel addition, or the pixel that has not undergone pixel addition. Of course, it is not necessary to perform a process of correcting the frequency characteristics by selecting a video signal in which a high frequency component is attenuated.

  In the above-described embodiment, the image size conversion in the horizontal direction and the vertical direction is performed by the imaging units 14r, 14g, and 14b. However, the image size conversion in one direction is performed by the imaging units 14r, 14g, and 14b. The image signal conversion in the other direction may be performed by the camera signal processing unit 15. As described above, if the image conversion is performed by the camera signal processing unit 15, pixel addition that cannot be performed by the imaging units 14r, 14g, and 14b is performed by the camera signal processing unit 15. Image size conversion to size can be performed.

  For example, when the imaging units 14r, 14g, and 14b set the pixels to be added by controlling the readout of the accumulated charges, if the charges of the pixels are read and added, as shown in FIG. Cannot be added repeatedly. In other words, after performing the addition by reading the charges of the first pixel and the second pixel, the second pixel and the third pixel cannot be added. However, if the camera signal processing unit 15 is provided with a memory, the second pixel and the third pixel are added after the first pixel and the second pixel are added by repeatedly reading the signal stored in the memory. The process of adding the pixels can be easily performed.

  Further, when image size conversion in one direction is performed by the imaging units 14r, 14g, and 14b and image size conversion in the other direction is performed by the camera signal processing unit 15, image size conversion in the vertical direction is performed by the imaging units 14r, 14g, and 14b. If the camera signal processing unit 15 performs horizontal image size conversion, it is not necessary to provide a line memory in the camera signal processing unit 15, so that the circuit scale and power consumption can be greatly reduced.

  Furthermore, in the above-described embodiment, the difference in frequency characteristics depending on the presence or absence of the pixel subjected to pixel addition is corrected by the horizontal filter unit 152, the vertical filter unit 153, and the control unit 20, but optical frequency correction means is optical. The low-pass filter 12 may be used, and the optical low-pass filter 12 may correct the difference in frequency characteristics. In this case, the optical low-pass filter 12 limits the frequency band according to the pixel on which the pixel addition has been performed, and attenuates the high frequency component of the pixel on which the pixel addition has not been performed, so that the pixel on which the pixel addition has been performed is attenuated. The difference in frequency characteristics due to the presence or absence is corrected. In this way, although the high-frequency component is lost in the video signal after the image size conversion, the boundary between the signal portion where the pixel addition is performed and the signal portion where the pixel addition is not performed becomes noticeable. Can be prevented.

It is a figure which shows the structure of an imaging device. It is a figure which shows the structure of an imaging part. It is a figure for demonstrating the image size conversion operation | movement by a thinning process. It is a figure for demonstrating the image size conversion operation | movement by pixel addition. It is a figure for demonstrating the pixel center. It is a figure which shows the structure of a camera signal processing part. It is a figure which shows the structure of a horizontal filter part. It is a figure for demonstrating pixel addition operation | movement. It is a figure which shows operation | movement of a filter output selection part. It is a figure for demonstrating operation | movement in the case of changing the phase of the pixel in which pixel addition was performed, and the pixel in which pixel addition was not performed.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Imaging device, 11 ... Imaging lens, 12 ... Optical low-pass filter, 13 ... Color separation prism, 14r, 14g, 14b ... Imaging part, 15 ... Camera signal processing part, DESCRIPTION OF SYMBOLS 20 ... Control part, 21 ... User interface part, 141 ... Pixel part, 142 ... A / D conversion part, 143 ... Addition processing part, 144 ... Drive signal generation part, 151 ... Pre-processing unit, 152 ... Horizontal filter unit, 153 ... Vertical filter unit, 154 ... Process unit, 155 ... Video signal output unit, 521, 522 ... Filter, 523 ... Filter output selection unit, 524 ... Variable ratio mixing unit

Claims (7)

Imaging means for generating a video signal using a pixel signal for which pixel addition has been performed and a pixel signal for which pixel addition has not been performed;
An image pickup apparatus comprising: frequency characteristic correction means for correcting a difference in frequency characteristic depending on presence or absence of the pixel addition in the video signal. The frequency characteristic correcting unit performs filter processing by switching filter characteristics between a pixel signal subjected to the pixel addition and a pixel signal not subjected to the pixel addition, and corrects the difference in the frequency characteristics. The imaging apparatus according to claim 1. The frequency characteristic correction unit extracts a high frequency component from a signal of a pixel that has not been subjected to pixel addition, and uses the extracted high frequency component to obtain a frequency characteristic of the pixel signal that has undergone the pixel addition. The imaging apparatus according to claim 2, wherein correction is performed. The imaging apparatus according to claim 3, wherein the frequency characteristic correcting unit performs low-pass filter processing on a signal of a pixel that has not been subjected to pixel addition. The imaging apparatus according to claim 1, wherein an optical low-pass filter is used as the frequency characteristic correcting unit. The imaging apparatus according to claim 1, wherein the imaging unit varies a phase of a pixel that performs the pixel addition and a pixel that does not perform the pixel addition for each frame. An imaging process for generating a video signal using a pixel signal for which pixel addition has been performed and a pixel signal for which pixel addition has not been performed;
And a frequency characteristic correcting step of correcting a difference in frequency characteristics depending on the presence or absence of the pixel addition in the video signal.

Images