JP4312121B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus, and more particularly to an image processing apparatus that is applied to, for example, a digital camera and generates image data of one screen based on a plurality of scene images captured in a plurality of imaging regions.

An example of a conventional image processing apparatus of this type is disclosed in Patent Document 1. According to this prior art, the image sensor has a right channel and a left channel. Image information generated in the right region of the imaging surface is output from the right channel, and image information generated in the left region of the imaging surface is output from the left channel. The output image information is subjected to black level correction processing for each channel. The image information is also subjected to gain correction processing so that the gain difference between channels is eliminated. Thereby, it is possible to prevent the boundary line between the right region and the left region from appearing in the reproduced image.
JP 2002-252808 A [H04N 5/335, 5/16, G06T 1/00, H01L 27/148]

    However, the conventional technique requires high-accuracy black level correction and gain adjustment, and there is a problem that signal processing becomes complicated.

    SUMMARY OF THE INVENTION Therefore, a main object of the present invention is to provide an image processing apparatus that can make an image boundary line inconspicuous with simple processing.

  The digital camera according to the invention of claim 1 is generated by an imaging means having an imaging surface on which a plurality of partial imaging areas are formed and a plurality of output paths respectively assigned to the plurality of partial imaging areas, and a plurality of partial imaging areas. Driving means for outputting a plurality of partial image signals from a plurality of output paths, and creating a one-screen image signal representing a scene image based on the plurality of partial image signals respectively output from the plurality of output paths And a compression means for compressing the image signal created by the creation means for each pixel block of a predetermined size in accordance with a compression method in which a high-frequency component is missing, and two adjacent partial imaging areas among a plurality of partial imaging areas are mutually A part of the plurality of pixel blocks allocated to the image signal by the compression means is a boundary line between two adjacent partial imaging regions. Ingredients.

  The imaging means has an imaging surface and a plurality of output paths. A plurality of partial imaging areas are formed on the imaging surface, and a plurality of output paths are assigned to the plurality of partial imaging areas. The driving unit outputs a plurality of partial image signals respectively generated in the plurality of partial imaging regions from a plurality of output paths. The creating means creates a one-screen image signal representing a scene image based on a plurality of partial image signals respectively output from the plurality of output paths. Further, the compression means compresses the created image signal for each pixel block of a predetermined size according to a compression method in which high frequency components are lost.

  Here, two adjacent partial imaging regions out of the plurality of partial imaging regions are in contact with each other. Further, some of the plurality of pixel blocks assigned to the image signal by the compression unit straddle the boundary line between two adjacent partial imaging regions. As a result, the edge component representing the boundary line of the partial imaging region is removed by the compression operation of the compression unit.

A digital camera according to the present invention of claim 2 depends on claim 1, the boundary line extending in a specific direction, the number of pixels definitive in a direction perpendicular to the specific direction of the image signal generated by the generating means, the specific pixel block divisible by the first number is the number of pixels definitive in a direction orthogonal to the direction, and N times the first number: the number of pixels can not be divided in the second number is (N 2 or more integer). As a result, the boundary line straddles the pixel block.

  The digital camera according to the invention of claim 3 is dependent on claim 2, and the specific direction is a vertical direction. In this case, the boundary line extends in the vertical direction, and a plurality of pixel blocks straddling the boundary line also extend in the vertical direction.

  The digital camera according to the invention of claim 4 is dependent on any one of claims 1 to 3, and the compression processing of the compression means follows the JPEG system. High-frequency components are lost due to lossy compression such as JPEG compression.

  A digital camera according to a fifth aspect of the invention is dependent on any one of the first to fourth aspects, wherein the plurality of partial image signals are signals in which each pixel has any one color information of a plurality of colors, and are created by a creation means. The image signal to be generated is a signal in which each pixel has all of a plurality of colors, and the creating unit matches the plurality of partial image signals generated in the plurality of partial imaging regions with the arrangement of the plurality of partial imaging regions. Write means for writing in a predetermined memory area, and color separation means for performing color separation on a plurality of partial image signals stored in the predetermined memory area.

  By writing the plurality of partial image signals into the predetermined memory area in a manner that matches the arrangement of the plurality of partial imaging areas, it becomes easy to control the read address for color separation processing.

  According to this invention, some of the plurality of pixel blocks assigned to the image signal by the compression unit straddle the boundary line between two adjacent partial imaging regions. The edge component representing the boundary line of the partial imaging region is removed by the compression operation of the compression unit. Therefore, the boundary line of the image can be made inconspicuous by simple processing.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

  Referring to FIG. 1, a digital camera 10 of this embodiment includes an optical lens 12. The optical image of the object scene is irradiated onto the imaging surface of the CCD imager 16 through the optical lens 12. The imaging surface is covered with a color filter 14 having a primary color Bayer array. For this reason, the electric charge produced | generated by each of the some light receiving element formed in the imaging surface has color information of R (Red), G (Green), or B (Blue).

  When a photographing operation is performed by the key input device 44, a TG (Timing Generator) 28 is activated by the CPU 42. The TG 28 generates a plurality of timing signals including a horizontal synchronization signal Hsync and a vertical synchronization signal Vsync. Each of the drivers 18a and 18b drives the CCD imager 16 in response to the timing signal. As a result, a charge corresponding to one frame, that is, a raw image signal is output from the CCD imager 16.

  The CCD imager 16 of this embodiment has channels CH1 and CH2 as output paths. As will be described later, a partial imaging region IML is formed on the left side of the imaging surface, a partial imaging region IMR is formed on the right side of the imaging surface, and drivers 18a and 18b are assigned to the partial imaging regions IML and IMR, respectively. As a result, the half-frame raw image signal generated in the partial imaging region IML is output from the channel CH1 by driving the driver 18a. In addition, the ½ frame raw image signal generated in the partial imaging region IMR is output from the channel CH2 by driving the driver 18b.

  The CDS / AGC / AD circuit 20 performs a series of processes of correlated double sampling, automatic gain adjustment, and A / D conversion on the raw image signal of the channel CH1. The preprocessing circuit 24 performs a series of processes such as clipping, clamping, pixel defect correction, and white balance adjustment on the raw image data output from the CDS / AGC / AD circuit 20. Similarly, the CDS / AGC / AD circuit 22 performs a series of processes of correlated double sampling, automatic gain adjustment, and A / D conversion on the raw image signal of the channel CH2. The preprocessing circuit 26 performs a series of processes of cutting, clamping, pixel defect correction, and white balance adjustment on the raw image data output from the CDS / AGC / AD circuit 20. The CDS / AGC / AD circuits 20 and 22 and the preprocessing circuits 24 and 26 also perform the above-described processing in synchronization with the timing signal output from the TG 28.

  Raw image data output from each of the preprocessing circuits 24 and 26 is written into the SDRAM 32 through the memory control circuit 30. As shown in FIG. 2, the SDRAM 32 has a raw image area 32a, a YUV image area 32b, and a compressed image area 32c, and raw image data is written in the raw image area 32a. The memory control circuit 30 stores the raw image data of the channel CH1 on the left side of the raw image region 32a, and stores the raw image data of the channel CH2 on the right side of the raw image region 32b. The raw image data stored in the raw image area 32b in this way represents a captured one-field scene image.

  The post-processing circuit 34 reads such raw image data from the SDRAM 32 through the memory control circuit 30, performs processing such as color separation and YUV conversion on the read raw image data, and performs memory control on the YUV format image data. The YUV image area 32b is written through the circuit 30. The JPEG codec 36 reads the image data stored in the YUV image area 32b through the memory control circuit 30, performs JPEG compression on the read image data, and passes the compressed image data into the compressed image area 32c through the memory control circuit 30. Write.

  When JPEG compression is completed, the CPU 42 reads the compressed image data from the compressed image area 32c through the memory control circuit 30, and records the read compressed image data on the recording medium 40 in a file format. Note that the recording medium 40 is detachable and can be accessed by the CPU 42 when it is mounted in a slot not shown.

  Referring to FIG. 3, the imaging surface of CCD imager 16 has partial imaging areas IML and IMR. The partial imaging region IML is formed on the left side of the boundary line BL extending in the vertical direction from the center of the imaging surface, and the partial imaging region IMR is formed on the right side of the same boundary line BL. As can be seen from FIG. 4, the imaging surface has a resolution of horizontal 2048 pixels × vertical 1536 pixels. Each of the partial imaging regions IML and IMR has a resolution of horizontal 1024 pixels × vertical 1536 pixels. Therefore, the partial imaging regions IML and IMR are in contact with each other at the boundary line.

  In the imaging surface, an area of horizontal 2020 pixels × vertical 1515 pixels with the coordinates (14, 15) as a reference is an effective area.

  A plurality of vertical transfer registers (not shown) are assigned to each of the partial imaging regions IML and IMR. Further, a horizontal transfer register HL is assigned to the partial imaging area IML, and a horizontal transfer register HR is assigned to the imaging area IMR. Therefore, the charges generated by the plurality of light receiving elements on the partial imaging region IML are output from the channel CH1 via the vertical transfer register and the horizontal transfer register HL (not shown). Similarly, charges generated by a plurality of light receiving elements on the imaging region IMR are also output from the channel CH2 via a vertical transfer register and a horizontal transfer register HR (not shown).

  That is, the driver 18a performs raster scanning on the partial imaging region IML based on the timing signal from the TG 28, and outputs the left half frame raw image signal from the channel CH1. Similarly, the driver 18b performs raster scanning on the imaging region IMR based on the timing signal from the TG 28, and outputs the raw image signal of the right half frame from the channel CH2.

  However, the transfer direction of the horizontal transfer register HR is opposite to the transfer direction of the horizontal transfer register HL. For this reason, the raster scanning direction is also reversed between the partial imaging regions IML and IMR.

  Referring to FIG. 5, TG 28 includes an oscillator 28a. The clock signal output from the oscillator 28a is applied to the CLK terminal of the H counter 28b. The H counter 28b is incremented in response to this clock signal, and at the same time the count value (H count value) reaches “1024”, outputs a carry signal as the horizontal synchronization signal Hsync. The output horizontal synchronization signal Hsync is applied to the RST terminal of the H counter 28b, thereby resetting the H count value.

  The horizontal synchronization signal Hsync is also supplied to the CLK terminal of the V counter 28c. The V counter 28c is incremented in response to the horizontal synchronization signal Hsync. The decoder 28d outputs the vertical synchronization signal Vsync at the same time when the count value (V count value) of the V counter 28c reaches “1536”. The output vertical synchronization signal Vsync is applied to the RST terminal of the V counter 28c, thereby resetting the V count value.

  Therefore, the H count value is updated between “0” and “1023”, and the V count value is updated between “0” and “1535”. The horizontal synchronization signal Hsync is output when the H count value returns to “0”, and the vertical synchronization signal Vsync is output when the V count value returns to “0”.

  Referring to FIG. 6, the preprocessing circuit 24 includes a register 24b. The reference coordinates (16, 17), the horizontal size “1008 pixels”, and the vertical size “1504 pixels” are set in the register 24b by the CPU 42. The cutout circuit 24a cuts out raw image data belonging to an area defined by the set value of the register 24b based on the set value, H count value, and V count value of the register 24b. The raw image data is output from the clipping circuit 24a only during a period in which the H count value indicates “16” to “1023” and the V count value indicates “17” to “1520”. The output raw image data belongs to the area on the left side of the boundary line BL in the cutout area shown in FIG.

  The raw image data output from the clipping circuit 24a is output from the preprocessing circuit 24 after undergoing clamping processing by the clamping circuit 24c, defect correction processing by the pixel defect correction circuit 24d, and white balance adjustment by the white balance adjustment circuit 24e.

  Referring to FIG. 6, the preprocessing circuit 26 is configured similarly to the preprocessing circuit 24. As described above, the reference coordinates (16, 17), the horizontal size “1008 pixels”, and the vertical size “1504 pixels” are set in the register 26b. Since the raster scanning direction of the imaging region IMR is opposite to the raster scanning direction of the imaging region IML, the raw image data belonging to the region on the right side of the boundary line BL in the clipping region shown in FIG. 4 is clipped by the clipping circuit 26a. . The cut out raw image data is output from the preprocessing circuit 26 via the clamp circuit 24c, the pixel defect correction circuit 26d, and the white balance adjustment circuit 26e.

  Therefore, the raw image data stored in the raw image area 32a shown in FIG. 2 has a size of horizontal 2016 pixels × vertical 1504 pixels. The post-processing circuit 34 performs processing such as color separation and YUV conversion on the raw image data. All color information of R, G, and B is assigned to each pixel by the color separation process. However, the four pixels at the upper and lower ends and the left and right ends are missing due to the color separation process. In the YUV image area 32b shown in FIG. 2, image data of horizontal 2008 pixels × vertical 1496 pixels corresponding to the YUV format is stored. The JPEG codec 36 performs JPEG compression processing on the image data thus obtained. JPEG compression is executed with a macroblock of 8 horizontal pixels × 8 vertical pixels as one unit.

  JPEG compression is irreversible compression, and the higher the frequency, the lower the reproducibility. In this embodiment, the phenomenon that noise corresponding to the boundary line BL appears in the reproduced image is mitigated by using such JPEG compression characteristics. Specifically, the number of horizontal pixels of the YUV format image data is set to a value that is divisible by “8” but not divisible by “16”. As a result, as shown in FIG. 7, the central macroblock in the horizontal direction straddles the boundary line BL, and high-frequency noise corresponding to the boundary line BL is removed by JPEG compression.

  As can be seen from the above description, the CCD imager 16 has an imaging surface on which the partial imaging areas IML and IMR are formed, and horizontal transfer registers HL and HR assigned to the partial imaging areas IML and IMR, respectively. The driver 18a outputs a 1/2 frame raw image signal generated in the partial imaging region IML from the horizontal transfer register HL, that is, the channel CH1, and the driver 18b generates a 1/2 frame raw image generated in the partial imaging region IMR. An image signal is output from the horizontal transfer register HR, that is, the channel CH2.

  The memory control circuit 30 outputs the half-frame raw image data output from each of the channels CH1 and CH2 and subjected to predetermined processing (CDS, AGC, A / D conversion, preprocessing) to the partial imaging region IML. In addition, data is written in the raw image area 32a of the SDRAM 32 in a manner consistent with the arrangement of the IMR. Subsequently, the raw image data stored in the raw image area 32a is subjected to processing such as color separation and YUV conversion by the post-processing circuit 34. The YUV format image data thus generated is then compressed by the JPEG codec 36 in units of macroblocks of 8 horizontal pixels × 8 vertical pixels.

  Here, the partial imaging regions IML and IMR are in contact with each other. Also, some of the plurality of macroblocks assigned to the image data by the JPEG codec 36 straddle the boundary lines between the partial imaging areas IML and IMR. As a result, the edge component representing the boundary line between the partial imaging regions IML and IMR is removed by the JPEG compression process. Therefore, the boundary line of the image can be made inconspicuous by simple processing.

  In this embodiment, the number of horizontal pixels of YUV format image data is set to a value that is divisible by “8” but not divisible by “16”. Any numerical value that is not divisible by a numerical value that is divisible by the horizontal size value of the block and that is N times the horizontal size of the micro block (N: an integer of 2 or more) may be used.

  In this embodiment, two partial imaging areas are assigned to the left and right of the imaging surface. However, the two partial imaging areas may be assigned to the upper and lower sides of the imaging surface. In this case, it is necessary to set the number of vertical pixels of the YUV format image data to a value that is not divisible by N times the vertical size of the macroblock (N: an integer equal to or greater than 2) although it is divisible by the vertical size of the macroblock.

  Furthermore, in this embodiment, two partial imaging areas are assigned to the imaging plane, but three or more partial imaging areas may be assigned to the imaging plane. In this case, it is necessary to assign the partial imaging area to the imaging surface so that the boundary line between two adjacent partial imaging areas is straddled by the macroblock.

  In this embodiment, the resolution of the recorded image is one, but the resolution may be changed by key operation. In this case, it is necessary to add a thinning circuit to the post-processing circuit. Furthermore, it is preferable to perform the thinning process so that the boundary line BL is arranged at the center in the horizontal direction of the macroblock.

It is a block diagram which shows the structure of one Example of this invention. It is an illustration figure which shows an example of the mapping state of SDRAM applied to FIG. 1 Example. It is an illustration figure which shows an example of a structure of the image sensor applied to FIG. 1 Example. It is an illustration figure which shows a part of operation | movement of FIG. 1 Example. It is a block diagram which shows an example of a structure of TG applied to the FIG. 1 Example. It is a block diagram which shows an example of a structure of the pre-processing circuit applied to FIG. 1 Example. It is an illustration figure which shows a part of other operation | movement of FIG. 1 Example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Digital camera 16 ... Image sensor 18a, 18b ... Driver 24, 26 ... Pre-processing circuit 28 ... TG
32 ... SDRAM
34 ... Post-processing circuit 36 ... JPEG codec

Claims (5)

An imaging means having an imaging surface on which a plurality of partial imaging areas are formed and a plurality of output paths respectively assigned to the plurality of partial imaging areas;
Driving means for outputting a plurality of partial image signals respectively generated in the plurality of partial imaging regions from the plurality of output paths;
Creating means for creating a one-screen image signal representing an object scene image based on a plurality of partial image signals respectively output from the plurality of output paths; and a high-frequency component missing from the image signal created by the creating means Compression means for compressing each pixel block of a predetermined size according to the compression method
Two adjacent partial imaging regions out of the plurality of partial imaging regions are in contact with each other,
The portion of the plurality of pixel blocks to be allocated before Kiga image signal by the compression means straddles the boundaries of two partial imaging region adjacent digital camera. The boundary line extends in a specific direction,
Number of pixels definitive in a direction perpendicular said in a specific direction of the image signal generated by said generating means, divided by the first number is the number of pixels definitive in a direction perpendicular to the specific direction of the pixel block, and the first The digital camera according to claim 1, wherein the number of pixels is not divisible by a second number that is N times the number (N: an integer equal to or greater than 2).   The digital camera according to claim 2, wherein the specific direction is a vertical direction.   The digital camera according to any one of claims 1 to 3, wherein the compression processing of the compression means conforms to a JPEG system. The plurality of partial image signals are signals in which each pixel has color information of any one of a plurality of colors,
The image signal created by the creating means is a signal in which each pixel has all color information of the plurality of colors,
The creating means writes a plurality of partial image signals respectively generated in the plurality of partial imaging areas into a predetermined memory area in a manner consistent with an arrangement of the plurality of partial imaging areas, and stores in the predetermined memory area 5. The digital camera according to claim 1, further comprising color separation means for performing color separation on the plurality of partial image signals.

Images