JP5847569B2 - Image compression method and imaging apparatus - Google Patents

Description

  The present invention relates to an image compression method and an imaging apparatus.

  In recent years, the number of pixels has been increasing in digital still cameras and digital video cameras. Here, image data transferred in an image processing LSI (Large Scale Integration) or between an image processing LSI chip and a chip such as a memory passes through an image bus. Therefore, as described in Patent Documents 1 and 2 below, when a captured image is once transferred to a memory and then transferred to an image processing LSI to perform image processing, transfer of image data whose size has been increased due to the increase in the number of pixels. In this case, a large load is applied to the image Bus.

  Therefore, it is conceivable to increase the clock frequency of the image processing LSI and the bit width of the image bus portion. However, when the clock frequency of the image processing LSI is increased, the power consumption of the imaging device increases. Further, when the bit width of the image bus portion is increased, the production cost of the product increases.

  To deal with the above problems, for example, in Patent Document 3 below, after various image processing including image compression is performed on acquired captured image data, only final image data is stored in the main memory. Thus, a method for reducing the number of times image data is transferred to the image bus is disclosed.

  In addition, there has been proposed a method for reducing the load on the image bus portion when transferring image data by compressing the image data using a variable length coding method. For example, in Patent Document 4 below, an image is expressed as an RL sequence using the coefficient values L of pixels and the number R of consecutive coefficient values L, and then the RL sequence is encoded while switching a plurality of code tables. Thus, a method for performing image compression is disclosed.

JP 2010-515326 A JP 2011-171842 A JP 2011-155696A JP 2010-161787 A

  The captured image may include pixel information that has been previously determined not to be used in image processing. In particular, when a wide dynamic range image having a wide dynamic range is generated by combining a plurality of captured images captured at different exposure times, the high-intensity region of the long-exposure captured image and the short-exposure captured image It has been known in advance that pixel information included in the low luminance area is not used.

  However, in the techniques described in the above cited references 3 and 4, the high-intensity region of the long-exposure captured image and the low-intensity region of the short-exposure captured image that are previously known to be unnecessary in the wide dynamic range processing after image compression Since the pixel information contained in is also uniformly compressed, the compression efficiency is poor.

  Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to efficiently compress an image used for wide dynamic range processing before the wide dynamic range processing. It is an object of the present invention to provide a new and improved imaging apparatus and image processing apparatus that are possible.

  In order to solve the above-described problem, according to an aspect of the present invention, a first image captured with a first exposure time and a first image captured with a second exposure time shorter than the first exposure time together with the first image A second image to be subjected to image composition processing is input, and high-intensity pixel values that are not used in the image composition processing of the first image are DC-converted, and the image composition of the second image is composed of the second image. There is provided an imaging apparatus including a DC component conversion unit that performs DC conversion on low-luminance pixel values that are not used during processing.

  With this configuration, a high-luminance pixel value that is not used in the image composition process in the first image and a low-luminance pixel value that is not used in the image composition process in the second image are DC-converted. As a result, the first image and the second image can be efficiently compressed.

  In addition, the DC component conversion unit includes a first pixel conversion unit that DC-converts a high-luminance pixel value that is not used in the image synthesis process in the first image with a first conversion value, and the second image A second pixel conversion unit configured to DC-convert low-brightness pixel values that are not used in the image composition processing using the second conversion values.

  With this configuration, a high-luminance pixel value that is not used in the image composition process in the first image is DC-converted with the first conversion value, and a low-luminance pixel value that is not used in the image composition process in the second image. Is DC-converted with the second conversion value. As a result, the first image and the second image can be efficiently compressed.

  In addition, the imaging apparatus further includes an image composition unit that performs the image composition process, and the image composition unit includes a first threshold value for determining a high-luminance pixel value in the first image, and the first value. Image synthesis based on a second threshold value for determining a low-luminance pixel value in the image, wherein the first pixel conversion unit calculates a pixel value of the first image that is greater than the first threshold value, DC conversion is performed using the first conversion value, and the second pixel conversion unit calculates a third threshold value for determining a low-luminance pixel value in the second image based on the second threshold value. The pixel value of the second image that is smaller than 3 threshold values is DC-converted with the second conversion value.

  With this configuration, a pixel value larger than the first threshold value that is not used in the image composition process in the first image is DC-converted with the first conversion value, and the second threshold value that is not used in the image composition process in the second image. A smaller pixel value is DC-converted with the second conversion value. As a result, it is possible to easily determine the high-luminance pixel value of the first image and the low-luminance pixel value of the second image, and to efficiently compress the first image and the second image. In addition, an image having a wide dynamic range is generated by combining the first image and the second image.

  Further, the first conversion value is the first threshold value, and the second conversion value is the third threshold value.

  With this configuration, a high-luminance pixel value in the first image is DC-converted at the first threshold value in the image synthesis process, and a low-luminance pixel value in the second image is DC-converted at the second threshold value in the image synthesis process. Is done. As a result, the difference between the pixel value that is DC-converted and the pixel value that is not DC-converted is small, and the first image and the second image can be efficiently compressed.

  In addition, the imaging device may include an image compression unit that compresses an image, and the number of pixels that are DC-converted by the DC component conversion unit in one of the first image and the second image, and the first image and the second image. A DC component prediction unit that predicts based on the other pixel value, and the image compression unit includes a first image compression unit and a second image compression unit having different image compression methods, and the DC component prediction unit The first image compression unit or the second image compression unit is selected on the basis of the number of pixels to be DC-converted predicted by the above, and one of the first image and the second image is compressed.

  With this configuration, before one image compression of the first image and the second image is performed, the number of pixels to be DC-converted in one of the first image and the second image is predicted, and a compression method is selected. As a result, it is possible to select an appropriate compression method for the input first image and second image.

  The compression method of the first image compression unit is a run-length encoding method, and the compression method of the second image compression unit is a method other than the run-length encoding method.

  With this configuration, one of the first image and the second image can be more efficiently compressed.

  In order to solve the above-described problem, according to another aspect of the present invention, a first image captured in a first exposure time and a second exposure time shorter than the first exposure time are captured and the first image is captured. A second image that is subjected to image composition processing together with one image, and DC conversion of a high-luminance pixel value that is not used in the image composition processing in the first image; An image processing method comprising a DC component conversion step of DC converting low-luminance pixel values that are not used in the image composition processing.

  With this configuration, a high-luminance pixel value that is not used in the image composition process in the first image and a low-luminance pixel value that is not used in the image composition process in the second image are DC-converted. As a result, the first image and the second image can be efficiently compressed.

  As described above, according to the present invention, it is possible to efficiently compress an image used for wide dynamic range processing before the wide dynamic range processing.

It is explanatory drawing which showed an example of the whole structure of the imaging device which performs a wide dynamic range process. It is explanatory drawing which showed the detailed functional structure about the image synthetic | combination part of the imaging device which performs a wide dynamic range process. In long-time exposure captured image and the short-exposure captured image P _2, which is an explanatory view showing a relationship between the amount of light and the signal value. It is explanatory drawing which showed an example of the luminance histogram in a long exposure picked-up image. It is explanatory drawing which showed an example of the brightness | luminance histogram in a short time exposure captured image. It is explanatory drawing which showed an example of the whole structure of the imaging device which concerns on the 1st Embodiment of this invention. It is explanatory drawing which showed the detailed functional structure about the DC component conversion part which concerns on the same embodiment. It is the flowchart which showed an example of the flow of the DC component conversion process which concerns on the same embodiment. It is the flowchart which showed an example of the flow of the DC component conversion process which concerns on the same embodiment. It is explanatory drawing which showed an example of the spatial frequency distribution of the DC conversion area | region of the image before DC component conversion process. It is explanatory drawing which showed an example of the spatial frequency distribution of the DC conversion area | region of the image after DC component conversion process. It is explanatory drawing which showed an example of the DC conversion area | region in the image used for a wide dynamic range process. It is explanatory drawing which showed an example of the whole structure of the imaging device which concerns on the 2nd Embodiment of this invention. 5 is a flowchart illustrating an example of a flow of image compression processing of the imaging apparatus according to the embodiment. It is the flowchart which showed an example of the flow of the image compression process in the modification of the embodiment. It is explanatory drawing which showed an example of the brightness | luminance histogram of the captured image imaged in the imaging device which concerns on the embodiment.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

<1. Wide dynamic range processing>
Prior to description of a preferred embodiment of the present invention, wide dynamic range (hereinafter referred to as WDR) image composition processing will be described with reference to FIGS.

  The dynamic range is the difference between the saturation signal output level of the image sensor and the maximum value of noise. A WDR image is an image having a dynamic range wider than the dynamic range that can be expressed by the image sensor when the image is captured with a constant exposure amount. A WDR image is generated by combining a plurality of images having different exposure times.

  FIG. 1 is an explanatory diagram illustrating an example of the overall configuration of an imaging apparatus that performs WDR processing. As shown in FIG. 1, an imaging apparatus that performs WDR processing includes a lens 101, an imaging element 103, an image compression unit 107, an image bus unit 109, an image storage unit 111, an image expansion unit 113, and an image composition. The unit 115 is mainly provided.

  The lens 101 collects light incident from the subject and forms an image with the image sensor 103. Note that the amount of light incident on the lens 101 is adjusted by a diaphragm (not shown).

  The image sensor 103 includes a plurality of photoelectric conversion elements that convert incident light that has passed through the lens 101 into electrical signals. The imaging element 103 has an imaging surface on which a subject image is incident, and generates an electrical signal corresponding to the amount of received light. Examples of usable image sensor 103 include a CCD image sensor and a CMOS image sensor.

  The image sensor 103 outputs the generated electrical signal to the image compression unit 107.

  The image compression unit 107 compresses (encodes) the input image. The compression method is, for example, a format such as JPEG (Joint Photographic Experts Group), PNG (Portable Network Graphics), or an original format, and lossless compression is generally used.

  The image compression unit 107 outputs the compressed image to the image storage unit 111 via the image bus unit 109.

  Further, the imaging apparatus may include a pack processing unit that performs pack processing instead of the image compression unit 107.

(Outline of Pack processing)
Pixel values input from the image sensor 103 are often expressed in 12 bits. On the other hand, the minimum unit transferred by the image processing LSI to the SDRAM is 1 byte (= 8 bits). When one pixel is expressed by 12 bits, 2 bytes are required to transfer, so if the data is transferred as it is, the upper 4 bits are wasted. For this reason, the method of inserting the next pixel in the upper 4 bits and not transferring useless bits as described above is called Pack processing.

Here, in the imaging apparatus performs the WDR process uses a long two image data of the exposure captured image P ₁ and the short-exposure captured image P _2. Therefore, first, the long-exposure captured image P ₁ is captured by an optical system such as the lens 101 and the image sensor 103, compressed by the image compression unit 107, and temporarily stored in the image storage unit 111. Next, the short-exposure captured image P ₂ is captured by an optical system such as the lens 101 and the image sensor 103, compressed by the image compression unit 107, and temporarily stored in the image storage unit 111.

  The image storage unit 111 is a memory such as an SDRAM (Synchronous DRAM), for example, and temporarily stores captured images. The compressed image stored in the image storage unit 111 is read when decompressed for image synthesis.

  The image decompression unit 113 decompresses (decodes) the long-exposure captured image and the compressed image obtained by compressing (encoding) the short-exposure captured image stored in the image storage unit 111. The image decompression unit 113 decompresses the compressed image by a method corresponding to the compression method of the image compression unit 107.

  The image decompression unit 113 outputs the decompressed image to the image composition unit 115.

An image expanded after compressing the long-exposure captured image (hereinafter referred to as long-exposure expanded image P ₁ ′) and an image expanded after compressing the short-exposure captured image (hereinafter referred to as short-exposure expanded image P ₂ ′) Are input to the image composition unit 115. The image synthesizing unit 115 synthesizes the long exposure extended image P ₁ ′ and the short exposure extended image P ₂ ′ to generate a WDR image P ₃ .

  Hereinafter, the functional configuration of the image composition unit 115 will be described in detail with reference to FIG. FIG. 2 is an explanatory diagram showing a functional configuration of the image composition unit 115.

As shown in FIG. 2, the image composition unit 115 includes a level determination unit 131 and a composition processing unit 133. The long exposure extended image P ₁ ′ is input to the level determination unit 131 and the composition processing unit 133 of the image composition unit 115. The short-time exposure expanded image P ₂ ′ is multiplied by the exposure ratio α and then input to the synthesis processing unit 133. The exposure ratio α is a coefficient for synthesizing a long exposure captured image and a short exposure captured image having different exposure times, and is represented by the following expression 101, for example.

・・・（１０１）

... (101)

The level determination unit 131 performs threshold determination for each pixel constituting the input long-time exposure stretched image P ₁ ′ using the threshold TH0 and threshold TH1 (TH1> TH0) recorded in advance in a ROM or the like. The level determination unit 131 determines whether each pixel belongs to one of three types of Cases (Case 1 to Case 3) based on the magnitude relationship between each pixel value of the long-time exposure expanded image P ₁ ′ and the threshold value.

Here, the pixel value at the position (x, y) of the long-time exposure expanded image P ₁ ′ is LP (x, y).

・・・(１０３)

・・・（１０５）

・・・（１０７）

... (103)

... (105)

... (107)

  A case where the above expression 103 is satisfied is referred to as Case1. A case where Expression 105 is satisfied is referred to as Case2. A case where Expression 107 is satisfied is referred to as Case3.

When the level determination unit 131 performs threshold determination for each pixel constituting the long-time exposure stretched image P ₁ ′, the level determination unit 131 outputs the determination result to the synthesis processing unit 133.

The level determination unit 131 may perform threshold determination for each pixel constituting the short-time exposure expanded image P ₂ ′ instead of performing threshold determination for each pixel that configures the long-time exposure expanded image P ₁ ′. .

Based on the determination result of the level determination unit 131, the composition processing unit 133 compares the pixel value LP (x, y) at the position (x, y) of the long-time exposure expanded image P ₁ ′ and the short-time exposure expanded image P ₂ ′. The pixel value P ₃ (x, y) at the position (x, y) of the generated WDR image P ₃ is calculated from the pixel value SP (x, y) at the position (x, y).

(Case 1)
In Case 1, there may be a blackout in which the difference in luminance cannot be expressed and image information is lost in the low luminance region of the short-time exposure expanded image P ₂ ′. Therefore, the composition processing unit 133 does not use the pixel value SP (x, y) of the short-time exposure expanded image P ₂ ′, but converts the pixel value LP (x, y) of the long-time exposure expanded image P ₁ ′ to WDR. The pixel value P ₃ (x, y) of the image is output.

(Case2)
In Case 2, the composition processing unit 133 calculates the pixel value P ₃ (x, y) of the WDR image based on the following formula 109, and outputs the calculated value. Note that β in Expression 109 is a coefficient representing the ratio of combining the long-time exposure extended image P ₁ ′ and the short-time exposure extended image P ₂ ′.

・・・（１０９）

... (109)

(Case3)
In Case 3, there may be a whiteout in which the difference in luminance cannot be expressed and the image information is lost in the high luminance region of the long-time exposure extended image P ₁ ′. For this reason, the composition processing unit 133 does not use the pixel value LP (x, y) of the long-time exposure extended image P ₁ ′, but uses the pixel value SP (x, y) of the short-time exposure extended image P ₂ ′. Is multiplied by the exposure ratio α, and is output as a pixel value P ₃ (x, y) of the WDR image.

  The detailed functional configuration of the image composition unit 115 has been described above with reference to FIG.

Here, image information that is not used in the image composition unit 115 will be considered with reference to FIGS. 3, the long-time exposure captured image P ₁ and the short-exposure captured image P _2, which is an explanatory view showing a relationship between the amount of light and the signal value. Figure 4 is an explanatory view showing an example of a luminance histogram in long-time exposure taken image P _1. Figure 5 is an explanatory view showing an example of a luminance histogram in the short-exposure captured image P _2.

As shown in FIG. 3, in the region of the LP <TH0 (Case1), the pixel value of the short-time exposure captured image _{P 2} is not used in the synthesis of WDR image _{P 3.} In the region of LP> TH1 (Case 3), the pixel value of the long exposure captured image P ₁ is not used in the synthesis of the WDR image P ₃ .

Here, as shown in FIG. 4, in the long-time exposure time is long exposure captured image P _1, is not used in the synthesis of the WDR image P _3, there is a case where pixels there are many with high signal value. Further, as shown in FIG. 5, in the short-exposure period is shorter exposure captured image P _2, is not used in the synthesis of the WDR image P _3, there is a case where pixels there are many with low signal value.

In other words, long exposure captured image P ₁ and the short-exposure captured image P ₂ encompasses many unnecessary pixel information for the synthesis of WDR image P _3.

The following describes effects of the image pickup apparatus according to an unwanted pixel information as described above long-time exposure captured image P ₁ and the short-exposure captured image P ₂ encompasses many.

As described above, WDR process uses a long two image data of the exposure captured image P ₁ and the short-exposure captured image P _2. Therefore, long-time exposure taken image _{P 1} and the short-exposure captured image _{P 2} is temporarily prior to WDR processing, are recorded in the image storage unit 111 via the image Bus 109.

  Here, in recent years, image data has become multi-pixel. Therefore, in order to realize the required processing speed, it is necessary to increase the clock frequency of the image processing LSI, and there is a phenomenon that power consumption generated when transferring image data to the image storage unit 111 increases. .

  In addition, in the image compression process or the Pack process performed before transferring to the image storage unit 111, the compression or Pack process is performed including the AC component of the area where unnecessary pixel information is formed in the WDR process. The phenomenon that is.

Here, unnecessary pixel information during WDR processing is long high luminance region and low luminance region of the short-time exposure captured image P ₂ of the exposure captured image P _1. As shown in FIG. 4, the long-time exposure captured image P _1, the number of pixels is large strong tendency to have a signal value of the long exposure time high luminance side. In other words, in the long-time exposure captured image P1, the ratio of the area of the high brightness area to the area of the entire image is likely to increase. Further, as shown in FIG. 5, the short-time exposure captured image P _2, a large number of pixels a strong tendency to have a signal value of the short exposure time low luminance side. In other words, the short-time exposure captured image P _2, a tendency that the ratio of the area of the low-intensity region to the area of the entire image increases strongly.

  That is, very inefficient image compression and transfer of compressed images are performed while a large amount of image information unnecessary for WDR processing is included.

As a result of intensive studies on a technique capable of efficiently compressing an image used for the wide dynamic range process before the wide dynamic range process, the present inventor has come up with a technique as described below. .
<2. First Embodiment>
[2-1: Overall Configuration of Imaging Device 10 According to this Embodiment]
First, the overall configuration of the imaging apparatus 10 according to the first embodiment of the present invention will be described with reference to FIG. FIG. 6 is an explanatory diagram illustrating an example of the overall configuration of the imaging apparatus 10 according to the present embodiment. The overall configuration shown in FIG. 6 is an example, and some components may be omitted, added, or changed. In addition, the imaging device 10 includes, for example, a digital still camera that can shoot a still image, a video camera that can shoot a moving image, or a mobile phone, game machine, or the like equipped with an imaging function equivalent to a digital still camera or a video camera. Information terminal, personal computer, etc.

  As illustrated in FIG. 6, the imaging apparatus 10 according to the present embodiment includes a lens 101, an imaging element 103, a DC component conversion unit 105, an image compression unit 107, an image bus unit 109, and an image storage unit 111. An image expansion unit 113 and an image composition unit 115.

_{P 11} in FIG. 6 represents a long-time exposure captured image used in the WDR processing. Further, P ₂₁ in FIG. 6 represents a short-time exposure captured image used for WDR image generation.

  The lens 101 guides light incident from the subject to the image sensor 103. Note that the amount of light incident on the lens 101 is adjusted by a diaphragm (not shown).

  The image sensor 103 includes a plurality of photoelectric conversion elements that convert incident light that has passed through the lens 101 into electrical signals. The imaging element 103 has an imaging surface on which a subject image is incident, and generates an electrical signal corresponding to the amount of received light. Examples of usable image sensor 103 include a CCD image sensor and a CMOS image sensor.

The image sensor 103 outputs the long-time exposure captured image P ₁₁ and the short-time exposure captured image P ₂₁ sequentially captured to the DC component conversion unit 105.

Prolonged exposure captured image _{P 11} and the short-exposure captured image _{P 21} to the DC component conversion unit 105 is input, the DC component converting unit 105, the long-time exposure captured image _{P 11} and the short-exposure captured image _{P 21} , DC conversion is performed for regions not used for WDR processing. In this paper, DC conversion means replacing a value with a constant value. Details of the DC conversion processing performed by the DC component conversion unit 105 will be described later.

When the DC conversion is performed, the DC component conversion unit 105 outputs the long-time exposure conversion image P ₁₂ and the short-time exposure conversion image P ₂₂ to the image compression unit 107.

Image compression unit 107 compresses the long entered exposed converted images _{P 12} and the short-exposure transformed image _{P 22.} The compression method is, for example, a format such as JPEG or PNG, or a unique format, and lossless compression is generally used.

When the long-time exposure converted image P ₁₂ and the short-time exposure converted image P ₂₂ are compressed, the image compression unit 107 outputs the compressed images to the image storage unit 111 via the image bus unit 109.

  In addition, the imaging apparatus 10 may include a pack processing unit that performs pack processing instead of the image compression unit 107.

  The image storage unit 111 is a memory such as an SDRAM (Synchronous DRAM), for example, and temporarily stores captured images. The compressed image stored in the image storage unit 111 is read when decompressed for image synthesis.

The compressed image compressed (encoded) a long exposure converted image P ₁₂ and short exposure conversion image P ₂₂ recorded in the image storage unit 111, the image decompression unit 113 decompresses (decodes). The image decompression unit 113 decompresses the compressed image by a method corresponding to the compression method of the image compression unit 107.

  When the compressed image is expanded, the image expansion unit 113 outputs the expanded image to the image composition unit 115.

Image synthesis unit 115, after compression the long-time exposure captured image, the expanded image, after compression the short exposure captured image, by combining the decompressed image to generate a WDR image P _3. The functional configuration of the image composition unit 115 is the same as that described in FIG. 2 and above.

[2-2: Functional Configuration of DC Component Conversion Unit 105]
Hereinafter, a detailed functional configuration of the DC component converter 105 will be described with reference to FIGS. 7 to 9. FIG. 7 is an explanatory diagram showing a functional configuration of the DC component converter 105. 8 and 9 are explanatory diagrams illustrating an example of a flow of processing performed by the DC component conversion unit 105. FIG.

  As illustrated in FIG. 7, the DC component conversion unit 105 includes a first level determination unit 121, a first pixel conversion unit 123, a second level determination unit 125, and a second pixel conversion unit 127.

TH0 ′ and TH1 are threshold values for determining image information used in the WDR process. Pixel value LP (x, y) of the long-time exposure captured image _{P 11} If is greater than TH1, and thus not using the long-time exposure captured image _{P 11} in the WDR processing. Moreover, so that the pixel value SP (x, y) of the short time exposure captured image _{P 21} is when TH0 'smaller, after a brief period of exposure captured image _{P 21} in the WDR processing.

First, referring to FIG. 8, when the inputted long-time exposure captured image P _11, a description is given of the flow of processing by the first level determining section 121 and the first pixel converting unit 123 performs.

As shown in FIG. 8, the long-time exposure captured image _{P 11} is input to the first level determining section 121 and the first pixel converting unit 123 (S101).

The first level determination unit 121 has a pixel value LP (x, y) at the position (x, y) of the input long-time exposure captured image P ₁₁ and a threshold value TH1 used for WDR processing recorded in advance in a ROM or the like. The magnitude relationship with is determined. When the pixel value LP (x, y) is greater than the threshold value TH1, the first level determination unit 121 sets the DC component conversion Flag to Hi. On the other hand, when the pixel value LP (x, y) is equal to or less than the threshold value TH1, the first level determination unit 121 sets the DC component conversion Flag to Low. The first level determination unit 121 outputs the DC component conversion flag to the first pixel conversion unit 123 (S103).

  When the input DC component conversion Flag is Hi, the first pixel conversion unit 123 replaces the pixel value LP (x, y) with the threshold value TH1 (S105).

  On the other hand, when the input DC component conversion Flag is Low, the first pixel conversion unit 123 holds the pixel value LP (x, y) (S107).

DC component conversion unit 105, for all the pixels constituting the long-time exposure captured image _{P 11,} performs processing of steps S103 to S107.

DC component conversion unit 105, a long-time exposure transformed image _{P 12} obtained by converting the long-time exposure captured image _{P 11,} and output to the image compression unit 107, and ends the series of processing (S109).

  In step S105, the pixel value LP (x, y) is replaced with the threshold value TH1 used in the WDR process. However, the replacement value is not limited to the threshold value TH1. For example, it may be replaced with an arbitrary value near the threshold value TH1.

Here, the long-time exposure region other than the DC conversion region of the captured image P _11, the pixel values of pixels positioned in the vicinity of the boundary between the DC conversion region has a value in the vicinity of the threshold value TH1 or threshold TH1.

As described above, the pixel value of the DC transform domain long exposure captured image P ₁₁ (region to be a DC component converted in the image) is replaced with the value in the vicinity of the threshold value TH1 or threshold TH1 used in WDR processing As a result, the pixel value gradually changes near the boundary between the DC conversion region and the region other than the DC conversion region, that is, no high-frequency AC component is generated. In particular, the pixel value of the DC transform domain long exposure captured image P _11, by replacing the threshold TH1, it is possible to more effectively suppress the occurrence of the AC component.

Next, referring to FIG. 9, when the short-time exposure captured image P ₂₁ is input will be described flow of processing second level determining section 125 and the second pixel conversion unit 127 performs.

As shown in FIG. 9, the short-time exposure captured image _{P 21} is input to the second level determining section 125 and the second pixel converting unit 127 (S201).

The second level determination unit 125 compares the pixel value SP (x, y) at the position (x, y) of the input short-time exposure captured image P ₂₁ and the threshold value TH0 ′ recorded in advance in the ROM or the like. Determine. When the pixel value SP (x, y) is smaller than the threshold value TH0 ′, the second level determination unit 125 sets the DC component conversion Flag to Hi. On the other hand, when the pixel value SP (x, y) is equal to or greater than the threshold value TH0 ′, the second level determination unit 125 sets the DC component conversion Flag to Low. The second level determination unit 125 outputs the DC component conversion flag to the second pixel conversion unit 127 (S203).

  Here, for example, the following formula 111 is established for the above-mentioned threshold value TH0 'and the threshold value TH0 for determining the low luminance region of the image used for the WDR processing.

・・・（１１１）

... (111)

  When the input DC component conversion Flag is Hi, the second pixel conversion unit 127 replaces the pixel value SP (x, y) with the threshold value TH0 '(S205).

  On the other hand, when the input DC component conversion Flag is Low, the second pixel conversion unit 127 holds the pixel value SP (x, y) (S207).

DC component conversion unit 105, for all the pixels constituting the short exposure captured image _{P 21,} performs processing of step S203～S207.

DC component conversion unit 105, a short-time exposure transformed image _{P 22} obtained by converting the short exposure captured image _{P 21,} and output to the image compression unit 107, and ends the series of processing (S209).

  In step S205, the pixel value SP (x, y) is replaced with the threshold value TH0 'determined based on the threshold value TH0 used in the WDR process. However, the replacement value is not limited to the threshold value TH0'. For example, it may be replaced with an arbitrary value near the threshold TH0 '.

Here, in the short time exposure captured image region other than the DC transform domain P _21, the pixel values of pixels positioned in the vicinity of the boundary between the DC conversion region has a value in the vicinity of the threshold value TH0 'or TH0'.

As described above, the pixel value of the DC transform domain short-time exposure captured image _{P 21} (region to be a DC component converted in the image), the threshold is determined by the threshold TH0 to be used in the WDR processing TH0 'or threshold TH0' By substituting with a value in the vicinity of the pixel value, the pixel value gradually changes near the boundary between the DC conversion region and the region other than the DC conversion region, that is, a high-frequency AC component does not occur. In particular, the pixel value of the DC transform domain short-time exposure captured image P _21, by replacing the threshold TH0 ', it is possible to more effectively suppress the occurrence of the AC component.

  The functional configuration of the DC component conversion unit 105 and the flow of processing performed by the DC component conversion unit 105 have been described above with reference to FIGS.

  Below, the effect by using the imaging device 10 which concerns on this embodiment is demonstrated, referring FIGS. 10-12. FIG. 10 is an explanatory diagram showing an example of the spatial frequency distribution in the DC conversion region of the image before the processing of the DC component conversion unit 105. FIG. 11 is an explanatory diagram showing an example of the spatial frequency distribution of the image after processing by the DC component converter 105.

Here, DC transform domain, the long-time exposure captured image _{P 11} or the short exposure captured image _{P 21,} made from the pixel information is not used in the WDR processing, threshold TH0 during WDR image generated by the DC component conversion section 105 ' Or the area | region where a pixel value is substituted by threshold value TH1.

As shown in FIG. 10, in a region not used for WR image synthesis during long exposure captured image P ₁₁ or the short exposure captured image P _21, include AC component spatial frequency is not 0. Here, in many compression methods, compression is performed by expressing an original image using a difference from neighboring pixels. Therefore, in the image compression, the compression efficiency improves as the difference (AC component) from the neighboring pixels is smaller. Therefore, the compression efficiency can be improved by setting the AC component in the region not used for WDR processing (DC conversion region) to 0 as shown in FIG.

FIG. 12 is an explanatory diagram showing an example of a DC conversion region in an image used for WDR processing. However, the images used for WDR processing, refers to long-time exposure captured image _{P 11} or the short exposure captured image _{P 21.}

  AR1 and AR2 in FIG. 12 are DC conversion regions in an image used for WDR processing.

In the case of long-time exposure captured image P _11, the pixel value is larger than the threshold value TH1, the pixel overexposure occurs is included in the DC conversion regions AR1 and AR2. As for the pixels included in the DC conversion areas AR1 and AR2, the pixel values are converted with the threshold value TH1 as described above.

In the case of short exposure captured image P _21, the pixel value is less than the threshold value TH0 ', pixels underexposure is occurring is included in DC conversion regions AR1 and AR2. As for the pixels included in the DC conversion areas AR1 and AR2, the pixel values are converted with the threshold value TH0 ′ as described above.

  AR3 in FIG. 12 is an area other than the DC conversion area in the image used for the WDR process, and holds the value of the image used for the WDR process.

DC conversion region, and the high luminance region in the long-time exposure captured image P _11, a low luminance region in the short-exposure captured image P _21, there are cases where extensive. As shown in FIG. 12, since the AC component of the DC conversion region (AR1 and AR2 in FIG. 12) is 0, when this region is compressed by the run-length encoding method as in this embodiment, the compressed image Is the minimum value of various compression methods.

  The imaging device 10 according to the first embodiment of the present invention has been described above with reference to FIGS.

  As described above, according to the first embodiment of the present invention, the imaging apparatus 10 according to the present embodiment uses the AC component as the DC component for a region that is previously determined not to be used for the WDR processing. The image spatial frequency distribution conversion is performed. For this reason, the compression efficiency of image compression is improved.

  Further, since the amount of transferred data is reduced by improving the compression rate, charging / discharging in the I / O (Input-Output) buffer of the image processing LSI is suppressed, and power consumption of a storage device such as an SDRAM is suppressed. Can do. For this reason, it is possible to reduce power consumption when transferring image information to a storage device such as an SDRAM.

<3. Second Embodiment>
[3-1: Overall Configuration of Imaging Device 20 According to this Embodiment]
First, the overall configuration of the imaging apparatus 20 according to the second embodiment of the present invention will be described with reference to FIG. FIG. 13 is an explanatory diagram showing an example of the overall configuration of the imaging apparatus 20 according to the present embodiment. Note that the overall configuration shown in FIG. 13 is an example, and some components may be omitted, added, or changed. In addition, the imaging device 20 includes, for example, a digital still camera that can shoot a still image, a video camera that can shoot a moving image, or a mobile phone, a game machine, or the like equipped with an imaging function equivalent to a digital still camera or a video camera. Information terminal, personal computer, etc.

  As illustrated in FIG. 13, the imaging device 20 includes a lens 101, an imaging element 103, a DC component conversion unit 105, an image compression unit 107, an image bus unit 109, an image storage unit 111, and an image expansion unit 113. And an image composition unit 115.

_{P 11} in FIG. 13 represents a long-time exposure captured image used in the WDR processing. Also, _{P 21} in FIG. 13 represents the short-time exposure captured image used in the WDR image generation.

  The functional configurations of the lens 101, the image sensor 103, the DC component conversion unit 105, the image bus unit 109, the image storage unit 111, and the image composition unit 115 are the same as those in the first embodiment of the present invention described above. Therefore, the description is omitted.

The subject image that has passed through the lens 101 is converted into an electrical signal by the image sensor 103 and input to the DC component converter 105 and the DC component predictor 141. In the imaging device 20, the long-exposure captured image P ₁₁ and the short-exposure captured image P ₂₁ are captured and input to the DC component conversion unit 105 and the DC component prediction unit 141.

DC component prediction unit 141, a long time when the input of the exposure captured image P _11, on the basis of the pixel information of the long-time exposure captured image P _11, predicts the number of pixels included in the DC conversion region of the short-exposure captured image P ₂₁ The prediction result is output to the image compression unit 107.

  In addition, the image compression unit 107 includes a first image compression unit 143 and a second image compression unit 145. The first image compression unit 143 compresses the input image by a compression method other than run-length encoding. The second image compression unit 145 compresses the input image by run length encoding.

  Here, run-length encoding is a method for reducing (compressing) the amount of data by recording the value of data and the number of arrangements when data of the same value in the image are arranged. It is. In general, in a situation where more DC components are included in an image than AC components, run-length encoding has higher compression efficiency than other compression methods.

  However, in a normal natural image, there are almost no situations where the DC component is contained more than the AC component (in other words, the situation where data of the same value is lined up in the image), and the run-length encoding method is natural. When applied to an image, the size may be larger than the original image, and the run-length encoding method has not been applied to a natural image.

  On the other hand, in the imaging apparatus 20 according to the present embodiment, the DC component conversion unit 105 performs DC conversion on an image region that is not used for WDR processing. In some cases, the encoding scheme can be effectively applied.

Image compression unit 107, based on the short-time exposure captured image P ₂₁ the number of pixels included in the DC conversion region of the prediction results, it selects the first image compression unit 143 or the second image compression unit 145 to function.

More specifically, a short time when the exposure captured image predictive value SP_DC_NUM number of pixels included in the DC transform domain _{P 21} is a compression method selection threshold ENC_SEL_TH smaller, the image compression unit 107, run-length encoding other schemes The first image compression unit 143 to which is applied is selected and functioned.

On the other hand, if the predicted value SP_DC_NUM the number of pixels included in the DC conversion region of the short-exposure captured image _{P 21} is larger than the compression method selection threshold ENC_SEL_TH, image compression unit 107, a high compression efficiency the run-length encoding scheme The applied second image compression unit 145 is selected and functioned.

Incidentally, a short time DC conversion area of the exposed imaged image P ₂₁ and, as described above, a long time corresponding pixel value LP (x, y) of a pixel at the position of the exposure captured image P ₁₁ is the threshold value TH0 a smaller area is there.

  The image expansion unit 113 includes a first image expansion unit 147 and a second image expansion unit 149. The first image decompression unit 147 decompresses the input compressed image by an decompression method corresponding to the compression method of the first image compression unit 143 of the image compression unit 107. The second image decompression unit 149 decompresses the input compressed image by an decompression method corresponding to the compression method of the second image compression unit 145 of the image compression unit 107, that is, run length decoding. The image expansion unit 113 selects the first image expansion unit 147 or the second image expansion unit 149 as an image expansion method corresponding to the image compression method selected by the image compression unit 107 and causes it to function.

[3-2: Flow of Image Compression Process Performed by Imaging Device 20 According to this Embodiment]
Hereinafter, the flow of image compression processing of the imaging apparatus 20 according to the present embodiment will be described with reference to FIG. FIG. 14 is a flowchart illustrating an example of a flow of image compression processing of the imaging device 20 according to the present embodiment. Incidentally, at the start of a series of processes, the predicted value SP_DC_NUM the number of pixels included in the DC conversion region of the short-exposure captured image _{P 21} is put to zero.

As shown in FIG. 14, when the long-time exposure captured image _{P 11} is inputted to the DC component conversion unit 105 and the DC component prediction unit 141, DC component conversion section 105, the long-time exposure captured image _{P 11,} the pixel value DC conversion is performed for the region where LP is greater than the threshold value TH1 (S301).

Then, DC component prediction unit 141 determines for each pixel of the long exposure captured image _{P 11,} and the pixel value LP, the magnitude relationship between the threshold TH0 (S303).

Position of the long-time exposure captured image _{P 11} (x, y) when the pixel value LP (x, y) of the pixels in the threshold TH0 smaller, DC component prediction unit 141, the short-time exposure the imaged image P corresponding to the pixel _It is predicted that ₂₁ pixels are included in the DC conversion region, and SP_DC_NUM is incremented by 1 as shown in Expression 113 below (S305).

・・・（１１３）

... (113)

Position of the long-time exposure captured image _{P 11} (x, y) pixel value LP (x, y) of a pixel in the case is larger than the threshold value TH0, DC component prediction unit 141, the short-time exposure the imaged image P corresponding to the pixel _It is predicted that the ₂₁ pixels are not included in the DC conversion region, and the value of SP_DC_NUM is held as in Expression 115 below (S307).

・・・（１１５）

... (115)

DC component prediction unit 141, the processing up to the above step S303~ step S307, performed for all the pixels included in the long-time exposure captured image _{P 11.}

Next, the first image compression unit 143 performs compression of the long-time exposure captured image _{P 11,} and transfers the compressed image into the image storage unit 111 (S309).

Next, the image compression unit 107 determines the predictive value SP_DC_NUM number of pixels included in the DC conversion region of the short-exposure captured image _{P 21,} the size relationship between the compression method selection threshold ENC_SEL_TH (S311).

If the short-time exposure captured image pixel number of the predicted value SP_DC_NUM included in DC transform domain _{P 21} is greater than the compression method selection threshold ENC_SEL_TH, the image compression unit 107, the compression scheme of the short-time exposure captured image _{P 21} It changes to the 2 image compression part 145 (S313).

On the other hand, if the predicted value SP_DC_NUM the number of pixels included in the DC conversion region of the short-exposure captured image _{P 21} is a compression method selection threshold ENC_SEL_TH smaller than, the image compression unit 107, compression method of short-time exposure captured image _{P 21} Is the first image compression unit 143 (S315).

Next, when the short-time exposure captured image _{P 21} is inputted to the DC component transformer 105, DC component conversion unit 105, the short-time exposure captured image _{P 21,} the pixel value SP is the threshold TH0 'smaller area DC Perform conversion. Further, the image compression unit 107 compresses the short exposure captured image _{P 21} in compression mode selected at step S311~ step S315. The image compression unit 107 transfers the compressed image to the image storage unit 111 and ends a series of processing (S317).

  The flow of the image compression process of the imaging device 20 according to the present embodiment has been described above with reference to FIG. Note that the processing order of some processing steps may be changed.

  In the above description, the image compression unit 107 selects the first image compression unit 143 or the second image compression unit 145. However, the application range of the technique according to the present embodiment is not limited to this example. For example, a control unit (not shown) included in the imaging device 20 may select the first image compression unit 143 or the second image compression unit 145.

  The imaging device 20 according to the second embodiment of the present invention has been described above with reference to FIGS.

According to the second embodiment of the present invention described above, the image pickup apparatus 20 according to the present embodiment, the high luminance region and the long-time exposure captured image P ₁₁ that is known not to use the pre-WDR processing for low luminance region of the short-time exposure captured image P ₂₁ performs the distribution converting the image spatial frequency AC components to the DC components. For this reason, the compression efficiency of image compression is improved.

  Further, since the amount of transferred data is reduced by improving the compression rate, charging / discharging in the I / O (Input-Output) buffer of the image processing LSI is suppressed, and power consumption of a storage device such as an SDRAM is suppressed. Can do. For this reason, it is possible to reduce power consumption when transferring image information to a storage device such as an SDRAM.

Further, according to this embodiment, while the input of the long-time exposure captured image P ₁₁ used to WDR image generation, then it predicts the number of pixels DC component conversion region of the short-exposure captured image P ₂₁ to be input, based on the predicted value of the number of pixels of the DC component transform domain, it is possible to select the compression method of the short-time exposure captured image P _21. That is, for an image having a large DC conversion area that is not used for WDR processing, compression is performed by a method with high compression efficiency for an image in which data of the same value is arranged. Therefore, the compression efficiency is greatly improved as compared with the conventional case.

  In the above description, an example in which run-length encoding is used as the compression method of the second image compression unit 145 and run-length decoding is used as the expansion method of the second image expansion unit 149 has been described. Is not limited to such an example. The compression method of the second image compression unit 145 only needs to be a method with high compression efficiency when data of the same value are arranged, and the expansion method of the second image expansion unit 149 is the compression method of the second image compression unit 145. Any expansion method corresponding to the method may be used.

[3-3: Modification of Second Embodiment]
DC component prediction unit 141, when the input of the short-time exposure captured image P _21, on the basis of the pixel information of the short exposure captured image P _21, predicts the number of pixels included in the DC conversion region of the long-exposure captured image P ₁₁ The prediction result may be output to the image compression unit 107. The overall configuration other than the DC component prediction unit 141 and the image compression unit 107 is the same as that of the imaging device 20 according to the second embodiment.

Pixel Hereinafter, DC component prediction unit 141, included in the input mode of the short exposure captured image P _21, on the basis of the pixel information of the short exposure captured image P _21, the DC component conversion region of the long-exposure captured image P ₁₁ The flow of processing for predicting the number will be described with reference to FIG. FIG. 15 is a flowchart illustrating an example of the flow of image compression processing of the imaging apparatus 20 according to the present embodiment. Incidentally, at the start of a series of processes, the predicted value LP_DC_NUM the number of pixels included in the DC conversion region of the long-exposure captured image _{P 11} is put to zero.

As shown in FIG. 15, when the short-time exposure captured image _{P 21} is inputted to the DC component conversion unit 105 and the DC component prediction unit 141, DC component conversion unit 105, the short-time exposure captured image _{P 21,} the pixel value DC conversion is performed for an area where SP is smaller than the threshold TH0 ′ (S401).

Then, DC component prediction unit 141, for each pixel of the short exposure captured image _{P 21,} determines a pixel value SP, a magnitude relation between the threshold TH1 '(S403). Table In this case, TH1 ', for example a threshold used in the WDR image generated TH1 (LP> if TH1 when WDR image generation using only short exposure captured image _{P 21.)} Using the following formula 117 Is done.

・・・（１１７）

... (117)

Position of the short-time exposure captured image _{P 21} (x, y) when the pixel value SP (x, y) of pixels in is larger than the threshold TH1 ', the pixels of the long exposure captured image _{P 11} corresponding to the pixel is DC converter It is predicted to be included in the area, and LP_DC_NUM is incremented by 1 as shown in the following expression 119 (S405).

・・・（１１９）

... (119)

Position of the short-time exposure captured image _{P 21} (x, y) when the pixel value SP (x, y) of the pixels in the threshold TH1 'smaller, DC component prediction section 141, long exposure captured image corresponding to the pixel pixels P ₁₁ is predicted not included in the DC transform domain, as shown in the following expression 121 holds the value of LP_DC_NUM (S407).

・・・（１２１）

... (121)

DC component prediction unit 141, the processing up to the above step S403~ step S407, performed for all the pixels included in the short-time exposure captured image _{P 21.}

Next, the first image compression unit 143 performs compression of short exposure captured image _{P 21,} and transfers the compressed image into the image storage unit 111 (S409).

Next, the image compression unit 107 determines the predictive value LP_DC_NUM number of pixels included in the DC conversion region of the long-exposure captured image _{P 11,} the size relationship between the compression method selection threshold ENC_SEL_TH (S411).

When long exposure captured image predictive value LP_DC_NUM number of pixels included in the DC transform domain _{P 11} is greater than the compression method selection threshold ENC_SEL_TH, the image compression unit 107, the compression scheme of the long-time exposure captured image _{P 11} It changes to the 2 image compression part 145 (S413).

On the other hand, if the number of pixels LP_DC_NUM included in DC conversion region of the long-exposure captured image _{P 11} is a compression method selection threshold ENC_SEL_TH smaller than, the image compression unit 107, the compression method of the long-time exposure captured image _{P 11} first The image compression unit 143 is set (S415).

Next, when the long-time exposure captured image _{P 11} is inputted to the DC component transformer 105, DC component conversion section 105, the long-time exposure captured image _{P 11,} DC converted pixel value SP is the threshold TH1 larger area I do. Further, the image compression unit 107 compresses the long-time exposure captured image _{P 11} in compression mode selected at step S411~ step S415. The image compression unit 107 transfers the compressed image to the image storage unit 111 and ends a series of processing (S417).

  The flow of the image compression process according to the modification of the second embodiment has been described above with reference to FIG. Note that the processing order of some processing steps may be changed.

  In the above description, the image compression unit 107 selects the first image compression unit 143 or the second image compression unit 145. However, the application range of the technique according to the present embodiment is not limited to this example. For example, a control unit (not shown) included in the imaging device 20 may select the first image compression unit 143 or the second image compression unit 145.

As described above, according to the modification of the second embodiment of the present invention, the long-time exposure captured image P _{11 that is} input next while the short-exposure captured image P ₁₂ used for WDR image generation is being input. predicting the number of pixels of the DC component transform domain, based on the predicted value of the number of pixels of the DC component transform domain, it is possible to select the compression method of the long-time exposure captured image P _11. That is, for an image having a large DC conversion area that is not used for WDR processing, compression is performed by a method with high compression efficiency for an image in which data of the same value is arranged. Therefore, the compression efficiency is greatly improved as compared with the conventional case.

  In the above description, an example in which run-length encoding is used as the compression method of the second image compression unit 145 and run-length decoding is used as the expansion method of the second image expansion unit 149 has been described. Is not limited to such an example. The compression method of the second image compression unit 145 only needs to be a method with high compression efficiency when data of the same value are arranged, and the expansion method of the second image expansion unit 149 is the compression method of the second image compression unit 145. Any expansion method corresponding to the method may be used.

  FIG. 16 is an explanatory diagram illustrating an example of a luminance histogram of a captured image captured by the imaging apparatus according to the present embodiment. As in the example shown in FIG. 16, by compressing an image with a large difference in brightness and darkness in an image and a small high-luminance region by a method with high compression efficiency for an image in which data of the same value is arranged. Especially, the compression efficiency is improved.

  In other words, when the following determination formulas 103 to 105 in the WDR image generation described above are satisfied, Case 1 to Case 3 have almost no image area corresponding to Case 2, and correspond to Case 1 compared to an image area corresponding to Case 3. When the image area to be covered covers a wide range of the entire image, the compression efficiency is particularly improved by performing compression by a method with high compression efficiency for an image in which data of the same value is arranged.

・・・(１０３)

・・・（１０５）

・・・（１０７）

... (103)

... (105)

... (107)

  The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

DESCRIPTION OF SYMBOLS 10,20 Image pick-up device 101 Lens 103 Image pick-up element 105 DC component conversion part 107 Image compression part 109 Image Bus part 111 Image memory | storage part 113 Image expansion part 115 Image composition part 121 1st level determination part 123 1st pixel conversion part 125 2nd Level determination unit 127 Second pixel conversion unit 131 Level determination unit 133 Composition processing unit 141 DC component prediction unit 143 First image compression unit 145 Second image compression unit 147 First image expansion unit 149 Second image expansion unit

Claims (5)

A first image captured at a first exposure time and a second image captured at a second exposure time shorter than the first exposure time and subjected to image composition processing together with the first image are input, A DC component that DC-converts a high-luminance pixel value that is not used in the image synthesis process in one image and DC-converts a low-luminance pixel value that is not used in the image synthesis process in the second image. A conversion unit ;
An image composition unit for performing the image composition processing,
The image composition unit
A first threshold for determining a high-luminance pixel value in the first image;
A second threshold value for determining a low-luminance pixel value in the first image;
Image composition based on
The DC component conversion unit performs DC conversion on a pixel value of the first image that is larger than the first threshold with the first threshold,
Based on the second threshold value, a third threshold value for determining a low-luminance pixel value in the second image is calculated, and a pixel value of the second image smaller than the third threshold value is calculated by the third threshold value. An imaging apparatus characterized by performing DC conversion . The DC component converter is
A first pixel conversion unit that DC-converts a high-luminance pixel value that is not used in the image synthesis process in the first image with the first threshold ;
A second pixel conversion unit that DC-converts a low-luminance pixel value that is not used in the image synthesis process in the second image with the third threshold ;
The imaging apparatus according to claim 1, further comprising: An image compression unit for compressing the image;
A DC component prediction unit that predicts the number of pixels DC-converted by the DC component conversion unit in one of the first image and the second image based on the other pixel value of the first image and the second image; Further comprising
The image compression unit
A first image compression unit and a second image compression unit having different image compression methods;
Based on the number of pixels to be DC-converted predicted by the DC component prediction unit, the first image compression unit or the second image compression unit is selected, and one of the first image and the second image is compressed. and performing image pickup apparatus according to claim 1 or 2. The compression method of the first image compression unit is a run-length encoding method,
The imaging apparatus according to claim 3 , wherein a compression method of the second image compression unit is a method other than a run-length encoding method. A first image captured at a first exposure time and a second image captured at a second exposure time shorter than the first exposure time and subjected to image composition processing together with the first image are input, A DC component that DC-converts a high-luminance pixel value that is not used in the image synthesis process in one image and DC-converts a low-luminance pixel value that is not used in the image synthesis process in the second image. Conversion process ;
An image composition step for performing the image composition process,
In the image composition step,
A first threshold for determining a high-luminance pixel value in the first image;
A second threshold value for determining a low-luminance pixel value in the first image;
Image composition based on
In the DC component conversion step, the pixel value of the first image that is larger than the first threshold is subjected to DC conversion using the first threshold,
Based on the second threshold value, a third threshold value for determining a low-luminance pixel value in the second image is calculated, and a pixel value of the second image smaller than the third threshold value is calculated by the third threshold value. An image processing method characterized by performing DC conversion .

Images