JP2008147751A - Imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging device capable of effectively correcting a pixel defect. <P>SOLUTION: The imaging device includes a main imaging element 106 and a sub imaging element 104 which acquire a subject image made incident from a photography optical system, a specifying means of specifying a corresponding pixel where the same subject part as a subject part imaged on a defective pixel of the main imaging element 106 is imaged among pixels of the sub imaging element 104, and a correcting means of correcting an image signal of the defective pixel using an image signal of the corresponding pixel. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an imaging apparatus.

  As a single-lens reflex camera capable of confirming an exposed subject image in a moving image form on a monitor screen, an image sensor that is installed in the finder optical system is a subject image separated in the photographing optical system and guided to the finder optical system. A technique has been proposed in which the subject image is acquired and displayed on an LCD monitor on the back (Patent Document 1).

  In Japanese Patent Application Laid-Open No. 2004-228561, a real captured image related to a subject image is acquired using a silver salt film. However, in recent years, a digital single lens reflex camera that acquires an actual captured image using an imaging element instead of a silver salt film. Cameras (DSLR) are becoming popular.

Japanese Patent Laid-Open No. 2001-133848

  By the way, in such a digital camera (such as a digital single-lens reflex camera), a defective pixel may exist in an image sensor for acquiring a captured image. The defective pixel includes, for example, a high-output pixel (white defect) that does not depend on the exposure amount and a low-output pixel (black defect) that does not depend on the exposure amount.

  A high-capacity point due to a high-output pixel or a low-brightness point due to a low-output pixel occurs in the actual captured image acquired by such an image sensor having defective pixels, and a good captured image is obtained. Can not do it.

  Accordingly, an object of the present invention is to provide an imaging apparatus capable of effectively correcting such pixel defects.

  The present invention has been made to solve the above-described problems, and is an imaging apparatus, which includes a first imaging element and a second imaging element that acquire a subject image incident from a photographing optical system, and the second imaging element. Specifying means for identifying a corresponding pixel on which the same subject part as the subject part imaged on the defective pixel of the first image sensor is formed, and an image signal of the corresponding pixel An image pickup apparatus comprising: a correcting unit that corrects an image signal of the defective pixel.

  According to the present invention, among the first image sensor and the second image sensor that acquire the subject image incident from the imaging optical system, and the pixels in the second image sensor, the defective pixels of the first image sensor are detected. Since there is provided a specifying means for specifying a corresponding pixel on which the same subject part as the subject part to be imaged is formed, and a correction means for correcting the image signal of the defective pixel using the image signal of the corresponding pixel. It is possible to effectively correct pixel defects in the image sensor.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<1. First Embodiment>
<1-1. Configuration>
1 and 2 are diagrams showing an external configuration of an imaging apparatus 1A according to the first embodiment of the present invention. Here, FIG. 1 is a front external view of the image pickup apparatus 1A, and FIG. 2 is a rear external view of the image pickup apparatus 1A. This imaging device 1A is configured as a lens interchangeable single-lens reflex digital camera.

  As shown in FIG. 1, the imaging device 1 </ b> A includes a camera body (camera body) 2. An interchangeable photographic lens unit (interchangeable lens) 3 can be attached to and detached from the camera body 2.

  The photographic lens unit 3 mainly includes a lens barrel 101, a lens group 37 (see FIG. 3) provided in the lens barrel 101, a diaphragm 38, and the like. The lens group 37 includes a focus lens that changes the focal position by moving in the optical axis direction.

  The camera body 2 includes an annular mount Mt to which the photographing lens unit 3 is attached at the front center, and an attach / detach button 89 for attaching / detaching the photographing lens unit 3 near the annular mount Mt. Yes.

  Further, the camera body 2 is provided with a mode setting dial 82 in the upper left part of the front surface and a control value setting dial 86 in the upper right part of the front surface. By operating the mode setting dial 82, various camera modes (such as various shooting modes (such as portrait shooting mode, landscape shooting mode, and continuous shooting mode), a playback mode for playing back captured images, and external devices can be used. It is possible to perform a setting operation (switching operation) including a communication mode for performing data communication. Further, by operating the control value setting dial 86, it is possible to set control values in various shooting modes.

  Further, the camera body 2 includes a grip portion 14 for a photographer to hold at the left end of the front. A release button 11 for instructing the start of exposure is provided on the upper surface of the grip portion 14. A battery storage chamber and a card storage chamber are provided inside the grip portion 14. For example, four AA batteries are housed in the battery compartment as a power source for the camera, and a memory card 109 (see FIG. 3) for recording image data of the photographed image is detachable in the card compartment. It is designed to be stored in.

  The release button 11 is a two-stage detection button that can detect two states, a half-pressed state (S1 state) and a fully-pressed state (S2 state). When the release button 11 is half-pressed to enter the S1 state, a preparation operation (for example, an AF control operation and an AE control operation) for acquiring a recording still image (main captured image) related to the subject is performed. Further, when the release button 11 is further depressed to enter the S2 state, a photographing operation of the actual photographed image (exposure operation related to the subject image is performed using the image sensor 106 (described later), and the image signal obtained by the exposure operation is converted to A series of operations for performing predetermined image processing) is performed.

  In FIG. 2, a rear monitor 12 is provided in the approximate center of the rear surface of the camera body 2. The rear monitor 12 is configured as a color liquid crystal display (LCD), for example. The rear monitor 12 can display a menu screen for setting shooting conditions and the like, and can reproduce and display a captured image recorded on the memory card 109 in the reproduction mode.

  An optical viewfinder 10 is provided at the upper center of the back of the camera body 2. The subject image from the photographic lens unit 3 is guided to the optical finder 10, and the photographer is equivalent to the subject image acquired by the image sensor (also referred to as “main image sensor”) 106 by looking into the optical finder 10. An image can be visually recognized. Specifically, the subject image incident on the photographing optical system is reflected by a mirror (reflecting mirror) 102 (102a) and a half mirror 103 (see FIG. 3), and is viewed through an eyepiece. In this way, it is possible to determine the composition using the optical finder 10.

  In the imaging apparatus 1 </ b> A, a subject image that can be visually recognized from the optical viewfinder 10 is acquired by an imaging device for live view display (also referred to as “sub-imaging device”) 104 and is displayed live on the rear monitor 12. As described above, the composition can be determined even using the rear monitor 12. Further, at the time of photographing the actual photographed image, the mirror 102a is retracted from the optical path of the subject image, and the light (subject image) from the photographing lens unit 3 reaches the main image sensor 106, and the photographed image (image data) relating to the subject is captured. ) Is obtained.

  A main switch 81 is provided at the upper left of the rear monitor 12. The main switch 81 is a two-point slide switch. When the contact is set to the left “OFF” position, the power is turned off. When the contact is set to the right “ON” position, the power is turned on.

  A direction selection key 84 is provided on the right side of the rear monitor 12. This direction selection key 84 has a circular operation button, and it is possible to detect a pressing operation in four directions of up, down, left and right, and a pressing operation in four directions of upper right, upper left, lower right and lower left on this operation button. It has become. In addition, the direction selection key 84 is configured to detect a pressing operation of a push button at the center, in addition to the pressing operations in the eight directions.

  A setting button group 83 including a plurality of buttons for setting a menu screen, deleting an image, and the like is provided on the left side of the rear monitor 12.

  Next, an overview of the functions of the imaging apparatus 1A will be described with reference to FIG. FIG. 3 is a block diagram illustrating a functional configuration of the imaging apparatus 1A.

  As illustrated in FIG. 3, the imaging apparatus 1A includes an image processing unit 108, an operation unit 113, an overall control unit 114, a shake correction control unit 117, and the like.

  The operation unit 113 includes various buttons and switches including the release button 11 (see FIG. 1). In response to a user input operation on the operation unit 113, the overall control unit 114 and the like implement various operations.

  The overall control unit 114 is configured as a microcomputer and mainly includes a CPU, a RAM 4a, a ROM 4b, and the like. The overall control unit 114 implements various functions by reading a program stored in the ROM 4b and executing the program by the CPU.

  The focus control unit 111 controls the movement of the focus lens in the optical axis direction. Specifically, the focus control unit 111 moves a focus lens included in the lens group 37 by generating a control signal based on a signal input from the overall control unit 114 and driving a motor.

  The mirror control unit 112 controls state switching between a state where the mirror 102a is retracted from the optical path (mirror up state) and a state where the mirror 102a blocks the optical path (mirror down state). Specifically, when the fully pressed state of the release button (shooting start instruction) is detected, the mirror control unit 112 generates a control signal based on a signal input from the overall control unit 114 and drives the motor. To switch between the mirror up state and the mirror down state.

  In the mirror-down state, the subject image from the photographing optical system is reflected upward by the mirror 102a and guided to the finder optical system. The light beam guided to the finder optical system is separated (branched) by the half mirror 103 and guided to the image sensor (sub-image sensor) 104 for live view display and the optical finder 10, respectively. The subject image acquired by the sub imaging element 104 is subjected to predetermined processing in the image processing unit 108 and then displayed on the rear monitor 12 as a live view image.

  On the other hand, in the mirror-up state, the subject image from the photographic optical system is formed on the main image sensor 106, and the main image sensor 106 can acquire the actual captured image.

  The main imaging element 106 performs exposure (charge accumulation by photoelectric conversion) of the subject image formed on the light receiving surface, and generates an image signal related to the subject image. Specifically, in the main image sensor 106, a plurality of pixels configured with photodiodes are two-dimensionally arranged in a matrix, and R (red), G (green), B ( It is configured as a Bayer array color sensor (for example, CCD) in which blue) primary color transmission filters are arranged in a checkered pattern. The main image sensor 106 converts the light image of the subject formed by the lens group 37 into analog electrical signals (image signals) of R (red), G (green), and B (blue) color components, An image signal (pixel value) for each of the R, G, and B pixels is generated.

  In addition, the sub image sensor 104 exposes the subject image guided to the finder optical system, and acquires an image signal related to an image for live view display. The sub image sensor 104 basically has the same function as the main image sensor 106, but here, a case where an image sensor having a smaller number of pixels (the number of low pixels) is used is illustrated.

  The main image sensor 106 or the sub image sensor 104 responds to read control signals input from the analog front end / timing generators (AFE / TG) 107 and 105, respectively, and sends acquired image signals to the AFE / TGs 107 and 105. Output each.

  The image signals acquired by the main image sensor 106 or the sub image sensor 104 are A / D converted by AFE / TGs 105 and 107, respectively, and converted into digital image data (image data). This image data is written into the image memory 115. Thereafter, various processes are executed by the image processing unit 108 on the image data stored in the image memory 115. The image memory 115 is a high-speed accessible image memory for temporarily storing generated image data, and has a capacity capable of storing image data for a plurality of frames.

  The image processing unit 108 performs digital signal processing on the image data stored in the image memory 115 to generate image data related to the captured image. The image processing unit 108 includes a defective pixel correction unit 18a, a pixel interpolation unit 18b, a white balance control unit 18c, a gamma correction unit 18d, a contour enhancement unit 18e, a resolution conversion unit 18f, and an image compression unit 18g.

  In the defective pixel correction unit 18a, the image signal output from the defective pixel in the main image sensor 106 is corrected. Details will be described later.

  In the pixel interpolation unit 18b, the pixel value of the color signal that is insufficient in each pixel of the Bayer array is calculated by interpolation. For example, in a pixel (G pixel) from which a G component pixel value can be acquired, the R component pixel value is determined as the average value of the R pixel values above and below the G pixel, and the B component Is determined as an average value of the pixel values of the B pixels existing on the left and right of the G pixel. Further, in the R pixel, the pixel value of the insufficient G component or B component is determined based on the pixel values of the four G pixels or the four B pixels existing around the R pixel. Similarly, in the B pixel, the pixel value of the insufficient G component or R component is determined based on the pixel values of the four G pixels or the four R pixels existing around the B pixel.

  The white balance control unit 18c performs image white balance control by independently performing gain correction on each of the RGB pixels.

  The gamma correction unit 18d performs non-linear conversion (specifically, gamma correction and offset adjustment) suitable for each output device on the pixel-interpolated image data.

  In the contour emphasizing unit 18e, edge emphasis processing is performed to make the contour stand out with a high-pass filter corresponding to the image data.

  In the resolution conversion unit 18f, horizontal or vertical reduction or thinning is performed to make the image data have a predetermined number of pixels.

  In the image compression unit 18g, compression processing is performed on the image data on which each of the above processes has been performed.

  The image data acquired by the main image sensor 106 is appropriately subjected to image processing in the image processing unit 108 and then recorded on the memory card 109 by the recording control unit 110.

  Further, the image data acquired by the sub imaging element 104 is appropriately subjected to image processing by the image processing unit 108, then transferred to the display control unit 116 and displayed on the rear monitor 12. In live view display, image data continuously acquired by the sub-imaging device 104 and subjected to necessary image processing is sequentially displayed on the rear monitor 12 in a moving image manner.

  The blur correction control unit 117 is configured as a microcomputer, like the blur correction control unit 114. The shake correction control unit 117 implements a shake correction control function and the like by executing software stored therein. Specifically, the shake correction control unit 117 uses the drive mechanism 119 to move the main image sensor 106 in a plane perpendicular to the optical axis based on the output signal (shake signal) of the gyro sensor 118, thereby causing camera shake. It has a function of correcting camera shake caused by the above. The shake correction control unit 117 functions as a shake correction control device that controls the shake amount, and also functions as a shake calculation device that calculates the shake amount.

<1-2. Operation>
Next, defective pixel correction in the imaging apparatus 1A will be described. FIG. 4 is a diagram illustrating defective pixels in the main image sensor 106. In FIG. 4, a 16 × 16 pixel area around the defective pixel in the main image sensor 106 is represented, and the defective pixel is represented by black background hatching. A 4 × 4 pixel area in the main image sensor 106 corresponds to one pixel in the sub image sensor 104, and one area partitioned by a solid line in FIG. 4 represents one pixel in the sub image sensor 104. . FIG. 5 is an operation flowchart of defective pixel correction.

  Here, a case where a plurality of defective pixels included in a region (also referred to as “defective pixel region”) in which defective pixels in units of one pixel as illustrated in FIG. 4 are adjacent to each other in a two-dimensional direction is corrected. Assume that there is no blurring of the imaging apparatus 1A due to camera shake or the like. In addition, although color sensors are used for both the main image sensor 106 and the sub image sensor 104 of the image pickup apparatus 1A, in the description of the defective pixel correction, the main image sensor 106 and the sub image sensor for simplicity. A case where a monochrome monochrome image pickup device capable of acquiring a monochrome image is used as the 104 is described. In the present embodiment, the defective pixel of the main image sensor 106 is specified in advance in the manufacturing process (inspection process), and the position of the defective pixel in the main image sensor 106 is stored in the ROM 4b in advance.

  In the defective pixel correction, each process shown in FIG. 5 is performed using image data relating to the same subject image acquired by the sub image sensor 104 and stored in the image memory 115 before the main captured image is acquired by the main image sensor 106. Executed by going through. In the present embodiment, a case in which defective pixel correction is executed using the image data last acquired by the sub imaging element 104 in the mirror-down state before acquiring the actual captured image is illustrated.

  Specifically, in step SP1, a pixel (corresponding pixel) on which the same subject part as the subject part imaged on the defective pixel of the main image sensor 106 is identified among the pixels in the sub image sensor 104. The In short, in the sub image sensor 104, a pixel (corresponding pixel) that will acquire the subject image of the same part as the defective pixel of the main image sensor 106 is specified.

  In the present embodiment, since there is no shake of the image pickup apparatus 1A due to camera shake or the like, the pixels that acquire the same subject image as each pixel in the main image pickup element 106 in the sub image pickup element 104 are manufactured by the image pickup apparatus 1A. It can be specified at the time. In other words, in the sub image sensor 104, a pixel (initial corresponding pixel) on which the same subject part as the subject part imaged on the defective pixel of the main image sensor 106 is defective at the time of manufacturing the imaging device 1A. It is specified in advance for each pixel. Specifically, each defective pixel is based on a known correspondence (initial correspondence) between the pixels of the two image sensors (the main image sensor 106 and the sub image sensor 104) when there is no blur in the image pickup apparatus 1A. And the corresponding relationship between the defective pixel and the corresponding pixel are specified, and the specified corresponding relationship is stored in the ROM 4b in advance.

  In step S1, the corresponding pixel for the defective pixel of the main image sensor 106 in the sub image sensor 104 is specified based on the correspondence relationship between each defective pixel stored in the ROM 4b and the corresponding pixel for each defective pixel. The

  For example, as illustrated in FIG. 4, in the sub image sensor 104, the pixel corresponding to the defective pixel PKd of the main image sensor 106 is the pixel PF7 (region surrounded by a broken line), and the pixel corresponding to the defective pixel PKs is This is the pixel PF10 (region surrounded by a two-dot chain line).

  In step SP2, normalization (adjustment) of pixel values between two image sensors (between the main image sensor 106 and the sub image sensor 104) is performed. The normalization of the pixel values is performed by averaging the pixel values of the respective pixels of the main image sensor 106 and the sub image sensor 104 and using two average pixel values. Specifically, the difference between the two average pixel values is calculated, and the pixel value of one image sensor is calculated using the difference between the two average pixel values so that the average pixel values of the two image sensors are equal. Increase or decrease.

  Hereinafter, for simplification of description, the defective pixel correction in the specific line LN9 of the main image sensor 106 (region surrounded by a one-dot chain line in FIG. 4) will be described. FIG. 6 is a diagram illustrating the pixel value of each pixel in the specific line LN9 of the main image sensor 106. FIG. 7 is a diagram illustrating pixel values of pixels corresponding to the pixels in the specific line LN9 of the main image sensor 106 in the sub image sensor 104. FIG. 8 is a diagram illustrating pixel values of the sub image sensor 104 after normalization. 6, 7, and 8, the horizontal axis represents the position of each pixel on the specific line, and the vertical axis represents the pixel value of each pixel.

  The average pixel value of 16 pixels shown in FIG. 6 in the specific line LN9 of the main image sensor 106 is about 6, and the average pixel value of each corresponding pixel in the sub image sensor 104 is about 9. That is, since the difference in pixel value between the two image sensors is about 3, in step SP2, adjustment of the pixel value between the two image sensors (by subtracting 3 from each pixel of the sub image sensor 104) Normalization). According to this, the pixel value of the corresponding pixel in the sub image sensor 104 after normalization is as shown in FIG. By normalizing the pixel values between the two image sensors as described above, it is possible to compensate for differences in sensitivity and exposure amount between the two image sensors. Here, normalization is performed using only pixels in the specific line LN9, but in actuality, normalization of pixel values between two image sensors using the average pixel value of all pixels in each image sensor. It is preferable to carry out.

  In the next step SP3, the pixel value of the defective pixel in the main image sensor 106 (the hatched area in FIG. 6) is replaced with the pixel value of the corresponding pixel in the sub image sensor 104. FIG. 9 is a diagram in which the pixel value of the defective pixel is replaced with the pixel value of the corresponding pixel. Note that the pixel values replaced in FIG. 9 are represented by vertical lines or grid hatching.

  Specifically, in the sub image sensor 104, the corresponding pixel corresponding to the defective pixels PKe, PKf, and PKg is the pixel PF6 (see FIG. 4), and therefore the pixel values of the defective pixels PKe, PKf, and PKg include The pixel value “5” of PF6 is used (see vertical line hatching in FIG. 9). Similarly, in the sub image sensor 104, the corresponding pixel corresponding to the defective pixels PKh and PKi is the pixel PF7, and therefore the pixel value “8” of the pixel PF7 is used as the pixel value of the defective pixels PKh and PKi ( (See FIG. 9, lattice hatching).

  In step SP4, the pixel values of the defective pixels PKe to PKi are finally determined by an appropriate interpolation method. FIG. 10 is a diagram illustrating a state in which the pixel value of each defective pixel is corrected, and FIG. 11 is a diagram illustrating the pixel value of each pixel after completion of the defective pixel correction. Note that the mark * in FIG. 11 indicates the original pixel value when there is no defective pixel in the main image sensor 106.

  As the interpolation method, for example, the following method can be employed. Specifically, in FIG. 9, the defective pixels replaced by the pixel values of the same corresponding pixels are regarded as one group (here, groups GP1, GP2), and the center positions CP1, CP2 of the groups GP1, GP2 are considered. The interpolation line HL is acquired by connecting the lines with a straight line and connecting the center position of normal pixels adjacent to the group with the center position of the group with a straight line (for example, the center position CPa and the center position CP1).

  Then, an integer value that is equal to or smaller than the value on the interpolation line HL at the center position of each defective pixel is determined as the pixel value of each defective pixel PKe to PKi. For example, the pixel value of the defective pixel PKe is determined to be the largest integer value “7” which is an integer value equal to or smaller than the value CVe (= 7.5) on the interpolation line HL at the center position of the defective pixel PKe. The

  The pixel values of the defective pixels PKe to PKi determined by such an interpolation method are as shown in FIG. 10 (horizontal line hatching).

  As described above, according to the defective pixel correction using the image data by the sub imaging element 104 on which the same subject image as the subject image formed on the main imaging element 106 is formed, as shown in FIG. A pixel value close to the original pixel value (marked with * in FIG. 8) when there is no defective pixel in the main image sensor 106 can be acquired.

  In the description of the defective pixel correction described above, the case where a monochrome image sensor is used for the main image sensor 106 and the sub image sensor 104 has been described for the sake of simplicity, but the same applies to the case where a color sensor is used. Defective pixel correction is possible. FIG. 12 is a diagram illustrating the corresponding positional relationship between the pixels of the main image sensor 106 (shown by dotted lines) and the pixels of the sub image sensor 104 (shown by solid lines).

  Specifically, when a Bayer array color sensor is used for the main image sensor 106 and the sub image sensor 104, pixel information (pixel value) of the same color as the defective pixel of the main image sensor 106 in the sub image sensor 104. ) Is used to correct the defective pixel. Specifically, pixel interpolation processing is performed on the image data acquired by the sub imaging element 104, and pixel values of color signals that are insufficient in each pixel are acquired. In step SP3, the pixel value of the defective pixel of the main image sensor 106 is replaced with the pixel value of the same color signal as the defective pixel among the three color signals in the corresponding pixel.

  For example, as shown in FIG. 12, the defective pixel PKa (hatched hatching area) in the main image sensor 106 is an R component pixel, and the corresponding pixel PF6 (area surrounded by a one-dot chain line) of the defective pixel in the sub image sensor 104. ) Is a G component pixel, the pixel value of the defective pixel PKa is replaced with the pixel value of the R component in the corresponding pixel PF6 calculated by pixel interpolation.

  As described above, the image pickup apparatus 1A according to the present embodiment includes the main image pickup element 106 and the sub image pickup element 104 that acquire the subject image incident from the photographing optical system, and the main image pickup element among the pixels in the sub image pickup element 104. The corresponding pixel on which the same subject part as the subject part imaged on the defective pixel 106 is identified, and the image signal (pixel value) of the defective pixel is obtained using the image signal (pixel value) of the corresponding pixel. Since correction is performed, it becomes possible to effectively correct pixel defects in the main image sensor 106.

  In particular, according to the imaging apparatus 1A, it is possible to effectively correct each pixel value obtained from a pixel defect region formed by a plurality of defective pixels adjacent to each other.

  As another method of correcting the defective pixel, for example, a method of acquiring and correcting the pixel value of the defective pixel using information (peripheral information) obtained from other pixels around the defective pixel can be considered. According to such a method, it is possible to correct the pixel value of the defective pixel in units of one pixel. However, since there is no peripheral information about a defective pixel located in the center of the defective pixel region for a defective pixel region (also referred to as a “giant defect”) composed of a plurality of defective pixels adjacent to each other, such It is difficult to correct the pixel value of each defective pixel included in the defective pixel region using a simple technique.

  On the other hand, in the imaging apparatus 1A according to the present embodiment, a defective pixel of the main imaging element 106 is obtained using information obtained from the main imaging element 106 for acquiring a captured image and other imaging elements that acquire the same subject image. Is corrected. For this reason, in the imaging apparatus 1A, since defective pixel correction is performed using information that actually exists as information regarding defective pixels, not only defective pixels in units of one pixel but also relatively large defective pixel regions are effectively used. It becomes possible to correct.

  Further, like the imaging apparatus 1A, the live view display sub-imaging element 104 is also used for defective pixel correction, so that the conventional configuration including the live view display imaging element is compared with the conventional configuration including the live view display imaging element. It is possible to effectively correct the defective pixels of the main image sensor 106 only by adding the above-described defective pixel correction processing.

<2. Second Embodiment>
Next, a second embodiment of the present invention will be described. In the first embodiment, the defective pixel correction in the case where there is no blur of the imaging apparatus 1A due to camera shake or the like has been described. The pixel correction will be described.

  An imaging apparatus 1A according to the second embodiment has a configuration similar to that of the imaging apparatus 1A according to the first embodiment, and the following description will focus on parts that are different from the first embodiment.

  FIG. 13 is a diagram illustrating an image captured by the sub image sensor 104, and FIG. 14 is a diagram illustrating an actual captured image captured by the main image sensor 106.

  As described above, the defective pixel correction in the image pickup apparatus 1 </ b> A is executed based on the image data acquired by the sub image sensor 104 before the main captured image is acquired by the main image sensor 106. Here, the exposure of the main image sensor 106 and the exposure of the sub image sensor 104 are executed at different timings, and there is a temporal shift at least for the save time (several tens of ms) of the mirror 102a between the two exposure timings. . For this reason, in the period from the exposure time of the sub image sensor 104 to the exposure time of the main image sensor 106, if there is camera shake or panning of the image pickup apparatus 1A, an image acquired by the sub image sensor 104 is acquired. There is a shift (image shift) between the subject and the subject of the image acquired by the main image sensor 106.

  For example, when a predetermined blur occurs in the imaging apparatus 1A between the exposures of two imaging elements, an image acquired by the sub imaging element 104 (see FIG. 13) and an image acquired by the main imaging element 106 (FIG. 14), an image shift corresponding to the blur vector ZU occurs.

  Therefore, when there is a shake of the image pickup apparatus 1A due to camera shake or panning, it is preferable to perform defective pixel correction taking into account the image shift.

  Hereinafter, defective pixel correction when a predetermined blur occurs in the imaging apparatus 1A between the exposures of the two imaging elements will be described. FIG. 15 is a diagram illustrating an initial correspondence relationship between two image sensors before occurrence of blurring, and FIG. 16 is a diagram illustrating a correspondence relationship between two image sensing devices when a predetermined blurring occurs.

  Even when there is a blur in the image pickup apparatus 1A, the defective pixel correction is executed by going through the steps shown in FIG. However, in the defective pixel correction when blurring exists, the method for identifying the corresponding pixel in step SP1 is different from that in the first embodiment.

Specifically, when there is a shake, the shake amount of the image pickup apparatus 1 </ b> A generated between the exposure time of the sub image sensor 104 and the exposure time of the main image sensor 106 is detected by the shake correction control unit 117 in the gyro sensor. It is calculated based on 118 output signals.
Then, in the defective pixel correction unit 18a, a blur vector ZU representing the degree of deviation of the same subject between the image acquired by the sub image sensor 104 and the image acquired by the main image sensor 106 based on the blur amount of the image sensor 1A. Is calculated.

  Specifically, the blur vector ZU is calculated by decomposing the blur amount of the image sensor 1A into an X-direction component and a Y-direction component, and converting the blur amount of each component into a shift in pixel units of the image sensor. .

  Then, by using the correspondence relationship (initial correspondence relationship) between the pixels of the two imaging elements when no blurring occurs and the calculated blur vector ZU, the same subject image as the subject image acquired by the defective pixel is obtained. The corresponding pixel to be acquired is specified. Note that the initial correspondence relationship between the pixels of the two image pickup elements when there is no blur in the image pickup apparatus 1A is stored in the ROM 4b in advance.

  For example, when each pixel of the two image sensors has an initial correspondence relationship as shown in FIG. 15 (specifically, the initial correspondence pixel of the defective pixel PKa is PF6), an image corresponding to the blur vector ZU. Assume that a gap has occurred. In this case, the corresponding pixel of the defective pixel PKa is the initial corresponding pixel PF7 of the pixel PX1 existing at the end point of the blur vector ZU when the defective pixel PKa is the start point of the blur vector ZU. That is, the corresponding pixel of the defective pixel PKa is the pixel PF7 in the sub imaging element 104. Actually, as shown in FIG. 16, in the sub image sensor 104, it can be seen that the corresponding pixel of the defective pixel PKa of the main image sensor 106 is the pixel PF7.

  With such a corresponding pixel specifying method, the corresponding pixels in the sub imaging element 104 are specified for the other defective pixels PKb to PKt.

  Then, defective pixel correction is executed by further passing through the steps SP2 to SP4.

  As described above, the imaging apparatus 1A according to the second embodiment detects the amount of blur of the imaging apparatus 1A, and exposes the captured image acquired by the main image sensor 106 and the captured image acquired by the sub image sensor 104. Based on the amount of blur occurring between the time points, a corresponding pixel for the defective pixel of the main image sensor 106 is specified from among a plurality of pixels in the sub image sensor 104. Accordingly, in the imaging apparatus 1A, it is possible to perform defective pixel correction in consideration of the influence of image shift due to camera shake or the like that occurs during the period from the exposure time of the sub image sensor 104 to the exposure time of the main image sensor 106. .

<3. Third Embodiment>
Next, a third embodiment of the present invention will be described. The imaging apparatus 1B in the third embodiment exemplifies a case where a half mirror (semi-transmissive mirror) is used as the mirror 102 (FIG. 3) that guides the subject image from the photographing optical system to the finder optical system.

  The imaging apparatus 1B according to the third embodiment has the same configuration as that of the imaging apparatus 1A except that a half mirror is used as the mirror 102, and the following description will focus on the differences.

  The half mirror 102b provided in the imaging apparatus 1B has a function of separating the subject image from the photographing optical system into two optical paths, a reflected light path and a transmitted light path. The subject image separated into the transmitted light path is formed on the main image sensor 106. On the other hand, the subject image separated into the reflected light path is guided to the finder optical system and formed on the sub-imaging device 104.

  In the image pickup apparatus 1B, since the same subject image is simultaneously guided to the main image sensor 106 and the sub image sensor 104, the exposure of the main image sensor 106 and the exposure of the sub image sensor 104 are executed at the same timing. Is possible. For this reason, even when there is a blur in the imaging device 1B, no image deviation occurs between the subjects of the two images acquired by each imaging device.

  Therefore, in the imaging apparatus 1B, even if there is a blur, it is not necessary to consider the influence of the blur. By applying the defective pixel correction process similar to that in the first embodiment, the main imaging of the imaging apparatus 1B is performed. It is possible to effectively correct defective pixels in the element 106.

<4. Modification>
Although the embodiments of the present invention have been described above, the present invention is not limited to the contents described above.

  For example, in each of the embodiments described above, an image sensor having a lower number of pixels than the main image sensor 106 is used as the sub image sensor 104. However, the present invention is not limited to this, and has the same number of pixels as the main image sensor 106. An image sensor (for example, the same image sensor) may be used as the sub image sensor 104.

  According to this, in the defective pixel correction, the defective pixel correction can be performed by replacing the pixel value of the defective pixel with the pixel value of the corresponding pixel in the sub-imaging device 104. Therefore, the pixel value of the defective pixel The determination (step SP4) process can be omitted.

  In each of the above embodiments, the case where a defective pixel found in the manufacturing process is corrected is illustrated, but the present invention is not limited to this. Specifically, even when a defective pixel is found at any point in time, if the position of the defective pixel in the main image sensor 106 is specified, the initial correspondence relationship between the pixels of the two image sensors when no blur occurs On the basis of the above, it becomes possible to apply the defective pixel correction processing based on the above idea.

1 is a diagram illustrating an external configuration of an imaging apparatus according to a first embodiment of the present invention. 1 is a diagram illustrating an external configuration of an imaging apparatus according to a first embodiment of the present invention. It is a block diagram which shows the function structure of an imaging device. It is a figure which shows the defective pixel in a main image pick-up element. It is an operation | movement flowchart of defective pixel correction | amendment. It is a figure which shows the pixel value of each pixel in the specific line of a main image pick-up element. In a sub image sensor, it is a figure showing a pixel value of a pixel corresponding to each pixel in a specific line of a main image sensor. It is a figure which shows the pixel value of the sub image pick-up element after normalization. It is the figure which replaced the pixel value of the defective pixel with the pixel value of the corresponding pixel. It is a figure which shows a mode that the pixel value of each defective pixel was correct | amended. It is a figure which shows the pixel value of each pixel after completion | finish of defective pixel correction | amendment. It is a figure which shows the corresponding positional relationship of each by superimposing each pixel of a main image sensor, and each pixel of a sub image sensor. It is a figure which shows the image image | photographed with the sub imaging element. It is a figure which shows the real picked-up image image | photographed with the main image sensor. It is a figure which shows the initial correspondence of two image sensors before blurring generate | occur | produces. It is a figure which shows the correspondence of two image sensors when a predetermined | prescribed blurring generate | occur | produces.

Explanation of symbols

1A, 1B Imaging device 2 Camera body 3 Shooting lens unit 11 Release button 102 Mirror 104 Sub imaging device 106 Main imaging device HL Interpolation line ZU Blur vector

Claims (6)

An imaging device,
A first image sensor and a second image sensor for acquiring a subject image incident from the photographing optical system;
A specifying unit that specifies a corresponding pixel on which a subject part that is the same as a subject part imaged on a defective pixel of the first image sensor among pixels in the second image sensor;
Correction means for correcting the image signal of the defective pixel using the image signal of the corresponding pixel;
An imaging apparatus comprising: The imaging apparatus according to claim 1,
Storage means for storing a correspondence relationship between the defective pixel and a corresponding pixel for the defective pixel;
Further comprising
The imaging device is characterized in that the specifying unit specifies the corresponding pixel based on the correspondence relationship stored in the storage unit. The imaging apparatus according to claim 1,
Blur detection means for detecting the blur amount of the imaging device;
Further comprising
The first image sensor acquires a first captured image related to the subject image,
The second image sensor acquires a second captured image related to the subject image,
The imaging device is characterized in that the identification unit identifies the corresponding pixel based on the blur amount generated between exposure time points of the first captured image and the second captured image. The imaging device according to claim 3.
A mirror disposed in the optical path of the photographing optical system so as to be retractable from the optical path;
Shooting instruction means for inputting a shooting start instruction;
Further comprising
The second photographed image is a photographed image related to the subject image reflected by the mirror and guided to the second image sensor when the mirror is located in the optical path of the photographing optical system. ,
The first photographed image is a photographed image related to the subject image guided to the first image sensor in a state where the mirror is retracted from the optical path of the photographing optical system in response to the photographing start instruction. An imaging device. The imaging apparatus according to claim 1,
A semi-transmissive mirror that separates the subject image into two optical paths;
Further comprising
The first imaging element is disposed on one of the two optical paths, acquires a first captured image,
The second image sensor is disposed on the other of the two optical paths, acquires a second captured image at the same exposure timing as the exposure timing of the first captured image,
The image pickup apparatus, wherein the correction unit corrects an image signal of the defective pixel in the first photographed image using an image signal of the corresponding pixel in the second photographed image. The imaging apparatus according to claim 1,
An image pickup apparatus, wherein an image captured by the second image sensor is used for correcting the defective pixel and also used for live view display.

Images