JP4323883B2 - Image processing apparatus and control method thereof - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a data processing method, an image processing apparatus, and a program for correcting a defective pixel of an output signal from a solid-state imaging device using correction data.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, an image processing apparatus such as an electronic camera that records and reproduces a still image or a moving image captured by a solid-state imaging device such as a CCD or CMOS using a memory card having a solid-state memory element as a recording medium is already on the market.
[0003]
Also, when imaging using a solid-state image sensor such as a CCD or CMOS, the normal color of the same color adjacent to the flawed pixel with respect to pixel defects due to minute flaws in the pixels unique to each image sensor, so-called flawed image quality degradation There is known a correction method for correcting a defective pixel by interpolation calculation processing using pixel image data. By this method, it is possible to further reduce image quality degradation caused by the solid-state imaging device.
[0004]
Conventionally, in the defect pixel correction method, when the sensor is shipped from the factory, the output in a standard accumulation time under a predetermined condition is evaluated, and a pixel exceeding a predetermined output range is determined as a defect pixel. Black scratches, white scratches, etc.) and addresses (horizontal coordinate x, vertical coordinate y) and output data of the scratches are obtained, and the scratch pixels are corrected using this data.
[0005]
In the inspection at the time of shipment, the sensor output is obtained under the conditions of high temperature and long seconds as much as possible so that the flawed pixels at high temperature and long seconds can be accurately detected.
[0006]
However, it is known that the degree of flaw pixels, especially white flaws, varies greatly depending on the accumulation time at the time of shooting.
[0007]
Next, a difference in accumulation time due to a difference in imaging elements will be described. As shown in FIG. 12, a normal CCD-type image pickup device is provided along a vertical direction on one side of each light receiving unit with a large number of light receiving units corresponding to each pixel arranged in a matrix. It consists of a vertical transfer register and a horizontal transfer register provided on each end side of each vertical transfer register.
[0008]
The charge stored in the pixel at the present time is transferred to the vertical transfer register by the drive signal before the accumulation starts, and the signal charge transferred to the vertical transfer register is swept away by high-speed driving and accumulation is started in the pixel. .
[0009]
After that, the mechanical shutter is opened and light enters, and after a predetermined set accumulation time has elapsed, the signal charges corresponding to the amount of light received by each light receiving unit are again applied to the vertical lines corresponding to each vertical line by the drive pulse. Transferred to transfer register. Next, the signal charges are transferred to the horizontal transfer register through each vertical transfer register unit by the drive pulse. Then, the signal charge for each horizontal line is read from the horizontal transfer register unit as an imaging output from the output terminal via the charge detector.
[0010]
In this way, accumulation starts when all the pixels in the screen are transferred to the first vertical transfer register at the same time, and then when the signal charge is transferred to the vertical transfer register, the accumulation time ends at the same time for all the pixels in the screen. To do. Therefore, the substantial accumulation time is the same as the set accumulation time for all the pixels in the screen.
[0011]
On the other hand, in the CMOS type image pickup device, a phenomenon occurs in which the substantial accumulation time varies depending on the region in the shooting screen.
[0012]
In the CMOS type image pickup device, for example, as disclosed in Embodiment 6 of Patent Document 1, in order to eliminate the charge remaining in the photoelectric conversion unit, the operation of resetting the accumulated charge simultaneously for all the pixels at the start of accumulation, A so-called batch reset operation is performed.
[0013]
This is shown in FIG. This figure shows an operation sequence in each row. One horizontal line indicates one row of the image sensor, and the horizontal axis indicates time. After the collective reset operation, the set accumulation time is started, the mechanical shutter is opened at a predetermined timing, and an appropriate amount of light is exposed to the image sensor. There is no variation in exposure time for all pixels. After the shutter is closed and a predetermined set accumulation time has elapsed, the accumulated shooting is read out.
[0014]
In the CMOS image pickup device, the charges of all the pixels in the row designated by the vertical scanning pulse are amplified by the amplifier provided for each pixel, and this output is transferred to the line memory of each column. Thereafter, the signals stored in the line memory for each column are sequentially read out from the output terminal via the amplifier by the horizontal scanning pulse. When a horizontal scanning pulse for one row is sent, the output voltage of the pixel in the corresponding row is read out.
[0015]
First, transfer is performed on the first pixel in the first row, and charge is transferred to the readout circuit. Thereafter, the reading operation for one row is completed by the horizontal scanning pulse. Subsequently, the vertical scanning pulse is driven to transfer and read the second row. Similarly, it is possible to obtain the output of the entire screen by scanning the pixels of the entire screen by the combination of the horizontal scanning pulse and the vertical scanning pulse.
[0016]
That is, the transfer of the pixel charges to the readout circuit is started when the vertical scanning pulse designates the corresponding row. Therefore, the pixels in the corresponding row remain charged until the vertical scanning pulse is received. For this reason, the actual accumulation time on the screen varies depending on the readout direction, and the shorter the first row and the slower the readout order, the longer the actual accumulation time that affects the output of this white scratch pixel. . This is shown at the bottom of the figure.
[0017]
In an image processing apparatus using an image sensor that performs such an operation, the charge accumulation time is an important parameter when determining the determination level for correcting a defective pixel. Judgment was made by regarding the time until completion as the accumulation time in the photographed image.
[0018]
[Patent Document 1]
JP 2000-201300 A
[0019]
[Problems to be solved by the invention]
However, the conventional flaw correction has the following problems.
[0020]
That is, if the shutter time is long and the set accumulation time is sufficiently long compared to the time required for reading, the difference in the accumulation time in this screen is not affected, but the shutter time is set short. If the accumulation time is short, the readout time will be considerably longer, and the difference between the initial readout start time and the last readout time will be several to tens of times on the screen. May be reached.
[0021]
For this reason, in an image taken in a short time, the distribution of white flaws is uneven in the photographing screen, the portion that is first read out with a short substantial accumulation time has few flaws, and the output level is also small.
[0022]
There are many scratches in the last read-out portion, and the level becomes high.
[0023]
FIG. 14A shows an image of scratches in the output image at the time of dark photographing in the sensor that reads from the upper part to the lower part of the screen.
[0024]
On the other hand, when acquiring flaw data, flaw pixels are detected based on image data taken at a fixed accumulation time of 1 second or longer, and the actual shooting time. It is not always necessary to have data corresponding to the storage time, and usually, it is possible to perform optimum detection by multiplying a certain conversion factor.
[0025]
In addition, since a long time is set as an accumulation time to be set for an image at the time of acquiring scratch data, the difference in the accumulation time in the screen hardly affects the image data at the time of acquisition.
[0026]
Therefore, in the image from which scratch data is acquired, the actual accumulation time does not differ greatly between the portion where reading is first performed and the portion where reading is last, so the distribution of scratches and the output level thereof are almost uniform in the screen. Become.
[0027]
FIG. 14B shows an image of a flaw in a flaw data acquisition image at the time of dark photographing in a sensor that reads from the upper part to the lower part of the screen.
[0028]
In this way, in an image processing apparatus using an image sensor that has a difference in the actual accumulation time in the screen, the actual accumulation time is the longest in the screen so that the entire screen can be corrected so far. Since the determination level for performing the correction process is determined in accordance with the part, for example, in the first part of the reading, even a defective pixel which is not necessary is corrected, and the correction process takes time and the defective pixel is corrected. I was squeezing the memory to remember.
[0029]
Further, when the number of pixels of the image sensor increases and the number of scratches to be corrected increases accordingly, the memory of the image processing apparatus is compressed to store the address information of the scratch to be corrected. there were. Further, when the number of pixels to be corrected increases, the time required for the correction processing also increases, which may hinder the photographing operation.
[0030]
In addition, these are serious problems in terms of cost reduction and operability improvement.
[0031]
In addition, when a defect correction process is performed on a pixel that does not need to be corrected, there is a possibility that the image quality is deteriorated due to the correction.
[0032]
The present invention is for solving the above-described problems, and it is possible to appropriately select the number of corrections for defective pixels to be corrected, and the image quality is deteriorated due to insufficient correction or overcorrection of defective pixels. An object of the present invention is to provide a data processing method, an image processing apparatus, and a program that can avoid the problem.
[0033]
It is another object of the present invention to provide a data processing method, an image processing apparatus, and a program capable of realizing high-speed and high-performance defective pixel correction processing without increasing system resources such as memory and complicating control software. And
[0034]
In addition, the number of pixels to be corrected can be selected more appropriately according to the shooting conditions and shooting environment for the defective pixels to be corrected, and it is possible to avoid insufficient correction of the defective pixels or deterioration in image quality due to overcorrection. It is an object to provide a data processing method, an image processing apparatus, and a program.
[0035]
[Means for Solving the Problems]
To solve the above problem, Storage means for storing basic correction data having coordinates of defective pixels and defect level of the image sensor, determination value selection means for selecting a determination value according to the coordinates of the image sensor, the basic correction data, and the determination Correction data output means for outputting correction data having coordinate information of defective pixels to be corrected from the values, and correction means for correcting output signals from the defective pixels of the image sensor using the correction data. Characteristic image processing apparatus and control method thereof I will provide a.
[0041]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0042]
FIG. 1 is a block diagram showing a configuration of an image processing apparatus according to the present invention.
[0043]
In FIG. 1, reference numeral 100 denotes a main body of the image processing apparatus. Reference numeral 12 denotes a shutter for controlling the amount of exposure to the image sensor 14, and reference numeral 14 denotes a CMOS image sensor that converts an optical image into an electrical signal.
[0044]
The light beam incident on the lens 310 can be guided through the diaphragm 312, the lens mounts 306 and 106, the mirror 130, and the shutter 12 by the single-lens reflex method, and can be formed on the image sensor 14 as an optical image.
[0045]
Reference numeral 16 denotes an A / D converter that converts an analog signal output from the image sensor 14 into a digital signal.
[0046]
A timing generation circuit 18 supplies a clock signal and a control signal to the image sensor 14, the A / D converter 16, and the D / A converter 26, and is controlled by the memory control circuit 22 and the system control circuit 50.
[0047]
An image processing circuit 20 performs predetermined pixel interpolation processing and color conversion processing on the data from the A / D converter 16 or the data from the memory control circuit 22.
[0048]
Further, the image processing circuit 20 performs predetermined calculation processing using the captured image data as necessary, and the system control circuit 50 performs shutter control means 40 and distance measurement control means based on the obtained calculation result. TTL (through-the-lens) AF (autofocus) processing, AE (automatic exposure) processing, and EF (flash light control) processing can be performed.
[0049]
Further, the image processing circuit 20 performs predetermined arithmetic processing using captured image data, and also performs TTL AWB (auto white balance) processing based on the obtained arithmetic result.
[0050]
In this embodiment, since the distance measuring circuit 42 and the photometry circuit 46 are exclusively provided, the AF (auto focus) processing, AE (automatic exposure) processing, EF using the distance measuring circuit 42 and the photometry circuit 46 are used. (Flash dimming) processing may be performed, and AF (auto focus) processing, AE (auto exposure) processing, and EF (flash dimming) processing using the image processing circuit 20 may not be performed. good.
[0051]
Alternatively, AF (autofocus) processing, AE (automatic exposure) processing, and EF (flash dimming) processing are performed using the distance measuring circuit 42 and the photometry means 46, and the image processing circuit 20 is used. An AF (autofocus) process, an AE (automatic exposure) process, and an EF (flash dimming) process may be performed.
[0052]
A memory control circuit 22 controls the A / D converter 16, the timing generation circuit 18, the image processing circuit 20, the image display memory 24, the D / A converter 26, the memory 30, and the compression / decompression circuit 32.
[0053]
The data of the A / D converter 16 is written into the image display memory 24 or the memory 30 via the image processing circuit 20 and the memory control circuit 22 or the data of the A / D converter 16 is directly passed through the memory control circuit 22. It is.
[0054]
Reference numeral 24 denotes an image display memory, 26 denotes a D / A converter, 28 denotes an image display unit including a TFT LCD, and the image data for display written in the image display memory 24 passes through the D / A converter 26. Displayed by the image display unit 28.
[0055]
If the image data captured using the image display unit 28 is sequentially displayed, the electronic viewfinder function can be realized.
[0056]
The image display unit 28 can arbitrarily turn on / off the display according to an instruction from the system control circuit 50. When the display is turned off, the power consumption of the image processing apparatus 100 can be greatly reduced. I can do it.
[0057]
Reference numeral 30 denotes a memory for storing captured still images and moving images, and has a sufficient storage capacity to store a predetermined number of still images and a predetermined time of moving images.
[0058]
Thus, even in continuous shooting or panoramic shooting in which a plurality of still images are continuously shot, it is possible to write a large amount of images to the memory 30 at high speed.
[0059]
The memory 30 can also be used as a work area for the system control circuit 50.
[0060]
Reference numeral 32 denotes a compression / decompression circuit that compresses and decompresses image data by adaptive discrete cosine transform (ADCT) or the like, reads an image stored in the memory 30, performs compression processing or decompression processing, and stores the processed data in the memory 30. Write to.
[0061]
Reference numeral 40 denotes a shutter control circuit that controls the shutter 12 in cooperation with an aperture control circuit 340 that controls the aperture 312 based on photometric information from the photometry circuit 46.
[0062]
Reference numeral 42 denotes a distance measuring circuit for performing AF (autofocus) processing. Light beams incident on the lens 310 are converted into a diaphragm 312, lens mounts 306 and 106, a mirror 130, and a distance measuring sub mirror (not shown) by a single-lens reflex system. , The in-focus state of the image formed as an optical image can be measured.
[0063]
A thermometer 44 can detect the temperature of the photographing environment. When the thermometer is in the sensor, it is possible to predict the dark current of the sensor more accurately.
[0064]
Reference numeral 46 denotes a photometric circuit for performing AE (automatic exposure) processing. Light rays incident on the lens 310 are converted into a diaphragm 312, lens mounts 306 and 106, mirrors 130 and 132, and a photometric lens (not shown) by a single-lens reflex system. The incident state of the image formed as an optical image can be measured by making the light incident on the photometric means 46 via.
[0065]
The photometry circuit 46 also has an EF (flash dimming) processing function in cooperation with the flash 48.
[0066]
A flash 48 has an AF auxiliary light projecting function and a flash light control function.
[0067]
The system control circuit 50 controls the shutter control unit 40, the aperture control unit 340, and the distance measurement control unit 342 based on the calculation result obtained by calculating the image data captured by the image sensor 14 by the image processing circuit 20. It is also possible to perform exposure control and AF (autofocus) control using the video TTL method.
[0068]
Further, AF (autofocus) control may be performed using both the measurement result by the distance measuring means 42 and the calculation result obtained by calculating the image data captured by the image sensor 14 by the image processing circuit 20.
[0069]
Then, exposure control may be performed using both the measurement result obtained by the photometry unit 46 and the calculation result obtained by calculating the image data captured by the image sensor 14 by the image processing circuit 20.
[0070]
Reference numeral 50 denotes a system control circuit that controls the entire image processing apparatus 100, and reference numeral 52 denotes a memory that stores constants, variables, programs, and the like for operation of the system control circuit 50.
[0071]
Reference numeral 54 denotes a display unit such as a liquid crystal display device or a speaker that displays an operation state or a message using characters, images, sounds, or the like in accordance with execution of a program in the system control circuit 50. One or a plurality of places are provided near the portion where they are easily visible, and are configured by a combination of, for example, an LCD, an LED, a sound generation element, and the like. In addition, the display unit 54 is partially installed in the optical viewfinder 104.
[0072]
Among the display contents of the display unit 54, what is displayed on the LCD or the like is, for example, single shot / continuous shooting display, self-timer display, compression rate display, ISO sensitivity display, recording pixel number display, recording number display, remaining number display, etc. Number of storable pictures, shutter speed display, aperture value display, exposure compensation display, flash display, red-eye reduction display, macro shooting display, buzzer setting display, clock battery level display, battery level display, error display, multi-digit display There are information display by numbers, attachment / detachment state display of the recording media 200 and 210, attachment / detachment state display of the lens unit 300, communication I / F operation display, date / time display, display showing a connection state with an external computer, and the like.
[0073]
In addition, among the display contents of the display unit 54, what is displayed in the optical viewfinder 104 is, for example, in-focus display, shooting preparation completion display, camera shake warning display, flash charge display, flash charge completion display, shutter speed display, There are an aperture value display, an exposure correction display, a recording medium writing operation display, and the like.
[0074]
Further, among the display contents of the display unit 54, what is displayed on the LED or the like includes, for example, in-focus display, photographing preparation completion display, camera shake warning display, camera shake warning display, flash charge display, flash charge completion display, recording medium There are a writing operation display, a macro shooting setting notification display, a secondary battery charge state display, and the like.
[0075]
And what is displayed on a lamp etc. among the display contents of the display part 54 includes a self-timer notification lamp, for example. This self-timer notification lamp may be used in common with AF auxiliary light.
[0076]
Reference numeral 56 denotes an electrically erasable / recordable nonvolatile memory such as an EEPROM. The nonvolatile memory 56 stores various parameters, setting values such as ISO sensitivity, and setting mode data.
[0077]
Reference numerals 60, 62, 64, 66, 68, 69 and 70 are operation units for inputting various operation instructions of the system control circuit 50, such as switches, dials, touch panels, pointing by line-of-sight detection, voice recognition devices, and the like. Consists of a single or a plurality of combinations.
[0078]
Here, a specific description of these operation units will be given. Reference numeral 60 denotes a mode dial switch, which is an automatic shooting mode, program shooting mode, shutter speed priority shooting mode, aperture priority shooting mode, manual shooting mode, depth of focus priority (depth) shooting mode, portrait shooting mode, landscape shooting mode, and close-up shooting. Each function shooting mode such as a mode, a sports shooting mode, a night view shooting mode, and a panorama shooting mode can be switched and set.
[0079]
Reference numeral 62 denotes a shutter switch SW1, which is turned on during the operation of a shutter button (not shown), and performs AF (autofocus) processing, AE (automatic exposure) processing, AWB (auto white balance) processing, EF (flash dimming) processing, and the like. Instruct to start operation.
[0080]
Reference numeral 64 denotes a shutter switch SW2, which is turned on when the operation of a shutter button (not shown) is completed, and an exposure process for writing a signal read from the image sensor 12 to the memory 30 via the A / D converter 16 and the memory control circuit 22. Development processing using operations in the image processing circuit 20 and the memory control circuit 22, recording processing for reading image data from the memory 30, compression in the compression / decompression circuit 32, and writing the image data to the recording medium 200 or 210. Instructs the start of a series of processing operations.
[0081]
Reference numeral 66 denotes a playback switch, which instructs the start of a playback operation in which a shot image is read from the memory 30 or the recording medium 200 or 210 and displayed by the image display unit 28 in the shooting mode state.
[0082]
68 is a single-shot / continuous-shooting switch. When the shutter switch SW2 is pressed, a single-shot mode is set in which one frame is shot and the camera is in a standby state, and continuous shooting is performed while the shutter switch SW2 is pressed. You can set the mode.
[0083]
Reference numeral 69 denotes an ISO sensitivity setting switch, which can set the ISO sensitivity by changing the gain setting in the image sensor 14 or the image processing circuit 20.
[0084]
Reference numeral 70 denotes an operation unit composed of various buttons, a touch panel, and the like. A menu button, a set button, a macro button, a multi-screen playback page break button, a flash setting button, a single shooting / continuous shooting / self-timer switching button, menu movement + (plus) Button, menu shift-(minus) button, playback image shift + (plus) button, playback image-(minus) button, shooting quality selection button, exposure compensation button, date / time setting button, panorama mode shooting and playback Selection / switching button for setting selection and switching of various functions at the time of execution, determination / execution button for setting determination and execution of various functions at the time of executing shooting and playback in panorama mode, etc., ON of the image display unit 28 Image display ON / OFF switch to set / OFF, image data taken immediately after shooting A quick review ON / OFF switch for setting a quick review function for automatic playback, a switch for selecting a compression rate of JPEG compression, or a CCD RAW mode for digitizing an image sensor signal and recording it on a recording medium. If you press the playback switch that can set each function mode such as compression mode switch, playback mode, multi-screen playback / erase mode, PC connection mode, etc. For example, there is an AF mode setting switch that can set a one-shot AF mode that keeps the in-focus state and a servo AF mode that keeps autofocusing continuously while the shutter switch SW1 is pressed.
[0085]
In addition, the functions of the plus button and the minus button can be selected more easily by providing a rotary dial switch.
[0086]
Reference numeral 72 denotes a power switch, which can switch and set the power-on and power-off modes of the image processing apparatus 100. Also, the power on and power off settings of various accessory devices such as the lens unit 300, the external strobe, and the recording media 200 and 210 connected to the image processing apparatus 100 can be switched.
[0087]
Reference numeral 80 denotes a power control circuit, which includes a battery detection circuit, a DC-DC converter, a switch circuit for switching a block to be energized, and the like. In addition, the DC-DC converter is controlled based on an instruction from the system control circuit 50, and a necessary voltage is supplied to each unit including the recording medium for a necessary period.
[0088]
Reference numeral 82 denotes a connector, 84 denotes a connector, and 86 denotes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, NiMH battery, or Li battery, an AC adapter, or the like.
[0089]
90 and 94 are interfaces with a recording medium such as a memory card or a hard disk, 92 and 96 are connectors for connecting to a recording medium such as a memory card or a hard disk, and 98 is a recording medium 200 or 210 attached to the connector 92 or 96. It is a recording medium attachment / detachment detection circuit that detects whether or not the recording medium is present.
[0090]
In the present embodiment, it is assumed that there are two systems of interfaces and connectors for attaching a recording medium. Of course, the interface and the connector for attaching the recording medium may have a single or a plurality of systems and any number of systems. Moreover, it is good also as a structure provided with combining the interface and connector of a different standard.
[0091]
The interface and connector may be configured using a PCMCIA card, a CF (Compact Flash (registered trademark)) card, or the like that conforms to a standard.
[0092]
In addition, when the interfaces 90 and 94 and the connectors 92 and 96 are configured using a PCMCIA card, a CF (Compact Flash (registered trademark)) card, or the like, a LAN card, a modem card, a USB card, IEEE1394. Image data and management information attached to image data are transferred to and from peripheral devices such as other computers and printers by connecting various communication cards such as cards, P1284 cards, SCSI cards, and PHS communication cards. I can meet each other.
[0093]
Reference numeral 104 denotes an optical viewfinder, which can guide and display a light beam incident on the lens 310 through an aperture 312, lens mounts 306 and 106, and mirrors 130 and 132 as an optical image by a single lens reflex system. Thereby, it is possible to perform photographing using only the optical viewfinder 104 without using the electronic viewfinder function of the image display unit 28. In the optical viewfinder 104, some functions of the display unit 54, for example, a focus display, a camera shake warning display, a flash charge display, a shutter speed display, an aperture value display, an exposure correction display, and the like are installed.
[0094]
A communication circuit 110 has various communication functions such as RS232C, USB, IEEE1394, P1284, SCSI, modem, LAN, and wireless communication.
[0095]
Reference numeral 112 denotes a connector for connecting the image processing apparatus 100 to another device by the communication unit 110 or an antenna in the case of wireless communication.
[0096]
Reference numeral 120 denotes an interface for connecting the image processing apparatus 100 to the lens unit 300 in the lens mount 106, 122 denotes a connector for electrically connecting the image processing apparatus 100 to the lens unit 300, and 124 denotes the lens mount 106 and / or the connector. Reference numeral 122 denotes a lens attachment / detachment detection circuit that detects whether or not the lens unit 300 is attached.
[0097]
The connector 122 communicates control signals, status signals, data signals, and the like between the image processing apparatus 100 and the lens unit 300, and also has a function of supplying currents of various voltages. The connector 122 may be configured to transmit not only electrical communication but also optical communication, voice communication, and the like.
[0098]
Reference numerals 130 and 132 denote mirrors that can guide the light beam incident on the lens 310 to the optical viewfinder 104 by a single-lens reflex system. The mirror 132 may be either a quick return mirror or a half mirror.
[0099]
Reference numeral 200 denotes a recording medium such as a memory card or a hard disk. The recording medium 200 includes a recording unit 202 composed of a semiconductor memory, a magnetic disk, or the like, an interface 204 with the image processing apparatus 100, and a connector 206 for connecting to the image processing apparatus 100.
[0100]
Reference numeral 210 denotes a recording medium such as a memory card or a hard disk. The recording medium 210 includes a recording unit 212 composed of a semiconductor memory, a magnetic disk, and the like, an interface 214 with the image processing apparatus 100, and a connector 216 that connects to the image processing apparatus 100.
[0101]
Reference numeral 300 denotes an interchangeable lens type lens unit. Reference numeral 306 denotes a lens mount that mechanically couples the lens unit 300 to the image processing apparatus 100. The lens mount 306 includes various functions for electrically connecting the lens unit 300 to the image processing apparatus 100.
[0102]
Reference numeral 310 denotes a photographing lens, and 312 denotes an aperture. Reference numeral 320 denotes an interface for connecting the lens unit 300 to the image processing apparatus 100 in the lens mount 306, and 322 denotes an h connector for electrically connecting the lens unit 300 to the image processing apparatus 100.
[0103]
The connector 322 transmits a control signal, a status signal, a data signal, and the like between the image processing apparatus 100 and the lens unit 300, and also has a function of supplying or supplying currents of various voltages. The connector 322 may be configured to transmit not only electrical communication but also optical communication, voice communication, and the like.
[0104]
Reference numeral 340 denotes an aperture control circuit that controls the aperture 312 in cooperation with the shutter control circuit 40 that controls the shutter 12 based on photometric information from the photometry circuit 46.
[0105]
A ranging control circuit 342 controls the focusing of the photographing lens 310, and a zoom control circuit 344 controls the zooming of the photographing lens 310.
[0106]
A lens system control circuit 350 controls the entire lens unit 300. The lens system control circuit 350 includes a memory for storing operation constants, variables, programs and the like, identification information such as a number unique to the lens unit 300, management information, function information such as an open aperture value, a minimum aperture value, a focal length, It also has a non-volatile memory function for holding current and past set values.
[0107]
The operation of the image processing apparatus 100 according to the present embodiment will be described with reference to FIGS.
[0108]
2 and 3 are flowcharts of the main routine of the image processing apparatus 100. FIG. The operation of the image processing apparatus 100 will be described with reference to FIGS.
[0109]
Upon power-on such as battery replacement, the system control circuit 50 initializes flags, control variables, and the like, and performs predetermined initial settings necessary for each unit of the image processing apparatus 100 (S101).
[0110]
The system control circuit 50 determines the setting position of the power switch 66, and if the power switch 66 is set to power OFF (S102), the display on each display unit is changed to the end state, and flags, control variables, etc. Required parameters, setting values, and setting modes including the image display unit 28 are recorded in the non-volatile memory 56, and a predetermined end process such as shutting off unnecessary power sources of the respective units of the image processing apparatus 100 including the image display unit 28 by the power control unit 80 is performed. After that (S103), the process returns to step S102.
[0111]
If the power switch 66 is set to ON (S102), the system control circuit 50 determines that the remaining capacity and operation status of the power source 86 constituted by a battery or the like by the power control means 80 are the operations of the image processing apparatus 100. It is determined whether or not there is a problem (S104), and if there is a problem, a predetermined warning is displayed by image or sound using the display unit 54 (S105), and the process returns to step S102.
[0112]
If there is no problem with the power supply 86 (S104), the system control circuit 50 determines the setting position of the mode dial 60. If the mode dial 60 is set to the photographing mode (S106), the process proceeds to step S108.
[0113]
If the mode dial 60 has been set to another mode (S106), the system control circuit 50 executes processing corresponding to the selected mode (S107), and returns to step S102 when the processing is completed.
[0114]
The system control circuit 50 determines whether or not the recording medium 200 or 210 is mounted, acquires management information of image data recorded on the recording medium 200 or 210, and determines whether the operation state of the recording medium 200 or 210 is image processing. It is determined whether or not there is a problem in the operation of the apparatus 100, particularly the recording / reproducing operation of the image data with respect to the recording medium (S108), and if there is a problem, a predetermined warning is displayed by an image or sound using the display unit 54. After performing (S105), it returns to step S102.
[0115]
Determination of whether or not the recording medium 200 or 210 is mounted, acquisition of management information of image data recorded on the recording medium 200 or 210, and the operation state of the recording medium 200 or 210 are the operations of the image processing apparatus 100, particularly As a result of determining whether there is a problem in the recording / reproducing operation of the image data with respect to the recording medium (S108), if there is no problem, the process proceeds to step S109.
[0116]
The system control circuit 50 displays various setting states of the image processing apparatus 100 using images and sounds using the display unit 54 (S109). If the image display of the image display unit 28 is ON, the image display unit 28 is also used to display various setting states of the image processing apparatus 100 using images and sounds.
[0117]
If the shutter switch SW1 has not been pressed (S121), the process returns to step S102. If the shutter switch SW1 is pressed (S121), the system control circuit 50 performs a distance measurement process to focus the photographing lens 10 on the subject, and performs a light measurement process to determine an aperture value and a shutter time. A photometric process is performed (S122), and the process proceeds to step S123. In the photometric process, the flash is set if necessary. Details of the distance measurement / photometry processing step S122 will be described later with reference to FIG.
[0118]
If the shutter switch SW2 has not been pressed (S132), the system control circuit 50 checks the state of the shutter switch SW1 in step S133, and if the shutter switch SW1 is on, returns to step S132. On the other hand, if the shutter switch SW1 is off, the process returns to step S102.
[0119]
If the shutter switch SW2 is pressed (S132), the system control circuit 50 determines whether there is an image storage buffer area in the memory 30 that can store the captured image data (S134), and stores the image in the memory 30. If there is no area where new image data can be stored in the buffer area, a predetermined warning is displayed by means of an image or sound using the display unit 54 (S135), and the process returns to step S102.
[0120]
For example, immediately after performing the maximum number of continuous shots that can be stored in the image storage buffer area of the memory 30, the first image to be read from the memory 30 and written to the storage medium 200 or 210 is still in the recording medium 200 or 210. An example of this state is an unrecorded state where one empty area cannot be secured on the image storage buffer area of the memory 30 yet.
[0121]
When the captured image data is compressed and stored in the image storage buffer area of the memory 30, it can be stored in consideration of the fact that the amount of compressed image data varies depending on the compression mode setting. Whether or not the area is on the image storage buffer area of the memory 30 is determined in S134.
[0122]
If there is an image storage buffer area capable of storing the image data captured in the memory 30 (S134), the system control circuit 50 reads out an image signal that has been imaged and accumulated for a predetermined time from the image sensor 14, and performs A / D conversion. The photographing process for writing the photographed image data in a predetermined area of the memory 30 is executed via the memory 16, the image processing circuit 20, the memory control circuit 22, or directly from the A / D converter via the memory control circuit 22 ( S136). Details of the photographing processing step S136 will be described later with reference to FIG.
[0123]
When the photographing process step S136 is completed, the process proceeds to the defective pixel correction process step S139. Details of the scratch correction processing step S139 will be described later with reference to FIG.
[0124]
The system control circuit 50 reads a part of the image data after the defect correction process written in a predetermined area of the memory 30 through the memory control circuit 22 and performs WB (white balance) integration necessary for performing the development process. Arithmetic processing and OB (optical black) integration calculation processing are performed, and the calculation result is stored in the internal memory or the memory 52 of the system control circuit 50.
[0125]
Then, the system control circuit 50 uses the memory control circuit 22 and, if necessary, the image processing circuit 20 to read the captured image data after the flaw correction written in a predetermined area of the memory 30, and the system control circuit 50 Various development processes including an AWB (auto white balance) process, a gamma conversion process, and a color conversion process are performed using the calculation results stored in the internal memory or the memory 52 (S140).
[0126]
Then, the system control circuit 50 reads the image data written in the predetermined area of the memory 30 and performs the image compression processing according to the set mode by the compression / decompression circuit 32 (S141), and the image storage buffer of the memory 30 Image data that has been photographed and completed a series of processing is written in the empty image portion of the area.
[0127]
Along with execution of a series of photographing, the system control circuit 50 reads out the image data stored in the image storage buffer area of the memory 30, and the memory card or compact flash (registration) via the interface 90 or 94 and the connector 92 or 96. A recording process for writing to the recording medium 200 or 210 such as a trademark card is started (S142).
[0128]
This recording start process is performed on the image data every time new image data that has been shot and finished a series of processes is newly written in the empty image portion of the image storage buffer area of the memory 30.
[0129]
Note that while writing image data to the recording medium 200 or 210, in order to clearly indicate that the writing operation is being performed, the display unit 54 performs a recording medium writing operation display such as blinking an LED.
[0130]
The system control circuit 50 proceeds to step S143 to determine whether or not the shutter switch SW1 is pressed. While the SW1 is being pressed, the system control circuit 50 waits for the release, and if not, the process proceeds to step S102. Return.
[0131]
FIG. 4 shows a detailed flowchart of the distance measurement / photometry process in step S122 of FIG.
[0132]
In the distance measurement / photometry process, the exchange of various signals between the system control circuit 50 and the aperture control circuit 340 or the distance measurement control circuit 342 is performed by the interface 120, the connector 122, the connector 322, the interface 320, and the lens control means. 350.
[0133]
The system control circuit 50 starts an AF (autofocus) process using the image sensor 14, the distance measurement circuit 42, and the distance measurement control circuit 342 (S201).
[0134]
The system control circuit 50 causes the light beam incident on the lens 310 to enter the distance measuring means 42 through the stop 312, the lens mounts 306 and 106, the mirror 130, and the distance measuring sub mirror (not shown), thereby forming an optical image. The focus state of the formed image is determined, and the distance measuring means 42 is moved while driving the lens 310 using the distance control means 342 until the distance measurement (AF) is determined to be in focus (S203). The AF control for detecting the in-focus state is executed (S202).
[0135]
If ranging (AF) is determined to be in focus (S203), the system control circuit 50 determines a focused distance point from a plurality of distance measuring points on the shooting screen, and the determined distance measuring point. The distance measurement data and / or setting parameters are stored in the internal memory of the system control circuit 50 or the memory 52 together with the data, and the process proceeds to S205.
[0136]
Subsequently, the system control circuit 50 starts an AE (automatic exposure) process using the photometry means 46 (S205).
[0137]
The system control circuit 50 causes the light beam incident on the lens 310 to enter the photometric means 46 through the stop 312, the lens mounts 306 and 106, the mirrors 130 and 132, and a photometric lens (not shown), thereby forming an optical image. The exposure state of the formed image is measured, and photometry is performed using the exposure control means 40 until the exposure (AE) is determined to be appropriate at the ISO sensitivity preset by the ISO sensitivity switch 69 (S206). Processing is performed (S206).
[0138]
If it is determined that the exposure (AE) is appropriate (S207), the system control circuit 50 stores the photometric data and / or the setting parameters in the internal memory or the memory 52 of the system control circuit 50, and proceeds to step S208.
[0139]
The system control circuit 50 determines the aperture value (Av value) and shutter speed (Tv value) according to the exposure (AE) result detected in the photometric processing step S206 and the shooting mode set by the mode dial 60. To do.
[0140]
Then, in accordance with the shutter speed (Tv value) determined here, the system control circuit 50 determines the charge accumulation time of the image sensor 14 and performs photographing processing with the same charge accumulation time.
[0141]
Based on the measurement data obtained in the photometric processing step S206, the system control circuit 50 determines whether or not the flash is necessary (S208), sets the flash flag if the flash is necessary, and completes the charging of the flash 48 ( S210), the flash 48 is charged (S209).
[0142]
If charging of the flash 48 is completed (S210), the distance measurement / photometry processing routine step S124 is terminated.
[0143]
FIG. 5 shows a detailed flowchart of the photographing process in step S136 of FIG. 3. In the photographing process, various signals are exchanged between the system control circuit 50 and the aperture control means 340 or the distance measurement control means 342. This is performed via the interface 120, the connector 122, the connector 322, the interface 320, and the lens control circuit 350.
[0144]
The system control circuit 50 moves the mirror 130 to the mirror-up position by a mirror drive circuit (not shown) (S301), and the aperture control circuit 340 performs the photometry data stored in the internal memory of the system control circuit 50 or the memory 52. The diaphragm 312 is driven to a predetermined diaphragm value (S302).
[0145]
The system control circuit 50 instructs the image pickup device 14 to perform a batch reset operation, performs a charge clear operation (S303), starts charge accumulation in the image pickup device 14 (S304), and then uses the shutter control unit 40 to perform shutter release. 12 is opened (S305), and exposure of the image sensor 14 is started (S306).
[0146]
Here, it is determined whether or not the flash 48 is necessary based on the flash flag (S307), and if necessary, the flash is emitted (S308).
[0147]
The system control circuit 50 waits for the exposure of the image sensor 14 to end according to the photometric data (S309), closes the shutter 12 by the shutter control means 40 (S310), and ends the exposure of the image sensor 14.
[0148]
The system control circuit 50 drives the diaphragm 312 to the open diaphragm value by the diaphragm control means 340 (S311), and moves the mirror 130 to the mirror down position by the mirror driving means (not shown) (S312).
[0149]
If the set charge accumulation time has elapsed (S313), the system control circuit 50 sequentially reads out charge signals from the image sensor 14, and through the A / D converter 16, the image processing circuit 20, and the memory control circuit 22, Alternatively, the photographic image data is written in a predetermined area of the memory 30 directly from the A / D converter 16 via the memory control circuit 22 (S315).
[0150]
When the series of processing is finished, the photographing processing routine step S136 is finished.
[0151]
FIG. 6 shows a detailed flowchart of the scratch correction process in step S137 of FIG.
[0152]
In step S401, correction data to be used when performing scratch correction processing is selected. Here, a plurality of correction data is stored in the nonvolatile memory 56 or the built-in memory of the system control circuit 50, and correction data that is actually used in the defect correction processing after photographing is selected. The correction data selection sequence will be described in detail with reference to FIG.
[0153]
FIG. 7 is a detailed flowchart of correction data selection for defect correction in step S401 of FIG.
[0154]
When the CMOS image sensor 14 is shipped, various scratch pixels are extracted from image data obtained at a predetermined environmental temperature and a predetermined accumulation time. Data to be stored in the memory in the image processing apparatus is generated based on the shipping data describing the type, address, and level of the flaw pixel. This process is performed outside the image processing apparatus.
[0155]
Specifically, the correction data is selected by determining white scratches that need correction processing according to the shooting situation. Many white scratches tend to increase in level according to the exposure time (accumulation time). Even if the white scratches are at the same level, the ISO sensitivity (the gain of the image sensor 14 and the gain of the image processing unit 20) is set. ) Also changes the level of scratches in the case of an image. For this reason, here, flaw data having an appropriate determination level is selected in accordance with the set accumulation time.
[0156]
That is, the system control circuit 50 determines the charge accumulation time (t) of the image sensor 14 according to the shutter speed (Tv value) determined in photometry (S206). Processing according to t) is performed.
[0157]
For example, as shown in FIG. 8, as the charge accumulation time (t) classification in step S206 in FIG. 4, a region that is faster (shorter) than T1 seconds, a region from T1 seconds to T2 (> T1) seconds, It is assumed that the table is divided into three zones at a time slower (longer) than T2 seconds.
[0158]
In general, as the charge accumulation time becomes longer, the scratch level of white scratches increases with the accumulation time, and the output of a pixel at a level that does not become a problem in short seconds may become a large scratch level in long seconds.
[0159]
Level that is determined by the scratch level value in the shipping data so that the white scratched pixels to be corrected are included in the respective areas in comparison with the shipping data when shooting at these seconds. Setting is required.
[0160]
Always correcting all the flawed pixels described in the shipping data presses the memory in the image processing device, hinders cost reduction of the image processing device, requires processing time required for correction, and deteriorates operability. Therefore, it is desirable to perform the correction process only on the flawed pixels that really need to be corrected for each photographing condition.
[0161]
Therefore, as shown in FIG. 8, correction data is selected according to the shutter speed set in step S501. If the accumulation time set in step S502 is longer than T2, Data3 is selected as correction data in step S503. Next, when the accumulation time set in step S504 is shorter than T1, data1 is selected as correction data in step S505. Further, if the accumulation time set in step S504 does not shorten T1, data2 is selected as correction data in step S506.
[0162]
For example, the correction data data1 is corrected within the range of charge accumulation time faster (shorter) than the time T1 by correcting only white flaws having a flaw level of 100 mV or more at the output level described in the shipping data. Since no white scratches can be discriminated from the entire area, the discrimination level in that area is set to 100 mV.
[0163]
This level is set as a basic discrimination level.
[0164]
These basic discrimination levels are determined based on the criteria that white scratches are not visible on the entire screen of the photographed image, that is, white scratches are not visible on the portion of the screen where the charge accumulation time is the longest. As can be seen, in a portion where the charge accumulation time is shorter than that, correction is performed even for minute white scratches that do not need to be corrected.
[0165]
Therefore, in the present embodiment, data obtained by correcting the basic discrimination level according to the actual accumulation time is described. The specific data generation process will be described later with reference to FIG.
[0166]
FIG. 8 shows a table in which basic discrimination levels are described based on the above operation.
[0167]
In this way, defect correction data corresponding to the difference in the accumulation time in the screen to be used is selected according to the determined charge accumulation time according to the shutter speed (Tv value) determined in the photometry (S206). .
[0168]
When the correction data to be used is selected in step S401, the system control circuit 50 indicates the address of the white defect pixel described in the correction data selected in step S401 in order to compensate for the white point defect of the image sensor. While referring to the information, point defect correction processing is performed on the corresponding pixels of the captured image written in the predetermined area of the memory 30 using the captured image data of the adjacent pixels of the same color.
[0169]
In step S402, first, scratch address information for one pixel is read from the top of the selected scratch data. By referring to this address information, it is possible to specify the address of the corresponding pixel in the captured image written in the memory 30.
[0170]
Next, in step S403, the captured image data of the same color pixel adjacent to the corresponding pixel specified in step S503 is read.
[0171]
Next, in step S404, the correction amount of the corresponding pixel is calculated from the value of the adjacent pixel obtained in step S403.
[0172]
Subsequently, in step S405, the correction amount obtained in step S404 is written in the address of the corresponding pixel in the memory 30. Thereby, the correction process of the corresponding pixel is completed.
[0173]
In step S406, it is determined whether or not the correction processing for all the flaw pixels described in the designated data has been completed. If the correction processing has not yet been completed, the process returns to step S402, and the next processing described in the correction data is performed. Read the flaw address information and repeat the same.
[0174]
If it is determined in step S406 that all the correction processes described in the designated data have been completed, all the correction processes are completed and the process ends.
[0175]
Next, a correction data generation method for correcting a substantial charge accumulation time difference in the screen with respect to the basic determination level of the scratch will be described with reference to FIG.
[0176]
For example, assume a sensor that reads a captured image from the top of the screen (numbers with small vertical coordinates) to the bottom of the screen (numbers with large vertical coordinates).
[0177]
In the case of this sensor, the substantial charge accumulation time at the top of the photographed image is substantially equal to the set charge accumulation time. However, the actual charge accumulation time at the bottom of the shooting screen is a time obtained by adding the readout time to the row in addition to the set charge accumulation time.
[0178]
For example, in the condition that the readout time and the set charge accumulation time are the same, the basic judgment level is a value determined so as to be appropriate in the last part of readout, so that the actual charge is in the first part of readout. Since the accumulation time is half that time, the level of damage is considered to be proportional to the charge accumulation time, and the determination level is set to twice the basic determination level.
[0179]
Similarly, it is possible to obtain a determination level that is corrected for each row or for a plurality of rows collectively according to the difference in charge accumulation time.
[0180]
That is, when the set charge accumulation time is Tacc and the read time required to read all the pixels is Tread, the charge accumulation time of the pixel read out first is almost Tacc, and the charge accumulation time of the pixel read out last is About Tacc + Tread. Therefore, the correction level is determined for each line as the scratch correction level. When the correction level corresponding to Tacc + Tread is V, the correction level of the m-th row among all n rows is
V × (Tacc + Tread) / (Tacc + (m / n) × Tread)
And Here, m represents the vertical coordinate of the flaw pixel described in the shipping data.
[0181]
Based on the determination level set here, determination is made for all the flaw pixels described in the shipping data, and the pixels to be corrected are extracted. FIG. 9 shows a flowchart executed outside the image processing apparatus.
[0182]
First, in step S601, a basic determination level is selected. This differs depending on the conditions used, and is set, for example, as shown in FIG. In the case of obtaining data1 in which the set charge accumulation time is the shortest, the basic determination level is 100 mV. As the set charge accumulation time becomes longer, the output of a small level scratch pixel also increases, so the basic determination level of the scratch pixel to be extracted also decreases. When obtaining data3 under the condition that the charge accumulation time is a certain value or more, the basic determination level is 25 mV.
[0183]
Next, in step S602, the data for one flaw pixel described from the shipping data, that is, the coordinates of the flaw pixel and the output level under a predetermined condition are read.
[0184]
Next, in step S603, the coordinates of the flawed pixel, the determination level determined from the basic determination level obtained in step S601, and the output level described in the shipping data are compared.
[0185]
Here, if the output level described in the shipping data is higher than the determination level obtained in step S603, it is determined that the defect pixel read here needs to be corrected, and the process proceeds to step S604. To extract.
[0186]
If the output level described in the shipping data does not exceed the determination level obtained in step S603, it is determined that the defect pixel read here does not need to be corrected, and the process proceeds to step S605.
[0187]
Next, in step S605, it is determined whether or not the processing has been completed for all of the data described in the shipping data. If the processing has been completed, in step S606, the extracted scratch pixel data is stored in the image processing apparatus. Convert to a format that can be written to memory. If the extraction process for all the data at the time of shipment has not been completed, the process proceeds to step S602 to proceed to the process for the next flaw pixel described in the data at the time of shipment, and the process proceeds to the process for the next flaw address.
[0188]
The writable data obtained in this way is named data1 to data3 and written in a predetermined memory area in the image processing apparatus, so that a defect pixel correction process can be performed.
[0189]
As described above, in this embodiment, compared with the conventional case where correction processing is performed on all pixels at the determination level V, the scratch determination level is higher in the direction closer to the read start position, and the relative Therefore, only a larger level of scratches is corrected, and the number of scratches to be corrected is greatly reduced in the entire screen.
[0190]
In particular, when the set charge accumulation time is significantly shorter than the readout time, the number of scratches to be corrected is greatly reduced.
[0191]
In this embodiment, the determination of the flaw pixel to be corrected is not performed at the time of shooting, but is performed before data is written in the image processing apparatus in advance. It is also possible to perform calculations.
[0192]
The correction data used for short seconds is greatly affected by the difference in the accumulation time in the screen, and there are few scratches to be corrected at the beginning of reading in the screen, and many scratches to be corrected at the end of reading. The data is arranged. On the other hand, in the correction data used in the long second range, since the influence of the difference in the accumulation time in the screen is small, there is not much difference in the distribution of scratches to be corrected in the screen.
[0193]
Therefore, the most effective effect in the present embodiment is the region where the shutter is at the shortest time.
[0194]
(Second Embodiment)
As shown in FIG. 9, when it is necessary to simplify the calculation process for extracting the flawed pixels from the shipping data, for example, when the detection is performed from scratch detection corresponding to the shipping data in the image processing apparatus. Alternatively, the reading direction may be divided into a plurality of blocks, and the calculation may be performed in the same manner for each block to make an appropriate level determination.
[0195]
For example, a schematic diagram of the determination level when the screen is divided into four is compared with the determination level in the first embodiment and shown in FIG. Here, the basic determination level is 100 mV.
[0196]
In other words, in the image sensor that reads first from the top of the screen, only a larger level of flaw pixels need be corrected at the top in the vertical direction. When dividing into a plurality of regions, a determination value that does not exceed the determination value (indicated by a dotted line) shown in the first embodiment may be set.
[0197]
As the most effective value, when the basic determination level is V, the determination level is the 1 / 4th line when the number of the first 1/4 lines from the reading start portion in the screen is reached.
V × (Tacc + Tread) / (Tacc + (1/4) × Tread)
Similarly, from the 1 / 4th line to 1/2,
V × (Tacc + Tread) / (Tacc + (1/2) × Tread)
From the 1/2 line to the 3/4 line,
V × (Tacc + Tread) / (Tacc + (3/4) × Tread)
And from the 3 / 4th line to the last part of the reading order
V
Correction is performed at the level.
[0198]
By performing such processing, it is not necessary to calculate the level to be determined for each scratch pixel, and the comparison level is determined by determining which region the vertical coordinate of the scratch pixel is in. The load can be greatly reduced.
[0199]
A flowchart in this case is shown in FIG. First, in step S701, a basic determination level is selected. This differs depending on the conditions used, and is set, for example, as shown in FIG.
[0200]
Next, in step S702, the data for one flaw pixel described from the shipping data, that is, the coordinates of the flaw pixel and the output level under a predetermined condition are read.
[0201]
Next, it is determined in which area the vertical coordinate of the scratch pixel is included.
[0202]
In step S702, it is determined whether or not the vertical coordinate is in the upper quarter area of the screen. If it is a flaw pixel in this area, the process proceeds to step S704, and the determination value of this area is
V × (Tacc + Tread) / (Tacc + (1/4) × Tread)
It becomes. Next, in step S710, the read output level described in the shipping data is compared with the determination level obtained in step S704.
[0203]
Similarly, in step S705, it is determined whether or not the vertical coordinate is in the upper 1/4 to 1/2 area in the screen. If it is a scratch pixel in this region, the process proceeds to step S706, and the determination value of this region is
V × (Tacc + Tread) / (Tacc + (1/2) × Tread)
It becomes. Next, in step S710, the output level written and read in the shipping data is compared with the determination level obtained in step S706.
[0204]
Similarly, in step S707, it is determined whether or not the vertical coordinate is in the region from the upper half to the third quarter in the screen. If it is a scratch pixel in this area, the process proceeds to step S708, and the determination value of this area is
V × (Tacc + Tread) / (Tacc + (3/4) × Tread)
It becomes. In step S710, the output level written and read in the shipping data is compared with the determination level obtained in step S708.
[0205]
Finally, in step S707, if the vertical coordinate is a scratch pixel in the area below the upper 3/4 in the screen, the process proceeds to step S709, and the determination value of this area is
V
It becomes. In step S710, the output level written in the shipping data and read out is compared with the determination level obtained in step S709.
[0206]
Here, in step S710, if the output level described in the shipping data is higher than the determination level obtained in steps S704, S706, S708, and S709, the defect pixel read here is corrected. It is determined that it needs to be performed, and extraction is performed in step S711. If the output level described in the shipping data does not exceed the determination level obtained in steps S704, S706, S708, and S709, it is determined that the defect pixel read here does not need to be corrected, Proceed to step S712. Next, in step S712, it is determined whether or not the processing has been completed for all the data described in the shipping data. If the processing has been completed, the scratch pixel data extracted in step S713 is stored in the memory inside the image processing apparatus. Convert to a writable format. If the extraction process for all the data at the time of shipment has not been completed, the process proceeds to step S702 to proceed to the process for the next flaw pixel described in the data at the time of shipment, and the process proceeds to the process for the next flaw address.
[0207]
By performing such processing, it is not necessary to obtain determination values for all the flawed pixels described in the shipping data, and only the four values obtained in advance are compared with the output data read from the shipping data. This makes it possible to greatly reduce the arithmetic processing.
[0208]
The processing here is not limited to four divisions. However, if the number of divisions is reduced, it is possible to reduce the load required for arithmetic processing. On the other hand, if the number of divisions is increased, more suitable defect pixels can be extracted. Needless to say.
[0209]
(Other embodiments)
In the description of the above embodiment, the table for selecting the defect correction data using the charge accumulation time is set. However, the present invention is not limited to this. For example, the table is configured by adding conditions such as ISO sensitivity and temperature. It may be.
[0210]
Further, in the description of the above embodiment, it has been described that the mirror 130 is moved to the mirror-up position and the mirror-down position to perform the photographing operation. However, the mirror 130 is configured as a half mirror and the photographing operation is performed without moving. There is no problem even if you do.
[0211]
The recording media 200 and 210 are not only memory cards such as PCMCIA cards and compact flash (registered trademark), hard disks, but also micro DAT, magneto-optical disks, optical disks such as CD-R and CD-WR, DVDs, and the like. Of course, there is no problem even if it is composed of a phase change type optical disk or the like.
[0212]
Of course, there is no problem even if the recording media 200 and 210 are composite media in which a memory card and a hard disk are integrated. Further, there is no problem even if a part of the composite medium is detachable.
[0213]
The recording media 200 and 210 have been described as being separated from the image processing apparatus 100 and can be arbitrarily connected. However, of course, any or all of the recording media may remain fixed to the image processing apparatus 100. No problem.
[0214]
Further, the recording medium 200 or 210 may be connected to the image processing apparatus 100 in any number of single or plural.
[0215]
In the above description, the recording media 200 and 210 are mounted on the image processing apparatus 100. However, the recording medium may have any combination of a single or a plurality of recording media.
[0216]
Although the correction data has been described as being generated outside the image processing apparatus based on the shipping data, the correction data may be generated by the image processing apparatus itself.
[0217]
Further, a configuration may be adopted in which a flaw pixel is detected in the image processing apparatus from a dark image photographed by the image processing apparatus, and correction data is generated as described in the above embodiment. It goes without saying that the flowchart of FIG.
[0218]
The correction data may be expanded from the nonvolatile memory to the built-in memory of the system control circuit 50 when the power is turned on.
[0219]
In the above embodiment, only the white point pixel is corrected, but it is desirable to correct other types of scratch pixels under all conditions. In that case, information regarding the addresses of these other types of flaw pixels must also be described in the respective correction data, and this process can be handled when generating correction data outside the image processing apparatus.
[0220]
Further, the division of the area in the second embodiment does not need to be divided at equal intervals in the shooting screen, and may be at unequal intervals.
[0221]
In each of the above embodiments, the present invention has been described by taking a digital camera as an example. Needless to say, the present invention can also be applied to other devices and systems such as a digital video movie camera and a mobile terminal with a camera.
[0222]
It goes without saying that the processing according to the present invention may be realized as software processing executed in an information processing apparatus such as a personal computer.
[0223]
That is, the present invention supplies a storage medium storing software program codes for implementing the functions of the above-described embodiments to a system or apparatus, and the computer (or CPU or MPU) of the system or apparatus stores the storage medium in the storage medium. Needless to say, the program code is also read and executed.
[0224]
In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. As a storage medium for supplying the program code, for example, a floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, nonvolatile memory card, ROM is used. Can do. In addition, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also the OS running on the computer based on the instruction of the program code performs the actual processing. Needless to say, a case where the function of the above-described embodiment is realized by performing part or all of the processing is also included.
[0225]
Furthermore, after the program code read from the storage medium is written in the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, the program code is read based on the instruction of the next program code. It goes without saying that the CPU of the expansion board or expansion unit performs the processing of the extended function to perform part or all of the actual processing, and the function of the above-described embodiment is realized by the processing.
[0226]
Further, the present invention can also be realized by arranging an apparatus including such a storage medium on a network, downloading a program stored in the storage medium to a predetermined apparatus via the network, and executing the downloaded program. It goes without saying that the functions of the above embodiment are realized.
[0227]
【The invention's effect】
As described above, according to the present invention, it is possible to appropriately select the number of pixels to be corrected for defective pixels to be corrected, and the effect of avoiding insufficient correction of defective pixels or deterioration of image quality due to overcorrection. There is.
[0228]
This also has the effect of realizing high-speed and high-performance correction processing for defective pixels without incurring an increase in system resources such as memory or complicating control software.
[0229]
In addition, it is possible to more appropriately select the number of pixels to be corrected for the defective pixel to be corrected according to the shooting conditions and shooting environment, and there is an effect of avoiding insufficient correction of the defective pixel or deterioration of image quality due to overcorrection. .
[0230]
In the case where the solid-state imaging device is a CMOS type imaging device, the number to be corrected for the defective pixel to be corrected is appropriately set in accordance with the substantial change in the accumulation time generated in the CMOS type imaging device. This makes it possible to select an image, and it is effective in avoiding insufficient correction of defective pixels or deterioration in image quality due to overcorrection.
[0231]
Further, when extracting the defective pixel, the correction data having at least the positional information of the defective pixel of the solid-state imaging device and the information regarding the output level is extracted with the output level exceeding the determination value. It is configured to do. As a result, the number of pixels to be corrected can be appropriately selected for the defective pixel to be corrected, and there is an effect of avoiding an insufficient correction of the defective pixel or a deterioration in image quality due to overcorrection.
[0232]
In addition, when the charge accumulation period and the ISO sensitivity building shooting conditions change, it is possible to more appropriately select the number to be corrected for the defective pixel to be corrected according to the charge accumulation time and the ISO sensitivity. This has the effect of avoiding poor image quality due to insufficient pixel correction or overcorrection.
[0233]
In addition, when the temperature in the shooting environment changes, it is possible to more appropriately select the number of pixels to be corrected for the defective pixel to be corrected according to the temperature. There is an effect to avoid the decrease of
[0234]
In addition, when the defective pixel has a white defect, it is possible to appropriately select the number of pixels to be corrected for the defective pixel to be corrected, particularly the white defect. There is an effect to avoid the decline.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of an image processing apparatus according to the present invention.
FIG. 2 is a part of a flowchart of a main routine in the embodiment of the present invention.
FIG. 3 is a part of a flowchart of a main routine in the embodiment of the present invention.
FIG. 4 is a flowchart of a distance measurement / photometry processing routine in the embodiment of the present invention.
FIG. 5 is a flowchart of a photographing process routine in the embodiment of the present invention.
FIG. 6 is a flowchart of a scratch correction processing routine in the embodiment of the present invention.
FIG. 7 is a flowchart illustrating a correction data selection method used in the defect correction processing according to the embodiment of the present invention.
FIG. 8 is a diagram illustrating a selection method of correction data used in the defect correction processing and the content thereof in the embodiment of the present invention.
FIG. 9 is a flowchart showing a correction data extraction method according to the first embodiment of the present invention.
FIG. 10 is a diagram comparing determination values in the first embodiment and the second embodiment of the present invention.
FIG. 11 is a flowchart showing a correction data extraction method according to the second embodiment of the present invention.
FIG. 12 is a diagram illustrating a configuration of a CCD image sensor.
FIG. 13 is a timing chart showing a driving sequence of the CMOS image sensor.
FIG. 14 is a diagram showing a substantial difference in accumulation time within a screen of a CMOS image sensor.
[Explanation of symbols]
14 Image sensor
20 Image processing circuit
50 System control circuit
100 Image processing apparatus

Claims (6)

Storage means for storing basic correction data having coordinates of defective pixels of the image sensor and defect level information;
Determination value selection means for selecting a determination value according to the coordinates of the image sensor;
Correction data output means for outputting correction data having coordinate information of defective pixels to be corrected from the basic correction data and the determination value;
Correction means for correcting an output signal from a defective pixel of the image sensor using the correction data;
An image processing apparatus comprising: In the correction data output means,
A defective pixel having a defect level in the basic correction data that exceeds the determination value is determined as a defective pixel to be corrected
The image processing apparatus according to claim 1. The judgment value selection means is
The determination value is selected for each of a plurality of coordinate sections set according to the coordinates of the image sensor.
The image processing apparatus according to claim 1, wherein: The image processing apparatus is
Having multiple basic correction data and selecting the basic correction data according to the shooting conditions
The image processing apparatus according to claim 1, wherein: The photographing conditions are at least a charge accumulation period and ISO sensitivity.
The image processing apparatus according to claim 4. A control method for an image processing apparatus comprising storage means for storing basic correction data having coordinates of defective pixels of an image sensor and defect level information,
A determination value selection step of selecting a determination value according to the coordinates of the image sensor;
A correction data output step for outputting correction data having coordinate information of defective pixels to be corrected from the basic correction data and the determination value;
A correction step of correcting an output signal from a defective pixel of the image sensor using the correction data;
A control method for an image processing apparatus, comprising:

Images