JP5014262B2 - Imaging apparatus, control method thereof, and program - Google Patents

Description

  The present invention relates to a technique for suppressing image quality deterioration due to foreign matter adhering to the surface of an optical low-pass filter or the like in an imaging apparatus using an imaging element such as a CCD or CMOS sensor.

  2. Description of the Related Art In recent years, many image pickup apparatuses that generate image signals using an image pickup device such as a CCD and record the data as data, such as digital cameras and digital video cameras, have come into circulation. In a digital camera, a photosensitive film that has been used as a conventional recording medium is no longer necessary, and instead of this, a data image is recorded on a data recording medium such as a semiconductor memory card or a hard disk device. Since these data recording media can be written and erased any number of times unlike films, the cost for consumables can be reduced, which is very convenient.

  Usually, a digital camera is equipped with an LCD (Liquid Crystal Display) monitor device capable of displaying captured images at any time and a detachable mass storage device.

  Using a digital camera equipped with these two devices not only eliminates the need for a film as a recording medium that has been used as a consumable item in the past, but also allows the captured image to be displayed on the LCD monitor device and immediately confirmed on the spot. . Therefore, unsatisfactory image data can be erased on the spot or re-photographed as needed, and the efficiency of photography is dramatically increased compared to film cameras using film. I can say.

Due to such convenience and technological innovations such as an increase in the number of pixels of the image sensor, the range of use of digital cameras has been expanded, and in recent years, there are many digital cameras capable of exchanging lenses, such as a single-lens reflex system.
JP 2004-222231 A

  However, in a digital camera, foreign matter such as dust or dust (hereinafter simply dust) on the surface of the image sensor protective glass fixed to the image sensor, an optical filter, or the optical system (hereinafter collectively referred to as an image sensor optical system component). May adhere. When dust adheres to the image sensor optical system component in this way, there is a problem that the quality of the photographed image is deteriorated, for example, light is blocked by the dust and the portion is not photographed.

  Not only digital cameras but also cameras using silver halide film have a problem of being captured if there is dust on the film, but in the case of film, the film moves from frame to frame, so it is the same for all frames It is very rare for trash to appear.

  However, since the image sensor of the digital camera does not move and images are taken with a common image sensor, once the dust adheres to the image sensor optical system parts, the same dust appears in many frames (photographed images). . Particularly in an interchangeable lens digital camera, there is a problem that dust easily enters the camera when the lens is replaced.

  Therefore, the photographer must always pay attention to the adhesion of dust to the image sensor optical system parts, and has spent a lot of labor on checking and cleaning the dust. In particular, since the image sensor is disposed at a relatively deep location inside the camera, it is not easy to clean or check for dust.

  In addition, with interchangeable lens digital cameras, not only dust can be easily infiltrated by attaching and detaching the lens, but many of the interchangeable lens digital cameras have a focal plane shutter immediately in front of the image sensor. Garbage easily adheres to the top.

  Since such dust on the image sensor usually adheres not to the surface of the image sensor but to the protective glass or optical filter, the imaging state differs depending on the aperture value and pupil position of the photographing lens. That is, if the aperture is close to the open value, the image is blurred, and even if small dust is attached, there is almost no effect. However, if the aperture value is increased, the image is clearly formed and the image is affected.

  Therefore, there is a method to make dust inconspicuous by taking a picture of a white wall etc. with the lens aperture closed and preparing an image showing only dust on the image sensor in combination with a normal shot image. It is known (see Patent Document 1). However, in this method, there is a problem that it is necessary to always be aware of the association between the image captured for dust detection and the actual captured image group associated therewith, which is troublesome.

  In recent years, some interchangeable lens type digital cameras have a function of taking a moving image. As a matter of course, the problem is that dust also appears in an image as with a still image. Further, in the case of moving image shooting, zoom-in / zoom-out can be performed during shooting, which causes a problem that the aperture value and pupil position of the shooting lens change, and the dust image formation state for each frame varies.

  Therefore, the present invention has been made in view of the above-described problems, and the object thereof is to suppress the influence on a captured moving image even when dust adheres to protective glass, a filter, or the like fixed to the image sensor. Is to do so.

In order to solve the above-described problems and achieve the object, an imaging apparatus according to the present invention includes an imaging unit having an imaging element that photoelectrically converts a subject image formed by a photographic lens , and a foreign object obtained from the imaging unit. from the image of the foreign matter contained in the detection image signal, and the foreign matter detection means for detecting the foreign substance information including information of the position and size of the foreign substance even without low, the lens including the information of the aperture value and the pupil position of the photographing lens Lens information acquisition means for acquiring information, and when moving image shooting is performed, moving image data generated based on image signals continuously output from the imaging means is recorded, and the moving image data is recorded in the moving image data before comprising recording means for recording added Kikoto product information and the lenses information, and a zoom operation detection means for detecting the start and end of the zoom operation of the imaging lens, the zoom When the operation detection unit detects the start of the zoom operation of the photographing lens, the recording unit creates a new fragment, and after the zoom operation detection unit detects the start of the zoom operation of the photographing lens, until detecting the end of the zoom operation of the imaging lens, the said lens information acquisition means and updating the lens information, the recording unit adds the updated lens information on the moving image data and the It is characterized by recording in a new fragment .

The image pickup apparatus control method according to the present invention is a method for controlling an image pickup apparatus including an image pickup unit having an image pickup element that photoelectrically converts a subject image formed by a photographing lens , and is obtained from the image pickup unit. from foreign matter images included in the foreign substance detection image signal, comprising the foreign substance detection step of detecting the foreign substance information including information of the position and size of the foreign substance even without less, the information of the aperture value and the pupil position of the photographing lens A lens information acquisition step for acquiring lens information, and when moving image shooting is performed, moving image data generated based on image signals continuously output from the imaging unit is recorded, and the moving image data before Kikoto product information and the lenses information recording step of recording added and the start of the zoom operation of the imaging lens and the zoom operation is detected detects the end stearyl Comprising a flop and, when the start of the zoom operation is detected in the photographing lens in the zoom operation detection step, the create new fragments in the recording step, the zoom operation of the photographing lens by the zoom operation detection step The lens information is updated in the lens information acquisition step and the updated lens information is updated in the recording step from when the start of detection is detected until the end of the zoom operation of the photographing lens is detected. It is added to moving image data and recorded in the new fragment .

  According to the present invention, even when dust adheres to protective glass, a filter, or the like fixed to the image sensor, it is possible to suppress the influence on the captured moving image.

  DESCRIPTION OF EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

(First embodiment)
FIG. 1 is a perspective view showing an appearance of a digital camera 120 according to the first embodiment of the present invention, and FIG. 2 is a vertical sectional view of FIG.

  In FIG. 1, an accessory shoe 110, an optical viewfinder 104, an AE (automatic exposure) lock button 112, an AF (autofocus) distance measuring point selection button 113, and a release button for performing a photographing operation are provided on the upper part of the camera body 100. 114 is provided. An electronic dial 411, a mode dial 60, and an external display device 409 are also provided. The electronic dial 411 is a multi-function signal input device that is used in combination with other operation buttons to input numerical values to the camera and to switch the shooting mode. The external display device 409 includes a liquid crystal display device, and displays shooting conditions such as shutter speed, aperture value, shooting mode, and other information.

  Also, on the back of the camera body 100, an LCD monitor device 417 for displaying captured images and various setting screens, a playback switch 66 for displaying captured images on the LCD monitor device 417, a single / continuous shooting switch 68, A cross placement switch 116, a menu button 124, and a power switch 72 are provided.

  The single-shot / continuous-shot switch 68 is in a single-shot mode in which a single frame is shot and placed in a standby state when a shutter switch SW2 (64), which will be described later, is pressed, and continuously while the shutter switch SW2 (64) is pressed. The continuous shooting mode can be set.

  The cross switch 116 has four buttons arranged vertically and horizontally and a SET button 117 arranged in the center. The user instructs the camera to select and execute menu items displayed on the LCD monitor device 417. Used to do.

  The menu button 124 is a button for causing the LCD monitor device 417 to display a menu screen for performing various camera settings. For example, when selecting and setting the shooting mode, after pressing the menu button 124, the user operates the up / down / left / right buttons of the cross switch 116 to select the desired mode, and the desired mode is selected. Pressing the SET button 117 completes the setting.

  Since the LCD monitor device 417 of the present embodiment is a transmissive type, an image cannot be visually recognized only by driving the LCD monitor device, and a backlight illumination device 416 is always required on the back side as shown in FIG. is there. In this way, the LCD monitor device 417 and the backlight illumination device 416 constitute the image display unit 28 as shown in FIG.

  As shown in FIG. 2, the imaging apparatus according to the present embodiment mainly includes a camera body 100 and an interchangeable lens type lens unit 300. In FIG. 2, 401 is a photographing optical axis.

  In the lens unit 300, 310 is an imaging lens composed of a plurality of lenses, 312 is a diaphragm, and 306 is a lens mount that mechanically couples the lens unit 300 to the camera body 100. The lens unit 300 can be attached to and detached from the camera body 100 via a lens mount 306.

  In the camera body 100, the mirror 130 is disposed in the photographing optical path, and a position for guiding the subject light from the lens unit 300 to the finder optical system (referred to as a position shown in FIG. 2, an oblique position) and a position for retreating from the photographing optical path. (Referred to as a retracted position). The mirror 130 may be either a quick return mirror or a half mirror.

  In FIG. 2, the subject light guided from the mirror 130 to the optical viewfinder 104 is imaged on the focus plate 204. Reference numeral 205 denotes a condenser lens for improving the visibility of the finder. Reference numeral 132 denotes a pentagonal roof prism.

  Reference numerals 209 and 210 respectively denote a rear curtain and a front curtain that constitute a shutter, and the rear and rear curtains 209 and 210 expose the image sensor 14 that is a solid-state image sensor disposed rearward for a necessary time. The image sensor 14 is provided with an optical low-pass filter 418 on the front surface thereof, and adjusts the spatial frequency of the subject image formed on the image sensor 14. Dust (foreign matter) affecting the captured image adheres to the optical low-pass filter 418 and appears as a shadow on the subject image formed on the image sensor 14, thereby degrading the quality of the captured image.

  The image sensor 14 is held on the printed circuit board 211. Behind this printed board 211, a display board 215, which is another printed board, is arranged. An LCD monitor device 417 and a backlight illumination device 416 are disposed on the opposite surface of the display substrate 215.

  Reference numeral 200 denotes a recording medium for recording image data, and 86 denotes a battery (portable power supply). The recording medium 200 and the battery 86 are detachable from the camera body 100.

  FIG. 3 is a block diagram illustrating a circuit configuration of the interchangeable lens digital camera according to the first embodiment of the present invention.

  First, the configuration of the lens unit 300 will be described.

  The lens mount 306 includes various functions for electrically connecting the lens unit 300 to the camera body 100. Reference numeral 320 denotes an interface for connecting the lens unit 300 to the camera body 100 in the lens mount 306, and 322 denotes a connector for electrically connecting the lens unit 300 to the camera body 100.

  The connector 322 has a function of transmitting control signals, status signals, data signals, and the like between the camera body 100 and the lens unit 300 and supplying currents of various voltages. The connector 322 may be configured to transmit not only electrical communication but also optical communication, voice communication, and the like.

  Reference numeral 340 denotes an aperture control unit that controls the aperture 312 in cooperation with a shutter control unit 40 that controls a shutter 12 of the camera body 100 described later based on photometric information from the photometry control unit 46. Reference numeral 342 denotes a focus control unit that controls focusing of the imaging lens 310, and 344 denotes a zoom control unit that controls zooming of the imaging lens 310.

  A lens system control circuit 350 controls the entire lens unit 300. The lens system control circuit 350 includes a memory for storing operation constants, variables, programs, and the like. Further, a non-volatile memory that holds identification information such as a unique number of the lens unit 300, management information, function information such as an open aperture value, a minimum aperture value, and a focal length, and current and past setting values is also provided.

  Next, the configuration of the camera body 100 will be described.

  A lens mount 106 mechanically couples the camera body 100 and the lens unit 300. A shutter 12 includes a rear curtain 209 and a front curtain 210. A light beam incident on the imaging lens 310 is guided by a single-lens reflex system through a diaphragm 312 which is a light amount limiting unit, lens mounts 306 and 106, a mirror 130, and a shutter 12, and photoelectrically converts a subject image as an optical image. Image on top.

  Reference numeral 16 denotes an A / D converter that converts an analog signal output from the image sensor 14 into a digital signal. A timing generation circuit 18 supplies a clock signal and a control signal to the image sensor 14, the A / D converter 16, and the D / A converter 26, respectively, and is controlled by the memory control circuit 22 and the system control circuit 50.

  An image processing circuit 20 performs predetermined pixel interpolation processing and color conversion processing on the data from the A / D converter 16 or the data from the memory control circuit 22. Further, the image processing circuit 20 performs predetermined arithmetic processing using the image data output from the A / D converter 16 as necessary. TTL (through-the-lens) type autofocus (AF) processing and automatic exposure (AE) for the system control circuit 50 to control the shutter control unit 40 and the focus adjustment unit 42 based on the obtained calculation result. Processing and flash pre-emission (EF) processing can be performed. Further, the image processing circuit 20 performs predetermined arithmetic processing using the image data output from the A / D converter 16, and also performs TTL auto white balance (AWB) processing based on the obtained arithmetic result. ing.

  In the example shown in FIG. 3 in the present embodiment, the focus adjusting unit 42 and the photometric control unit 46 are provided exclusively. Therefore, the AF adjustment process, the AE process, and the EF process are performed using the focus adjustment unit 42 and the photometry control unit 46, and the AF process, the AE process, and the EF process using the image processing circuit 20 are not performed. It doesn't matter. Further, AF processing, AE processing, and EF processing are performed using the focus adjustment unit 42 and the photometry control unit 46, and further, AF processing, AE processing, and EF processing using the image processing circuit 20 are performed. It is good also as a structure.

  A memory control circuit 22 controls the A / D converter 16, the timing generation circuit 18, the image processing circuit 20, the image display memory 24, the D / A converter 26, the memory 30, and the compression / decompression circuit 32. Image data output from the A / D converter 16 is written into the image display memory 24 or the memory 30 via the image processing circuit 20, the memory control circuit 22, or only through the memory control circuit 22.

  Reference numeral 24 denotes an image display memory, 26 denotes a D / A converter, 28 denotes an image display unit including an LCD monitor device 417, a backlight illumination device 416, and the like. The display image data written in the image display memory 24 is D The image is displayed by the image display unit 28 via the / A converter 26. An electronic viewfinder (EVF) function can be realized by sequentially displaying captured image data using the image display unit 28. Further, the image display unit 28 can arbitrarily turn on / off the display according to an instruction from the system control circuit 50, and when the display is turned off, the power consumption of the camera body 100 can be significantly reduced. it can.

  Reference numeral 30 denotes a memory for storing captured still images, which has a storage capacity sufficient to store a predetermined number of still images. This makes it possible to write a large amount of images to the memory 30 at high speed even in continuous shooting or panoramic shooting in which a plurality of still images are continuously shot. In moving image shooting, it is used as a frame buffer for images written continuously at a predetermined rate. Further, the memory 30 can be used as a work area for the system control circuit 50.

  A compression / decompression circuit 32 compresses / decompresses image data using a known compression method. The compression / decompression circuit 32 reads an image stored in the memory 30, performs a compression process or an expansion process, and writes the processed data to the memory 30 again.

  Reference numeral 40 denotes a shutter control unit that controls the shutter 12 in cooperation with an aperture control unit 340 that controls the aperture 312 based on photometric information from the photometry control unit 46. A focus adjusting unit 42 performs AF (autofocus) processing. A light beam incident on the imaging lens 310 in the lens unit 300 is incident as a single lens reflex system through a diaphragm 312, lens mounts 306 and 106, a mirror 130, and a focus adjustment sub-mirror (not shown), thereby forming an optical image. The in-focus state of the captured image is measured.

  A photometry control unit 46 performs AE (automatic exposure) processing. A light beam incident on the imaging lens 310 in the lens unit 300 is incident as a single-lens reflex system through a diaphragm 312, lens mounts 306 and 106, a mirror 130, and a photometric sub-mirror (not shown), thereby forming an optical image. Measure the exposure of the image. A flash 48 has an AF auxiliary light projecting function and a flash light control function. The photometry control unit 46 also has an EF (flash dimming) processing function in cooperation with the flash 48.

  Further, the AF control may be performed using the measurement result by the focus adjustment unit 42 and the calculation result obtained by calculating the image data from the A / D converter 16 by the image processing circuit 20. Furthermore, exposure control may be performed using the measurement result obtained by the photometry control unit 46 and the calculation result obtained by calculating the image data from the A / D converter 16 by the image processing circuit 20.

  Reference numeral 50 denotes a system control circuit that controls the entire camera body 100, and incorporates a known CPU. A memory 52 stores constants, variables, programs, and the like for operating the system control circuit 50.

  Reference numeral 54 denotes a notification unit for notifying the outside of an operation state, a message, and the like using characters, images, sounds, and the like in accordance with execution of a program in the system control circuit 50. As the notification unit 54, for example, a display unit that performs visual display using an LCD, an LED, or the like, or a sound generation element that performs voice notification, etc., is used, and the notification unit 54 is configured by a combination of one or more of these. In particular, in the case of a display unit, for example, as in the case of the external display device 409, the display unit is installed at a single or a plurality of locations in the vicinity of the operation unit 70 of the camera body 100, which is easy to see. In addition, the notification unit 54 has a part of its functions installed in the optical viewfinder 104.

  Among the display contents of the notification unit 54, the following are displayed on the image display unit 28 such as an LCD. First, there are displays relating to shooting modes such as single-shot / continuous-shot shooting display, self-timer display, and the like. In addition, there are displays relating to recording such as compression rate display, recording pixel number display, recording number display, and remaining image number display. In addition, there are displays relating to photographing conditions such as shutter speed display, aperture value display, exposure correction display, dimming correction display, external flash emission amount display, and red-eye reduction display. In addition, there are a macro shooting display, a buzzer setting display, a battery remaining amount display, an error display, an information display with a multi-digit number, and a recording medium 200 and PC 210 attachment / detachment state display. Furthermore, the attachment / detachment state display of the lens unit 300, the communication I / F operation display, the date / time display, the display indicating the connection state with the external computer, and the like are also performed.

  In addition, among the display contents of the notification unit 54, examples of what is displayed in the optical finder 104 include the following. In-focus display, shooting preparation completion display, camera shake warning display, flash charge display, flash charge completion display, shutter speed display, aperture value display, exposure correction display, recording medium writing operation display, and the like.

  Reference numeral 56 denotes an electrically erasable / recordable non-volatile memory in which programs and the like to be described later are stored. For example, an EEPROM or the like is used.

  Reference numerals 60, 62, 64, 66, 68, and 70 denote operation means for inputting various operation instructions of the system control circuit 50. A single unit such as a switch, a dial, a touch panel, pointing by line-of-sight detection, a voice recognition device, or the like. Alternatively, it is composed of a plurality of combinations.

  Here, a specific description of these operating means will be given.

  Reference numeral 60 denotes a mode dial, which can switch and set various function shooting modes such as an automatic shooting mode, a program shooting mode, a shutter speed priority shooting mode, an aperture priority shooting mode, a manual shooting mode, and a depth of focus priority (depth) shooting mode. . In addition, each function shooting mode such as portrait shooting mode, landscape shooting mode, close-up shooting mode, sports shooting mode, night view shooting mode, panoramic shooting mode, and moving image shooting mode can be switched and set.

  Reference numeral 62 denotes a shutter switch SW1, which is turned ON during the operation of the release button 114 (for example, half-pressed), and instructs to start operations such as AF processing, AE processing, AWB processing, and EF processing.

  Reference numeral 64 denotes a shutter switch SW2, which is turned on when the operation of the release button 114 is completed (for example, fully pressed), and instructs the start of a series of processing including exposure processing, development processing, and recording processing. First, in the exposure process, a signal read from the image sensor 14 is written to the memory 30 via the A / D converter 16 and the memory control circuit 22, and further, an operation in the image processing circuit 20 and the memory control circuit 22 is used. Development processing is performed. Further, in the recording process, the image data is read from the memory 30, compressed by the compression / decompression circuit 32, and written or transmitted to the recording medium 200 or the PC 210.

  Reference numeral 66 denotes a playback switch, which instructs to start a playback operation for reading an image shot in the shooting mode state from the memory 30, the recording medium 200, or the PC 210 and displaying it on the image display unit 28. In addition, each function mode such as a playback mode, a multi-screen playback / erase mode, and a PC connection mode can be set.

  68 is a single-shot / continuous-shot switch. When the shutter switch SW2 (64) is pressed, a single-shot mode in which one frame is shot and the camera is in a standby state, and while the shutter switch SW2 (64) is being pressed, continuous shooting is performed. The continuous shooting mode can be set.

  An operation unit 70 includes various buttons and a touch panel. As an example, a live view start / stop button, a moving image recording start / stop button, a menu button 124, a SET button 117, a multi-screen playback page break button, a flash setting button, a single shooting / continuous shooting / self-timer switching button, and a cross switch 116, an AE (automatic exposure) lock button 112, an AF AF point selection button 113, and an electronic dial 411. Furthermore, a playback image movement + (plus) button, a playback image movement-(minus) button, a photographing image quality selection button, an exposure correction button, a light adjustment correction button, an external flash emission amount setting button, a date / time setting button, and the like are also included. The functions of the up / down / left / right buttons of the cross-arrangement switch 116 can be more easily selected by providing a rotary dial switch.

  In addition, there is an image display ON / OFF switch for setting ON / OFF of the image display unit 28, and a quick review ON / OFF switch for setting a quick review function for automatically reproducing image data taken immediately after shooting. Further, there is a compression mode switch that is a switch for selecting a compression rate for JPEG compression or for selecting a RAW mode in which a signal from an image sensor is directly digitized and recorded on a recording medium. In addition, there is an AF mode setting switch that can set a one-shot AF mode and a servo AF mode. In the one-shot AF mode, an autofocus operation is started when the shutter switch SW1 (62) is pressed, and once focused, the focused state is maintained. In the servo AF mode, the autofocus operation is continued while the shutter switch SW1 (62) is being pressed. Further, as will be described later, it includes a setting switch that can set a dust information acquisition mode for capturing dust information by capturing dust detection images.

  Reference numeral 72 denotes a power switch, which can switch and set the power-on and power-off modes of the camera body 100. Also, the power on and power off settings of various accessory devices such as the lens unit 300, the external flash 112, the recording medium 200, and the PC 210 connected to the camera body 100 can be switched.

  A power control unit 80 includes a battery detection circuit, a DC-DC converter, a switch circuit that switches a block to be energized, and the like. The power supply control unit 80 detects the presence / absence of a battery, the type of battery, and the remaining battery level, controls the DC-DC converter based on the detection result and the instruction of the system control circuit 50, and requires the necessary voltage. It is supplied to each part including the recording medium for a period.

  Reference numerals 82 and 84 denote connectors, and 86 denotes a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, NiMH battery, Li-ion battery, or Li polymer battery, an AC adapter, or the like.

  Reference numerals 90 and 94 denote recording media such as a memory card and a hard disk, and an interface with a PC. Reference numerals 92 and 96 denote connectors for connecting the recording medium such as a memory card and a hard disk and the PC. A recording medium attachment / detachment detection circuit 98 detects whether the recording medium 200 or the PC 210 is attached to the connectors 92 and / or 96.

  Although the present embodiment has been described as having two systems of interfaces and connectors for attaching the recording medium, the interface and connectors for attaching the recording medium may have a single or a plurality of systems. Moreover, it is good also as a structure provided with combining the interface and connector of a different standard.

  As the interface and the connector, it is possible to configure using interfaces that comply with various storage medium standards. For example, a PCMCIA (Personal Computer Memory Card International Association) card, a CF (Compact Flash (registered trademark)) card, an SD card, or the like. When the interfaces 90 and 94 and the connectors 92 and 96 are configured using a standard conforming to a PCMCIA card or a CF card, various communication cards can be connected. Communication cards include LAN cards, modem cards, USB (Universal Serial Bus) cards, and IEEE (Institute of Electrical and Electronic Engineers) 1394 cards. In addition, there are P1284 card, SCSI (Small Computer System Interface) card, PHS, and the like. By connecting these various communication cards, image data and management information attached to the image data can be transferred to and from other computers and peripheral devices such as a printer.

  The optical viewfinder 104 can guide the light beam incident on the imaging lens 310 through the diaphragm 312, the lens mounts 306 and 106, and the mirrors 130 and 132 by the single-lens reflex method, and can display the image as an optical image. . Thus, it is possible to take an image using only the optical viewfinder without using the electronic viewfinder function of the image display unit 28. Further, in the optical viewfinder 104, some functions of the notification unit 54, for example, in-focus state, camera shake warning, flash charging, shutter speed, aperture value, exposure correction, and the like are displayed.

  Reference numeral 112 denotes an external flash device mounted via the accessory shoe 110.

  Reference numeral 120 denotes an interface for connecting the camera body 100 to the lens unit 300 in the lens mount 106.

  A connector 122 electrically connects the camera body 100 to the lens unit 300. Further, whether or not the lens unit 300 is attached to the lens mount 106 and the connector 122 is detected by a lens attachment / detachment detection unit (not shown). The connector 122 transmits a control signal, a status signal, a data signal, and the like between the camera body 100 and the lens unit 300, and has a function of supplying currents of various voltages.

  Various optical information (aperture, zoom position, pupil position, focal length, etc.) of the lens unit 300 communicated via the connector 122 is stored in the memory 30 of the camera body 100. The communication request may be made from the camera body 100 side, or may be communicated from the lens unit 300 side every time information is updated.

  Further, the connector 122 may be configured to transmit not only electrical communication but also optical communication, voice communication, and the like.

  Reference numeral 200 denotes a recording medium such as a memory card or a hard disk. The recording medium 200 includes a recording unit 202 composed of a semiconductor memory, a magnetic disk, and the like, an interface 204 with the camera body 100, and a connector 206 that connects to the camera body 100.

  As the recording medium 200, a memory card such as a PCMCIA card or compact flash (registered trademark), a hard disk, or the like can be used. Of course, it may be composed of a micro DAT, a magneto-optical disk, an optical disk such as a CD-R or CD-RW, a phase change optical disk such as a DVD.

  Reference numeral 210 denotes a PC, which includes a recording unit 212 composed of a magnetic disk (HD) or the like, an interface 214 with the camera body 100, and a connector 216 for connecting with the camera body 100. Examples of the interface 94 include USB and IEEE1394, but are not particularly limited.

(Dust detection process)
In the present embodiment, first, dust information (foreign matter information) that is information on dust (foreign matter) attached to the optical low-pass filter 418 arranged in front of the image sensor 14 (information on dust in the imaging screen) is obtained. A dust detection image (foreign object detection image signal) is taken. Then, dust data is extracted and dust correction data (dust information profile) is generated. Here, the dust detection image is preferably an image obtained by photographing a luminance surface that is as uniform as possible. However, since it is desirable that the dust detection image can be easily photographed in a familiar place, strict uniformity is not required. For example, it is assumed that a blue sky or a white wall is photographed.

  First, an example of processing for detecting the position of dust attached to the optical low-pass filter 418 will be described with reference to the flowchart of FIG. This processing is performed by the system control circuit 50 executing a dust detection processing program stored in the nonvolatile memory 56.

  The dust detection process is performed by capturing a dust detection image. When performing dust detection processing, a camera is installed with the photographing optical axis 401 of the lens unit 300 facing a surface having a uniform color, such as an emission surface of a surface light source device or a white wall, and preparation for dust detection is performed. Alternatively, a dust detection light unit (a small point light source device mounted instead of a lens) is mounted on the lens mount 106 to prepare for dust detection. For example, a white LED is considered as the light source of the light unit, and it is desirable to adjust the size of the light emitting surface so as to correspond to a predetermined aperture value (for example, F32).

  In this embodiment, a case where a normal photographing lens is used will be described. However, dust detection may be performed by attaching the light unit to the lens mount 106. Thus, in this embodiment, the dust detection image is an image having a uniform color.

  After the preparation is completed, when the start of dust detection processing is instructed from the cross arrangement switch 116, the system control circuit 50 first sets the aperture. The image formation state of dust near the image sensor changes depending on the aperture value of the lens, and the position changes depending on the pupil position of the lens. Therefore, in the dust correction data, in addition to the position and size of dust, it is necessary to hold the aperture value at the time of detection and the pupil position of the lens.

  However, it is not always necessary to hold the aperture value in the dust correction data if it is determined in advance that the same aperture value is always used even when different lenses are used at the stage of creating dust correction data. Similarly, by using a light unit or permitting the use of only a specific lens for the pupil position, it is not always necessary to hold the pupil position in the dust correction data.

  In other words, at the stage of creating dust correction data, if you want to allow multiple lenses to use or change the aperture value to narrow down appropriately, it is necessary to keep the aperture value and lens pupil position at the time of detection in the dust correction data It can be said that there is. Here, the pupil position refers to the distance from the imaging plane (focal plane) of the exit pupil.

  Here, for example, F22 is designated (step S21).

  Next, the system control circuit 50 causes the aperture controller 340 to control the aperture blades of the lens unit 300 via the connector 122, and sets the aperture to the aperture value specified in step S21 (step S22). Further, the focus control unit 342 is instructed to set the focus position to infinity (step S23).

  When the aperture value and focus position of the photographing lens are set, photographing in the dust detection mode is executed (step S24). Details of the imaging processing routine performed in step S24 will be described later with reference to FIG. The captured image data is stored in the memory 30.

  When shooting is completed, the aperture value and lens pupil position at the time of shooting are acquired (lens information acquisition) (step S25). Data corresponding to each pixel of the captured image stored in the memory 30 is called into the image processing circuit 20 (step S26). The image processing circuit 20 performs the processing shown in FIG. 6 and acquires the position and size of the pixel where dust is present (step S27). The position and size of the pixel where the dust exists acquired in step S27, and the aperture value and lens pupil position information acquired in step S25 are registered in the nonvolatile memory 56 (step S28). Here, when the light unit described above is used, lens information cannot be acquired. Therefore, when the lens information cannot be acquired, it is determined that the light unit is used, and predetermined lens pupil position information and a converted aperture value calculated from the light source diameter of the light unit are registered.

  Here, in step S28, the position of the defective pixel (pixel defect) from the time of manufacture recorded in advance in the nonvolatile memory 56 is compared with the position of the read pixel data to confirm whether the pixel defect is present. Only the area determined not to be due to the pixel defect may be registered in the nonvolatile memory 56.

  An example of the data format of dust correction data stored in the nonvolatile memory 56 is shown in FIG. As shown in FIG. 5, the dust correction data stores lens information and dust position and size information when the detection image is captured.

  Specifically, the actual aperture value (F value) at the time of detection image capture and the lens pupil position at that time are stored as lens information at the time of detection image capture. Subsequently, the number of detected dust areas (integer value) is stored in the storage area, and subsequently, individual specific dust area parameters are repeatedly stored by the number of dust areas. The parameter of the dust area is a set of three numerical values: a dust radius (for example, 2 bytes), a center x coordinate (for example, 2 bytes) in the effective image area, and a similar y coordinate (for example, 2 bytes).

  When the dust correction data size is limited due to the size of the nonvolatile memory 56 or the like, the data is stored with priority from the beginning of the dust area obtained in step S27. This is because in the dust area acquisition routine in step S27, the dust areas are sorted in the order of noticeable dust, as will be described later.

(Trash area acquisition routine)
Next, details of the dust region acquisition routine performed in step S27 will be described with reference to FIGS.

  As shown in FIG. 7, the called image data is expanded on the memory 30 and processed in units of predetermined blocks. This is to cope with peripheral dimming caused by lens and sensor characteristics. Peripheral dimming is a phenomenon in which the luminance at the peripheral portion is lower than that at the central portion of the lens, and it is known that it is reduced to some extent by reducing the aperture of the lens. However, there is a problem that even if the aperture is reduced, if the dust position is determined based on a predetermined threshold value for the photographed image, dust in the peripheral portion cannot be detected accurately depending on the lens. Therefore, the image is divided into blocks to reduce the influence of peripheral dimming.

  When the blocks are simply divided, there is a problem that the detection result of the dust straddling the blocks is shifted when the threshold values are different between the blocks. Therefore, the blocks are overlapped, and pixels determined to be dust in any of the blocks constituting the overlap region are treated as dust regions.

  The determination of the dust area in the block is performed according to the processing flow shown in FIG. First, the maximum luminance Lmax and average luminance Lave in the block are calculated, and the threshold value T1 in the block is calculated using the following equation.

T1 = Lave × 0.6 + Lmax × 0.4
Next, pixels that do not exceed the threshold value are defined as dust pixels (step S61), and isolated regions constituted by dust pixels are each defined as one dust region di (i = 0, 1,..., N) (step S62). . As shown in FIG. 8, for each dust region, the maximum value Xmax and the minimum value Xmin of the horizontal coordinate of the pixels constituting the dust region, the maximum value Ymax and the minimum value Ymin of the vertical coordinate are obtained, and the dust region di is obtained. A radius ri representing the size of is calculated by the following equation (step S63).

ri = √ [{(Xmax−Xmin) / 2} ² + {(Ymax−Ymin) / 2} ² ]
FIG. 8 shows the relationship between Xmax, Xmin, Ymax, Ymin and ri.

  Thereafter, in step S64, an average luminance value for each dust region is calculated.

  The data size of the dust correction data may be limited due to the limitation due to the size of the nonvolatile memory 56 or the like. In order to cope with such a case, the dust position information is sorted according to the size and the average luminance value of the dust region (step S65). In this embodiment, sorting is performed in descending order of ri. When ri is equal, the sort is performed in ascending order of the average luminance value. In this way, it is possible to preferentially register noticeable dust in the dust correction data. The sorted dust area is Di, and the radius of the dust area Di is Ri.

  If there is a dust area larger than a predetermined size, it may be excluded from sorting and placed at the end of the sorted dust area list. This is because a large dust region may lower the image quality if interpolation processing is performed later, and is preferably treated as the lowest priority for correction.

(Imaging processing routine)
Next, the details of the imaging processing routine performed in step S24 of FIG. 4 will be described using the flowchart shown in FIG. This process is performed by the system control circuit 50 executing an imaging process program stored in the nonvolatile memory 56.

  When this imaging processing routine is executed, in step S201, the system control circuit 50 operates the mirror 130 shown in FIGS. 2 and 3, performs so-called mirror-up, and retracts the mirror 130 out of the imaging optical path.

  Next, in step S202, charge accumulation in the image sensor 14 is started, and in the next step S203, the shutter 12 shown in FIG. In step S204, the charge accumulation of the image sensor 14 is terminated. In the next step S205, the image signal is read from the image sensor 14, and the image data processed by the A / D converter 16 and the image processing circuit 20 is temporarily stored in the memory 30. Remember.

  When the reading of all image signals from the image sensor 14 is completed in the next step S206, the mirror 130 is mirrored down in step S207, the mirror is returned to the oblique position, and a series of imaging operations is completed.

  In step S208, it is determined whether normal shooting or dust detection image shooting is performed. When normal shooting is performed, the process proceeds to step S209, and the shot image is recorded on the recording medium 200.

(Dust removal process)
Next, a flow of operations for removing dust by image processing on a still image file using the dust correction data described above will be described with reference to FIG. In the present embodiment, dust removal processing is performed on a moving image file. However, a method similar to dust removal processing on a still image file can be applied to a moving image file. Debris removal processing will be described.

  First, a still image file to be subjected to dust removal processing is designated, and is read by a device that performs dust removal processing (either the image processing circuit 20 in the camera or an image processing device outside the camera) (step S1801).

  Next, the apparatus that performs dust removal processing acquires the dust correction data created in step S65 of FIG. 6 (step S1802).

  Next, from the dust correction data acquired in step S1802, the coordinate sequence Di (i = 1, 2,... N), the radius sequence Ri (i = 1, 2,..., N), the aperture value f1, and the lens pupil position L1. Is obtained (step S1803). Here, Ri is the size of the dust at the coordinate Di calculated in step S65 of FIG. In step S1804, the aperture value f2 and the pupil position L2 of the lens are acquired. In step S1805, Di is converted by the following equation. Here, d is a distance from the image center to the coordinate Di, and H is a distance between the surface of the image sensor 14 and dust. The coordinate Di ′ after conversion and the radius Ri ′ after conversion are defined by the following equation, for example.

Di ′ (x, y) = (L2 × (L1−H) × d / ((L2−H) × L1)) × Di (x, y)
Ri ′ = (Ri × f1 / f2 + 3) (1)
The unit here is a pixel, and “+3” for Ri ′ is a margin amount.

  In step S1806, dust in the area indicated by the coordinates Di ′ and the radius Ri ′ is detected, and interpolation processing is applied as necessary. Details of the interpolation processing will be described later. In step S1807, it is determined whether dust removal processing has been applied to all coordinates. If processing has been completed for all coordinates, the processing ends. If not, processing returns to step S1806.

  Next, details of dust region interpolation processing will be described. FIG. 11 is a flowchart showing the flow of the interpolation routine.

First, in step S1901, dust region determination is performed. The dust area is an area that satisfies all of the following conditions.
(1) Next, using the average luminance Yave and the maximum luminance Ymax of the pixels included in the center coordinate Di ′ and radius Ri ′ (Di ′, Ri ′ obtained in Expression (1)) calculated in step S1805 in FIG. An area darker than the threshold value T2 obtained by the equation.

T2 = Yave × 0.6 + Ymax × 0.4
(2) A region not in contact with the circle having the center coordinate Di ′ and the radius Ri ′.
(3) A region in which the radius value calculated by the same method as in step S63 in FIG. 6 is not less than l1 pixels and less than l2 pixels with respect to the isolated region constituted by the low-luminance pixels selected in (1).
(4) A region including the center coordinates Di of the circle.

  In this embodiment, l1 is 3 pixels and l2 is 30 pixels. In this way, only an isolated small area can be handled as a dust area. If the lens pupil position cannot be obtained accurately, the condition (4) may have a width. For example, if the region of interest includes coordinates within a range of ± 3 pixels in the X direction and the Y direction from the coordinate Di, a condition such as determining as a dust region can be considered.

  If there is such an area in step S1902, the process proceeds to step S1903, and dust area interpolation is performed. If there is no such area, the process ends. The dust region interpolation processing executed in step S1903 is performed by a known defect region interpolation method. As a known defect area interpolation method, for example, there is a pattern replacement disclosed in Japanese Patent Laid-Open No. 2001-223894. In Japanese Patent Laid-Open No. 2001-223894, a defective region is specified using infrared light. In this embodiment, the dust region detected in step S1901 is treated as a defective region, and the dust region is treated as a normal pixel by pattern replacement. Interpolate with. For pixels that are not filled by pattern replacement, p normal pixels are selected in the order closest to the interpolation target pixel from the image data after pattern interpolation, and interpolation is performed using the average color.

  Next, MP4, which is a moving image file format used in recent years for recording moving image data in digital cameras and digital video cameras, will be described.

  MP4 file format (ISO / IEC 14496-14; “Information technology—Coding of audio-visual objects—Part 14: MP4 file format”; ISO / IEC; see 2003-11-24) In order to record video / audio content data such as MPEG standardized by JTC1 / SC29 / WG11 (International Organization for Standardization / International Engineering Consortium) in a file, “ISO Base Media File 49 EC / F14” 12; “Inform tion technology - Coding of audio-visual objects - Part 12: ISO base media file format "; is a reference to 2004-01-23) file format, which has been extended on the basis of the file format of the general purpose of; ISO / IEC. Note that the present embodiment is not limited to MP4 and can be applied to a case using a similar file format. For example, ISO standard file formats such as “Motion JPEG 2000 file format” (ISO / IEC 15444-3) and “AVC file format” (ISO / IEC 14496-15) are standard file formats having the same basic structure as MP4. Has been enacted.

  FIG. 12 is a conceptual diagram for explaining the data structure in the MP4 file format.

  The MP4 file 1001 is composed of metadata (header information) 1002 indicating the physical position, temporal position and characteristic information of the video / audio data, and media data 1003 indicating the actual state of the encoded video / audio data. The In the MP4 format, the presentation of the entire content is called “movie”, and the presentation of the media stream constituting the content is called “track”. The metadata 1002 typically includes a video track 1004 that logically handles the entire moving image data and an audio track 1005 that logically handles the entire audio data. The basic configuration contents of the video track 1004 and the audio track 1005 are almost the same. That is, each track records various metadata information of actual media data, and the contents thereof are only slightly different depending on the characteristics of the media data.

  The data included in the video track 1004 includes, for example, so-called decoder configuration information for decoding encoded data and information such as a rectangular size of a moving image. In addition, an offset 1006 indicating the position on the file where the media data is actually recorded, a sample size 1007 indicating the size of each frame data (sometimes referred to as a picture) of the media data, and each frame data A time stamp 1008 indicating the decoding time is recorded.

  On the other hand, the media data 1003 includes moving picture data and audio data by means of a data structure called “chunk” in which one or more “samples” indicating the basic unit of encoded data are continuously recorded. It is recorded. This chunk is composed of a video chunk 1009 including moving image media data and an audio chunk 1010 including audio media data in accordance with the track of the metadata 1002.

  The configuration shown in FIG. 12 shows that video chunks 1009 and audio chunks 1010 are recorded alternately, but the recording position and order are not necessarily in this way. This example is merely an example of a format that is generally recorded. However, such alternate arrangement (interleaving) has the effect of improving the accessibility of the data recorded in the file by arranging the video and audio data to be played back at approximately the same time at close positions. This is a very common method.

  A chunk contains one or more samples of individual media data. For example, as shown in FIG. 12, video samples (frames) 1011 are continuously recorded in the video chunk 1009. In general, this video sample (frame) 1011 corresponds to one frame data (picture) of video. Each track and chunk is related as follows.

  For example, in the case of moving image data, the information included in the video track 1004 includes information on each video chunk 1009 included in the media data 1003. The offset 1006 is composed of a table of information indicating the relative position of each video chunk 1009 on the file, and by referring to the individual entries in the table, the actual video chunk is recorded at any position. You can know its position. The sample size 1007 describes a plurality of samples included in a plurality of chunks, that is, a size of each video frame as a table. More precisely, information describing the number of samples included in each individual chunk is also described in the video track 1004. From this information, the samples included in each individual video chunk 1009 are accurately determined. It is possible to get to. The time stamp 1008 records the decoding time of each sample in a table as a difference between samples. By referring to this table, the so-called time stamp of each sample can be obtained by calculating the accumulated time. Such a relationship between the track and the chunk is defined so that the audio track 1005 and the audio chunk 1010 are similarly established. As a result, in the MP4 file and the ISO Base Media File Format, it is possible to acquire encoded data from an arbitrary position with additional information such as a time stamp in a necessary unit using the metadata 1002 and the media data 1003. It has become. In order to simplify the explanation, all the standardized recording information is not described here. Details of the standardized definition contents can be known by referring to the corresponding part of ISO / IEC 14496.

In the MP4 file format, data recorded in the file is described in a data structure called “BOX”, and is recorded in the file in units of BOX. The BOX is composed of the following fields.
Size: the size of the entire BOX, including the size field itself.
Type: 4-byte type identifier representing the type of BOX. Usually expressed in 4 alphanumeric characters.

  Since the other fields are optional depending on the BOX, the description is omitted here.

  Data recorded in the file is held in different types of BOX depending on the type. For example, the media data 1003 is a Media Data BOX for storing encoded data (the content of the type field is 'mdat', and if an identifier indicating the BOX type is used in the following description, the media data 1003 represents the BOX indicated by that type. The metadata 1002 is recorded as a Movie BOX ('moov') that stores metadata information of the entire content. Information on the above chunks and samples is also recorded for each track in the moov as a BOX having a unique identifier.

  In addition, the MP4 file format allows not only a form in which all metadata is recorded in moov but also a form in which metadata is divided and recorded in a plurality of areas in time series. This format is called “Fragmented Movie”.

  FIG. 13 shows the structure of a fragment movie format file. In the fragment movie format, the media data and metadata of content can be divided in arbitrary time units, and the divided “fragments” are recorded in chronological order from the beginning of the file. For example, in FIG. 13, moov 1101 indicates the metadata of the first fragment, and holds information regarding the data included in mdat 1102. The next appearing moof 1103 is the metadata of the second fragment, and is recorded in the same manner so as to hold the information of mdat 1104. When the fragment movie format is adopted, it is necessary to add a Movie Extends Box ('mvex') 1104 indicating that a fragment exists in the moov 1101. The information included in mvex 1104 is the duration (time length) of the entire content including all fragments. As described above, in the MP4 file format file, various attributes relating to the media data are stored as metadata areas separately from the media data, so that the desired data can be obtained regardless of how the media data is physically stored. It is possible to easily access the sample data.

  Hereinafter, the moving image file format used for recording moving images and audio data in the present embodiment is a fragment movie format as shown in FIG. 13 of the MP4 format. A method of associating the dust correction data with the video sample (frame) 1011 at the time of moving image recording will be described.

  Standards such as the aforementioned “Motion JPEG 2000 file format” (ISO / IEC 15444-3) and “AVC file format” (ISO / IEC 14496-15), and wireless terminals centering on third-generation mobile phones 3GPP file format (Through Generation Partnership Project) (3GPP TS 26.244 “Technical Specialist Group Transport Services and Systems Specified Spectral Spects and Systems Specified Spectral Spects and Systems). service (PSS); 3GPP file format (3G ) (Release 6) "3rd Generation Partnership Project; see 2003-02-28), and other standards that employ file formats and architectures similar to those defined in MP4 may be applied to the present invention. Is possible.

(Video shooting routine)
Next, the operation at the time of moving image shooting in the present embodiment will be described.

  This process is performed by the system control circuit 50 executing a moving image capturing process program stored in the nonvolatile memory 56.

  In order to shoot a moving image, it is first necessary to change the shooting mode from the still image shooting mode to the moving image shooting mode using the mode dial 60 or the like.

  When this moving image shooting routine is executed, the system control circuit 50 operates the quick return mirror 130 shown in FIG. 3, performs so-called mirror up, and retracts the quick return mirror 130 out of the shooting optical path. Then, the shutter 12 is opened, and the image sensor 14 is exposed to subject light. The image data generated by exposing the image sensor 14 is continuously written in the memory 30 acting as a frame buffer at a predetermined rate. The LCD monitor device 417 functions as an electronic viewfinder (EVF), and the written image data is sequentially displayed. When the moving image recording start button is turned ON in the moving image shooting mode (for example, when the SET button 117 is pressed in the moving image shooting mode), the moving image shooting is started, and the image data is sequentially converted into the MP4 file format. To record on the recording medium 200.

  When movie recording is started by turning on the movie recording start button in the movie shooting mode, a new file is first generated, and mov which is the BOX of the first fragment metadata and mdat which is the BOX of the media data. create.

  Next, dust position correction data is created. In the dust position correction data, an aperture value, which is lens information of a lens used for moving image shooting, lens pupil position information, and dust correction data shown in FIG. 5 are stored. The created dust position correction data is stored in the memory 52. The dust position correction data stored in the memory 52 is read and written in the metadata moov of the current fragment.

  FIG. 14 is a flowchart showing an operation when the lens unit 300 is zoomed during moving image shooting.

  In the present embodiment, the following processing is performed when the system control circuit 50 detects that the zoom drive is started from the zoom control unit 344 during moving image shooting.

  When it is detected that zoom driving has started, the system control circuit 50 creates a new fragment (step S1401). Note that the captured moving image data is divided and recorded in fragment units.

  Next, the system control circuit 50 exposes the image sensor 14 to perform moving image imaging processing, and stores the generated moving image data in the memory 30. The image processing circuit 20 sequentially performs image processing for each frame of moving image data and records it in the memory 30 (step S1402).

  Next, the system control circuit 50 receives information on whether or not the lens is being zoom-driven from the zoom control unit 344, and determines whether or not the lens unit 300 is being zoom-driven (zooming operation) ( Step S1403).

  If it is determined in step S1403 that zoom driving is being performed, the system control circuit 50 acquires lens information. The lens information is information on the aperture value and the pupil position (step S1404).

  It is determined whether the lens information of the current frame acquired in step S1404 has changed from the lens information of the previous frame (step S1405).

  If the lens information changes in step S1405, the lens information of the current frame is recorded in the metadata moof of the current fragment (step S1406).

  If no change in lens information is detected in step S1405, the fragment is not updated, moving image shooting, image processing, and compression processing are performed, and moving image data is written in the mdat of the current fragment.

  During zoom driving, the lens information that changes is written in the metadata moof of one fragment. Therefore, when the lens information is recorded in the header portion, information such as the number of frames indicating the range of frames corresponding to the same lens information is simultaneously written. Note that the information indicating the frame range corresponding to the same lens information is not limited to the number of frames, and may be other information as long as the information can clarify the lens information and the corresponding frame.

  If it is determined in step S1403 that the zoom drive is not being performed, the fragment is divided to newly create a fragment, and the operation during the zoom drive is ended.

  The above series of processing (steps S1402 to S1406) is repeated until it is determined that the zoom driving is completed.

  During moving image reproduction, dust conversion is performed by reading lens information corresponding to each frame from the fragment moof when converting dust correction parameters in step S1805 in the dust removal processing of FIG.

  According to the first embodiment described above, the following effects can be obtained.

  By attaching dust correction data to an image as described above, there is an advantage that it is not necessary to be aware of the correspondence between dust correction image data and captured image data. Further, since the dust correction data is compact data composed of position, size, and conversion data (aperture value, lens pupil position information), the captured image data size does not become extremely large. Further, by performing interpolation processing only on the region including the pixel specified by the dust correction data, the probability of erroneous detection can be greatly reduced.

  In addition, since the plurality of pieces of lens information is recorded in one fragment without dividing the fragment during the zoom driving of the lens, the fragment is not divided more than necessary, and the data amount can be reduced.

  Further, the fact that the fragment is not divided can reduce the load in the reproduction process of the moving image file.

(Second Embodiment)
In the first embodiment described above, lens information is acquired at the time of moving image recording for each frame during zoom driving of the lens, and lens information is recorded when there is a change in the lens information.

  On the other hand, in the second embodiment, lens information is recorded only at the start and end of zoom driving of the lens. Since the configuration of the digital camera according to the second embodiment is the same as that of the first embodiment, description thereof will be omitted, and only operations different from those of the first embodiment will be described.

  FIG. 15 is a flowchart showing a moving image recording process during zoom driving in the present embodiment.

  In the present embodiment, the following processing is performed when the system control circuit 50 detects that the zoom drive is started from the zoom control unit 344 during moving image shooting.

  First, when it is detected that zoom driving has started, the system control circuit 50 creates a new fragment (step S1501).

  Next, the system control circuit 50 acquires lens information. This lens information is recorded in the metadata moof of the current fragment as lens information at the start of zoom driving (step S1502). The lens information is information on the aperture value and the pupil position.

  Next, the system control circuit 50 exposes the image sensor 14 to perform moving image shooting processing, and stores the shot moving image in the memory 30. The captured moving image is sequentially processed by the image processing circuit 20 for each frame and recorded in the memory 30 (step S1503).

  Next, the system control circuit 50 receives information on whether or not the lens is being zoom-driven from the zoom control unit 344, and determines whether or not the lens is being zoom-driven (step S1504). Steps S1503 and S1504 are repeated until it is determined that zoom driving is not being performed.

  If it is determined in step S1504 that the zoom drive is not being performed, that is, the zoom drive has been completed, the system control circuit 50 acquires lens information (step S1505). This lens information is recorded in the metadata moof of the current fragment as lens information at the end of zoom driving. The lens information is information on the aperture value and the pupil position.

  Next, the system control circuit 50 divides the fragment, creates a new fragment, and ends the flow during zoom driving (step S1506).

  Then, when converting dust correction parameters in step S1805 in the dust removal process of FIG. 10, the lens information at the start and end of zooming is read from the fragment moof, and the lens information is stored in the intermediate frame during zoom driving from the transition. Dust removal is performed by interpolation.

  According to the second embodiment described above, substantially the same effect as that of the first embodiment can be obtained.

  Furthermore, in the second embodiment, since lens information is recorded only at the start and end of zoom driving, the amount of data can be reduced.

(Third embodiment)
Next, a third embodiment will be described. In the third embodiment, lens information is recorded together with moving image recording at a certain frame interval during zoom driving of the lens. Since the configuration of the digital camera according to the third embodiment is the same as that of the first embodiment, description thereof will be omitted and only operations different from those of the first embodiment will be described.

  FIG. 16 is a flowchart showing a moving image recording process during zoom driving in the present embodiment.

  In the present embodiment, the following processing is performed when the system control circuit 50 detects that the zoom drive is started from the zoom control circuit 344 during moving image shooting.

  First, when it is detected that zoom driving is started, the system control circuit 50 creates a new fragment (step S1601).

  Next, the system control circuit 50 acquires the lens information and records it in the metadata moof of the current fragment. The lens information is information on the aperture value and the pupil position (step S1602).

  Next, the system control circuit 50 starts counting for counting the frame interval. Specifically, 1 is substituted for the count value (step S1603).

  The system control circuit 50 exposes the image sensor 14 to perform moving image shooting processing, and stores the shot moving image in the memory 30. The captured moving image is subjected to image processing by the image processing circuit 20 sequentially for each frame and recorded in the memory 30 (step S1604).

  When one frame of the moving image is captured, the system control circuit 50 increments the count value by 1 to count the frame interval (step S1605).

  The system control circuit 50 receives information on whether or not the lens is zoom-driven from the zoom control unit 344, and determines whether or not the lens is zoom-driven (step S1606).

  A series of processing (steps S1602 to S1607) is repeated until it is determined that zoom driving is not in progress.

  In step S1607, the system control circuit 50 determines whether the count value has reached a predetermined number of frames (10 frames in FIG. 16). A series of processing (steps S1604 to S1607) is repeated until the count value reaches the predetermined number of frames. When the count value reaches the predetermined number of frames, lens information is recorded in step S1602, and counting is started again in step S1603, and a series of processing (steps S1604 to S1607) is performed.

  If it is determined in step S1606 that the zoom drive is not in progress, that is, it is determined that the zoom drive has ended, a fragment is divided in step S1608, a new fragment is created, and the flow during the zoom drive is ended.

  Then, during the dust correction parameter conversion in step S1805 in the dust removal process of FIG. 10, the lens information of the frames at regular intervals is read from the fragment moov, and the intermediate frame in which no lens information is recorded from the transition is read. The lens information is interpolated from the front and rear lens information. In this way, dust removal is performed.

  As described above, according to the above embodiment, substantially the same effect as that of the first embodiment can be obtained. Furthermore, in the third embodiment, since lens information is recorded at a constant frame interval, the amount of data can be reduced.

(Other embodiments)
In each of the embodiments described above, the dust removal process is performed using an image processing apparatus prepared separately instead of the digital camera body. Of course, the dust removal process may be performed on the digital camera body. In that case, the processing is performed by the system control circuit 50 executing a dust removal processing program stored in the nonvolatile memory 56.

  The object of each embodiment is also achieved by the following method. That is, a storage medium (or recording medium) in which a program code of software that realizes the functions of the above-described embodiments is recorded is supplied to the system or apparatus. Then, the computer (or CPU or MPU) of the system or apparatus reads and executes the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but the present invention includes the following cases. That is, based on the instruction of the program code, an operating system (OS) running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

  Furthermore, the following cases are also included in the present invention. That is, the program code read from the storage medium is written into a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer. Thereafter, based on the instruction of the program code, the CPU or the like provided in the function expansion card or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

  When the present invention is applied to the above storage medium, the storage medium stores program codes corresponding to the procedure described above.

It is the external appearance perspective view which looked at the digital camera concerning the 1st Embodiment of this invention from the back side. 1 is a vertical sectional view showing an internal structure of a digital camera according to a first embodiment of the present invention. 1 is a block diagram illustrating a circuit configuration of a lens interchangeable single-lens reflex digital camera as an imaging apparatus according to the present invention. 4 is a flowchart for explaining dust detection processing in the digital camera according to the first embodiment. It is a figure which shows the data format example of dust correction data. 5 is a flowchart for explaining details of a dust region acquisition routine performed in step S27 of FIG. 4. It is a figure which shows the processing unit of the dust area | region determination process performed by step S62 of FIG. It is a figure which shows the outline | summary of dust area size calculation performed by step S63 of FIG. FIG. 5 is a flowchart illustrating details of an imaging process routine performed in step S <b> 24 of FIG. 4. FIG. It is a flowchart which shows a dust removal process. It is a flowchart explaining the detail of the preservation | save routine performed by FIG.10 S1806. It is a figure explaining the concept of the metadata and media data in MP4 or a similar file format. It is a figure explaining the concept of a fragment movie. It is a flowchart which shows the division | segmentation method of the fragment at the time of the zoom drive in 1st Embodiment. It is a flowchart which shows the division | segmentation method of the fragment at the time of the zoom drive in 2nd Embodiment. It is a flowchart which shows the division | segmentation method of the fragment at the time of the zoom drive in 3rd Embodiment.

Claims (7)

An image pickup means having an image pickup device for photoelectrically converting a subject image formed by the taking lens ;
From the image of the foreign matter contained in the foreign substance detection image signal obtained from the imaging unit, and the foreign matter detection means for detecting the foreign substance information including information of the position and size of the foreign substance even without low,
Lens information acquisition means for acquiring lens information including information on the aperture value and pupil position of the photographing lens ;
When the moving image shooting is performed, and records the moving image data generated based on the image signal continuously outputted from the image pickup means, wherein the lenses information before and Kikoto product information to the moving picture data And a recording means for adding and recording,
Zoom operation detecting means for detecting the start and end of the zoom operation of the photographic lens ,
When the zoom operation detecting means detects the start of the zoom operation of the photographing lens, the recording means creates a new fragment,
Between the time when the zoom operation detecting unit detects the start of the zoom operation of the photographing lens and the end of the zoom operation of the photographing lens is detected, the lens information acquisition unit updates the lens information, The recording device adds the updated lens information to the moving image data and records it in the new fragment . The lens information acquisition unit repeatedly updates the lens information after the zoom operation detection unit detects the start of the zoom operation of the photographic lens until the end of the zoom operation of the photographic lens is detected. Get multiple updated lens information,
The imaging apparatus according to claim 1, wherein the recording unit adds the plurality of updated lens information to the moving image data and records the new fragment . When the zoom operation detecting means detects the end of the zoom operation is started or the photographing lens of the zoom operation of the photographing lens, the lens information acquisition unit in claim 1, characterized in that updating the lens information The imaging device described. When the zoom operation detecting means detects the end of the zoom operation of the photographing lens, said recording means imaging apparatus according to claim 1, characterized in that to create a new fragment. The imaging apparatus according to claim 2, wherein the lens information acquisition unit updates the lens information repeatedly at regular frame intervals. A method for controlling an image pickup apparatus including an image pickup unit having an image pickup device that photoelectrically converts a subject image formed by a photographing lens ,
From the image of the foreign matter contained in the foreign substance detection image signal obtained from the imaging unit, and the foreign matter detection step of detecting the foreign substance information including information of the position and size of the foreign substance even without low,
Lens information acquisition step for acquiring lens information including information on the aperture value and pupil position of the photographing lens ;
When the moving image shooting is performed, and records the moving image data generated based on the image signal continuously outputted from the image pickup means, wherein the lenses information before and Kikoto product information to the moving picture data And a recording step for adding and recording,
A zoom operation detecting step for detecting the start and end of the zoom operation of the photographing lens ,
When the start of the zoom operation of the photographing lens is detected in the zoom operation detection step, a new fragment is created in the recording step,
The lens information is updated in the lens information acquisition step after the zoom operation detection step detects the start of the zoom operation of the photographing lens until the end of the zoom operation of the photographing lens is detected, A method for controlling an imaging apparatus , wherein the updated lens information in the recording step is added to the moving image data and recorded in the new fragment .   A program for causing a computer to execute the control method according to claim 6.

Images