JP2008306307A - Image processing unit, image processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To improve visibility in correction result confirmation by explicitly displaying a place where dirt is corrected automatically. <P>SOLUTION: An image processing unit 1120 corrects a photographed image including the shadow of a foreign matter adhered to an optical element arranged in front of an image pickup device, so that the shadow of the foreign matter is eliminated, and transmits the corrected image to an output device which outputs it as a visible image. The image processing unit 1120 has: an image correction section 425 that performs eliminates the shadow of the foreign matter in a plurality of different degrees of correction to the photographed image including the shadow of the foreign matter, and generates the corrected image of a plurality of patterns; a layout section 402 that lays out the generated corrected images of the plurality of patterns, and generates an image signal for outputting them as one image; and a transmission section 404 for transmitting the generated image signal to the output device. At the layout section, in each of the corrected images of the plurality of patterns, at least one portion of the region of the shadow of the foreign matter subjected to the correction processing by the image correction section is enlarged to size corresponding to the output size of the output device for layout. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image correction technique for removing foreign matter from an image when a foreign matter adhering to an optical system is reflected in an image in an imaging apparatus such as a digital camera.

  Conventionally, in a single-lens reflex digital camera with an interchangeable lens, a photographed image is obtained when foreign matter such as dust adheres to the imaging optical system (particularly the surface of an optical element such as a low-pass filter disposed in front of the imaging element). In some cases, the shadow of the foreign object is reflected. In this case, the quality of the captured image is deteriorated. Therefore, in recent years, development of a technique for preventing foreign matter from adhering to the imaging optical system, a technique for removing foreign matter attached to the imaging optical system, and a technique for removing foreign substances reflected in an image have been actively performed.

  Among these techniques, there are the following techniques for removing foreign substances from an image. First, a uniform subject of a single white color is photographed, an image in which only dust in the imaging optical system is captured is photographed, and position information of dust in the imaging optical system is obtained from the image. Then, correction processing (removal, color correction, interpolation by peripheral pixels, etc.) is automatically performed on the actually captured image based on the position information, and dust is removed from the image.

  In such automatic correction processing, it is practically impossible to achieve a correction result desired by all users according to user preferences and shooting scenes. You may be dissatisfied with the location.

In order to make the user determine whether or not to perform automatic correction processing for such a problem, the result of performing automatic correction processing on a plurality of images is displayed as a sample in an index display. A proposal has been made (see Patent Document 1).
JP 2000-221618 A

  However, it is very difficult to grasp the result of correcting dust in each image with an index display image size. In addition, the user has only two choices of automatic correction or not, and cannot reflect the user's preference such as "I want to correct dust a little more" or "I don't want to correct this place" .

  The present invention has been made in view of the above-described problems, and an object of the present invention is to explicitly display a place where dust is automatically corrected, thereby improving the visibility with which the user confirms the correction result. It is.

  In order to solve the above-described problems and achieve the object, the image processing apparatus according to the present invention captures a shadow of a foreign object attached to an optical element arranged in front of an image sensor that photoelectrically converts a subject image. An image processing apparatus that corrects an image so as to remove a shadow of the foreign object and transmits the corrected image to an output device that outputs the image as a visible image. A correction process for removing the shadow of the foreign substance at a plurality of different correction levels, and a correction image of the plurality of patterns generated by the image correction unit and an image correction unit that generates a correction image of a plurality of patterns are laid out Layout means for generating an image signal to be output as a single image, and transmission means for transmitting the image signal generated by the layout means to the output device. The layout means includes, in each of the plurality of patterns of corrected images, at least a part of the shadow area of the foreign matter that has been corrected by the image correction means, and the output size of the output device specified by the user The layout is considerably enlarged.

  Further, the image processing method according to the present invention removes the shadow of the foreign object from a captured image in which the shadow of the foreign object attached to the optical element disposed in front of the image sensor that photoelectrically converts the subject image is reflected. An image processing method that corrects and transmits the corrected image to an output device that outputs the corrected image as a visible image, wherein the shadow of the foreign object is captured at a plurality of different degrees of correction with respect to a captured image in which the shadow of the foreign object is reflected. An image correction step for performing correction processing to remove a plurality of patterns and generating a correction image of a plurality of patterns, and an image for laying out the correction images of the plurality of patterns generated in the image correction step and outputting them as a single image A layout step for generating a signal, and a transmission step for transmitting the image signal generated in the layout step to the output device. In each of the corrected images of the screen, at least a part of the shadow area of the foreign matter that has been subjected to the correction process in the image correction step is enlarged to correspond to the output size of the output device specified by the user. It is characterized by doing.

  According to the present invention, it is possible to improve visibility of a user confirming a correction result by explicitly displaying a place where dust is automatically corrected.

  An embodiment in which the present invention is used in a direct printing environment based on the PictBridge standard by connecting a lens interchangeable single-lens reflex digital camera and a printer as an output destination will be described below with reference to attached images. The single-lens reflex digital camera and the printer constitute the image processing system of the present embodiment.

  FIG. 1 is a diagram illustrating a state where a lens interchangeable single-lens reflex digital camera and a printer according to an embodiment of the present invention are connected.

  In FIG. 1, the single-lens reflex digital camera 1120 includes a liquid crystal display 417 and a cross switch switch (SW) 116. The communication cable 1130 is a communication cable such as a USB that connects the single-lens reflex digital camera 1120 and the printer 1110.

  FIG. 2 is a block diagram of the control system of the printer 1110 in this embodiment.

  In FIG. 2, reference numeral 1210 denotes a CPU that controls the entire apparatus, 1211 denotes a ROM that stores an operation procedure (program) and fonts of the CPU 1210, and 1212 denotes a RAM that is used as a work area of the CPU 1210. Reference numeral 1213 denotes an operation panel. Reference numeral 1214 denotes an interface for connecting to a PC (personal computer), and reference numeral 1215 denotes an interface (USB host side) for connecting to the single-lens reflex digital camera 1120. Reference numeral 1216 denotes a card interface to which an adapter (PCMCIA) 1217 equipped with a storage medium such as a memory card can be connected. The card interface 1216 is used by the printer 1110 to read a description file relating to a print instruction in a memory card that is photographed by the DPOF, that is, a single-lens reflex digital camera 1120, and prints accordingly. Reference numeral 218 denotes a printer engine.

  FIG. 3 is a perspective view showing an appearance of the single-lens reflex digital camera 1120 according to this embodiment, and FIG. 4 is a vertical sectional view of the single-lens reflex digital camera 1120 shown in FIG.

  In FIG. 3, an eyepiece window 111 for finder observation, an AE (automatic exposure) lock button 112, and an AF (automatic focus adjustment) distance measuring point selection button 113 are provided on the upper part of the camera body 100. In addition, a release button 114 for instructing photographing is provided, and two-stage input of a half-pressed state (SW1) and a fully-pressed state (SW2) is possible. Further, an electronic dial 411 for selecting various setting values, a photographing mode selection dial 117, and an external display device (OLC) 409 are provided. The external display device 409 includes a liquid crystal display device, and displays shooting conditions such as shutter speed, aperture, and shooting mode, and other information.

  On the back surface of the camera body 100, an LCD monitor device 417 for displaying a photographed image, an image subjected to correction processing, various setting screens such as a menu screen, and the like is provided.

  Further, a monitor switch 121 for turning on / off the LCD monitor device 417, a cross arrangement switch 116, and a menu button 124 are provided on the back surface of the camera body 100.

  The cross placement switch 116 has four buttons arranged vertically and horizontally and a SET button arranged in the center. The user instructs the camera to select and execute menu items displayed on the LCD monitor device 417. Used for.

  The menu button 124 is a button for causing the LCD monitor device 417 to display a menu screen for performing various camera settings. For example, the shooting mode is set by pressing the menu button 124, operating the up / down / left / right buttons of the cross switch 116, and pressing the SET button while the desired mode is selected. The menu button 124 and the cross placement switch 116 are also used for setting for printing after correcting dust in image data, which will be described later.

  Since the LCD monitor device 417 of the present embodiment is a transmissive type, an image cannot be visually recognized only by driving the LCD monitor device, and a backlight illumination device 416 is always required on the back side as shown in FIG. is there. As described above, the LCD monitor device 417 and the backlight illumination device 416 constitute an image display device.

  As shown in FIG. 4, the photographic lens 200 that is an imaging optical system can be attached to and detached from the camera body 100 via a lens mount 202. In FIG. 4, 201 is a photographing optical axis, and 203 is a quick return mirror.

  The quick return mirror 203 is disposed in the photographing optical path, and a position for guiding the subject light from the photographing lens 200 to the finder optical system (referred to as a position shown in FIG. 4, an oblique position) and a position for retracting out of the photographing optical path (retraction position). Is called).

  In FIG. 4, the subject light guided from the quick return mirror 203 to the finder optical system is imaged on the focus plate 204. Reference numeral 205 denotes a condenser lens for improving the visibility of the finder, and 206 denotes a pentagonal roof prism.

  Reference numerals 209 and 210 respectively denote a rear curtain and a front curtain that constitute a shutter, and an image sensor 418 that is a solid-state image sensor that photoelectrically converts a subject image disposed behind the rear curtain 209 and the front curtain 210 is necessary. Exposed for time. A captured image converted into an electrical signal for each pixel by the image sensor 418 is processed by an A / D converter 423, an image processing circuit 425, and the like, which will be described later, and recorded on the recording medium 419 as image data. The image sensor 418 is provided with an optical low-pass filter 418 a on the front surface thereof, and adjusts the spatial frequency of the subject image formed on the image sensor 418. Dust (foreign matter) affecting the captured image adheres to the optical low-pass filter 418a and appears as a shadow on the subject image formed on the image sensor 418, thereby degrading the quality of the captured image.

  The image sensor 418 is held on the printed circuit board 211. Behind this printed board 211, a display board 215, which is another printed board, is arranged. An LCD monitor device 417 and a backlight illumination device 416 are disposed on the opposite surface of the display substrate 215.

  Reference numeral 419 denotes a recording medium for recording image data, and reference numeral 217 denotes a battery (portable power supply). The recording medium 419 and the battery 217 are detachable from the camera body.

  FIG. 5 is a block diagram showing a circuit configuration of the single-lens reflex digital camera 1120 according to the first embodiment of the present invention.

  In FIG. 5, the microcomputer 402 controls the operation of the entire camera including processing of image data output from the image sensor 418 and display control of the LCD monitor device 417.

  The switch (SW1) 405 is turned on when the release button 114 is half-pressed, and when the switch (SW1) 405 is turned on, a shooting preparation state is set. The switch (SW2) 406 is turned on when the release button 114 is pressed to the end (fully pressed state), and the photographing operation is started when the switch (SW2) 406 is turned on.

  The lens control circuit 407 performs communication with the photographic lens 200, drive control of the photographic lens 200 during AF, and drive control of the diaphragm blades.

  In FIG. 5, an external display control circuit 408 controls an external display device (OLC) 409 and a display device (not shown) in the finder. The switch sense circuit 410 transmits signals of a large number of switches including the electronic dial 411 provided in the camera to the microcomputer 402.

  The strobe light emission dimming control circuit 412 is grounded via the X contact 412a and controls the external strobe. The distance measuring circuit 413 detects the defocus amount with respect to the subject for AF. The photometry circuit 414 measures the luminance of the subject.

  A shutter control circuit 415 controls the shutter and performs appropriate exposure on the image sensor 418. The LCD monitor device 417 and the backlight illumination device 416 constitute an image display device. The recording medium 419 is, for example, a hard disk drive or a semiconductor memory card that can be attached to and detached from the camera body.

  Further, the microcomputer 402 includes an A / D converter 423, an image buffer memory 424, an image processing circuit 425 including a DSP, and a pixel defect position memory 426 that stores that a predetermined pixel in the image sensor itself is defective. Is connected. Also connected is a dust position memory 427 that stores the pixel position in the image pickup device causing the image defect due to dust. Note that the pixel defect position memory 426 and the dust position memory 427 are preferably non-volatile memories. Further, the pixel defect position memory 426 and the dust position memory 427 may be stored using different addresses in the same memory space.

  Reference numeral 428 denotes a nonvolatile memory that stores programs executed by the microcomputer 402.

  Reference numeral 404 denotes an external interface including a USB interface (USB slave side) for connecting to a PC or the printer 1110 in this embodiment.

  As communication means between the single-lens reflex digital camera and the printer, SCSI, wireless TCP / IP, or the like can be considered, but here, a USB interface is used. Further, the result of removing the shadow of the foreign matter from the photographed image is not displayed on the liquid crystal display 417 of the camera, but is transmitted to the printer and printed (preprinted) as a visible image.

  However, the present invention is not limited to this example, and the result of removing the shadow of the foreign object may be displayed on the liquid crystal display 417 of the camera, or may be displayed on an external display device such as a television.

  Next, an example of processing for detecting the position of dust attached to the imaging optical system will be described with reference to the flowchart of FIG.

  This processing is performed by the microcomputer 402 executing a dust detection processing program stored in the memory 428.

  The dust detection process is performed by capturing a dust detection image. When performing dust detection processing, a camera is installed with the photographing optical axis 201 of the lens 200 facing a surface having a uniform color, such as an emission surface of a surface light source device or a white wall, and preparation for dust detection is performed. Alternatively, a dust detection light unit (a small point light source device mounted instead of a lens) is mounted on the lens mount 202 to prepare for dust detection. For example, a white LED is considered as the light source of the light unit, and it is desirable to adjust the size of the light emitting surface so as to correspond to a predetermined aperture value (for example, F32).

  In this embodiment, a case where a normal photographing lens is used will be described. However, dust detection may be performed by attaching the light unit to the lens mount 202. Thus, in this embodiment, the dust detection image is an image having a uniform color.

  After the preparation is completed, when the start of dust detection processing is instructed from the cross arrangement switch 116, the microcomputer 402 first sets the aperture. The image formation state of dust near the image sensor changes depending on the aperture value of the lens, and the position changes depending on the pupil position of the lens. Therefore, in the dust correction data, in addition to the position and size of dust, it is necessary to hold the aperture value at the time of detection and the pupil position of the lens.

  However, it is not always necessary to hold the aperture value in the dust correction data if it is determined in advance that the same aperture value is always used even when different lenses are used at the stage of creating dust correction data. Similarly, by using a light unit or permitting the use of only a specific lens for the pupil position, it is not always necessary to hold the pupil position in the dust correction data.

  In other words, at the stage of creating dust correction data, if you want to allow multiple lenses to use or change the aperture value to narrow down appropriately, it is necessary to keep the aperture value and lens pupil position at the time of detection in the dust correction data It can be said that there is. Here, the pupil position refers to the distance from the imaging plane (focal plane) of the exit pupil.

  Here, for example, F16 is designated (step S21).

  Next, the microcomputer 402 causes the lens control circuit 407 to perform aperture blade control of the photographing lens 200, and sets the aperture to the aperture value designated in step S21 (step S22). Further, the focus position is set to infinity (step S23).

  When the aperture value and focus position of the photographing lens are set, photographing in the dust detection mode is executed (step S24). Details of the imaging processing routine performed in step S24 will be described later with reference to FIG. The captured image data is stored in the buffer memory 424.

  When shooting is completed, the aperture value and lens pupil position at the time of shooting are acquired (step S25). Data corresponding to each pixel of the photographed image stored in the image buffer memory 424 is called to the image processing circuit 425 (step S26). The image processing circuit 425 performs the process shown in FIG. 8, and acquires the position and size of the pixel where dust exists in the image (step S27). The position and size of the pixel where the dust exists in step S27 and the aperture value and lens pupil position information acquired in step S25 are registered in the dust position memory 427 (step S28). Here, when the light unit described above is used, lens information cannot be acquired. Therefore, when the lens information cannot be acquired, it is determined that the light unit is used, and predetermined lens pupil position information and a converted aperture value calculated from the light source diameter of the light unit are registered.

  Here, in step S28, the position of the defective pixel (pixel defect) from the time of manufacture recorded in the pixel defect position memory in advance is compared with the position of the read pixel data to check whether the pixel defect is present. Only the area determined not to be due to the pixel defect may be registered in the dust position memory 427.

  An example of the data format of dust correction data stored in the dust position memory 427 is shown in FIG. As shown in FIG. 7, the dust correction data stores lens information and dust position and size information when the detection image is captured. The dust correction data is added to the image together with the shooting information of the image data during normal shooting.

  Specifically, the actual aperture value (F value) at the time of capturing the detected image and the lens pupil position at that time are stored as lens information at the time of capturing the detected image. Subsequently, the number of detected dust areas (integer value) is stored in the storage area, and subsequently, individual specific dust area parameters are repeatedly stored by the number of dust areas. The parameter of the dust area is a set of three numerical values: a dust radius (for example, 2 bytes), a center x coordinate (for example, 2 bytes) in the effective image area, and a similar y coordinate (for example, 2 bytes).

  When the dust correction data size is limited due to the size of the dust position memory 427 or the like, the data is stored with priority from the head of the dust area obtained in step S27. This is because in the dust area acquisition routine in step S27, the dust areas are sorted in the order of noticeable dust, as will be described later.

  Next, details of the dust region acquisition routine performed in step S27 will be described with reference to FIGS.

  As shown in FIG. 9, the called image data is developed on a memory and processed in units of predetermined blocks. This is to cope with peripheral dimming caused by lens and sensor characteristics. Peripheral dimming is a phenomenon in which the luminance at the peripheral portion is lower than that at the central portion of the lens, and it is known that it is reduced to some extent by reducing the aperture of the lens. However, there is a problem that even if the aperture is reduced, if the dust position is determined based on a predetermined threshold value for the photographed image, dust in the peripheral portion cannot be detected accurately depending on the lens. Therefore, the image is divided into blocks to reduce the influence of peripheral dimming.

  When the blocks are simply divided, there is a problem that the detection result of the dust straddling the blocks is shifted when the threshold values are different between the blocks. Therefore, the blocks are overlapped, and pixels determined to be dust in any of the blocks constituting the overlap region are treated as dust regions.

  The dust region determination in the block is performed according to the processing flow shown in FIG. First, the maximum luminance Lmax and average luminance Lave in the block are calculated, and the threshold value T1 in the block is calculated using the following equation.

T1 = Lave × 0.6 + Lmax × 0.4
Next, pixels that do not exceed the threshold value are defined as dust pixels (step S61), and isolated regions constituted by dust pixels are each defined as one dust region di (i = 0, 1,..., N) (step S62). . As shown in FIG. 10, for each dust region, the maximum value Xmax and the minimum value Xmin of the horizontal coordinate of the pixels constituting the dust region, the maximum value Ymax and the minimum value Ymin of the vertical coordinate are obtained, and the dust region di A radius ri representing the size of is calculated by the following equation (step S63).

ri = √ [{(Xmax−Xmin) / 2} ² + {(Ymax−Ymin) / 2} ² ]
The relationship between Xmax, Xmin, Ymax, Ymin and ri is shown in FIG.

  Thereafter, in step S64, an average luminance value for each dust region is calculated.

  In some cases, the data size of the dust correction data is limited due to the limitation due to the size of the dust position memory 427. In order to cope with such a case, the dust position information is sorted according to the size and the average luminance value of the dust region (step S65). In this embodiment, sorting is performed in descending order of ri. When ri is equal, the sort is performed in ascending order of the average luminance value. In this way, it is possible to preferentially register noticeable dust in the dust correction data. The sorted dust area is Di, and the radius of the dust area Di is Ri.

  If there is a dust area larger than a predetermined size, it may be excluded from sorting and placed at the end of the sorted dust area list. This is because a large dust region may lower the image quality if interpolation processing is performed later, and it is desirable to treat it as the lowest priority for correction.

  Next, details of the imaging processing routine performed in step S24 of FIG. 6 will be described using the flowchart shown in FIG. The processing is performed by the microcomputer 402 executing an imaging processing program stored in the memory 428.

  When this imaging processing routine is executed, in step S201, the microcomputer 402 operates the quick return mirror 203 shown in FIG. 4, performs so-called mirror up, and retracts the quick return mirror 203 outside the imaging optical path.

  Next, charge accumulation in the image sensor is started in step S202, and in the next step S203, exposure is performed by running the front curtain 210 and the rear curtain 209 of the shutter shown in FIG. In step S204, the charge accumulation of the image sensor is terminated, and in the next step S205, the image signal is read from the image sensor, and the image data processed by the A / D converter 423 and the image processing circuit 425 is temporarily stored in the buffer memory 424. .

  When reading of all image signals from the image sensor is completed in the next step S206, the quick return mirror 203 is mirrored down in step S207, the quick return mirror is returned to the oblique position, and a series of imaging operations is completed.

  In step S208, it is determined whether normal shooting or dust detection image shooting is performed. At the time of normal shooting, the process proceeds to step S209, and dust correction data shown in FIG. 7 is added to the Exif area, which is the header area of the image data, together with the camera setting value at the time of shooting, and is recorded on the recording medium 419 in association with the image data.

  In the following embodiment, description will be made using an image file in which dust correction data and image data are integrally held. Note that the present embodiment can be applied even if dust correction data and image data are not integrated, and are recorded as, for example, separate files associated with each other.

Example 1
FIG. 12 is a flowchart showing a printing process executed in the present embodiment until automatic dust correction is performed and an image is printed. In FIG. 12, the user uses the liquid crystal display 417 and the cross switch 116 of the single-lens reflex digital camera 1120 to select an image to be printed, and a print setting screen as shown in FIG. 14 displayed on the liquid crystal display 417. A “paper setting” button 603 on the screen 601 is pressed to change the screen in the order of FIGS. 15, 16, and 17 and specify the paper size, paper type, and layout, which are necessary conditions for printing, as shown in FIG. 18 shows a processing procedure inside the camera after the “dust correction” button 602 is pressed and an operation such as “ON” / “OFF” for automatic dust correction is performed on the screen of FIG. It is assumed that all the necessary information such as the set information, the image data of the selected image, and the dust correction data associated therewith are stored in the memory 428.

  First, in step S401, if the “cancel” button 605 in FIG. 14 is pressed, the printing process is terminated without printing, and if not, the process proceeds to step S402. In step S402, if the “pre-print” button 604 in FIG. 14 is pressed, the process proceeds to step S403, and if not, the process proceeds to step S408.

  When the “pre-print” button 604 is pressed and the process proceeds to step S403, a plurality of patterns of images that have been subjected to automatic dust correction processing by changing the correction level (level) of the original image are generated. For example, in the case of three patterns, as the correction level, a level “correcting only a uniform color portion such as the sky”, a level “correcting all dust”, and a level between them can be considered.

  After creating a plurality of patterns of automatically corrected images, in step S404, the dust-corrected portion of each automatically corrected image is enlarged to a size corresponding to the paper size specified in FIG. And a composite corrected image). This process is performed by the CPU 402 which is also an image composition unit. This assumes that in pre-printing, m + 1 images obtained by adding original images to a plurality of patterns (number of patterns: m) are arranged in n planes (n> 1) on one page of paper. For this reason, each image has a size smaller than the designated paper size, so that the visibility of the correction result is lowered. Therefore, by enlarging only the part (part) of the automatically corrected dust to the paper size (equivalent to the output size) and combining the enlarged image with the dust part to generate a composite image, the visibility of the correction result is improved. Improve.

  FIG. 20 (1201, 1202, 1203) shows an example in which automatic correction is performed on the original image as shown in FIG. 13 and a composite corrected image is created. In the present embodiment, since the output destination is a printer, the output size is the paper size. However, for example, when outputting to a television or a liquid crystal monitor, the output size is enlarged corresponding to the resolution.

  In step S405, the combined correction image prepared for the pattern is printed in an n-plane arrangement. Note that an index number (selection number) for selecting a correction pattern preferred by the user is printed in the portion where the file number is printed. The index number is added to the composite corrected image and transmitted from the single-lens reflex digital camera 1120 to the printer 1110.

  FIG. 21 shows an example in which four patterns are arranged in three patterns (m = 3, n = 4). If n = 1, one image is printed on one page, so step S404 is omitted, and the selection number is displayed on the border as a file number in step S405 and printed. The number of patterns m and the value of n in the n-plane arrangement may be fixed values or may be set by the user as menu items displayed on the liquid crystal display 417 of the single-lens reflex digital camera 1120.

  In step S406, the user looks at the pre-printed image output from the printer, and selects a correction pattern selection number that suits the user's preference on a screen (index selection screen) as shown in FIG. In step S407, it is confirmed whether or not “Do not automatically correct” is selected. If not selected, in step S408, the correction image of the correction pattern of the selected number is designated as the specified paper size and paper layout.・ Print with paper type. For example, no. If 4 is selected and L plate, 1 up, and Photo are specified, as shown in FIG. 25, the correction image of the selected correction pattern is printed on the L plate size in a single-sided layout. If “not automatically correct” is selected, the original image is printed in the designated paper size, paper layout, and paper type in step S412.

  If the “preprint” button 604 is not pressed in step S402 and the process proceeds to step S409, the “dust correction” button 602 in FIG. 14 is pressed in step S409. If “Yes (On)” is selected in the screen of FIG. 18, dust correction processing is automatically performed in step S410. In step S411, the image subjected to dust correction processing is transferred to the designated paper size / Print with paper layout and paper type. If “YES (ON)” is not selected in step S409, the original image is printed in the designated paper size / paper layout / paper type in step S412.

  As a result, the user can obtain an image of the automatic dust correction processing result that suits the user's preference by pre-printing. In addition, when the pre-printing is performed, the corrected dust spot is enlarged to a size equivalent to the paper size, thereby improving the visibility with which the user can confirm the automatically corrected effect.

(Example 2)
In the second embodiment, a method for further improving the visibility when a user automatically confirms a place where dust has been corrected will be described.

  If there are places where dust is dense (1401 to 1405, 1408 to 1412) like 1401 to 1412 in FIG. 22, the part where dust is corrected as shown in FIG. Enlarged images will overlap. Therefore, it is difficult to confirm the result of correcting dust.

  Therefore, in step S404 of FIG. 12, “locations where dust is dense and dense” are automatically detected in the image, and peripheral portions (peripheral regions) of the detected regions are detected as 1501-1502 in FIG. Enlarge and combine to the paper size equivalent. In addition, as a method of determining the degree of congestion, a search is made for dust existing within an appropriate radius centered on a certain dust from the dust position information, and if there is a predetermined number or more, it is determined that the dust is dense. Such a method can be considered. However, the method for determining the degree of congestion is not limited to this.

  As a result, even when pre-printing is performed on an image with a large amount of dust, the user can easily confirm the location of the dust that has been automatically corrected.

(Example 3)
In the third embodiment, a method for further improving the visibility when a user automatically confirms a place where dust has been corrected will be described.

  When there are places where the dust is dense (1401 to 1405, 1408 to 1412) and isolated places (1406 and 1407) like 1401 to 1412 in FIG. 22, the places corrected as in the first embodiment Is enlarged to a size equivalent to the paper size and combined as follows. That is, the enlarged locations overlap in the dense locations (1401 to 1405, 1408 to 1412), making it difficult to confirm the corrected result. In addition, when the dusty area is enlarged and displayed as in the second embodiment, the isolated portions (1406 and 1407) are not enlarged, and it is necessary to look closely to confirm the corrected result. .

  Therefore, in step S404 in FIG. 12, the method for enlarging and displaying the dust area as in the second embodiment and the method for enlarging and displaying the dust location as in the first embodiment are automatically switched depending on the situation around the dust location. . As a result, as shown in FIG. 24, it is easy to check the periphery of the dust that has been automatically corrected even in an isolated location (1602) or a dense location (1601).

  As a result of the above, not only the dust that is concentrated is enlarged as in the second embodiment, but also the dust that is isolated is enlarged as in the first embodiment, and the user exists on the image. It is possible to easily check the state of automatic correction of all dust in pre-printing.

(Other embodiments)
In the present invention, a software program (in the embodiment, a program corresponding to the communication procedure shown in the figure) that realizes the functions of the above-described embodiments may be directly or remotely supplied to each device. Further, it includes a case where the processing unit of the apparatus reads out and executes the supplied program code so that the above-described embodiment can be executed.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, or the like.

It is a figure which shows the state which connected the lens interchangeable single-lens reflex digital camera which is one Embodiment of this invention, and a printer. 2 is a block configuration diagram of a printer control system according to an embodiment of the present invention. FIG. It is an external appearance perspective view of a digital camera. It is a vertical sectional view showing the internal structure of the digital camera. It is a block block diagram of the control system of a single-lens reflex digital camera. It is a flowchart explaining the dust detection process in a digital camera. It is a figure which shows the data format example of dust correction data. It is a flowchart explaining the detail of the dust area | region acquisition routine performed by step S27 of FIG. It is a figure which shows the processing unit of the dust area | region determination process performed by step S62 of FIG. It is a figure which shows the outline | summary of dust area size calculation performed by step S63 of FIG. It is a flowchart explaining the detail of the imaging process routine performed by step S24 of FIG. 6 is a flowchart illustrating a printing process that is executed in one embodiment of the present invention until automatic dust correction is performed and an image is printed. It is a figure which shows the original image before performing a dust removal process. It is a figure which shows a print setting screen. It is a figure which shows the screen which designates the paper size which is conditions required for printing. It is a figure which shows the screen which designates the paper type which is conditions required for printing. It is a figure which shows the screen which designates the layout which is conditions required for printing. It is a figure which shows the screen which designates whether dust correction processing is performed automatically. It is a figure which shows the screen which selects the correction pattern suitable for a user preference. It is a figure which shows the example which performed the automatic correction | amendment with respect to the original image, and produced the composite correction | amendment image. It is a figure which shows the example which prints the correction image of 3 patterns by 4 surface arrangement | positioning. It is a figure which shows the state which has the location where the dust is crowded, and the location where it is isolated. FIG. 5 is a diagram illustrating a state in which dust is densely gathered in an image and a portion where a large amount of dust is automatically detected, and a detected area is enlarged to be equivalent to a paper size and combined. It is a figure which shows a mode that the method of enlarging and displaying the area | region of a dust and the method of enlarging and displaying the location of a dust are automatically switched according to the condition around a dust location. It is a figure which shows the example which printed the result of having performed automatic dust correction.

Claims (5)

A captured image in which a shadow of a foreign object attached to an optical element arranged in front of an image sensor that photoelectrically converts a subject image is reflected so as to remove the shadow of the foreign object, and the corrected image is made a visible image. An image processing device that transmits to an output device that outputs,
Image correction means for performing a correction process to remove the shadow of the foreign object at a plurality of different correction levels on the captured image in which the shadow of the foreign object is reflected, and generating a corrected image of a plurality of patterns;
Layout means for laying out the corrected images of the plurality of patterns generated by the image correction means and generating an image signal for outputting as a single image;
Transmission means for transmitting the image signal generated by the layout means to the output device,
The layout means includes, in each of the plurality of patterns of corrected images, at least a part of the shadow area of the foreign matter that has been corrected by the image correction means, and the output size of the output device specified by the user An image processing apparatus characterized in that the layout is considerably enlarged.   The image processing apparatus according to claim 1, wherein the output device is a printer, and the output size is a paper size of a print output from the printer.   The image processing apparatus according to claim 1, wherein the image correction unit performs correction based on position information relating to the shadow of the foreign matter recorded in association with the captured image. A captured image in which a shadow of a foreign object attached to an optical element arranged in front of an image sensor that photoelectrically converts a subject image is reflected so as to remove the shadow of the foreign object, and the corrected image is made a visible image. An image processing method for transmitting to an output device for outputting,
An image correction step of performing correction processing to remove the shadow of the foreign matter at a plurality of different correction degrees on the captured image in which the shadow of the foreign matter is reflected, and generating a corrected image of a plurality of patterns;
A layout step of laying out the corrected images of the plurality of patterns generated in the image correction step and generating an image signal for outputting as a single image;
A transmission step of transmitting the image signal generated in the layout step to the output device,
In the layout step, in each of the corrected images of the plurality of patterns, at least a part of the shadow area of the foreign matter subjected to the correction process in the image correction step is output by the output size of the output device specified by the user An image processing method characterized in that the layout is considerably enlarged.   A program causing a computer to execute the image processing method according to claim 4.

Images