JP2010239514A - Imaging system, image processing method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To obtain a formal photographic image improved and to provide in image quality with respect to an imaging system, an image processing method and a program. <P>SOLUTION: The imaging system 1 includes an imaging unit 2 for imaging a subject; an operation detecting unit 34 for detecting a photographing operation due to a photographer; a photographic control unit 20, in which the photographing operation is performed, formal photographing is performed and prior to the formal photographing, previous photographing is performed under photographing conditions different from the formal photographing; an SDRAM 41 for storing information, relating to a previous image captured in the previous photographing; and an image processing unit 25 for correcting a regular photographed image, captured in the formal photographing by using the previous image. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an imaging system such as a digital camera, an image processing method, and a program.

Since still images such as cameras can only be captured at the moment they are taken, depending on the information situation, images with exposure and subject colors that are not as intended by the photographer may be acquired, or the acquired images may be noisy. It may be included, or the image may be blurred due to camera shake or subject shake.
On the other hand, a method of correcting an image by performing image processing on a main captured image acquired at the time of main shooting using a similar image is known (see, for example, Patent Document 1).
In addition, image processing is performed on the actual captured image using information obtained from the captured image used for the pre-photographing display such as a through image, and the color of the subject such as white balance is corrected. A method is known (see, for example, Patent Document 2).
In addition, a technique is known in which a plurality of images are captured with the same resolution and combined to reduce noise, improve exposure, and reduce blur in the captured image.

JP 2006-344166 A Japanese Patent No. 3934611

  However, in the conventional technique disclosed in Patent Document 1 or the like, image processing of the captured image is not necessarily performed using an appropriate image, and as a result, the captured image satisfying the image quality desired by the photographer is satisfied. There is a concern that an image cannot be obtained.

  SUMMARY An advantage of some aspects of the invention is that it provides an imaging system, an image processing method, and a program that can obtain a real captured image with improved image quality.

In order to solve the above problems, the present invention employs the following means.
According to a first aspect of the present invention, there is provided a photographing optical system, an image pickup device that picks up an image of a subject via the photographing optical system, a photographing operation detecting unit that detects a photographing operation of a photographer, and the photographing operation. In addition, the first shooting control means for performing the main shooting, the second shooting control means for performing the pre-shooting under shooting conditions different from the main shooting prior to the main shooting, and the advance acquired in the previous shooting. An imaging system comprising storage means for storing information relating to an image, and image processing means for correcting an actual captured image acquired by the actual imaging using the prior image.

According to such a configuration, image processing is performed on the main captured image acquired in the main shooting using a pre-image acquired under shooting conditions different from that of the main shooting, and thus included in the main captured image. It is possible to perform image processing of the actual captured image using an image having information different from the existing image information. As a result, it is possible to perform image processing of the main photographic image using new information not included in the main photographic image, and it is possible to obtain a main photographic image with improved image quality.
Furthermore, since the pre-image captures a subject that is substantially similar to the image acquired in the main shooting, more appropriate information can be obtained for correcting the main shooting.

  For example, it is possible to reduce the memory capacity and the processing time by using an image having a lower resolution than the actual captured image as the prior image. In addition, since the low-resolution image has a clear structure such as the edge and texture of the image and is easy to analyze, there is a special solution when noise reduction processing is adjusted according to the edge and texture of the image during noise reduction processing. It can be used effectively without performing image conversion processing. In addition, an image created by performing gain-up processing after adding a plurality of pixels in the image sensor is a high-sensitivity image with low resolution but low noise. By using such a high-sensitivity image with less noise as a prior image, noise reduction processing can be performed more effectively.

  Also, for example, in the case of actual shooting, even if the shooting exposure condition setting fails or the exposure is under-emphasized with an emphasis on avoiding camera shake, By using images with different exposure conditions, it is possible to perform image correction processing. Furthermore, even if the subject has a brightness difference and the image is partially blackened or skipped only by actual shooting, it can be corrected by using a preliminary image with different exposure conditions. It is possible to reproduce the gradation and color of the overexposed part.

  Also, for example, in actual shooting, even if subject blurring or camera shake occurs due to reduced sensitivity and exposure for a long time with an emphasis on image quality, the actual shooting image and shutter speed are used as prior images. By using different images, for example, it is possible to improve the image by correcting the influence of subject shake and camera shake using a pre-image taken with high sensitivity and high shutter speed.

  According to a second aspect of the present invention, an imaging device that performs pre-photographing with a resolution different from that of main photographing prior to the main photographing and corrects an image acquired in the main photographing using an image obtained by the pre-photographing. This is an image processing method.

  According to a third aspect of the present invention, there is provided an image processing program for performing image processing, which is performed prior to a main shooting and an image acquired in a preliminary shooting performed at a resolution different from the main shooting. It is an image processing program for causing a computer to execute correction processing for correcting an image acquired in actual photographing.

  According to the present invention, it is possible to obtain a main captured image with improved image quality.

1 is a block diagram illustrating a schematic configuration of an imaging system according to a first embodiment of the present invention. It is the figure which showed the outline of the sequence flow from a power-on to the storage process of this imaging | photography image. It is the flowchart which showed the process sequence of the through image imaging | photography display process and the through image storage process which concern on the 1st Embodiment of this invention. It is the flowchart which showed the process sequence of the image process of the real picked-up image which concerns on the 1st Embodiment of this invention. It is the flowchart which showed the process sequence of the through image selection process which concerns on the 1st Embodiment of this invention. It is the flowchart which showed the process sequence of the matching process which concerns on the 1st Embodiment of this invention. It is the flowchart which showed the process sequence of the image process of the real picked-up image which concerns on the 2nd Embodiment of this invention. It is the flowchart which showed the process sequence of the image process of the picked-up image which concerns on the 3rd Embodiment of this invention.

[First Embodiment]
Hereinafter, an imaging system according to a first embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a block diagram illustrating a schematic configuration of an imaging system according to the present embodiment. An imaging system 1 according to the present embodiment is, for example, a digital camera, and includes an imaging unit 2 and an image processing device 3. The imaging unit 2 includes a lens 10, a shutter 11, a CCD 12, a CCD control unit 13, a lens driving unit 14, and a strobe 15.
The lens 10 is provided with a photographing lens for focus adjustment and focal length adjustment, and a diaphragm 10a for adjusting the aperture amount. The role of the diaphragm 10a is to adjust the brightness and depth of light irradiated to the imaging surface based on a control command from the imaging control unit 20, but in an inexpensive imaging system with little need for depth adjustment. For the purpose of adjusting the brightness, for example, an ND filter for adjusting the amount of light can be substituted.

The lens 10 is driven by the operation of the lens driving unit 14 under the control of the photographing control unit 20 described later. Thereby, focusing, zoom driving, and the like are performed based on a control command from the imaging control unit 20. The strobe 15 can irradiate the subject with light under the control of the photographing control unit 20.
Behind the lens 10 is an exposure time control shutter 11. The shutter 11 is driven and controlled by the photographing control unit 20.
The shutter 11 is always open during through image capturing. At this time, exposure amount control of the CCD 12 is realized by using an electronic shutter function of the CCD 12. The exposure amount to the CCD 12 is controlled by the shutter 11 during so-called still image (hereinafter referred to as “still shooting”) shooting.

  A CCD 12 as a two-dimensional imaging device is disposed behind the shutter 11 and photoelectrically converts a subject image formed by the lens 10 into an electrical signal. In the present embodiment, the CCD is used as the image pickup device. However, the present invention is not limited to this, and it is needless to say that a two-dimensional image pickup device such as a CMOS (Complementary Metal Oxide Semiconductor) can be used.

  The CCD controller 13 is connected to the CCD interface 21. The CCD controller 13 receives a control signal from a sequence controller (hereinafter referred to as “body CPU”), which will be described later, via the CCD interface 21, and performs power on / off control of the CCD 12 based on the control signal. The imaging timing is adjusted, and the photoelectric conversion signal is amplified (gain adjustment).

The analog image signal acquired by the CCD 12 is converted into a digital signal by the CCD interface 21 and input to the image processing device 3.
The image processing apparatus 3 is, for example, an ASIC, and the above-described photographing control unit (first photographing control unit, second photographing control unit) 20, CCD interface 21, body CPU 22, luminance calculation unit 23, and AF calculation unit 24. And an image processing unit (image processing means) 25 and the like. These units are connected to each other via a data bus 30 in the image processing apparatus 3.

The body CPU 22 controls each unit included in the imaging system 1.
The luminance calculation unit 23 averages the image signal for each predetermined divided area, converts it into a luminance signal, and calculates luminance distribution information of the subject.
The AF calculation unit 24 divides the image signal into predetermined regions, calculates contrast information for each region, and synchronizes with the control of the lens driving unit 14 so that the lens 10 is adjusted so that the contrast in the predetermined region is maximized. Focus on the subject by driving.

  The image processing unit 25 performs various types of image processing such as OB subtraction, color correction, gradation conversion, monochrome / color mode processing, and through image processing on the image signal acquired by the imaging unit 2. Details of processing realized by the image processing unit 25 will be described later.

  In addition to the above-described components, the data bus 30 includes a compression unit 31, an SDRAM (Synchronous Dynamic Access Memory) control unit 32, a flash memory control unit 33, an operation detection unit (shooting operation detection unit) 34, a recording A medium control unit 35, a video signal output unit 36, and the like are connected.

  The compression unit 31 is a block for compressing image data or the like stored in an SDRAM 41 described later with JPEG. Note that image compression is not limited to JPEG, and other compression methods can be applied. The flash memory control unit 33 is connected to the flash memory 42. The flash memory 42 stores an image processing program for controlling each process in the imaging system 1, and the body CPU 22 controls each unit according to the program stored in the flash memory 42. The flash memory 42 is an electrically rewritable nonvolatile memory. The SDRAM 41 is connected to the data bus 30 via the SDRAM control unit 32. The SDRAM 41 is a memory for temporarily storing image information processed by the image processing unit 25 or the like or image information compressed by the compression unit 31.

  The imaging control unit 20 is connected to each unit such as the body CPU 22 via the data bus 30. The recording medium control unit 35 is connected to the recording medium 43 and controls recording of image data and the like on the recording medium 43. The recording medium 43 includes a rewritable recording medium such as an xD picture card (registered trademark), a compact flash (registered trademark), an SD memory card (registered trademark), a memory stick (registered trademark), or a hard disk drive (HD). The imaging system main body can be attached and detached.

  The video signal output unit 36 is connected to a display monitor (display means) 46 via a display monitor control unit 45. The video signal output unit 36 is a circuit for converting the image data stored in the SDRAM 41 or the recording medium 43 into a video signal for display on the display monitor 46. The display monitor 46 is, for example, a liquid crystal display device disposed on the back surface of the imaging system main body. However, the display monitor 46 is not limited to the back surface and may be other display devices as long as the photographer can observe. Absent.

  The operation unit 47 includes a switch (shutter button) for detecting a release indicating a shooting instruction of the imaging system 1, a mode dial, a power switch, a control dial, a playback button, a menu button, a cross key, an OK button, and the like. It is connected to the data bus 30 via the unit 34.

Next, the operation of the imaging system 1 having the above configuration will be described with reference to FIGS. FIG. 2 is a diagram showing an outline of a sequence flow from power-on to storage processing of the actual captured image.
In FIG. 2, when the digital camera is turned on by the photographer, the memory for storing the through image is cleared (step SA1). This memory is, for example, the SDRAM (storage means) 41 described above. Subsequently, user setting information such as a shooting mode and zoom is detected (step SA2). The user setting is performed by, for example, the operation detection unit 34 (see FIG. 1).

Subsequently, autofocus (step SA3) and exposure calculation for through image shooting are performed (step SA4). Here, in the through image acquisition, the frame rate is prioritized and the resolution is set small. In the present embodiment, the through image sensitivity can be set from the lowest sensitivity of the image sensor to a high sensitivity equivalent to ISO 6400, and the exposure amount is determined so as to correspond to the highest possible frame rate. As a result, depending on the setting mode, setting sensitivity, setting shutter speed, and setting aperture at the time of actual shooting, a through image has a wider exposure interlocking range than the actual shooting image, and even an image that is under or over at the time of actual shooting. In the through image, it is possible to shoot with appropriate exposure.
In addition, when the sensitivity is high, a process of increasing the gain after adding a plurality of pixels in the image sensor is performed, so that an image with low noise but less noise can be obtained due to the effect of pixel addition.

Subsequently, a through image shooting display process and a through image storage process are performed (step SA5). Details of this processing will be described later.
Next, it is determined whether or not the shutter button is half-pressed (step SA6). If the shutter button is not half-pressed, it is confirmed whether or not the power switch that is the main operation switch of the camera is on (step S6). SA15) If the power switch is in the ON state, the process returns to step SA2, and the processes after step SA2 are repeated. If the power switch is off, an end process (step SA16) is performed to end the camera sequence. On the other hand, if the shutter button is half-pressed in step SA6, autofocus (step SA7) and exposure calculation (step SA8) are performed, and then it is determined again whether or not the half-pressed state is released. (Step SA9).

  As a result, when the half-pressed state is released (“NO” in step SA9), step SA15 is performed. On the other hand, if the half-pressed state has continued, it is determined whether or not the full-press has been made (step SA10). As a result, when the shutter button is not fully pressed and the half-pressed state is still continued (“NO” in step SA10), after the through image shooting display process and the through image storage process are performed (step S10). SA11), returning to step SA9, the processing after step SA9 is repeated.

  On the other hand, if it is determined in step SA10 that the shutter button has been fully pressed (“YES” in step SA10), a main photographing process is performed (step SA12), and then the main photographing image acquired in the main photographing. Is subjected to image processing (step SA13), the actual photographed image is recorded (step SA14), the photographing process is terminated, and then whether or not the power switch, which is the main operation switch of the camera, is on. Is confirmed (step SA15), and if the power switch is on, the process returns to SA2 and returns to the normal loop.

[Through image shooting display processing and through image storage processing]
Next, the through image shooting and display process and the through image storage process performed in step SA5 and step SA11 will be described with reference to FIG. FIG. 3 is a flowchart showing the processing procedure of the through image shooting display processing and the through image storage processing.

In FIG. 3, when a through image is captured (step SB1), various known images such as OB subtraction, gradation processing, γ conversion, and auto white balance processing are applied to the through image acquired in the through image capturing. After the processing is performed and the image processing for display is further performed, the through image after the image processing is displayed on the display monitor 46 (see FIG. 2) (step SB2).
Next, the image processing for storing the memory is performed on the image acquired in the through image shooting in step SB1 (step SB3). In the present embodiment, a through image is stored. However, in order to save memory, only luminance information may be extracted and stored depending on the purpose.

  Subsequently, it is determined whether the memory capacity of the memory for storing the through image, for example, SDRAM 41 (see FIG. 1) is sufficient (step SB4). As a result, when a sufficient memory capacity can be secured to store the through image that has been subjected to image processing for memory storage (“YES” in step SB4), the through image is stored in the memory (step SB6). ), The process ends. On the other hand, if there is not enough memory capacity to store the live view image, the oldest live view image stored in the memory is deleted to secure the memory capacity (step SB5), and this time created The through image is stored in the memory (step SB6), and the process ends.

[Image processing of actual shot image]
Next, image processing of the actual captured image performed in step SA13 in FIG. 2 will be described with reference to FIG. Here, among various image processes performed on the actual captured image, an image process performed using the above-described through image will be described.

First, a through image selection process used in the image processing of the actual captured image is performed (step SC1 in FIG. 4). Details of the through image selection processing will be described later.
Next, the through image selected in step SC1 is set as a reference image (step SC2), the reference image is compared with the actual captured image (step SC3), and it is determined whether or not the quality of the actual captured image can be improved. To do. For example, when the actual captured image has a lower sensitivity than a preset standard sensitivity, when the luminance average value of the reference image is a predetermined value or less, it is determined that high image quality is impossible.

As a result, if it is determined that high image quality is impossible, the process is terminated. On the other hand, if it is determined that high image quality is possible (“YES” in step SC4), level correction, color conversion processing, and the like are performed so that the exposure conditions and the like of the reference image are substantially the same as those of the captured image. Image processing is performed on the reference image (step SC5).
Subsequently, the actual captured image is divided into a plurality of areas, and one of the areas (hereinafter referred to as “selected area”) is selected (step SC6). Then, the selected area is compared with the reference image, and an area similar to the selected area is specified in the reference image (step SC7). At this time, for example, a similar area is selected from the reference image using a known technique such as block matching based on the luminance signal of the selected area in the actual captured image. In this case, the area may be selected after correction is performed so that the sizes of the reference image and the actual captured image match.

  Subsequently, using the information on the similar area selected in the reference image, noise reduction processing is performed on the selected area in the actual captured image (step SC8). Specifically, by resizing either the area selected in the reference image or the selected area of the actual photographic image, the low frequency components in both areas are extracted after matching the area sizes of both areas. After weighting the low-frequency component, the noise reduction processing in the selected area of the actual captured image is performed by combining the high-frequency component of the selected area of the actual captured image with the low-frequency component after the weighted calculation based on the size of the actual captured image. Do.

  Subsequently, it is determined whether or not noise reduction processing has been performed for all areas divided in the captured image (step SC9). If noise reduction processing has not been completed for all areas (step SC9, “ NO "), select another area that has not been processed (step SC10), and return to step SC7 to perform the subsequent processing. On the other hand, when the noise reduction process is completed in all areas (“YES” in step SC9), all areas after the noise reduction process are weighted and synthesized to create a final actual captured image (step SC11). This process is terminated.

[Through image selection processing]
Next, the through image selection process performed in step SC1 of the image processing of the actual captured image described with reference to FIG. 4 will be described with reference to FIG.
First, the through image and the actual captured image are compared, and image analysis of the actual captured image is performed to extract a through image that can be used for noise reduction processing of the actual captured image (step SD1). For example, a through image in which at least one of resolution and sensitivity is different from a captured image is extracted from through images stored in the memory.
Next, each extracted through image and the actual captured image are matched (step SD2), and the through image most similar to the actual captured image is selected (step SD3).

[Matching process]
Details of the matching process performed in step SD2 of FIG. 5 will be described with reference to FIG.
First, an image obtained by resizing the central portion of about 50% of the actual captured image to the size of the through image is set as a template image (step SE1), and then a comparison initial position is set on the through image (step SE2). The similarity between the template image and the through image is calculated at the comparison position (step SE3). For example, the similarity is calculated by calculating a difference between pixel values of pixels at corresponding positions, and calculating an integrated value of all the differences as the similarity. The similarity calculation method is not limited to the above calculation method. For example, the histogram of each image may be calculated, the histogram similarity may be used as the image similarity, and color information may be taken into account. The similarity may be obtained.

Subsequently, it is determined whether or not the calculation of the similarity is completed over the entire search area (step SE4). If the calculation is not completed, the comparison position is changed (step SE5), and then the process returns to step SE3 and the similarity is calculated. Is calculated.
When the calculation of the similarity over the entire search region is completed (“YES” in step SE4), the similarity at each position calculated in step SE3 is evaluated, and the similarity with the highest value is determined as the similarity of the through image. At the same time (step SE6), the position having the highest degree of similarity is set as a reference for alignment between the actual captured image and the through image (step SE7), and the present process is terminated.
Then, the matching process shown in FIG. 6 is performed on all the through images extracted in step SD1 in FIG. 5, whereby the similarity of each through image is calculated. In step SD3 in FIG. A high through image is selected.

  In the above example, the similarity is calculated by template matching. However, other known methods may be used as long as it can be determined whether or not the scene is the same as the scene at the time of actual shooting. Further, when creating a template image, color information may be left and color similarity may be taken into account. Further, the search area and the search method are not limited to the above methods. For this, for example, a technique used in known tracking or the like may be used.

  Moreover, it is good also as matching by changing an angle. By changing the angle in this way, the matching accuracy can be improved. Alternatively, the subject may be detected from the framing information, and only the peripheral portion of the subject may be cut out and block matching may be performed.

As described above, according to the imaging system according to the present embodiment, a through image is acquired prior to the main shooting, and noise of the main shooting image is reduced using information of the through image. In this case, since the through image has a lower resolution than the actual captured image, it is possible to reduce the processing burden by reducing the noise of the actual captured image using such a low resolution image. In addition, the noise reduction effect can be improved by synthesizing with the main image a through image that expresses features such as image edges because of low resolution. Furthermore, since the through image has a smaller data capacity than the actual captured image, it does not impose a memory capacity and is advantageous in that a large-capacity memory is not required.
In addition, by adopting through images with different sensitivities as through images used for image processing of the actual captured image, it is possible to improve noise reduction performance by correcting with images with low sensitivity and less noise. it can. Further, even if the sensitivity is high, correction can be performed using a through image in which noise is reduced by pixel addition.

  In the above embodiment, instead of the through image selection process shown in FIG. 5, for example, a through image acquired immediately before the shutter button is fully pressed or a predetermined number of images may be selected. At this time, the through image to be selected is required to have different shooting conditions from the actual shooting.

[Second Embodiment]
Next, a second embodiment of the present invention will be described.
In the first embodiment described above, noise reduction processing is performed using a through image. However, the present embodiment is different in that color correction is performed using a through image. More specifically, the image processing shown in FIG. 7 is executed instead of the image processing of the actual captured image described with reference to FIG. 4 in the first embodiment.
Hereinafter, the imaging system according to the present embodiment will be described mainly with respect to differences from the first embodiment.

[Image processing of actual shot image]
FIG. 7 is a diagram illustrating a processing procedure of image processing of a main captured image according to the present embodiment.
First, a through image selection process used in the image processing of the actual captured image is performed (step SF1 in FIG. 7). Details of the through image selection processing are substantially the same as those in the first embodiment described above, but in this embodiment, the through image extracted in step SD1 in FIG. Different through images are used.

  Next, the through image selected in step SF1 is set as a reference image (step SF2), the reference image is compared with the actual captured image (step SF3), and it is determined whether or not color correction of the actual captured image is possible. (Step SF4). For example, when the difference between the exposure conditions of the reference image and the actual captured image is equal to or smaller than a predetermined value, it is determined that color correction is not performed because the effect of color correction is small, in other words, color correction is impossible.

As a result, when it is determined that color correction is impossible (“NO” in step SF4), this process ends. On the other hand, if it is determined that color correction is possible (“YES” in step SF4), the white balance and the like are corrected so that the conditions regarding the color at the time of shooting the reference image are substantially the same as those of the actual shot image. (Step SF5).
Next, alignment processing and resizing processing of the main captured image and the reference image are performed (step SF6), and then the main captured image and the reference image are divided into a plurality of areas (step SF7). Next, a color map is created in each area of the reference image (step SF8), and a color map is created in each area of the actual captured image (step SF9), and these are stored in the memory.

  The color map includes an average value of R / G (red / green), an average value of B / G (blue / green), and an average value of G (green) in each area of the reference image and the actual captured image. It is created by calculating.

  Next, the color map of the reference image and the color map of the actual captured image are compared. Specifically, the average value of R / G (red / green) and the average of B / G (blue / green) in each area Each value is compared, and a value closer to the intermediate gradation is selected (step SF10). Next, the average value of each selected R / G (red / green) and the average value of B / G (blue / green) are set as the target colors of the respective values, and these values are used as R / G ( The average value of red / green) and the average value of B / G (blue / green) are divided to obtain color correction coefficients in each area (step SF11). Alternatively, each selected value may be set as a target color, and this value may be divided by an average value of G (green) of the main image to obtain a color correction coefficient in each area.

  Next, it is determined whether or not color correction coefficients have been calculated in all areas (step SF12). If color correction coefficients have not been calculated for all areas ("NO" in step SF12), step SF8 is performed. Returning to the above, the above process is repeated in an unprocessed area.

  On the other hand, when the color correction coefficient is calculated in all areas (“YES” in step SF12), the final actual photographing is performed by performing color correction on the actual captured image based on the color correction coefficient in each area. An image is created (step SF13), and this process ends. Here, as a specific color correction process, R / G (red / green) and B / G (blue / green) of each pixel of the main image are multiplied by the above-described correction coefficient for each area. Correction is performed, and color correction processing is performed by calculating R, G, and B again from the result.

  When each value is a target color and this value is divided by the average value of G (green) of the main image to obtain a color correction coefficient in each area, it is converted to G (green) of each pixel of the image. On the other hand, R and B are calculated again by multiplying the correction coefficients that differ for each area, and color correction processing is performed.

  At the time of these color corrections, the correction coefficient is calculated based on the color map closer to the intermediate gradation in the color map of the reference image and the color map of the actual captured image. Therefore, even if the color information of the reference image remains even in the portion where the color information is not accurate, the color can be reproduced using them.

As described above, according to the imaging system according to the present embodiment, a through image is acquired prior to the main shooting, and color correction processing of the main shooting image is performed using information of the through image. In this case, since the resolution of the through image is lower than that of the actual captured image, it is possible to reduce the processing burden by performing color correction of the actual captured image using such a low resolution image. Become. Furthermore, since the through image has a smaller data capacity than the actual captured image, it does not impose a memory capacity and is advantageous in that a large-capacity memory is not required.
In addition, by performing color correction of the actual captured image using a through image with different exposure conditions from the actual captured image, even if the color information is not accurate due to underexposure or overexposure in actual capture, If the color information of the reference image remains, it is possible to reproduce the color using them.

[Third Embodiment]
Next, a third embodiment of the present invention will be described.
In the first embodiment described above, noise reduction processing is performed using a through image. However, the present embodiment is different in that blur correction is performed. More specifically, the image processing shown in FIG. 8 is executed instead of the image processing of the actual captured image described with reference to FIG. 4 in the first embodiment.
Hereinafter, the imaging system according to the present embodiment will be described mainly with respect to differences from the first embodiment.

[Image processing of actual shot image]
FIG. 8 is a diagram illustrating a processing procedure of image processing of a main captured image according to the present embodiment.
First, it is determined whether or not blur correction is necessary in the actual captured image (step SG1). For example, it is determined that blur correction is necessary when the shooting shutter speed is longer than a predetermined shutter speed determined in accordance with the focal length at the time of actual shooting. As other methods, for example, the determination may be made using information such as the contrast of the actual captured image, or a determination may be made by using a camera shake correction sensor or the like and using its output.

  As a result, when it is determined that blur correction is not necessary (“NO” in step SG1), this process is terminated. On the other hand, if it is determined that blur correction is necessary (“YES” in step SG1), subsequently, a through image selection process used in the image processing of the actual captured image is performed (step SG2). The details of the through image selection processing are substantially the same as those in the first embodiment described above, but in this embodiment, the through image extracted in step SD1 in FIG. It is a through image acquired at a speed.

Next, the through image selected in step SG2 is set as the reference image (step SG3). Subsequently, the exposure condition of the reference image, the white balance, etc. so that the shooting condition of the reference image is substantially the same as that of the actual shot image. Is corrected (step SG4).
Next, each of the actual captured image and the reference image is divided into a plurality of areas, block matching is performed for each area, and the fluctuation of coordinates between the actual captured image and the reference image of the highly correlated block is calculated. Each area motion vector is obtained (step SG5).

Next, from the motion vectors for each area, the motion vector with the least frequency is estimated as the blur area of the subject, the area with the motion vector with the highest frequency is estimated as the motion area due to camera shake between frames, and the motion with the highest frequency The vector is calculated as a blur amount (step SG6).
Subsequently, each pixel of the reference image is aligned with each pixel of the reference image while shifting the same amount in the opposite direction to the amount of blur so as to cancel out the amount of blur, and the pixels are added at a predetermined ratio. Thus, blur correction is performed, a final actual captured image is created (step SG7), and the present process is terminated.

As described above, according to the imaging system according to the present embodiment, a through image is acquired prior to the main shooting, and blur correction processing of the main shooting image is performed using information of the through image. In this case, since the resolution of the through image is lower than that of the actual captured image, it is possible to reduce the processing burden by performing color correction of the actual captured image using such a low resolution image. Become. Furthermore, since the through image has a smaller data capacity than the actual captured image, it does not impose a memory capacity and is advantageous in that a large-capacity memory is not required.
In addition, by performing blur correction on the actual captured image using a through image whose shutter speed is faster than at the time of actual shooting, it is possible to perform highly accurate blur correction, and even when subject blur occurs. It becomes.

  Note that the noise reduction processing in the first embodiment, the color correction processing in the second embodiment, and the blur correction processing in the third embodiment have been described, but these embodiments are combined as appropriate. Is possible. For example, the image processing unit 25 illustrated in FIG. 1 may perform all of noise reduction processing, color correction processing, and blur correction processing, or may arbitrarily execute any combination of these processing. .

  In the above-described embodiment, hardware processing is assumed as the image processing apparatus 3, but it is not necessary to be limited to such a configuration. For example, a configuration in which processing is performed separately by software is possible. In this case, the image processing apparatus 3 includes a main storage device such as a CPU and a RAM, and a computer-readable recording medium on which a program for realizing all or part of the above processing is recorded. Then, the CPU reads out the program recorded in the storage medium and executes information processing / calculation processing, thereby realizing processing similar to that of the image processing apparatus 3 described above.

  Here, the computer-readable recording medium means a magnetic disk, a magneto-optical disk, a CD-ROM, a DVD-ROM, a semiconductor memory, or the like. Alternatively, the computer program may be distributed to the computer via a communication line, and the computer that has received the distribution may execute the program.

DESCRIPTION OF SYMBOLS 1 Imaging system 2 Imaging part 3 Image processing apparatus 20 Shooting control part 25 Image processing part 34 Operation detection part 41 SDRAM
46 Display monitor

Claims (10)

Photographic optics,
An image sensor for imaging a subject via the imaging optical system;
Photographing operation detecting means for detecting a photographing operation of the photographer;
First shooting control means for performing actual shooting when the shooting operation is performed;
Prior to the main shooting, a second shooting control means for performing shooting in advance under shooting conditions different from the main shooting;
Storage means for storing information relating to the preliminary image acquired in the preliminary shooting;
An image processing system comprising: an image processing unit that corrects a main captured image acquired by the main shooting using the prior image.   The imaging system according to claim 1, wherein the second imaging control unit performs preliminary imaging with a resolution or exposure different from the main imaging.   The imaging system according to claim 1, further comprising display means for displaying the preliminary image. The image processing means calculates the degree of similarity by comparing the information about the prior image and the information about the actual captured image,
The imaging system according to any one of claims 1 to 3, wherein the actual captured image is corrected using information related to the prior image in which the similarity satisfies a predetermined condition. The image processing means includes
Aligning the information related to the previous image and the information related to the actual captured image,
5. The imaging system according to claim 1, wherein the pre-image and the actual captured image are synthesized in a state where the alignment is performed.   The imaging system according to any one of claims 1 to 5, wherein the image processing unit reduces noise components of the main captured image using information about the prior image.   The imaging system according to claim 1, wherein the image processing unit corrects the color of the main captured image using information about the prior image.   8. The imaging system according to claim 1, wherein the image processing unit corrects camera shake or subject blur in the main captured image using information related to the prior image. 9. .   An image processing method for an imaging apparatus, characterized in that prior to main shooting, pre-shooting is performed with a resolution different from that of main shooting, and an image acquired in the main shooting is corrected using an image obtained by the previous shooting. . An image processing program for performing image processing,
For performing a correction process for correcting the image acquired in the main shooting using the image acquired in the previous shooting performed at a resolution different from that of the main shooting, prior to the main shooting. Image processing program.

Images