JP2006325274A - Imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging device capable of performing excellent image synthesis processing by using a plurality of images with different exposure previously photographed. <P>SOLUTION: The image forming device is provided with a CCD image sensor 1 for photoelectrically converting a subject image and outputting as an electrical signal, an image processing device 2 for performing color correction or edge emphasis, etc. of images, a memory 3 for storing image data, an image synthesizing apparatus 4 for synthesizing a plurality of images, and a CPU 5 for controlling the respective devices. The memory stores a plurality of images with different exposure previously photographed by the image sensor. The image synthesizing apparatus inputs a plurality of images with different exposure stored in the memory and applies synthesis processing. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an imaging apparatus including an image composition device that synthesizes and outputs a plurality of images having different exposure amounts.

  In an imaging device having a solid-state image sensor, the dynamic range of the solid-state image sensor is narrower than that of a silver salt film. For example, when shooting a subject under a bright background, the subject may be blacked out or the background may be overexposed. Arise. In order to prevent such a phenomenon, a method has been proposed in which images of appropriate exposure are taken for each of the subject and the background, and a plurality of images having different exposure amounts are combined.

  FIG. 20 shows a conceptual diagram of a method for obtaining an image having a wide dynamic range by combining two images having different exposure amounts. In FIG. 20, it is assumed that the long-exposure image shown in FIG. 20 (B) has a person or tree exposed and the background is overexposed. Further, it is assumed that the short-time exposure image shown in FIG. 20A is exposed in the background and the person is blackened. These images can be synthesized by the following method, for example. First, an appropriate threshold value is set for the long-exposure image so that the region where the pixel value is equal to or greater than the threshold value matches the overexposed region. Next, for the area equal to or greater than the threshold, an image corresponding to the short exposure image (at the same position) is pasted and combined. By synthesizing in this way, an image with a wide dynamic range in which both the background and the person are exposed can be obtained as shown in FIG.

  However, when shooting multiple images with different exposure amounts with a single image sensor, the shooting timing of each image is shifted, so that for example when the subject is moving or camera shake occurs, there is a positional shift between the images. Arise. When a plurality of images having such positional deviations are combined, an unnatural image such as a double subject is formed as shown in FIG.

  Conventionally, Japanese Patent No. 3110797 has proposed a method for correcting such image misalignment. Next, a conventional correction method disclosed in Japanese Patent No. 3110797 will be described. FIG. 22 is a conceptual diagram showing a conventional correction method. As shown in FIG. 22, when the image a and the image b are shifted on the whole screen due to camera shake, the conventional correction method shifts these images so that the common area portion in FIG. 22 is obtained. Synthesizing.

FIG. 23 is a block diagram illustrating a configuration example of an image composition device for performing a conventional correction method. The memory control unit 205 controls the field memory 204a based on the screen blur information detected by the motion detection 206 using the input image, and reads an image corresponding to the common area in FIG. The common area image is enlarged to the same size as the input image by the enlargement interpolation unit 202a, and then stored in the field memory 204b. Subsequently, the same processing is performed, and the image output from the enlargement interpolation unit 202a and the image of the previous frame read out from the field memory 204b are combined by the adder 202b, so that an image with corrected blur is obtained. ing. In FIG. 23, 201 is an A / D converter, and 203 is a D / A converter.
Japanese Patent No. 3110797

  As described above, the conventionally proposed correction technique shifts the entire image to obtain a common area and performs image blur correction, which is a suitable method for correcting camera shake that blurs the entire image. However, the image blur includes a subject blur in which a part of the subject moves. In the two images shown on the left side of FIG. 24, it is assumed that the entire image is shaken, and that the airplane is shaken in a direction different from the shake direction. Therefore, when the image as shown in FIG. 24 is corrected by shifting the entire screen in the direction in which the airplane moves, the image is synthesized as an image in which other subjects such as people and trees are shifted. As described above, when the entire image blurred in the subject is shifted and combined, there is a case where the combined image may not be appropriately combined. However, in the conventionally proposed correction technique, this viewpoint is not taken into consideration.

  The present invention has been made in view of the above circumstances, and provides an imaging device including an image composition device that can perform a good image composition process using a plurality of images that have been captured in advance and have different exposure amounts. With the goal.

  In order to solve the above-mentioned problem, the invention according to claim 1 is directed to an image sensor that photoelectrically converts a subject image and outputs it as an electrical signal, a memory for storing image data, and an image composition for synthesizing a plurality of images. The memory stores a plurality of images with different exposure amounts previously captured by the image sensor, and the image composition device stores the exposure amount stored in the memory. A plurality of different images are input and subjected to synthesis processing.

  According to a second aspect of the present invention, in the imaging apparatus according to the first aspect, the image synthesizing apparatus detects a motion vector using the plurality of images having different exposure amounts read from the memory, and has different exposure amounts. A motion vector detection unit that generates a binarized image of a plurality of images and stores the binarized image in the memory, and a combination using the binarized images of the motion vector and the plurality of images stored in the memory with different exposure amounts And switching means for switching and outputting the image.

  According to a third aspect of the present invention, there is provided an imaging apparatus comprising: an imaging element that photoelectrically converts a subject image and outputting the result as an electrical signal; a memory that stores image data; The memory stores a plurality of images with different exposure amounts previously captured by the imaging device, and the image composition device inputs the plurality of images with different exposure amounts stored in the memory. Then, the type of blur is determined, and based on the determination result, either the camera shake correction process or the subject blur correction process is switched and output, and a composite image is output.

  According to a fourth aspect of the present invention, in the imaging device according to the third aspect, the image composition device detects a motion vector using a plurality of images having different exposure amounts read from the memory, and has different exposure amounts. A motion vector detection unit that generates a binarized image of a plurality of images and stores the binarized image in the memory, and blurs using the binarized images of the plurality of images having different exposure amounts stored in the motion vector and the memory. A blur determination unit that determines whether or not the image has occurred and outputs the resultant image, and based on the result of the blur determination unit, performs either a camera shake correction process or a subject blur correction process to obtain a composite image. And switching means for switching output.

  According to a fifth aspect of the present invention, in the imaging device according to any one of the first to fourth aspects, the image synthesis device further includes a low-pass filter, and applies the low-pass filter to the image signal subjected to the synthesis process. It is comprised so that it may be comprised.

  According to a sixth aspect of the present invention, in the imaging apparatus according to the second or fourth aspect, the image synthesizing device further includes a motion vector detection region specifying unit that specifies a region for detecting a motion vector, and the switching unit includes the switching unit, The image signal corresponding to the motion vector detection region is configured to be subjected to a low pass filter.

  According to a seventh aspect of the present invention, in the imaging device according to the sixth aspect, the image synthesizing apparatus includes a blur type determination unit that determines and outputs the type of blur, and the switching unit determines the blur type determination. When the result output from the unit indicates subject blur, a low-pass filter is applied to an image signal corresponding to the motion vector detection region for an image obtained by combining the one image and the other image. It is characterized by being comprised.

  According to an eighth aspect of the present invention, in the imaging apparatus according to the fourth aspect, the image synthesizing apparatus sets horizontal and vertical coordinates of one of the plurality of images having different exposure amounts based on the motion vector. A mismatch area between the images is obtained from the image shifted in the direction and the other image of the plurality of images having different exposure amounts, and the switching unit horizontally converts the one image based on the motion vector. And a low-pass filter is applied to a pixel image signal corresponding to the inconsistent region with respect to an image stabilization image obtained by synthesizing an image whose coordinates are shifted in the vertical direction and the other image. It is a feature.

  The invention according to claim 9 is the imaging apparatus according to claim 4, wherein the image composition device calculates coordinates of a binary image related to one of the plurality of binary images based on the motion vector. A mismatch area between images is obtained from an image shifted in the horizontal and vertical directions and the other image of the plurality of binary images, and the switching means determines the one image based on the motion vector. It is configured to apply a low-pass filter to an image signal corresponding to the inconsistent region with respect to a camera shake correction image obtained by synthesizing an image whose coordinates are shifted in the horizontal and vertical directions and the other image. It is a feature.

  According to a tenth aspect of the present invention, in the imaging apparatus according to the ninth aspect, the switching means coordinates the one image in the horizontal and vertical directions based on the motion vector when the blur determination signal indicates camera shake. The camera shake correction image obtained by combining the image with the shifted image and the other image is configured to apply a low-pass filter to the pixel image signal corresponding to the mismatch region. is there.

  ADVANTAGE OF THE INVENTION According to this invention, the imaging device which can perform a favorable image composition process is realizable using the several image image | photographed in advance from which the exposure amount differs. Further, it is possible to easily obtain image information capable of image processing with less influence of camera shake and subject shake using a plurality of images obtained by shooting a moving subject.

  Next, the best mode for carrying out the present invention will be described.

Example 1
First, a first embodiment of the imaging apparatus according to the present invention will be described. FIG. 1 is a block diagram illustrating a basic configuration of the imaging apparatus according to the first embodiment. 1 is a CCD image sensor that photoelectrically converts a subject image and outputs it as an electrical signal, 2 is an image processing device that performs color correction and edge enhancement of the image, 3 is a memory for storing image data, and 4 is a plurality of images 5 is a CPU for controlling these devices, and 6 is a bus for connecting the memory and other devices (image processing device, image composition device, CPU).

  An image captured by the CCD 1 is stored in the memory 3 via the bus 6. A plurality of images with different exposure amounts are captured and stored in the memory 3 in advance. The image synthesizing apparatus 4 inputs a plurality of images with different exposure amounts stored in the memory 3, determines the type of blur, performs a synthesizing process according to the determination result, and stores the memory 3 as a blur-corrected image. To store. The corrected image is read from the memory 3 as needed and processed in the image processing apparatus 2.

  Next, the image composition apparatus in Embodiment 1 will be described in detail. In the following description, one of the plurality of images having different exposure amounts will be referred to as an image a and the other as an image b. FIG. 2 is a block diagram showing a configuration example of the image composition device 4 in FIG. In FIG. 2, reference numeral 11 denotes a motion vector detection unit. The motion vector detection unit 11 detects one motion vector as a whole image using the images a and b read from the memory 3. At the time of motion vector detection, binary images a and b obtained by binarizing the images a and b are generated and stored in the memory 3. The shake determination unit 12 determines whether or not camera shake has occurred using the motion vector and the binary images a and b and outputs the result. Based on the result of the camera shake determination, the switching unit 13 appropriately switches the switches 16 and 17 to perform either the camera shake correction processing unit 14 or the subject shake correction processing unit 15 and output a composite image.

  Next, details of each component of the image composition apparatus shown in FIG. 2 will be described. FIG. 3 is a block configuration diagram showing a configuration example of the motion vector detection unit 11 in FIG. The motion vector detection area specifying means 21 specifies an area for detecting a motion vector. As shown in FIG. 4, a portion where the subject is shifted between the images a and b is specified as a motion vector detection region. Information on the identified motion vector detection area is temporarily stored in the memory 3. In addition, binary images a and b (described later) generated when the motion vector detection area is specified are also stored in the memory 3. The block setting means 22 reads information on the motion vector detection area from the memory 3 and sets one specific block as a reference block in motion vector detection. As shown in FIG. 5, a block set by the block setting means 22 is used as a reference block, a search range is provided around the block, and a motion vector calculation unit 23 detects a motion vector. The reference block and the search range are set in different images. For example, when the reference block is set to the image b, the search range is set to the image a.

  The motion vector is calculated only for the reference block set by the block setting means 22, and the result is used as one motion vector for the entire image. As a method for calculating the motion vector, for example, a block matching method is used. For the reference block set by the block setting means 22, the similarity to each block in the motion vector search range is obtained, and the vector to the block with the highest similarity is used as the motion vector. The validity determination unit 24 determines whether the block set by the block setting unit 22 is appropriate as a reference block for motion vector detection, and the validity of the motion vector calculated by the motion vector calculation unit. If these are not appropriate, the motion vector calculation unit 23 is controlled not to output the calculated motion vector.

  In the above description, the motion vector detection area information is stored in the memory 3. However, a dedicated memory may be provided between the motion vector detection area specifying means 21 and the block setting means 22.

FIG. 6 is a block diagram showing a configuration example of the motion vector detection region specifying means 21 in FIG. Images a and b are binarized by binarization units 31 and 32, respectively, and binary images a and b are generated. At this time, the binarization threshold values of the images a and b are set so as to satisfy the relationship of the following expression (1).
Threshold b = threshold a × (exposure amount b / exposure amount a) (1)
By setting each threshold value as in Expression (1), the same binary image can be obtained if there is no deviation between the images a and b. Therefore, on the contrary, the portion where the deviation occurs has different values in the binary images a and b.

  The binary images a and b are compared with each other by the comparison unit 33, and a motion vector detection region is output as a binary image whose region having a different value is “1”. FIG. 7 is a block configuration diagram showing a more detailed configuration example of the comparison unit 33. By taking the exclusive OR of the binary images a and b by the exclusive OR circuit 41, binary image mismatch region information is generated so that the mismatch region is “1”. This binary image mismatch region information is counted when the number of consecutive “1” s is counted by the counter 42 that enables it, and the count value exceeds a preset continuous detection amount threshold by the comparator 43. Only “1” is output through the selector 44. By adopting this configuration, inconsistencies caused by noise and the like are removed.

  FIG. 8 is an explanatory diagram of a block setting method in the block setting means 22 in FIG. The block setting means 22 sets one specific block in the motion vector detection area stored in the memory 3 as a reference block for motion vector detection. In the illustrated example shown in FIG. 8, the image is divided into blocks, and information on the motion vector detection area is searched spirally from the central block on the screen toward the outer edge. When a block including “1” (mismatch area) is found first, the block of the image b corresponding to the position of the block is read from the memory 3 and output as a reference block. According to such a method, a block as close to the center of the image as possible can be set as a reference block, and the possibility that a motion vector can be found can be increased as compared with the case where the reference block is taken at the edge of the screen.

  Of course, the block setting method is not limited to the above method. For example, the motion vector detection area may be searched in order from the upper left of the screen, and the block including the mismatch area first may be set as the reference block. According to such a method, it is possible to perform addressing when information on the motion vector detection region is read from the memory 3 by a simple method. Further, a block included in an area where the number of consecutive mismatches in the horizontal direction of the screen is the maximum may be set as a reference block. Such a non-matching region that is long in the horizontal direction of the screen is highly likely to be displaced due to camera shake occurring in the vertical direction of the screen, and can be highly likely to detect a motion vector due to camera shake. Naturally, a block included in an area having the largest number of consecutive mismatches in the vertical direction of the screen may be set as the reference block. It is also possible to set a block by combining a plurality of the above methods.

  FIG. 9 is a block diagram illustrating a configuration example of the validity determination unit 24 in FIG. The image b is determined to be overexposed, underexposed, or gray. If all pixel values in an arbitrary area of the image b are equal to or greater than the white threshold value, the whiteout determination unit 51 determines that the area is overexposed. If the entire pixel value of the arbitrary area of the image b is equal to or less than the black threshold value, the blackout determination unit 52 determines that the area is blackout. Further, the pixel value maximum value and minimum value of an arbitrary region of the image b are obtained from the maximum value extraction unit 53 and the minimum value extraction unit 55, respectively, and the difference value is calculated by the subtractor 57. The difference value is compared with a luminance difference threshold value by the comparator 59, and if it is equal to or smaller than the threshold value, it is determined that there is no luminance difference in the region and that the color is gray.

  Also, the validity of the motion vector is determined from the similarity. For the set of similarities output from the motion vector calculation unit 23, the maximum value extraction unit 54 and the minimum value extraction unit 56 obtain the maximum value and the minimum value. The difference value is calculated by the subtractor 58, and the result is compared with the similarity difference threshold value by the comparator 60. If the difference value is equal to or less than the threshold value, the vector to the maximum value of the set of similarity values is set as the motion vector. Is determined to be invalid.

  Assuming that an arbitrary area of the image b is a reference block set by the block setting means 22, if any of overexposure, blackout, or gray is determined, it is determined that the set reference block is not suitable for motion vector detection. can do. Further, if a search range (image a) in motion vector calculation from the reference block is input to these determination units, it can be determined whether or not the search range is suitable for motion vector detection.

  A valid signal is generated by combining the results of whiteout determination, blackout determination, gray determination, and motion vector invalidity determination in FIG. 9, and is output to the block setting means 22 and the motion vector calculation unit 23. Any combination is possible. For example, when no overexposure, underexposure, or gray is determined, a valid signal is output to the block setting unit 22, and when it is determined that the motion vector is invalid, the motion vector An effective signal is output to the calculation unit 23.

  FIG. 10 is a block diagram illustrating a configuration example of the shake determination unit 12 in FIG. The coordinate conversion means 61 shifts the binary image b read from the memory 3 in the horizontal and vertical directions based on the motion vector and outputs it as a binary image b ′. The difference calculating means 62 compares the binary images a and b ′ and outputs a difference image and difference information. FIG. 11 is a block diagram showing a more detailed configuration example of the difference calculating means 62. As shown in FIG. The exclusive OR circuit 71 calculates the exclusive OR of the binary images a and b 'to obtain a mismatch area between the binary images a and b' and outputs it as a difference image. Further, the number of “1” in the difference image is counted by the counter 72, and the number of pixels in the mismatch area between the binary images a and b ′ is output as difference information. As shown in FIG. 12, the difference image indicates a region that is inconsistent when the two images are combined on the assumption that the entire image has a shake in the direction of the motion vector, and the difference information indicates the mismatch region. It shows the size. The shake determination signal generation means 63 determines whether the hand shake or the subject shake from the difference information and the motion vector, and outputs the result as a shake determination signal.

  FIG. 13 is a block diagram showing a more detailed configuration example of the shake determination signal generation means 63. As shown in FIG. Here, camera shake is indicated when the shake determination signal is “1”, and subject shake is indicated when “0”. The motion vector presence / absence determination unit 81 outputs “1” when a motion vector is output from the motion vector calculation unit 23, and outputs “0” when a motion vector is not output, that is, when a motion vector is not detected. On the other hand, the difference information is compared with the difference threshold value by the comparator 82, and “1” is output when the difference information is equal to or less than the difference threshold value, and “0” is output when the difference information is equal to or greater than that. The two binary signals are ANDed by AND83 and output as a blur determination signal. According to such a configuration, when the motion vector is detected and the difference information is equal to or smaller than the threshold value, the shake determination signal is “1”, and it is determined that the camera shake has occurred. When the difference information is equal to or greater than the threshold value or when no motion vector is detected, the shake determination signal is “0” and it is determined that the subject is shaken.

  Here, the difference threshold can be arbitrarily set. For example, if a certain specific value is set, a case where the size of the mismatch area when it is corrected as a camera shake is less than a certain value can be regarded as a camera shake. Alternatively, the number of pixels in the mismatch area between the binary images a and b may be used as the difference threshold value. In this case, a result obtained by adding means for counting the number of pixels in the mismatched areas of the binary images a and b to the motion vector detection area specifying means 21 shown in FIG. If the difference threshold value is set in this way, a case where the mismatch area decreases before and after the correction as a result of the camera shake can be regarded as a camera shake.

  FIG. 14 is a block diagram showing a specific configuration example of the switching means 13 in FIG. The coordinate conversion means 91 generates an image obtained by shifting the image b in the horizontal and vertical directions based on the motion vector. The combining unit 92 is a combining unit that combines two images. Pixels corresponding to the region where the value of the binary image a is “1” (or “0”) is the image a, and pixels corresponding to the region “0” (or “1”) is the other image b. Synthesized. The low-pass filter 93 performs low-pass filter processing only on the area in the input image corresponding to the motion vector detection area. The switches 94 and 95 are switches whose connection is switched according to the shake determination signal.

  Next, an image composition operation when the shake determination signal is subject shake and camera shake will be described. When the blur determination signal is subject blur, a blur region of the synthesized image is corrected by applying a low-pass filter. First, the switch 94 is connected to the s2 side, and the images a and b are combined by the combining means 92. As shown in FIG. 15, the image obtained here is an image obtained by blurring and synthesizing the image of the region corresponding to the motion vector detection region. The synthesized image is subjected to low-pass filter processing in a region corresponding to the motion vector detection region. The switch 95 is connected to the s2 side, and stores the image subjected to the low-pass filter in the memory 3 as a composite image. As a result, the blurred portion of the image can be made inconspicuous, and an image with good image quality and a wide dynamic range can be obtained.

  When the camera shake determination signal is camera shake, the entire image is shifted in the direction of the motion vector. First, the coordinate conversion means 91 generates an image in which the image b is shifted in the direction of the motion vector. The switch 94 is connected to the s1 side, and the combining means 92 combines the image a and the image obtained by shifting the image b in the direction of the motion vector. The switch 95 is connected to s1 and stored in the memory 3 without applying a low-pass filter. As shown in FIG. 16, the image obtained by this processing is an image in which image blur due to camera shake is corrected, and an image with good image quality and wide dynamics can be obtained.

  As described above, according to the present embodiment, it is determined whether an image has a camera shake or a subject shake by using only one motion vector and an arithmetic operation between binary images as an image, and the image depends on the type of the shake. Since the combining method is appropriately switched, it is possible to determine blurring with simple processing, and it is possible to obtain an image with good image quality and a wide dynamic range.

(Example 2)
Next, a second embodiment of the present invention will be described. FIG. 17 is a block diagram illustrating a configuration of an image composition device in the imaging apparatus according to the present embodiment. Since the configuration other than the switching means 13a is the same as that of the image synthesizing apparatus of the first embodiment, the description thereof is omitted. In the switching means 13a in the present embodiment, the switches 16 to 18 are appropriately switched, and either the camera shake correction processing unit 14 or the subject shake correction processing unit 15 or both processes are performed to output a composite image.

  FIG. 18 is a block diagram showing a specific configuration example of the switching means 13a in the present embodiment. The coordinate conversion means 91, the synthesis means 92, the low-pass filter 93, and the switches 94 and 95 are the same as the switching means 13 in the first embodiment shown in FIG. The switch 96 is a switch for switching the region to be subjected to the low-pass filter between the motion vector detection region and the difference image in accordance with the shake determination signal.

  Next, an image composition operation when the shake determination signal is subject shake and camera shake will be described. When the blur determination signal is subject blur, the switch 96 is connected to the s2 side, and the motion vector detection region is selected as the region to be subjected to the low-pass filter, so the same processing as in the first embodiment is performed. When the camera shake determination signal is camera shake, the process is the same as that in the first embodiment until the entire image is shifted in the direction of the motion vector. That is, an image in which the image b is shifted in the direction of the motion vector is generated by the coordinate conversion means 91, the switch 94 is connected to the s1 side, and the image a and the image in which the image b is shifted in the direction of the motion vector are generated. Synthesis is performed by the synthesis means 92.

  Next, the switch 96 is connected to the s1 side, and low-pass filter processing is performed on the region corresponding to the difference image. As shown in FIG. 12, the difference image is a mismatch area between the binary image a and an image obtained by shifting the binary image b in the direction of the motion vector, and matches the shake occurrence area of the corrected image due to camera shake. . Accordingly, the low-pass filter is applied only to the blur region remaining after correction due to camera shake. Finally, the switch 95 is connected to the s2 side, and the image subjected to the low-pass filter is stored in the memory 3 as a synthesized image.

  FIG. 19 shows an example in which images in which both camera shake and subject shake are generated are combined. Even after the camera shake is corrected, the blur remains in the airplane part. By applying the low pass filter by the above method, the blur part of the airplane becomes inconspicuous, and an image with better image quality and a wide dynamic range can be obtained.

  As described above, according to the present embodiment, when an image is shaken, the shake is corrected by shifting the image to be combined, and the shake remaining after the correction is applied with a low-pass filter so that the shake is not noticeable. it can. Therefore, an image with good image quality and a wide dynamic range can be obtained even for an image in which camera shake and subject shake are mixed.

It is a block diagram which shows the basic composition of the imaging device which concerns on this invention. It is a block diagram which shows the structure of the image composition apparatus in the imaging device which concerns on Example 1 of this invention. FIG. 3 is a block diagram illustrating a configuration of a motion vector detection unit in the image composition device illustrated in FIG. 2. Image a. It is a figure which shows the motion vector detection area | region where the to-be-photographed object has shifted between b. It is explanatory drawing which shows the reference | standard block set with a block setting means, and the search range around it. It is a block diagram which shows the structure of the motion vector detection area specific | specification means in the motion vector detection part shown in FIG. It is a block diagram which shows the structure of the comparison part in the motion vector detection area specific | specification means shown in FIG. It is explanatory drawing which shows the block setting method of the block setting means in FIG. It is a block diagram which shows the structure of the validity determination part in FIG. It is a block diagram which shows the structure of the blur determination means in FIG. FIG. 11 is a block diagram showing a configuration of difference calculation means in the shake determination means shown in FIG. It is a figure which shows the mismatch area | region produced when two images are synthesize | combined supposing that the whole image has produced the camera shake in the direction of the motion vector. FIG. 11 is a block diagram illustrating a configuration of a shake determination signal generation unit in the shake determination unit illustrated in FIG. It is a block diagram which shows the structure of the switching means in FIG. It is explanatory drawing which shows the low-pass filter process area | region of a synthesized image. It is explanatory drawing which shows the image by which the image blurring by camera shake was correct | amended. It is a block diagram which shows the image composition apparatus in Example 2 of the imaging device which concerns on this invention. FIG. 18 is a block diagram showing a configuration of switching means in FIG. It is explanatory drawing which shows a low-pass filter process area | region at the time of synthesize | combining the image in which both the hand movement and the subject blur have occurred. It is a conceptual diagram which shows the method of synthesize | combining the image from which exposure amount differs, and obtaining an image with a wide dynamic range. It is a figure which shows a synthetic | combination image when a to-be-photographed object moves or camera shake has generate | occur | produced. It is explanatory drawing which shows the camera-shake correction method in the conventional synthesized image. It is a block diagram which shows the structural example of the image synthesis apparatus to which the conventional camera shake correction method is applied. It is a figure which shows the synthesized image by the conventional image synthesizing | combining apparatus.

Explanation of symbols

1 CCD
2 Image processing device 3 Memory 4 Image composition device 5 CPU
6 Bus
11 Motion vector detector
12 Shake detection means
13, 13a switching means
14 Image stabilizer
15 Subject blur correction processor
16, 17, 18 switches
21 Motion vector detection area identification means
22 Block setting method
23 Motion vector calculator
24 Validity judgment part
31, 32 Binarization part
33 Comparison part
41 Exclusive OR circuit
42 counter
43 Comparator
44 selector
51 White spot determination unit
52 Black crush detection section
53, 54 Maximum value extraction unit
55, 56 Minimum value extraction unit
57, 58 subtractor
59, 60 comparator
61 Coordinate transformation means
62 Difference calculation means
63 Shake detection signal generation means
71 Exclusive OR circuit
72 counter
81 Motion vector presence / absence judgment unit
82 Comparator
83 AND circuit
91 Coordinate transformation means
92 Synthesis means
93 Low-pass filter
94, 95, 96 switches

Claims (10)

An image sensor that photoelectrically converts a subject image and outputs it as an electrical signal;
A memory for storing image data;
An image capturing apparatus including an image composition device that performs composition processing of a plurality of images,
The memory stores a plurality of images with different exposure amounts previously captured by the image sensor,
The image synthesizing apparatus is configured to input a plurality of images having different exposure amounts stored in the memory and perform a synthesizing process. The image composition device includes:
Detecting a motion vector using a plurality of images with different exposure amounts read from the memory, and generating a binary image of the plurality of images with different exposure amounts and storing them in the memory;
The imaging apparatus according to claim 1, further comprising: a switching unit that switches and outputs a composite image using a binarized image of the plurality of images having different exposure amounts stored in the memory and the motion vector. An image sensor that photoelectrically converts a subject image and outputs it as an electrical signal;
A memory for storing image data;
An image capturing apparatus including an image composition device that performs composition processing of a plurality of images,
The memory stores a plurality of images with different exposure amounts previously captured by the image sensor,
The image synthesizer inputs a plurality of images with different exposure amounts stored in the memory, determines the type of blur, and based on the determination result, either the camera shake correction process or the subject shake correction process An imaging apparatus characterized by switching the correction processing and outputting a composite image. The image composition device includes:
Detecting a motion vector using a plurality of images with different exposure amounts read from the memory, and generating a binary image of the plurality of images with different exposure amounts and storing them in the memory;
A blur determination unit that determines whether or not blur has occurred using a binarized image of a plurality of images having different exposure amounts stored in the motion vector and the memory; and
4. The imaging according to claim 3, further comprising switching means for performing a correction process of either a camera shake correction process or a subject shake correction process based on a result of the shake determination unit to switch and output a composite image. apparatus.   The imaging according to any one of claims 1 to 4, wherein the image synthesizing apparatus further includes a low-pass filter, and is configured to apply a low-pass filter to the image signal subjected to the synthesizing process. apparatus.   The image synthesizing apparatus further includes a motion vector detection region specifying unit that specifies a region for detecting a motion vector, and the switching unit is configured to apply a low-pass filter to an image signal corresponding to the motion vector detection region. The imaging apparatus according to claim 2 or 4, wherein   The image synthesizing apparatus includes a blur type determination unit that determines and outputs the type of blur, and the switching unit includes the one image when the result output from the blur type determination unit indicates subject blur. The image pickup apparatus according to claim 6, wherein a low-pass filter is applied to an image signal corresponding to the motion vector detection region for an image obtained by combining the second image and the other image. The image composition device includes: an image obtained by shifting the coordinates of one of the plurality of images having different exposure amounts based on the motion vector in a horizontal and vertical direction; and a plurality of images having the different exposure amounts. From the other image of
The switching means is arranged in the inconsistent region with respect to a camera shake correction image obtained by synthesizing the one image with the other image and a coordinate shifted in the horizontal and vertical directions based on the motion vector. The imaging apparatus according to claim 4, wherein a low-pass filter is applied to the corresponding pixel image signal. The image composition device includes: an image obtained by shifting the coordinates of a binary image related to one of the plurality of binary images based on the motion vector in the horizontal and vertical directions; and the plurality of binary images. From the other image, find the mismatch area between the images,
The switching means is arranged in the inconsistent region with respect to a camera shake correction image obtained by synthesizing the one image with the other image and a coordinate shifted in the horizontal and vertical directions based on the motion vector. The imaging apparatus according to claim 4, wherein the imaging apparatus is configured to apply a low-pass filter to a corresponding image signal.   When the blur determination signal indicates a camera shake, the switching means is a camera shake obtained by combining the image obtained by shifting the coordinates of the one image in the horizontal and vertical directions with the other image based on the motion vector. The imaging apparatus according to claim 9, wherein a low-pass filter is applied to a pixel image signal corresponding to the mismatch area with respect to the corrected image.

Images