JP4015646B2 - Information generating apparatus, photographing apparatus, and photographing method - Google Patents

Description

  The present invention relates to an information generation apparatus that generates information indicating a still image, an imaging apparatus that captures a still image during a predetermined exposure period, and an imaging method.

  As a method for correcting camera shake of a photographed image, there are an optical camera shake correction method and an electronic camera shake correction method (see, for example, Patent Document 1).

  A technique using an optical camera shake correction method is known as a camera shake correction technique for a shot still image.

A technique using an electronic camera shake correction method is known as a camera shake correction technique for a captured moving image. Since this technique does not require an optical drive unit, the apparatus can be miniaturized.
JP 60-143330 A

  However, the technique using the optical camera shake correction method is effective for photographing a still image without deterioration in image quality, but requires an optical drive unit, and thus there is a limit to downsizing of the apparatus.

  Since the technique using the electronic camera shake correction method does not require an optical drive unit, it is possible to reduce the size of the apparatus. However, it has been considered that the technique is not effective for camera shake correction of a still image. This is because this technique only performs correction between frames, and it is impossible in principle to correct camera shake of a single frame.

  The present invention solves the above-described problems, and an object thereof is to provide an information generation apparatus, an imaging apparatus, and an imaging method using an electronic camera shake correction method that can correct camera shake of a captured still image.

An image display device according to the present invention is a photographing device that captures a still image during a predetermined exposure period, and includes a swing amount detecting unit that detects a swing amount between a plurality of frames representing the captured still image, and the detection A plurality of frame information indicating the plurality of frames according to the determined amount of swing, and a resolution of the plurality of frames according to at least one of the detected amount of swing and zoom ratio. A resolution changing means for changing, a storage means for storing the plurality of frame information with the fluctuation correction and the resolution changed , and a still image indicating the still image based on the plurality of frame information stored in the storage means Information generating means for generating information, thereby achieving the above object.

  The information generation unit may generate the still image information by collectively calculating a plurality of frame information stored in the storage unit.

  The information generation unit may generate the still image information by sequentially calculating each of a plurality of frame information stored in the storage unit.

The resolution changing unit further includes a frame rate changing unit that changes the resolution of the plurality of frames according to the detected amount of fluctuation, and changes a frame rate according to the amount of fluctuation. The number of a plurality of frames representing still images taken per unit time is shown.

  The swing amount detection means detects the swing amount based on information generated by adding information indicating a plurality of pixels included in the imaging surface of the image sensor, and the swing correction means determines the swing amount. Accordingly, the plurality of frame information may be corrected by cutting out a part of the plurality of frame information.

  The apparatus further comprises determination means for determining whether or not the predetermined exposure time is greater than a predetermined value, and when the predetermined exposure time is determined to be greater than a predetermined value, the swing amount detection means You may detect the said rocking | swiveling amount based on the information produced | generated by adding the information which shows the some pixel contained in the imaging surface of an element.

The photographing method of the present invention is a photographing method for photographing a still image during a predetermined exposure period, the step of detecting a swing amount between a plurality of frames representing the shot still image, and the detected swing amount. According to the step, correcting a plurality of frame information indicating the plurality of frames , changing a resolution of the plurality of frames according to at least one of the detected swing amount and zoom ratio, and the swing A step of storing a plurality of pieces of frame information subjected to dynamic correction and the resolution change , and a step of generating still image information indicating the still image based on the plurality of pieces of frame information stored in the storage unit, This achieves the above object.

  According to the information generation apparatus, the imaging apparatus, and the imaging method of the present invention, it is possible to generate information indicating a still image by correcting the information indicating a plurality of frames, and thus, one still image that is corrected for camera shake. Can be obtained.

  The present invention captures a plurality of images during a predetermined exposure period and corrects camera shake of each of the plurality of images. By obtaining a single still image by performing arithmetic processing on a plurality of corrected images, it is possible to obtain a still image with a good S / N corrected for camera shake.

  The present invention increases the frame rate by increasing the clock speed or decreasing the resolution when the frame rate is low during camera shake correction. Accordingly, there is little deterioration in image quality due to electronic camera shake correction.

  According to the photographing apparatus of the present invention, the photographer can check the degree of camera shake. By checking the degree of camera shake, the photographer can change the camera fixing method, and as a result, a still image with less camera shake than usual can be obtained by human operation.

  Embodiments of the present invention will be described below with reference to the drawings.

1. Imaging Device FIG. 1 shows an imaging device 100 according to an embodiment of the present invention.

  The imaging apparatus 100 captures a still image during a predetermined exposure period. The photographing apparatus 100 includes a lens unit 2 for entering light from the outside of the photographing apparatus 100, an automatic focusing unit 4 that automatically adjusts the light 3, a zoom unit 6 that sets a zoom ratio of the lens unit 2, and imaging. Part 5. An optical image is formed on the imaging unit 5. The imaging unit 5 outputs output data indicating the formed optical image. The imaging unit 5 is, for example, a CCD or MOS type imaging device. The light 3 incident from the lens unit 2 is focus-adjusted by the automatic focusing unit 4. A zoom ratio of the lens unit 2 is set by the zoom unit 6, and an optical image 7 is formed on the imaging unit 5.

  The photographing apparatus 100 further includes a display switching unit 10, a display circuit 11, a display unit 12, a recording unit 13, and a recording medium 14. When a still image is captured without correcting the fluctuation of the still image representing the still image, the output data output from the imaging unit 5 is sent directly to the display switching unit 10. When displaying a still image, the output data output from the imaging unit 5 is displayed on the display unit 12 by the display circuit 11. When recording a still image, the output data output from the imaging unit 5 is recorded on the recording medium 14 by the recording unit 13.

  The photographing apparatus 100 detects the shutter button 25, the sub-image memory 8 for storing data, and the rocking detection means 15 for detecting the rocking amount between a plurality of images (frames) representing a captured still image. A swing correction unit 9 that corrects a plurality of pieces of image information indicating a plurality of images (frames) according to a swing amount, a swing correction control unit 21, and trimming that is controlled to remove the vertical swing. A unit 22, a resolution changing unit 24 that changes the resolution of the image indicated by the screen data, a thinning control unit 25a, a pixel transfer unit 23, and a frame rate changing unit 40 that changes the frame rate in accordance with the swing amount. In addition. The swing detection unit 15 includes a calculation unit 18 that calculates data input to the swing detection unit 15, a first memory 16, and a second memory 17. The swing correction control unit 21 controls each component in order to correct the output data output from the imaging unit 5 in accordance with the detected swing amount.

  When a still image representing a still image is corrected to shoot a still image (when the shake correction setting is ON), the output data output from the imaging unit 5 before the shutter button 25 is pressed. Is temporarily stored in the sub-image memory 8. The output data output from the imaging unit 5 is input to the swing detection means 15. The calculation unit 18 calculates a plurality of input output data (for example, the (n-1) th screen data (that is, data of the previous screen) and the nth image data (that is, data of the current screen)). Find rocking information. The swing information is a motion vector 19. Even when the photographing apparatus 100 includes the vertical vibration gyro and the horizontal vibration gyro, the vertical swing amount and the horizontal swing amount can be detected. In this case, the swing information is a vertical swing amount and a horizontal swing amount.

  The swing correction control unit 21 controls the trimming unit 22 and the pixel transfer unit 23 to remove the vertical swing. When the swing correction control unit 21 controls the swing correction unit 9, the horizontal swing is removed. Accordingly, it is possible to correct the vertical fluctuation of the image data and the horizontal fluctuation of the image data, and it is possible to obtain image data that has been corrected for fluctuation.

  The image data subjected to the swing correction is sequentially output to the display unit 12 via the display switching unit 10. The user can visually recognize the continuous image corrected for shaking from the display unit 12 at a predetermined frame rate.

  The display unit 12 can display an image of a part of the entire image indicated by the image data that has been subjected to the rocking correction. Therefore, the framing of the subject can be accurately performed. In this case, the resolution is set to a resolution lower than that at the time of still image shooting by the resolution changing unit 24 and the thinning control unit 25a. Therefore, the frame rate can be increased, and the number of displayed images per second can be increased. As a result, the user can visually recognize the subject image more smoothly.

  The recording unit 13 may record a plurality of images with a high frame rate on the recording medium 14 as moving images.

  For example, the resolution changing unit 24 changes the resolution of the plurality of frames according to at least one of the brightness, the swing amount, and the zoom ratio.

  The photographing apparatus 100 further includes a clock control unit 27, a transfer clock supply unit 32, a processing clock supply unit 28, and a CPU 26. When the user presses the shutter button 25 halfway, the CPU 26 gives a command to the clock control means 27 so that the processing clock supply unit 28 starts the clock operation of the arithmetic unit 29 or the like or increases the clock speed.

  The photographing apparatus 100 includes a main image memory 30 that stores a plurality of pieces of image information that has been subjected to rocking correction, and a calculation unit 29 that generates still image information indicating a still image based on the plurality of pieces of image information stored in the storage unit. And a sub-operation unit 29a. The details of the function of the calculation unit 29, the function of the main image memory 30, and the function of the sub calculation unit 29a will be described later.

  The photographing apparatus 100 further includes a masking unit 20 and a bright part extraction unit 39. Details of the function of the masking unit 20 and the function of the bright part extraction unit 39 will be described later.

  The photographing apparatus 100 further includes a vibrator 36 and a speaker 37. Details of the function of the vibrator 36 and the function of the speaker 37 will be described later.

  FIG. 2 shows the configuration of the swing detection means 15. The swing detection means 15 includes a calculation unit 18, a first memory 16, and a second memory 17.

The first memory 16 stores image data indicating an image D _n−1 taken at time t = t _n−1 . The second memory 17 stores image data indicating an image D _n photographed at time t = t _n . Based on the (image data showing the image data and the image D _n indicating the image D _n-1) data indicating two images, the swing amount between the image D _n-1 and the image D _n (e.g., motion vector (X ₁ , y ₁ )) is detected, and data indicating the swing amount is output.

2. Operation of camera shake correction FIG. 3 shows an operation of camera shake correction according to the embodiment of the present invention.

Image D _{n = 1} shows an image taken at time n = 1, image D _{n = 2} shows an image taken at time n = 2, and image D _{n = 3} shows an image taken at time n = 3. A photographed image is shown, and an image Dn _{= 4} is an image photographed at time n = 4.

Image data indicating the image D _{n = 1} is stored in the main image memory unit 30.

Image data indicating the image D _{n = 2} is corrected according to the swing amount M ₂ (M ₂ = (x ₀ + x ₁ , y ₀ + y ₁ )). The image data corrected by the calculation unit 29 is added to the image data indicating the image D _{n = 1} stored in the main image memory unit 30. Data indicating the addition result is stored in the main image memory unit 30. Since the corrected image data is added to the image data indicating the image D _{n = 1} , the stationary portions of the subject are accurately overlapped, and the SN ratio of the image is improved.

The image data indicating the image D _{n = 3} is corrected according to the swing amount M ₃ (M ₃ = (x ₀ + x ₁ + x ₂ , y ₀ + y ₁ + y ₂ )). The corrected image data is added to the data indicating the addition result stored in the main image memory unit 30 by the calculation unit 29. Data indicating the addition result is stored in the main image memory unit 30.

The image data indicating the image D _{n = 4} is corrected according to the swing amount M ₄ (M ₄ = (x ₀ + x ₁ + x ₂ + x ₃ , y ₀ + y ₁ + y ₂ + y ₃ )). The corrected image data is added to the data indicating the addition result stored in the main image memory unit 30 by the calculation unit 29. Data indicating the addition result is stored in the main image memory unit 30.

  As described above, by repeating the addition of data, the main image memory unit 30 stores the image data indicating the four substantially the same added images, and generates data indicating one still image. Therefore, when the shutter is opened 100% during the exposure period, an image having almost the same brightness as that obtained when the shutter is continuously opened during the period from time n = 1 to time n = 4 is obtained. The camera shake is corrected for each screen.

  Note that the camera shake can be corrected electronically without degrading the SN by appropriately setting, for example, the shutter opening time (exposure time) and the frame rate in accordance with the degree of camera shake (swing amount) and the zoom ratio. it can. Note that if the shutter opening time when each image is captured is shortened, image deterioration due to camera shake correction is reduced, but the amount of light is reduced. In this case, the number of frames to be shot is increased.

  In the embodiment of the present invention, the optimum frame rate can be obtained by increasing the frame rate by increasing the transfer clock or increasing the frame rate by decreasing the resolution. Therefore, there is an effect that the application range is wide.

  The calculation performed by the calculation unit 29 is not limited to addition. As long as data indicating one still image can be generated based on a plurality of image data (frame information), the calculation may be integration, for example.

3. Masking Operation FIG. 4 shows a masking operation according to the embodiment of the present invention.

Image D _{n = 1} shows an image taken at time n = 1, image D _{n = 2} shows an image taken at time n = 2, and image D _{n = 3} shows an image taken at time n = 3. A photographed image is shown, and an image Dn _{= 4} is an image photographed at time n = 4.

  The flash fires at time n = 1. Therefore, for example, the person shown in the image 35e becomes brighter than the background (for example, a night view). Here, the person represented in the image 35e is defined as the bright part 38a. The bright part extraction unit 39 extracts data indicating the bright part 38a from the data indicating the image 35e, and generates masking data 31 based on the extracted data indicating the bright part a.

At time n = 2, the masking data 31 is cut out from the image data indicating the image D _{n = 2} . Data indicating the corrected image 33a in which the image of the bright part 38b is removed and the camera shake is corrected is generated. Data indicating the corrected image 33a is added to data indicating the image 35e, and data indicating the integrated image 35a is generated.

At time n = 3, the masking data 31 is cut out from the image data indicating the image D _{n = 3} . Data indicating the corrected image 33b in which the image of the bright portion 38c is removed and the camera shake is corrected is generated. Data indicating the corrected image 33b is added to data indicating the integrated image 35a, and data indicating the integrated image 35b is generated.

At time n = 4, the masking data 31 is cut out from the image data indicating the image D _{n = 4} . Data indicating the corrected image 33c in which the image of the bright portion 38d is removed and the camera shake is corrected is generated. Data indicating the corrected image 33c is added to data indicating the integrated image 35b, and data indicating the integrated image 35c is generated.

  Data indicating the integrated image 35c is resized, and data indicating the integrated image 35d is generated.

  As described above, the bright part of a person or the like that has become bright due to the light emission of the strobe is captured at time n = 1, and the areas other than the bright part of the captured image are captured at time n = 2, 3, and 4. Import images. When shooting with a slow shutter with a slow shutter on a person in night view shooting, for example, the image of the person's face that was exposed during the slow shutter period overlaps the image of the person's face during the flash firing (double exposure). ) The image is blurred. However, according to the masking operation of the embodiment of the present invention, the face of a person who is a bright part or the like is not double-exposed due to camera shake correction. As a result, the bright part can be clearly photographed.

4). Change of Frame Rate FIG. 5 shows the relationship between the number of pixels and the frame frequency.

  If the resolution in the imaging unit 5 (see FIG. 1) is lowered by the resolution changing unit 24 (see FIG. 1), the frame rate (fps) can be raised. The frame rate (fps) can be increased even when the transfer clock speed is increased by the clock control means 27, the processing clock supply section 28, and the transfer clock supply section 32.

  In the embodiment of the present invention, when an image is taken out for the purpose of camera shake correction, the transfer clock is raised or the resolution is lowered to substantially increase the frame rate, and an afterimage (image degradation) that is unique to electronic camera shake correction. ) Can be eliminated. In this case, since the normal imaging unit has 2 million pixels and the frame rate is about 7.5 fps, the influence of the afterimage remains. If the frame rate is not 20 fps or more, it is difficult to eliminate afterimages specific to electronic image stabilization.

  FIG. 6 shows a configuration of the imaging unit 5 including a pixel region divided into four.

  The imaging unit 5 includes a pixel region 40. The pixel area 40 is divided into four pixel areas (pixel area 40a, pixel area 40b, pixel area 40c, and pixel area 40d). The imaging unit 5 is divided into four horizontal transfer units (horizontal transfer unit 41a, horizontal transfer unit 41b, horizontal transfer unit 41c and horizontal transfer unit 41d) and four vertical transfer units (vertical). A direction transfer unit 42a, a vertical transfer unit 42b, a vertical transfer unit 42c, and a vertical transfer unit 42d). Accordingly, the sweep time for all pixels is ¼, the frame rate is quadrupled, and the frame rate is 30 fps under the condition of a 2.1 million pixel CCD-type imaging device and a clock speed = 18 MHz. As a result, image degradation caused by camera shake correction can be made inconspicuous. The imaging unit 5 may be divided into left and right parts.

  FIG. 7 is an operation diagram when the frame rate is increased in the embodiment of the present invention.

When the exposure time is 1/4 second, a frame is obtained every 1/8 second. At this time, the shake amount in the x direction = ∫ ₀ (x _i + x _{i + 1} ) dt≈ ((x ₁ + x ₂ ) + (x ₃ + x ₄ )) / 2.

When the exposure time is 1/2 second, a frame is obtained every 1/16 second by increasing the frame rate. At this time, the amount of camera shake in the x direction≈ (x ₁ + x ₂ + x ₃ + x ₄ + x ₅ + x ₆ + x ₇ + x ₈ ) / 8≈ ((x ₁ + x ₂ + x ₃ + x ₄ ) / 2 + (x ₅ + x ₆ + x ₇ + _x 8) / 2) / 4.

  If the exposure time is long, the frame rate is increased to obtain many frames. As a result, it is possible to detect the amount of camera shake in detail, and to reduce image deterioration due to camera shake correction.

  As described above, according to the photographing apparatus of the present invention, since information indicating a plurality of frames can be corrected by swinging and information indicating a still image can be generated, one still image with camera shake correction can be obtained. Can do.

  As described above, in (1. photographing apparatus) to (4. change of frame rate), an example of the embodiment of the present invention has been described with reference to FIGS.

  For example, in the embodiment shown in FIGS. 1 to 7, the swing amount detecting means 15 corresponds to “a swing amount detecting means for detecting a swing amount between a plurality of frames representing a captured still image”. The correction unit 9 corresponds to “a swing correction unit that corrects a plurality of pieces of frame information indicating a plurality of frames according to the detected swing amount”, and the main image memory 30 has “a plurality of pieces of frame information whose swing is corrected”. Corresponding to the “storage means for storing”, and the calculation unit 29 corresponds to “information generation means for generating still image information indicating a still image based on a plurality of pieces of frame information stored in the storage means”.

  However, the photographing apparatus of the present invention is not limited to the embodiment shown in FIGS. Each component included in the photographing apparatus indicates a plurality of frames according to the above-described “swing amount detecting means for detecting a swing amount between a plurality of frames representing a captured still image” and “a swing amount detecting unit”. "Standing correction means for correcting a plurality of frame information", "Storage means for storing a plurality of pieces of frame information corrected for shaking" and "Still image indicating a still image based on a plurality of frame information stored in the storage means" As long as each function of the “information generating means for generating image information” is provided, it may have an arbitrary configuration.

5. Shooting method 1
FIG. 8 shows the procedure (step 50a to step 50f) of the photographing process according to the embodiment of the present invention.

  FIG. 9 shows the procedure (step 51a to step 51y) of the photographing process according to the embodiment of the present invention.

  FIG. 10 shows the procedure (step 52a to step 52t) of the photographing process according to the embodiment of the present invention.

  Hereinafter, with reference to FIG. 1 and FIGS. 8 to 10, the procedure of the photographing process according to the embodiment of the present invention will be described step by step.

  Steps 50a to 50f will be described with reference to FIG.

  Step 50a: An operator prepares to take a still image.

  Step 50b: When the operator half-presses the shutter button 25, the CPU 26 gives a command to the clock control means 27 so that the processing clock supply unit 28 starts the clock operation of the arithmetic unit 29 or the like or improves the clock speed. . When the processing clock supply unit 28 starts operating the clock of the arithmetic unit 29 or the like or increases the clock speed, the process proceeds to step 50c.

  Step 50c: The imaging unit 5 extracts an image smaller than the set resolution or a thinned image. Based on the data indicating the (n−1) -th image and the data indicating the n-th image, difference information on the position of a specific point or a specific region between the (n−1) -th image and the n-th image is generated, and Dynamic information (swing amount) is obtained.

  Step 50d: It is determined whether or not the swing information (swing amount) is larger than a predetermined value under the condition that the brightness of the shooting location is dark and the set resolution is a certain value or more. If it is larger than the predetermined value (Yes), the process proceeds to step 50e. If it is equal to or smaller than the predetermined value (No), the process proceeds to step 50f.

  Step 50e: A warning “shake hand shake” is displayed on the display unit 12 according to the value of the swing information.

  Step 50f: The operator determines whether or not to press the shutter button 25. If the button is pressed (Yes), the process proceeds to step 51a (see FIG. 7). If it is not pressed (No), the process of step 50f is repeated.

  Hereinafter, step 51a to step 51y will be described with reference to FIG.

Step 51a: Shutter speed (exposure time) whether longer than t ₁ is determined. For example, CPU 26 is, the shutter speed (exposure time) is equal to or longer or not than _{t 1.}

If the zoom ratio of the zoom unit 6 is equal to or less than a certain value and the shutter speed (exposure time) is equal to t ₁ or shorter than t ₁ (No), the process proceeds to step 51b. If the shutter speed (exposure time) is longer than _{t 1} (Yes), then the process proceeds to step 51d.

  Step 51b: Shooting without shaking correction (without camera shake correction).

  Step 51c: Shooting is completed and the process ends.

  Step 51d: The means shake correction priority switch is turned ON.

Step 51e: shutter speed (exposure time) whether longer than t ₂ is determined.

If the shutter speed (exposure time) is shorter than the same or _{t 2} and _{t 2} in the (No), then the process proceeds to step 51f. If the shutter speed (exposure time) is longer than _{t 2} (Yes), then the process proceeds to step 51h.

  Step 51f: It is determined whether or not the camera shake is severe and whether or not the zoom ratio is larger than a certain value.

  If the camera shake is not severe and the zoom ratio is greater than a certain value (No), the process proceeds to step 51g. If camera shake is severe and the zoom ratio is equal to or less than a certain value (Yes), the process proceeds to step 51i.

Step 51 g: proceed to image stabilization routine while setting the resolution N ₀ that is set resolution in advance (step 51r).

Step 51h: set resolution _{N 0} is whether higher than the predetermined resolution _{N 1} is determined.

If the set resolution N ₀ is the predetermined resolution N ₁ or lower than the predetermined resolution N ₁ (No), the process proceeds to step 51r. If the set resolution N ₀ is higher than the predetermined resolution N ₁ (Yes), the process proceeds to step 51i.

  Step 51i: The clock control means 27 speeds up the transfer clock of the pixel transfer unit 23. Thus, the frame rate is increased.

  Step 51j: It is determined whether or not the camera shake is intense.

  If the camera shake is not intense and the zoom ratio is less than or equal to the predetermined value, that is, the camera shake is very small (No), the process proceeds to step 51k. When camera shake is intense or when the zoom ratio or the like is larger than a predetermined value, that is, when there is a certain amount of camera shake (Yes), the process proceeds to step 51m.

Step 51k: Leave a predetermined resolution _{N 1,} the process proceeds to step 51r.

Step 51m: set resolution whether higher than a predetermined resolution N _2, or the frame rate is whether lower than a predetermined value fn is determined.

If the set resolution is lower than the predetermined resolution N ₂ same or predetermined resolution N _2,, or, if the frame rate is higher than the same or a predetermined value fn, the predetermined value fn (No), the process proceeds to step 51r.

If the set resolution is higher than the predetermined resolution N _2, or when the frame rate is lower than the predetermined value fn (Yes), the process proceeds to step 51n.

Step 51n: In order to set the resolution to the resolution N ₂ lower than the predetermined resolution N ₁ , the process proceeds to Step 51p.

Step 51p: The resolution changing unit 24 and the thinning control unit 25a thin out the pixel output from the imaging unit 5 or add information indicating a plurality of pixels in the in-plane direction to generate information indicating one pixel. number (i.e., resolution) lowers (setting resolution _{N 2).}

Step 51 q: resolution a result of setting the lower resolution N ₂ than the predetermined resolution N _1, go up the maximum speed of the value of the frame rate. Increase the frame rate.

  Step 51r: In order to capture a plurality of frames (images) into the photographing apparatus 100, it is determined whether or not to start inputting a photographed image for a camera shake correction routine. When the input of the captured image is started (Yes), the process proceeds to step 51y.

Step 51y: Based on the exposure time (that is, the shutter time), the aperture value, and the frame rate, the total number n _last required for the divided exposure is calculated. When camera shake is severe, shorten the shutter time for each still image.

  Step 51s: n = 0 is set.

  Step 51t: Increment n by 1 (n = n + 1).

  Step 51u: The nth image is taken, and the nth still image is taken into the sub memory 8 from the imaging unit 5 (the nth still image data is obtained).

  Step 51v: It is determined whether or not the still image data is the first still image data.

  If the still image data is the first still image data (Yes), the process proceeds to step 51w. If the still image data is not the first still image data (No), the process proceeds to step 52a (see FIG. 10).

Step 51w: cutting out a portion of the image of the imaging unit 5 to obtain the image data _{I 1.}

Step 51x: storing the image data _{I 1} in the main image memory 30.

  Hereinafter, step 52a to step 52t will be described with reference to FIG.

  Step 52a: The movement detection means 15 calculates the movement of a specific point between the first image data and the second image data, and calculates the fluctuation amount Mn (see FIG. 2).

Image data indicating the _first image D ₁ taken at time t = t ₁ is stored in the first memory 16 included in the swing detection unit 15, and the first memory 16 included in the swing detection unit 15 includes 2 memory 17, if the image data indicating the time t = t ₂ 2 sheet image D ₂ taken in is stored, the shaking motion detecting section 15, data indicating two images (image Based on the image data indicating D ₁ and the image data indicating image D ₂ ), a fluctuation amount M ₁ (for example, a motion vector (x1, y1)) between the image D ₁ and the image D ₂ is detected. Outputs data indicating the amount of movement.

  Step 52b: It is determined whether or not the integral value of the swing amount Mn is equal to or greater than a certain value.

  If the integral value of the swing amount Mn is greater than or equal to a certain value (Yes), it is determined that the captured image has protruded from the specific area, and the process proceeds to step 52c. If the integral value of the swing amount Mn is smaller than a certain value (No), the process proceeds to step 52s.

  Step 52c: 1 is added to the error register. Without storing the nth image in the main image memory 30, the process proceeds to step 52h.

  Step 52s: It is determined whether or not the integral value of the swing amount Mn is equal to or greater than another constant value. If the integral value of the swing amount Mn is equal to or greater than another constant value, 1 is added to the second error register. .

Step 52 d: among the image data output from the image pickup unit 5 stores the image data I _n which is cut in the longitudinal direction in accordance with the shaking motion amount Mn in the sub-image memory 8.

  Step 52e: It is determined whether or not the strobe is turned on. If the strobe is to be turned on (Yes), the process proceeds to step 52f. If the strobe is not turned on (No), the process proceeds to step 52g.

Step 52f: the masking section 20, in advance masked image data _{I n} (see FIG. 4 and (3. Operation masking)).

Step 52 g: from shaking motion correcting section 9 and the oscillation of the oscillating and vertical direction of the transverse direction to obtain image data I _n corrected.

For example, sending the image data I _n the operation unit 29, the image data and calculates the image data I _n the main image memory 30 (e.g., addition, integration) and, again, saves the result to the main image memory 30.

Step 52h: The oscillation correction control unit 21 determines whether or not n = n _last (that is, the last value).

If n = n _last (Yes), the process proceeds to step 52i. If n = n _last is not satisfied (No), the process proceeds to step 51t (see FIG. 9) in order to capture another piece of image data.

  Step 52i: The clock control means 27 lowers the transfer clock of the imaging unit 5. Alternatively, the transfer clock of the imaging unit 5 is stopped for power saving.

  Step 52j: It is determined whether or not the value of the second error register is a certain value or more.

  If the value of the second error register is smaller than the predetermined value (No), the process proceeds to step 52n. If the value of the second error register is greater than or equal to a certain value (Yes), the process proceeds to step 52k.

  Step 52k: It is determined whether or not the missing portion can be eliminated by resizing the integral image (whether or not the range of the missing portion is within the range that can be eliminated).

  For example, a missing portion 34a is generated in the corrected image 33c (see FIG. 4). In this case, the lacking part 34b also occurs in the integrated image 35c. Therefore, it is necessary to resize the integral image 35c in order to eliminate the lacking part 34b. In this case, it is determined whether or not the missing part 34b can be eliminated by resizing the integral image 35c (whether or not the range of the missing part 34b is within a range that can be eliminated).

  If it can be eliminated (Yes), the process proceeds to step 52m. If it cannot be excluded (No), the process proceeds to step 52p.

  Step 52m: The integral image 35c is resized to eliminate the lacking portion 34b, and an integral image 35d without the lacking portion is obtained (see FIG. 4).

  Step 52n: Data indicating the integral image 35d is recorded on the recording medium 14.

  Step 52p: Since the missing part cannot be removed even if the camera shake correction is performed, the operator is notified of the camera shake correction failure. For example, the meaning of “camera shake correction error (out of range)” is displayed on the display unit 12 (see FIG. 1). In addition, an error warning sound is output from the speaker 37. Further, the vibrator 36 is vibrated.

  Step 52q: It is determined whether or not the main display setting is ON.

  If the main display setting is ON (Yes), the process proceeds to step 52r. If the main display setting is not ON (No), the process proceeds to step 52t.

  Step 52r: The integrated image corrected in camera shake stored in the main image memory 30 or the resized image is displayed on the display unit 12.

  Step 52t: The image subjected to camera shake correction is recorded on the recording medium 14. When another image subjected to camera shake correction is taken after a predetermined time has elapsed, the process returns to the first step 50a again (see FIG. 8).

  As described above, according to the photographing method of the present invention, information indicating a plurality of frames can be corrected by swinging and information indicating a still image can be generated, so that one still image corrected by camera shake can be obtained. Can do.

  The example of the embodiment of the present invention has been described above in (5. Imaging method 1) with reference to FIGS. 1 and 8 to 10.

  For example, in the embodiment shown in FIGS. 8 to 10, step 52 a corresponds to “a step of detecting a swing amount between a plurality of frames representing a captured still image”, and steps 52 b to 52 g are “detection”. Step 52g corresponds to “Step of correcting a plurality of pieces of frame information indicating a plurality of frames according to the amount of swinging performed”, and Step 52g corresponds to “Step of storing a plurality of pieces of frame information subjected to swing correction”. Alternatively, step 52m corresponds to “a step of generating still image information indicating a still image based on a plurality of pieces of frame information stored in the storage unit”.

  However, the photographing method of the present invention is not limited to the embodiment shown in FIGS. Each step included in the imaging method includes the above-described “step of detecting the swing amount between a plurality of frames representing a captured still image” and “a plurality of frames indicating a plurality of frames according to the detected swing amount. “Step of correcting frame information”, “Step of storing a plurality of pieces of frame information subjected to fluctuation correction” and “Step of generating still image information indicating a still image based on a plurality of pieces of frame information stored in the storage means” As long as each of the functions has a function, any procedure can be included.

  For example, as described with reference to FIGS. 8 to 10, the calculation unit 29 calculates still image information by sequentially calculating each of a plurality of image data (frame information) stored in the main image memory 30. Generate. Further, the calculation unit 29 may generate still image information by calculating a plurality of image data (frame information) stored in the main image memory 30 at once.

6). Shooting method 1 (Sequential calculation)
FIG. 11 shows a sequential calculation processing procedure according to the embodiment of the present invention. The sequential calculation processing procedure is a procedure for generating still image information by sequentially calculating each of a plurality of image data (frame information).

  Hereinafter, with reference to FIG. 1 and FIG. 11, the sequential calculation processing procedure after the completion of the imaging preparation (after step 51 s in FIG. 9) will be described step by step.

  Step 10a: Set n = 0.

  Step 10b: Increment n by 1 (n = n + 1).

  Step 10c: The nth image is captured and taken into the sub memory 8.

Step 10d: The n-th image is subjected to camera shake correction, and an image P _n subjected to camera shake correction is obtained.

Step 10e: The computing unit 29 multiplies the data indicating the image _Pn subjected to the camera shake correction by m (P _n × m).

Step 10f: The computing unit 29 adds data indicating the image P _n multiplied by m to the image data in the main image memory 30 (Σ ⁿ⁻¹ _{i = 1} (P _i × m) + (P _n × m) ).

  Step 10g: The addition result is stored in the main image memory 30.

Step 10h: It is determined whether n = n _last (that is, the last value).

If n = n _last (Yes), the process proceeds to step 10i. If n = n _last is not satisfied (No), the process proceeds to Step 10b in order to capture another image data.

Step 10i: The sub-operation unit 29a multiplies the image data multiplied by m and sequentially added by 1 / s to generate image data P _x indicating a still image (P _x = (Σ ⁿ _{i = 1} (P _i × m)) / s).

Step 10j: outputting the generated image data _{P x} to the recording unit 13.

  After outputting to the recording unit 13, the process ends.

  As described above, according to the sequential calculation processing procedure, still image information is generated by sequentially calculating each of a plurality of image data (frame information), so that the generation time of the still image information can be shortened.

Further, the calculation unit 29 adds data indicating the image P _n multiplied by m to the image data in the main image memory 30, and the sub calculation unit 29a adds the image data multiplied by m and sequentially added to 1 / s times. In order to generate image data P _x indicating a still image, one still image having a desired brightness can be obtained by arbitrarily setting the value of m and the value of s.

7). Shooting method 1 (batch calculation)
FIG. 12 shows a batch operation processing procedure according to the embodiment of the present invention. The batch calculation processing procedure is a procedure for generating still image information by calculating a plurality of image data (frame information) at once.

  Hereinafter, with reference to FIG. 1 and FIG. 12, the batch calculation processing procedure after the completion of the preparation for shooting (after step 51s in FIG. 9) will be described step by step.

  Step 20a: Set n = 0.

  Step 20b: Increment n by 1 (n = n + 1).

  Step 20c: The nth image is taken.

  Step 20 d: Data indicating the nth image is stored in the main image memory 30.

Step 20e: The calculation unit 29 determines whether n = n _last (that is, the last value).

If n = n _last (Yes), the process proceeds to Step 20f. If n = n _last is not satisfied (No), the process proceeds to step 20b in order to capture another image data.

Step 20f: Camera shake correction is performed on n images. Each of the n images subjected to the camera shake correction is subjected to pixel integration to generate image data P _x indicating one still image.

Step 20 g: outputting the generated image data _{P x} to the recording unit 13.

  After outputting to the recording unit 13, the process ends.

  Thus, according to the batch calculation processing procedure, since still image information is generated by batch calculation of a plurality of image data (frame information), the load on the calculation unit 29 can be reduced.

  Note that, as described above in (6. Shooting Method 1 (Sequential Calculation)), the sub-operation unit 29a can appropriately m-fold and 1 / s-fold the image data, and thus has a desired brightness. One still image can be obtained.

8). Shooting method 2
FIG. 13 shows a processing procedure for correcting camera shake by integrating a plurality of images (divided images) according to the brightness of the shooting location and the shutter speed (exposure time).

  Hereinafter, this processing procedure will be described step by step.

  Step 99a: The resolution, the number of pixels, and the number of divided images are set.

  Step 99b: It is determined whether or not the means shake correction priority switch is ON.

  If it is not ON (No: resolution priority mode), for example, the process proceeds to step 80c (see FIG. 19 described later). If it is ON (Yes: means shake correction priority mode), the process proceeds to step 99c.

  Step 99c: It is determined whether or not the brightness of the shooting location is smaller than a predetermined value determined according to the resolution.

  If it is smaller than the predetermined value (Yes), the process proceeds to step 99f. If it is equal to or greater than the predetermined value (No), the process proceeds to step 99d.

  Step 99d: It is determined whether or not the shutter opening time (exposure time) S is longer than a predetermined value determined according to the resolution.

  If it is larger than the predetermined value (Yes), the process proceeds to step 99f. If it is equal to or smaller than the predetermined value (No), the process proceeds to step 99e.

  Step 99e: It is determined whether or not the camera shake amount is larger than a predetermined value.

  If it is larger than the predetermined value (Yes), the process proceeds to step 99f. When it is equal to or smaller than the predetermined value (No), normal photographing (photographing without pixel addition in the time direction) is performed (step 99m).

Step 99f: brightness of the shooting location, the shutter opening time according to at least one of (exposure time) and the frame rate, the camera shake is set the resolution (resolution limit) N ₁ inconspicuous.

  After the setting, the process proceeds to step 99g.

Step 99 g: resolution _{N 1} is equal to or greater than the initial resolution _{N 0.}

  If it is larger (Yes), normal shooting (shooting without pixel addition in the time direction) is performed (step 99m). If they are the same or smaller (No), the process proceeds to step 99h.

Step 99h: changing the resolution _{N 1} to the initial resolution _{N 0} is smaller than the resolution _{N 2.}

Step 99i: performing at least one of a horizontal pixel addition (horizontal addition process) and vertical pixel addition (vertical addition process) to set the resolution to N _2. Details of the horizontal addition process and the vertical addition process will be described later.

  Step 99j: Increase the frame rate.

  Step 99k: Multiple shooting is performed (time direction, pixel addition mode). Next, the process proceeds to, for example, step 51y (see FIG. 9).

9. Changing Resolution FIG. 14 shows a procedure for changing the resolution by in-plane pixel addition and time-axis pixel addition.

  Hereinafter, a procedure for changing the resolution by in-plane pixel addition and time-axis pixel addition will be described step by step with reference to FIG.

  Step 70a: Data indicating nine pixels (pixel 60a to pixel 60i) in the image sensor are added in the in-plane direction to generate data indicating one pixel 62.

  Step 70b: Set more virtual addresses than actual addresses (set virtual addresses by expanding the amount of actual addresses). A virtual cutout unit 65 is set according to the camera shake correction information (swing information).

  Step 70c: The data indicating the image 61 is shifted on the virtual address according to the camera shake correction information. In this case, data indicating a new pixel 66 is generated based on data indicating the original pixel 62 and data indicating surrounding pixels.

  FIG. 15 shows the principle of correcting camera shake by setting a larger number of pixels than the actual number of pixels according to the embodiment of the present invention. Since the camera shake correction amount has a resolution that is 1/10 of the pixel, a virtual pixel 67 obtained by dividing the pixel 62 into ten parts is generated and the virtual pixel 67 is shifted in order to perform a precise correction.

  After shifting the virtual pixel 67 in the virtual space, the process proceeds to Step 70d.

  Step 70d: The image is cut out.

  Step 70e: Data indicating the cut-out image 64 is obtained. Data indicating the protruding portion 68 is discarded.

  Step 70f: Data indicating the cut-out image 64 is recorded in the main image memory 30. The camera shake correction amount at this time is recorded in the main image memory 30.

  Step 70g: When data indicating a new image 61a is input, the same processing as in steps 70a to 70d is performed.

  Step 70h: Based on the camera shake correction amount, data indicating the cut-out image 64a is obtained.

  Step 70i: Data indicating the synthesized image 71 is obtained by adding (or integrating) the data indicating the pixels of the clipped image 64 and the data indicating the pixels of the clipped image 64a in the time axis direction.

  Step 70j: Data indicating the composite image 71 is recorded in the main image memory 30.

  Step 70k: When data indicating a new image 61b is input, the same processing as in steps 70a to 70e is performed. Data indicating the cut-out image 64b is obtained.

  Step 70m: Data indicating the composite image 71a is obtained by adding data indicating the pixel of the composite image 71 and data indicating the pixel of the cutout image 64b in the time axis direction.

  Step 70n: The camera shake correction amount 72 is generated by calculating the first camera shake correction amount 69, the second camera shake correction amount 69a, and the third camera shake correction amount 69b. Based on the camera shake correction amount 72, an overlapping area 73 in which three images are overlapped and added is specified from the synthesized image 71a.

  Step 70p: Enlargement interpolation is performed by performing a zooming operation on the data indicating the overlapping area 73, and data indicating the enlarged image 74 is obtained. Details of enlargement interpolation and reduction interpolation will be described later.

  Data indicating the still image 74 corrected for camera shake is obtained, and the process ends.

  In the embodiment described with reference to FIG. 14, an example in which three images are integrated has been described. However, the number of images to be integrated is not limited to three. For example, as the exposure time becomes longer, more images are integrated. By integrating more images, it is possible to shoot in dark places.

  FIG. 16 shows an addition method in the in-plane direction. In-plane direction addition includes vertical direction addition and horizontal direction addition.

  FIG. 16A shows a vertical addition method. R (red) (m, n + 1) and R (m, n) are vertically added at the time of reading in the vertical direction to generate R (m, n + 1) + R (m, n).

  FIG. 16B shows a horizontal addition method. Pixels of the same color are added in the horizontal direction. For example, G (m, n + 1) + G (m, n) and G (m + 1, n + 1) + G (m + 1, n) are horizontally added, and G (m, n + 1) + G (m, n) + G (m + 1, n) n + 1) + G (m + 1, n).

  As described with reference to FIG. 16, data indicating one pixel can be generated from data indicating four pixels by addition in the in-plane direction.

  As will be described below, in the in-plane direction addition process, the image can be cut out more accurately by shifting the cutout position of the image.

  FIG. 17 is a diagram for explaining the shift of the image cutout position.

  The addition switching unit 102a and the addition switching unit 102b change the addition mode between the A mode 103 and the B mode 104 according to the correction signal or detection signal (swing information) output from the swing detection unit 15 (see FIG. 1). Switch between. In this way, in the in-plane direction addition process (see FIG. 16), the image can be cut out more accurately by shifting the cutout position of the image by one pixel.

  FIG. 18 shows the principle of reduction interpolation, the principle of enlargement interpolation, and the principle of camera shake correction with high resolution. FIG. 18A shows the principle of reduced interpolation. It is possible to obtain pixels (six pixels) subjected to reduction interpolation from the original pixels (eight pixels). FIG. 18B shows the principle of enlargement interpolation. Pixels (8 pixels) subjected to enlargement interpolation can be obtained from the original pixels (6 pixels). FIG. 18C shows the principle of high-resolution camera shake correction.

10. Removal of Image Failed in Camera Shake Detection FIG. 19 shows a procedure for removing an image failed in camera shake detection.

  Hereinafter, a procedure for removing an image for which camera shake detection has failed will be described step by step with reference to FIG.

  Step 80a: It is determined whether or not the shutter speed (exposure time) is longer than t '.

  If the shutter speed (exposure time) is longer than t ′ (Yes), the process proceeds to step 80b.

  Step 80b: It is determined whether the camera shake is intense.

  If the camera shake is not severe (No), the process proceeds to step 80c. If the camera shake is intense (Yes), the process proceeds to step 80d.

  Step 80c: Shooting is performed with the resolution set to a preset resolution.

  Step 80d: It is determined whether or not camera shake correction is performed with priority. If the setting for preferentially performing camera shake correction is ON (Yes), the process proceeds to step 80e. If the setting for preferentially performing camera shake correction is not ON (No), the process proceeds to step 80c.

  Step 80e: The camera shake correction mode is displayed.

Step 80f: It is determined whether or not the exposure time t is t ₁ <t <t ₂ .

If the exposure time t is t ₁ <t <t ₂ (Yes), the process proceeds to step 81d, the in-plane pixels are added (step 81d), and the exposure time t is set to t <t ₁ ( In step 81e), photographing is started (step 81f).

In the case the exposure time t is not _{t 1 <t <t 2 (} No), the process proceeds to step 80 _g, it is determined whether or not t 2 <t _{<t 3,} not t 2 <t _{<t 3} In the case (No), the process is stopped (step 81g). If t ₂ <t <t ₃ (Yes), the process proceeds to step 80h.

  Step 80h: In-plane pixel addition is set.

Step 80i: The exposure time t to t _{<t 2,} determine the number of images P to be taken for image stabilization.

  Step 80j: Shooting is started. Set R = 0.

  Step 80k: n = 0 is set.

  Step 80m: n = n + 1 is set.

  Step 80n: Pixel addition in the in-plane direction of the nth image is performed.

  Step 80p: Camera shake is detected.

  Step 80q: It is determined whether or not the camera shake has been successfully detected.

If detection of camera shake has failed (No), R = R + 1 is set (step 80r), it is determined whether R <R ₀ (whether R is smaller than the set value R ₀ ), and R < If it is not R ₀ (No), the processing is stopped (step 80t). If R <R ₀ (Yes), at least one of the detection point of the motion vector and the number of detections of the motion vector is changed (step 80v), and the processing is performed in step 80k in order to start correction work from the beginning. Proceed to

  If the camera shake is successfully detected (Yes), the process proceeds to step 80u.

  Step 80u: The corrected image is stored in the main image memory 30.

  Step 80w: It is determined whether n = P.

  If n = P is not satisfied (No), the process proceeds to step 80m. If n = P (Yes), it is determined that the processing of all images (all divided images) taken for camera shake correction has been completed, and the processing proceeds to step 80x.

  Step 80x: A plurality of corrected images in the main image memory 30 are added or integrated in the time axis direction.

  Step 80y: Data indicating one image is generated.

  Step 80z: Thinning processing or the like is performed on the generated data representing one image and displayed on the display unit 12.

  Step 81a: The operator determines whether to turn on the image storage switch.

  Step 81b: A compression process (JPEG or the like) is performed on the data indicating the image to reduce the capacity of the image data.

  Step 81c: Recording is performed on the recording medium 14 (for example, an IC card).

  As described with reference to FIG. 19, according to the embodiment of the present invention, it is possible to prevent the addition (integration) of corrected image data that has failed to detect camera shake. For example, even in the case of an image that is difficult to detect such that the detection that is added (integrated) in the time axis direction fails, an image that is corrected for camera shake can be obtained. Also, since integration in the time axis direction can be started from the image next to the image for which the detection of camera shake has failed, the time utilization efficiency is good.

11. Display of Camera Shake Amount FIG. 20 shows the configuration of the photographing apparatus 200 according to the embodiment of the present invention.

  The image capturing apparatus 200 can display the amount of camera shake as with the image capturing apparatus 100. The photographing apparatus 200 includes a camera shake amount calculation unit 92, a locus calculation unit 91, a display unit 95, a speaker 97, a vibrator 98, a CPU 99, a vibration gyro 101a, and a vibration gyro 101b.

  A camera shake amount calculation unit 92 (swing detection means 15: see FIG. 1) calculates a hand shake amount (swing amount) and outputs it to the display unit 95 via a display circuit. The trajectory calculation unit 91 calculates a trajectory of camera shake that could not be corrected even after camera shake correction, and outputs it to the display unit 95 via the display circuit.

  The CPU 99 determines whether or not the amount of camera shake is greater than a predetermined value. If the amount is greater than the predetermined value, the CPU 99 outputs the determination result to at least one of the display unit 95, the speaker 97, and the vibrator 98. Instruct.

  The display unit 95 displays the determination result in accordance with an instruction from the CPU 99. The speaker 97 generates a warning sound in accordance with an instruction from the CPU 99. Vibrator 98 vibrates in accordance with an instruction from CPU 99.

  FIG. 21 shows an example of the display unit 95 included in the photographing apparatus 200.

  On the display unit 95, the amount of camera shake is displayed by the indicators 93, 93a, 93b, and 93c. When the photographer visually observes this display, the photographer can confirm the camera shake amount and the camera shake direction. By checking the camera shake amount and the camera shake direction, the photographer changes the camera fixing method. As a result, a still image with less camera shake than usual can be obtained by human operation.

  FIG. 22 shows another example of the display unit 95 included in the photographing apparatus 200.

  On the display unit 95, the camera shake trajectory that cannot be corrected by the camera shake correction is displayed as a trajectory 94b and a trajectory 94d. The photographer can check how much the still image is shaken after photographing by viewing this display. Since the camera shake failure can be confirmed on the small display part of the camera, the photographer can check the camera shake correction failure. In the case of a panning mode (for example, panning or panoramic shooting), only hand shake in the vertical direction may be checked.

In the display unit 95, when the means shake amount (x, y) is larger than a predetermined value (x ₀ , y ₀ ) ((x> x ₀ , or y> y ₀ ) or (x> x ₀ , and A warning may be displayed for y> y ₀ )). The speaker may generate a warning sound. The predetermined values (x ₀ , y ₀ ) are set according to the zoom ratio, for example.

Further, in the display unit 95, when the means shake amount (x, y) is smaller than a predetermined value (x ₀ , y ₀ ) ((x <x ₀ , or y <y ₀ ) or (x <x ₀ , and y <y ₀ )) may be displayed (for example, “OK”). The speaker may generate sound. The predetermined values (x ₀ , y ₀ ) are set according to the zoom ratio, for example.

  FIG. 23 shows the display of the boundary indicator 97 during panning or panoramic photography.

  FIG. 23A shows a landscape for shooting in three frames.

  FIG. 23B shows a frame 98a. FIG. 23C shows the frame 98b. FIG. 23D shows a frame 98c. FIG. 23 (e) shows a frame 98d.

  When panoramic shooting of the landscape shown in FIG. 23 (a) is performed in the right direction, detection points 96a, 96b, and 96c (see FIG. 23 (b)) that are representative points for detecting a motion vector for camera shake correction. The detection point 96a moves on the frame and comes to the left end of the frame 98b (see FIG. 23C). At this time, the swing detection means 15 for detecting motion detects that the screen has shifted to the right by L1, and the boundary indicator 97a indicating the boundary of the right end of the frame in FIG. 23B is positioned at L1 from the right end of the frame. (Refer to FIG. 23C).

  Similarly, the swing detection means 15 detects that the screen has shifted to the right by L2, and displays the boundary indicator 97b at a position at L2 from the right end of the frame (see FIG. 23 (d)).

  Similarly, in FIG. 23E, the boundary indicator 97c comes to the left end of the screen. At this stage, the photographer can know that he has come to the next shooting position. If necessary, a notification sound can be generated by the speaker 37 (see FIG. 1) to notify the photographer. At this time, when the photographer presses the shutter button, almost complete panoramic shooting can be performed.

  In the foregoing, a method has been described in which a plurality of detection points are set on the screen, and the movement of the photographer is regarded as a frame motion from the detection point motion vector.

In this method, panning detection is performed by camera shake detection means for correcting camera shake. However, as shown in FIG. 20, in an imaging apparatus that detects camera shake using the vibration gyro 101a and the vibration gyro 101b, the panning rotation angle of the photographer is detected by the vibration gyro and is necessary for panning in the horizontal direction for one frame. It is also possible to obtain the correct rotation angle θ ₀ according to the zoom ratio of the zoom detection unit.

First, in the state of FIG. 23B, the boundary indicator 97 is displayed at the right end. In this state, the photographer takes the first panoramic shot. Next, when the photographer pans the camera to the right by the rotation angle θ ₀ (that is, rotates), the photographer knows that the camera has come to the photographing position of the second frame.

When the rotation angle θ ₀ is reached, a boundary indicator 97c is displayed at the left end (see FIG. 23 (e)). At this time, the photographer is notified by the speaker or display that the rotation angle θ ₀ has been reached. By having the photographer release the next shutter, panoramic photography can be performed accurately in the horizontal direction.

  By displaying the boundary indicators 99a and 99b in the vertical direction as well as the horizontal direction on the display unit 12, the photographer can easily take a panoramic image in which the vertical and horizontal directions match (FIG. 23B). reference). In this case, the panning direction is indicated by an arrow on the display screen, and the correct panning direction is displayed, so that the photographer can easily take a panoramic image simply by pointing the camera as indicated by the arrow. In general, since a photographer holds a camera and shoots, the shooting direction cannot be determined accurately. However, by using the camera shake correction function of the camera to automatically adjust the left, right, top and bottom of the screen, extremely accurate panoramic shooting can be performed.

  Even when the left / right / up / down of the screen is automatically adjusted using the camera shake correction function of the camera, an error occurs between the captured panorama screen and the ideal panorama screen. By additionally recording this error in attribute data (Exif etc.) indicating the attribute of the shooting state, it is possible to more accurately align left and right and up and down when combining a plurality of images and converting them to a single panorama image. Is possible. This is because the error can be corrected based on the error information.

  The above description can be similarly realized even when the camera shake detection method is an electronic detection method.

  FIG. 24 shows detection points for detecting camera shake in the frame. When there are many camera shake detection failures or when the frame rate is slow, the success rate of camera shake detection can be increased by changing the position of the detection points 96 or increasing the number of detection points 96 in the frame 98. it can.

  Thus, according to the photographing apparatus of the present invention, the photographer can check the degree of camera shake. By checking the degree of camera shake, the photographer can change the camera fixing method, and as a result, a still image with less camera shake than usual can be obtained by human operation.

  As described above, in (11. Display of camera shake amount), an example of the embodiment of the present invention has been described with reference to FIGS. 1 and 20 to 23.

  For example, in the embodiment shown in FIG. 1 and FIG. 20 to FIG. 23, the swing detection means 15 (the camera shake amount calculation unit 92) is “a swing that detects the swing amount between a plurality of frames representing a captured still image”. Corresponding to “movement amount detecting means”, the CPU 99 corresponds to “determination means for determining whether or not the swing amount is larger than a predetermined value”, and the display unit 95, the speaker 97 and the vibrator 98 are “output means for outputting a determination result”. ".

  However, the photographing apparatus of the present invention is not limited to the embodiment shown in FIG. 1 and FIGS. Each component included in the photographing apparatus determines whether or not the rocking amount is larger than a predetermined value as described above, “the rocking amount detection means for detecting the rocking amount between a plurality of frames representing a captured still image”. As long as each of the functions of “determination means” and “output means for outputting a determination result” is provided, it can have any configuration.

  As mentioned above, although this invention has been illustrated using preferable embodiment of this invention, this invention should not be limited and limited to this embodiment. It is understood that the scope of the present invention should be construed only by the claims. It is understood that those skilled in the art can implement an equivalent range based on the description of the present invention and the common general technical knowledge from the description of specific preferred embodiments of the present invention. Patents, patent applications, and documents cited herein should be incorporated by reference in their entirety, as if the contents themselves were specifically described herein. Understood.

  According to the information generation apparatus, the imaging apparatus, and the imaging method of the present invention, it is possible to generate information indicating a still image by correcting the information indicating a plurality of frames, and thus, one still image that is corrected for camera shake. Can be obtained.

  The present invention captures a plurality of images during a predetermined exposure period and corrects camera shake of each of the plurality of images. By obtaining a single still image by performing arithmetic processing on a plurality of corrected images, it is possible to obtain a still image with a good S / N corrected for camera shake.

  The present invention increases the frame rate by increasing the clock speed or decreasing the resolution when the frame rate is low during camera shake correction. Accordingly, there is little deterioration in image quality due to electronic camera shake correction.

  According to the photographing apparatus of the present invention, the photographer can check the degree of camera shake. By checking the degree of camera shake, the photographer can change the camera fixing method, and as a result, a still image with less camera shake than usual can be obtained by human operation.

It is a figure which shows the imaging device 100 of embodiment of this invention. FIG. 3 is a diagram illustrating a configuration of a swing detection unit 15. It is a figure which shows the operation | movement of camera shake correction of embodiment of this invention. It is a figure which shows the masking operation | movement of embodiment of this invention. It is a figure which shows the relationship between a pixel number and a frame frequency. It is a figure which shows the structure of the imaging part 5 containing the pixel area | region divided into four. FIG. 5 is an operation diagram when the frame rate is increased in the embodiment of the present invention. It is a flowchart which shows the procedure (step 50a-step 50f) of the imaging | photography process of embodiment of this invention. It is a flowchart which shows the procedure (step 51a-step 51y) of the imaging | photography process of embodiment of this invention. It is a flowchart which shows the procedure (step 52a-step 52t) of the imaging | photography process of embodiment of this invention. It is a flowchart which shows the sequential calculation processing procedure of embodiment of this invention. It is a flowchart which shows the batch calculation processing procedure of embodiment of this invention. It is a flowchart which shows the process sequence for performing a camera shake correction by integrating a several image (divided image) according to the brightness of an imaging | photography place, or a shutter speed (exposure time). It is a flowchart which shows the procedure which changes the resolution by in-plane pixel addition and time-axis pixel addition. It is a figure which shows the principle which corrects camera shake by setting more pixel numbers than the actual pixel number by embodiment of this invention. It is a figure which shows the addition method of an in-plane direction. It is a figure for demonstrating the shift of the cutout position of an image. It is a figure which shows the principle of reduction interpolation, the principle of expansion interpolation, and the principle of high-resolution camera-shake correction. It is a flowchart which shows the procedure of removing the image which failed in camera shake detection. It is a figure which shows the structure of the imaging device 200 of embodiment of this invention. 6 is a diagram illustrating an example of a display unit 95 included in the imaging apparatus 200. FIG. FIG. 10 is a diagram illustrating another example of the display unit 95 included in the photographing apparatus 200. It is a figure which shows the display of the boundary indicator 97 at the time of panning or panoramic photography. It is a figure which shows the detection point for the camera shake detection in a flame | frame.

Explanation of symbols

2 Lens unit 4 Auto focus unit 5 Imaging unit 6 Zoom unit 8 Sub image memory 9 Oscillation correction unit 10 Display switching unit 11 Display circuit 12 Display unit 13 Recording unit 14 Recording medium 15 Oscillation detection means 16 First memory 17 Second Memory 18 Arithmetic unit 20 Masking unit 21 Oscillation correction control unit 22 Trimming unit 23 Pixel transfer unit 24 Resolution change unit 25 Shutter button 25a Thinning control unit 26 CPU
27 Clock control means 28 Processing clock supply unit 29 Operation unit 29a Sub-operation unit 30 Main image memory 32 Transfer clock supply unit 36 Vibrator 37 Speaker 39 Bright portion extraction unit 40 Frame rate change unit 100 Imaging device

Claims (7)

A photographing device for photographing a still image during a predetermined exposure period,
A swing amount detecting means for detecting a swing amount between a plurality of frames representing the captured still image;
A swing correcting means for correcting a plurality of pieces of frame information indicating the plurality of frames according to the detected swing amount;
Resolution changing means for changing the resolution of the plurality of frames according to at least one of the detected swing amount and zoom ratio;
Storage means for storing a plurality of pieces of frame information that has been subjected to the oscillation correction and the resolution change ;
An imaging apparatus comprising: information generation means for generating still image information indicating the still image based on a plurality of pieces of frame information stored in the storage means. The imaging apparatus according to claim 1 , wherein the information generation unit generates the still image information by collectively calculating a plurality of frame information stored in the storage unit. The photographing apparatus according to claim 1 , wherein the information generation unit generates the still image information by sequentially calculating each of a plurality of frame information stored in the storage unit. The resolution changing means changes the resolution of the plurality of frames in accordance with the detected swing amount,
A frame rate changing means for changing a frame rate according to the swing amount;
The imaging apparatus according to claim 1 , wherein the frame rate indicates the number of a plurality of frames representing a still image captured per unit time. The swing amount detection means detects the swing amount based on information generated by adding information indicating a plurality of pixels included in the imaging surface of the image sensor,
The imaging apparatus according to claim 1 , wherein the swing correction unit corrects the plurality of frame information by cutting out a part of the plurality of frame information according to the swing amount. Determination means for determining whether or not the predetermined exposure time is greater than a predetermined value;
When it is determined that the predetermined exposure time is greater than a predetermined value, the swing amount detection means is based on information generated by adding information indicating a plurality of pixels included in the imaging surface of the imaging element. The imaging device according to claim 1 , wherein the swing amount is detected. A photographing method for photographing a still image during a predetermined exposure period,
Detecting a swing amount between a plurality of frames representing the captured still image;
Correcting a plurality of pieces of frame information indicating the plurality of frames according to the detected swing amount;
Changing the resolution of the plurality of frames according to at least one of the detected swing amount and zoom ratio;
Storing the swing correction and the plurality of frame information with the resolution changed ;
Generating still image information indicating the still image based on a plurality of pieces of frame information stored in the storage means.

Images