JP2007221291A - Image processing apparatus, photographing apparatus, image processing method and control program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect the blur of a photographed image without using hardware for executing complicated arithmetic operation processing and a physical sensor etc. and to discriminate a handshake from a camera work. <P>SOLUTION: A portable video camera 1 includes a photographing unit 20 for sequentially photographing frames constituting a motion image; and a control unit 10 for acquiring a blur quantity and a blur direction for each photographed image, and determining whether the blur of the frame is caused by the handshake or by camera work in photographing. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing apparatus for processing an image, a photographing apparatus having the function of the image processing apparatus, an image processing method in the image processing apparatus, and a control program.

2. Description of the Related Art Conventionally, in a photographing apparatus such as a portable digital video camera or a digital camera to which a moving image shooting function is added, there is known a device equipped with a camera shake correction function that corrects a shake generated in a moving image due to a camera shake at the time of shooting. Yes. Since it is necessary to detect the shake amount when correcting the shake by this camera shake correction function, for example, a method of obtaining the shake amount by detecting a motion vector between temporally continuous frames is adopted ( For example, see Patent Document 1).
Japanese Patent Laid-Open No. 2-117276

  However, the above-described conventional method has a problem in that a calculation process for obtaining a motion vector is very complicated and has a high load, so that a dedicated hardware device or the like for executing this calculation process is required. In addition, in order to obtain the blur direction and blur amount of the photographing operation without obtaining the motion vector, if a physical sensor that physically detects the shake amount of the photographing device is provided, the detection value of the physical sensor is used. The direction and amount of blur can be detected. However, this method requires the use of a physical sensor.

  In addition, at the time of moving image shooting, camera work intentionally performed by the photographer, such as camera work (so-called panning) in which the photographer shakes the photographing apparatus left and right or camera work (so-called tilt) in which the photographing apparatus is moved up and down, may be performed. is there. Therefore, when correcting the shake, it is desirable to distinguish whether the shake is caused by camera shake or camera work. However, in order to realize the function of separating camera shake and camera work, it is essential to use the hardware device that performs the above-described complicated arithmetic processing, a physical sensor, etc., and it is difficult to reduce costs and reduce size. There was a problem.

  The present invention relates to an image processing apparatus, a photographing apparatus, and an image that can detect a shake of a photographed image and separate a camera shake from a camerawork without using a hardware or a physical sensor that performs complicated arithmetic processing. It is an object to provide a processing method and a control program.

  To achieve the above object, the present invention provides a frame acquisition unit that sequentially acquires frames constituting a moving image, a blur analysis unit that acquires a blur amount and a blur direction for each frame, and a blur acquired by the blur analysis unit. There is provided an image processing apparatus comprising: a discriminating unit that discriminates whether the blur of the frame is caused by camera shake or camera work at the time of photographing based on the amount and the blur direction.

  According to this configuration, the blur amount and the blur direction are obtained for each frame, and the blur due to the camera shake and the blur due to the camera work are discriminated based on the blur amount and the blur direction without using a physical sensor or the like. , Camera shake and camera work can be separated. In addition, since complicated and high-load processing such as arithmetic processing between a plurality of frames is not performed, the processing can be realized without using a hardware device that enables complex arithmetic processing.

In the present invention, the blur analysis unit may acquire a blur amount and a blur direction based on a change in luminance between a plurality of pixels in the frame. In this case, camera shake and camera work can be separated by simple and low-load processing.
Further, in the above configuration, the determination unit may determine that the blur of the frame is caused by camera shake when the amount of blur is equal to or less than a predetermined threshold. In this case, camera shake and camera work can be separated by low-load processing based on a threshold.
Further, in the above configuration, when the blur analysis unit detects blurs in the same direction in a predetermined number of consecutive frames or more by the blur analysis unit, these blurs are caused by camera work. It may be determined. In this case, based on the blur amount and the blur direction acquired for each frame, camera shake and camera work are separated depending on whether blur in the same direction is detected in successive frames. For this reason, it is possible to perform analysis over a plurality of frames by simple and low-load processing, and to accurately distinguish between camera shake and camera work.
Still further, the image forming apparatus may further include a correcting unit that corrects the blur of the frame when the determining unit determines that the blur of the frame is caused by camera shake. In this case, it is possible to correct blur caused by camera shake and output a natural image that is easy to view.

  In order to achieve the above object, the present invention is obtained by a photographing unit that sequentially captures frames constituting a moving image, a blur analyzing unit that acquires a blur amount and a blur direction for each frame, and the blur analyzing unit. There is provided a photographing apparatus comprising: a discriminating means for discriminating whether the blur of the frame is caused by camera shake or camera work at the time of photographing based on a blur amount and a blur direction.

  In order to achieve the above object, the present invention sequentially acquires frames constituting a moving image, acquires a blur amount and a blur direction for each frame, and based on the acquired blur amount and blur direction, Disclosed is an image processing method characterized by determining whether blur is caused by camera shake or camera work at the time of shooting.

  In order to achieve the above object, the present invention provides a computer, a frame acquisition unit that sequentially acquires frames constituting a moving image, a blur analysis unit that acquires a blur amount and a blur direction for each frame, and the blur analysis. Provided is a control program that functions as discriminating means for discriminating whether the blur of the frame is caused by camera shake or camera work at the time of shooting based on the blur amount and the blur direction acquired by the means To do.

  In addition to being applied to the above-described image processing apparatus, photographing apparatus, image processing method, and control program, the present invention enables the control program to be downloaded via an electric communication line, or a magnetic recording medium or optical recording. The present invention can also be implemented in such a manner that the program is recorded and distributed on a computer-readable recording medium such as a recording medium using a semiconductor storage element.

  According to the above configuration, camera shake and camera work can be separated without using a physical sensor or the like and without performing complicated and high-load calculation processing.

Embodiments of the present invention will be described below with reference to the drawings. In this embodiment, a case where the present invention is applied to a portable digital video camera (hereinafter simply referred to as a “portable video camera”) as one embodiment of an electronic device will be described.
FIG. 1 is a block diagram showing a configuration of a portable video camera 1 according to the present embodiment. This portable video camera 1 is an application of the image processing device and the photographing device of the present invention. As shown in FIG. 1, the control unit 10, the photographing unit 20, the operation unit 40, the removable media 50, the I / O An F unit 51 and a video output terminal 52 are provided.

  The control unit 10 functions as a frame acquisition unit, a blur analysis unit, a determination unit, and a correction unit that control each unit of the portable video camera 1, and includes a CPU 11 that executes various programs and performs arithmetic processing, and the CPU 11 A rewritable flash ROM 12 storing various control data and various data, a RAM 13 functioning as a work area for temporarily storing calculation results and various data of the CPU 11, and a timer circuit 14 for counting time And. The control program 100 stored in the ROM 12 includes a shake correction program 100A, and the control unit 10 executes the shake correction program 100A at the time of shooting to eliminate the influence of camera shake at the time of shooting. , Realize motion-free video display. Further, the control unit 10 that executes the control program 100 including the shake correction program 100A functions as a computer.

  The control program 100 can be recorded and distributed on a computer-readable recording medium 60 such as a portable memory using a CD-ROM, DVD-ROM, flexible disk, or semiconductor storage element. Further, the personal computer and the portable video camera 1 are communicably connected by a cable or the like, and the control program 100 of the recording medium 60 read by the personal computer is output to the portable video camera 1 so that the flash ROM 12 is controlled. It is also possible to store the program 100.

  The photographing unit 20 functions as photographing means for photographing a subject as a moving image, and includes a camera control circuit 21, a photographing camera 22, a photographing unit RAM 23, and a display panel 24. The camera control circuit 21 controls each unit of the photographing unit 20 under the control of the control unit 10. The photographing camera 22 sequentially outputs frame data (hereinafter simply referred to as “frame”) in which the subject is imaged to the camera control circuit 21 at a predetermined sampling rate. An image sensor in which elements are arranged in a matrix or honeycomb, an optical lens system having a plurality of optical lenses, and a lens driving device for driving the optical lens system to achieve zooming, focusing, a diaphragm, etc. An analog signal image acquired by the image sensor is converted into a digital signal, and an A / D conversion circuit that outputs image data is provided. Here, the number of light receiving elements (so-called number of pixels) and the number of effective pixels of the image sensor included in the photographing camera 22 are arbitrary. A moving image shot by the shooting unit 20 is recorded as a set of a plurality of continuous still images of 30 frames per second (30 fps), for example, and image processing can be performed for each frame.

The photographing unit RAM 23 functions as a buffer for temporarily storing frames. Further, the display panel 24 has a function of displaying various information such as a captured moving image and a setting screen, and is configured by a flat display panel such as a liquid crystal display panel or an organic EL panel, for example. The removable medium 50 is a recording medium for storing moving image data at the time of shooting, and includes, for example, a video tape, a recordable optical disk, and a removable hard disk.
Under such a configuration, the frame output from the photographing camera 22 is subjected to predetermined image processing by the camera control circuit 21 and then temporarily stored in the photographing unit RAM 23, and is also removable media via the control unit 10. 50 are sequentially stored as moving image data. The frame stored in the imaging unit RAM 23 is used when displayed on the display panel 24, and the moving image data stored in the removable medium 50 is used when displaying (reproducing) the captured moving image after shooting.

  The operation unit 40 includes a plurality of operation elements operated by the user, and includes operation keys for inputting various instructions such as a power button and shooting start / end, for example. The I / F unit 51 is an interface for connecting the portable video camera 1 to a personal computer through a cable or the like so that the video data stored in the removable medium 50 can be output to the personal computer. Is output to the personal computer via the I / F unit 51. The video output terminal 52 is a terminal for outputting a video signal of a moving image to an external display device such as a television or a projector. In addition to the above-described components, the portable video camera 1 includes an audio circuit for capturing and recording / reproducing audio signals, an audio output terminal for outputting audio signals to an external speaker, an external amplifier, and the like. It is configured with.

  Then, the portable video camera 1 executes a shake correction process for creating a blur-free output image from the frames shot by the shooting unit 20 by executing the shake correction program 100A stored in the flash ROM 12.

FIG. 2 is a diagram showing an outline of blur correction by the portable video camera 1.
As shown in FIG. 2A, when camera shake occurs during shooting with the portable video camera 1, the relative position of the subject and the shooting target range moves. Therefore, the subject moves unnaturally in the captured frame, making it difficult to see the image.
Here, the frame 70 photographed by the photographing unit 20 is an image having more pixels than the image output from the portable video camera 1 via the video output terminal 52 or the like. An image obtained by trimming a part of the photographed frame 70 is generated, and this image is output. The area to be trimmed is an output unit 71.
The position to be trimmed as the output unit 71 in the frame 70 is movable. In the shake correction process, the control unit 10 determines whether the output unit 71 has the direction and amount of shake in accordance with the direction and amount of the shake when the camera shake occurs. By changing the position, processing to reduce the effects of camera shake is performed.

That is, as shown in FIG. 2B, the portable video camera 1 trims the output unit 71 from the frame 70 photographed by the photographing unit 20 to obtain an output image. The position of the output unit 71 in the frame 70 is the center position of the frame 70 in the initial state, but can be arbitrarily changed from this initial state. For this reason, when camera shake occurs at the time of shooting and the subject in the frame 70 moves, as shown in FIG. 2C, the output unit reverses the direction of the camera shake so as to compensate for the movement due to camera shake. 71 is moved for trimming. By this processing, as shown in FIG. 2D, a corrected image 72 is obtained in which the position of the subject is the same as before the occurrence of blurring.
Thus, according to the shake correction process, the movement of the frame 70 due to camera shake (apparent movement of the subject in the frame 70) is canceled by the movement of the output unit 71, and the influence of camera shake can be eliminated.

FIGS. 3A and 3B are diagrams showing an example of a change in the frame due to blur. FIG. 3A shows a frame shot without blur, and FIG. 3B shows a frame where blur occurs during shooting.
As shown in FIG. 3A, a subject that appears round in a frame without blurring appears in a widened shape as shown in FIG. 3B in a frame in which blurring occurs. The direction and shape of the spread differ depending on the blur direction, and the subject of FIG. 3 (B) appears to have a tail on the left side in the figure, so the portable video camera 1 has moved to the opposite side, that is, the right side. I understand that.

  FIG. 4 is an example of a frame without blur, (A) shows a part of the frame, and (B) schematically shows a change in luminance between PP 'in FIG. 4 (A). FIG. 5 is an example of a frame in which blur occurs, (A) shows a part of the frame, and (B) schematically shows a change in luminance between Q-Q ′ in FIG. 5 (A). Shown in

In the frame illustrated in FIG. 4A, the boundary between the high luminance portion 75 and the low luminance portion (background) 76 is clear. At the boundary between the high luminance portion 75 and the low luminance portion 76, as shown in FIG. 4B, the luminance of adjacent pixels changes extremely.
On the other hand, in the blurred frame, the boundary between the high luminance portion 75 and the background 76 is expanded as shown in FIG. 5A, and a plurality of pixels with medium luminance are generated. In these pixels, as shown in FIG. 5B, the luminance gradually changes across a plurality of pixels. The width indicated by the symbol W in FIG. 5B is a portion that exists only as a boundary between pixels in a non-blurred frame, and the width W indicates the amount expanded by the blur. Hereinafter, the amount of spread indicated by the width W is referred to as a blur amount.

  When the frames constituting the moving image are sequentially shot by the shooting unit 20, the control unit 10 obtains the blur amount indicated by the width W for each frame. For example, the blur amount is determined by comparing the luminance difference between adjacent pixels at one or a plurality of positions in a predetermined frame, and specifying a position where the edge (edge) of the image is expanded by blur. It can be obtained by finding the width of the spread. Further, the control unit 10 obtains the blur direction together with the blur amount. The process for obtaining the direction of blur can be obtained, for example, by identifying a location where the edge of the image has been expanded due to blur as described above and then determining the direction of change in luminance at that location, that is, the direction of expansion. Here, when the direction of expansion is obtained, for example, a pattern of change in luminance may be stored in advance, and this pattern may be compared with the change in luminance at one or more locations in the frame. . For example, the sharpness (sharpness) may be obtained for the processing target frame, and the blur amount may be obtained based on the sharpness.

FIG. 6 is a chart showing the correlation between the movement of the portable video camera and the amount of blur, (A) shows the operating speed of the portable video camera, and (B) shows the amount of blur.
In FIG. 6A, the vertical axis indicates the speed in a specific direction of the portable video camera, and on the vertical axis, 0 (zero) indicates zero speed, and “+” and “−” indicate the direction of the speed. In addition, the vertical axis in FIG. 6B indicates the amount of blur in a specific direction in the photographed image. On the vertical axis, 0 (zero) indicates zero blur, and “+” and “−” indicate the direction of blur. In the charts of FIGS. 6A and 6B, the horizontal axis indicates the passage of time.

The movement of the portable video camera 1 during shooting includes movement due to camera shake of the photographer and movement due to camera work (panning, tilt, etc.). Of these, it is desirable to correct blur caused by camera shake, but it is preferable not to correct blur caused by camera work.
Therefore, the portable video camera 1 distinguishes the cause of the blur based on the blur amount and the blur direction in the frame.
For example, the moving direction (+, −) of the portable video camera 1 frequently changes during the period indicated by the symbol T1 in FIG. This is a feature of movement caused by camera shake during shooting.
On the other hand, in the period indicated by T2 in FIG. 6A, the movement in the same direction (+) continues for a relatively long time. Further, in the period indicated by T3 in FIG. 6A, the movement in the same direction (−) continues for a relatively long time. Further, in both the periods T2 and T3, the speed is extremely large as compared with the period T1. This is a feature of the movement of the portable video camera 1 by camera work.
Correspondingly, the blur amount in the period T1 shown in FIG. 6B is frequently switched in the blur direction (+, −). On the other hand, the blur amount in the period T2 is continuously blurred in the same direction (+), and the blur in the same direction (−) is continuously generated in the period T3. Furthermore, the amount of blur is clearly greater in both periods T2 and T3 than in period T1. As shown in FIG. 6B, when the threshold values TH + and TH− are set for the magnitude of the blur amount, the blur amount in the period T1 falls between the threshold values TH + to TH−, and the blur in the periods T2 and T3 is the threshold value TH +. Or greatly exceeds TH-.

In summary, the blur caused by camera shake has a smaller blur amount than the blur due to camera work. For example, by determining the magnitude of the blur amount using a threshold value, the camera shake and the camera work can be separated.
In addition, if blur in the same direction occurs in a plurality of consecutive frames, it can be said that the blur is caused by camera work.For example, camera shake and camera work are determined based on the number of frames in which blur in the same direction occurs. Can be carved.
That is, the following two pieces of information can be used as a reference for discriminating camera shake and camera work.
I. Threshold of blur amount
II. The number of frames that have consecutive blurs in the same direction

FIG. 7 is an explanatory diagram showing the relationship between the movement of the portable video camera 1 and the frame. In the figure, α indicates the angle of view [rad], L indicates the focal length, β indicates the movement angle [rad] that the portable video camera 1 has moved, and x indicates the amount of blur.
FIG. 7 shows the movement of the portable video camera 1 centered on an axis extending in a specific direction out of the three-dimensional movement of the portable video camera 1. The movement of the portable video camera 1 is centered on the axis. This is expressed as rotation. Here, when the portable video camera 1 is rotated by an angle β, the blur amount x in this case is expressed by the following equation (1).
x = L {1 + tan ² (α / 2)} β (1)

Based on this equation (1), when the blur amount X is obtained when the portable video camera 1 is rotated in the range of -θ ° to + θ °, the calculation equation is as shown in the following equation (2).
X = L {1 + tan ² (α / 2)} {2θ × π / 180} (2)
Generally, the angle at which the portable video camera 1 rotates due to camera shake at the time of shooting is assumed to be within a range of ± 2 °. Therefore, the blur amount X when θ = 2 in the above formula (2) can be used as a reference for separating camera shake and camera work.
However, depending on the moving speed of the portable video camera 1, the blur amount X may appear across a plurality of frames. In this case, the blur amount E per frame is obtained by the following equation (3).
E = X / (N × T) (3)
T is the time [seconds] that the portable video camera 1 moved.
N is a frame rate [fps (frame / second)] at the time of shooting.
As described above, the blur amount E per frame when the portable video camera 1 moves in a range of ± 2 °, for example, can be obtained and used as a threshold for the blur amount.

Further, the number of frames in which blurring in the same direction has occurred continuously is obtained as follows.
The number of frames D per time T [seconds] is obtained as shown in the following equation (4) based on the frame rate N [fps] at the time of shooting.
D = N × T (4)
In general, movement in the same direction of the portable video camera 1 due to camera shake converges within 0.3 seconds. Therefore, if the direction in which the portable video camera 1 moves changes within approximately 0.3 seconds, it can be said that the movement is due to camera shake. In other words, if the moving direction of the portable video camera 1 does not change over 0.3 seconds, it can be said that the movement of the portable video camera 1 is due to camera work.
For example, when the frame rate is 15 [fps], the number of frames captured in 0.3 seconds is obtained as D = 15 × 0.3 = 4.5 by the above equation (4). Therefore, when blurs in the same direction occur in frames exceeding 4.5, it can be determined that the blur is caused by camera work.
Thus, for example, when the portable video camera 1 moves in the same direction for 0.3 seconds, the number D of frames in which blur in the same direction occurs can be obtained and used as a threshold value for the number of frames. Here, since the number of frames is always an integer, the threshold number of frames D is preferably an integer. For this reason, when the number of frames D obtained by the above equation (4) is a decimal number, rounding up is performed. For example, when D = 4.5 as described above, the threshold number of frames may be set to 5.
The blur amount E and the frame number D as threshold values obtained as described above are stored in advance in the flash ROM 12 when the blur correction process is executed.

FIG. 8 is a flowchart showing a shake correction process executed in the portable video camera 1. The shake correction process shown in FIG. 8 is realized by the CPU 11 reading and executing the shake correction program 100A stored in the flash ROM 12.
This blur correction process is a blur caused by camera shake by detecting the occurrence of blur in the shot frame during shooting by the portable video camera 1 and determining whether the blur is caused by camera shake or camera work. In this case, correction is performed and a corrected image is output, and in the case of blur due to camera work, the image is output without correction.

  When the blur correction process is started (step S11), the CPU 11 reads the blur amount E and the frame number D as the threshold values stored in advance in the flash ROM 12, sets them as the threshold values for this process, and stores them in the RAM 13. (Step S12). Furthermore, the CPU 11 sets the same direction counter, stores this count value as 1 in the RAM 13 (step S13). The same direction counter is a counter that counts the number of frames when blurring in the same direction occurs in a plurality of consecutive frames.

  Subsequently, the CPU 11 obtains a blur amount and a blur direction for the frame photographed by the photographing unit 20 (step S14), and whether the obtained blur direction is the same as the blur direction in the frame photographed by the previous photographing unit 20 or not. Is determined (step S15). Here, when the obtained blur direction is the same as the blur direction in the previous frame (step S15; Yes), the CPU 11 increments the count value of the same direction counter stored in the RAM 13 by 1 (step S16). On the other hand, if the frame for which the blur amount and the blur direction are obtained in step S14 is the first frame, and if the blur direction is different from the previous frame (step S15; No), the CPU 11 stores the same stored in the RAM 13. The count value of the direction counter is set to 1 (step S17).

  Thereafter, the CPU 11 compares the number of frames D as the threshold set in step S12 with the count value of the same direction counter stored in the RAM 13 (step S18). Here, when the count value is smaller than the frame number D (step S18; No), the CPU 11 determines whether or not the blur amount calculated in step S14 is equal to or less than the blur amount E as the threshold set in step S12. (Step S19). If the calculated blur amount is equal to or less than the blur amount E (step S19; Yes), blur due to camera shake has occurred, and thus the CPU 11 performs a process of correcting the blur of the frame. That is, an amount (correction amount) for moving the output unit 71 (FIG. 2) in the frame 70 (FIG. 2) is obtained (step S20), and the output unit 71 is trimmed at a position based on this correction amount to create a corrected image. Are output (step S21). Thereafter, the CPU 11 proceeds to step S23.

If the count value is greater than or equal to the number of frames D in step S18, that is, if blurring in the same direction has occurred in consecutive frames greater than or equal to the number of frames D, the CPU 11 The output unit 71 is trimmed and output as a captured image (step S22), and the process proceeds to step S23.
Furthermore, when the amount of blur exceeds the threshold value E in step S19, the CPU 11 proceeds to step S22 described above.
In step S23, the CPU 11 determines whether or not the photographing is finished. When the photographing is continued, the process returns to step S15, and when the photographing is finished, the present process is finished.

In this way, the portable video camera 1 obtains the blur amount and the blur direction for each frame that is continuously shot during moving image shooting, and obtains the number of consecutive frames in which blur in the same direction has occurred. Using the blur amount E and the number of frames D stored in the flash ROM 12 as threshold values, it is determined whether the cause of blur is camera shake or camera work.
Then, when the obtained blur amount exceeds the threshold value E, and when blurring in the same direction occurs in consecutive frames that are equal to or greater than the threshold value D, it is determined that these blurs are caused by camera work, and correction is performed. Output the shot image without doing it. On the other hand, in a case not corresponding to the above case, that is, when the obtained blur amount is equal to or less than the threshold value E and blurs in the same direction are not continued in a frame equal to or greater than the threshold value D, blurring due to camera shake has occurred. Discriminate and output the corrected image.
Thereby, in the portable video camera 1 that does not include a physical sensor such as an acceleration sensor or a gyro sensor, it is possible to accurately separate camera shake and camera work. Further, the process for determining the type of blur is an extremely simple process based on the blur amount E and the number of frames D as threshold values, and does not involve high-load arithmetic processing such as complex analysis processing over a plurality of frames, for example. . Therefore, it can be executed without using high-performance hardware resources. Since the correction is performed only when the blur is caused by camera shake, an easy-to-see and natural moving image can be obtained.

Furthermore, since the blur amount and the blur direction in the frame are obtained based on the change in luminance between a plurality of pixels in the frame, camera shake and camera work can be separated by simple and low-load processing.
Further, in the process of discriminating camera shake and camera work, since the magnitude comparison using the blur amount E and the number of frames D as threshold values stored in advance in the flash ROM 12 is performed, the discrimination can be performed by a process with a very light load. is there. Furthermore, camera shake and camera work are separated depending on whether or not blur in the same direction is detected in consecutive frames, so analysis that spans multiple frames is achieved through simple and low-load processing, and camera shake and the camera are accurately detected. Can be separated from the workpiece.
When it is determined that the blur of the frame is caused by camera shake, the blur is corrected and a corrected image is output, so that a natural image that is easy to view can be output.

The above-described embodiment is merely an aspect of the present invention, and can be arbitrarily modified and applied within the scope of the present invention.
For example, in the above-described embodiment, the example in which the blur correction process illustrated in FIG. 8 is performed on the frame 70 photographed by the photographing unit 20 has been described. For example, the frame 70 includes a plurality of frames stored in the removable medium 50 or the like. You may make it perform a blurring correction process at the time of the display (reproduction | regeneration) about a moving image. That is, if the data of the shot frame and the data indicating the amount of blur at the time of shooting each frame are stored in the removable medium 50 or the like, the blur correction process can be executed in another device that does not have the shooting function. Is possible.
Furthermore, in the above-described embodiment, the blur amount E and the number of frames D as threshold values are preliminarily stored in the flash ROM 12 when executing the blur correction process. However, the present invention is not limited to this. Instead, for example, the blur amount E and the number of frames D may be obtained based on information input by operating the operation unit 40, or based on the blur amount and the blur direction in a frame shot in the past. Thus, the blur amount E and the number of frames D may be obtained as needed.
Further, the present invention can also be applied to a photographing apparatus having a moving image photographing function other than the portable video camera 1 described in the above embodiment and the modified examples. The present invention can also be applied to a digital still camera and various electronic devices such as a mobile phone, a PDA, and a notebook personal computer equipped with such a digital still camera.

It is a block diagram which shows the structure of the digital camera which concerns on embodiment. It is a figure which shows the mode of blurring correction. It is a figure which shows the change of the flame | frame by blurring. It is a figure which shows the example of a flame | frame without blurring. It is a figure which illustrates the state of the blur produced by blurring. It is a figure which shows the correlation with a motion and the amount of blurs of a portable video camera. It is explanatory drawing which shows the relationship between a motion of a portable video camera, and a frame. It is a flowchart which shows a blurring correction process.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Portable digital video camera (an imaging device, an image processing device), 10 ... Control part (a frame acquisition means, a blur analysis means, a discrimination means, a correction means, a computer), 20 ... An imaging part (imaging means), 24 ... Display Panel: 50 ... Removable medium, 60 ... Recording medium, 70 ... Frame, 71 ... Output unit, 72 ... Corrected image, 75 ... High brightness part, 76 ... Low brightness part, 100 ... Control program, 100A ... Shake correction program.

Claims (8)

Frame acquisition means for sequentially acquiring frames constituting the video;
Blur analysis means for obtaining a blur amount and a blur direction for each frame;
Based on the blur amount and the blur direction acquired by the blur analysis unit, a discriminating unit for discriminating whether the blur of the frame is caused by camera shake or camera work at the time of shooting;
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the blur analysis unit acquires a blur amount and a blur direction based on a change in luminance between a plurality of pixels in the frame.   The image processing apparatus according to claim 1, wherein the determination unit determines that the blur of the frame is caused by camera shake when the amount of blur is equal to or less than a predetermined threshold.   The discriminating means discriminates that the blur is caused by camera work when blur in the same direction is detected in a predetermined number or more of the consecutive frames by the blur analyzing means. The image processing apparatus according to claim 1.   5. The image according to claim 1, further comprising a correcting unit that corrects the blur of the frame when the determining unit determines that the blur of the frame is caused by camera shake. Processing equipment. Photographing means for sequentially photographing the frames constituting the video;
Blur analysis means for obtaining a blur amount and a blur direction for each frame;
Based on the blur amount and the blur direction acquired by the blur analysis unit, a discriminating unit for discriminating whether the blur of the frame is caused by camera shake or camera work at the time of shooting;
Correction means for correcting blur of the frame when the determination means determines that the blur of the frame is due to camera shake;
An imaging apparatus comprising: Obtain the frames that make up the video in sequence,
Obtaining a blur amount and a blur direction for each frame, and determining whether the blur of the frame is caused by camera shake or camera work at the time of shooting based on the obtained blur amount and blur direction;
An image processing method characterized by the above. Computer
Frame acquisition means for sequentially acquiring frames constituting the video;
Blur analysis means for obtaining a blur amount and a blur direction for each frame;
Based on the blur amount and the blur direction acquired by the blur analysis unit, a discriminating unit for discriminating whether the blur of the frame is caused by camera shake or camera work at the time of shooting;
A control program characterized by functioning as a function.

Images