JP2010004113A - Imaging apparatus, imaging method, image processing device and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide: an imaging apparatus capable of changing moving images and capturing the moving images that can be appreciated; an imaging method; an image processing device; and an image processing method. <P>SOLUTION: The imaging apparatus includes: a telephoto photographing section 1a for generating first image data indicating the first image of a first photographing range; a wide angle photographing section 1b for generating second image data indicating the second image of a second photographing range wider than the first photographing range; an image recording control section 2 for storing the first image data and the second image data; a shake determining section 3 for determining whether or not the movement of the first image data is unstable; and an image synthesis section 7 for complementing the image data of a section determined as being unstable by the shake determining section 3 in the first image data by the second image data. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging device, an imaging method, an image processing device, and an image processing method, and more specifically, an imaging device, an imaging method, an image processing device, and an image processing method that can shoot a moving image that can be easily viewed by anyone. About.

  Conventionally, a camera has been a device for taking a still image. However, in recent years, since image signals have been digitized, moving images can be shot by performing inter-frame compression on images that have been acquired continuously. It has become possible to use as a device. For this reason, recent digital cameras are capable of shooting moving images as well as still images.

  In this way, digital cameras can shoot both still images and movies, but for users who have only taken still images, they do not know when to shoot movies, Even when I started shooting, I didn't know when to stop shooting, so it was often unsightly and sluggish.

As a solution to this problem, for example, Patent Document 1 discloses a camera capable of switching display between a self-portrait image taken toward the photographer and a partner-taken image taken toward the opponent. An attached portable image display device has been proposed. Further, Patent Document 2 proposes an electronic camera that compensates for a still image when a moving image is interrupted due to a still image interruption or the like during moving image shooting. Further, Patent Document 3 proposes an imaging apparatus that simultaneously captures a first area and a second area during moving image shooting.
JP 2005-92657 A JP 2003-8948 A Japanese Patent No. 3778163

  As described above, there has been proposed a photographing apparatus that allows a user who has become a still image to be familiar with a moving image, but it has not been sufficient for photographing a moving image that can be appreciated. That is, the camera-equipped portable image display device disclosed in Patent Document 1 simply switches between a still image and a moving image, and does not shoot a moving image that can be changed and can be enjoyed. In addition, since the electronic camera disclosed in Patent Document 2 becomes a still image when the movie is cut off, it cannot continue to capture the movement of the subject, so that the movie can be changed and can be appreciated. It is not intended to shoot the video. Furthermore, the image pickup apparatus disclosed in Patent Document 3 simply shoots a plurality of moving images at the same time, and by appropriately selecting two moving images, it does not perform shooting that can change the moving images and endure viewing. Absent.

  The present invention has been made in view of such circumstances, and makes it easy to shoot difficult moving images, and to easily change moving images that tend to be monotonous, and to shoot moving images that can be appreciated. It is an object of the present invention to provide an imaging device, an imaging method, an image processing device, and an image processing method that can be used.

  In order to achieve the above object, an imaging apparatus according to a first aspect of the present invention represents first image data representing a first image in a first imaging range and a second image in a second imaging range wider than the first imaging range. An imaging unit that generates second image data, an image data storage unit that stores the first image data and the second image data, and a determination unit that determines whether or not the movement of the first image data is unstable. An image data synthesis unit that supplements image data of a section determined to be unstable by the determination unit in the first image data with the second image data.

In the imaging apparatus according to a second aspect of the present invention, in the first aspect, the image data composition unit is configured to determine a section of the image data determined to be more unstable than the determination unit in the first image data. Control is performed so that the second image data corresponding to the section is stored in the image data storage unit.
According to a third aspect of the present invention, there is provided an image pickup apparatus according to the first aspect, wherein the image data storage unit stores the first image data and the second image data in parallel. The image data of the section determined to be unstable by the determination unit in the first image data is replaced with the second image data corresponding to the section.

According to a fourth aspect of the present invention, there is provided an imaging apparatus according to the first aspect, wherein the imaging unit includes a first imaging unit having a first zoom unit, and a second zoom unit having a different angle of view from the first imaging unit. A second imaging unit provided;
According to a fifth aspect of the present invention, in the fourth aspect of the present invention, the photographing field angle adjustable by the first zoom means and the photographing field angle adjustable by the second zoom means do not overlap each other. .

In the imaging device according to a sixth aspect of the present invention, in the first aspect, the determination unit determines that the movement of the first image is unstable when the first image is moving at a predetermined speed or more. judge.
According to a seventh aspect of the present invention, in the first invention, the imaging apparatus further includes a blur detection unit that detects a blur amount of the first image, and the determination unit includes the blur amount of the first image. Is determined to be greater than or equal to a predetermined value, it is determined that the movement of the first image is unstable.

  An imaging method according to an eighth aspect of the invention generates first image data representing a first image in a first imaging range and second image data representing a second image in a second imaging range wider than the first imaging range. The step of storing, the step of storing the first image data and the second image data, the step of determining whether the movement of the first image is unstable, and the determining unit of the first image data Supplementing image data of a section determined to be unstable with the second image data.

  According to a ninth aspect of the invention, there is provided an image processing apparatus comprising: first image data representing a first image in a first photographing range; and second image data representing a second image in a second photographing range wider than the first photographing range. An image data storage unit to be stored, a determination unit that determines whether or not the movement of the first image is unstable, and image data of a section determined to be unstable by the determination unit in the first image data Is supplemented with the second image data.

According to a tenth aspect of the present invention, in the ninth aspect, the image data synthesizing unit is configured such that the image data of the section determined to be unstable by the determination unit in the first image data is Replace with the second image data corresponding to the section.
An image processing apparatus according to an eleventh aspect of the invention is the image processing apparatus according to the ninth aspect, wherein when the first image is moving at a speed greater than or equal to a predetermined speed, the movement of the first image is unstable. Is determined.
In an image processing apparatus according to a twelfth aspect, in the ninth aspect, when the determination unit determines that the blur amount of the first image is equal to or greater than a predetermined amount, the movement of the first image is unstable. Judge that there is.

  According to a thirteenth aspect of the present invention, there is provided an image processing method comprising: first image data representing a first image in a first photographing range; and second image data representing a second image in a second photographing range wider than the first photographing range. A step of storing, a step of determining whether or not the movement of the first image is unstable, and image data of a section determined to be unstable by the determination unit in the first image data. Complementing with two image data.

  An image pickup apparatus according to a fourteenth aspect of the present invention is a first lens configured with a first fixed focal length, a first image pickup element disposed near the image plane of the first lens, and the first lens. A first imaging unit that changes the effective range of the image data of the image sensor and performs electronic zooming, and a second lens that is configured with a second fixed focal length different from the first fixed focal length, A second imaging device disposed in the vicinity of the imaging surface of the second lens, and a second imaging unit that changes the effective range of image data of the second imaging device and performs electronic zooming. However, the focal length variable range by the first photographing unit and the focal length variable range by the second photographing unit do not overlap.

An image pickup apparatus according to a fifteenth aspect of the present invention is the image pickup apparatus according to the fourteenth aspect, wherein the image pickup apparatus further includes a zoom operation member, and the focal length is changed to a discrete focal length according to an operation of the zoom operation member. .
According to a sixteenth aspect of the present invention, in the fifteenth aspect, the zoom operation member includes a wide-angle zoom operation member that gives an instruction to the wide-angle side and a telephoto zoom operation member that gives an instruction to the telephoto side. The focal length to be changed is different between the instruction to the wide angle side and the instruction to the telephoto side.

  Advantageous Effects of Invention According to the present invention, an imaging apparatus, an imaging method, and an image that can easily shoot difficult moving images, can easily change moving images that tend to be monotonous, and can shoot moving images that can be appreciated. A processing device and an image processing method can be provided.

  Hereinafter, a preferred embodiment will be described using a camera to which the present invention is applied according to the drawings. The camera according to the first embodiment of the present invention includes two imaging units, a wide-angle imaging unit that captures images at a wide angle and a telephoto imaging unit that captures images at a telephoto range, in order to acquire moving images. In response to the operation of the release button, acquisition of moving image data is started and recorded on a recording medium. In addition, when the image in the first embodiment is disturbed due to unstable framing such as camera shake or panning, the camera is switched from the telephoto shooting unit to the wide-angle shooting unit so as not to cause an uncomfortable scene. Yes.

  FIG. 1 is a block diagram showing the configuration of a camera according to the first embodiment of the present invention. A camera 10 is a digital camera that captures an image of a subject. The camera 10 includes a wide-angle photographing unit 1a, a telephoto photographing unit 1b, an image recording control unit 2, a blur determination unit 3, an operation unit 4, a recording unit 5, an image composition unit 7, and a display unit 8.

  The wide-angle photographing unit 1a shoots the entire subject at the wide-angle focal length and outputs moving image data, and the telephoto-side photographing unit 1b shoots the details of the subject at the telephoto focal length. Output data. Switching between the wide-angle imaging unit 1a and the telephoto imaging unit 1b is performed by arranging the imaging element 6 near the focal plane of the lens 9 and changing the effective range of the imaging element 6 as shown in FIG. ing.

  That is, at the time of photographing by the wide-angle photographing unit 1a, the subject is photographed at the wide-angle focal length by using the entire area of the image sensor 6. On the other hand, at the time of shooting by the telephoto shooting unit 1b, by using a part of the image pickup area of the image pickup device 6, the subject is shot at the telephoto side focal length. In the present embodiment, when the camera shake increases during telephoto shooting with a narrow effective range, control is performed to switch to wide-angle shooting with a wide effective range.

  FIG. 2B shows another example of the wide-angle photographing unit 1a and the telephoto photographing unit 1b, in which a telephoto lens 9t and a wide-angle lens 9w are arranged on the front surface of the camera 10 main body. Image pickup elements are arranged in the vicinity of the focal image planes of the wide-angle lens 9w and the telephoto lens 9t. These image sensors output moving image data.

  In the present embodiment, the configurations of the wide-angle imaging unit 1a and the telephoto imaging unit 1b may employ either the configuration shown in FIG. 2A or FIG. The photographing optical systems of the photographing units 1a and 1b may employ a zoom lens. However, in the present embodiment, a photographing optical system with a fixed focal length is employed to change the effective range of the image sensor. The electronic zoom system is used to change the focal length. Note that there are various ways to change the effective range of the image sensor, such as changing the range in which the image signal is read from the image sensor, or cutting out image data in the effective range from the image data read from the image sensor. There is a method, and may be selected as appropriate.

  The image recording unit 2 performs overall control of the camera 10 and also controls recording of moving image data output from the wide-angle imaging unit 1a and the telephoto imaging unit 1b. The auxiliary recording unit 2a temporarily stores the image data output from the wide-angle imaging unit 1a, and the auxiliary recording unit 2b temporarily stores the image data output from the telephoto imaging unit 1b. In the present embodiment, wide-angle moving image data and telephoto moving image data are selectively recorded as moving images, but are temporarily stored in the auxiliary recording units 2a and 2b, respectively, when moving images are recorded.

  The blur determination unit 3 determines whether or not the camera 10 is performing an unstable movement such as a camera shake applied to the camera 10 or a panning operation by a photographer. That is, the blur determination unit 3 inputs image data from the telephoto imaging unit 1b, and determines whether or not the movement is unstable, such as an image changing irregularly due to a change in the image data. In addition to analyzing image data, a sensor such as an acceleration sensor, an angular acceleration sensor, or a gyro may be provided, and the blur determination may be performed based on the sensor output. Based on the output of the blur determination unit 3, known blur reduction processing can also be performed.

  The operation unit 4 is an operation member such as a release button or a zoom button provided on the camera 10. When the release button is operated, the camera 10 starts moving image shooting, and when the release button is operated again, the moving image shooting is stopped. The zoom buttons include a telephoto zoom operation member and a wide-angle zoom operation member, and zooming is performed according to the operation of these operation members. The operation state of the operation unit 4 is transmitted to the image recording control unit 2. The recording unit 5 has a microphone and records surrounding sounds during shooting.

  The display unit 8 includes a display device such as a liquid crystal monitor disposed on the back surface of the camera 10, and performs playback display of moving images recorded in the image recording control unit 2. In addition, a moving image acquired by the wide-angle photographing unit 1a or the telephoto photographing unit 1b is displayed as a so-called live view for observing a subject image.

  The image synthesizing unit 7 synthesizes the image acquired by the wide-angle imaging unit 1a and the image acquired by the telephoto imaging unit 1b. The composite image generated by the image composition unit 7 is displayed on the display unit 8. The auxiliary recording units 2a and 2b described above are temporary storage units for synthesizing images using the image data acquired by each imaging unit.

  In the image synthesis in the image synthesis unit 7, the determination result of the blur determination unit 3 is input, and the image is switched based on the blur output. As will be described later, in the present embodiment, the moving image acquired by the telephoto shooting unit 1b is recorded. However, when the blur determination unit 3 detects a blurring state, the moving image acquired by the wide-angle shooting unit 1a is recorded. Replace with the video obtained by, and record. The image composition unit 7 has a function of displaying two images side by side on the same screen as well as switching between the two images. At this time, the image determined to be blurred by the blur determination unit 3 may be displayed small.

  Next, the operation in the present embodiment will be described. First, a scene that could not be shot with a still image will be described with reference to FIG. In general, when photographing a large building such as a temple or shrine at an historic site, first, as shown in FIG. However, since the details are not clearly shown, a part of the building is photographed as shown in FIG. In other words, in order to record both the overall force and the detailed description, two still images were taken, but the relationship between the two images was difficult to understand.

  This problem can be overcome by shooting the moving image of the process of FIG. 3A → FIG. 3B. In order to shoot the details, framing is done in the telephoto state, but if the movement is regular and slow at this time, the viewer can be seen calmly, but often it can not be planned shooting at the destination If the screen is moved in a hurry, the image tends to be distorted and unsightly as shown in FIG. This is because, in particular, when shooting with a telephoto lens, the small angle of view makes a large change even with small movements.

  Since such a moving image makes the viewer feel uncomfortable like a motion sickness, it is required to shoot without an unpleasant change scene. In the present embodiment, when an unstable movement is detected, a more easily viewable image that is simultaneously captured is preferentially displayed and recorded. Specifically, instead of taking a picture on the telephoto side as shown in FIG. 3C, the picture is taken on the wide-angle side as shown in FIG.

  Further, even when shooting is performed by switching to the wide-angle side as shown in FIG. 3 (e), when unstable movement is lost, the original shooting is restored as shown in FIG. 3 (d). ing. In this way, by switching the angle of view according to the blurring state, it is possible to express the entire subject and details from the viewpoint, and it is possible to prevent the captured image from being disturbed.

  Subsequently, switching of the angle of view in the present embodiment will be described with reference to FIG. FIG. 4 shows a type having two photographing lenses, that is, the wide-angle lens 9w and the telephoto lens 9t shown in FIG. As shown in FIG. 4A, an image sensor 6w is disposed in the vicinity of the image plane of the wide-angle lens 9w. The image sensor 6w is a solid-state image sensor such as a CCD or CMOS having 9 million pixels. Similarly, as shown in FIG. 4B, an image sensor 6t is disposed in the vicinity of the image plane of the telephoto lens 9t. This imaging element 6t is also a solid-state imaging element such as a CCD or CMOS having 9 million pixels.

  In the image pickup device 6w, the shooting range R1 is the entire angle of view of the image pickup device 6, and the shooting range R2 indicates a portion limited to 1/9 of the entire range of the image pickup device 6w. This is in the range of 1/3 as the angle of view, and corresponds to a change width of 3 × zoom. In other words, if the photographing lens 9w is equivalent to a 35 mm photographic lens, the electronic zoom can be achieved up to 105 mm, which is three times as much, due to imaging limitations. If the image sensor 6w is 9 million pixels, the effective pixel is 100 pixels because the electronic zoom is limited to 1/9 in the range of R2.

  In the image pickup device 6t, the telephoto lens 9t is set so that the shooting range R3 using the entire area of the image pickup device 6t is narrower than the above-described shooting range R2. However, the narrowness of the shooting range is set such that the shooting range R3 corresponds to 120 mm if the shooting range R2 corresponds to 105 mm.

  Similarly to the image sensor 6w shown in FIG. 4A, in the case of the image sensor 6t shown in FIG. 4B, when the shooting range is narrowed, the electronic zoom can cover from 120 mm to 3 times 360 mm. Therefore, the combination of the imaging elements 6w and 6t, the wide-angle lens 9w, and the telephoto lens 9t can cover from 35 mm to 360 mm, and a 10 × electronic zoom can be obtained. Many video cameras currently on the market are equipped with about 10 × zoom. However, in the camera 10 of this embodiment, the effect of 10 × zoom is achieved by two fixed lenses (a wide-angle lens 9w and a telephoto lens 9t). Obtainable.

  If R2 = R3, the shooting range R3 is equivalent to 105 mm, and the shooting range R4 is 315 mm. In other words, the focal length at the telephoto end is equivalent to 315 mm, and the zoom is not 10 times.

  In the present embodiment, as described above, the wide-angle lens 9w and the telephoto lens 9t are fixed focal length lenses. In the case of a fixed focal length lens, a driving sound is not generated during driving, unlike a zoom lens that is mechanically driven, so that it is possible to improve sound quality during recording. The interval between the shooting range R2 and the shooting range R3 is an electronic zoom non-corresponding section, but between 105 mm and 120 mm is about 10%, and it is sufficient to correspond with a focal length of either 105 mm or 120 mm. The difference is negligible.

  In the present embodiment, when determining the effective range, the minimum number of pixels is set to 100 in consideration of the resolution at the time of reproduction and display on a high-definition television. That is, in a high-definition television, there are 1920 × 1080 display pixels, which corresponds to about 2 million pixels. As shown in FIGS. 5A and 5B, when the left and right sides of the television screen 15 are left and the shooting screen 16 is arranged only in the center of the screen, or when two screens are arranged on the television screen 15 (FIG. 11B )) Considers that about half a million pixels is sufficient. When the image sensors 6w and 6t have 9 million pixels, only 1/9 of the entire area may be used.

  Next, lens angle-of-view control of the zoom camera as shown in FIG. 4 will be described using the flowchart shown in FIG. If the flow of view angle switching is entered, it is first determined whether or not the camera is on the telephoto side (S101). In this step, it is determined whether or not the telescopic zoom operation member in the operation unit 4 has been operated.

  If the result of determination in step S101 is that it has been operated on the telephoto side, it is next determined whether or not it is within the first limit (S111). In this step, the focal length of the electronic zoom currently set by a focal length setting unit (not shown) is detected, and it is determined whether this focal length is within the first limit. As shown in FIG. 6B, the first limit is between the wide angle end (wide end) and the first limit (105 mm in the example shown in FIG. 4).

  If the result of determination in step S111 is within the first limit, the focal length is increased pseudo 10 mm (S114). In this step, the effective range of the image sensor 6w on which the subject image is formed by the wide-angle lens 9w is gradually limited to the telephoto side by a focal length of 10 mm. After 10mm up, return to the original flow.

  If the result of determination in step S111 is not within the first limit, it is next determined whether or not the first limit has been exceeded (S112). In this step, the focal length of the electronic zoom currently set by a focal length setting unit (not shown) is detected, and it is determined whether or not the focal length on the telephoto side exceeds the first limit (105 mm in the example shown in FIG. 4). judge.

  If the result of determination in step S112 is that the first limit has been exceeded, the process moves to the second lens (tele) (S115). That is, since the focal length on the telephoto side exceeds the first limit, processing is performed using image data from the image sensor 6t formed by the telephoto lens 9t. Note that the shooting range R2 (see FIG. 4A) immediately before the first limit is exceeded, and if it is determined that the first limit is exceeded, the shooting range R3 (see FIG. 4B) is entered. Therefore, in this embodiment, the focal length changes from 105 mm to 120 mm. When moving to the second lens (telephoto lens 9t), the original flow is restored.

  If the result of determination in step S112 is that the first limit has not been exceeded, that is, if it has not been the changing point from the first limit to the second limit (120 mm in the example shown in FIG. 4), then the second limit It is determined whether it is within the limit (S113). Within the second limit is the focal length from the second limit to the telephoto end, and is performed by detecting the focal length in electronic zoom currently set by a focal length setting unit (not shown). As shown in FIG. 6B, the second limit is between the telephoto end (tele end) and the second limit (120 mm in the example shown in FIG. 4).

  If the result of determination in step S113 is that it is within the second limit, the focal length is artificially increased by 15 mm (S116). In this step, the effective range of the image sensor 6t on which the subject image is formed by the telephoto lens 9t is gradually limited to a telephoto distance of 15 mm. After 15mm up, return to the original flow.

  If the result of determination in step S113 is not within the second limit, the flow returns to the original flow.

  If the result of determination in step S101 is that there has been no operation on the telephoto side, it is next determined whether or not the zooming operation is on the wide angle side (S102). In this step, it is determined whether or not the wide-angle zoom operation member in the operation unit 4 has been operated. If the result of this determination is that the operation is not to the wide-angle side, the zooming operation has not been performed in any direction, and the original flow is restored.

  If the result of determination in step S102 is that zooming to the wide angle side has been made, it is determined whether or not it is within the second limit (S121). In this step, the focal length in the electronic zoom currently set by a focal length setting unit (not shown) is detected, and it is determined whether or not the focal length is within the second limit.

  If the result of determination in step S121 is within the second limit, the focal length is artificially reduced by 15 mm (S124). In this step, the effective range of the image sensor 6t on which the subject image is formed by the telephoto lens 9t is gradually limited, and the field angle is widened by a focal length of 15 mm. If it goes down by 15mm, it returns to the original flow.

  If the result of determination in step S121 is not within the second limit, it is determined whether or not the second limit has been exceeded (S122). In this step, the focal length in the electronic zoom currently set by a focal length setting unit (not shown) is detected, and it is determined whether or not the telephoto side focal length exceeds the second limit (120 mm in the example shown in FIG. 4). judge.

  If the result of determination in step S122 is that the second limit has been exceeded, the process moves to the second lens (wide) (S125). That is, since the focal length is on the wide angle side exceeding the second limit, processing is performed using image data from the image sensor 6w formed by the wide angle lens 9w. Note that immediately before the second limit is exceeded, the shooting range is R3 (see FIG. 4B), and when it is determined that the second limit is exceeded, the shooting range is R2 (see FIG. 4A). Therefore, in this embodiment, the focal length changes from 120 mm to 105 mm. When moving to the first lens (wide-angle lens 9w), the original flow is restored.

  If the result of determination in step S123 is that it is within the first limit, the focal length is artificially reduced by 10 mm (S126). In this step, the effective range of the image sensor 6w on which the subject image is formed by the wide-angle lens 9w is gradually limited to a wide angle by a focal length of 10 mm. When it goes down by 10 mm, it returns to the original flow.

  If the result of determination in step S123 is that it is not within the first limit, processing returns to the original flow.

  Thus, in the present embodiment, the angle of view can be controlled in the range of about 10 times zoom by the user's control. In the present embodiment, the focal length is increased or decreased by 10 mm or 15 mm (S114, S126, S116, S124). That is, in the present embodiment, the focal length is not continuously changed, but is changed stepwise to the focal length. For this reason, it is possible to achieve an electronic zoom with a large zoom ratio without a sense of incongruity even if a gap is provided in the change in focal length between the first limit and the second limit.

  Further, the focal length is changed every 10 mm within the first limit on the wide angle side and every 15 mm within the second limit on the telephoto side. This is because, on the wide angle side, the change in the angle of view is generally larger than the change in the focal length.

  In the present embodiment, the first limit is set to 105 mm and the second limit is set to 120 mm. However, this is an example, and the focal length may be changed to another focal length. Further, although the focal length is changed every 10 mm within the first limit and every 15 mm within the second limit, this value is also an example and may be changed to another focal length.

  Next, a change in the angle of view in the present embodiment will be described using a case where a mountain range and a building in the mountain range are photographed with a moving image. FIG. 7 shows a landscape where the buildings HA1 and HA2 are located on the ridgeline of the mountain range Mt, and shows a situation where the user is photographing the landscape from the left side to the right side of the screen. At this time, the user is photographing the buildings H2 from the buildings H1 with the telephoto lens 9t in order of the photographing ranges Ft1, Ft2, and Ft3.

  However, if the user moves the camera 10 at a high speed, there is no overlap between the shooting ranges Ft1 and Ft2 and between the shooting ranges Ft2 and Ft3, and the user does not know how the camera 10 moves during this time. The eyes flicker and become uncomfortable. However, when the angle of view of the photographic lens is wider and the positions of the photographic ranges Fw1, Fw2, and Fw3 are changed, there is an overlapping portion, and the image is less irritating and easy to see. As described above, in the example illustrated in FIG. 7, the wide-angle side can capture an image that is easy to see corresponding to the fast movement of the camera 10 than the telephoto side.

  Next, the operation of the camera 10 in the first embodiment of the present invention will be described using the flowchart shown in FIG. In the present embodiment, as described with reference to FIG. 7, when an unpleasant movement is detected at the time of viewing by moving the camera quickly or the like, moving image shooting with reduced discomfort can be performed by changing the angle of view.

  If the camera control flow is entered, it is first determined whether or not the camera 10 is set to the shooting mode (S1). The camera 10 can be set in a shooting mode and a playback mode by a photographer. If the result of determination in step S1 is that a shooting mode has been selected, it is determined whether or not to perform shooting (S2). Whether or not to perform this shooting is determined by detecting whether or not the release button of the operation unit 4 has been pressed. In the present embodiment, when the release button is pressed once, moving image shooting starts, and when the release button is pressed again, moving image shooting ends.

  If the result of determination in step S <b> 2 is that there was no shooting, a field angle switching subroutine is executed (S <b> 3). This subroutine for changing the angle of view is the flowchart described in FIG. In the present embodiment, after the shooting mode is set, the view angle switching subroutine is executed before the shooting operation is started. However, the present invention is not limited to this, and the view angle switching subroutine is set before the shooting mode is set. May be executed.

  If the result of determination in step S2 is shooting, movie shooting and recording are started (S11). In this step, a moving image shooting operation by electronic zoom is performed based on the set focal length. That is, the wide-angle photographing unit 1a or the telephoto photographing unit 1b acquires the image data of the moving image of the subject image and records this image data in the image recording control unit 2. At the same time, surrounding sound is recorded by the recording unit 5.

  Subsequently, it is determined whether or not the image change is large (S12). This step is performed using the determination result of the blur determination unit 3, and whether the camera 10 performs an unstable movement, that is, the movement of the camera 10 does not move in one direction but moves irregularly. Determine if you have moved too fast. As a result of this determination, if the image change is not large, the process returns to step S11 to continue shooting and recording.

  Here, the determination of whether or not the image change in step S12 is large will be described using FIG. 9 and FIG. The moving image is constituted by a combination of continuous still images. In the example shown in FIG. 9, images are acquired continuously from the shooting point 21a and the shooting frames 21b to 21d. In this case, the shooting frame 21b and the shooting frame 21c, and the shooting frame 21c and the shooting frame 21d do not overlap. That is, the frames are compared from the stop shooting point, and when there is no overlapping image, for example, when it continues for 1 second or more, it is determined that the movement is uncomfortable as a too fast movement.

  Further, in the example shown in FIG. 10, the shooting points 22b, 22c, and 22d change from the stop point 22a with a large undulation. That is, when the motion vector of the image is determined, the image changes irregularly in the X direction and the Y direction. From the stop point 22a to the shooting frame 22b, the change in the X direction> the change in the Y direction, From the shooting frame 22b to the shooting frame 22c, the change in the X direction <the change in the Y direction. In such a case, the movement is determined to be unstable and uncomfortable. In detecting the locus of the image, the output is not limited to a motion vector, and for example, an output from a hand shake sensor or the like may be used.

  If the result of determination in step S12 is that the image change is large (see FIGS. 9 and 10), it is determined whether or not telephoto shooting is being performed (S13). This step is performed based on the currently set focal length of the electronic zoom. If the focal length is greater than or equal to a predetermined value, it is determined that the telephoto shooting is performed.

  If the result of determination in step S <b> 13 is not telephoto shooting, the process proceeds to step S <b> 15. On the other hand, if the result of determination is telephoto shooting, wide-angle image shooting and recording are performed (S14). In this step, instead of moving image shooting with the telephoto lens 9t and the image sensor 6t, switching to moving image shooting with the wide-angle lens 9w and the image sensor 6w is performed. Accordingly, as described with reference to FIG. 7, since it is possible to shoot in the wide-angle shooting ranges Fw1, Fw2, and Fw3, it is possible to prevent an unpleasant screen during viewing. Even when a wide-angle image is captured, the surrounding sound is recorded by the recording unit 5.

  In the present embodiment, during telephoto shooting, switching to wide-angle shooting is performed in step S14, and shooting is performed so as to display a wide-angle image as shown in FIG. However, the present invention is not limited to this. For example, as shown in FIG. 12B, two-image simultaneous recording may be performed so that two wide-angle and telephoto images can be displayed. Of course, at this time, the wide image may be enlarged, the disordered image may be made smaller, and the display may be devised so that the eyes are not tired. In addition, this wide-angle shooting may not be completely set to the wide-angle side, but the angle of view may be switched according to the speed. In this case, it is only necessary to detect whether a common part is generated on the two screens and to control the angle of view so that the common part remains.

  Subsequently, it is determined whether or not the photographing end operation is performed (S15). In this step, it is determined whether or not the release button has been operated again. If the result of this determination is that the operation has not ended, it is determined whether or not the set angle of view image change is small (S21).

  If the result of determination in step S21 is that the change in image at the set angle of view is not small, the flow returns to step S11 to shoot a moving image without changing the angle of view and record the image data. continue. On the other hand, when the image change at the set angle of view becomes small, shooting and recording are performed at the set angle of view (S22). That is, in step S14, the moving image shooting is continued by returning to the set angle of view before switching to the wide-angle image.

  When shooting and recording are performed at the set angle of view, a detail routine for changing the angle of view is executed similarly to step S3 (S23). That is, the angle of view is switched by the electronic zoom even during the shooting operation. When the subroutine for changing the angle of view is executed, the process returns to step S11 and the above-described operation is executed.

  If the result of determination in step S15 is that a shooting end operation has been made, a shooting end operation is performed (S16). That is, the moving image shooting operation ends. When the photographing operation is finished, the operation is finished, and the processing is executed again from step S1.

  If the result of determination in step S1 is not shooting mode, it is playback mode. In this case, it is determined whether or not a playback image has been selected (S31). In this step, it is determined whether or not a reproduction image is selected from a plurality of moving images by a reproduction operation member (not shown) of the operation unit 4. If the result of this determination is that a replay image has not been selected, this flow is terminated and executed again from step S1.

  If the result of determination in step S3 is that a playback image has been selected, it is determined whether or not playback is to be performed (S32). If the result of this determination is not playback, processing returns to step S31. On the other hand, in the case of reproduction, the image is reproduced (S33). In this step, the moving image recorded in the image recording control unit 2 is reproduced.

  When shooting a moving image from step S11 to step S23, if the image change is large, the image is recorded by the wide-angle lens 9t with less blurring. Therefore, the image reproduction in step S33 may have a large image change. There is little possibility of giving an unpleasant feeling. When the image reproduction is finished, the process is finished, and the process is executed again from step S1.

  As described above, in this embodiment, it is determined whether or not the motion of the image is unstable during shooting (see S12). If the motion of the image is unstable, a wide-angle image is shot and recorded. Yes. For this reason, when an image is reproduced, an unpleasant image is reduced from being reproduced and displayed, and an easy-to-view moving image can be captured. In the present embodiment, since images at two angles of view are switched and photographed at important points, it is possible to automatically change the screen and to prevent a sluggish video. it can.

  Next, a second embodiment of the present invention will be described with reference to FIGS. In the first embodiment of the present invention, when the movement of an image becomes unstable at the time of shooting, and in the case of telephoto shooting, recording is performed by switching to wide-angle shooting. In the second embodiment, both telephoto and wide-angle images are recorded, it is determined whether or not there is an unstable movement in the image during playback, and the image to be displayed is switched based on the determination result. Yes.

  The configuration of the second embodiment is the same as the configuration of the first embodiment shown in FIG. 1, and only the camera control flow shown in FIG. 8 is replaced with the flowchart shown in FIG. Will be described.

  When the flow shown in FIG. 12 is entered, it is determined whether or not the photographing mode is set, as in the first embodiment (S1). If the result of this determination is that a shooting mode has been selected, it is determined whether or not to perform shooting (S2). If the result of this determination is that shooting is not to be performed, a field angle switching subroutine is executed (S3), and upon execution, the flow returns to step S1.

  If the result of determination in step S2 is that photographing is to be carried out, photographing and image recording are carried out (S11). In this step, telephoto shooting is performed by the telephoto lens 9t and the image sensor 6t, and image data is recorded. Subsequently, wide-angle image shooting and recording are performed (S51). In this step, wide-angle shooting is performed by the wide-angle lens 9w and the image sensor 6w, and image data is recorded. In step S11 and step S51, telephoto image shooting and wide-angle image shooting are performed, and each image data is recorded in the image recording control unit 2. Note that the focal length set by the electronic zoom is also recorded when the image data is recorded in steps S11 and S51.

  Subsequently, similarly to step S15, it is determined whether or not the operation is finished (S52). If the result of this determination is that a shooting end operation has not been performed, an angle-of-view switching subroutine is executed as in step S3 (S56). This is to cope with a zooming operation during shooting.

  If the result of determination in step S52 is that shooting has ended, a shooting end operation is performed as in step S16 (S25). When the photographing end operation is performed, the operation ends, and the process is executed again from step S1. Thus, at the time of shooting, in steps S11 and S51, shooting of a telephoto image and a wide-angle image is simultaneously recorded until it is determined that shooting is finished.

  If the result of determination in step S <b> 1 is not shooting mode, it is determined whether or not a playback image has been selected as in step S <b> 31 (S <b> 41). If the result of this determination is that it has not been selected, processing returns to step S1. On the other hand, if it has been selected, it is determined whether or not the reproduction is started (S42), as in step S32. If the result of this determination is that playback has not started, processing returns to step S41.

  On the other hand, if the result of determination in step S42 is that playback has started, playback of the set image is started (S43). As described above, both the telephoto image and the wide-angle image are recorded in the image recording control unit 2, and an image corresponding to the focal length set as the electronic zoom is read out from this, and this is displayed on the display unit. 8 is reproduced and displayed.

  Subsequently, it is determined whether the change in the setting image is irregular or large (S44). In this step, it is determined whether or not there is an irregular movement or a change in speed such that a change in the front image impedes viewing.

  Here, determination of whether or not the moving image recorded using FIG. 13 and FIG. 14 moves in an unstable manner will be described. As described above, a moving image is composed of a combination of continuous still images. In the example shown in FIG. 13, the shooting frames 23b to 23d are continuously recorded between the first stop shooting point 23a that has been stopped for one second or more and the second stop point 23e that has been stopped for one second or more. Has been. In this case, the stop point 21a and the shooting frame 21b, the shooting frame 21b and the shooting frame 21c, the shooting frame 21c and the shooting frame 21d, and the like do not overlap. That is, the frames are compared from the stop shooting point, and when there is no overlapping image, for example, when it continues for 1 second or more, it is determined that the movement is uncomfortable as a too fast movement.

  In the example shown in FIG. 14, the imaging frames 24b and 24c have a large undulation between the first stop point 24a that has been stopped for 1 second and the second stop point 24d that has been stopped for 1 second. It has changed. That is, the motion vector of the image is determined, and the waviness width W is more than half of the straight line distance L between the first stop point 24a and the second stop point 24d instead of moving in a straight line. In such a case, it is determined that the movement is unpleasant as an unstable movement. In detecting the locus of the image, the output is not limited to a motion vector, and for example, an output from a hand shake sensor or the like may be used.

  If the result of determination in step S44 is that the change in the setting image is irregular or large, it is determined whether or not this setting image is telephoto shooting (S45). If the result of this determination is telephoto shooting, the wide-angle image is reproduced for a predetermined time (S46). In this step, the image data of the wide-angle image recorded in step S51 is read and reproduced and displayed for a predetermined time. The predetermined time is about 1 to 3 seconds, but if it is too short, the image frequently changes and is troublesome. On the other hand, if it is too long, reproduction of the image intended by the photographer may be omitted. Allow selection.

  When the wide-angle image is reproduced in step S46, or if the result of determination in step S44 is that the image change of the setting image is irregular or not significantly changed, or the result of determination in step S45 is that there is no telephoto shooting. If YES in step S47, it is determined whether or not the reproduction has ended (S47). If the result of this determination is that playback has not ended, it is determined whether or not the change in the set image is irregular or large (S48).

  If the result of determination in step S48 is that the change in the setting image is irregular or large, the process returns to step S46, and reproduction and display of the wide-angle image is continued. On the other hand, when the change in the setting image is not irregular or large, the movement of the setting image is not unstable, and the process returns to step S43 to reproduce and display the setting image.

  If the result of determination in step S47 is that reproduction has been completed, the reproduction operation is terminated (S49), this flow is terminated, and execution is repeated from step S1.

  Thus, in the second embodiment of the present invention, at the time of shooting, both the telephoto image and the wide-angle image are recorded simultaneously, and at the time of playback, the set image is reproduced and displayed. If the setting image is a telephoto image, it is determined whether the set image is a telephoto image. If the set image is a telephoto image, it is switched to a wide-angle image and displayed. . For this reason, even if the telephoto image moves in an unstable manner, it is possible to view a stable image because it can be switched to a wide-angle image, and to prevent the viewer from becoming uncomfortable with the unstable motion. Can do. In addition, since the telephoto image and the wide-angle image are appropriately switched and reproduced and displayed, it is possible to reproduce with a variety of changes.

  In this embodiment, the image is automatically switched when an unstable movement is made. However, in such a case, if it is desired to view the telephoto image, the image is manually returned to the telephoto image. Also good. Further, as shown in FIG. 11B, a two-screen display may be used, and a telephoto image may be displayed on a small screen and a wide-angle image may be displayed on a large screen.

  As described above, in each embodiment of the present invention, it is determined whether or not the motion of the setting image is unstable. If it is unstable, the setting image is switched to a wide-angle image. For this reason, it is possible to easily shoot difficult moving images, to easily change monotonous moving image shooting, and to shoot moving images that can be enjoyed for viewing. In addition, if a moving image is shot, image disturbance caused by moving the camera is compensated and reduced by an undisturbed image, so that even an amateur can take an image that is easy to see.

  In each embodiment of the present invention, the effective range of image data captured by at least two fixed focus lenses is changed in steps, thereby realizing high-magnification electronic zoom. In particular, in the imaging devices 6t and 6w in which an image is formed by the first and second fixed focus lenses (the telephoto lens 9t and the wide-angle lens 9w), the distance between the focal lengths is not overlapped. The zoom magnification can be set to a magnification.

  In each embodiment of the present invention, an example is shown in which shooting is performed with the camera 10 and playback is performed. However, the present invention is not limited to this, and playback can also be performed on a personal computer, a television, or the like. In the case of enjoying with a personal computer, a TV, or the like, image data of a moving image taken with the camera 10 may be stored.

  In each embodiment of the present invention, the camera 10 was shot and played back with a moving image. However, it is a matter of course that a still image shooting and playback display function may be provided as in a normal camera. Furthermore, in each embodiment of the present invention, the telephoto photographing unit and the wide-angle photographing unit have been described mainly with a type having a lens, but an image with a large angle of view is obtained from an image sensor formed by one lens. You may do it. Also in this case, when the captured image in a narrow range is disturbed, the effective range of the image sensor is widened to facilitate difficult video shooting and monotonous video as in the embodiments of the present invention. You can shoot videos that make it easy to change the shooting and endure viewing.

  Furthermore, in each embodiment of the present invention, a digital camera has been described as an apparatus for photographing. However, the camera may be a digital single lens reflex camera or a compact digital camera, such as a video camera or a movie camera. A camera for moving images may be used, and a camera built in a mobile phone, a personal digital assistant (PDA), or the like may of course be used.

  The present invention is not limited to the above-described embodiments as they are, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. Moreover, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above embodiments. For example, you may delete some components of all the components shown by embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

It is a block diagram which shows the structure of the camera concerning 1st Embodiment of this invention. 2A and 2B are side views of a wide-angle photographing unit and a telephoto photographing unit in the camera according to the first embodiment of the present invention, in which FIG. 1A is one example and FIG. 2B is another example. In the first embodiment of the present invention, it is a diagram showing a state of shooting a building or the like that is a subject, (a) (b) (c) (d) in order to zoom up and down the subject sequentially (C) shows a state where the image is unstable due to zooming up, and (e) shows a wide-angle image at the same time as (c). FIG. 2 is a diagram illustrating a configuration of a wide-angle / telephoto photographing unit in the first embodiment of the present invention, (A) illustrating a configuration of a wide-angle photographing unit, and (B) illustrating a configuration of a telephoto photographing unit. It is a figure which shows a mode that the picked-up image is reproduced and displayed on the television screen in 1st Embodiment of this invention. In the first embodiment of the present invention, it is a diagram for explaining the operation of angle of view switching, (A) is a flowchart showing a subroutine of angle of view switching, (B) is within the first limit and within the second limit. It is a figure explaining. It is a figure which shows a mode that a to-be-photographed object is image | photographed in a moving image in 1st Embodiment of this invention. It is a flowchart which shows the operation | movement of the camera control concerning 1st Embodiment of this invention. It is a figure explaining a quick movement in 1st Embodiment of this invention. It is a figure explaining the unstable movement in 1st Embodiment of this invention. In the first embodiment of the present invention, it is a diagram showing a state of reproduction display of a photographed image, (A) is a reproduction display of only one photographed image, and (B) is a state of reproducing and displaying two photographed images. FIG. It is a flowchart which shows the operation | movement of the camera control concerning 2nd Embodiment of this invention. It is a figure explaining a quick movement in 2nd Embodiment of this invention. It is a figure explaining the unstable movement in 2nd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1a ... Telephoto imaging part, 1b ... Wide-angle imaging | photography part, 2 ... Image recording control part, 2a ... Auxiliary recording part, 2b ... Auxiliary recording part, 3 ... Blur determination part, 4・ ・ ・ Operation unit, 5 ... Recording unit, 6 ... Image sensor, 6t ... Image sensor (telephoto), 6w ... Image sensor (wide angle), 7 ... Image composition unit, ..Display unit, 9 ... lens, 9t ... telephoto lens, 9w ... wide-angle lens, 10 ... camera, 15 ... TV screen, 16 ... shooting screen, 21a ... stop Shooting point, 21b ... Shooting frame, 21c ... Shooting frame, 21d ... Shooting frame, 22a ... Stop point, 22b ... Shooting frame, 22c ... Shooting frame, 22d ... Shooting Frame, 23a, first stop point, 23b, shooting frame, 23c, shooting frame, 23d - photographing frame, 23e ··· the second stopping point, 24a ··· first stopping point, 24b ··· photographic frame, 24c ··· photographic frame, 24d ··· second stop point

Claims (16)

An imaging unit that generates first image data representing a first image in a first imaging range, and second image data representing a second image in a second imaging range wider than the first imaging range;
An image data storage unit for storing the first image data and the second image data;
A determination unit for determining whether or not the movement of the first image data is unstable;
An image data synthesizing unit that complements the image data of the section determined to be unstable by the determination unit in the first image data with the second image data;
An imaging apparatus comprising:   The image data synthesizing unit stores the second image data corresponding to the section of the image data determined to be more unstable than the determination unit in the first image data in the image data storage unit. The imaging apparatus according to claim 1, wherein the imaging apparatus is controlled so as to perform.   The image data storage unit stores the first image data and the second image data in parallel, and the image data composition unit is unstable due to the determination unit of the first image data. The imaging apparatus according to claim 1, wherein the image data of the section determined to be replaced with the second image data corresponding to the section.   The imaging unit includes a first imaging unit including a first zoom unit, and a second imaging unit including a second zoom unit having a different angle of view from the first imaging unit. Item 2. The imaging device according to Item 1.   5. The imaging apparatus according to claim 4, wherein a shooting field angle adjustable by the first zoom unit and a shooting field angle adjustable by the second zoom unit do not overlap each other.   The imaging apparatus according to claim 1, wherein the determination unit determines that the movement of the first image is unstable when the first image is moving at a predetermined speed or more.   The imaging apparatus further includes a blur detection unit that detects a blur amount of the first image. When the determination unit determines that the blur amount of the first image is equal to or greater than a predetermined value, the movement of the first image is detected. The imaging apparatus according to claim 1, wherein the imaging apparatus is determined to be unstable. Generating first image data representing a first image in a first imaging range and second image data representing a second image in a second imaging range wider than the first imaging range;
Storing the first image data and the second image data;
Determining whether the movement of the first image is unstable;
Complementing the second image data with image data of a section determined to be unstable by the determination unit in the first image data;
An imaging method comprising: An image data storage unit for storing first image data representing a first image in the first photographing range and second image data representing a second image in the second photographing range wider than the first photographing range;
A determination unit for determining whether or not the movement of the first image is unstable;
An image data synthesizing unit that complements the image data of the section determined to be unstable by the determination unit in the first image data with the second image data;
An image processing apparatus comprising:   The image data synthesizing unit replaces image data of a section determined to be unstable by the determination unit in the first image data with the second image data corresponding to the section. Item 10. The image processing apparatus according to Item 9.   The image processing apparatus according to claim 9, wherein the determination unit determines that the movement of the first image is unstable when the first image is moving at a speed greater than or equal to a predetermined speed.   The image processing apparatus according to claim 9, wherein the determination unit determines that the movement of the first image is unstable when it is determined that a blur amount of the first image is greater than or equal to a predetermined value. . Storing first image data representing a first image in a first imaging range and second image data representing a second image in a second imaging range wider than the first imaging range;
Determining whether the movement of the first image is unstable;
Complementing the second image data with image data of a section determined to be unstable by the determination unit in the first image data;
An image processing method comprising: A first lens configured with a first fixed focal length;
A first imaging device disposed in the vicinity of the imaging surface of the first lens;
A first imaging unit that changes an effective range of image data of the first image sensor and performs electronic zooming;
A second lens configured with a second fixed focal length different from the first fixed focal length;
A second imaging element disposed in the vicinity of the imaging plane of the second lens;
A second imaging unit that changes the effective range of image data of the second image sensor and performs electronic zooming;
Comprising
An imaging apparatus, wherein a variable focal length range by the first imaging unit and a variable focal length range by the second imaging unit do not overlap.   The imaging apparatus according to claim 14, further comprising a zoom operation member, wherein the focal length is changed to a discrete focal length according to an operation of the zoom operation member.   The zoom operation member has a wide-angle zoom operation member that gives an instruction to the wide-angle side and a telephoto zoom operation member that gives an instruction to the telephoto side. The instruction is changed according to the instruction to the wide-angle side and the instruction to the telephoto side. The imaging apparatus according to claim 15, wherein the focal lengths are different.