JP2015156595A - Imaging apparatus, method for controlling the same, program, and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging apparatus which is an imaging apparatus capable of performing a zoom operation and a perspective operation and is advantageous for the reduction in size and cost of the imaging apparatus.

SOLUTION: The imaging apparatus includes: a plurality of imaging parts (101 and 102) different in viewpoint; instruction means (106b) for instructing the start of a focusing motion; an operation member (106a) for performing a first operation for changing the magnification of a specific object in a screen or a second operation for changing a distance up to the specific object, while keeping the magnification of the specific object constant; and control means (107) for performing control so that one of the first operation and the second operation is executed by operating the operation member, before the instruction from the instruction means and the other of the first operation and the second operation is executed by operating the operation member, after the instruction from the instruction means.

COPYRIGHT: (C)2015,JPO&INPIT

Description

  The present invention relates to an imaging apparatus capable of performing a zoom operation and a perspective operation.

  2. Description of the Related Art Conventionally, in camera photography, an operation called so-called perspective change is known in which the relationship between the main subject and the angle of view in which the background is reflected is changed by changing the subject distance and the focal length. For example, in movie shooting, a technique called perspective change shooting technique is known. This is because the distance to the main subject (subject distance) is changed by moving the camera on the rail, and at the same time, the focal length of the lens is changed so that the size of the main subject in the captured image remains unchanged. This is a technique for continuously changing the reflection.

  In recent years, there has been disclosed an imaging apparatus that captures a parallax image obtained by simultaneously acquiring two-dimensional intensity distribution of light in a subject space and angle information of light rays, that is, images with different viewpoints. By obtaining a parallax image, three-dimensional information of the subject space can be obtained. By performing image processing on this parallax image, it is possible to synthesize an image at a virtual viewpoint in which the subject distance is changed after shooting (Patent Document 1). With the technique of Patent Document 1, it is possible to change the perspective that has been performed while changing the position of the camera in the past by image composition processing after shooting.

US Patent Application Publication No. 2008/013019

  However, with the technique of changing the subject distance after shooting as in Patent Document 1, it is difficult to take a picture while grasping the image after the perspective change at the time of shooting. Even if an operation unit for performing a parsing operation is separately provided to synthesize an image in which the perspective is changed interactively and aiming while displaying on the display unit, the addition of the operation unit increases the size and cost of the imaging apparatus. It leads to up. Even in an imaging apparatus capable of acquiring subject space information from multi-viewpoint parallax images, zoom operations and focus operations are performed in the conventional manner, and it is not preferable that the number of operation units increase and the operations become complicated.

  Therefore, an object of the present invention is to provide an imaging apparatus capable of performing a zoom operation and a parsing operation, which is advantageous for downsizing and cost reduction of the imaging apparatus.

  An imaging apparatus according to an aspect of the present invention includes a plurality of imaging units with different viewpoints, an instruction unit for instructing start of a focusing operation, and a first operation for changing a magnification of a specific subject in a screen, or An operation member for performing a second operation for changing the distance to the specific subject while keeping the magnification of the specific subject constant, and the operation member is operated before an instruction from the instruction means To control that one of the first operation and the second operation is executed, and after the instruction from the instruction means, the operation member is operated to operate the first operation and the second operation. Control means for performing control so that the other operation of the second operations is executed.

  Other objects and features of the present invention are illustrated in the following examples.

  ADVANTAGE OF THE INVENTION According to this invention, it is an imaging device which can perform zoom operation and a perspective operation, Comprising: The imaging device advantageous to size reduction and cost reduction of this imaging device can be provided.

1 is a block diagram illustrating a system configuration of an imaging apparatus according to Embodiment 1. FIG. 1 is an external view of an image pickup apparatus according to Embodiment 1. FIG. It is a flowchart which shows the flow of the still image photography of this invention. It is a flowchart explaining the flow of the moving image photography of this invention. It is a figure explaining the image of the recorded image of this invention. It is a figure explaining a parallax image acquisition concept and image composition. It is a figure explaining the relationship of the virtual viewpoint position at the time of a parsing operation, a focal distance, and an angle of view. It is a figure explaining the change of the display image according to operation. 6 is a block diagram illustrating a system configuration of an imaging apparatus according to Embodiment 2. FIG. 6 is an external view of an imaging apparatus according to Embodiment 2. FIG. It is a figure explaining the picked-up image image at the time of zoom and perspective change.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  With reference to FIG. 1 to FIG. 11, an imaging device according to a first embodiment of the present invention, and an image processing device and a playback device included in the imaging device will be described.

  First, a photographed image image when the zoom change and the perspective change, which are the main parts of the present invention, are described with reference to FIG.

  FIG. 11A shows the change of the image due to the zoom change. Reference numeral 145 denotes a main subject, 148 denotes a background subject located behind the main subject, and 147 denotes a frame of the aiming image. In a general projection lens, when a zoom operation is performed, the subject in the image is enlarged or reduced at a substantially uniform magnification on the entire image surface about the optical axis. The scaling factor does not depend on the subject distance. By performing the zoom operation from the telephoto to the wide angle, the aiming frame is changed to 147A, 147B, and 147C. In the zoom operation from the image 147A to the wide-angle image 147B, the main subject is uniformly reduced from 145A to 145B and the background subject is reduced from 148A to 148B. Therefore, the central portion of the image acquired with the aiming frame 147C can be clipped to obtain the same angle of view of the image acquired with the aiming frame 147A.

  FIG. 11B shows a change in the image due to the perspective change. When the perspective change is performed, the main subject 145 and the background subject 148 are enlarged or reduced at different ratios. Conventional parsing operations are performed by changing the subject distance and changing the focal length. The main subject 145 and the background subject 148 with different original subject distances differ in the amount of change in the expected angle due to the subject distance change. Therefore, for example, if the subject distance and the focal length of the lens are adjusted so that the magnification of the main subject 145 does not change as shown in FIG. 11B, the magnification of only the background subject 148 changes. The aiming frame is changed to 147D, 147E, and 147F by performing the parsing operation from the state where the main subject distance is long and the lens is telephoto to the state where the main subject distance is short and the lens is wide angle. At this time, there is no change in the magnification of the main subject 145, but the background subject is reduced from 148D to 148F. In FIG. 11B, the reflection of the background subject 148 on the aiming frame 147 can be changed by performing a parsing operation so that the aiming frame changes from 147D to 147F. In the aiming frame 147D, only a part of the background subject 148D is included in the aiming frame, but in the aiming frame 147F, almost the entire background subject 148F is included in the aiming frame. From the above, it is not possible to clip the central portion of the image acquired with the aiming frame 147F and obtain the same angle of view of the image acquired with the aiming frame 147A.

  In an imaging device capable of acquiring a parallax image, the subject distance can be changed by image processing during or after shooting without moving the photographer in the direction of the subject. Therefore, by performing the change of the focal length and the image composition for changing the subject distance in conjunction with each other, it is possible to perform a parsing operation at the time of shooting or after shooting without moving the photographer. An object of the present invention is to achieve both zoom change and perspective change at the time of shooting with a simple operation in an imaging apparatus capable of acquiring this parallax image. In other words, an imaging apparatus capable of performing a zoom operation and a parsing operation, which is advantageous for downsizing and cost reduction of the imaging apparatus is provided.

  FIG. 1 is a block diagram illustrating a system configuration of the imaging apparatus according to the first embodiment of the present invention.

  Reference numeral 100 denotes a camera which is an imaging apparatus of the present invention. Reference numerals 101 a, 101 b, and 101 c are imaging optical systems capable of changing the focal length (so-called optical zooming), and constitute the imaging optical system group 101. Reference numerals 102 a, 102 b, and 102 c are image sensors that photoelectrically convert light beams guided from the plurality of imaging optical systems 101 a, 101 b, and 101 c, and constitute a plurality of image sensors 102. However, the imaging device of the present invention does not have to be plural, and may be a single imaging device as long as images formed by the imaging optical systems 101a to 101c can be acquired. Each image sensor 102a, 102b, 102c has a plurality of pixels. The plurality of imaging optical systems 101 and the plurality of imaging elements 102 constitute a plurality of imaging units with different viewpoints. Reference numeral 103 denotes a lens driving unit that drives the zoom lens included in the imaging optical system groups 101a to 101c and changes the focal length. In addition to zoom driving, the lens driving unit 103 also performs lens driving such as focus driving for driving a focus lens to focus on a subject. In this embodiment, the plurality of imaging optical systems are uniformly driven to change the focal length. The imaging optical system group 101, the plurality of imaging elements 102, and the lens driving unit 103 constitute a parallax image generating unit that generates a parallax image. An image processing unit 104 is an image synthesizing unit, and can synthesize an image obtained by synthesizing an acquired parallax image by giving a shift amount for each viewpoint, and can generate an image at a virtual viewpoint with a changed subject distance. . Reference numeral 105 denotes an image display unit which is an image display unit, and displays a generated image (composite image) generated by the image processing unit 104. Reference numeral 106 denotes an operation unit. The operation unit includes timing determining means for determining a switching timing between a zoom operation and a perspective operation, which will be described later, and an operation member that can be operated by switching between the zoom operation and the perspective operation. Reference numeral 107 denotes a camera system control unit (control unit) that controls the entire camera system, and also serves as a focal length control unit that sends an instruction to the lens driving unit to perform control. The camera system control unit 107 measures the subject using the parallax image transmitted from the image processing unit 104 in combination with the imaging optical system group 101, the plurality of imaging elements 102, and the image processing unit 104. A ranging means is configured. Since the method of distance measurement is not the main part of the present invention, the details are omitted, but a so-called phase difference method for evaluating the image shift amount for images of two or more different viewpoints, or a composition that gives a shift amount for each viewpoint. A so-called contrast method for evaluating the contrast of an image is used. With these distance measurement methods, distance measurement is performed on a subject in a region defined by an image. Reference numeral 108 denotes a memory unit that stores acquired images.

  FIG. 2 is an external view of the camera 100 that is the imaging apparatus according to the first embodiment of the present invention.

  FIG. 2A shows the front surface of the camera 100. A plurality of optical systems (imaging optical system groups) 101 capable of optical zoom are disposed on the front surface of the camera 100. Reference numeral 106a denotes an operation unit, which is a rotation ring input unit serving as an operation member, and a rotation operation input in the direction of an arrow 120 is possible. By rotating the rotary ring input unit 106a, a zoom operation can be performed, and the operation can be switched to a parse operation at the timing of a switch S1 described later. One of the operation units 106b is an automatic return type push button switch, which is a timing determination means (instruction means) for determining the switching timing between the zoom operation and the perspective operation. The push button switch 106b is operated with a finger or the like. When the push button switch 106b is lightly pushed, the switch S1 is turned on, and when pushed completely, the switch S2 is turned on. When the finger to be pressed is released, the state before the pressing is restored, and each of the switch S1 and the switch S2 is turned OFF. By turning on the switch S1, the start of the focusing operation is instructed, and the switching timing between the zoom operation and the perspective operation is determined. The switch S2 is turned on to start and end still image capturing and moving image capturing.

  FIG. 2B shows the back of the camera 100. Reference numeral 105 denotes a display unit which is an image display unit provided with a touch center 106c, which is one of the operation units, on the surface, and is configured by a flat panel display in this embodiment. The flat panel display 105 updates an image that changes due to a zoom operation and a perspective operation, which will be described later, as a display image as needed, thereby enabling interactive display for the operation. The touch sensor 106c receives a single touch input that is touched and released with one finger, or a pinch input that is touched with two fingers and slides the surface like an arrow 121 as shown in FIG.

  What has been described so far is an example of a suitable operating method, but other examples are also conceivable. For example, the switching between the parsing operation and the zooming operation by the push button switch 106b described above may be performed by an input to the touch sensor 106c (instruction means). Further, the input of the parsing operation and the zoom operation by the rotating ring input unit 106a may be performed by the pinch operation shown in FIG. 2C on the touch sensor 106c (operation member). Also in these methods, a simple operation can be realized while suppressing an increase in size and cost of the imaging apparatus.

  FIG. 3 is a flowchart showing the flow of still image shooting according to the present invention.

  The flowchart process starts when the power is turned on. First, in step S101, the initial setting is read. Here, the current focal length of the imaging optical system group 101 is read, and for example, the virtual viewpoint position when no shift amount is given to each viewpoint of the parallax image is read as an initial value.

  Next, in step S102, a conditional branch is made as to whether the switch S1 is ON. Here, the timing at which the switch S1 is turned on is determined by the input to the operation unit 106b, which is the timing determining means as described above.

  When the switch S1 is ON, the process proceeds to step S103, and autofocus processing is performed. In the autofocus process, the detection result of the distance measuring means described above is converted into a drive amount of the focus lens, and the focus lens is driven to focus on the subject in the defined area of the image.

  In step S104, conditional branching is performed as to whether or not the mode is a parse change prohibition mode. The setting of the perspective change prohibition mode is performed by an input operation (not shown). Details will be described later.

  In the case of the perspective change prohibition mode, the following steps S105 and S106 are skipped.

  When the mode is not the perspective change prohibition mode, the process proceeds to step S105, and the reference subject is determined. The reference subject (specific subject) is a subject whose image magnification is not changed when the perspective is changed, and needs to be determined when the perspective operation is performed. At the time of step S105, the photographer operates to focus on the subject (specific subject) in the area defined by the image. For this reason, it is convenient to determine the focused subject as a reference subject (specific subject). As the timing for switching to the perspective operation, the autofocus timing at which a subject suitable for the reference subject is determined is suitable.

  Next, in step S106, the parse processing flag is turned ON. With this parsing processing flag, in the conditional branch in the subsequent steps, the input to the operation member is divided into the zoom operation and the parsing operation.

  Returning to the conditional branch in step S102, the case where the switch S1 is not ON will be described. If the switch S1 is not ON, the process proceeds to step S107. In step S107, the parsing processing flag is returned to OFF because the parsing processing flag remains ON.

  Next, the process proceeds to step S108. In step S108, conditional branching is performed as to whether or not there is an input to the rotary ring input unit 106a, which is the operation member. If there is an input to the operation member, the process proceeds to step S109.

  In step S109, a conditional branch is performed as to whether or not the above-described parse processing flag is ON. Based on the parsing processing flag, the input to the operation member is switched to step S110 which is a zoom change instruction operation or step S111 which is a perspective change instruction operation. At the timing when the parsing process flag is turned on in step S106, the condition in step S109 is switched. Therefore, at the timing determined by the timing determining means, the input to the operation member is switched between the zoom change process and the perspective change process.

  In step S110 when the parsing processing flag is OFF, zoom change processing is performed. The zoom change process is a change in focal length, and the input to the operation member is distributed to the instruction operation to the focal length control means. In this embodiment, the focal length is changed by driving the zoom lens as described above.

  In step S111 when the parse change flag is ON, a parse change process is performed. As will be described in detail later, the perspective change process is performed in conjunction with the change of the focal length and the image composition for changing the subject distance. Therefore, the input to the operation member is distributed to an instruction operation in which the focal length control unit and the image synthesis unit are linked. The focal length is changed according to the interlocking operation, and shift amount information for each parallax of the parallax image for changing the subject distance described later is generated. In subsequent steps, image composition is performed based on this information. This image composition completes the perspective change.

  Next, in step S112, image composition based on the change processing in step S110 and step S111 is performed. The perspective change process based on step S111 is as described above. In this embodiment, since zoom change is performed by driving a zoom lens, there is no need for image composition based on step S110. However, when the lenses constituting the imaging optical system group are single focal lenses that cannot change the focal length, it is conceivable that the zoom is changed by image enlargement / reduction processing. In this case, the image enlargement / reduction amount may be determined in step S110, and image composition may be performed in step S112.

  In step S113, the image displayed on the image display unit is updated. The image updated here is the image synthesized in step S112. By updating the display image, the image displayed on the display unit 105 configured by the flat panel display in FIG. 2 is interactively changed according to the input to the operation member. In addition, even when there is no input to the operation member and the conditional branch in step S108 is NO, the display image is updated in step S113 so that the change in the image due to movement of the photographer or the subject is reflected.

  Next, in step S114, a conditional branch is made as to whether the switch S2 is ON. As described above, the switch S2 is turned on by completely pushing the operation unit 106b.

  If step S114 is ON, the process proceeds to step S115.

  In step S115, shooting is performed.

  In step S116, the photographed image is recorded. Details of the image recording will be described later with reference to FIG.

  If the switch S2 is not ON in step S114, step S115 and step S116 are skipped.

  Next, in step S117, a conditional branch is made as to whether the power is off.

  If the power is not turned off, the step returns to immediately before step S102, and the main part of the flowchart is repeated. Thus, while the power is on, image composition and image display according to the flowchart are interactively executed in accordance with the input to the rotary ring input unit 106a and the push button switch 106b.

  If the power is turned off, the flowchart ends.

  With the above control flowchart, it is possible to perform the aiming operation while switching between the zoom operation and the perspective operation during shooting.

  Further, in the present embodiment, as a preferred example, switching from the zoom operation to the perspective operation is shown. However, the switching from the perspective operation to the zoom operation may be made possible by a user setting. In this case, it is necessary to separately set a step for determining the reference subject. For example, a specific position in the image to be focused by reading the initial setting in step S101 is also read at the same time, and the subject located at the specific position is set as the reference subject. Etc.

  Depending on the photographer, it may be possible to perform a zoom operation to focus on the subject and then perform a further zoom operation. Therefore, a perspective change prohibiting mode that does not switch from the zoom operation to the perspective operation can be selected by the user setting. For example, it is conceivable to allocate an operation unit for input to the touch sensor 106 c on the flat panel display 105. In the parse change prohibition mode, step S106 for turning on the parse processing flag is skipped as described above, and the process does not proceed to the parse processing step.

  FIG. 4 is a flowchart showing the flow of moving image shooting in this case. Since many steps are common to the still image shooting flowchart shown in FIG. 3, the same reference numerals are given to the common flows, and only the differences will be described.

  In moving image shooting, the shooting start moving image based on the switch S2 is different from the still image shooting flowchart.

  In step S214, a conditional branch is made as to whether the switch S2 is ON. If the switch S2 is ON, the process proceeds to step S215.

  In the switch S215, a conditional branch is made as to whether or not moving image shooting is already in progress. If moving image shooting is in progress (Yes in step S215), the process proceeds to step S217 to end moving image shooting. If not moving image shooting is in progress (No in step S215), the process proceeds to step S216 to start moving image shooting.

  If the switch S2 is not ON in step S214, steps S215, S216, and S217 are skipped.

  Next, in step S218, conditional branching is again performed as to whether or not moving image shooting is in progress. If the moving image is being shot, the image is recorded in step S219. If the moving image is not being shot, the image is not recorded.

  As described above, according to the control flowchart of FIG.

  The image of the recorded image of the present invention will be described with reference to FIG. FIG. 5A shows the case of a still image, and FIG. 5B shows the case of a moving image.

  The description starts with a still image. Reference numeral 301 denotes all images acquired by the plurality of image sensors 102. On the other hand, reference numeral 302 denotes a synthesized image synthesized at the virtual viewpoint. As will be described later, the composite image 302 is determined by shooting conditions such as the angle of view and focal length of images acquired by the plurality of image sensors 102 and image composition conditions such as the virtual viewpoint position. It is possible to obtain various composite images 302 by combining these imaging conditions and image composition conditions. However, in the past, it is not known what kind of composite image 302 is finally obtained until composition after shooting is performed. It was. In the present invention, the photographer can grasp the composite image 302 at the time of photographing by synthesizing the image interactively according to the operation and displaying it on the display unit.

  If the resultant composite image 302 cannot be grasped at the time of shooting, the entire image 301 needs to be recorded. However, when the number of the imaging optical system group 101 and the plurality of imaging elements 102 to be arranged is large, the recording capacity of all the images 301 is increased. On the other hand, in the present invention, it is possible to take a picture while confirming whether the image is a desired one at the time of photographing. Therefore, by recording only the composite image 302 that is a photographer's desired image in the memory unit 108, the capacity of the recorded image can be reduced.

  Furthermore, in order to acquire only the composite image 302, the signal acquired from the image sensor 102 can be limited. In that case, the power required for signal processing and image processing can be reduced.

  Next, the case of a moving image will be described. Reference numerals 301a to 301c denote all images acquired in time series. Reference numerals 302a to 302c denote composite images in time series. In the case of moving image shooting, if all the images 301a having a large recording capacity are recorded as 301b and 301c in time series, the recording capacity becomes larger than that of a still image. Therefore, it is more preferable to record only the composite image 302 confirmed on the display unit 105 at the time of shooting in the moving image.

  The concept of acquiring a parallax image using the optical system group of the present invention will be described with reference to FIG. The relationship between the image composition given the shift amount by the image composition means and the subject distance will also be described.

  FIG. 6A schematically shows light rays viewed from the upper surface or the side surface. The explanation here is valid from both the top and side. Reference numeral 101 denotes a plurality of coupling optical systems, and reference numeral 102 denotes a plurality of imaging elements. Reference numeral 140 denotes a pixel included in each image sensor. Here, for easy understanding, the number of divisions is roughly divided into five. Reference numeral 143 denotes chief rays incident on the imaging optical system at respective angles. A chief ray 143 indicated by a solid line enters the imaging optical system located at the center of the plurality of imaging optical systems 101 and enters each of the five pixels 140 of the corresponding imaging element among the plurality of imaging elements 102. doing. The relationship between the light beam 143 and the pixel 140 is established for each imaging optical system and imaging element.

  Next, the relationship between the light rays and the pixels 140 among the plurality of image sensors 102 will be described with reference to FIG.

  In FIG. 6B, the pixel of interest in each image sensor is shifted. For the image sensor located at the center, the pixel 140i-3 located at the center, for the image sensor one level above the center, the pixel 140h-2 shifted one level up from the center, and for the image sensor one level above Pay attention to the shifted pixel 140g-1 which is one level higher. With regard to the image sensor located below the center, attention is paid to each pixel shifted by the same rule. The chief rays incident on these pixels of interest are indicated by solid lines. At this time, the plurality of chief rays intersect at one point. Now, consider placing a virtual imaging optical system 101x at a point where the principal rays intersect. Then, it can be seen that the chief rays incident on these different image sensors coincide with the chief rays incident on the imaging optical system when the imaging optical system is located at the position 101x. That is, an image acquired when the imaging optical system located at 101x has the image sensor 102x can be virtually generated using a signal acquired by the target pixel on which these principal rays are incident. In FIG. 6B, different shift amounts are given to pixels of different image sensors for each image height, and 140g-1, 140h-2, 140i-3, 140j-4, and 140k-5 are arranged in order from the top and synthesized. . By doing so, a signal acquired by the element 140x included in the imaging element 102x can be generated.

  Different imaging optical systems have different viewpoints. It can be seen that parallax images acquired by the plurality of imaging optical systems 101 as a whole can be combined at different virtual viewpoints by applying different shift amounts to different viewpoints and combining them at a virtual viewpoint. In the example shown in FIG. 6B, the virtual imaging optical system 101x is shifted in the optical axis direction with respect to the actual imaging optical system 101, which corresponds to changing the subject distance. ing.

  In the above example, every time the viewpoint (imaging device) is shifted from the center by one, the target pixel is shifted up by one. Here, it is considered that when the viewpoint (imaging device) is shifted up by one, the pixel of interest is shifted up by two. At this time, the target pixel is 140h-1. The principal ray 143y incident on 140h-1 is incident on the imaging optical system at a steeper angle than the principal ray incident on 140h-2. The subject distance to be changed is different. As can be seen from this example, the position of the virtual viewpoint can be changed and the subject distance at the virtual viewpoint can be changed by changing the shift amount at the time of synthesis for each viewpoint.

  In FIG. 6, for easy explanation, the number of pixels is roughly divided into five. Therefore, although the position of the virtual viewpoint that can be set is skipped, the virtual viewpoint position can be set at a finer pitch with respect to the number of pixels of a general image sensor, and the focal length can be changed finely.

  In the present invention, a shift amount is given for each parallax of a parallax image acquired based on this relationship, and image synthesis is performed at a virtual viewpoint in which the object distance is changed. Then, the perspective change is performed in conjunction with the change of the focal length.

  The relationship between the subject distance, the focal length, and the angle of view when performing a parsing operation will be described with reference to FIG.

  In FIG. 7, reference numeral 101 denotes an optical system group that acquires a parallax image, 145 denotes a main subject, and 148 denotes a background subject located behind the main subject. Reference numeral 149 denotes an interval (distance) between the main subject 145 and the background subject 148. As described with reference to FIG. 6, it is possible to perform composition by virtually changing the subject distance from the acquired parallax image. The position 140a in FIG. 7A and the position 140b in FIG. 7B indicate the position of the virtual viewpoint at the set subject distance. Reference numerals 146a and 146b denote subject distances, and 146a is shorter than 146b. Reference numerals 141a and 141b indicate the angles of light rays acquired at respective subject distances. The angle of the light beam shown here changes according to the change in focal length accompanying the driving of the zoom lens of the imaging optical system. Reference numeral 147 denotes the width of the light beam incident at each subject distance at the position of the main subject 145. The operation from FIG. 7A to FIG. 7B is a parsing operation, and it is necessary to adjust so that the magnification of the main subject 145 does not change in the images acquired at the subject distances 140a and 140b. . Therefore, when changing the subject distance, the focal length is also changed in association with the operation so that the light beam width 147 is kept equal by the operation from FIG. 7A to FIG. 7B. As described above, when the subject distance is changed without changing the magnification of the main subject 145, the background subject 148 is entirely included in the angle 141a as shown in FIG. In this way, only a part of the angle 141b is included. At this time, an image acquired at the subject distance 146a in FIG. 7A is as shown in FIG. 7C. Further, the image image acquired at the position of the subject distance 146b in FIG. 7B is as shown in FIG. In this way, the change of the subject distance and the change of the focal length are performed in conjunction with each other so that the subject magnification is kept equal, thereby realizing the perspective change.

  In the description of FIG. 7, it has been described that the angle of the light beam changes according to the change in the focal length. However, even if the angle of view is changed by enlarging or reducing the image, the angle of the light beam is changed. Therefore, in a system configured with an imaging optical system whose focal length is not variable, the image may be enlarged or reduced instead of changing the focal length.

  Next, changes in the display image according to the operation will be described with reference to FIG.

  In FIG. 8, reference numerals 145 and 148 denote the main subject and the background subject as before. Reference numeral 147 denotes an acquired image in each state when the zoom operation and the perspective operation are performed. Reference numeral 160 denotes a predetermined area of the image to be focused.

  First, an operation before the switch S1 is turned on (before an instruction from the instruction unit) will be described using images 147A, 147B, and 147C. Before the switch S1 is turned on, a zoom operation (the first operation for changing the magnification of the specific subject in the screen described above) is performed. By performing the zoom operation, the image continuously changes as it moves back and forth according to the operation of 147A, 147B, and 147C. At this time, the main subject (specific subject) 145 and the background subject 148 are uniformly enlarged and reduced.

  Next, an operation after the switch S1 is turned on (after an instruction from the instruction unit) will be described using images 147D, 147E, and 147F. After the switch S1 is turned on, a parsing operation (second operation for changing the subject distance to the specific subject while keeping the magnification of the specific subject in the screen constant) is performed. If the switch S1 is turned on when the display image during the zoom operation is 147A, the image in the initial state before the operation is 147D. By performing the parsing operation, the image can be continuously changed between 147D, 147E, and 147F. As shown in the figure, the image magnification of the main subject (specific subject) 145 does not change, and the reflection of the background subject 148 in the image changes.

  Note that, as described above, the present invention is not limited to only switching from the zoom operation to the perspective operation by the instruction unit, and conversely, switching from the perspective operation to the zoom operation is performed by the instruction unit. You can also That is, the control means of the present invention can be controlled so that one of the first operation and the second operation is executed by operating the operation member before the instruction from the instruction means. It may be configured. In addition, after the instruction from the instruction unit, the operation member may be operated so that the other operation of the first operation and the second operation can be performed.

  As described above, a simple operation can be realized while suppressing an increase in the size and cost of the imaging apparatus in the realization of the perspective operation at the time of imaging of the imaging apparatus capable of acquiring multi-viewpoint parallax images by the imaging apparatus having the configuration described above. .

  With reference to FIG. 9 and FIG. 10, an imaging device according to a second embodiment of the present invention, and an image processing device and a playback device included in the imaging device will be described. In addition, the same number is provided about the component similar to Example 1. FIG. In addition, since the contents not shown in the figure are common contents, the description is omitted.

  FIG. 9 is a block diagram illustrating a system configuration of the imaging apparatus according to the present embodiment.

  The system configuration in FIG. 9 is different from the system configuration in FIG. 1 in a plurality of optical systems (imaging optical system group). The imaging optical system group 101 in FIG. 1 is composed of a plurality of imaging optical systems with variable focal lengths, whereas the imaging optical system group 201 in FIG. 9 has a plurality of imaging optical systems with different focal lengths. Consists of. For example, in the imaging optical system group indicated by 201a, 201b, and 201c, the focal length becomes longer in the order of the imaging optical system 201a, the imaging optical system 201b, and the imaging optical system 201c. Although the focal lengths of the imaging optical system group 201 are different, they are different viewpoints and can constitute a parallax image generating means. Therefore, it is possible to synthesize an image at a virtual viewpoint as in the first embodiment by combining processing for correcting differences in focal length such as enlargement / reduction of an image as appropriate.

  The zoom operation in the present embodiment is performed by the same operation as in the first embodiment. In the first embodiment, after the zoom operation is performed, the lens driving unit 103 performs zoom driving of the imaging optical system group 101. In this embodiment, since a plurality of imaging optical systems having different focal lengths are provided, an image corresponding to zoom driving can be obtained without actually moving the lens. Although not described in detail because it is not a main part of the present invention, for example, the image pickup device that acquires an image to be output to the display unit 105 is changed from 102a to 102b in response to a zoom operation. In a practical system configuration, the focal length of the imaging optical system group 201 is not continuous but intermittent. Therefore, the focal length between the focal lengths included in the system is obtained by clipping from the image of the imaging element having the imaging optical system having the closest focal length shorter than the focal length.

  FIG. 10 is an external view of a camera 100 that is an imaging apparatus according to the second embodiment.

  The difference from the first embodiment is an imaging optical system group 201. A to D shown in the imaging optical system group 201 indicate focal lengths of the imaging optical system. In the imaging optical system 201 of the present embodiment shown in FIG. 10, the imaging optical systems indicated by A to D are configured to have a plurality (four each).

  As described above, in the realization of the parsing operation at the time of shooting of the imaging apparatus capable of acquiring a parallax image, a simple operation is realized while suppressing the increase in size and cost of the imaging apparatus.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

  Also, a software program in which a procedure for realizing the functions of the above-described embodiments is described is supplied from a recording medium directly to a system or apparatus having a computer that can execute the program using wired / wireless communication. Cases are also included in the present invention. Further, the present invention includes a case where the computer executes the program.

  Accordingly, the program code itself supplied and installed in the computer in order to implement the functional processing of the present invention by the computer also realizes the present invention. That is, the computer program itself for realizing the functional processing of the present invention is also included in the present invention.

  In this case, the program may be in any form as long as it has a program function, such as an object code, a program executed by an interpreter, or script data supplied to the OS.

  As a recording medium for supplying the program, for example, a magnetic recording medium such as a hard disk or a magnetic tape, an optical / magneto-optical storage medium, or a nonvolatile semiconductor memory may be used.

  As a program supply method, a computer program that forms the present invention is stored in a server on a computer network, and a connected client computer downloads and programs the computer program.

  The present invention can be suitably used for an imaging apparatus such as a compact digital camera, a single-lens reflex camera, and a video camera.

DESCRIPTION OF SYMBOLS 100 Camera 101 Imaging optical system 102 Image pick-up element 106 Operation part 107 Camera system control part

Claims (11)

A plurality of imaging units with different viewpoints;
Instruction means for instructing the start of a focusing operation;
An operation member for performing a first operation for changing the magnification of the specific subject in the screen, or a second operation for changing the distance to the specific subject while keeping the magnification of the specific subject constant;
Before the instruction from the instruction means, the operation member is operated so that one of the first operation and the second operation is executed,
After the instruction from the instruction means, a control means for controlling the other operation of the first operation and the second operation by the operation member being operated,
An imaging device comprising:   The imaging apparatus according to claim 1, wherein the plurality of imaging units includes an optical system having a variable focal length. The plurality of imaging units include a plurality of optical systems having different focal lengths,
The imaging apparatus according to claim 1, further comprising an image processing unit configured to perform enlargement / reduction processing of a plurality of parallax images obtained from the plurality of imaging units. Comprising image combining means for combining a plurality of parallax images obtained from the plurality of imaging units;
4. The image synthesizing unit according to any one of claims 1 to 3, wherein the image synthesizing unit synthesizes the parallax image with a shift amount for each viewpoint, and generates an image in which a distance to the specific subject is changed. The imaging apparatus according to item 1.   The imaging apparatus according to claim 4, further comprising a recording unit that records a synthesized image synthesized by the image synthesizing unit.   The imaging apparatus according to claim 1, further comprising a distance measuring unit that measures a subject distance according to an instruction from the instruction unit.   The imaging apparatus according to claim 1, wherein the operation member is a button switch.   The imaging apparatus according to claim 1, wherein the operation member is a touch sensor. A plurality of imaging units with different viewpoints, instruction means for instructing the start of a focusing operation, a first operation for changing the magnification of the specific subject in the screen, or while maintaining the magnification of the specific subject constant An operation member for performing a second operation for changing a distance to the specific subject, and a control method for the imaging apparatus,
A first control step for performing control so that one of the first operation and the second operation is executed by operating the operation member before an instruction from the instruction means;
After the instruction from the instruction means, a second control step for controlling the other operation of the first operation and the second operation by operating the operation member;
A method for controlling an imaging apparatus, comprising:   A computer-executable program in which a procedure of a control method for an imaging apparatus according to claim 9 is described.   A computer-readable storage medium storing a program for causing a computer to execute each step of the control method of the imaging apparatus according to claim 9.