JP2022048079A - Information processing device, information processing method, and program - Google Patents

Abstract

To provide an information processing device which performs gate control with a simple hardware configuration while a point of focus is set in an arbitrary region in an imaging region.SOLUTION: Gate control is performed by inclining an imaging element relative to a surface orthogonal to an optical axis of an imaging optical system. An object region is set in a subject image of the imaging element. The correction amount of a position of focus is calculated on the basis of a position of the object region according to the gate control. The position of focus is corrected on the bases of the correction amount.SELECTED DRAWING: Figure 4

Description

The present invention relates to an information processing apparatus, an information processing method, and a program.

A shine proof that changes the depth of field of the imaging scene by adjusting the focal plane by rotating the imaging surface so that it tilts (tilt control) with respect to the surface orthogonal to the optical axis of the imaging system that images the subject. A technique called the principle is known. According to this technology, it is possible to obtain an image in which the depth of field is shallow and the image other than the main subject is blurred, and by increasing the depth of field without narrowing the aperture, subject blurring or noise due to insufficient light amount is suppressed. While doing so, it is possible to obtain an image in which the entire imaging region is in focus.

Patent Document 1 discloses an image pickup apparatus that performs portrait photography in which a subject other than the main subject is blurred by performing tilt control after focusing on the main subject. Further, Patent Document 2 discloses a technique of rotating the image pickup surface around an in-focus point designated in the tilt control by adopting a configuration in which the image pickup surface is rotated around an arbitrary place as a rotation axis.

Japanese Unexamined Patent Publication No. 2008-205569 Japanese Unexamined Patent Publication No. 2018-056810

However, in the technique described in Patent Document 1, since the rotation axis of the tilt control is fixed and the focus is adjusted for the subject on the center line of the image sensor, the focus changes when the subject is not on the center line. And it becomes blurry. On the other hand, in the technique described in Patent Document 2, it is possible to focus on a designated in-focus point, but a complicated hardware configuration for operating an arbitrary place as a rotation axis is required.

An object of the present invention is to perform tilt control while focusing on an arbitrary region in the imaging region with a simple hardware configuration.

In order to achieve the object of the present invention, for example, the information processing apparatus according to one embodiment has the following configurations. That is, depending on the control means for tilting the image sensor with respect to the plane orthogonal to the optical axis of the image pickup optical system, the setting means for setting the target area in the subject image by the image sensor, and the tilt control. A calculation means for calculating the correction amount of the focus position based on the position of the target region and a correction means for correcting the focus position based on the correction amount are provided.

With a simple hardware configuration, it is possible to control the tilt while keeping the focus on any area within the imaging area.

The figure which shows an example of the functional structure of the information processing apparatus which concerns on Embodiment 1. FIG. The figure which shows an example of the correction of the focus position in the tilt control which concerns on Embodiment 1. FIG. The figure which shows an example of the correction of the focus position in the tilt control which concerns on Embodiment 1. FIG. The figure for demonstrating the calculation of the correction amount of the focus position which concerns on Embodiment 1. The graph which shows an example of the relationship between the tilt angle and the focus correction amount which concerns on Embodiment 1. The flowchart of the processing example in the information processing method which concerns on Embodiment 1. The flowchart of the process example of the tilt control which concerns on Embodiment 1. The flowchart of the processing example including the focus correction which concerns on Embodiment 1. The graph which shows an example of the relationship between the tilt angle and the focus correction amount which concerns on Embodiment 2. The flowchart of the processing example including the focus correction which concerns on Embodiment 2. The figure which shows the functional structure of the computer which concerns on Embodiment 4. The figure which shows the example of the captured image which captures two subjects which concerns on Embodiment 3. The flowchart of the processing example in the information processing method which concerns on Embodiment 3.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiment, not all of the plurality of features are essential for the invention, and the plurality of features may be arbitrarily combined. Further, in the attached drawings, the same or similar configurations are given the same reference numbers, and duplicate explanations are omitted.

[Embodiment 1]
The information processing apparatus according to this embodiment includes an image pickup device and an image pickup optical system. Further, in the information processing apparatus according to the present embodiment, the target area is set in the subject image by the image pickup element, and the target area is focused according to the tilt control that changes the relative posture between the image pickup element and the image pickup optical system. The position correction amount is calculated and the focus position is corrected.

FIG. 1 shows an example of the functional configuration of the information processing apparatus according to the present embodiment. Since the image pickup optical system of the information processing apparatus 100 according to the present embodiment forms a subject image by the same process as a general image pickup optical system, detailed description of each process will be omitted. The information processing apparatus 100 includes a zoom lens 101 that moves in the optical axis direction to change the focal length, a focus lens 102 that moves in the optical axis direction to adjust the focus position, and an aperture unit 103 that adjusts the amount of light. The light that has passed through the image pickup optical system forms a subject image as an optical image on the image pickup element 106 via the bandpass filter (BPF) 104 and the color filter 105. The BPF 104 may be able to move forward and backward with respect to the optical path of the imaging optical system.

The image sensor 106 photoelectrically converts the subject image. The analog electric signal (imaging signal) output from the image pickup element 106 is gain-adjusted by the AGC 107, converted into a digital signal by the AD converter 108, and then input to the signal processing unit 109. The signal processing unit 109 performs various image processing on the input digital signal to generate a video signal. Next, the signal processing unit 109 outputs a video signal to the monitoring device 111 connected to the information processing device 100 by wired or wireless communication via the communication unit 110. The monitoring device 111 outputs control signals such as commands to the area setting unit 112, the tilt control unit 114, or the focus control unit 117 in the information processing device 100 in response to the user's instruction, and the processing described later by each processing unit is performed. To do.

The area setting unit 112 sets a target area to be focused in the subject image on the image sensor based on the instruction from the communication unit 110. The target area is an area consisting of one or more pixels, and for example, an area in which a desired subject to be focused is captured is set. Details thereof will be described later with reference to the flowchart of FIG. The evaluation value calculation unit 113 acquires RGB pixel values or luminance values from the AD converter 108 or the signal processing unit 109 for the target area. Next, the evaluation value calculation unit 113 calculates the focus evaluation value used in the autofocus (AF) control. In the present embodiment, the focus evaluation value is an evaluation value related to the contrast of a specific frequency, but the present invention is not particularly limited to this, and may be a value indicating distance information due to, for example, a phase difference.

The tilt control unit 114 instructs the tilt drive unit 115 of the target position of the tilt control based on the instruction from the communication unit 110. In the present embodiment, the target position of the tilt control is represented by the rotation angle from the basic posture of the image sensor 106 or the image pickup optical system. Further, in the following, the control amount (rotation angle) by the tilt control from the basic posture to a certain posture of the image sensor 106 is referred to as a tilt angle. In the present embodiment, the following description will be made assuming that the basic posture of the main surface of the image pickup optical system and the basic posture of the image pickup element 106 are parallel to each other.

The tilt control according to the present embodiment is a control in which the in-focus surface is tilted by tilting the image pickup surface with respect to the plane orthogonal to the optical axis for imaging the subject. In the present embodiment, the tilt control is performed by rotationally driving the image sensor 106 or the image pickup optical system, and it is possible to focus on the entire subject on a plane such as the ground or a building. The tilt control is performed, for example, by the user operating an input device such as a button or a dial.

The tilt drive unit 115 tilts the image pickup device 106 or the image pickup optical system based on the target position instructed by the tilt control unit 114. Hereinafter, the tilting drive unit 115 shall tilt the image pickup device 106, and the operation thereof will be described simply as “tilt control”. Further, in the present embodiment, as shown in FIGS. 2 or 3 to be described later, the tilting drive unit 115 controls the central axis in the center of the image pickup element 106 as a rotation axis. Is not particularly limited to this position. In this example, the central axis of the image pickup device will be described as an axis connecting the midpoints of the two short sides of the image pickup device. In the following, for the sake of explanation, the tilt control shall be performed in the range of −90 ° to 90 ° from the basic posture.

As described above, in the present embodiment, the tilt control is performed by rotating the image pickup device 106, but the tilt control may be performed by rotating the image pickup optical system. In that case, the relationship between the position and orientation of the image pickup element 106 and the image pickup optical system is such that the image pickup element 106 is rotated in the direction opposite to the rotation direction of the image pickup optical system and the same rotation width with the rotation axis of the image pickup optical system as the center of rotation. It is the same as when it is rotated. Therefore, the value of the focus correction is also calculated by the method described later in the same manner as in that case.

The correction amount calculation unit 116 determines the correction amount (focus correction amount) of the focus correction performed by the focus control unit 117 when the tilt control is performed. The focus correction amount according to the present embodiment is a correction amount in the focus correction for maintaining the focus in the target area set by the area setting unit 112 according to the tilt control. The focus correction amount will be described later with reference to FIG. The focus control unit 117 designates the focus position with respect to the focus drive unit 118 based on the calculated focus correction amount or the instruction from the communication unit 110. The focus drive unit 118 controls the position of the focus lens 102 based on the focus position instructed by the focus control unit 117. Further, when the user gives an instruction to stop the tilt control, the correction amount calculation unit 116 recalculates the correction amount with the position corresponding to the stop instruction (posture at the time when the image sensor is stopped) as the target position. May be good.

[Aori control]
Hereinafter, the correction of the focus position according to the tilt control performed by the information processing apparatus according to the present embodiment will be described with reference to FIGS. 2 and 3. FIG. 2 is a schematic diagram showing an example of a state before focusing on a point A on a subject on the optical axis and performing tilt control. In the example of FIG. 2A, the focus is adjusted so that the subject distance L to the point A is in focus. Further, since the main surface of the optical system and the image pickup device are arranged in parallel, those surfaces and the focus surface are parallel to each other. Further, the point A exists on the rotation axis of the focus surface described later. Focusing means keeping the subject within the range of the depth of field determined by the setting of the allowable confusion circle of the imaging optical system, and in the present embodiment, the focus is basically achieved by AF. Although it is adjusted, the description is omitted because it is a general process.

FIG. 2B is a schematic diagram showing an example of a state after tilt control is performed with the central axis of the image sensor as the rotation axis. When tilt control is performed, the focus surface also rotates based on the Scheimpflug principle. Therefore, it is possible to focus on the entire range from a short distance to a long distance with respect to a subject on a certain plane by the tilt control. At this time, the focus surface rotates about the position corresponding to the tilt rotation axis (rotation axis of the image sensor). In the present embodiment, the axis corresponding to the tilt rotation axis of the focus surface is a straight line on the focus surface set at a position corresponding to the subject image on the tilt rotation axis by the image pickup element. In FIG. 2B, the focus surface is rotated by tilt control so that the focus surface matches the entire surface of the subject on the plane. Since the principle of Scheimpflug is a known principle, detailed explanation is omitted, but when the main surface of the optical system and the image pickup surface of the image pickup element intersect on one straight line, the focus plane is also the same straight line as that straight line. The principle is that they meet above. Assuming that the focal length is f, the subject distance is L, and the depression angle is α, the tilt angle b is expressed by the following equation (1) based on the Scheimpflug principle.
b = tan ^-1 (f / (Ltanα)) Equation (1)

When the tilt control is performed in the opposite direction to the example of FIG. 2, the focus surface also rotates in the opposite direction, so that the depth of field becomes shallow and the effect of blurring the part other than the point A on the subject more greatly. There is.

FIG. 3 is a schematic diagram showing another example of a state before focusing on a point B on a subject that is not on the optical axis and performing tilt control. In the example of FIG. 3A, the focus is adjusted so that the subject distance L to the point B is in focus. Further, since the main surface of the image pickup optical system and the image pickup element are arranged in parallel, those surfaces and the focus surface are parallel to each other. Further, the rotation axis of the focus surface corresponding to the tilt control passes through the point A in FIG. 2, and the point B is off the rotation axis. When the tilt control is performed in the same manner as in the example of FIG. 2, the focus surface rotates with the position corresponding to the rotation axis of the image sensor 106 as the rotation axis. Therefore, the point B of the subject that is not on the rotation axis of the focus surface is the focus surface. It deviates from the image and causes blurring.

FIG. 3B is a schematic diagram showing an example of a state after performing tilt control in the same manner as in FIG. 2B. As described above, in this example, the focus is out of focus from the point B, and it is preferable to perform focus correction in order to prevent blurring at this point B. FIG. 3C is a schematic diagram when the information processing apparatus 100 according to the present embodiment performs such focus correction. In the example of FIG. 3C, the focus plane is rotated while the focus is maintained at the point B by the focus correction.

Hereinafter, an example of calculating the focus correction amount for the focus position correction performed according to the tilt control will be described with reference to FIG. In FIG. 4, the area setting unit 112 sets the target area in the captured image. In this example, the area setting unit 112 sets the area representing the subject existing below the tilt rotation axis in the captured image as the target area. Next, the correction amount calculation unit 116 calculates the focus correction amount so that the target area is in focus according to the control amount of the tilt control. Here, the focus correction amount β is calculated according to the following equation (2). Here, it is assumed that the image sensor 106 is rotationally controlled from the tilt angle α1 to α2 by the tilt control, and k is the distance from the rotation axis on the image sensor 106 to the center pixel of the subject.
β = (tanα2-tanα1) × k equation (2)

FIG. 5 is a graph showing the relationship between the tilt angle and the focus correction amount β. In this example, the graph is not linear, and even when the same amount of tilt control is performed, the fluctuation amount of the focus correction amount changes according to the control amount with respect to the basic posture. In the present embodiment, the tilt control of the image sensor 106 is performed in the range of −90 ° to 90 ° from the basic posture. Therefore, the larger the absolute value of the control amount from the basic posture, the larger the focus correction amount due to the same control amount. Become.

The method for calculating the focus correction amount is not particularly limited to the equation (2). For example, the correction amount calculation unit 116 may calculate a simple correction amount by dividing β in the equation (2) by the sensitivity of the focus lens, and solves a high-order equation or a polynomial according to the sensitivity. By doing so, a more accurate correction amount may be calculated.

Hereinafter, the processing performed by the information processing apparatus 100 according to the present embodiment will be described with reference to FIG. FIG. 6 is a flowchart showing an example of the main processing performed by the information processing apparatus 100.

In step S601, the area setting unit 112 sets the target area. In the present embodiment, the area setting unit 112 sets an arbitrary area specified by the user as a target area. Further, the area setting unit 112 may automatically set the position detected by the function of detecting a specific object as the target area. For example, the area setting unit 112 may focus on the detected subject by the AF function and set the area indicating the subject as the target area.

In step S602, the focusing drive unit 118 performs focusing on the target area. The focus drive unit 118 may automatically focus by, for example, an AF function. Further, for example, the focus drive unit 118 may perform focusing according to the manual input of the user who visually observes the captured image by the manual focus (MF) function. In this case, the focus drive unit 118 may display information that assists the user, such as enlarging the target area for easy viewing or presenting the degree of focus of the target area to the user.

In step S603, the tilt control unit 114 determines whether or not there is an instruction for tilt control by the user. If there is an instruction for tilt control, the process proceeds to step S604, and if not, it is confirmed whether or not there is an instruction again.

In step S604, the correction amount calculation unit 116 calculates the distance from the rotation axis in the subject image by the image sensor to the target region. In the present embodiment, the correction amount calculation unit 116 calculates the distance from the rotation axis to the central pixel of the target area. In step S605, the tilt control unit 114 acquires the target position of the tilt control in the instruction confirmed in step S603. As the target position of the tilt control, only one point may be specified. For example, a grace range such as 1 ° in the front-rear direction is set, and if it is within that range, it may be regarded as the target position. In step S606, the correction amount calculation unit 116 determines whether or not the value of the distance from the rotation axis on the image sensor 106 to the subject is 0. If it is 0, the focus does not shift in the target area even if the tilt control is performed, and the process proceeds to step S607 assuming that the focus control does not need to be performed. If not, the process proceeds to step S608, assuming that focus control needs to be performed. In step S607, tilt control is performed, and in step S608, tilt control and focus control are performed. After step S607 or step S608, the process ends.

Hereinafter, the processes in steps S607 and S608 will be described with reference to FIGS. 7 and 8, respectively. FIG. 7 is a flowchart showing an example of detailed processing of tilt control performed in step S607.

In step S701, the tilt drive unit 115 starts tilt control toward the target position of the tilt control acquired in step S605. In step S702, the tilt drive unit 115 determines whether or not the tilt angle at that time has reached the target position. If it has been reached, the process ends, and if not, the process proceeds to step S703.

In step S703, the tilt control unit 114 determines whether or not there is a stop instruction by the user during the tilt drive. If there is a stop instruction, the process proceeds to step S704, and if not, the process returns to step S702. In step S704, the tilting drive unit 115 stops driving at a position corresponding to the stop instruction, and ends the process.

FIG. 8 is a flowchart showing an example of detailed processing of tilt control and focus control performed in step S608. In step S801, the tilt control unit 114 acquires the current tilt angle as the tilt start position. In step S802, the correction amount calculation unit 116 calculates the correction amount to be executed according to the tilt control and the focus target position according to the correction amount. In this embodiment, the correction amount is calculated by the above equation (2).

In step S803, the correction amount calculation unit 116 sets the focus drive speed for performing focus correction. In the present embodiment, the focus drive speed refers to the speed at which the position of the focus lens is controlled for focus correction. The correction amount calculation unit 116 may set the focus drive speed according to a predetermined control speed of the tilt control drive. If the tilt control and the focus control are not synchronized with each other, the effect of maintaining the focus in the focus area during the tilt drive is reduced. From this point of view, the correction amount calculation unit 116 according to the present embodiment sets the focus drive speed so that the focus drive time becomes equal to the total drive time of the tilt control. That is, the correction amount calculation unit 116 sets the focus drive speed so that the correction of the focus position is completed at the same time as the tilt control is completed. The tilt control drive time is calculated, for example, by dividing the tilt drive amount (difference between the tilt start position and the tilt target position) by the tilt drive speed (° / sec).

In the present embodiment, the tilting drive unit 115 controls tilting at a constant speed. The larger the absolute value of the tilt control amount from the basic posture, the larger the focus correction amount due to the same control amount. Therefore, in the correction amount calculation unit 116, the larger the absolute value of the tilt angle is within the range of 0 ° to 90 °. , The focus drive speed may also be set to be high. Further, the correction amount calculation unit may set the focus drive speed during the tilt speed according to the value of the absolute value of the tilt angle before the tilt control.

In step S804, the tilt drive unit 115 starts tilting drive toward the target position acquired in step S605, and the focus driving unit 118 starts controlling the position of the focus lens toward the focus target position calculated in step S802. .. In step S805, the tilt drive unit and the focus drive unit 118 determine whether the tilt angle and the focus position have reached the target positions, respectively. If both of them have reached the target position, the process ends, and if not, the process proceeds to step S806.

In step S806, the tilt control unit 114 determines whether or not there is a stop instruction by the user during the tilt drive. If there is a stop instruction, the process proceeds to step S807, and if not, the process returns to step S805. In step S807, the tilting drive unit 115 stops driving at a position corresponding to the stop instruction.

In step S808, the focus control unit 117 recalculates the focus correction amount and the focus target position corresponding to the position corresponding to the tilt drive stop instruction as the tilt target position. In step S809, the focus drive unit 118 controls the position of the focus lens to the recalculated focus target position, and ends the process.

According to such a process, it is possible to calculate a correction amount according to the position of the target region according to the tilt control and correct the focus position using the correction amount. Therefore, with a simple hardware configuration in which the rotation axis of the tilt control is fixed, it is possible to change the depth of field by the tilt control while keeping the focus on an arbitrary region in the imaging region. Further, by setting the drive times of the tilt control and the focus control to be the same, it is possible to continue focusing on the target area during one tilt drive.

[Embodiment 2]
In the information processing apparatus according to the first embodiment, the focus control is set so that the drive times of the tilt control and the focus control are the same. That is, the setting is made so that the focus control is performed once for the tilt control once. On the other hand, the information processing apparatus according to the present embodiment repeatedly calculates the focus correction amount and corrects the focus position a plurality of times during the one-time tilt control. By this multiple corrections, it is possible to control the focus position to the optimum state each time it is repeated. The configuration and processing of the information processing apparatus according to the present embodiment are the same as those shown in FIGS. 1 and 6 of the first embodiment except for the processing shown in step S608 (the processing of FIG. 8). , Duplicate explanations are omitted.

Hereinafter, the processing performed by the information processing apparatus according to the present embodiment will be described with reference to FIGS. 9 and 10. In the present embodiment, the focus correction amount is recalculated for each step with the predetermined tilt angle control amount S as one step, and the focus position is corrected. FIG. 9 is a diagram in which an auxiliary line is drawn so that the focus correction amount for each control amount S of the tilt angle can be easily referred to the graph showing the relationship between the tilt angle and the focus correction amount β shown in FIG. be.

In the example of FIG. 9, the start position of the tilt control is P1 and the target position is P2, and the focus correction amount for each control amount S between them is shown. In the present embodiment, the focus correction amount as shown in FIG. 9 is calculated for each control amount S, and the focus position is corrected each time. Here, it is assumed that the focus drive speed during one focus correction is constant, but it may be finely adjusted as necessary. Further, in the present embodiment, the focus correction amount is calculated according to the equation (2), but the table data or the coefficient for acquiring the focus correction amount is stored, and the correction is performed with reference to them. May be. Further, in the present embodiment, the control angle S is set so that the tilt angle can reach exactly the target position from the start position by driving each step, but the control angle S is not particularly limited in this way. For example, the focus correction amount is calculated for each control angle S until the control immediately before reaching the target position, and the focus correction amount is calculated by setting the control angle from the tilt angle at that time to the target position only at the last control. You may do it.

FIG. 10 is a flowchart showing an example of detailed processing of tilt control and focus control performed by the information processing apparatus according to the present embodiment in step S608. In step S1001, the tilt control unit 114 acquires the current tilt angle as the tilt start position. In step S1002, the tilt control unit 114 sets the control amount S for one step of the tilt control. The value of the control amount S is set to a value desired by the user. The smaller the value, the higher the focus maintenance effect during the tilt control, and the larger the value, the shorter the time required for the tilt control.

In step S1003, the correction amount calculation unit 116 calculates the focus correction amount for one step. In this example, the correction amount calculation unit 116 calculates the focus correction amount based on the tilt angle at that time and the control amount S of one step. In step S1004, the tilt drive unit 115 and the focus drive unit 118 perform tilt drive and focus drive for one step. When both the tilt drive and the focus drive are completed for one step, the process proceeds to step S1005.

In step S1005, the tilt control unit 114 determines whether or not the tilt angle has reached the target position set in step S608. If the target position has been reached, the process ends, and if not, the process proceeds to step S1006. In step S1006, the tilt control unit 114 determines whether or not there is a stop instruction by the user during the tilt drive. When there is a stop instruction, the tilt drive unit 115 stops the tilt drive at the position corresponding to the stop instruction and ends the process. If this is not the case, the process returns to step S1003, assuming that the tilt drive and focus drive for the next step are performed.

According to such a process, it is possible to perform two or more focus corrections for one tilt control. Therefore, by sequentially controlling the focus position to the optimum state each time the focus correction is repeated, the effect of maintaining the focus in the target region can be improved.

[Embodiment 3]
The information processing apparatus according to the present embodiment automatically adjusts the focus position so as to focus on two target areas in the subject image by tilt control and focus control. The information processing apparatus 100 can automatically adjust the focus position so as to focus on two subjects having different distances from the information processing apparatus 100, for example. Since the information processing apparatus 100 according to the present embodiment has the same configuration as that shown in FIG. 1 of the first embodiment and can perform the same processing, duplicate description will be omitted.

Hereinafter, the processing performed by the information processing apparatus 100 according to the present embodiment will be described with reference to FIGS. 12 and 13. FIG. 12 shows an example of an captured image captured by the information processing apparatus according to the present embodiment. In this example, the subject 1201 is located closer to the information processing apparatus 100 than the subject 1202. When there is a difference in the distances to the two subjects in this way, the depth of field may be insufficient in normal imaging, and it may not be possible to focus on both. Therefore, the tilt control unit 114 according to the present embodiment determines the control amount of the tilt control so that both the subjects 1201 and 1202 are in focus by correcting the focus position. That is, the information processing apparatus according to the present embodiment determines the tilt control amount and the focus correction amount so that the two subjects are both on the focus plane by the tilt control and the focus control as in the first embodiment.

The area setting unit 112 according to the present embodiment sets two target areas in the subject image on the image sensor in the same manner as in the first embodiment. Here, the focus drive unit 118 can preferentially focus on one of the two target areas based on the focus evaluation value calculated by the evaluation value calculation unit 113 calculated from each target area. .. The focus drive unit 118 may focus on the target area having a higher priority set in the target area, for example, and focus on the target area selected based on the focus evaluation value before focusing. The method of selecting the target area to be focused is not particularly limited. The priority set in the target area may be set by input such as user selection. Further, this priority may be set according to the type of the detection target (priority for focusing on a specific subject, etc.), and may be set for a target area in which a subject closer to the information processing apparatus 100 is captured. The priority may be set according to a predetermined condition, such as setting a higher priority. Hereinafter, for the sake of explanation, the subjects 1201 and 1202 in FIG. 12 will be used as the subjects of the two target areas set by the area setting unit 112, and the subject 1201 will be preferentially focused.

The tilt control unit 114 determines a target position for tilt control such that the subject 1202 is focused by the same tilt control and focus control as in the first embodiment while maintaining the focus on the subject 1201. The tilt control unit 114 focuses on the subject 1202 based on the focus evaluation value of the area on the subject 1202 during the tilt control (and the focus correction associated with maintaining the focus on the subject 1201). The target position of may be detected. Further, the tilt control unit 114 focuses on a case where the tilt control is performed while maintaining the focus on the subject 1201 and a case where the tilt control is performed while first focusing on the subject 1202 and maintaining the focus. You may calculate a common tilt control position where the positions of are the same. In this case, for example, by using the equation (2), it is possible to calculate the above-mentioned common tilt control position.

When the distance (k) from the rotation axis in the subject image by the image sensor to the subject 1201 is zero, the in-focus state of the subject 1201 does not change depending on the tilt control. Therefore, when k is zero, the correction amount calculation unit 116 does not consider the in-focus of the subject 1201 and corrects the focus of the subject 1202 with respect to the position of the arbitrary tilt control in the same manner as in the first embodiment. The amount can be calculated. Further, when k is close to zero, it is considered that the change in the in-focus state of the subject 1201 due to the tilt control is so small that it can be ignored. Therefore, a predetermined range is set before and after zero, and the value of k is the range. If it is within (for example, −0.5 or more and 0.5 or less), it may be zero.

FIG. 13 is a flowchart showing an example of processing performed by the information processing apparatus according to the present embodiment. In step S1301, the area setting unit 112 sets two target areas in the subject image on the image element. In step S1302, the area setting unit 112 sets one of the two target areas as a preferentially focused area (hereinafter, focus area). In step S1303, the area setting unit 112 sets the target area, which was not the focus area in step S1302, as the area to be focused by the tilt control (hereinafter referred to as the tilt area).

In step S1304, the tilt control unit 114 adjusts the focus based on the focus evaluation value of the focus region. Here, the AF function focuses on the focus area.

In step S1305, the correction amount calculation unit 116 calculates the distance k from the rotation axis in the subject image by the image sensor to the focus region. If k is 0, the process proceeds to step S1307, and if not, the process proceeds to step S1308.

In step S1307, the correction amount calculation unit 116 calculates the focus correction amount for the position of the tilt control. The position of the tilt control may be determined, for example, from an image or may be determined by a user operation. Following step S1307, processing proceeds to step S1309.

In step S1308, the tilt control unit 114 determines the target position of the tilt control so that the focus is focused on the tilt region by the same tilt control and focus control as in the first embodiment while maintaining the focus on the focus region. do. Following step S1308, processing proceeds to step S1309.

In step S1309, the tilt drive unit 115 and the focus drive unit 118 drive based on the position of the tilt control determined in step S1307 or step S1308 and the position of the focus lens determined accordingly, and the process ends.

According to such processing, it is possible to focus on two subjects to be focused at the same time by performing tilt control and focus control. For example, it is possible to first focus on one subject with the AF function, and then perform tilt control and focus control so that the other subject is in focus while maintaining the focus. Therefore, it is possible to automatically focus on both of two subjects having different distances from the image pickup device.

[Embodiment 4]
In the above-described embodiment, for example, each processing unit shown in FIG. 1 and the like is realized by dedicated hardware. However, a part or all of the processing unit of the information processing apparatus 100 may be realized by a computer. In this embodiment, at least a part of the processing according to each of the above-described embodiments is executed by a computer.

FIG. 11 is a diagram showing a basic configuration of a computer. In FIG. 11, the processor 1110 is, for example, a CPU, and controls the operation of the entire computer. The memory 1120 is, for example, a RAM, and temporarily stores programs, data, and the like. The computer-readable storage medium 1130 is, for example, a hard disk, a CD-ROM, or the like, and stores programs, data, and the like for a long period of time. In the present embodiment, the program that realizes the functions of each part stored in the storage medium 1130 is read out to the memory 1120. Then, the processor 1110 operates according to the program on the memory 1120, so that the functions of each part are realized.

In FIG. 11, the input interface 1140 is an interface for acquiring information from an external device. Further, the output interface 1150 is an interface for outputting information to an external device. The bus 1160 connects the above-mentioned parts and enables data exchange.

(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

The invention is not limited to the above embodiment, and various modifications and modifications can be made without departing from the spirit and scope of the invention. Therefore, a claim is attached to publicize the scope of the invention.

101: Zoom lens, 102: Focus lens, 103: Aperture unit, 104: Bandpass filter, 105: Color filter, 106: Image sensor, 107: AGC, 108: AD conversion unit, 109: Signal processing unit, 110: Communication Unit, 111: Monitor device, 112: Area setting unit, 113: Evaluation value acquisition unit, 114: Aperture control unit, 115: Aperture drive unit, 116: Correction amount calculation unit, 117: Focus control unit, 118: Focus drive unit

Claims (19)

A control means for tilting the image pickup element with respect to a plane orthogonal to the optical axis of the image pickup optical system around the axis of rotation, and a focus control for moving the focus lens in the direction of the optical axis.
A setting means for setting a target area in an image captured by an image pickup unit, and
A determination means for determining a correction amount of the position of the focus lens based on the position of the rotation axis and the position of the target area is provided.
The control means is an information processing device that moves the focus lens based on the correction amount. The determination means is characterized in that the correction amount is determined so that the larger the distance from the rotation axis of the image pickup device to the position corresponding to the target region, the larger the correction amount of the position of the focus lens. , The information processing apparatus according to claim 1. The information according to any one of claims 1 or 2, wherein the determination means determines a correction amount of the position of the focus lens based on the rotation angle of the rotation drive by the tilt control. Processing equipment. The determination means is characterized in that the rotation angle of the rotation drive is determined based on the current rotation angle of the image pickup element or the image pickup optical system and the rotation angle which is a target of tilt control. Item 3. The information processing apparatus according to Item 3. The information processing according to claim 3 or 4, wherein the determination means determines the correction amount so that the correction amount of the position of the focus lens becomes larger as the rotation angle of the rotation drive becomes larger. Device. The information processing apparatus according to any one of claims 1 to 5, wherein the control means controls the position of the focus lens at a speed corresponding to a control speed of rotation drive by tilt control. The information processing apparatus according to claim 6, wherein the control means moves the focus lens so that the focus control ends at the same time as the tilt control ends. The control means according to claim 6 or 7, wherein the control means controls the correction of the position of the focus lens at a speed corresponding to the rotation angle of the image pickup element or the image pickup optical system in the tilt control. Information processing device. The control means increases the speed of controlling the position of the focus lens as the absolute value of the rotation angle of the image pickup element or the image pickup optical system in the tilt control is larger in the range of 0 ° to 90 °. 8. The information processing apparatus according to claim 8. The determination means is characterized in that, when the tilt control is stopped, the correction amount is determined according to the rotation angle of the image pickup device or the image pickup optical system at the time when the tilt control is stopped. The information processing apparatus according to any one of 1 to 9. The determination means determines a correction amount of two or more for one instruction of the tilt control, and determines the correction amount.
The control means according to claim 1 to 10, wherein the control means adjusts the position of the focus lens in a plurality of times in response to the one-time tilt control instruction based on the two or more correction amounts. The information processing device according to any one of the items. 11. The control means is characterized in that, by adjusting the position of the focus lens divided into a plurality of times, the position of the focus lens is sequentially adjusted during the one-time tilt control. The information processing device described. The target area according to any one of claims 1 to 12, wherein the target area is a region designated by a user or a region automatically detected as an object to be detected from within the captured image. Information processing equipment. The method according to any one of claims 1 to 13, wherein the determination means determines the amount of correction of the position of the focus lens according to the defocus of the target area due to the tilt control. Information processing device. It is further provided with a determination means for determining the control amount of the tilt control so as to focus on both the first target area and the second target area by correcting the position of the focus lens. , The information processing apparatus according to any one of claims 1 to 14. Further provided with an adjusting means for adjusting the position of the focus lens so as to focus on the first target region before performing the tilt control.
The information processing apparatus according to claim 15, wherein the tilt control is performed while maintaining a state of being in focus on the first target region. The setting means further sets a priority for each target area, and the setting means further sets the priority.
The information processing apparatus according to claim 16, wherein the priority set in the first target area is higher than the priority set in the second target area. A control step of tilting the image pickup element with respect to a plane orthogonal to the optical axis of the image pickup optical system around the axis of rotation, and performing focus control of moving the focus lens in the direction of the optical axis.
The process of setting the target area in the captured image captured by the imaging unit, and
A step of determining a correction amount of the position of the focus lens based on the position of the rotation axis and the position of the target area is provided.
In the control step, the focus lens is moved based on the correction amount.
Information processing method. A program for making a computer function as each means of the information processing apparatus according to any one of claims 1 to 17.

Images