JP7293309B2 - IMAGING DEVICE, CONTROL METHOD AND STORAGE MEDIUM - Google Patents

Description

The present invention relates to an imaging device capable of tilt control.

Conventionally, a surveillance camera is installed at a high place, and the optical axis of the surveillance camera is directed obliquely downward to monitor people passing by on the road or photograph a vehicle and its license plate.

A focal plane on which an image is focused is a plane perpendicular to the optical axis. When the optical axis of the surveillance camera is directed obliquely downward, the focal plane that is in focus when capturing an image does not match the imaging plane of the subject that is actually captured. Therefore, the in-focus area is part of the screen, and the other areas are out of focus.

To solve this problem, there is a method of narrowing down the aperture of the optical system to deepen the depth of field and prevent the out-of-focus blur.

However, in surveillance cameras that take pictures under low illuminance, there are many cases where the aperture is opened to near maximum. As a result, the depth of field becomes shallow, the entire screen is out of focus, and the image is shot in an out-of-focus state.

A technique called Scheimpflug's theorem, which widens the range of depth of field by relatively tilting a lens or an image pickup device, is generally known for such a problem.

Japanese Patent Application Laid-Open No. 2002-200001 discloses a technique for determining a focus position and a tilt control amount so as to focus on a plurality of subjects. In this technique, it is necessary to detect the in-focus position for each subject. As a method of detecting a focus position, a method of executing a process for controlling the focus position for each subject so as to maximize the contrast evaluation value is described. In the imaging apparatus disclosed in Patent Document 1, the control amount of the tilt angle and focus position is calculated from the object distance and image height of two points.

JP-A-2003-75716

However, Patent Document 1 does not mention the positions of the two points used when controlling the tilt angle and the focus position. Therefore, when there are a plurality of subjects, depending on the selection of two points, the control amount of the tilt angle and the focus position may not be calculated correctly.

As a result, deviation from the ideal tilt plane position occurs. 2. Description of the Related Art In recent years, with the increase in the number of pixels of imaging devices and the increase in size of display devices, there is a demand for higher definition images. For this reason, the method of Patent Document 1 cannot achieve sufficient focusing when using a 4K or 8K camera.

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide an imaging apparatus capable of calculating a tilt angle and a focus position with high accuracy by selecting an appropriate subject when there are a plurality of subjects.

An image pickup device as one aspect of the present invention is a tilt drive unit that tilts an image pickup device to change a tilt angle that is an angle between an image pickup surface of the image pickup device and an orthogonal plane perpendicular to an optical axis of an image pickup optical system. a focus driving unit that drives the focus lens of the imaging optical system to change the focus position; a subject detection unit that detects a subject in an image captured by the imaging device; a distance measurement unit that measures the distance to the subject; a determination unit that determines the tilt angle and the focus position based on the distance measurement result of the distance measurement unit; and a control unit that controls the tilt driving unit and the focus driving unit based on the focus position, and the control unit controls the detection size of the subject detected by the subject detection unit, the size of the image sensor , and detection of the subject based on reliability determined by a detection position, which is a distance from a position corresponding to the rotation axis to the subject, and accuracy of the distance measurement result of the subject measured by the distance measuring unit; selects a subject to be used for determining the tilt angle and the focus position from the plurality of subjects detected by the determination unit, and determines the tilt angle based on the distance measurement result of the selected subject. and the focus position is determined, and the reliability is higher as the distance from the position corresponding to the rotation axis of the imaging element to the subject is longer .

Other objects and features of the invention are described in the following embodiments.

According to the present invention, it is possible to provide an imaging apparatus capable of calculating a tilt angle and a focus position with high accuracy by selecting an appropriate subject when there are a plurality of subjects.

It is a block diagram which shows the structure of an imaging device. It is a figure which shows the pixel structure of an image pick-up element. It is a figure explaining the calculation method of bit deviation|shift amount. 4 is a diagram for explaining a method of calculating a conversion coefficient K; FIG. It is a figure explaining swing control. FIG. 5 is a diagram for explaining tilt control and focus control; 4 is a flowchart for explaining tilt/focus control according to the first embodiment; It is a figure of the situation of the subject of 1st Embodiment. FIG. 9 is a diagram showing a focus position on the imaging plane when the situation in FIG. 8 is measured by the imaging plane phase difference method; It is a reliability table. 9 is a flowchart for explaining tilt/focus control according to the second embodiment; It is a figure of the situation of the subject of 3rd Embodiment. FIG. 13 is a diagram showing a focus position on the imaging plane when ranging in the situation of FIG. 12 by the imaging plane phase difference method; FIG. 11 is a flowchart for explaining tilt/focus control according to the third embodiment; FIG. FIG. 14 is a flowchart for explaining tilt/focus control according to the fourth embodiment; FIG. It is a figure of the situation of the subject of 4th Embodiment. FIG. 17 is a diagram showing the focus position on the imaging plane when ranging in the situation of FIG. 16 by the imaging plane phase difference method; It is a reliability table.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(overall structure)
First, with reference to FIG. 1, an imaging device according to the present embodiment will be described. FIG. 1 is a block diagram of an imaging device (control device) 100. As shown in FIG. The zoom lens 101 moves in the optical axis direction to change the focal length. The focus lens 102 moves in the optical axis direction to control focus (focus control).

A diaphragm unit 103 adjusts the amount of light. In this embodiment, the zoom lens 101, the focus lens 102, and the diaphragm unit 103 constitute an imaging optical system.

Light that has passed through the imaging optical system is received by an imaging device 106 via a bandpass filter (BPF) 104 and a color filter 105 . The bandpass filter 104 may be movable with respect to the optical path of the imaging optical system.

The imaging device 106 has a CMOS sensor or the like, and photoelectrically converts an optical image (object image) formed via an imaging optical system.

An analog electric signal (imaging signal) output from the imaging device 106 is gain-adjusted by an AGC (Auto Gain Control) 107 . The A/D converter 108 converts the analog electrical signal into a digital signal (digital imaging signal).

A camera signal processing unit 109 performs various types of image processing (for example, gamma conversion and white balance adjustment) on the digital imaging signal to generate a video signal. The video signal is output via the communication unit 110 to the monitoring monitor device 111 connected to the imaging device 100 by wire or wireless communication.

The communication unit 110 also receives a command from an external device such as an external PC and outputs a control signal such as a command to a tilt/focus control unit (control unit) 115 in the imaging apparatus 100 .

In this embodiment, a system including the monitoring device 111 is called an imaging device. On the other hand, in FIG. 1, components other than the surveillance monitor device 111 are housed in one housing to constitute a surveillance camera.

A distance measurement information calculation unit (distance measurement unit) 112 receives RGB pixel values or luminance values from the camera signal processing unit 109 for each target subject area, and acquires distance information based on a phase difference.

The subject detection unit 113 receives the result from the camera signal processing unit 109 and detects the subject within the captured image. The subject detection unit 113 can also detect subjects specified by the user or preset subjects such as people and cars.

A tilt/focus drive amount calculation unit (determination unit) 114 acquires distance measurement information from the distance measurement information calculation unit 112 and object information from the object detection unit 113 .

Further, the tilt/focus driving amount calculation unit 114 calculates an appropriate (optimal) tilt angle and focus position according to the scene based on the distance measurement information and the subject information, and the tilt angle and focus position from the control unit 115 . Calculate

The control unit 115 incorporates a CPU as a computer, and executes various operations of the entire apparatus as control means based on computer programs stored in the memory 120 . The control unit 115 controls the tilt angle and the focus position based on the tilt angle and the focus position calculated by the tilt/focus driving amount calculation unit 114 .

The control unit 115 performs focus control, zoom control, and aperture control in autofocus (AF) and manual focus (MF) according to instructions from an external device such as an external PC via the communication unit 110 . Further, the control unit 115 receives current position information from the image sensor driving unit 116, the diaphragm driving unit 117, the focus driving unit 118, and the zoom driving unit 119, and sends the current position information to the tilt/focus driving amount calculation unit 114. Output.

The control unit 115 also instructs the image sensor driving unit 116 and the focus driving unit 118 of the tilt angle and the focus position calculated by the tilt/focus driving amount calculation unit 114 .

The imaging device driving unit 116 functions as a tilting driving unit, and controls the tilting angle by tilting the imaging device 106 based on the tilting angle instructed by the control unit 115 . That is, the imaging device driving unit 116 tilts the imaging device 106 to change the tilt angle, which is the angle between the imaging plane of the imaging device 106 and the orthogonal plane perpendicular to the optical axis of the imaging optical system.

Normally, the axis of rotation for tilting the image pickup device 106 is a horizontal axis (along the longitudinal direction of the image pickup device) that passes through the center of the captured image. relatively tilted. However, this embodiment is not limited to this.

The aperture drive unit 117 controls the position of the aperture unit 103 based on the aperture setting value transmitted from the control unit 115 . The focus drive unit 118 controls the position of the focus lens 102 (performs focus control) based on the focus setting value (based on the defocus amount) instructed by the control unit 115 . The zoom drive unit 119 controls the position of the zoom lens 101 based on the zoom setting value transmitted from the control unit 115 .

The distance measurement information calculation unit 112 uses focus detection data obtained by the camera signal processing unit 109 to perform distance measurement information calculation processing using a phase difference detection method. More specifically, the camera signal processing unit 109 generates a pair of image data formed by light beams passing through a pair of pupil regions of the imaging optical system as focus detection data. The distance measurement information calculation unit 112 detects the defocus amount as the distance measurement result based on the amount of deviation between the pair of image data. In this manner, the distance measurement information calculation unit 112 performs distance measurement (focus detection) by the imaging plane phase difference detection method based on the output of the image sensor 106 without using a dedicated AF sensor.

Details of the ranging information calculation operation by the imaging plane phase difference detection method of the ranging information calculation unit 112 will be described later. Further, the distance measurement information calculation unit 112 acquires an evaluation value (contrast evaluation value) related to contrast at a specific frequency, and calculates a defocus amount based on the difference between the original focus position and the focus position where the contrast evaluation value peaks. can be calculated.

The memory (storage means) 120 stores data related to programs executed by the control unit 115, and also stores data related to shading correction coefficients and data related to conversion coefficients.
(Ranging information calculation operation by imaging plane phase difference detection method)
Next, the pixel configuration of the image sensor 106 will be described with reference to FIG. FIG. 2 is a pixel configuration diagram of the image sensor 106. As shown in FIG.

In the image sensor 106, all pixels 201 are divided in the X direction into two photoelectric conversion units 201a and 201b. Further, the image sensor 106 can independently read the photoelectric conversion signals of the photoelectric conversion units 201a and 201b and the sum of the photoelectric conversion signals of the two photoelectric conversion units 201a and 201b.

Further, by subtracting the photoelectric conversion signal of one photoelectric conversion unit (for example, the photoelectric conversion unit 201a) from the sum of the photoelectric conversion signals of the two photoelectric conversion units 201a and 201b, the other photoelectric conversion unit (for example, the photoelectric conversion unit 201b) can obtain a signal corresponding to the photoelectric conversion signal of .

The photoelectric conversion signals in the photoelectric conversion units 201a and 201b are used as focus detection data for phase difference AF. Also, the sum of the photoelectric conversion signals of the two photoelectric conversion units 201a and 201b is used as normal captured image data.

The distance measurement information calculation unit 112 calculates the relative value of the signal (A image) corresponding to the photoelectric conversion signal of the photoelectric conversion unit 201a generated in this way and the signal (B image) corresponding to the photoelectric conversion signal of the photoelectric conversion unit 201b. A typical image shift amount is calculated by correlation calculation. Thereby, the distance measurement information calculation unit 112 can calculate the bit shift amount [bit], which is the degree of correlation between the pair of image signals. By multiplying the bit shift amount by the conversion coefficient, the bit shift amount is converted into the defocus amount [mm] of the predetermined area.

In this embodiment, the sum of the output signal of one of the photoelectric conversion units 201a and 201b and the output signal of the two photoelectric conversion units 201a and 201b is read from each pixel 201 from the image sensor . For example, when the output signal of the photoelectric conversion unit 201a and the sum of the output signals of the photoelectric conversion units 201a and 201b are read out, the output signal of the photoelectric conversion unit 201b is obtained from the sum of the output signals of the photoelectric conversion units 201a and 201b. It can be obtained by subtracting the output of the conversion unit 201a. As a result, both the A image and the B image can be obtained, and distance measurement (focus detection) can be realized by the imaging plane phase difference detection method. Also, the sum of the output signals of the photoelectric conversion units 201a and 201b generally forms one pixel (output pixel) of the output image. In addition, since such an image sensor is well-known, detailed description is omitted.

Next, a distance measurement information calculation operation based on the imaging plane phase difference detection method will be described. The following ranging information calculation operation is executed mainly by the ranging information calculation unit 112 .

First, the distance measurement information calculation unit 112 sets the focus detection position. Subsequently, the distance measurement information calculation unit 112 reads focus detection data from the set focus detection position. By using signals read from pixels within the focus detection position set by the distance measurement information calculation unit 112, the distance measurement information calculation unit 112 generates respective signals of the A image and the B image. Subsequently, the distance measurement information calculation unit 112 acquires the bit shift amount P [bit] by calculating the relative image shift amount between the A image and the B image by correlation calculation.

Here, an example of the correlation calculation method will be described with reference to FIG. FIG. 3 is an explanatory diagram of the correlation calculation method (calculation method of bit deviation amount).

In FIG. 3, the vertical axis indicates signal values, and the horizontal axis indicates bits (positions). Assume that the signals of the A image and the B image are read from the pixels (focus detection pixels) of the image sensor 106 . The camera signal processing unit 109 first performs digital filter processing on each of the A image and the B image in order to reduce noise. FIG. 3(a) shows the waveform after filtering.

As shown in FIGS. 3A to 3D, the ranging information calculator 112 bit-shifts either the A image signal or the B image signal or both the A image signal and the B image signal. A correlation amount COR at that time is calculated. The correlation amount COR is an area when the A image and the B image are overlapped, a value obtained by subtracting the area of the B image from the area of the A image, or a calculated value representing the degree of correlation, but is limited to these. not a thing A case where the correlation amount COR is the area when the A image and the B image are overlapped will be described below.

When the A image and the B image match, the overlap between the A image and the B image increases, so the correlation is highest and the correlation amount COR increases. Here, the shift amount [bit] when the correlation amount COR takes the maximum value is the bit shift amount P [bit].

Subsequently, the ranging information calculator 112 acquires the conversion coefficient K. FIG. By multiplying the bit shift amount P by the conversion coefficient K, the defocus amount DEF is obtained.

Calculation of the conversion coefficient K will now be described with reference to FIG. FIG. 4 is an explanatory diagram of calculation of the conversion coefficient K. FIG. In FIG. 4, the Z axis indicates the optical axis direction of the imaging optical system, and Z=0 indicates the plane of the imaging element 106 (imaging plane). Zep indicates the exit pupil distance. The pupil intensity distribution PI_A and the pupil intensity distribution PI_B, which are the light amount distributions of the focus detection light beams of the A image and the B image on Z=Zep, are the signals (focus detection signals) output from the photoelectric conversion units 201a and 201b, respectively. It is a projection image projected onto the exit pupil plane.

PI_A and PI_B in FIG. 4 denote one-dimensional pupil intensity distributions. Let the distance between the centers of gravity of the pupil intensity distributions PI_A and PI_B be the base line length BL. At this time, the change amount [mm] in the optical axis direction with respect to the bit shift amount P [bit] between the A image and the B image can be obtained from the ratio between the exit pupil distance Zep and the base length BL.

Therefore, the transform coefficient K can be expressed as in Equation (1) below.

K=Zep/BL (1)
Next, the distance measurement information calculation unit 112 calculates the defocus amount DEF [mm] by the following formula (2).

DEF=P×K (2)
When focusing at the focus detection position, the tilt/focus drive amount calculator 114 calculates the drive amount M [lensmm] of the focus lens 102 based on the defocus amount DEF using the following equation (3). The control unit 115 then controls the focus driving unit 118 to drive the focus lens 102 .

M=DEF×FS (3)
In Expression (3), FS is the sensitivity for converting the defocus amount DEF [mm] to the lens driving amount M [lensmm].

The control unit 115 sets the lens driving amount M, and the focus driving unit 118 drives the focus lens 102 in the optical axis direction based on the set lens driving amount M. FIG. As a result, an in-focus image is obtained at the focus detection position.

Thus, the distance measurement information calculation unit 112 calculates the defocus amount DEF based on the distance measurement information obtained by the imaging plane phase difference detection method, and the tilt/focus driving amount calculation unit 114 calculates the defocus amount DEF A focus drive amount is calculated based on.
(Description of tilt control)
The swing control will be described with reference to FIG. FIG. 5 is an explanatory diagram of tilt control. FIG. 5A shows a state in which the optical system (imaging optical system) and the imaging device 106 are parallel. The focal length L is in focus, and the focal plane is parallel to the optical system and the image sensor 106, respectively.

FIG. 5B shows a state in which tilt control is performed by rotating the image sensor 106 around the image sensor rotation axis from the state of FIG. 5A. When the tilt control is performed, the focal plane also rotates about the focal plane rotation axis corresponding to the imaging element rotation axis based on the Scheimpflug principle. For this reason, it is possible to bring all objects in focus from short distances to long distances with respect to a certain plane. The Scheimpflug principle is that when the main plane of the optical system and the imaging plane of the imaging element 106 intersect on a straight line, the focal plane also intersects on the same straight line.

The tilt angle b is calculated by the following equation (4) according to the Scheimpflug principle using the focal length f, the focal length L, and the depression angle a.

b=tan ⁻¹ (f/(Ltan(a))) (4)
FIG. 5(c) shows a scene in which subject X and subject Y exist as a plurality of subjects. In this case, as shown in FIG. 5C, it is desirable to control the focus plane so that the faces of subjects X and Y are in focus. Here, the subject includes a part of the subject such as the subject's face. Therefore, it is necessary to perform not only tilt control but also focus control. Also, the optimum focus plane (that is, the optimum tilt angle and the optimum focus position) differs for each subject, and it is difficult for the user to manually adjust it.

Therefore, with reference to FIG. 6, an example of calculating the optimum tilt angle and the optimum focus position according to the subject will be described.

FIG. 6 is an explanatory diagram of tilt control and focus control. Subjects X and Y exist as subjects of interest, as in FIG. 5(c).

The current tilt angle and the position of the focus lens 102 have the positional relationship shown in the upper part of FIG. x is the amount of correction from the focal plane of the imaging optical system required to bring the subject X into focus, and y is the correction from the focal plane of the imaging optical system required to bring the subject Y into focus. quantity.

Also, let the distance (image height) from the tilt axis to the subject on the imaging element 106 be k1 [um] for subject X and k2 [um] for subject Y. FIG.

Here, let α be the tilt angle for focusing on the objects X and Y at the same time, and β be the focus correction amount (focus position) on the focus plane due to the movement of the focus lens 102. Then, the following equation ( 5) and (6) are established.

x−β=k1×tan(α)+β (5)
y=k2×tan(α)−β (6)
Solving the simultaneous equations of Equations (5) and (6), the tilt angle α and the focus correction amount β are expressed by Equations (7) and (8) below, respectively.

α=tan ⁻¹ ((x+y)/(k1+k2)) (7)
β=(k2×x−k1×y)/(k1+k2) (8)
The drive amount of the focus lens 102 can be calculated simply by dividing the focus correction amount β by the sensitivity FS of the focus lens 102 .

On the other hand, the driving amount of the focus lens 102 can be accurately calculated by solving a higher-order equation or polynomial according to the sensitivity FS. However, this embodiment is not limited to this, and may be calculated by other methods.

As described above, unless the distance from the tilt axis on the image pickup device 106 to the subject is accurately calculated, the focus of the subject is poor even if the focus position and tilt angle are corrected.
[First embodiment]
Next, a method of tilt/focus position control in the first embodiment will be described with reference to FIG.

FIG. 7 is a flowchart of tilt/focus position control processing in this embodiment. The processing of this embodiment is executed by the control unit 115 according to a processing program as a computer program. Note that the processing program is stored in, for example, a computer-readable storage medium (memory 120 or the like).

In step S<b>701 , communication section 110 sends a tilt control start command to control section 115 .

In step S<b>702 , the subject detection unit 113 detects a subject within the image captured by the imaging device 100 . Subject detection may be human body detection or object detection.

In step S703, the control unit 115 counts the number of subjects and determines whether the number of subjects is three or more.

If the number of subjects is three or more, the process proceeds to step S704. In step S<b>704 , the control unit 115 acquires the detection position (image height) and detection size (size) of the subject detected by the subject detection unit 113 . Here, the detection position (image height) is the distance from the rotational axis of the sensor to the subject. Also, the detection size (size) is determined by human body recognition or face recognition.

In step S705, the distance measurement information calculation unit 112 measures the defocus amount (distance) as the distance measurement result at the detection position of each subject detected in step S702. Specifically, a defocus amount (distance) is measured by aligning a frame with each subject. Although the frame here is an AF frame, it is not limited to this. A phase difference method or a contrast method may be used to measure the defocus amount. The defocus amount is measured in step S705. In the case of distance measurement by the phase difference method, the distance measurement accuracy decreases as the distance from the current focus position to the subject increases. In addition, when the object is far from the center of the image, the accuracy of distance measurement deteriorates due to the influence of lens shading and the like. Therefore, if a subject (or subject frame) that is far from the center of the image is selected as the range-finding frame, the focus position and tilt angle correction accuracy will be low, resulting in a poorly focused image. Sometimes. In the contrast method as well, since the object distance is calculated from the focused focus position, mechanical errors (backlash) and stopping accuracy errors are involved, resulting in errors in the calculation of the object distance. Therefore, when the two subjects (or two frames) are close to each other, the error rate increases, and the accuracy of correcting the focus position and the tilt angle decreases, resulting in an image that is poorly focused.

For example, FIG. 8 is a diagram of a situation when there are three subjects. FIG. 9 is a diagram showing the focus position on the imaging surface when the situation in FIG. 8 is measured by the imaging surface phase difference method. The subject Z has a high image height (a long distance from the rotation axis of the sensor) and has a focal plane at a position far from the sensor. Therefore, the defocus amount measured by the phase difference method may differ greatly from the actual distance due to the influence described above. In this case, if the defocus amount of the object Z is used to correct the focus position and the tilt angle, the focused plane becomes a soft image. For the above reasons, in order to perform tilt/focus position control with high accuracy, it is necessary to appropriately select the subject used for calculating the tilt angle and focus position.

In step S706, the control unit 115 calculates the reliability (applicability) for each subject (for each frame of the subject) from the detected size of the subject, the detected position (image height), and the precision of the defocus amount (distance measurement precision). judge. For calculation of reliability, a table such as that shown in FIG. 10 may be referred to. FIG. 10 is an example, and various modifications and changes are possible within the scope of the gist of the method of creating the table. In FIG. 10, the reliability is calculated (determined) in consideration of all of the detection size of the subject, the detection position, and the accuracy of the defocus amount. The control unit 115 may calculate the reliability using at least one of the detection size of the object, the detection position, and the precision of the defocus amount. The elements and weights for calculating the reliability may be changed according to the distance measurement method used to calculate the defocus amount and the shooting environment.

FIG. 10 shows a case where the detection size Ta, the detection position Tb, and the precision Tc of the defocus amount, which are the respective elements, are evaluated from 0 to 1. In FIG. The reliability is represented by Equation (9).

T=Ta×Tb×Tc (9)
The reliability calculation formula is only an example, and the weighting (specific gravity) for each element may be changed according to the distance measurement method used to calculate the defocus amount and the shooting environment, or each element may be added. good. Various modifications and changes are possible within the scope of the gist of the reliability calculation method. This is performed for each subject to calculate the reliability for each subject. In the case of FIG. 10, X and Y with high reliability are selected. Also, in FIG. 10, since Y is closer to the center of the image than X, the Y ranging accuracy is higher than the X ranging accuracy.

In step S707, the control unit 115 selects two subjects (or subject frames) with high reliability. The image height difference between the two objects may be taken into consideration when selecting the object as described above.

In step S708, the tilt/focus drive amount calculation unit 114 calculates the focus drive amount d and the tilt angle drive amount θ from the defocus amounts DEF1 and DEF2 as the distance measurement results for the two selected subjects (or subject frames). do.

In step S709, the control unit 115 drives the focus driving unit 118 and the image sensor driving unit 116 according to the driving amount calculated by the tilt/focus driving amount calculation unit 114 in step S708.

In step S703, if the number of subjects is not three or more, the process proceeds to step S710.

In step S710, the control unit 115 determines whether the number of subjects is two. If the number of subjects is two, the process proceeds to step S708. If the number of subjects is one, the process proceeds to step S711.

In step S711, the distance measurement information calculation unit 112 measures the defocus amount up to the subject.

In step S712, the control unit 115 drives the focus driving unit 118 according to the defocus amount measured in step S711.

When there is one subject, the image pickup device driving section 116 is not driven, and only the focus driving section 118 is driven.

In step S713, the control unit 115 terminates this control process.

According to the present embodiment, by selecting two subjects (or two frames) for performing tilt/focus position control according to the reliability, subjects suitable for tilt/focus position control (or subject frame) can be selected. Therefore, it is possible to calculate the tilt angle and the focus position with high accuracy.
[Second embodiment]
In the first embodiment, an example of implementing the tilt/focus position control method with selection based on reliability has been described.

Below, tilt/focus position control processing according to the second embodiment of the present invention will be described with reference to the flowchart shown in FIG. In this embodiment, a control method when a new subject is recognized during tilt/focus position control or when a subject within an image moves will be described. A detailed description of the same configuration as in the first embodiment will be omitted.

What differs from the first embodiment is a series of processes in steps S1113 and S1114.

In step S1113, the control unit 115 determines whether the subject detection unit 113 has detected a new subject in the image, or whether the subject has not moved since the first detection.

If a new subject has not been detected, or if the subject has not moved since it was first detected, the process advances to step S1115 to end this control process. If a new subject is detected, or if the subject has moved since it was first detected, the process proceeds to step S1114.

In step S1114, the control unit 115 determines whether the distance measurement method is the contrast method.

If the distance measurement method is the contrast method, the focus lens 102 must be driven in order to perform distance measurement, resulting in poor quality. Therefore, the distance measurement is not performed again, and the process proceeds to step S1115. If the distance measurement method is the image plane phase difference method, it is not necessary to drive the focus lens 102 for distance measurement, so the process returns to step S1103. That is, the control unit 115 determines whether or not to recalculate (redetermine) the reliability according to the distance measurement method.

In this embodiment, when a new subject is detected in an image during tilt/focus position control, or when the subject moves within an image, if the reliability of the subject is high, the first calculated tilt angle, focus It is possible to perform tilt/focus position control with high accuracy based on the position.
[Third embodiment]
In the first embodiment, an example of implementing the tilt/focus position control method with selection based on reliability has been described.

However, in a real environment, not all objects detected as subjects are on the tilted focus plane. For example, FIG. 12 is a diagram showing a situation in which there are subjects A to D walking on the road, and subject E, which is a poster of a person such as a billboard on a building. FIG. 13 is a diagram showing the focal position on the imaging surface when the situation in FIG. 12 is measured by the imaging surface phase difference method. In this way, it is impossible to place subject E and other subjects A to D on the same focal plane. In that case, the subjects A to D excluding the subject E must be used to perform tilt/focus position control.

Hereinafter, tilt/focus position control processing according to the third embodiment of the present invention will be described with reference to the flowchart shown in FIG. In the present embodiment, a tilt/focus position control method when a subject that is clearly out of focus (clearly not on the focus plane) is detected in subject detection will be described. A detailed description of the same configuration as in the first embodiment will be omitted.

The difference from the first embodiment is the processing of step S1406.

In step S<b>1406 , the control unit 115 determines the subject correlation based on the subject detection position detected by the subject detection unit 113 and the distance measurement result of the subject by the distance measurement information calculation unit 112 . As a result, a subject that is clearly out of focus (not on the focus plane) in subject detection is detected. A general method such as the Smirnoff-Grubbs test is used for detecting such an object. In the Smirnoff-Grubbs test, outliers are excluded.

Also, an outlier may be detected such that the correlation coefficient is greater than or equal to a threshold.

The control unit 115 excludes the subject E corresponding to the outlier and calculates the reliability in step 1407 .

According to this embodiment, if there is a subject among the detected subjects that is not on the same focal plane even if tilt/focus position control is performed, the subject is excluded using correlation. By excluding subjects that are not on the same focal plane, high-precision tilt/focus position control becomes possible.
[Fourth embodiment]
In the first embodiment, an example of implementing the tilt/focus position control method with selection based on reliability has been described.

However, when selecting a subject (or subject frame), if there are only subjects with low reliability, and the distance measurement for the selected subject is different from the actual subject distance, the tilt angle calculated for that subject will greatly defocus the subject. there is a possibility.

When there are only subjects with low reliability as described above, the information of a plurality of subjects (or subject frames) is averaged, and a virtual frame having an average image height and subject distance is set. By calculating the tilt angle using the virtual frame, it is possible to prevent the subject from being largely out of focus.

Hereinafter, tilt/focus position control processing according to the fourth embodiment of the present invention will be described with reference to the flowchart shown in FIG. In this embodiment, a tilt/focus position control method when there are only subjects (or subject frames) with low reliability will be described. A detailed description of the same configuration as in the first embodiment will be omitted.

What differs from the first embodiment is a series of processes in steps S1507 and S1508.

In step S1507, the control unit 115 determines whether the reliability calculated in step S1506 is equal to or less than the threshold.

As a result, regardless of whether the reliability is high or low, it is possible to avoid greatly defocusing the subject.

If the reliability is lower than the threshold, the process proceeds to step S1508; otherwise, the process proceeds to step S1509.

In step S1508, the control unit 115 creates and selects a virtual frame from information on multiple subjects (or subject frames). The virtual frame is calculated by averaging image heights and subject distances of multiple subjects.

For example, FIG. 16 is a diagram of a situation when there are four subjects. FIG. 17 is a diagram showing the focus position on the imaging surface when the situation in FIG. 16 is measured by the imaging surface phase difference method.

In FIG. 18, the reliability is calculated (determined) in consideration of all of the detection size of the object, the detection position, and the precision of the defocus amount.

Since the reliability of subjects A to D in FIG. 16 is low, a virtual frame X obtained by averaging subjects A and B and a virtual frame Y obtained by averaging subjects C and D are created.

By calculating the tilt angle from the image height of the virtual frame X, object distance information, image height of the virtual frame Y, and object distance information, it is possible to prevent the object from being largely out of focus.

In the present embodiment, information on the subject (real frame) is averaged for the image height of the virtual frame and the subject distance, but they may be calculated by weighting according to the degree of reliability. Also, when selecting a subject (real frame) in creating a virtual frame, the selection may be made separately for the upper and lower sides of the rotation axis of the tilt angle. If a subject (subject frame) divided above and below the rotation axis is selected, the image height of the virtual frame will be low, the error will increase, and there is a possibility that the focus plane calculation accuracy will decrease. As described above, by separating the upper and lower sides of the rotation axis, the image height of the virtual frame can be maintained, and a decrease in the accuracy of the tilt angle can be suppressed.

According to this embodiment, when there is only a subject (subject frame) with a low reliability, the information of a plurality of subjects (subject frames) is averaged, and a virtual frame having an average image height and subject distance is set. By calculating the tilt angle using the virtual frame, it is possible to avoid the subject from being largely out of focus.
(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

According to each embodiment, it is possible to provide a blur evaluation device, a blur evaluation method, and a program capable of highly accurately evaluating the blur of a photographing means. Further, according to each embodiment, it is possible to provide a method for manufacturing an imaging means capable of highly accurate blur correction.

Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist.

116 tilt drive unit 118 focus drive unit 113 subject detection unit 112 distance measurement unit 114 determination unit 115 control unit

Claims (17)

a tilt driving unit that tilts an imaging device to change a tilting angle that is an angle between an imaging surface of the imaging device and an orthogonal plane perpendicular to an optical axis of an imaging optical system;
a focus driving unit that drives the focus lens of the imaging optical system to change the focus position;
a subject detection unit that detects a subject in an image captured by the imaging device;
a distance measuring unit that measures the distance to the subject detected by the subject detecting unit;
a determination unit that determines the tilt angle and the focus position based on the result of distance measurement by the distance measurement unit;
a control unit that controls the tilt driving unit and the focus driving unit based on the tilt angle and the focus position determined by the determining unit;
The control unit controls the detection size of the subject detected by the subject detection unit, the detection position that is the distance from the position corresponding to the rotation axis of the imaging device to the subject , and the distance measured by the distance measurement unit. Selecting a subject to be used for determining the tilt angle and the focus position from the plurality of subjects detected by the subject detection unit based on the reliability determined by the accuracy of the distance measurement result of the subject. death,
The determination unit determines the tilt angle and the focus position based on the distance measurement result of the selected subject ,
The imaging apparatus , wherein the reliability increases as the distance from the position corresponding to the rotation axis of the imaging element to the subject increases . 2. The imaging apparatus according to claim 1 , wherein the control unit selects two subjects with the highest reliability from among the plurality of subjects. The control unit adjusts the specific gravity of the detection size and detection position of the subject detected by the subject detection unit, and the precision of the distance measurement result of the subject measured by the distance measurement unit, according to the distance measurement method. 3. The image pickup apparatus according to claim 1 , wherein the image pickup apparatus is changed. The control unit
determining the correlation of the subject based on the detection position of the subject detected by the subject detection unit and the distance measurement result of the subject by the distance measurement unit;
4. The imaging apparatus according to any one of claims 1 to 3 , wherein a subject to be excluded from the plurality of subjects detected by the subject detection unit is determined based on the correlation. 5. The imaging apparatus according to claim 4 , wherein the Smirnoff-Grubbs test is used to determine the subject to be excluded. After calculating the focus position and the tilt angle, the control unit
When a new subject is detected in the image captured by the imaging device, or when it is determined that the subject in the image has moved,
3. The imaging apparatus according to claim 1 , wherein whether or not to re-determine the reliability is determined according to a distance measurement method. 7. The image pickup apparatus according to claim 6 , wherein the ranging method includes a phase difference method and a contrast method. 8. The imaging apparatus according to claim 7 , wherein the control unit re-determines the reliability when the distance measurement method is a phase difference method. When the number of subjects in the image captured by the imaging device is three or more, the control section controls the detection size and detection position of the subject detected by the subject detection section, and the distance measurement section. 2. A subject used for determining said tilt angle and said focus position is selected from said plurality of subjects based on at least one accuracy of said distance measurement result of said subject distanced. 9. The imaging device according to any one of 8 to 8 . 3. The imaging apparatus according to claim 1 , wherein the control unit determines the reliability for each subject in an image captured by the imaging apparatus. The imaging apparatus according to any one of claims 1 to 10, wherein the distance measurement result is a defocus amount. The control unit is characterized in that a frame is aligned for each of the plurality of subjects detected by the subject detection unit, and a subject frame used for determining the tilt angle and the focus position is selected from the frame. The imaging device according to any one of claims 1 to 11 . A tilting driving unit that tilts an imaging device to change a tilting angle, which is an angle between an imaging surface of the imaging device and an orthogonal plane perpendicular to an optical axis of the imaging optical system, and drives a focus lens of the imaging optical system. and a focus drive section for changing the focus position, wherein
a subject detection step of detecting a subject in an image captured by the imaging device;
a distance measurement step of measuring the distance to the subject detected by the subject detection step;
A detection size of the subject detected by the subject detection step, a detection position that is a distance from the position corresponding to the rotation axis of the imaging device to the subject, and a distance measurement of the subject that is measured by the distance measurement step. a selection step of selecting a subject to be used for determining the tilt angle and the focus position from the plurality of subjects detected by the subject detection step, based on the reliability determined by the accuracy of the result;
a determination step of determining the tilt angle and the focus position based on the distance measurement result of the distance measurement step of the subject selected in the selection step;
a control step of controlling the tilt driving section and the focus driving section based on the tilt angle and the focus position determined by the determining step ;
The control method , wherein the reliability increases as the distance from the position corresponding to the rotation axis of the imaging element to the subject increases . A computer-readable storage medium storing a computer program for causing a computer of an imaging device having an imaging element for imaging an optical image formed by an imaging optical system to execute processing according to the control method according to claim 13 . The control unit
generating a virtual frame if the confidence of the subject is lower than a threshold;
2. The imaging apparatus according to claim 1 , wherein the image height of the virtual frame and the subject distance information of the virtual frame are calculated from information of a plurality of subjects. 16. The imaging apparatus according to claim 15 , wherein the control unit generates the virtual frame by performing weighting according to the reliability of the subject. The imaging apparatus according to claim 15 or 16 , wherein the control unit selects a subject to be used for generating the virtual frame according to the vertical direction of the rotation axis of the tilt angle.

Images