JP2020129788A - Imaging apparatus, control method of the same, and program - Google Patents

Abstract

To provide an imaging apparatus capable of capturing a moving image in focus even when a plurality of subjects move while focusing properly on multiple subjects.SOLUTION: An imaging apparatus (100) includes: an optical unit (1001) having a lens (101) and an imaging sensor (102); detection means (103) for detecting a specific subject based on image data from the imaging sensor; setting means (104) for setting an object tilt axis by focusing on the specific subject; and tilt means (105) for tilt rotating the imaging sensor around a tilt rotation axis of the imaging sensor.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image pickup apparatus having a mechanism for tilting an image pickup sensor.

2. Description of the Related Art Conventionally, there is known an image pickup apparatus that tilts an image sensor with respect to an optical axis of a lens to perform shooting (tilt shooting). Patent Document 1 discloses an image pickup apparatus that adjusts the tilt of an image pickup element so as to focus on a plurality of designated areas and performs tilted image pickup. In Patent Document 2, by adjusting the tilt angle of the light receiving surface (imaging sensor) or the imaging lens according to the tilt angle of the surface (vertical surface) including the faces of the plurality of people, the faces of the plurality of people are focused. There is disclosed an image pickup device capable of adjusting the above.

JP, 2016-042194, A JP, 2017-093904, A

However, the image pickup device disclosed in Patent Document 2 adjusts the tilt angle of the light receiving surface or the image pickup lens so that the focus surface is aligned with the tilted vertical plane including the three faces. Generally, it is a special case when three faces are in the same vertical plane, and complicated processing is required when the three faces are not in the same vertical plane. Further, when considering a plane including four or more faces, there may be a case where there is no face including all the faces. Further, when a person moves and the inclination angle of the face changes, it is necessary to recalculate the inclination angle of the face and the inclination angle of the light receiving surface or the imaging lens in order to reduce the error. However, since it is difficult to change the tilt angle and the lens position in real time by following the moving face, it is not suitable for movie shooting.

Therefore, the present invention is capable of appropriately focusing on a plurality of subjects, and also capable of performing moving image shooting in focus even when a plurality of subjects move, an imaging device control method, Also, the purpose is to provide a program.

An image pickup apparatus according to one aspect of the present invention includes an optical unit having a lens and an image pickup sensor, a detection unit that detects a specific subject based on image data from the image pickup sensor, and a focus on the specific subject, It has a setting means for setting an object tilt axis and a tilt means for tilt-rotating the image sensor about a tilt rotation axis of the image sensor.

According to another aspect of the present invention, there is provided a method of controlling an image pickup apparatus, which comprises a step of detecting a specific subject based on image data from an image sensor, focusing on the specific subject, and an image formation relationship with the tilt rotation axis. Setting the object tilt axis, and tilting the image sensor about a tilt rotation axis of the image sensor.

A program as another aspect of the present invention causes a computer to execute the method for controlling the imaging device.

Other objects and features of the present invention will be described in the following embodiments.

ADVANTAGE OF THE INVENTION According to the present invention, it is possible to properly focus on a plurality of subjects, and even when a plurality of subjects move, it is possible to capture a moving image in focus, and an imaging device control method. , And a program can be provided.

It is a block diagram of the imaging device in a 1st embodiment. 6 is a flowchart illustrating a method for controlling the image pickup apparatus according to the first embodiment. It is explanatory drawing of the control method of the imaging device in 1st Embodiment. It is explanatory drawing of the captured image in each embodiment. It is explanatory drawing of the effect of each embodiment. It is explanatory drawing of the effect of each embodiment. It is a block diagram of an imaging device in a 2nd embodiment. 9 is a flowchart showing a method for controlling the image pickup apparatus according to the second embodiment. It is explanatory drawing of the focus surface set by the imaging device in 2nd Embodiment. It is explanatory drawing of the control method of the imaging device in 3rd Embodiment. It is explanatory drawing of the object tilt axis|shaft set by the imaging device in 4th Embodiment. It is explanatory drawing of the focus surface set by the imaging device in 4th Embodiment. It is explanatory drawing of the focus surface set by the imaging device as a comparative example.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

First, shooting (tilt shooting) performed by tilting the image sensor with respect to the optical axis of the lens will be described. When the image sensor is not tilted (not tilted), the image pickup surface of the image sensor is parallel to the lens main plane, and the image pickup surface of the image sensor and the lens main plane are parallel to each other (a plane perpendicular to the optical axis). Focuses on the subject existing in. In this way, when a subject existing on a certain virtual surface can be focused, this virtual surface is called a focus surface. By raising the image sensor, it is possible to focus on an object on a surface other than the surface perpendicular to the optical axis, that is, to tilt the focus surface arbitrarily (Scheinproof principle). When the image sensor is tilted, the image sensor is rotated around a rotation axis that supports the image sensor by a predetermined angle (amount of tilt). In each of the following embodiments, the rotation axis (tilt rotation axis) of the image sensor is referred to as “imaging tilt axis”. Further, as described above, the focus surface can be tilted by tilting the image sensor, that is, the focus surface can be rotated around a predetermined rotation axis. In each of the following embodiments, the rotation axis of the focus plane is referred to as "object tilt axis".

When the surface of the image sensor is perpendicular to the optical axis, that is, when the surface does not move, the object is focused only on a subject having a specific distance in the optical axis direction. That is, since the focus plane is a plane perpendicular to the optical axis, it is difficult to focus from a subject in the foreground to a subject in the back in a scene with depth. On the other hand, by tilting (moving) the image sensor with respect to the optical axis, the focus surface can be tilted with respect to the optical axis, and the focus surface can be a surface other than the surface perpendicular to the optical axis. In this case, it is possible to focus on subjects having different distances in the optical axis direction. That is, it is possible to focus on a surface where a distant subject and a near subject exist. For this reason, by moving the image sensor according to the distribution of the object distance within the angle of view, it is possible to focus from a subject in the foreground to a subject in the back, as compared with the case where the image sensor is not moved. That is, it is possible to focus on a subject in a wider range in the depth direction (ground direction) by inclining the focusing surface on the object (subject) side in the depth direction (to approach the ground).

(First embodiment)
First, an image pickup apparatus according to the first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram of an image pickup apparatus 100 according to this embodiment.

The optical unit 1001 has a lens (imaging optical system) 101 and an imaging sensor (imaging element) 102. The optical unit 1001 captures input light such as reflected light of a subject, converts the light into an image signal, outputs the image signal as an output image of the imaging device 100, and outputs the image signal to the subject detection unit (detection unit) 103. The lens 101 collects the input light and forms a subject image on the image sensor 102. Further, the lens 101 may be composed of a plurality of lenses, and in the present embodiment, by moving the position of the focus lens included in the lens 101, an object located at a predetermined object distance is focused. be able to. The image sensor 102 generates an image signal of the subject by converting the subject image formed on the image pickup surface formed of a plurality of two-dimensionally arranged image pickup elements into an electric signal, and outputs image data. Further, the image sensor 102 can be tilted by a predetermined tilt angle (tilt angle) in the rotation direction about the image tilt axis.

The subject detection unit 103 acquires image data from the image sensor 102, detects a specific subject existing at a desired height, and sets the position (coordinates) of the detected subject in an object tilt axis setting unit (setting unit) 104. Output. The specific subject is, for example, a human face. Depending on the angle of view, the height of a person's face while walking and the height of a traffic light on the road are almost constant, so by detecting a specific subject such as a person's face, An existing subject can be detected.

The object tilt axis setting unit 104 compares the position of the specific subject acquired from the subject detection unit 103 with the position of the imaging tilt axis to determine whether or not a subject having a desired height exists on the imaging tilt axis. It is determined and the determination result is output to the sensor tilt unit (tilt means) 105. For example, when the image area based on the position information of the specific subject overlaps the imaging tilt axis, the object tilt axis setting unit 104 determines that a subject having a desired height exists on the imaging tilt axis. When the object tilt axis setting unit 104 determines that a subject having a desired height exists on the imaging tilt axis, it drives the lens 101 to focus on the subject, thereby setting the object tilt axis to the subject. To do.

When the sensor tilt unit 105 obtains from the object tilt axis setting unit 104 a determination result that a subject having a desired height exists on the image tilt axis, the sensor tilt unit 105 tilts and rotates the image sensor 102 around the image tilt axis. By tilting the image sensor 102 around the image tilt axis, the focus plane rotates around the object tilt axis on the subject side as well, and includes a subject (face or the like) having a desired height and is substantially parallel to the ground plane. Rotate the focusing surface until it matches.

With such a configuration, the image pickup apparatus 100 of the present embodiment detects a plurality of subjects (specific subjects such as a plurality of faces) having a desired height within the screen. When the imaging device 100 detects a subject (a specific subject such as a face) having a desired height on the imaging tilt axis, the imaging apparatus 100 sets the object tilt axis by focusing on the detected subject. Then, the image pickup apparatus 100 causes the image sensor 102 to tilt and rotate around the image pickup tilt axis, so that the subject side also rotates the focus plane around the object tilt axis. At that time, the focusing surface is rotated so that the horizontal surface including the subject (face or the like) having a desired height and substantially parallel to the ground surface and the focusing surface substantially match each other.

Next, with reference to FIG. 2, a control method of the image pickup apparatus 100 according to the present embodiment will be described. FIG. 2 is a flowchart of the control method of the image pickup apparatus 100. Each step of FIG. 2 is executed by the subject detection unit 103, the object tilt axis setting unit 104, or the sensor tilt unit 105 of the imaging device 100.

First, in step S01, the subject detection unit 103 acquires image data (input image) from the image sensor 102, and detects a specific subject (human face or the like) present at a desired height from the input image. Then, the subject detection unit 103 calculates the position of the image region (subject region) of the detected specific subject.

Subsequently, in step S02, the object tilt axis setting unit 104 includes at least part of the position of the imaging tilt axis of the imaging sensor 102 in the input image in at least part of the position of the subject region detected in step S01. Or not. That is, the object tilt axis setting unit 104 determines whether or not there is a subject having a desired height on the imaging tilt axis. When there is no subject having a desired height on the imaging tilt axis, the process returns to step S01, and the object detection unit object tilt axis setting unit 104 performs the processes of steps S01 and S02 on the input image of the next frame and subsequent frames. On the other hand, when there is a subject having a desired height on the imaging tilt axis, the process proceeds to step S03. In step S03, the object tilt axis setting unit 104 moves the lens 101 (focus lens) so that the object area on the imaging tilt axis is in focus. After focusing on the subject area, the process proceeds to step S04.

In step S04, the sensor tilt unit 105 calculates a sensor tilt rotation amount (sensor tilt amount) required to rotate the focus surface to a horizontal plane substantially parallel to the ground. Then, the sensor tilt unit 105 tilts and rotates the imaging sensor 102 around the imaging tilt axis by the amount of sensor tilt rotation (sensor tilt amount).

Next, with reference to FIG. 3, a method of rotating the focus plane by the sensor tilt unit 105 to a horizontal plane substantially parallel to the ground (reference plane) will be described. That is, a method of calculating the sensor tilt rotation amount (sensor tilt amount) when the sensor tilt is performed so as to rotate the focus surface to a horizontal plane substantially parallel to the ground (reference plane) will be described. In FIG. 3, the object tilt axis setting unit 104 detects a subject (face of the subject 10) having a desired height in a captured image on the imaging tilt axis, and the lens 101 (focusing) is performed so that the face of the subject 10 is focused. The lens) is moved.

Here, when the subject distance to the subject (face of the subject 10) having a desired height is A, the focal length of the lens 101 is f, and the angle formed by the optical axis OA and the perpendicular PL to the ground (reference plane) is θ. , The tilt angle S _p can be calculated by the following equation (1). That is, when the tilt angle Sp is set to satisfy the expression (1), the sensor tilt rotation amount (sensor tilt amount) can be calculated. In this way, by controlling the sensor tilt rotation amount (sensor tilt amount), the focus plane is made to coincide with a horizontal plane that includes a subject (human face, etc.) of a desired height and is substantially parallel to the ground (reference plane). be able to.

The angle θ formed by the optical axis OA and the perpendicular PL to the ground (reference plane) may be calculated and measured when the image pickup apparatus 100 is installed. However, an acceleration sensor provided in the image pickup apparatus 100 or a distance measuring unit. You may calculate based on the measured value of a sensor. Note that the subject distance A can be calculated using a lens formula based on a value B determined according to the position of the lens 101 at the time of focusing shown in FIG. You may calculate based on the measured value of the distance measuring sensor provided.

Next, the effect of this embodiment will be described. First, a comparative example will be described with reference to FIG. FIG. 13 is an explanatory diagram of a focus plane set by the image pickup apparatus 300 as a comparative example. In this comparative example, the tilt angle of the light receiving surface or the imaging lens is adjusted based on the tilt angle of the face. Therefore, when the subjects (faces) 41 and 42 are distributed as shown in FIG. 13A, the focus plane as shown in FIG. 13A is set. On the other hand, when the subjects (faces) 41 and 42 are distributed as shown in FIG. 13B, the focus plane as shown in FIG. 13B is set.

The difference between the tilt angle of the focus surface shown in FIG. 13A and the tilt angle of the focus surface shown in FIG. 13B is large. Therefore, as shown in FIG. 13C, when the subjects 41 and 42 move while being fixed at the position of the focus plane of FIG. 13A, the position of the focus plane and the positions of the subjects (faces) 41 and 42. And the captured image becomes blurry. In order to prevent the captured image from being blurred, it is necessary to change the position of the focus surface in real time by following the movement of the subjects (faces) 41 and 42. However, since changing the focus plane involves mechanical fluctuations, it is difficult to change the position of the focus plane in real time, which is not suitable for capturing a moving image.

Next, with reference to FIG. 4, an image (captured image) acquired by the image capturing apparatus 100 according to the present embodiment will be described. FIG. 4 is an explanatory diagram of a captured image according to this embodiment. A broken-line rectangular area in FIG. 4 shows an example of detection of subjects (specific subjects such as faces) 10 and 11 having a desired height. FIG. 4 shows a state in which a subject (face of the subject 10) having a desired height is detected on the object tilt axis and a subject (face of the subject 11) having a desired height behind the object is detected. ing. Further, FIG. 4A shows an imaged image before the image sensor 102 is tilted (moved), and FIG. 4B shows an imaged image after the image sensor 102 is tilted (moved).

In the case of FIG. 4, a subject (a specific subject such as a face) having a desired height exists on the object tilt axis. Therefore, the imaging apparatus 100 sets the object tilt axis by focusing on the subject. In the state before performing the sensor tilt (tilt), the focus surface is a vertical surface, so that the whole body of the subject 10 is in focus as shown in FIG.

Then, the imaging sensor 102 is raised by a predetermined amount so that the face of the person in the back (the subject 11) is also in focus, thereby forming a focus surface on a horizontal plane at the height of the face. As a result, as shown in FIG. 4B, both the face of the person in the center (subject 10) and the face of the person in the back (subject 11) are in focus. Since the height of the face does not change even if a person moves, the face cannot be out of focus even if a person moves.

Next, the effect of this embodiment will be further described with reference to FIG. FIG. 5 is an explanatory diagram of the effect of this embodiment. In the image pickup apparatus 100 of the present embodiment, the position of the object tilt axis is set to the detected face position by focusing on the face (the subject 12 in FIG. 5A) detected on the image tilt axis. Set. Then, the inclination of the focusing surface is not controlled based on the distribution of the position of the face, but is fixed to an inclination that is substantially parallel to the reference plane (ground) as shown in FIG. With the above control, the focus surface is fixed to a plane that is substantially parallel to the reference plane and that is the height of the face from which the height from the reference plane is detected. Therefore, as shown in FIG. 5B, even if the subject 12 or 13 moves and the position of the face changes, the error between the position of the focus surface and the position of the face is the error E of the height of the face. The above can be prevented.

In the present embodiment, the process of setting the position of the focus surface is performed as pre-setting (when the camera is installed, etc.) before shooting, and the position of the focus surface is fixed during shooting. Therefore, it is not necessary to recalculate the position of the focus surface each time the position of the face changes, or to drive the mechanical member to change the position of the focus surface. Therefore, it is possible to shoot a smooth moving image while preventing the faces of all people from being blurred.

Next, with reference to FIG. 6, the effect of the present embodiment will be supplemented. FIG. 6 is an explanatory diagram of the effect of this embodiment. As shown in FIG. 6A, when the face of the subject O1 (adult) is detected in the captured image on the imaging tilt axis, the subject distance is A1 by focusing on the face of the subject O1 (adult). Becomes As a result, the position of the object tilt axis becomes the position of the object tilt axis T1. Further, by controlling the tilt angle of the image sensor 102 so that the focusing plane becomes substantially parallel to the reference plane, the position of the focusing plane becomes the focusing plane F1.

On the other hand, as shown in FIG. 6B, when the face of the subject O2 (child) is detected in the captured image on the imaging tilt axis, the subject distance is adjusted by focusing on the face of the subject O2 (child). Is A2. As a result, the position of the object tilt axis becomes the position of the object tilt axis T2. Further, by controlling the tilt angle of the image sensor 102 so that the focusing plane becomes substantially parallel to the reference plane, the position of the focusing plane becomes the focusing plane F2. In this way, by determining the position of the object tilt axis according to the detected subject distance of the subject, the position of the focus plane can be set according to the height of the subject.

Although the subject detection unit 103 detects a human face as the specific subject in the present embodiment, the method of determining the subject is not limited to this. For example, the captured image and the marker representing the position of the captured tilt axis may be displayed on a viewer operating on the PC, and the user may manually specify the subject region on the viewer using a mouse or the like. That is, a subject having a desired height (such as a person's face) and a subject having a desired height (such as a person's face) existing behind or in front of the imaging tilt axis may be manually designated as areas. ..

Further, in the present embodiment, the subject detection unit 103 detects a human face as the specific subject, but the specific subject to be detected is not limited to this. If the subject has a constant height from the reference plane (road or floor), such as traffic lights on the road, traffic signs, car license plates, tables in restaurants, desks in classrooms at schools and cram schools, It can be detected as a specific subject. Further, a unit that allows the user to set an object to be detected as the specific subject may be further provided.

Further, in the present embodiment, the object tilt axis setting unit 104 calculates the tilt angle S _p based on the above equation (1), but the method of calculating the tilt angle is not limited to this. The tilt angle calculated based on the equation (1) may be held in advance as table data, and the tilt angle corresponding to the angle θ and the subject distance A may be read from the table data. Further, the mathematical formula for calculating the tilt angle is not limited to the formula (1), and for example, a formula obtained by simplifying the formula (1) may be used. Further, the tilt angle S _p may be obtained without using a mathematical formula. For example, the tilt angle S _p obtained experimentally may be held as table data.

Further, in the present embodiment, the position of the tilt angle S _p of the image sensor 102 is searched and set so that the contrast value of the face of the subject 11 (the person behind the subject 10) shown in FIG. The surface may be formed. That is, for a plurality of face image areas of a predetermined size or more detected in the captured image, the contrast is evaluated for each face image area, so that the contrast evaluation value of more face image areas is near the peak, The image sensor tilt angle may be calculated.

Further, in the present embodiment, a unit that allows manual adjustment of the tilt angle S _p of the image sensor 102 is provided, and an image is picked up by the user while visually checking the picked-up image so that both the face of the subject 10 and the face of the subject 11 are focused. The tilt angle S _p of the sensor 102 may be manually adjusted.

Next, an example of a use case assumed in this embodiment will be described. For example, a surveillance camera is a use case suitable as an image pickup apparatus to which the present embodiment is applied. The surveillance camera needs to capture a moving image. Further, since the surveillance camera sets various settings for shooting at the time of installation and the shooting is automatically performed unattended, the settings of the surveillance camera are rarely changed during shooting. Since the surveillance camera often photographs a specific subject according to the environment in which it is fixedly installed, the structure of the subject rarely changes. Therefore, the position and inclination angle of the reference plane such as the ground or road does not change much. For this reason, the height of the specific subject (for example, the face of a person) rarely changes significantly during shooting. Since the shooting environment of the surveillance camera has the characteristics as described above, even if the height of the focusing surface is fixed to the height of the face in advance as in the image pickup apparatus according to the present embodiment, the height of the focusing surface is increased. And blurring due to face height error are unlikely to occur. That is, an image in which a specific subject (face) existing in a wide range from the front to the back is in focus can be obtained.

(Second embodiment)
Next, a second embodiment of the present invention will be described. In the present embodiment, when a subject (a specific subject such as a human face) having a desired height is not detected on the imaging tilt axis, the optical axis of the imaging device is rotated using the pan-tilt function of the imaging device such as a surveillance camera. .. More specifically, the optical axis is rotated so that the object of the desired height closest to the imaging tilt axis out of the detected plural objects of desired height (face, etc.) overlaps the imaging tilt axis. Then, the position of the subject having the desired height on the angle of view is moved. Then, focusing is performed so that the subject having a desired height moved on the imaging tilt axis is in focus, and the object tilt axis is set. After that, by tilting the image sensor around the image tilt axis, the subject side also rotates the focus plane around the object tilt axis to form a horizontal plane substantially parallel to the ground including the object (face, etc.) of the desired height. Match the focus surface.

Hereinafter, the imaging device 100a according to the present embodiment will be described with reference to FIG. FIG. 7 is a block diagram of the imaging device 100a. The same components as those in the first embodiment are designated by the same reference numerals and the description thereof will be omitted.

The optical unit 2001 has a pan-tilt mechanism 201, and the tilt of the optical axis of the optical unit 2001 can be changed by driving the pan-tilt mechanism 201 based on the pan-tilt control information acquired from the object tilt axis setting unit 204. The object tilt axis setting unit 204 compares the position of the specific object acquired from the object detection unit 103 with the position of the imaging tilt axis to determine whether or not there is a subject of a desired height on the imaging tilt axis. Then, the determination result is output to the sensor tilt unit 105. For example, when the image area based on the position information of the specific subject overlaps the imaging tilt axis, the object tilt axis setting unit 204 determines that a subject having a desired height exists on the imaging tilt axis. When it is determined that there is a subject having a desired height on the imaging tilt axis, the object tilt axis setting unit 204 drives the lens 101 to focus the object, thereby setting the object tilt axis as the object.

On the other hand, when the object tilt axis setting unit 204 determines that there is no subject having a desired height on the imaging tilt axis, it drives the pan-tilt mechanism 201 of the optical unit 2001 to change the tilt of the optical axis of the optical unit 2001. .. The tilt of the optical axis is changed so as to move the subject of a desired height on the imaging tilt axis.

With such a configuration, the imaging apparatus 100a of the present embodiment detects a plurality of subjects (specific subjects such as faces) having a desired height within the screen. Then, when a subject (a specific subject such as a face) having a desired height is detected on the imaging tilt axis, the imaging device 100a performs focusing on the object and sets the object tilt axis. On the other hand, when the imaging device 100a determines that there is no subject having a desired height on the imaging tilt axis, it drives the pan-tilt mechanism 201 of the optical unit 2001 to change the tilt of the optical axis of the optical unit 2001 to perform imaging tilt. A subject of desired height is moved on the axis. Then, focusing is performed on the subject having a desired height moved on the imaging tilt axis to set the object tilt axis. After that, the imaging device 100a rotates the image sensor in the tilt plane around the imaging tilt axis to rotate the focus plane around the object tilt axis on the subject side as well. At this time, the focusing surface is rotated so that the horizontal surface including the subject (face, etc.) having a desired height and substantially parallel to the ground surface and the focusing surface substantially match each other.

Next, with reference to FIG. 8, a control method of the image pickup apparatus 100a according to the present embodiment will be described. FIG. 8 is a flowchart of the control method of the imaging device 100a. Each step in FIG. 8 is executed by the subject detection unit 103, the object tilt axis setting unit 204, the sensor tilt unit 105, or the pan tilt mechanism 201 of the imaging device 100a. In FIG. 8, the same steps as those in the first embodiment are designated by the same reference numerals, and the description thereof will be omitted.

In step S12, the object tilt axis setting unit 204 determines that at least a part of the position of the imaging tilt axis of the imaging sensor 102 in the input image is at least the position of the subject region (for example, a human face) detected in step S01. It is determined whether it is included in a part. That is, the object tilt axis setting unit 204 determines whether or not there is a subject having a desired height on the imaging tilt axis. When a subject having a desired height exists on the imaging tilt axis, the process proceeds to step S03. On the other hand, when there is no subject having a desired height on the imaging tilt axis, the process proceeds to step S15.

In step S15, the object tilt axis setting unit 204 controls the pan-tilt mechanism 201 of the optical unit 2001 so that the direction of the optical axis of the optical unit 2001 (the imaging device is adjusted so that the subject area having a desired height overlaps the imaging tilt axis. The angle of 100a) is changed. Then, the process returns to step S12. In the present embodiment, the pan/tilt mechanism 201 of the optical unit 2001 is used as a means for rotating the optical axis of the image pickup apparatus 100a, but the invention is not limited to this. The optical axis of the image pickup apparatus 100a may be changed by using other means such as manually changing the angle of the image pickup apparatus 100a (changing the camera angle) or changing the image pickup mechanism.

Next, the effect of this embodiment will be described with reference to FIG. FIG. 9 is an explanatory diagram of the focus plane set by the image capturing apparatus 100a, and shows an example of moving the object tilt axis. Here, it is assumed that the specific subject is a region from the human chest to the face (face region, the height at the intermediate point is 1.5 m), and the focus surface is desired to be aligned with a horizontal plane having a height of 1.5 m. On the object tilting axis (imaging tilting axis) in FIG. 9A, not the face area of the person of the subject 21, but the foot area overlaps. In this state, the sensor tilt cannot adjust the focus plane to the horizontal plane of 1.5 m in height, so it is necessary to move the object tilt axis to the face area of the person of the subject 21.

Therefore, in the present embodiment, as shown in FIG. 9B, the pan/tilt function of the pan/tilt mechanism 201 of the image pickup apparatus 100a is used to raise the optical axis of the image pickup apparatus 100a (angle up) to perform image pickup tilt. The axis is superimposed on the face area of the person of the subject 21. Then, focusing is performed on the face area of the subject 21, and an object tilt axis is set on the face of the subject 21. Thereafter, the image sensor 102 is swung until the face of the subject 22 in the front or the face of the subject 23 in the back comes into focus. As a result, the focus surface is in focus on the horizontal plane of 1.5 m at the face height, and the face areas of all persons are in focus. Further, even if a person moves, the person's face is not out of focus.

(Third Embodiment)
Next, a third embodiment of the present invention will be described. Similar to the second embodiment, the present embodiment uses the pan-tilt function to detect the image pickup apparatus by using the pan-tilt function when an object having a desired height (a specific object such as a face) is not detected on the imaging tilt axis at the initial angle of view. Rotate the optical axis. On the other hand, in the present embodiment, unlike the second embodiment, after the object tilt axis is set to the position of the subject (for example, face) having a desired height by using the pan-tilt function of the imaging device, and the focus plane is set. , Use the pan/tilt function again to restore the original angle of view.

Further, the image pickup apparatus according to the present embodiment adjusts the position of the focus lens and the tilt of the image pickup sensor according to the change amount of the angle of view so as to maintain a substantially horizontal focus surface having a desired height from the set ground surface (reference plane). Change the angle (orientation angle). In other words, when the angle of view is returned, the position of the object tilt axis is moved to a virtual position on the substantially horizontal focus plane of the desired height and where there is no actual subject of the desired height. Let As a result, while maintaining the desired angle of view (original angle of view) set by the user, even if the object of the desired height does not exist on the imaging tilt axis while maintaining the initial angle of view, the desired height The object tilt axis can be set at a virtual position of.

The image pickup apparatus according to this embodiment will be described below. The image pickup apparatus according to the present embodiment is the same as the image pickup apparatus 100a described in the second embodiment with reference to FIG. 7, and thus description thereof will be omitted. In the present embodiment, the object tilt axis setting unit 204, as in the second embodiment, when the inclination of the optical axis of the optical unit 2001 (optical axis direction) is the first inclination (first direction), It is determined whether or not a subject having a desired height exists on the imaging tilt axis. If the object tilt axis setting unit 204 determines that there is no subject having a desired height on the imaging tilt axis, it drives the pan-tilt mechanism 201 of the optical unit 2001 to set the tilt of the optical axis of the optical unit 2001 to the second tilt. Change to (second direction). The second tilt of the optical axis is changed so as to move the subject having a desired height on the imaging tilt axis. Then, if it is determined that a subject having a desired height exists on the imaging tilt axis, the lens 101 is driven to adjust the focus to the subject, thereby setting the object tilt axis as the subject.

The sensor tilt unit 105 causes the imaging sensor 102 to tilt and rotate about the image tilting axis, thereby rotating the focus plane around the object tilting axis on the subject side as well. At that time, the focusing surface is rotated so that the horizontal surface including the subject (face or the like) having a desired height and substantially parallel to the ground surface and the focusing surface substantially match each other. Then, in the present embodiment, unlike the second embodiment, the object tilt axis setting unit 204 drives the pan-tilt mechanism 201 of the optical unit 2001 again to set the tilt of the optical axis of the optical unit 2001 to the first tilt. Return to. At this time, the position of the focus lens and the tilt angle (tilt) of the image sensor are adjusted according to the change of the tilt of the optical axis so as to maintain a substantially horizontal focusing surface having a desired height from the preset ground (reference plane). Change) and. It should be noted that a specific amount of change between the two will be described later. By changing the position of the focus lens and the tilt angle (tilt angle) of the image sensor, the position of the object tilt axis is moved on a substantially horizontal focus plane having a desired height.

With the above configuration, even if the angle of view is returned, as shown in FIG. 10D, even if there is no specific subject on the imaging tilt axis, the peripheral specific subject (for example, face) can be focused. , The position of the object tilt axis can be moved. That is, even when the specific subject is not imaged on the imaging tilt axis, the plane including the specific object is focused while the position of the object tilt axis is set to a virtual position on the substantially horizontal focus plane at a desired height. Can be matched. In other words, as shown in FIG. 10D, the specific subject (face) is set on the plane including the specific subject (face) while setting the object tilt axis at a virtual position where the specific subject does not exist. You can focus on the plane containing it.

Hereinafter, with reference to FIGS. 10A to 10D, the position of the focus lens and the tilt angle of the image sensor are changed according to the change of the tilt of the optical axis while maintaining the substantially horizontal focus surface having a desired height. A method of changing (orientation angle) and will be described. 10A to 10D are explanatory diagrams of the control method of the image pickup apparatus according to the present embodiment.

First, as shown in FIG. 10A, consider a case where a face is not imaged on the imaging tilt axis in a state where the focus plane is aligned with a reference plane (for example, the ground). The tilt angle (reference tilt angle) θ of the focusing surface is expressed by the following equation (2).

Next, as shown in FIG. 10B, the image pickup apparatus 100a performs tilt drive using the pan-tilt mechanism 201 so that a face is picked up on the image pickup tilt axis, and the image pickup apparatus 100a is moved in the optical axis direction. change. At this time, if the tilt angle of the image pickup apparatus 100a is φ, the subject distance a′ and the tilt angle S′ are represented by the following equations (3) and (4), respectively.

Then, as shown in FIG. 10C, the subject distance A′ can be calculated by moving the focus lens so that the face on the imaging tilt axis is in focus. The tilt angle Sp' at this time is expressed by the following equation (5). This allows the position of the object tilt axis to be moved to the position of the specific object at the initial angle of view shown in FIG. 10A even when the specific object does not exist on the imaging tilt axis.

Finally, as shown in FIG. 10D, the image pickup apparatus 100a performs tilt drive using the pan-tilt mechanism 201 to return the optical axis direction to the optical axis direction shown in FIG. As a result, the subject distance A _r can be calculated geometrically and is expressed by the following equation (6).

Then, the position B _{r of the} focus lens is determined as in the following Expression (7) based on the subject distance A _r .

Furthermore, by setting the tilt angle S _r as in the following expression (8), the focus surface can be set so that a virtual specific subject (for example, face) is focused on the imaging tilt axis. With this, it is possible to focus on the plane including the specific subject while setting the position of the object tilt axis to the position of the virtual specific subject (face) on the substantially horizontal focusing surface of the desired height. In other words, as shown in FIG. 10A, even when the face of the specific subject is not imaged on the imaging tilt axis, the angle of view shown in FIG. It is possible to raise a substantially horizontal focusing surface of a desired height so that a plane including the above) is in focus.

According to the present embodiment, it is possible to focus on a specific subject that is not imaged on the tilt axis at a predetermined angle of view in a wide range from the front to the back while maintaining the predetermined angle of view. Become.

(Fourth Embodiment)
Next, a fourth embodiment of the present invention will be described. In the present embodiment, when a subject (a specific subject such as a face) having a desired height is not detected on the imaging tilt axis, focus is performed on the position of the subject having a desired height detected in an imaging region other than the imaging tilt axis. Set the distance A. As a result, the object tilt axis is set at the position (the position on the optical axis) of the intersection of the perpendicular line drawn from the position of the specific subject having the desired height to the optical axis and the optical axis. Then, the image pickup surface is rotated by a predetermined angle S tilt so that the optical axis vertical focus surface rotates by a predetermined angle ω. At this time, a focus shift occurs in a subject other than the imaging tilt axis. Therefore, refocusing for correcting the shift is executed (focus distance A′ is set), and another object tilt axis is set on the optical axis. Finally, the focus distance A″ is set so that the object focusing surface is horizontal with the ground, and the object tilting axis is set on the optical axis. Rotate to S''. As a result, the focus surface is horizontal, so no focus shift occurs (refocusing is unnecessary, the final object tilt axis is set on the optical axis), and the focus surface including the subject (face, etc.) It becomes a horizontal focus surface that is substantially parallel to the ground.

Unlike the first to third embodiments, the present embodiment performs focusing so that an object (specific object such as a face) having a desired height detected in an imaging region other than the imaging tilt axis is in focus, Set the object tilt axis on the optical axis. After that, the image sensor is tilted around the imaging tilt axis, and refocused to correct the focus shift of the object at the desired height, and another object tilt axis is set on the optical axis. To do. This sensor tilt rotation about the image tilting axis and refocusing (setting another object tilting axis on the optical axis) to correct the focus shift for a subject having a desired height initially focused are repeated. Then, finally, the horizontal plane substantially parallel to the ground including the subject (face etc.) of a desired height is made to coincide with the focus plane. In other words, unlike the second and third embodiments, the present embodiment rotates the optical axis of the image pickup apparatus by the pan-tilt function when a subject of a desired height is not detected on the image pickup tilt axis at the initial angle of view. Without doing this, the focus surface is adjusted to a surface parallel to the ground, including the subject (face, etc.) of the desired height.

The image pickup apparatus according to this embodiment will be described below. The block diagram of the image pickup apparatus according to the present embodiment is the same as the image pickup apparatus 100 described in the first embodiment with reference to FIG. In the following, among the functional blocks forming the block diagram shown in FIG. 1, only functional blocks having functions different from those of the first embodiment will be described.

In the present embodiment, the object tilt axis setting unit 104 sets the object tilt axis by focusing on the position of the specific subject acquired from the subject detection unit 103. That is, when the specific subject does not exist in the image pickup area on the image pickup tilt axis, the object tilt axis setting unit 104 sets the object at the position of the intersection of the perpendicular line drawn from the position of the focused particular object to the optical axis and the optical axis. Set the tilt axis. Then, the object tilt axis setting unit 104 outputs the tilt angle of the image sensor to the sensor tilt unit 105 according to a predetermined focus surface rotation amount, and drives the lens 101 according to the predetermined focus surface rotation amount. Correct the focus. More specifically, the tilt angle of the image sensor and the subject distance to be focused are calculated so that the focused subject can be continuously focused even if the focusing surface is rotated by a predetermined amount. ,Output. In other words, in the present embodiment, while rotating the focus surface, the position of the object tilt axis is shifted according to the rotation of the focus surface. A method for calculating the tilt angle of the image sensor and the correction amount of the subject distance for focusing will be described later.

The sensor tilt unit 105 tilts and rotates the image sensor 102 around the image tilt axis according to the tilt angle of the image sensor acquired from the object tilt axis setting unit 104. By tilting the image sensor 102 around the image tilt axis, the focus plane rotates around the object tilt axis on the subject side as well, and includes a subject (face or the like) having a desired height and is substantially parallel to the ground plane. Rotate the focusing surface until it matches.

Hereinafter, the method for setting the object tilt axis and the method for controlling the focus plane in this embodiment will be further described with reference to FIGS. 11 and 12.

FIG. 11A shows an example of the position of the object tilt axis in this embodiment. In FIG. 11A, the object tilt axis is set to a position on the optical axis where the optical axis intersects the perpendicular line drawn from the position of the face of the subject 10 to the optical axis by focusing on the position of the face of the subject 10. Has been done. Unlike the first to third embodiments, in the present embodiment, the position of the specific subject does not have to have an imaging relationship with the position of the imaging tilt axis as shown in FIG. That is, the specific subject may not be imaged in the imaging area on the imaging tilt axis.

FIG. 11B shows an example of a captured image in this embodiment. In FIG. 11B, a face is detected from the captured image and the face of the subject 10 is focused so that no specific subject (face) exists, but the object tilt axis is located at a position having an imaging relationship with the image tilt axis. Represents the state of setting. P _y in FIG. 11 (b) in the number of pixels in the vertical direction of the captured image, P _f in FIG. 11 (b) is the position of the face of the combined position and the focus corresponding to the optical axis of the captured image It represents the number of pixels, which is the distance between them.

In such a case, the object tilt axis setting unit 104 calculates the distance between the optical axis and the position of a specific subject such as a focused face, which is indicated by H in FIG. 11A. That is, the distance H between the focused specific subject and the object tilt axis is calculated. First, at a subject distance A, which is indicated by V in FIG. 11A, falls within the angle of view when the focal length of the lens 101 is f and the vertical size (imaging length) y of the image sensor is y. The width of the field of view is expressed by the following equation (9).

Then, the distance H between the position of a specific subject such as a focused face and the optical axis is represented by the following expression (10) using V.

In FIG. 12A, a specific subject (face) does not exist, and a position having an imaging relationship with the position of the imaging tilt axis is set as the object tilt axis, and the focus surface is rotated around the position of the face of the object 10. It shows the state of being done. That is, in FIG. 12A, the focusing surface is rotated by an angle ω about the position of the face of the subject 10 with respect to the focusing surface of FIG. The state is shown. Furthermore, FIG. 12A shows a state in which the position of the object tilt axis is corrected by A ₊ so that the focus surface is rotated while the face of the subject 10 is being focused. That is, the virtual subject distance A′ in FIG. 12A is the subject distance A in the state where the image sensor shown in FIG. 11A is not covered, corrected by A ₊ . The virtual subject distance A′ is expressed by the following equation (11) using the distance H between the position of a specific subject such as a focused face and the optical axis and the focus plane rotation amount ω. ..

The tilt angle S at this time is expressed by the following equation (12) using the virtual subject distance A′ and the focus plane rotation amount ω.

FIG. 12B shows a state in which the focus surface is rotated about the position of the face of the subject 10 until the focus surface substantially coincides with the horizontal plane at the height of the face of the subject 10. As shown in FIG. 12B, when the angle formed by the perpendicular to the horizontal plane and the optical axis is θ _H , when the focus surface rotation amount ω=θ _H , the focus surface coincides with the horizontal plane at the face height. To do. The virtual subject distance A′ at this time is represented by the following equation (13), and the tilt angle S is represented by the following equation (14).

With the above configuration, the position of the object tilt axis, which has an image-forming relationship with the position of the image tilt axis, is determined according to the distance between the position of a specific subject such as a focused face and the optical axis, and the focus plane rotation amount. to correct. As a result, the focus surface can be rotated around the position of the face that is not in the image formation relationship with the position of the imaging tilt axis. As a result, even when the specific subject is not imaged in the imaging region on the imaging tilt axis, it is possible to focus on a substantially horizontal plane including the specific subject and having a desired height. That is, an image in which a specific subject (face) existing in a wide range from the front to the back is in focus can be obtained.

When θ _H , which is the angle formed by the perpendicular to the horizontal plane and the optical axis, is known based on the camera installation conditions and the like, the focus plane rotation amount ω=θ _H , as shown in FIG. 12B. Therefore, the focus plane may be made to substantially coincide with the horizontal plane in one step. On the other hand, when θ _H is an unknown angle between the perpendicular to the horizontal plane and the optical axis, the process of rotating the focus surface by an arbitrary focus surface rotation amount is repeated a plurality of times. That is, the focus surface rotation amount ω may be gradually changed until an image in which a specific subject (face) existing in a wide range from the front to the back is focused. That is, the method of setting the focus surface rotation amount ω may be switched depending on whether θ _H is a known angle formed by the perpendicular to the horizontal plane and the optical axis.

In the example described above, the object tilt axis setting unit 104 calculates the focus correction amount and the sensor tilt angle according to the focus surface rotation amount ω, but the present invention is not limited to this. That is, the arbitrary focus correction amount A ₊ is first determined. Then, the object tilt axis setting unit 104 calculates the focus surface rotation amount ω and the tilt angle S at that time so that the object to be focused can be continuously focused on the specific subject that is initially focused even if the focus is corrected by A _+. May be.

The method of calculating the distance H between the focused specific subject position and the optical axis is not limited to the method described above. For example, the relationship between the vertical position coordinates on the captured image and the angle ψ shown in FIG. 11A is obtained in advance and stored as table data. Then, based on the vertical coordinates on the captured image of the position of the focused specific subject, the angle ψ is determined by referring to the table data, and the distance H between the focused position of the specific subject and the optical axis is determined. You may calculate like a following formula (15).

As described above, in each of the embodiments, the imaging device 100 (100a) includes the optical unit 1001 (2001), the detection unit (subject detection unit 103), the setting unit (object tilt axis setting unit 104), and the tilt unit (sensor tilt). Part 105). The optical unit has a lens 101 and an image sensor 102. The detection means detects the specific subject based on the image data from the image sensor. The setting means sets the object tilt axis by focusing on the specific subject. The tilt unit tilts the image sensor around a tilt rotation axis of the image sensor.

Preferably, the setting unit sets the object tilt axis by focusing on the subject when the detection unit detects the specific subject. Further, preferably, when the specific subject is not detected by the detection unit, the setting unit does not set the object tilt axis until the specific subject is detected.

Preferably, the setting means changes the optical axis direction of the optical unit when the specific subject is not detected by the detecting means. More preferably, the imaging device has a changing unit (pan-tilt mechanism 201) that changes the direction of the optical unit. The setting means controls the changing means to change the optical axis direction of the optical unit. Further preferably, the setting means changes the position of the object tilt axis by changing the optical axis direction of the optical unit. Further preferably, the setting unit changes the optical axis direction of the optical unit based on a specific subject existing in an image pickup area different from the tilt rotation axis of the image sensor. Further, preferably, the setting means changes the optical axis direction of the optical unit so that the specific subject existing in the imaging region moves to a position corresponding to the tilt rotation axis (the specific subject overlaps the imaging tilt axis).

Preferably, the detection unit detects the specific subject existing at the position corresponding to the tilt rotation axis of the image sensor based on the image data from the image sensor. The setting means sets an object tilt axis that is in an image-forming relationship with the tilt rotation axis by focusing on a specific subject. Further preferably, the setting means sets the object tilt axis according to the rotation amount of the focus surface. The tilt means tilts the image sensor around the tilt rotation axis according to the amount of rotation of the focus surface.

Preferably, the tilt means tilts and rotates the image sensor so that a plurality of subjects including a specific subject are in focus. Further, preferably, the tilting unit tilts and rotates the image sensor so that the second plane parallel (substantially parallel) to the first plane (reference plane) serving as a reference of the height of the specific subject is in focus. Also preferably, the first plane includes at least one of a ground, a road, a floor, and a corridor. Further preferably, the setting means sets the object tilt axis by changing the optical axis direction of the optical unit from the first direction (first tilt) to the second direction (second tilt). After that, the setting means returns the optical axis direction of the optical unit from the second direction to the first direction while maintaining the height of the object tilt axis from the first plane. Further, preferably, the specific subject is a face area of a person.

(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. It can also be realized by the processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

According to each of the embodiments, the object tilt axis is set to a subject (specific subject) having a desired height such as a face, and the image sensor tilt (imaging sensor tilt) is used to set the horizontal plane (to the ground surface) of the desired height (face). It can be set to focus on parallel horizontal planes). As a result, it is possible to focus on the height surface of every person's face, and even if the person's face moves, it is possible to shoot a moving image without deviating from the focus surface. Therefore, according to each embodiment, it is possible to properly focus on a plurality of subjects, and even when a plurality of subjects move, an image capturing apparatus and an image capturing apparatus capable of capturing a moving image in focus. It is possible to provide a control method and a program.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes can be made within the scope of the gist thereof.

100, 100a Imaging device 101 Lens 102 Imaging sensor 103 Subject detection unit (detection means)
104 Object tilt axis setting unit (setting means)
105 Sensor tilt unit (tilt means)
1001, 2001 Optical unit

Claims (17)

An optical unit having a lens and an image sensor;
Based on the image data from the image sensor, a detection unit that detects a specific subject,
Setting means for setting the object tilt axis by focusing on the specific subject,
An image pickup apparatus comprising: a tilt unit configured to rotate the image sensor by tilting about a tilt rotation axis of the image sensor. The image pickup apparatus according to claim 1, wherein the setting unit sets the object tilt axis by focusing on the subject when the detection unit detects the specific subject. The image pickup apparatus according to claim 1, wherein the setting unit does not set the object tilt axis until the specific subject is detected by the detecting unit, unless the specific subject is detected. .. The image pickup apparatus according to claim 1, wherein the setting unit changes the optical axis direction of the optical unit when the detection unit does not detect the specific subject. Further comprising a changing means for changing the direction of the optical unit,
The image pickup apparatus according to claim 4, wherein the setting unit controls the changing unit to change the optical axis direction of the optical unit. The image pickup apparatus according to claim 4, wherein the setting unit changes a position of the object tilt axis by changing a direction of the optical axis of the optical unit. 7. The setting unit changes the optical axis direction of the optical unit based on the specific subject existing in an image pickup area different from the tilt rotation axis of the image sensor. The imaging device according to any one of items. 8. The setting unit changes the optical axis direction of the optical unit so that the specific subject existing in the imaging region moves to the position corresponding to the tilt rotation axis. The imaging device described. The detection means detects a specific subject existing at a position corresponding to a tilt rotation axis of the image sensor based on image data from the image sensor,
9. The image pickup according to claim 1, wherein the setting unit sets the object tilt axis in an image-forming relationship with the tilt rotation axis by focusing on the specific subject. apparatus. The setting means sets the object tilt axis according to the rotation amount of the focusing surface,
9. The image pickup apparatus according to claim 1, wherein the tilting unit tilts the image pickup sensor around the tilt rotation axis according to a rotation amount of the focus surface. The image pickup apparatus according to claim 1, wherein the tilting unit tilts and rotates the image pickup sensor so that a plurality of subjects including the specific subject are in focus. 12. The tilting unit tilts and rotates the image sensor so as to focus on a second plane parallel to a first plane which is a reference of the height of the specific subject. The imaging device according to any one of 1. The imaging device according to claim 12, wherein the first plane includes at least one of a ground, a road, a floor, and a corridor. The setting means,
Changing the optical axis direction of the optical unit from a first direction to a second direction to set the object tilt axis,
14. The optical axis direction of the optical unit is returned from the second direction to the first direction while maintaining the height of the object tilt axis from the first plane. The imaging device according to. The image pickup apparatus according to claim 1, wherein the specific subject is a face area of a person. Detecting a specific subject based on image data from the image sensor,
Focusing on the specific subject and setting an object tilt axis,
Tilting the image sensor around a tilt rotation axis of the image sensor. A program that causes a computer to execute the method for controlling an imaging device according to claim 16.