JP7076988B2 - Imaging device and control method - Google Patents

Description

The present invention relates to an image pickup apparatus and a control method.

Network cameras that can remotely control cameras and monitor images via networks or leased lines are known. Some network cameras have a pan-tilt mechanism (PT mechanism) in which the camera head unit (imaging unit) rotates, and a model having an electric zoom function. The pan-tilt mechanism performs panning (horizontal rotation) or tilting (vertical rotation) of the camera head portion. In the following description, horizontal rotation is also described as pan rotation, and pan rotation direction is also described as pan direction. Further, the vertical rotation is also described as a tilt rotation, and the direction of the tilt rotation is also described as a tilt direction. An image pickup device having a pan-tilt mechanism and an electric zoom function can freely change the image pickup direction and the image pickup angle of view.

Further, in order to reduce the shaking of the captured image generated by the vibration of the installation environment, an imaging device having an image shake correction function (vibration isolation function) has been proposed. Examples of the vibration isolation method include electronic vibration isolation using correction by image processing and optical vibration isolation in which correction is performed optically by driving a lens. Further, it is conceivable to use a pan-tilt mechanism to drive a lens barrel unit including a lens and an image sensor in response to shaking to realize vibration isolation control for image shake correction. Anti-vibration control using the pan-tilt mechanism is hereinafter referred to as PT anti-vibration. PT anti-vibration can be corrected even for shaking with a large amplitude, and is effective in an installation environment where large shaking occurs such as on a ship.

Patent Document 1 discloses a system in which the anti-vibration function of a camera is temporarily disabled while the pan-tilt mechanism is being driven. Further, Patent Document 2 acquires a sway signal in the direction of gravity by an acceleration sensor, decomposes the sway signal into a component parallel to the image sensor and a component perpendicular to the image sensor using the tilt angle, and uses the parallel sway component. Discloses a device that corrects translational sway.

Japanese Unexamined Patent Publication No. 2004-31138 Japanese Unexamined Patent Publication No. 2009-139827

An image pickup device having a pan-tilt mechanism has, for example, a base portion and a camera head portion, a turntable for pan rotation is provided on the base portion, and a support column for supporting a shaft for tilt rotation is provided on the turntable, and the camera head is provided. It has a mechanical configuration that rotates pan and tilt. The drive unit that rotates the camera head in a pan is called a pan drive unit. Further, the drive unit that tilts and rotates the camera head is called a tilt drive unit. In the case of such a mechanical configuration, the movement locus in the captured image when the camera head is pan-rotated differs depending on the angle (tilt angle) of the tilt drive unit. Specifically, when the tilt angle is oriented in the horizontal direction, the movement locus of the image due to the pan rotation is the movement in the horizontal direction. However, when the tilt angle is directed diagonally upward, the movement locus of the image is a movement that draws a curve. Then, when the tilt angle is in the vertical direction, the movement locus of the image becomes a rotational movement. As described above, depending on the tilt angle, the movement locus of the image due to the horizontal shake and the pan rotation do not match, and the image shake (image shake) due to the horizontal shake cannot be corrected. An object of the present invention is to realize good image shake correction in an image pickup apparatus having a plurality of drive units for rotationally driving the image pickup unit in different directions.

The image pickup apparatus of one embodiment of the present invention has a first driving means for rotating the image pickup direction of the image pickup means in the first direction, and a second direction in which the image pickup direction of the image pickup means is orthogonal to the first direction. A second drive means for rotating the image, and a control means for controlling the first drive means or the second drive means to perform runout correction control for correcting image shake caused by shake applied to the image pickup means. The control means invalidates the runout correction control using the first drive means and the second drive means when the rotation angle by the second drive means is equal to or larger than the threshold value.

According to the present invention, good image shake correction can be realized in an image pickup apparatus having a plurality of drive units for rotationally driving the image pickup unit in different directions.

It is a figure which shows the structure of the image pickup apparatus. It is a figure which shows the mechanical mechanism of a network camera. It is a figure which shows the example of the movement locus in the captured image by pan rotation. It is a flowchart explaining operation process of a network camera. It is a flowchart explaining the vibration isolation control using a pan-tilt mechanism. It is a figure which shows the relationship between the imaging position and the imaging direction. It is a figure which shows the movement locus on the captured image at the time of pan rotation. It is a flowchart explaining the setting process of the correction amount upper limit value of vibration isolation control. It is a figure which shows the movement locus by pan rotation when a zoom position changes. It is a figure which shows the example of the threshold table. It is a figure explaining an example of switching of anti-vibration control according to a zoom position. It is a figure explaining the method of finding the movement locus based on a motion vector. It is a figure explaining the example of the vibration isolation control based on the analysis of the captured image.

(Example 1)
FIG. 1 is a diagram showing a configuration of an image pickup apparatus according to the present embodiment.
The network camera 1000 is connected to a client device (information processing device) (not shown) via the network 3000 so as to be able to communicate with each other. The network camera 1000 includes an image pickup unit 1001, an image processing unit 1002, a system control unit 1003, a shake detection unit 1004, a panning drive unit 1005, a tilting drive unit 1006, a pan / tilt control unit 1007, and a communication unit 1008. In the following description, the panning drive unit is simply referred to as a "pan drive unit". Further, the tilting drive unit is simply referred to as a "tilt drive unit".

The image pickup unit 1001 includes a lens, an image pickup device, and a control group thereof. The image pickup unit 1001 photoelectrically converts the subject light and outputs a signal related to the captured image. The image processing unit 1002 performs predetermined development processing and compression coding processing on the signal output by the image pickup unit 1001, generates image data, and transmits the image data to the system control unit 1003. The system control unit 1003 controls the entire network camera 1000. The system control unit 1003 distributes image data to a client device (not shown) via the communication unit 1008. Further, the communication unit 1008 receives the camera control command transmitted from the client device and transmits the camera control command to the system control unit 1003. Further, the system control unit 1003 transmits a response to the camera control command to the client device. The system control unit 1003 analyzes the transmitted camera control command and performs processing according to the command. The system control unit 1003 gives an instruction to adjust the image quality to the image processing unit 1002, for example. Further, for example, the system control unit 1003 instructs the pan / tilt control unit 1007 of the pan / tilt operation. The pan-tilt operation indicates a panning (horizontal rotation) operation or a tilting (vertical rotation) operation.

The image processing unit 1002 performs image processing based on an instruction from the system control unit 1003. Further, the pan / tilt control unit 1007 controls the pan drive unit 1005 or the tilt drive unit 1006 based on the instruction from the system control unit 1003. The shake detection unit 1004 is configured by a gyro sensor or the like, detects shake (shake) applied to the image pickup unit 1001, and outputs a shake detection signal. Specifically, the shake detection unit 1004 outputs the angular velocities in the pan direction and the tilt direction of the image pickup unit 1001 as the shake detection signal. The angular velocity information is transmitted to the image processing unit 1002 and the pan / tilt control unit 1007, and is used for vibration correction control (vibration isolation control). That is, the system control unit 1003 and the pan / tilt control unit 1007 function as control means for controlling the pan drive unit 1005 or the tilt drive unit 1006 to perform shake correction control for correcting image shake caused by shake applied to the image pickup unit 1001. do.

The pan drive unit 1005 functions as a first drive means for pan-rotating the image pickup unit 1001 by a panning operation. Due to the pan rotation, the imaging direction of the imaging unit 1001 is rotated in the first direction (horizontal direction). The pan drive unit 1005 has a mechanical drive system that performs a panning operation, a motor of the drive source thereof, and an angle sensor that detects an angle of the drive unit. The tilt drive unit 1006 functions as a second drive means for tilting and rotating the image pickup unit 1001 by a tilting operation. Due to the tilt rotation, the imaging direction of the imaging unit 1001 rotates in the second direction (vertical direction) orthogonal to the first direction. The tilt drive unit 1006 has a mechanical drive system that performs a tilting operation, a motor of the drive source thereof, and an angle sensor that detects the angle of the drive unit. The operations of the pan drive unit 1005 and the tilt drive unit 1006 are controlled by the pan tilt control unit 1007 instructed by the system control unit 1003. That is, the configuration (pan / tilt mechanism) for performing the pan / tilt operation of the network camera 1000 is realized by the system control unit 1003, the pan / tilt control unit 1007, the pan drive unit 1005, and the tilt drive unit 1006.

FIG. 2 is a diagram showing a mechanical mechanism of the network camera of the present embodiment.
FIG. 2A shows a state in which the network camera 1000 is viewed from above. FIG. 2B shows a state in which the network camera 1000 is viewed from the side. The bottom case 1101 is a case provided on the bottom surface of the network camera. The turntable 1102 is a table that rotates in the horizontal direction. The camera head support column 1103 is a support column provided on the turntable 1102. The camera head 1104 is provided with an image pickup unit 1001 shown in FIG.

With reference to FIG. 2, the mechanical mechanism of the pan drive unit 1005 and the tilt drive unit 1006 shown in FIG. 1 will be described. The pan drive unit 1005 is composed of a bottom case 1101 and a turntable 1102, and is rotated in the horizontal direction by the turntable 1102. In the present embodiment, the pan drive unit 1005 can rotate (pan rotation) from -175 degrees to +175 degrees in the left-right direction.

The tilt drive unit 1006 is composed of a camera head support column 1103 and a camera head 1104, and rotates in the vertical direction (tilt rotation). In the present embodiment, the tilt drive unit 1006 can rotate from 0 degrees in the horizontal direction to 90 degrees in the straight upward direction. That is, the network camera 1000 can take an image by changing the image pickup direction by rotating the camera head 1104 in the horizontal direction or the vertical direction.

FIG. 3 is a diagram showing an example of a movement locus in a captured image due to pan rotation when the tilt angle changes.
The tilt angle is the angle of the tilt drive unit 1006, that is, the angle of the camera head 1104 when the camera head 1104 is tilt-rotated by the tilt drive unit 1006. FIG. 3A shows a state in which the tilt angle is 0 degrees in the horizontal direction. FIG. 3B shows a state where the tilt angle is 45 degrees. FIG. 3C shows a state in which the tilt angle is 90 degrees in the vertical direction. The movement locus 1202A shows a movement locus in the captured image 1201 when the camera head 1104 pan-rotates in a state where the tilt angle is 0 degrees. The movement locus 1202B shows a movement locus in the captured image 1201 due to pan rotation in a state where the tilt angle is 45 degrees. The movement locus 1202C shows a movement locus in the captured image 1201 due to pan rotation in a state where the tilt angle is 90 degrees.

In the pan-tilt mechanism of the network camera 1000 of the present embodiment, the movement locus in the captured image due to the pan rotation in the state where the tilt angle is in the horizontal direction is the movement in the horizontal direction as in the movement locus 1202A. However, the movement locus in the captured image due to the pan rotation in the state where the tilt angle is 45 degrees is a curved movement like the movement locus 1202B. Further, when the camera head 1104 is pan-rotated while the tilt angle is 90 degrees, the movement locus in the captured image becomes a rotation movement like the movement locus 1202C. In this way, as the tilt angle changes from the horizontal state to the vertical state, the movement locus in the captured image due to the pan rotation changes from the horizontal movement to the rotation direction movement. Therefore, the vibration isolation control that corrects the horizontal vibration using the pan drive unit 1005 may not function effectively depending on the tilt angle. Therefore, the network camera 1000 of the present embodiment switches between valid and invalid anti-vibration control using the pan drive unit 1005 or the tilt drive unit 1006 according to the tilt angle. The network camera 1000 performs anti-vibration control in a tilt angle range in which anti-vibration control by the pan drive unit 1005 effectively functions, for example.

FIG. 4 is a flowchart illustrating the operation processing of the network camera of the first embodiment.
In S1000, the tilt drive unit 1006 starts driving. In S1001, the system control unit 1003 acquires the tilt angle from the pan / tilt control unit 1007. Subsequently, in S1002, the system control unit 1003 determines whether or not the tilt angle is less than the threshold value. In the following description, the threshold value of the tilt angle used for the process related to the switching of the vibration isolation control as in the determination process in S1002 is described as the tilt angle threshold value. For example, when the tilt angle is less than 30 degrees, the movement locus in the image due to pan rotation is close to horizontal, and the vibration isolation control by the pan drive unit functions effectively, the tilt angle threshold value is set to 30 degrees. If the tilt angle is less than the tilt angle threshold value, the process proceeds to S1003. If the tilt angle is equal to or greater than the tilt angle threshold value, the process proceeds to the process of S1004.

In S1003, the system control unit 1003 enables (on) the vibration isolation control setting. As a result, as described with reference to FIG. 5, the pan / tilt control unit 1007 executes image shake correction by driving the pan drive unit 1005 and the tilt drive unit 1006. Further, in S1004, the system control unit 1003 invalidates (turns off) the anti-vibration control setting. As a result, image shake correction by driving the pan drive unit 1005 and the tilt drive unit 1006 is not executed.

FIG. 5 is a flowchart illustrating anti-vibration control using the pan-tilt mechanism.
In S2000, the network camera 1000 starts imaging. In S2001, the pan / tilt control unit 1007 determines whether the anti-vibration control setting is on. If the anti-vibration control setting is on, the process proceeds to the process of S2002. If the anti-vibration control setting is off, the process ends. That is, in this case, the pan / tilt control unit 1007 does not perform vibration isolation control using the pan drive unit 1005 or the tilt drive unit 1006.

In S2002, the pan / tilt control unit 1007 acquires the angular velocity in the pan direction and the angular velocity in the tilt direction from the shake detection unit 1004. Subsequently, in S2003, the pan / tilt control unit 1007 integrates the acquired angular velocities in the pan direction to calculate the pan angle value. The pan angle is the angle of the pan drive unit 1005, that is, the angle of the camera head 1104 when the camera head 1104 is pan-rotated by the pan drive unit 1005. Further, the pan-tilt control unit 1007 integrates the acquired angular velocities in the tilt direction to calculate the tilt angle value.

Next, in S2004, the pan / tilt control unit 1007 calculates a shake correction amount (sway correction amount) in the pan direction based on the calculated pan angle value. Further, the pan-tilt control unit 1007 calculates a shake correction amount (sway correction amount) in the tilt direction based on the calculated tilt angle value. The runout correction amount is a correction amount for canceling the angle amount generated by the shake applied to the network camera 1000. For example, when the pan angle value is 0.1 degree, the runout correction amount is −0.1 degree.

Next, in S2005, the pan / tilt control unit 1007 gives a drive instruction to the pan drive unit 1005 and the tilt drive unit 1006 according to the calculated amount of runout correction in the pan direction and the tilt direction. As a result, vibration isolation control (sway correction operation) is executed.

As described above, the network camera 1000 switches between valid and invalid anti-vibration control using the pan-tilt mechanism according to the tilt angle. Specifically, when the tilt angle is equal to or greater than the tilt angle threshold value, the vibration isolation control using the pan drive unit 1005 and the tilt drive unit 1006 is invalidated. As a result, the vibration isolation control can be performed in the tilt angle range in which the vibration isolation control by the pan drive unit 1005 effectively functions.

As an example of switching the anti-vibration control using the pan-tilt mechanism according to the tilt angle, the network camera 1000 changes the upper limit value (correction amount upper limit value) of the image shake correction by the anti-vibration control according to the tilt angle. You may try to do it. An example of changing the correction amount upper limit value will be described below with reference to FIGS. 6 to 8.

FIG. 6 is a diagram showing the relationship between the imaging position and the imaging direction.
In FIG. 6, position 4001 indicates an imaging position. The pan rotation axis 4002 indicates a predetermined pan rotation axis. The imaging spherical surface 4003 indicates an imaging spherical surface. Direction 4005A indicates an imaging direction corresponding to a tilt angle of 0 degrees. The pan rotation trajectory 4006A shows a pan rotation trajectory at a tilt angle of 0 degrees. The imaging direction 4005B indicates a predetermined imaging direction. The pan rotation trajectory 4006B indicates a pan rotation trajectory in the imaging direction 4005B. The captured image 4008 shows a captured image in the imaging direction 4005B.

In FIG. 6, the camera at the imaging position 4001 pan-rotates around the pan rotation axis 4002 and tilts around the tilt rotation axis (not shown), so that the imaging direction is the imaging spherical surface centered on the imaging position 4001. Move on 4003. When the tilt angle is 0 degrees, the imaging direction is the imaging direction 4005A, and the pan rotation trajectory is the pan rotation trajectory 4006A. When the image pickup direction of the camera is 4005B at a certain tilt angle, the pan rotation trajectory becomes the pan rotation trajectory 4006B. Further, the captured image 4008 is arranged so as to be orthogonal to the imaging direction 4005B.

また、撮像画像４００８の撮像画角をα、撮像画像４００８の長さをｄとする。システム制御部１００３は、撮像方向４００５Ｂの長さl を、式（２）に基づいて算出する。

また、システム制御部１００３は、パン回転軌道の回転半径４００７の長さｒを、撮像方向４００５Ｂの長さl と角度βを用いて、式（３）で算出する。

Let θ be the tilt angle formed by the imaging direction 4005A and the imaging 4005B. The system control unit 1003 calculates the angle β formed by the pan rotation axis 4002 and the image pickup direction 4005B based on the equation (1).

Further, the angle of view of the captured image 4008 is α, and the length of the captured image 4008 is d. The system control unit 1003 calculates the length l of the imaging direction 4005B based on the equation (2).

Further, the system control unit 1003 calculates the length r of the radius of gyration 4007 of the pan rotation trajectory by the equation (3) using the length l of the imaging direction 4005B and the angle β.

The system control unit 1003 uses the obtained pan rotation radius 4007 to obtain a pan rotation trajectory 4006B which is the circumference of a circle orthogonal to the pan rotation axis 4002. Then, the system control unit 1003 projects the pan rotation trajectory 4006B seen from the image pickup position 4001 onto the image pickup image 4008.

FIG. 7 is a diagram showing a movement locus on a captured image when pan rotation is performed at a certain tilt angle.
In FIG. 7, the pan movement locus 4009 shows a movement locus (movement locus of the camera head) in the imaging direction on the image pickup image 4008. The asymptote 4010 indicates an asymptote with respect to the pan movement locus 4009.

係数ａ，ｂは、チルト角度θを用いて、式（５）で表すことができる。

When the tilt angle is less than 45 degrees, the pan movement locus 4009 projected on the captured image 4008 can be approximated by a hyperbola. For example, the pan movement locus 4009 can be represented by the equation (4) when expressed in coordinates with the center of the captured image 4008 as the origin, the x-axis in the horizontal direction, and the y-axis in the vertical direction.

The coefficients a and b can be expressed by the equation (5) using the tilt angle θ.

The value of b / a indicates the slope of the asymptote 4010 with respect to the hyperbola 4009. The larger the tilt angle, the larger the slope of the asymptote, that is, the larger the curvature of the hyperbola. As shown in FIG. 7, the pan movement locus 4009 draws an upward curve from the center of the image to the outside, and does not coincide with the horizontal shaking direction. However, if the locus is regarded as horizontal in a range where the bending width of the central portion of the pan movement locus 4009 is minute, the anti-vibration control by the pan drive unit 1005 can effectively function.

For example, let yL be the upper limit of the bending width regarded as the horizontal direction with respect to the vertical angle of view yA. The system control unit 1003 obtains x1 and x2 such that y = yL by using the equation (4). The system control unit 1003 sets the obtained x1 and x2 as the correction amount upper limit value, and performs vibration isolation control by driving the pan drive unit 1005 within the range of xL from x1 to x2. In this way, the system control unit 1003 obtains the pan movement locus on the captured image by projecting the pan rotation trajectory according to the tilt angle, and corrects the anti-vibration control based on the upper limit of the bending width of the pan movement locus. Calculate the upper limit. In this embodiment, the system control unit 1003 calculates the correction amount upper limit value of the vibration isolation control in advance, and stores the table (upper limit value table) associated with the tilt angle in a predetermined storage unit (not shown). Keep it.

FIG. 8 is a flowchart illustrating a process of setting a correction amount upper limit value of vibration isolation control according to a tilt angle.
In S3000, the tilt drive unit 1006 starts driving. In S3000, the system control unit 1003 acquires the tilt angle from the pan / tilt control unit 1007. Subsequently, in S3002, the system control unit 1003 acquires the correction amount upper limit value of the vibration isolation control driven by the pan drive unit 1005 according to the acquired tilt angle. Specifically, the system control unit 1003 refers to the upper limit value table stored in the storage unit and acquires the correction amount upper limit value according to the tilt angle. Then, in S3003, the system control unit 1003 sets the correction amount upper limit value for the pan / tilt control unit 1007.

In the process of S2004 in FIG. 5, when the runout correction amount exceeds the set correction amount upper limit value, the pan / tilt control unit 1007 in which the correction amount upper limit value is set limits the runout correction amount to the correction amount upper limit value. do. As described above, the network camera 1000 is good in the range of the tilt angle at which the vibration isolation control by the pan drive unit 1005 functions effectively by changing the correction amount upper limit value of the vibration isolation control according to the tilt angle. It is possible to perform various image shake corrections.

Further, in addition to the pan-tilt mechanism, the network camera 1000 has a shake correction means (optical vibration isolation means) for optically correcting image shake or a shake correction means (electronic vibration isolation means) for correcting image shake by image processing on an captured image. ) May be provided. The optical anti-vibration means is, for example, an image shake correction lens or an image pickup element that moves in a direction orthogonal to the optical axis. When the tilt angle is equal to or greater than the tilt angle threshold value, the system control unit 1003 invalidates the anti-vibration control using the pan-tilt mechanism, and controls the optical anti-vibration means or the electronic anti-vibration means to correct the image shake. You may.

Further, when the tilt angle is equal to or larger than the tilt angle threshold value, the system control unit 1003 reduces the correction amount of the image shake correction by the vibration isolation control using the pan-tilt mechanism, and controls the optical vibration isolation means or the electronic vibration isolation means. Then, image shake correction may be performed.

Further, the present invention is not limited to an application example in which the system control unit 1003 simultaneously switches between the use and non-use of vibration isolation control in the pan direction and the tilt direction. The system control unit 1003 may switch the control between the pan direction and the tilt direction under different conditions. For example, even if the anti-vibration control using the pan drive unit 1005 does not function effectively for horizontal sway when the tilt angle is facing an angle equal to or higher than a certain threshold, tilt drive for vertical sway. Anti-vibration control using unit 1006 may function effectively. Therefore, when the tilt angle is equal to or greater than the tilt angle threshold value, the system control unit 1003 invalidates the horizontal vibration isolation control using the pan drive unit 1005, and the vertical vibration isolation control using the tilt drive unit 1006. May be valid.

Further, in the present embodiment, the anti-vibration control by the pan drive unit 1005 functions effectively for the horizontal shaking when the tilt angle faces the horizontal direction (a state below the threshold value). Therefore, in this state, the vibration isolation control functions effectively. Anti-vibration control by the pan drive unit 1005 was enabled. However, the anti-vibration control by the pan drive unit 1005 effectively functions for the shaking in the rotation direction in the state where the tilt angle faces the vertical direction. Therefore, the system control unit 1003 may enable the anti-vibration control by the pan drive unit 1005 when the tilt angle is equal to or larger than the tilt angle threshold value.

(Example 2)
The second embodiment will be described with reference to FIGS. 9 to 11. For the same components as in the first embodiment, the reference numerals already used will be used, detailed description thereof will be omitted, and the differences from the first embodiment will be mainly described. The method of omitting such an explanation is the same in other examples described later. In the second embodiment, the image pickup unit 1001 includes a zoom drive unit capable of changing the image pickup angle of view from a wide angle to a telephoto lens. In this embodiment, the zoom drive unit is an optical zoom mechanism.

FIG. 9 is a diagram showing a movement locus in the imaging direction in the captured image due to pan rotation when the zoom position changes when the tilt angle is 45 degrees.
FIG. 9A shows a state in which the zoom position of the zoom drive unit is on the wide-angle side. FIG. 9B shows a state in which the zoom position of the zoom drive unit is on the telephoto side. The movement locus 1202D is a movement locus in the captured image 1201 due to pan rotation in a state where the zoom position is on the wide-angle side. The movement locus 1202E is a movement locus in the captured image 1201 due to pan rotation in a state where the zoom position is on the telephoto side.

In the pan-tilt mechanism of the network camera 1000 of the second embodiment, as shown in FIG. 9A, the movement locus in the captured image when the pan is rotated with the tilt angle of 45 degrees is as shown in the movement locus 1202D. It becomes a curve movement. However, as shown in FIG. 9B, when the zoom position changes to the telephoto side, the angle of view becomes narrower, so that the pan movement amount becomes relatively small, and the movement as if the central part of the movement locus 1202D is extracted. The locus is 1202E. Therefore, the movement locus 1202E on the telephoto side has a smaller bending width than the movement locus 1202D on the wide-angle side. As described above, even at the same tilt angle, the curve of the movement locus in the image due to pan rotation changes according to the zoom position, so vibration isolation control using the pan drive unit 1005 is effective according to the zoom position. The tilt angle range that works for is different. Therefore, in the second embodiment, the system control unit 1003 changes the tilt angle threshold value used for the process related to the switching of the vibration isolation control according to the zoom position. For this purpose, the system control unit 1003 creates in advance a table (threshold value table) in which the zoom position and the tilt angle threshold value are associated with each other, as shown in FIG. 10, and stores them in a predetermined storage unit. In the example of the threshold table shown in FIG. 10, the tilt angle threshold is smaller as the zoom position is on the wide-angle side, and the tilt angle threshold is larger as the zoom position is on the telephoto side. The system control unit 1003 acquires the tilt angle threshold value according to the zoom position using the threshold value table and uses it for switching the anti-vibration control to perform highly accurate image shake correction according to the zoom position and the tilt angle. be able to

FIG. 11 is a flowchart illustrating an example of switching the vibration isolation control according to the zoom position.
In S4000, the tilt drive unit 1006 starts driving. In S4001, the system control unit 1003 acquires the zoom position from the image pickup unit 1001. In S4002, the system control unit 1003 refers to the threshold value table stored in the storage unit and acquires the tilt angle threshold value corresponding to the zoom position acquired in S4001. Then, the process proceeds to S4003.

In S4003, the system control unit 1003 acquires the tilt angle from the pan / tilt control unit 1007. Subsequently, in S4004, the system control unit 1003 determines whether or not the tilt angle is less than the tilt angle threshold value acquired in S4002. If the tilt angle is less than the tilt angle threshold, the process proceeds to S4005. If the tilt angle is equal to or greater than the tilt angle threshold value, the process proceeds to the process of S4006. Since S4005 and S4006 are the same as S1003 and S1004 in FIG. 4, the description thereof will be omitted. According to the network camera 1000 of the second embodiment, highly accurate image shake correction can be performed by using the tilt angle threshold value according to the zoom position.

As in the first embodiment, the system control unit 1003 may change the correction amount upper limit value of the image shake correction using the pan drive unit 1005 as an example of switching the vibration isolation control. For example, the system control unit 1003 refers to an upper limit value table in which the tilt angle, the zoom position, and the correction amount upper limit value are associated with each other, which is stored in the storage unit in advance, and the correction amount according to the tilt angle and the zoom position. Get the upper limit. Then, the system control unit 1003 limits the correction amount of the image shake correction using the pan drive unit 1005 to the acquired correction amount upper limit value.

Further, the network camera 1000 may be provided with a digital zoom mechanism for cutting out a part of the captured image and displaying it in an enlarged manner, instead of the optical zoom mechanism. The bending width of the movement locus in the captured image due to pan rotation changes depending on the cutout position of the captured image. Therefore, the system control unit 1003 may switch the tilt angle threshold value according to, for example, the cutting position of the captured image by the digital zoom mechanism. Further, the system control unit 1003 may change the correction amount upper limit value according to the cutout position of the captured image and the tilt angle. For example, the system control unit 1003 refers to an upper limit value table in which the tilt angle, the cutout position of the captured image, and the upper limit value of the correction amount are associated with each other, which is stored in the storage unit in advance, and cuts out the tilt angle and the captured image. Acquires the upper limit of the correction amount according to the position. Then, the system control unit 1003 limits the correction amount of the image shake correction using the pan drive unit 1005 to the acquired correction amount upper limit value.

(Example 3)
The third embodiment will be described with reference to FIGS. 12 and 13. For the same components as in the first embodiment, the reference numerals already used will be used, detailed description thereof will be omitted, and the differences from the first embodiment will be mainly described.
The network camera 1000 of the third embodiment analyzes and obtains the movement locus of the captured image when the image pickup unit 1001 is driven (pan rotation) by the pan drive unit 1005. The network camera 1000 switches anti-vibration control using the pan-tilt mechanism according to the size of the bending width of the obtained movement locus. Hereinafter, a process of obtaining the movement locus of the captured image when pan-rotating will be described based on the motion vector detected from the captured image.

FIG. 12 is a diagram illustrating a method of obtaining a movement locus based on a motion vector.
FIG. 12A shows a state in which the tilt angle faces the horizontal direction. FIG. 12B shows a state where the tilt angle is 45 degrees. The motion vector 5001A is a motion vector of the pixel block of the captured image 4008 in FIG. 12 (A). The motion vector 5002A is a motion vector of the entire captured image 4008 in FIG. 12 (A).

Further, the motion vector 5001B is a motion vector of the pixel block of the captured image 4008 in FIG. 12B. The motion vector 5002B is a motion vector of the entire captured image 4008 in FIG. 12 (B). In this embodiment, the image processing unit 1002 divides the captured image into a plurality of small pixel blocks under the control of the system control unit 1003, and derives the feature points of each pixel block. The image processing unit 1002 detects the feature points derived from the image of the current frame and the feature points of the image of the previous frame derived in advance by pattern matching. The system control unit 1003 connects the positions from the feature points of the image of the previous frame to the feature points of the image of the current frame, and detects a motion vector indicating the direction and magnitude of the image change. Further, the image processing unit 1002 detects a motion vector in the entire image based on each motion vector of each block.

As shown in FIG. 12A, when the tilt angle is oriented horizontally, the motion vector of the entire captured image indicates the horizontal direction as in the motion vector 5002A. Further, as shown in FIG. 12B, when the tilt angle is 45 degrees, the motion vector of the entire captured image shows a curve like the motion vector 5002B. The locus indicated by the motion vector of the entire captured image corresponds to the movement locus of the captured image. Therefore, the system control unit 1003 controls the image processing unit 1002 to analyze the movement locus of the captured image obtained based on the motion vector detected from the captured image when the pan is rotated. The system control unit 1003 switches the anti-vibration control using the pan-tilt mechanism according to the magnitude of the bending width of the analyzed movement locus.

FIG. 13 is a flowchart illustrating an example of vibration isolation control based on analysis of captured images.
In S5000, the pan drive unit 1005 starts driving. In S5001, the system control unit 1003 analyzes the image captured by the image pickup unit 1001 and obtains the movement locus of the image captured by pan rotation.

Next, in S5002, the system control unit 1003 determines whether the bending width of the movement locus is less than the threshold value. The system control unit 1003 sets in advance, for example, in a range where the bending width of the moving locus is small, the moving locus is horizontal, and the upper limit of the horizontal bending width is set as a threshold value. If the bending width of the movement locus is less than the threshold value, the process proceeds to S5003. If the bending width of the movement locus is equal to or greater than the threshold value, the process proceeds to S5004. S5003 and S5004 are the same as S1003 and S1004 in FIG. 4, respectively.

The network camera 1000 of the third embodiment obtains the movement locus of the captured image by analyzing the captured image when pan-rotated at a predetermined tilt angle, and the condition that the vibration isolation control functions effectively from the bending width of the movement locus. Anti-vibration control is used in. As a result, image shake correction can be performed with high accuracy according to the tilt angle.

In Example 3, an application example of enabling or disabling vibration isolation control using the pan-tilt mechanism according to the size of the bending width of the movement locus is shown, but the present invention is not limited to this application example. For example, as described in the first embodiment, the system control unit 1003 may change the correction amount upper limit value of the image shake correction according to the bending width of the movement locus obtained by the analysis of the captured image.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these examples, and various modifications and modifications can be made within the scope of the gist thereof. The above-mentioned Examples 1 to 3 may be applied in combination as appropriate.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

1000 Network camera 1003 System control unit 1005 Pan drive unit 1006 Tilt drive unit

Claims (8)

A first driving means for rotating the image pickup direction of the image pickup means in the first direction, and
A second driving means that rotates the image pickup direction of the image pickup means in a second direction orthogonal to the first direction, and
A control means for controlling the first drive means or the second drive means to perform shake correction control for correcting image shake caused by shake applied to the image pickup means is provided.
The control means invalidates the runout correction control using the first drive means and the second drive means when the rotation angle by the second drive means is equal to or larger than the threshold value.
An imaging device characterized by this. It is provided with a shake correction means for optically correcting the image shake or a shake correction means for correcting the image shake by image processing on the captured image.
The control means invalidates the runout correction control using the first drive means and the second drive means when the rotation angle by the second drive means is equal to or larger than the threshold value, and the optical image The image pickup apparatus according to claim 1, further comprising controlling a shake correction means for correcting shake or a shake correction means for correcting image shake by image processing on an image captured by the image pickup means. A zoom drive means for changing the image-taking angle of view of the image-taking means is provided.
The image pickup apparatus according to claim 1 or 2, wherein the control means changes the threshold value according to the zoom position of the zoom drive means. A cutting means for cutting out a part of the captured image acquired by the imaging means is provided.
The image pickup apparatus according to any one of claims 1 to 3, wherein the control means changes the threshold value according to the cut-out position of the image captured by the cut-out means. The control means is characterized in that the upper limit value of the correction amount of the image shake by the shake correction control using the first drive means is changed according to the rotation angle by the second drive means. The image pickup apparatus according to the above. A zoom drive means for changing the image-taking angle of view of the image-taking means is provided.
The imaging according to claim 5, wherein the control means changes the upper limit of the correction amount of the image shake according to the zoom position of the zoom drive means and the rotation angle of the second drive means. Device. A cutting means for cutting out a part of the captured image acquired by the imaging means is provided.
6. The control means is characterized in that the upper limit value of the correction amount of the image shake is changed according to the cutting position of the captured image by the cutting means and the rotation angle by the second driving means. The imaging device described. It includes a first driving means for rotating the imaging direction of the imaging means in the first direction, and a second driving means for rotating the imaging direction of the imaging means in a second direction orthogonal to the first direction. It is a control method of the image pickup device.
It has a control step of controlling the first driving means or the second driving means to execute the shake correction control for correcting the image shake caused by the shake applied to the image pickup means.
In the control step, when the rotation angle by the second drive means is equal to or larger than the threshold value, the runout correction control using the first drive means and the second drive means is invalidated.
A control method characterized by that.

Images