JP2014103490A - Robot camera control device, program for the same, and multi-viewpoint robot camera system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a multi-viewpoint robot camera system which can perform depth adjustment easily.SOLUTION: A multi-viewpoint robot camera system 100 comprises: an operation interface section 10 which calculates a depth value which is changed nonlinearly, from an encoder value; a camera calibration section 20 which calculates a camera parameter by a camera calibration; a master camera control section 30 which controls a master camera MC; a gazing point calculation section 40 which calculates a rotation matrix on the basis of the difference between a camera posture at the time of camera calibration and a camera posture at the time of imaging, and calculates a position of the gazing point by adding a translation matrix to the product of a depth value and a vector of a depth component included in the rotation matrix; and a slave camera control section 50 which controls the slave camera SC so as to face the gazing point.

Description

  The present invention relates to a robot camera control device that controls the posture of a robot camera, a program thereof, and a multi-view robot camera system.

  Conventionally, a multi-view robot camera system capable of shooting a multi-view video obtained by pan-following a moving subject has been proposed (Non-Patent Document 1). In this multi-viewpoint robot camera system, one of a plurality of robot cameras is set as a master camera, and a cameraman operates to direct the master camera toward a subject. Here, a gazing point is set on the optical axis of the master camera, and slave cameras other than the master camera are automatically controlled in direction by changing the control angle so as to face the gazing point. At this time, the cameraman moves the gazing point on the optical axis of the master camera by continuously changing the depth value that is the distance between the master camera and the gazing point. Then, the photographer changes (increases or decreases) the depth value using the depth operation device so that the gazing point overlaps the subject (that is, the subject is positioned at the center of the screen of all slave cameras). You can shoot a viewpoint video.

Kenichi Isa and four others, "The latest sports broadcast technology, the world's first! Utilizing EyeVisionTM for professional baseball broadcasts", Broadcast Technology, Kenrokukan Publishing, November 2001, p. 96-p. 105

  However, in the conventional multi-viewpoint robot camera system, the depth operation device linearly changes the depth value. Therefore, in the conventional multi-view robot camera system, when the gazing point is close to the master camera, the control angle of the slave camera changes greatly. On the other hand, in the conventional multi-viewpoint robot camera system, when the gazing point is far from the master camera, the control angle of the slave camera hardly changes. Therefore, the change amount of the control angle of the slave camera is not constant with respect to the operation amount of the depth operation device. For this reason, the conventional multi-viewpoint robot camera system has a problem that it is difficult to perform depth adjustment so that the gazing point overlaps the subject.

The problem of the conventional multi-viewpoint robot camera system will be described in detail with reference to FIG.
Gaze points Q _{1 to} Q _{4 in} FIG. 9A indicate positions at time t = 1 to 4 with respect to the gaze point moving on the optical axis β of the master camera MC.
As described above, since the depth value changes linearly, the distance between the gazing points Q ₁ and Q ₂ , the distance between the gazing points Q ₂ and Q ₃ , and the distance between the gazing points Q ₃ and Q ₄ are all equal. . That is, at time t = 1 to 2, the slave camera SC follows the gazing points Q _{1 to} Q ₂ , so that the control angle A increases. On the other hand, at time t = 3 to 4, the slave camera SC follows the gazing points Q _{3 to} Q ₄ , so the control angle B becomes small.

FIG. 9B illustrates the index positions corresponding to the gazing points Q _{1 to} Q ₄ on the rotary dial 91 of the depth operation device 90. The interval between the index positions is shorter in the order of the gazing points Q _{1 to} Q ₄ .
As shown in FIG. 9B, when the index 91a engraved on the rotary dial 91 overlaps the index position “Q ₁ ”, the point of sight Q ₁ in FIG. 9A is indicated. Similarly, it refers to the gazing point _{Q 2} in FIG. 9 when the indicator 91a overlaps the index position _{"Q 2"} (a), note shown in FIG. 9 (a) when the index 91a overlaps the index position _{"Q 3"} refers viewpoint _{Q 3,} refer to FIG. 9 fixation point _{Q 4} in (a) when the index 91a overlaps the index position _{"Q 4".}

Here, as shown in FIG. 9 (a), when moving from the gazing point Q ₁ to the gazing point Q _2, for controlling the angle A is large, as shown in FIG. 9 (b), the operation amount of the rotary dial 91 Will increase. On the other hand, as shown in FIG. 9 (a), when moving from the gazing point Q ₃ on the fixation point Q _4, for controlling the angle B is smaller, as shown in FIG. 9 (b), the operation amount of the rotary dial 91 Less. As described above, in the conventional multi-viewpoint robot camera system, the change amount of the control angle of the slave camera SC is not constant with respect to the operation amount of the rotary dial 91, and it becomes difficult to adjust the depth.

  Accordingly, an object of the present invention is to provide a robot camera control device, a program thereof, and a multi-view robot camera system that can easily adjust the depth.

  In view of the above-described problems, the robot camera control device according to the first invention of the present application is a gaze point that can move on the optical axis of a master camera that is a preset robot camera among a plurality of robot cameras. Is a robot camera control device that controls the posture of a slave camera that is a robot camera other than the master camera, and includes a camera posture operation unit, a depth value calculation unit, a camera calibration unit, and a gaze point calculation unit And a slave camera control unit.

  According to this configuration, the robot camera control apparatus operates the camera posture of the master camera by the camera posture operation unit. Also, in the robot camera control device, the depth value calculation unit performs a moving operation to move the gazing point, and from the detected value of the gazing point moving operation, the distance the gazing point moves per unit time as the distance from the master camera increases. Is calculated by changing the depth value, which is the distance from the master camera to the gazing point, in a non-linear manner.

  Also, the robot camera control device calculates, by camera calibration, the camera orientation of the master camera when the camera calibration is performed and the translation vector that is an external parameter of the master camera by the camera calibration unit.

  In addition, the robot camera control device calculates a rotation matrix in the master camera based on the difference in camera posture between the camera posture operation unit and the camera calibration unit by the gazing point calculation unit, and the depth included in the calculated rotation matrix The position of the gazing point is calculated by adding the translation vector to the product of the component vector and the depth value. In the robot camera control device, the slave camera control unit controls the posture of the slave camera so as to face the gazing point calculated by the gazing point calculation unit.

Here, the depth value calculation unit includes an operation unit in which a gazing point movement operation is performed, and an encoder that detects a movement operation with respect to the operation unit, and the maximum distance B between the baselines of the master camera and the slave camera, the depth value The maximum value d _max , the minimum value d _min of the depth value, and the maximum detected value t _max of the moving operation in the encoder are set in advance using the following equations (1) to (4). Preferably, k indicating the depth value is calculated from the detected value t (where 0 ≦ t ≦ t _max ) (second invention of the present application).

The robot camera control device according to the third invention of the present application further includes a master camera control unit that controls the posture of the master camera so as to take the camera posture operated by the camera posture operation unit, and the slave camera control unit includes: The focus of the slave camera is further controlled so as to focus on the gazing point calculated by the gazing point calculation unit.
According to such a configuration, the robot camera control apparatus can control the posture of the master camera and accurately focus the slave camera on the point of sight.

  Further, in view of the above-described problems, a multi-viewpoint robot camera system according to the fourth invention of the present application includes a master camera that is a preset robot camera among a plurality of robot cameras, and a robot other than the master camera. A multi-viewpoint robot camera system comprising a slave camera that is a camera and a robot camera control device that controls the posture of the slave camera so as to face a gazing point that can move on the optical axis of the master camera. Comprises a camera posture input unit, a depth value calculation unit, a camera calibration unit, a gaze point calculation unit, and a slave camera control unit.

  According to this configuration, the robot camera control apparatus operates the camera posture of the master camera by the camera posture operation unit. Also, in the robot camera control device, the depth value calculation unit performs a moving operation for moving the gazing point, and the moving distance of the gazing point increases as the distance from the master camera increases from the gazing point moving operation amount. In this way, the depth value, which is the distance from the master camera to the gazing point, is calculated in a non-linear manner.

  Also, the robot camera control device calculates, by camera calibration, the camera orientation of the master camera when the camera calibration is performed and the translation vector that is an external parameter of the master camera by the camera calibration unit.

  In addition, the robot camera control device calculates a rotation matrix in the master camera based on the difference in camera posture between the camera posture operation unit and the camera calibration unit by the gazing point calculation unit, and the depth included in the calculated rotation matrix The position of the gazing point is calculated by adding the translation vector to the product of the component vector and the depth value. In the robot camera control device, the slave camera control unit controls the posture of the slave camera so as to face the gazing point calculated by the gazing point calculation unit.

  Here, the robot camera control device according to the first invention of the present application is provided with hardware resources such as a CPU (Central Processing Unit), a memory, and a hard disk of a computer including a camera posture operation unit and a depth value calculation unit, a camera calibration unit, It can also be realized by a robot camera control program for cooperative operation as a gazing point calculation unit and a slave camera control unit (the fifth invention of the present application). This program may be distributed through a communication line, or may be distributed by writing in a recording medium such as a CD-ROM or a flash memory.

According to the present invention, the following excellent effects can be obtained.
According to the first, second, fourth, and fifth inventions of the present application, the robot camera control device, the program thereof, and the multi-viewpoint robot camera system have a depth so that the moving distance of the gazing point per unit time increases as the distance from the master camera increases. Since the value is changed non-linearly, the change of the control angle of the slave camera becomes constant regardless of the distance between the gazing point and the master camera, and the depth adjustment can be easily performed.
According to the third aspect of the present invention, the robot camera control device controls the posture of the master camera and accurately focuses the slave camera on the point of sight, so that it is possible to easily shoot a multi-viewpoint video.

It is a block diagram which shows the structure of the multiview robot camera system which concerns on embodiment of this invention. (A) is a figure explaining the operation part and encoder with which the depth operation part of FIG. 1 is provided, (b) is explanatory drawing explaining operation of a depth operation part. It is explanatory drawing explaining the change of a depth value in the depth operation part of FIG. It is explanatory drawing explaining the change of a depth value in the depth operation part of FIG. (A) And (b) is explanatory drawing explaining the change of the control angle of a slave camera in the multiview robot camera system of FIG. It is a flowchart which shows operation | movement of the multiview robot camera system of FIG. It is explanatory drawing explaining the Example of this invention. In the Example of this invention, it is a graph which shows the measurement result of the change of the control angle of a slave camera. (A) And (b) is explanatory drawing explaining the change of the control angle of a slave camera in the conventional multiview robot camera system.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate. In each embodiment, means having the same function are denoted by the same reference numerals and description thereof is omitted.

(Embodiment)
[Configuration of multi-viewpoint robot camera system]
A configuration of a multi-viewpoint robot camera system 100 according to an embodiment of the present invention will be described with reference to FIG.
As shown in FIG. 1, the multi-viewpoint robotic camera system 100 is for capturing multi-view image of an object, a robot camera controller 1, and the master camera MC, the slave camera _{SC 1, ..., SC n (} SC ).

In the present embodiment, the master camera MC and the slave camera SC are arranged in a straight line at a predetermined interval.
The number of slave cameras SC is n (where n is an integer satisfying n ≧ 1).

The robot camera control device 1 controls the master camera MC and the slave camera SC, and includes an operation interface unit 10, a camera calibration unit 20, a master camera control unit 30, a gazing point calculation unit 40, and a slave camera. Control units 50 ₁ ,..., 50 _n (50).

  The operation interface unit 10 is used by a cameraman to remotely control the master camera MC and the slave camera SC. The pan / tilt operation unit (camera posture operation unit) 11, the zoom / iris operation unit 13, and the depth operation unit ( A depth value calculation unit) 15.

  The pan / tilt operation unit 11 is used by a cameraman to operate pan and tilt (camera posture) of the master camera MC. Then, the pan / tilt operation unit 11 detects the pan value and the tilt value according to the operation of the cameraman, and outputs them to the pan / tilt / zoom / iris control unit 31 and the gaze point calculation unit 40.

  The zoom / iris operation unit 13 is used by a cameraman to operate zoom and iris of the master camera MC. Then, the zoom / iris operation unit 13 detects the zoom value and the iris value according to the operation of the cameraman, and outputs them to the pan / tilt / zoom / iris control unit 31 and the zoom / iris control unit 55.

The depth operation unit 15 performs a moving operation for moving the gazing point, and the depth value such that the moving distance of the gazing point per unit time increases as the distance from the master camera increases from the detected value of the gazing point moving operation. Is calculated.
The depth value indicates the distance from the master camera MC to the gazing point.

In the present embodiment, as shown in FIG. 2A, the depth operation unit 15 includes a rotary dial (operation unit) 16 that performs a gazing point movement operation, and an encoder 17 that detects a movement operation on the rotation dial 16. Is provided.
Here, as shown in FIG. 2B, the encoder value of the encoder 17 (detected value of the movement operation) increases as the rotary dial 16 is rotated clockwise. When the index 16a engraved on the rotary dial 16 reaches the index position “Δmaximum”, the encoder value of the encoder 17 becomes maximum.
On the other hand, the encoder value of the encoder 17 decreases as the rotary dial 16 is rotated counterclockwise. When the index 16a engraved on the rotary dial 16 reaches the index position “Δminimum”, the encoder value of the encoder 17 becomes minimum.
The depth operation unit 15 is preset with depth value conversion information in which the encoder value and the depth value are associated with each other, and converts the encoder value detected by the encoder 17 into a depth value with reference to the depth value conversion information. To do. Further, the depth operation unit 15 changes (increases / decreases) the converted depth value as described below, and outputs it to the focus control unit 33 and the gaze point calculation unit 40.

<Change in depth value>
With reference to FIG. 3 and FIG. 4, the change of the depth value by the depth operation part 15 is demonstrated (refer FIG. 1, 2 suitably).
As shown in FIG. 3, the cameraman operates the pan / tilt operation unit 11 and the zoom / iris operation unit 13 so that the master camera MC captures the subject α while viewing the captured video 90 of the master camera MC. Therefore, in the captured video 90 of the master camera MC, the subject α is captured at the center of the screen. On the other hand, in the captured video 91 of the slave camera SC, the subject α is not captured.
The zoom / iris operation unit 13 may be remotely operated by a person other than the cameraman.

  Next, the cameraman moves the gazing point Q on the optical axis β by continuously changing the depth value by operating the rotary dial 16. Then, as shown in FIG. 4, the cameraman adjusts the depth value by operating the rotary dial 16 so that the subject α is positioned at the center of the captured image 91 while viewing the captured image 91 of the slave camera SC.

  At this time, the depth operation unit 15 changes the depth value nonlinearly so that the moving distance of the gazing point Q per unit time increases as the distance from the master camera MC increases. Specifically, the depth operation unit 15 calculates the depth value k using the following equations (1) to (4).

The base line distance (camera interval) between the master camera MC and the slave camera SC farthest from the master camera MC is B, the maximum depth value is d _max , the minimum depth value is d _min , and is detected by the encoder 17. The maximum possible encoder value is t _max , and the encoder value detected by the encoder 17 is t (provided that 0 ≦ t ≦ t _max is satisfied).
The maximum distance B, maximum value d _max , minimum value d _min, and maximum detection value t _max are set in advance.

Returning to FIG. 1, the description of the multi-viewpoint robot camera system 100 will be continued.
The camera calibration unit 20 includes camera parameters (A _m , R _m , T _m , p _0m , t _0m , z _0m , f _0m , i _0m , A _n , R _n , T _n of the master camera MC and slave camera SC. , P _0n , t _0n , z _0n , f _0n , i _0n ) are calculated by camera calibration and stored in a storage device (not shown) such as a memory or a hard disk.

Here, the camera calibration unit 20 calculates the pan value p _0m and the tilt value t _0m of the master camera MC as the camera posture of the master camera MC when the camera calibration is performed, and stores it in the storage device. . The camera calibration unit 20 includes an internal parameter A _m of the master camera MC when performing camera calibration, and external parameters (rotation matrix R _m and the translation vector T _m), the master camera MC zoom value z _{0 m} Then, the focus value f _0m and the iris value i _0m are calculated and stored in the storage device.
Note that the subscript m indicates the master camera MC.

The camera calibration unit 20 includes an internal parameter A _n of the slave camera SC _n when performing camera calibration, and external parameters (rotation matrix R _n and the translation vector T _n), the pan value of the slave camera SC _n p _0n , tilt value t _0n , zoom value z _0n , focus value f _0n , and iris value i _0n are calculated and stored in the storage device.

The camera calibration method is described in, for example, the following references, and thus detailed description thereof is omitted.
Reference "Digital Image Processing: Association for Promotion of Image Information Education"

  The master camera control unit 30 controls the master camera MC, and includes a pan / tilt / zoom / iris control unit 31 and a focus control unit 33.

  The pan / tilt / zoom / iris control unit 31 is based on the pan value and tilt value input from the pan / tilt operation unit 11 and the zoom value and iris value input from the zoom / iris operation unit 13. This controls the pan, tilt, zoom and iris of the camera MC.

  Specifically, the pan / tilt / zoom / iris control unit 31 converts the pan value, tilt value, zoom value, and iris value into pan / tilt / zoom / iris control signals corresponding to the magnitudes of these values. . Then, the pan / tilt / zoom / iris control unit 31 outputs the converted pan / tilt / zoom / iris control signal to the master camera MC.

  The focus control unit 33 controls the focus of the master camera MC based on the depth value input from the depth operation unit 15. Specifically, the focus control unit 33 is preset with information that associates the depth value with the focus value, and refers to this information to calculate the focus value corresponding to the depth value. The focus control unit 33 converts the calculated focus value into a focus control signal corresponding to the magnitude of the value, and outputs the focus control signal to the master camera MC.

  The master camera MC is a preset robot camera among a plurality of robot cameras provided in the multi-viewpoint robot camera system 100. This master camera MC is, for example, a fixed robot camera mounted on an electric head. The master camera MC drives pan, tilt, zoom, and iris in accordance with the pan / tilt / zoom / iris control signal input from the pan / tilt / zoom / iris control unit 31. Further, the master camera MC drives the focus according to the focus control signal input from the focus control unit 33.

  The gazing point calculation unit 40 is based on the pan value and tilt value input from the pan / tilt operation unit 11, the depth value input from the depth operation unit 15, and the camera parameters stored in the camera calibration unit 20. The world coordinates of the point of interest are calculated.

Specifically, the gazing point calculation unit 40 uses the following equation (5), the master camera MC, and pan values p _m from the pan-tilt control member 11, the pan value p _{0 m} when camera calibration And the angle difference θ _pm is calculated.

The gaze point calculation unit 40 uses the following equation (6) to determine the angle between the tilt value t _m from the pan / tilt operation unit 11 and the tilt value t _0m at the time of camera calibration for the master camera MC. The difference θ _tm is calculated.

Next, the gaze point calculation unit 40 generates a rotation matrix _{R′rect_Cm} in the camera coordinate system using the angle difference θ _pm and the angle difference θ _tm as shown in the following equation (7).

Then, the _gaze point calculation unit 40 uses the rotation matrix R ′ _{rect — Cm} and the rotation matrix R _m that is an external parameter of the master camera MC, as shown in the following equation (8), to determine the camera posture in the world coordinate system. The rotation matrix R ′ _rectm is generated. This rotation matrix _R′rectm is a rotation matrix for changing the camera posture at the time of camera calibration from the camera posture at the time of camera calibration to the camera posture after being operated by the operation interface unit 10 for the master camera MC.
The rotation matrix _R′rectm corresponds to the “rotation matrix in the master camera” recited in the claims.

Here, as _shown in the following equation (9), the rotation matrix _R′rectm of equation (8) is a vector e ′ _xm indicating the X axis of the camera posture in the world coordinate system and the camera posture in the world coordinate system. It can be defined using a vector e ′ _ym indicating the Y axis and a vector (depth component vector) e _zm indicating the Z axis of the camera posture in the world coordinate system.
Note that the vectors e ′ _xm and e ′ _ym in the equation (9) are not used for the subsequent calculations.

Then, as shown in the following formula (10), the _gazing point calculation unit 40 calculates the rotation matrix R ′ _rectm , the translation vector T _m of the master camera MC, and the depth value k input from the depth operation unit 15. Using this, the world coordinate (position) P of the gazing point Q is calculated. This equation (10) indicates that the translation vector T _m is added to the product of the vector e _zm and the depth value k. Thereafter, the gazing point calculation unit 40 outputs the calculated world coordinates P of the gazing point Q to the pan / tilt control unit 51 and the focus control unit 53.

A change in the control angle of the slave camera SC will be described with reference to FIG. 5 (see FIGS. 1 and 2 as appropriate).
The gazing points Q _{1 to} Q _{4 in} FIG. 5A indicate the positions at the time t = 1 to 4 with respect to the gazing point moving on the optical axis β of the master camera MC.
As described above, since the depth operation unit 15 changes the depth value nonlinearly, the moving distance per unit time of the gazing points Q _{1 to} Q ₄ becomes wider as the distance from the master camera MC increases. That is, the distance between the gazing points Q ₂ and Q ₃ is larger than the distance between the gazing points Q ₁ and Q _{2, and} the distance between the gazing points Q ₃ and Q ₄ is larger than the distance between the gazing points Q ₂ and Q ₃ . Therefore, the control angle A of the slave camera SC at the time t = 1 to 2 is equal to the control angle B of the slave camera SC at the time t = 3 to 4.

FIG. 5B illustrates the index positions corresponding to the gazing points Q _{1 to} Q ₄ on the rotary dial 16. The index positions corresponding to these gazing points Q _{1 to} Q ₄ are all equally spaced.
As shown in FIG. 5B, when the index 16a overlaps the index position “Q ₁ ”, the point of sight Q ₁ in FIG. 5A is indicated. Similarly, when the index 16a overlaps the index position “Q ₂ ”, it points to the gazing point Q ₂ in FIG. 5A, and when the index 16a overlaps the index position “Q ₃ ”, the note of FIG. refers viewpoint _{Q 3,} refers to the fixation point _{Q 4} shown in FIG. 5 (a) when the index 16a overlaps the index position _{"Q 4".}

Here, as shown in FIG. 5 (a), when moving from the gazing point Q ₁ to the gazing point Q _2, and, even when moving from the gazing point Q ₃ on the fixation point Q _4, the control angle A, and B Therefore, as shown in FIG. 5B, the moving operation amount of the rotary dial 16 is constant. As described above, in the multi-viewpoint robot camera system 100, the change amount of the control angle of the slave camera SC is constant with respect to the movement operation amount of the rotary dial 16.

Returning to FIG. 1, the description of the multi-viewpoint robot camera system 100 will be continued.
The slave camera control units 50 ₁ ,..., 50 _n control the slave cameras SC ₁ ,..., SC _n , and the pan / tilt control units 51 ₁ , ..., 51 _n (51) and the focus control unit 53. ₁ ,..., 53 _n (53) and zoom / iris control units 55 ₁ ,..., 55 _n (55).
Incidentally, the slave camera control unit ₅₀ 1, ..., 50 _n, the slave camera _SC 1, ..., and arranged to correspond to SC _n, are all identical configuration.

The pan / tilt control unit 51 uses the world coordinates P of the gazing point Q input from the gazing point calculation unit 40 and the camera parameters (rotation matrix R _n and translation vector T _n ) stored in the camera calibration unit 20. Based on this, the posture (pan and tilt) of the slave camera SC is controlled.

Specifically, the pan-tilt control unit 51, as shown in equation (11) below, in order to control the pan and tilt unit vector e directed to the world coordinate P of the fixation point Q from the slave camera SC _{n zn} is calculated.
“‖‖” represents a norm.

In addition, the pan / tilt control unit 51 uses the calculated unit vector e _nz and the inverse matrix R ⁻¹ _n of the rotation matrix R _n as shown in the following equation (12) to set the slave in the camera coordinate system. A unit vector e _Czn from the camera SC _n toward the world coordinate P of the _gazing point Q is calculated.

Then, the pan / tilt control unit 51 calculates the pan value θ _Pn and the tilt value θ _Tn of the slave camera SC _n as shown in the following equations (13) to (15).
Note that e ₁ , e ₂ , and e _{3 represent} the X-axis, Y-axis, and Z-axis components of the unit vector e _Czn , respectively.

Further, the pan / tilt control unit 51 converts the calculated pan value θ _Pn and tilt value θ _Tn into pan / tilt control signals according to the magnitudes of these values. Thereafter, the pan / tilt control unit 51 outputs the converted pan / tilt control signal to the slave camera SC.

The focus control unit 53 sets the gaze point Q based on the world coordinates P of the gaze point Q input from the gaze point calculation unit 40 and the camera parameters (translation vector T _n ) stored in the camera calibration unit 20. The focus of the slave camera SC is controlled so that it is in focus.

Specifically, the focus control unit 53 calculates a vector E _zn from the slave camera SC _n toward the world coordinate P of the _gazing point Q in order to control the focus, as shown in the following formula (16).

Then, the focus control unit 53, as shown in the following equation (17) and (18), from the vector _{E zn,} calculates the distance _{k n} of the world coordinates P of the slave cameras SC _n and fixation point Q.
E ₁ , E ₂ , and E ₃ indicate the X-axis, Y-axis, and Z-axis components of the vector E _zn , respectively.

Further, the focus control unit 53, the distance k _n and set information associating a focus value in advance, by referring to this information, calculates a focus value corresponding to the distance k _n. Thereafter, the focus control unit 53 converts the calculated focus value into a focus control signal corresponding to the magnitude of the value, and outputs the focus control signal to the slave camera SC.

  The zoom / iris control unit 55 controls the zoom and iris of the slave camera SC based on the zoom value and the iris value input from the zoom / iris operation unit 13.

  Specifically, the zoom / iris control unit 55 converts the zoom value and the iris value into a zoom / iris control signal corresponding to the magnitude of the values. Then, the zoom / iris control unit 55 outputs the converted zoom / iris control signal to the slave camera SC.

  The slave camera SC is a robot camera other than the master camera MC among a plurality of robot cameras provided in the multi-viewpoint robot camera system 100. The slave camera SC is, for example, a fixed robot camera mounted on an electric head. In addition, the slave camera SC drives pan and tilt according to the pan / tilt control signal input from the pan / tilt control unit 51. Then, the slave camera SC drives the focus according to the focus control signal input from the focus control unit 53. Furthermore, the slave camera SC drives the zoom and the iris in accordance with the zoom / iris control signal input from the zoom / iris control unit 55.

[Operation of multi-viewpoint robot camera system]
The operation of the multi-viewpoint robot camera system 100 of FIG. 1 will be described with reference to FIG. 6 (see FIG. 1 as appropriate).
In the multi-viewpoint robot camera system 100, the camera calibration unit 20 calculates camera parameters of the master camera MC and the slave camera SC (step S1) and stores them in the storage device (step 2).

The multi-viewpoint robot camera system 100 detects a pan value and a tilt value according to the operation of the cameraman by the pan / tilt operation unit 11, and performs a pan / tilt / zoom / iris control unit 31, a gaze point calculation unit 40, and the like. Output to.
In the multi-viewpoint robot camera system 100, the zoom / iris operation unit 13 detects the zoom value and the iris value according to the operation of the cameraman, and the pan / tilt / zoom / iris control unit 31 and the zoom / iris control unit 55. And output.
The multi-viewpoint robot camera system 100 uses the expressions (1) to (4) by the depth operation unit 15 to increase the moving distance of the gazing point per unit time as the distance from the master camera increases. Such a depth value is calculated (step S3).

The multi-viewpoint robot camera system 100 converts a pan value, a tilt value, a zoom value, and an iris value into a pan / tilt / zoom / iris control signal by the pan / tilt / zoom / iris control unit 31 to obtain a master camera MC. Output to.
In the multi-viewpoint robot camera system 100, the focus control unit 33 calculates a focus value corresponding to the depth value, converts the calculated focus value into a focus control signal, and outputs the focus control signal to the master camera MC (step S4).

In the multi-viewpoint robot camera system 100, the gaze point calculation unit 40 calculates the rotation matrix _R′rectm of Expression (8) based on the difference in camera posture between the pan / tilt operation unit 11 and the camera calibration unit 20. .
In the multi-viewpoint robot camera system 100, the _gaze point calculation unit 40 adds the translation vector T _m to the product of the vector e _zm and the depth value k included in the rotation matrix R ′ _rectm as in Expression (10). Thus, the world coordinate P of the gazing point Q is calculated (step S5).

In the multi-viewpoint robot camera system 100, the pan / tilt control unit 51 controls the pan and tilt of the slave camera SC so as to face the gazing point Q.
In the multi-viewpoint robot camera system 100, the focus control unit 53 controls the focus of the slave camera SC so as to focus on the gazing point Q.
In the multi-viewpoint robot camera system 100, the zoom / iris controller 55 controls the zoom and iris of the slave camera SC based on the zoom value and the iris value (step S6).

  As described above, since the multi-viewpoint robot camera system 100 according to the embodiment of the present invention changes the depth value nonlinearly by the depth operation unit 15, the slave camera SC can be controlled regardless of the distance between the gazing point and the master camera. Changes in the control angles A and B are linear (constant), and depth adjustment can be easily performed (see FIG. 5).

The present invention is not limited to the embodiments, and various modifications can be made without departing from the spirit of the present invention. Hereinafter, the modification of this invention is demonstrated concretely.
In the above-described embodiment, the depth operation unit 15 is described as including the rotary dial 16, but the present invention is not limited to this.
For example, the depth operation unit 15 may include a slider instead of the rotary dial 16.

With reference to FIG. 7 and FIG. 8, the experiment result of the change in the control angle of the slave camera SC will be described as an embodiment of the present invention (see FIG. 1 as appropriate).
Using the multi-viewpoint robot camera system 100 of FIG. 1, an experiment for measuring a change in the control angle of the slave camera SC was performed. Here, as shown in FIG. 7, nine robot cameras are arranged in a horizontal line, one central robot camera is set as a master camera MC, and the remaining eight robot cameras are set as slave cameras SC. . At this time, the maximum depth value d _max = 100 meters, the minimum value range d _min = 1 meter, the baseline distance B = 20 meters, and the maximum encoder value t _max = 10000.

Then, in response to operation of the operation unit 16 by the photographer, to determine the change in the control angle of the slave camera SC _1. Further, as a comparative example, the same camera arrangement and 7, using a conventional multi-view robotic camera system, to measure the change in the control angle of the slave camera SC _1.

In FIG. 8, the measurement results of the example are illustrated by solid lines, and the measurement results of the comparative example are illustrated by broken lines.
8, in the comparative example, a rather dashed gradient is constant, it can be seen that the control angle of the slave camera SC ₁ greatly changes. That is, in the comparative example, (when the small encoder value) fixation point Q is when close to the master camera MC, the change in the control angle of the slave camera SC ₁ increases. In the comparative example, when the gazing point Q is far from the master camera MC (when the encoder value is large), the change in the control angle of the slave camera SC is small. Therefore, in the comparative example, it is considered that it is difficult to adjust the depth so that the change in the control angle of the slave camera SC is not constant and the gazing point Q overlaps the subject.

Meanwhile, in the embodiment, a solid line slope is constant, it can be seen that the control angle of the slave camera SC ₁ is changed to the constant. That is, in the embodiment, regardless of the distance between the gazing point Q and the master camera MC, the change in the control angle of the slave camera SC ₁ is constant, is considered in comparison with the comparative example, it is easy depth adjustment .

DESCRIPTION OF SYMBOLS 1 Robot camera control apparatus 10 Operation interface part 11 Pan / tilt operation part (camera attitude | position operation part)
13 Zoom / iris operation section 15 Depth operation section (depth value calculation section)
16 Rotating dial (operation unit)
17 Encoder 20 Camera calibration unit 30 Master camera control unit 31 Pan / tilt / zoom / iris control unit 33 Focus control unit 40 Gaze point calculation unit 50 Slave camera control unit 51 Pan / tilt control unit 53 Focus control unit 55 Zoom / iris Control unit 100 Multi-viewpoint robot camera system MC Master camera SC Slave camera

Claims (5)

Of the plurality of robot cameras, the posture of the slave camera, which is a robot camera other than the master camera, is set so as to face a gazing point that can be moved on the optical axis of the master camera that is a preset one robot camera. A robot camera control device for controlling,
A camera attitude operation unit for operating the camera attitude of the master camera;
From the master camera, a moving operation for moving the gazing point is performed, and from the detected value of the gazing point moving operation, the moving distance of the gazing point per unit time increases as the distance from the master camera increases. A depth value calculation unit that calculates a depth value that is a distance to the gazing point;
A camera calibration unit that calculates a camera orientation of the master camera when performing camera calibration and a translation vector that is an external parameter of the master camera by the camera calibration;
A rotation matrix in the master camera is calculated based on a difference in camera attitude between the camera attitude operation unit and the camera calibration unit, and a product of a depth component vector and the depth value included in the calculated rotation matrix A gazing point calculation unit for calculating the position of the gazing point by adding the translation vector to
A slave camera control unit that controls the posture of the slave camera so as to face the gazing point calculated by the gazing point calculation unit;
A robot camera control device comprising: The depth value calculation unit
An operation unit for performing an operation of moving the gazing point;
An encoder for detecting a movement operation on the operation unit;
The maximum baseline B of the master camera and the slave camera, the maximum value d _{max of} the depth value, the minimum value d _{min of} the depth value, and the maximum detection value t _max of the moving operation in the encoder are preset. In addition, the following equation (1) to equation (4) are used to calculate k indicating the depth value from the detected value t (where 0 ≦ t ≦ t _max ) of the movement operation in the encoder. The robot camera control device according to claim 1.

A master camera control unit that controls the posture of the master camera so as to take the camera posture operated by the camera posture operation unit;
The robot camera according to claim 1, wherein the slave camera control unit further controls the focus of the slave camera so as to focus on the gazing point calculated by the gazing point calculation unit. Control device. Among a plurality of robot cameras, a master camera that is a preset robot camera, a slave camera that is a robot camera other than the master camera, and a gaze point that can move on the optical axis of the master camera A multi-viewpoint robot camera system comprising a robot camera control device for controlling the posture of the slave camera so as to face,
The robot camera control device is
A camera attitude operation unit for operating the camera attitude of the master camera;
From the master camera, a moving operation for moving the gazing point is performed, and from the detected value of the gazing point moving operation, the moving distance of the gazing point per unit time increases as the distance from the master camera increases. A depth value calculation unit that calculates a depth value that is a distance to the gazing point;
A camera calibration unit that calculates a camera orientation of the master camera when performing camera calibration and a translation vector that is an external parameter of the master camera by the camera calibration;
A rotation matrix in the master camera is calculated based on a difference in camera attitude between the camera attitude operation unit and the camera calibration unit, and a product of a depth component vector and the depth value included in the calculated rotation matrix A gazing point calculation unit for calculating the position of the gazing point by adding the translation vector to
A slave camera control unit that controls the posture of the slave camera so as to face the gazing point calculated by the gazing point calculation unit;
A multi-viewpoint robot camera system comprising: Of the plurality of robot cameras, the posture of the slave camera, which is a robot camera other than the master camera, is set so as to face a gazing point that can be moved on the optical axis of the master camera that is a preset one robot camera. In order to control, a camera posture operation unit for operating the camera posture of the master camera and a moving operation for moving the gazing point are performed, and from the detection value of the gazing point moving operation, the farther from the master camera, A computer comprising a depth value calculation unit that calculates a depth value that is a distance from the master camera to the gazing point so that a moving distance of the gazing point per unit time is increased.
A camera calibration unit that calculates a camera orientation of the master camera when camera calibration is performed and a translation vector that is an external parameter of the master camera by the camera calibration;
A rotation matrix in the master camera is calculated based on a difference in camera attitude between the camera attitude operation unit and the camera calibration unit, and a product of a depth component vector and the depth value included in the calculated rotation matrix A gazing point calculation unit for calculating the position of the gazing point by adding the translation vector to
A slave camera control unit that controls the posture of the slave camera so as to face the gazing point calculated by the gazing point calculation unit;
Robot camera control program to function as

Images