JP2013105002A - 3d video photography control system, 3d video photography control method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a 3D video photography control system which controls the positions and direction of a pair of imaging apparatuses to obtain a suitable 3D effect or maintain safety, in comprehensive consideration of a photographing condition and a viewing condition.SOLUTION: Base line lengths and angles of convergence of a pair of imaging apparatuses are controlled on the basis of an integrated model into which 3D photographic/broadcast models and a 3D subject models are integrated. A base line length determination unit 6d-1 determines the base line lengths on the basis of a relation represented by an integrated model expression 1 (the relation that a ratio of a base line length of an imaging apparatus to a base line length of viewer's both eyes is proportional to a viewing ratio), and a camera control unit 6e moves the pair of imaging apparatuses so that the pair of imaging apparatuses have the determined base line lengths. A convergence angle control unit 2d-2 controls the angles of convergence by a viewing ratio obtained on the basis of a relation represented by an integrated model expression 2 (the relation that the result from multiplication of a viewing ratio and an angle of view of an imaging apparatus is a constant), and the base line length determination unit 6d-1 determines the base line lengths on the basis of the viewing ratio and the integrated model expression 1.

Description

  The present invention relates to a system for controlling the position and direction of an imaging apparatus (camera) for capturing 3D video.

  In recent years, as represented by 3D movies and 3D television, three-dimensional (3D) images capable of three-dimensional expression are often used in various fields. Such 3D video is usually shot by a pair of 3D shooting cameras (a left-eye video camera and a right-eye video camera).

  The position of the pair of 3D shooting cameras is always fixed, and shooting is performed in that state. At this time, the distance (baseline length) of the 3D camera is usually kept at about 65 mm so that problems such as the miniature garden effect (giantness of the giant) do not occur. Further, the relationship between the interval between the 3D imaging cameras and the angle of view is manually adjusted (for example, manually by the user).

  Such adjustment of the base line length for the 3D effect is disclosed in “Three-dimensional imaging apparatus and method and program” in Patent Document 1. In this document, two 3D images are obtained in order to obtain a three-dimensional effect of 3D video. The point which adjusts the base line length of the camera for imaging | photography is described.

JP 2010-81010 A

  In the above-described shooting method using two 3D shooting cameras, that is, a shooting method in which the interval between the cameras is fixed and the relationship between the interval and the angle of view is manually adjusted, basically, Only 3D images at a distance can be taken.

  When shooting an object at a long distance, it is necessary to use a zoom or a telephoto lens. Therefore, if the camera interval is set to 65 mm at this time, it is not possible to obtain a stereoscopic effect of the shooting target. As described above, in the conventional 3D shooting, the shooting conditions including the camera interval and the angle of view are set based on the experience and intuition of the photographer (for example, a cameraman or a 3D grapher).

  For example, in the conventional 3D shooting method, the stereoscopic effect is different between 3D shooting of a near scene and a far scene. In particular, it is very difficult to perform 3D shooting such as sports that are difficult to shoot at close range.

  However, up to now, no relational expression has been provided that represents a suitable relationship between shooting conditions such as camera angle of view, base line length, convergence angle, and distance to the target, and comprehensive consideration of shooting conditions and viewing conditions has been provided. The theoretical system of the 3D imaging system has not been established. Furthermore, since there is no clear relational expression of shooting conditions that considers both safe parallax (for example, parallax is within 50 mm) and 3D effect, how to set the baseline length and angle of view of the camera in this case as well It depends on the experience and intuition of the photographer.

  As described above, none of the conventional techniques including the above-described Patent Document 1 specifically show a relational expression for obtaining a suitable photographing condition in 3D photographing or a suitable control method. In the above-mentioned Patent Document 1, there is a description that the base line lengths of the two 3D photographing cameras are changed in order to solve the problem that stereoscopic viewing in the photographed image becomes impossible when the zoom operation is performed. However, what kind of information is used to determine the baseline length after the change is not specified.

  Accordingly, an object of the present invention is to comprehensively consider shooting conditions and viewing conditions, and control at least one of the position and direction of a pair of cameras in order to obtain a suitable 3D effect or maintain safety. The object is to provide a 3D video shooting control system.

  In order to solve the above-described problem, first, all the parameters related to the 3D effect and the safety parallax are theoretically extracted with respect to the two 3D photographing cameras, and their relationship is expressed by a mathematical expression. Next, from the photographer's standpoint, parameters (for example, subject parallax, angle of view (zoom), convergence angle, etc.) that need to be adjusted in actual shooting are determined, and other parameters are based on the parameters based on the parameters. Calculated by Thereafter, the 3D imaging camera is controlled based on the obtained parameters.

  A first embodiment of the present invention is a 3D video shooting control system for performing shooting control of 3D video. The 3D video imaging control system includes: a parallax calculation unit that calculates parallax related to a subject from video obtained from a pair of imaging devices that shoot a subject; a baseline length determination unit that determines a baseline length of the pair of imaging devices; Baseline length control means for controlling the pair of imaging devices to move so as to have the determined baseline length. Here, the baseline length determination means is configured to determine the ratio between the baseline length of the pair of imaging devices and the baseline length of both human eyes, the distance (actual distance) from the baseline of the pair of imaging devices to the subject at the time of shooting, and the 3D video. The base line length of the pair of imaging devices is determined based on the relationship (the pull-in rate) with the distance (sensing distance) from the baseline of both human eyes to the subject at the time of viewing (sensing distance) being correlated by a predetermined function. The sensing distance is determined based on the parallax.

  With such a 3D video shooting control system according to the first embodiment of the present invention, a baseline length that provides a suitable 3D effect is “the ratio between the baseline length of a pair of imaging devices and the baseline length of both human eyes” The pull-in rate is obtained based on “a relationship associated with a predetermined function”, and the pair of imaging devices (cameras) is adjusted to the determined baseline length. Note that the baseline length of both human eyes may be a constant set in advance in addition to the one specified by the user or the like.

  According to a second embodiment of the present invention, in the first embodiment, a parallax receiving unit that receives a designated parallax and a convergence angle control unit that controls a pair of imaging devices to move in order to adjust the convergence angle. Is a 3D video shooting control system. Here, the convergence angle control means is configured to move the pair of imaging devices so as to obtain a convergence angle corresponding to the designated parallax, and the baseline length determination means is configured to obtain the sensing distance based on the designated parallax. Is done.

  When the parallax is specified by the user or the like by the 3D video shooting control system according to the second embodiment of the present invention, the vergence angle and the base line length at which a suitable 3D effect is obtained are based on the parallax. The pair of imaging devices (cameras) is adjusted to the baseline length and the convergence angle.

  According to a third embodiment of the present invention, in the first embodiment, a pull-in rate receiving unit that receives a specified pull-in rate, and a convergence angle that controls to move a pair of imaging devices to adjust the convergence angle. A 3D video shooting control system further comprising control means. Here, the convergence angle control means moves the pair of imaging devices so as to obtain a convergence angle corresponding to the parallax obtained based on the designated pull-in rate, and the baseline length determining means is based on the designated pull-in rate. It is comprised so that the base line length of a pair of imaging device may be determined.

  When the pull-in rate is specified by the user or the like by the 3D video shooting control system according to the third embodiment of the present invention, the convergence angle and the base line that provide a suitable 3D effect based on the pull-in rate The length is determined, and the pair of imaging devices (cameras) is adjusted to the baseline length and the convergence angle.

  A fourth embodiment of the present invention is a 3D video shooting control system further comprising a convergence angle control unit that controls to move a pair of imaging devices in order to adjust a convergence angle in the first embodiment. Here, the pull-in rate is obtained from the angle of view based on a relationship in which the multiplication of the pull-in rate and the angle of view of the pair of imaging devices is a constant, and the convergence angle control means is obtained based on the obtained pull-in rate. The pair of imaging devices are moved so as to obtain a convergence angle corresponding to the parallax to be obtained, and the baseline length determining means is configured to determine the baseline length of the pair of imaging devices based on the obtained pull-in rate.

  By such a 3D video shooting control system according to the fourth embodiment of the present invention, the pull-in rate is obtained based on “a relationship in which the multiplication between the pull-in rate and the angle of view of the pair of imaging devices is a constant”, The baseline length and the convergence angle are different from this relationship (that is, “a relationship in which the ratio between the baseline length of the pair of imaging devices and the baseline length of both human eyes and the pull-in rate is associated by a predetermined function”). Is required. A suitable 3D effect is obtained based on the convergence angle and the baseline length thus obtained, and the pair of imaging devices (cameras) are adjusted to the baseline length and the convergence angle.

  According to a fifth embodiment of the present invention, in the fourth embodiment, the angle of view is measured by an angle-of-view measuring unit that measures the angle of view, or is calculated by a focal length and an imaging element width in a pair of imaging devices. Configured as follows.

  A sixth embodiment of the present invention is a 3D video shooting control system further comprising a horizontal magnification enlargement receiving means for receiving a designated horizontal magnification in the fourth or fifth embodiment. Here, the pull-in rate is configured to be obtained from the angle of view and the horizontal magnification factor based on a relationship in which the multiplication factor of the pull-in rate, the angle of view of the pair of imaging devices, and the specified horizontal magnification factor becomes a constant. .

  A seventh embodiment of the present invention is a 3D video shooting control system further comprising a depth width ratio receiving means for receiving a specified depth width ratio in the first to sixth embodiments. Here, the base line length determining unit is configured to determine whether the ratio between the base line length of the pair of imaging devices and the baseline length of both human eyes is proportional to the multiplication of the pull-in rate and the depth width ratio. Configured to determine a baseline length.

  According to an eighth embodiment of the present invention, in the second to seventh embodiments, an allowable parallax range receiving unit that receives a specified allowable parallax range, and shooting that adjusts various parameters so that the parallax is within the allowable parallax range. A 3D video shooting control system further comprising condition adjusting means. Here, the parallax calculation unit calculates the parallax range of the entire image in the video obtained from the pair of imaging devices, and the shooting condition adjustment unit calculates the pair of parallax from the calculated parallax range of the entire image and the allowable parallax range. At least one of an allowable range for the base line length of the imaging device and an allowable range for the convergence angle is determined, and the base line length or the convergence angle of the pair of imaging devices is adjusted to the determined allowable range.

  By such a 3D video shooting control system according to the eighth embodiment of the present invention, the base line length and the convergence angle of the camera are controlled so that 3D video is shot within a safe parallax range.

  According to a ninth embodiment of the present invention, in any one of the first to eighth embodiments, the baseline length determining means obtains the sensing distance based on parallax and viewing conditions, and the viewing conditions are the baseline lengths of human eyes. , The width of the screen, and the distance between the base line of the human binocular and the screen.

  According to a tenth embodiment of the present invention, in the first to ninth embodiments, the actual distance is measured by an actual distance measuring unit that measures the actual distance, or the parallax, the baseline length of the pair of imaging devices, and the convergence angle. , And the angle of view.

  In an eleventh embodiment of the present invention, in the first to tenth embodiments, 3D video shooting control is performed for each of two or more pairs of imaging devices configured by three or more imaging devices that shoot a subject. Configured as follows.

  A twelfth embodiment of the present invention is a 3D video shooting control method for performing shooting control of 3D video. The 3D video shooting control method includes: a parallax calculation step that calculates parallax related to a subject from video obtained from a pair of imaging devices that shoot a subject; a baseline length determination step that determines a baseline length of the pair of imaging devices; A baseline length control step for controlling to move the pair of imaging devices so as to have the determined baseline length. Here, in the baseline length determination step, the ratio between the baseline length of the pair of imaging devices and the baseline length of both human eyes is the distance from the baseline of the pair of imaging devices to the subject at the time of shooting (actual distance) and the 3D video The base line length of the pair of imaging devices is determined based on the relationship (the pull-in rate) with the distance (sensing distance) from the baseline of both human eyes to the subject at the time of viewing (sensing distance) being correlated by a predetermined function. The sensing distance is determined based on the parallax.

  In a thirteenth embodiment of the present invention, in the twelfth embodiment, a parallax receiving step for receiving a designated parallax and a convergence angle control step for controlling the pair of imaging devices to move in order to adjust the convergence angle Is a 3D video shooting control method. Here, the convergence angle control step is configured to move the pair of imaging devices so that the convergence angle corresponds to the designated parallax, and the baseline length determination step is configured to obtain the sensing distance based on the designated parallax. Is done.

  In a fourteenth embodiment of the present invention, in the twelfth embodiment, a pull-in rate receiving step for receiving a specified pull-in rate, and a convergence angle for controlling the pair of imaging devices to move to adjust the vergence angle. A 3D video shooting control method further comprising a control step. Here, the convergence angle control step moves the pair of imaging devices so as to obtain a convergence angle corresponding to the parallax obtained based on the specified pull-in rate, and the baseline length determination step is based on the specified pull-in rate. It is comprised so that the base line length of a pair of imaging device may be determined.

  The fifteenth embodiment of the present invention is a 3D video shooting control method according to the twelfth embodiment, further comprising a convergence angle control method for controlling the pair of imaging devices to move in order to adjust the convergence angle. Here, the pull-in rate is obtained from the angle of view based on a relationship in which the multiplication of the pull-in rate and the angle of view of the pair of imaging devices is a constant, and the convergence angle control step is obtained based on the obtained pull-in rate. The pair of imaging devices are moved so as to have a convergence angle corresponding to the parallax, and the baseline length determination step is configured to determine the baseline length of the pair of imaging devices based on the obtained pull-in rate.

  In a sixteenth embodiment of the present invention, a computer calculates parallax calculation means for calculating parallax related to a subject from video obtained from a pair of imaging devices that shoot the subject, and a baseline length for determining a baseline length of the pair of imaging devices. This is a program for performing 3D video shooting control for performing 3D video shooting control, which functions as a determination unit and a baseline length control unit that controls to move the pair of imaging devices so as to have the determined baseline length. Here, the baseline length determining means is configured such that the ratio between the baseline length of the pair of imaging devices and the baseline length of both human eyes is the distance (actual distance) from the baseline of the pair of imaging devices to the subject at the time of shooting and the 3D video. The base line length of the pair of imaging devices is determined based on the relationship (the pull-in rate) with the distance (sensing distance) from the baseline of both human eyes to the subject at the time of viewing (sensing distance) being correlated by a predetermined function. The sensing distance is determined based on the parallax.

  According to a seventeenth embodiment of the present invention, in the sixteenth embodiment, a parallax receiving unit that receives a designated parallax, and a convergence angle control that controls to move a pair of imaging devices to adjust the convergence angle It is a program further including a program that functions as means. Here, the convergence angle control means is configured to move the pair of imaging devices so as to obtain a convergence angle corresponding to the designated parallax, and the baseline length determination means is configured to obtain the sensing distance based on the designated parallax. Is done.

  According to an eighteenth embodiment of the present invention, in the sixteenth embodiment, a pull-in rate receiving unit that receives a specified pull-in rate, and a congestion that controls to move a pair of imaging devices to adjust a convergence angle The program further includes a program that functions as an angle control means. Here, the convergence angle control means moves the pair of imaging devices so as to obtain a convergence angle corresponding to the parallax obtained based on the designated pull-in rate, and the baseline length determining means is based on the designated pull-in rate. It is comprised so that the base line length of a pair of imaging device may be determined.

  A nineteenth embodiment of the present invention is a program that further includes a program that functions as a convergence angle control unit that controls to move a pair of imaging devices in order to adjust a convergence angle in the sixteenth embodiment. Here, the pull-in rate is obtained from the angle of view based on a relationship in which the multiplication of the pull-in rate and the angle of view of the pair of imaging devices is a constant, and the convergence angle control means is obtained based on the obtained pull-in rate. The pair of imaging devices are moved so as to have a convergence angle corresponding to the parallax to be obtained, and the baseline length determining means is configured to determine the baseline length of the pair of imaging devices based on the obtained pull-in rate.

  In the 3D video shooting control system according to the present invention, a parameter that comprehensively considers shooting conditions and viewing conditions is obtained. In order to obtain a suitable 3D effect by the parameters or to maintain safety, a pair of cameras At least one of the position and direction is controlled.

It is a basic diagram which shows the outline of the 3D image | video imaging | photography control system which concerns on one Embodiment of this invention, and the other apparatus related to this. 1 is a schematic diagram schematically illustrating a 3D shooting mechanism of a 3D video shooting control system according to an embodiment of the present invention. It is a basic diagram for demonstrating imaging | photography conditions and viewing conditions regarding the 3D image | video imaging | photography control system which concerns on one Embodiment of this invention. It is a basic diagram for demonstrating the formula which calculates | requires an actual distance and a sensing distance regarding the 3D image | video imaging | photography control system which concerns on one Embodiment of this invention. It is a functional block diagram regarding the 3D imaging | photography mechanism and control apparatus of the 3D image | video imaging | photography control system which concern on one Embodiment of this invention. It is a basic diagram which shows the example of the hardware constitutions of the computer which comprises the control apparatus of the 3D video imaging control system which concerns on one Embodiment of this invention.

  A 3D video imaging control system according to an embodiment of the present invention is a pair of 3D imaging cameras so that a suitable 3D effect is obtained or a suitable safety parallax is obtained based on imaging conditions and viewing conditions. And controlling at least one of the interval (baseline length) and the direction (convergence angle).

  First, referring to FIG. 1, a 3D video shooting control system 7 according to an embodiment of the present invention and an apparatus for displaying 3D video data acquired by the 3D video shooting control system 7 to a viewer. Will be described.

  1 includes a 3D imaging mechanism 5 and a control device 6. The 3D imaging mechanism 5 includes a camera 1, an actuator 2, a movement amount measuring device 3 (see FIG. 2), and a camera control platform 4. including. The camera 1 is an imaging device for capturing 3D video images, and includes a pair of cameras, that is, a left image capturing camera 1-L and a right image capturing camera 1-R. The camera 1 is directly linked and controlled without using the control device 6. In the present specification, when these two cameras are collectively referred to, they are referred to as a camera 1.

  The actuator 2 moves the camera 1 (each of the left image capturing camera 1-L and the right image capturing camera 1-R) in accordance with a control signal from the control device 6. The actuator 2 realizes the rotational movement of the camera 1 in the three axis directions and the horizontal movement in the baseline direction. In this example, the actuator 2 is used as the moving means, but the camera 1 can be moved to an arbitrary position and direction by other known means. The movement amount measuring device 3 measures the movement amount (rotation angle, movement distance) of the camera 1 moved by the actuator 2 and transmits the measurement result to the control device 6.

  Two cameras 1 are basically provided at the same height on the base 4 a of the camera control platform 4. The left image capturing camera 1-L is attached to the base 4a by the actuator 2 so as to be able to rotate and move in three axial directions and horizontally in the baseline direction. Similarly, the right image capturing camera 1-R is also attached to the base 4a by the actuator 2 so as to be able to rotate and move in the three axis directions and horizontally in the baseline direction. The base 4a is attached to the camera control pan head 4 in a state in which pan rotation is possible.

  The control device 6 performs predetermined processing on the 3D video data received from the camera 1 and transmits this to the transmission device 8. Thereafter, the 3D video data is transmitted to the receiving device 9 (for example, far away from the transmitting device 8) via a network such as the Internet or WAN, and is further arranged in a movie theater, a broadcasting station, a relay station, a home, etc. Sent to the 3D image quality adjustment apparatus 10. The 3D image quality adjustment apparatus 10 performs image quality adjustment of the 3D video data and other video processing.

  Next, the 3D video data that has undergone image quality adjustment and video processing is provided for viewing by the viewer. For example, as shown in FIG. 1, in a movie theater, 3D video data is transmitted to the 3D projector 11, and 3D video is projected from the 3D projector 11 onto the screen 12. On the other hand, in a general home, the transmitted 3D video data is displayed on the monitor of the 3D television 13.

  In the example of FIG. 1, the 3D image capturing control system 7 is configured to include the 3D image capturing mechanism 5, but does not include the camera 1, the camera control platform 4, or the like, and substantially controls the camera 1. It may be configured to include only the control device 6.

  Next, adjustment / movement of the position and direction of the camera 1 in the 3D imaging mechanism 5 will be described with reference to FIG.

  The optical axis 15L of the left camera (left image capturing camera) 1-L and the optical axis 15R of the right camera (right image capturing camera) 1-R are ½ of the angle formed on the plane including these optical axes. Is the convergence angle (see FIG. 4), and the subjects (targets) of the left camera 1-L and the right camera 1-R are shown in FIG.

The angle of the subject 20 with respect to the optical axis 15L of the left camera 1-L is expressed by an angle φ _{l-p in the} pan direction, an angle φ _{l-t in the} chilled direction, and an angle φ _l-r in the roll direction, respectively. . Similarly, the angle of the object 20 relative to the optical axis 15R of the right camera 1-R, the angle in the pan direction phi _r-p, chilled direction of angle phi _r-t, with the roll direction of the angle phi _r-r, respectively expressed.

  FIG. 2 shows the base line length b of the left camera 1-L and the right camera 1-R. Here, the baseline length is the distance between the optical centers of two cameras (or human eyes), and the optical center is the position of a pinhole when the camera is approximated to a pinhole model. The baseline length b can also be expressed as the distance between the rotation axes of the two cameras in the pan direction. In addition, as the base line length b of the two cameras, another base point that approximates the distance can be used (however, in this case, it is necessary to consider the error of the parameter).

  The actuator 2 is configured to have, for example, three degrees of freedom for each of the two cameras, and is arranged to rotate the camera about each rotation axis. Further, an actuator 2 for changing the base line length is provided. The base 4a on which the two cameras 1 are installed can be rotated in the pan direction as described above, and this rotation can be controlled by the actuator 2.

In FIG. 2, the actuator 2-LP is an actuator for rotating the pan direction of the left camera 1-L, the rotation angle is represented by θ _1-p , and the actuator 2-RP is represented by the right camera 1-P. R is an actuator for rotating in the pan direction, and the rotation angle is represented by θ _r−p . The actuator 2-LT is a tilt direction rotation actuator of the left camera 1-L, and the rotation angle is represented by θ _1-t , and the actuator 2-RT is the tilt direction of the right camera 1-R. It is a rotation actuator, and its rotation angle is represented by θ _r−t . Actuator 2-LR is an actuator for rotating the left camera 1-L in the roll direction (visual axis direction), and its rotation angle is represented by θ _l-r , and actuator 2-RR is the right camera 1. It is an actuator for rotating in the roll direction (visual axis direction) of −R, and the rotation angle is represented by θ _r−r .

The actuator 2-B is an actuator that changes the interval between the two cameras 1 (changes the baseline length), and the interval between the cameras 1 is represented by the baseline length b as described above. The actuator 2-N is the foundation 4a of the camera control camera platform 4 an actuator for rotating the pan direction, its rotation angle is expressed by theta _n.

  Here, when a control signal for changing the base length b to a predetermined distance is transmitted from the control device 6 shown in FIG. 1, the actuator 2-B operates to change the base length to the predetermined distance. Similarly, when a control signal for changing the convergence angle of the two cameras 1 to a predetermined angle is transmitted from the control device 6, the actuator 2-LP and the actuator 2-LR are set to the predetermined angle. Operates to change the convergence angle.

  In FIG. 2, the base line length is schematically represented as a mechanism that is changed by one actuator 2-B. However, as shown in FIG. 1, the base line length is controlled by two actuators that move each camera 1. May be. Further, the two actuators (actuator 2-LP and actuator 2-LR) that change the convergence angle basically operate so as to have a predetermined convergence angle by rotating the respective cameras by the same angle. .

  In the embodiment described in this specification, the position and direction of the camera 1 are adjusted by an actuator (motor). However, the present invention is not limited to this, and the camera 1 may be moved by another mechanism. .

  Here, with reference to FIG. 3, modeling of shooting and broadcasting of 3D video will be described. FIG. 3A shows a 3D shooting model in which two cameras (the left camera 1-L and the right camera 1-R) perform 3D shooting by the convergence shooting method. The left camera 1-L and the right camera 1-R are arranged at an interval of the base line length b, and 1/2 of the angle formed by the optical axes of the two cameras is represented by the convergence angle β. The angle of view of each of the left camera 1-L and the right camera 1-R is α, and the distance from the optical center of the camera (the distance of a straight line extending perpendicularly to the base line from the midpoint of the base line connecting the two optical centers) A plane in which) is 1 is called a normalized plane, and a field angle range on the normalized plane is a normalized field angle r.

  The plane passing through the intersection of the optical axes of the pair of cameras and having the same field angle range of both cameras is the convergence plane, and the distance from the optical center of the camera to the convergence plane (from the midpoint of the base line connecting the two optical centers) Let g be the distance of a straight line extending perpendicularly to the base line. The convergence angle β and the distance g are changed by the pan operation of the two cameras, and the view angle α and the normalized view angle r are changed by the zoom operation. The distance g is determined by the base line length b and the convergence angle β. The base line length b, the convergence angle β, and the angle of view α are parameters representing imaging conditions.

  FIG. 3B shows a 3D broadcast model in which 3D video is broadcasted or screened and viewed with both human eyes. The distance between human eyes (the distance between the optical centers) is represented by the baseline length m, and the screen width is represented by w. The distance from the optical center of both human eyes to the screen (the distance of a straight line extending perpendicularly to the base line from the midpoint of the base line connecting the two optical centers) is represented by d. The base line length m of human eyes, the screen width w, and the distance d between the base lines of human eyes and the screen are parameters representing viewing conditions.

Here, the normalized angle of view r is approximated by the following equation.
r = 2 tan (α / 2) (Formula 1)
The distance g is approximated by the following expression.
g = (b / 2) cot (β) (Formula 2)

  Next, a subject model in 3D video shooting and broadcasting will be described with reference to FIG. FIG. 4A shows a subject model corresponding to the 3D shooting model in FIG. 3A. Here, the midpoint of the straight line (base line) connecting the optical centers of the two cameras at the time of 3D shooting is the origin, the axis from the origin to the optical center of the right camera 1-R is the x axis, and from the origin A two-dimensional plane having the y-axis as the axis toward the optical axis intersection of the two cameras is defined as an imaging plane.

  In the example of FIG. 4A, the subject 20a is captured by two cameras at a distance y from the base line. When two cameras (the left camera 1-L and the right camera 1-R) photograph the subject 20a, a left eye image (an image photographed by the left camera 1-L) and a right eye image of the subject are taken from the photographed image. A difference (parallax) in the horizontal axis coordinate of (an image photographed by the right camera 1-R) is calculated, and parallax information p, which is a ratio between the parallax and the screen width, is obtained.

The coordinates of the subject 20a on the photographing plane are defined as (x, y), and y is defined as the actual distance of the subject. The relationship between the actual distance y of the subject 20a, the base line length b of the two cameras, the distance g between the camera and the convergence plane, the normalized angle of view r, and the parallax information p of the subject can be expressed by the following equations.
(G−y) / y = grp / b (Formula 3)
From this, the actual distance is obtained as follows.
y = bg / (b + grp) (Formula 4)

  On the other hand, FIG. 4B shows a subject model corresponding to the 3D broadcast model in FIG. 3B. Here, the origin of the midpoint of the straight line (base line) connecting the optical centers of the human eyes when viewing 3D video broadcasts or screenings is the origin, and the axis from the origin to the optical center of the right eye is the a-axis. A sensing plane is defined as a two-dimensional plane whose z axis is an axis extending in a direction perpendicular to the base line connecting the optical centers of both human eyes. In the 3D shooting model and the 3D broadcast model, spatial conversion from the shooting plane to the sensing plane occurs.

In the example of FIG. 4B, the subject 20b corresponding to the subject 20a shown in FIG. 4A is detected at a distance z from the base line. The coordinates on the sensing plane of the subject 20b are (a, z), and z is defined as the sensing distance of the subject. The relationship between the sensing distance z of the subject 20b, the baseline length m of both human eyes, the distance d between the baseline of the human eyes and the screen, the screen width w, and the parallax information p of the subject can be expressed by the following equations.
(D−z) / z = wp / m (Formula 5)
From this, the sensing distance is obtained as follows.
z = md / (m + wp) (Formula 6)

  When a scene is broadcasted or screened after being shot, the extent to which the subject can be seen closer to the actual size (that is, the ratio between the actual distance of the subject and the perceived distance of the subject) is defined as the pulling rate h, and the magnification of the depth of the subject Is defined as a depth enlargement magnification t. Further, the magnification of the width of the subject is defined as the width magnification s, and the ratio between the depth magnification and the width magnification is defined as the depth width ratio q.

  The miniature garden effect (giant's vision) occurs when the horizontal magnification is small. On the other hand, the book split effect occurs when the depth width ratio is small.

  Next, an integrated model obtained by integrating the 3D shooting model, the 3D broadcast model, and the subject model described above will be described. First, regarding the 3D shooting model and the 3D broadcasting model, a spatial conversion function for converting one point (x, y) on the shooting plane to one point (a, z) on the sensing plane is obtained.

When A = m− (bw / gr), B = bw / r, C = lw / r, and D = dl, the coordinates a and the coordinates z are obtained as follows.
a = Cx / (Ay + B) (Expression 7)
z = Dy / (Ay + B) (Formula 8)

Next, using the space conversion function, the pull-in rate h, the depth enlargement magnification t, the width enlargement magnification s, and the depth width ratio q regarding the subject in the scene are obtained.
Pull-in rate h = y / z = (Ay + B) / D (formula 9)
Depth magnification t = ∂z / ∂y = BD / (Ay + B) ² (Equation 10)
Width magnification factor s = ∂a / ∂x = C / (Ay + B) (Formula 11)
Depth width ratio q = t / s = BD / C (Ay + B) (Formula 12)

Finally, the following two equations that can be applied to any subject in the scene with any 3D shooting model and 3D broadcast model are obtained.
hq = B / C = b / m (Formula 13)
hs = C / D = w / dr (Expression 14)

Here, b / m in Expression 13 is a ratio between the base line length b of the two cameras and the base line length m of both human eyes, and this is defined as the base line length magnification. Then, Expression 13 can be expressed by the following Expression 15 (Integrated Model Expression 1).
Subject pull-in rate x Subject depth width ratio = Baseline length enlargement magnification (Equation 15)

Further, d / w in Expression 14 is a ratio of the distance from both human eyes to the screen and the width of the screen, and this is defined as a viewing distance magnification. Then, the equation 14 can be expressed by the following equation 16 (integrated model equation 2).
Subject pull-in rate × Subject width magnification × Viewing distance magnification × Normalized angle of view = 1
... (Formula 16)

  Next, each function of the control device 6 will be described with reference to FIG. As shown in FIG. 5, the control device 6 according to an embodiment of the present invention includes a video data receiving unit 6a, a shooting condition receiving unit 6b, a parallax calculating unit 6c, a shooting control processing unit 6d, a camera control unit 6e, and viewing conditions. An equal reception unit 6f, a set value reception unit 6g, an encoder unit 6h, and a convergence angle calculation unit 6i are configured. Furthermore, the imaging control processing unit 6d includes a baseline length determination unit 6d-1, a convergence angle control unit 6d-2, and an imaging condition adjustment unit 6d-3.

  The video data receiving unit 6a is connected to two cameras 1 (the left camera 1-L and the right camera 1-R) with a cable or the like, and these cameras 1 are used to perform 3D shooting by the convergence shooting method (on the left and right sides). Receive synchronized video data. The video data receiving unit 6a also receives data (for example, coordinate data) indicating a subject specified by the user in the camera 1 or generated by automatic designation by subject tracking or the like as needed. To do.

  The imaging condition receiving unit 6b is connected to the two cameras 1 with cables or the like, and receives parameters representing the imaging conditions from these cameras 1. Further, the movement amount measuring device 3 (view angle measuring device 3a, actual distance measuring device 3b) is connected by a cable or the like, and parameters representing imaging conditions are also received from these devices. Further, the base line length b measured by the base line length measuring device 3c and the convergence angle β stored by the convergence angle calculation unit 6i are received as necessary.

  As parameters, for example, the focal length and the image sensor width are received from the camera 1 as necessary. The focal length and the imaging element width are used when calculating the angle of view α. Further, the measured angle of view α is received from the angle of view measuring device 3a, and the actual distance y or the actual distance to the subject (from the base line midpoint of the camera 1) is measured from the actual distance measuring device 3b. Receive distance data used to determine y. The angle-of-view measuring device 3a and the actual distance measuring device 3b are configured as separate devices outside the camera 1 in FIG. For example, a laser distance meter such as an active laser sensor can be used as the actual distance measuring device 3b, and in this case, it is particularly suitable for measuring a subject at a short distance. However, at least some of the functional units of the angle-of-view measurement device 3a and the actual distance measurement device 3b may be configured to be incorporated in the camera 1.

  The parallax calculation unit 6c extracts the feature points of the subject from the left and right video data of the camera 1 received by the video data reception unit 6a, and matches these feature points to obtain the difference (parallax) in the horizontal axis. calculate. The subject is a specific object in the captured video, and is selected by a photographer or 3D grapher at the time of shooting (for example, by inputting using the input means of the camera) or automatically. It can be specified in various ways. The coordinate data relating to the subject thus designated is transmitted from the camera 1 to the parallax calculation unit 6c via the video data reception unit 6a in the same manner as the video data.

  Feature point extraction methods include feature points within a radius where the subject is selected, feature points within a specific segmented area, and all feature points (all parallax averages and weighted average values are parallaxes) Various methods can be employed. The feature point matching method can be performed using various known methods such as the SIFT method.

  The parallax obtained here is obtained by calculating the ratio (pixel base or physical length (meter) base) with the screen width w in the photographing control processing unit 6d described later, and the obtained value is the parallax information p. Used for.

  The imaging control processing unit 6d determines the baseline length of the camera 1 suitable for 3D imaging according to the conditions derived from the above-described integrated model and the like, and controls the convergence angle. It also determines the safe baseline length and range of convergence angles. A detailed method will be described later.

  The camera control unit 6e receives the baseline length determined by the baseline length determination unit 6d-1 of the imaging control processing unit 6d and receives measurement data from the movement amount measurement device 3 (baseline length measurement device 3c). The horizontal position (interval) of the camera 1 is adjusted without changing the angle of the camera 1 so that the camera 1 has the baseline length. For this purpose, the camera control unit 6 e transmits control data for operating the actuator 2 to the actuator 2. For example, in order to adjust the base line length b of the camera 1, the operation of the actuator 2-B is controlled (see FIG. 2).

  In addition, the camera control unit 6e feedback-controls the convergence angle β of the camera 1 by the convergence angle control unit 6d-2 of the imaging control processing unit 6d. The convergence angle control unit 6d-2 adjusts the angle of the camera 1 according to the parallax received from the parallax calculation unit 6c. For this purpose, the camera control unit 6 e transmits control data for operating the actuator 2 to the actuator 2. In order to adjust the convergence angle β of the camera 1, the operations of the actuator 2-LP and the actuator 2-RP are controlled (see FIG. 2).

  For the control of the actuator (motor), for example, the digital signal output from the camera control unit 6e is D / A converted, the converted analog signal is input to the motor driver, and the motor is designated by the driver. Driven to move only. The movement amount measuring device 3 is, for example, a movement amount detection sensor installed in a motor. A rotation angle signal of the motor detected thereby is A / D converted, and the converted digital signal is sent to the camera control unit 6e. Sent.

  The change in the baseline length b of the camera 1 that occurs as a result of the control by the actuator 2-B is measured by the baseline length measuring device 3c, and the measurement result is transmitted to the camera control unit 6e. The camera control unit 6e performs feedback control such as PID control, feedforward control, and the like on the operation of the actuator 2-B using the measurement result.

  In FIG. 5, for the convenience of explanation, the baseline length measuring device 3 c and the rotation angle measuring device 3 d are configured as separate devices outside the camera 1 and the actuator 2, but at least some of these functional units are included. The camera 1 or the actuator 2 may be incorporated (as described above, it may be a movement amount detection sensor installed in the motor). The actuator 2 is premised on feedback control or feedforward control here, but such control is not necessarily required.

  The viewing condition receiving unit 6f receives data related to viewing conditions for the viewer to view the 3D video, such as the baseline length m of the human eyes, the screen width w, and the distance d between the baseline of the human eyes and the screen. In addition, data used for obtaining the base line length b of the camera 1 such as the width of the image sensor of the camera 1 or for controlling the convergence angle β is received. These data may be input from a user or the like via input means such as a keyboard, or may be automatically transmitted by any means. In this embodiment, the base line length m of both human eyes is configured to be received by the receiving unit 6f such as viewing conditions, but a preset constant (for example, 60 mm) can be registered. In this case, the constant can be used as the baseline length m of both human eyes.

  The set value receiving unit 6g receives designated data designated by a user or the like, which is a precondition for obtaining the base line length b of the camera 1 or controlling the convergence angle β. For example, a parallax receiving unit that receives a specified parallax and a pull-in rate receiving unit that receives a specified pull-in rate are included. Further, a depth width ratio receiving unit that receives the specified depth width ratio and a width expansion magnification unit that receives the specified width expansion ratio are included. Also included is an allowable parallax range receiving means for receiving the allowable parallax range, which is used for adjusting the safe parallax. These designation data are usually entered and registered by a user or the like via an input means such as a keyboard.

  The encoder 6h performs necessary processing on the video received by the video data receiver 6a, encodes it into a predetermined data format, and outputs this as 3D video data.

  The convergence angle calculation unit 6i receives the rotation angle θ from the rotation angle measurement device 3d, and calculates and stores the convergence angle β of the camera 1 based on this data. The value of the convergence angle β is transmitted to the camera control unit 6e and to the imaging condition reception unit 6b.

  Next, a control method in the imaging control processing unit 6d will be described.

[Control based on integrated model formula 1]
The imaging control processing unit 6d controls several patterns using the relationship expressed by the integrated model equation 1 described above. Here, four typical patterns will be described.

First Control As shown in the integrated model equation 1, the preferred baseline length enlargement ratio (b / m) is proportional to the subject pull-in rate h × the subject depth width ratio q. The pull-in rate h of the subject is calculated using the actual distance y and the sensing distance z of the subject. Here, the depth width ratio q of the subject of the integrated model equation 1 is set to a default value 1 so that the problem of the book split effect can be effectively eliminated.

In such a case, that is, when the depth width ratio q of the subject is designated as a constant, a suitable baseline length enlargement rate (b / m) is proportional to the subject pulling rate h. However, the depth width ratio q of the subject may be another predetermined function (here, the constant is a kind of function (constant function)), and in this case, a suitable baseline length enlargement ratio (b / m) And the subject pull-in rate h have a relationship associated with each other by this predetermined function. Here, the predetermined function can be expressed as a function of the actual distance y, for example, 1 when the distance between the subject and the camera 1 is short and 0.5 when the distance is long. The depth width ratio q of the subject can also be expressed by the following formula (predetermined function) (where k is a coefficient).
q = k ^2/3 (Formula 17)
In the integrated model equation 1, the depth width ratio q of the subject is a function of the actual distance y of the subject, the base length b of the camera, the distance g between the camera and the convergence plane, and the normalized angle of view r.

  Then, the baseline length of the camera 1 can be obtained by the integrated model equation 1. The base line length determination unit 6d-1 determines the base line length of the camera 1 in this way, transmits the determined base line length b to the camera control unit 6e, and is controlled by the camera control unit 6e, so that the base line length of the camera 1 is determined. Is adjusted to the determined baseline length b.

  In order to obtain the baseline length b of the camera 1, it is only necessary to know the actual distance y of the subject, the sensing distance z of the subject, and the baseline length m of both human eyes. As the actual distance y of the subject, the measurement result of the actual distance measuring device 3b can be used. The actual distance y of the subject can be calculated from the parallax information p of the subject, the base line length b of the camera 1, the distance g between the base line of the camera 1 and the convergence plane, and the normalized angle of view r. By obtaining a value other than the length b, the base line length b of the camera 1 can be finally obtained.

  The parallax information p of the subject is obtained from the parallax transmitted from the parallax calculation unit 6c and the screen width w. The distance g between the base line of the camera 1 and the convergence plane can be obtained from the base line length b of the camera 1 and the convergence angle β as described above. The convergence angle β can be obtained based on the rotation angle measured by the rotation angle measurement device 3d. For example, the convergence angle β stored by the convergence angle calculation unit 6i is imaged via the imaging condition reception unit 6b. It is transmitted to the control processing unit 6d. The normalized angle of view r can be obtained from the angle of view α of the camera 1 as described above. The angle of view α is received by the angle of view measured by the angle of view measuring device 3a or the photographing condition receiving unit 6b. Obtained from the focal length of the camera 1 and the image sensor width of the camera 1. As the image sensor width of the camera 1, a value input by the user or the like from the viewing condition receiving unit 6f can be used.

  The sensing distance z of the subject is calculated using the parallax information p of the subject and viewing conditions (that is, the baseline length m of the human eyes, the screen width w, and the distance d between the baseline of the human eyes and the screen).

Second control In the second control, a parallax value is designated by the user or the like. When the setting value receiving unit 6g provides the parallax value designated by the user or the like to the shooting control processing unit 6d, the convergence angle control unit 6d-2 uses the parallax calculation unit based on the video data shot by the camera 1. While receiving the parallax calculated in 6c, the camera control unit 6e is controlled so that the designated parallax is maintained, and the convergence angle β is adjusted (feedback control (PID control)).

  At this time, in the integrated model equation 1, by specifying the parallax, the parallax information p is obtained (w is known), and then the parallax information p and the viewing conditions (base line length m of human binoculars, screen width w, The sensing distance z of the subject can also be obtained from the distance d) between the base line of both human eyes and the screen. Then, the actual distance y of the subject changed by the adjusted convergence angle β is grasped. The baseline length determination unit 6d-1 of the imaging control processing unit 6d obtains the changed pulling rate h from the actual distance y of the subject and the sensing distance z of the subject, determines the baseline length b based on this, and the camera control unit The base line length of the camera 1 is controlled to the determined base line length b by 6e.

  As the actual distance y of the subject, the measurement result of the actual distance measuring device 3b can be used, but as described in the first control, the baseline length b is finally determined without using the measurement result. You can also

  In the second control, after the parallax value is specified by the user or the like, the process of adjusting the convergence angle β and the process of controlling the baseline length b can be executed asynchronously.

Third Control In the third control, the value of the subject pull-in rate h is designated by the user or the like. The set value receiving unit 6g provides the value of the pull-in rate designated by the user or the like to the imaging control processing unit 6d. The baseline length determination unit 6d-1 of the imaging control processing unit 6d determines the baseline length b based on the specified pull-in rate h, and the camera control unit 6e controls the baseline length of the camera 1 to the determined baseline length b. Is done.

  Further, the photographing control processing unit 6d obtains the sensing distance z of the subject from the designated pull-in rate h and the actual distance y of the subject, and further from the sensing distance z of the subject and the viewing conditions (m, w, and d are known). The parallax information p and the parallax are obtained. Then, the convergence angle control unit 6d-2 receives the parallax calculated by the parallax calculation unit 6c based on the video data captured by the camera 1, and maintains the calculated parallax so as to maintain the parallax. The convergence angle β is feedback controlled (PID control) by the above control.

  As the actual distance y of the subject, the measurement result of the actual distance measuring device 3b can be used. However, as described in the first control, the actual distance y can be calculated without using the measurement result. it can.

  In the third control, the process of adjusting the convergence angle β and the process of controlling the baseline length b are asynchronous after the subject pull-in rate h is specified by the user or the like, as in the second control. Can be executed in

-Fourth control In the fourth control, the value of the depth width ratio q that has been treated as the default value 1 or other predetermined function so far is set to a value designated by the user or the like. The setting value receiving unit 6g provides the photographing control processing unit 6d with the value of the depth width ratio designated by the user or the like. The imaging control processing unit 6d determines the baseline length using the value specified here (instead of the previous value) for the depth width ratio q in the first to third control processes described above. Etc.

[Control based on integrated model formula 1 and integrated model formula 2]
The imaging control processing unit 6d controls several patterns using the relationship expressed by the integrated model equation 1 and the integrated model equation 2 described above. Here, two typical patterns will be described.

-Fifth Control As shown in the integrated model equation 2, the multiplication of the subject pull-in rate h, the subject horizontal magnification s, the viewing distance magnification (d / w), and the normalized angle of view r is a constant (= 1). Here, in order to effectively control the problem of the miniature garden effect, the zoom of the camera 1 is adjusted to change the angle of view α, and the horizontal magnification factor s of the subject is held at 1. Since the viewing distance magnification (d / w) is input in advance as viewing conditions by the user or the like, the multiplication of the subject drawing rate h and the normalized angle of view r is also a constant.

  The horizontal width magnification factor s of the subject is not a constant, but can be a predetermined function, like the depth width ratio q of the subject described above. According to the integrated model equation 2, the horizontal width magnification factor s of the subject is a function of the actual distance y of the subject, the base length b of the camera, the distance g of the camera and the convergence plane, and the normalized angle of view r.

  At this time, the imaging control processing unit 6d obtains the normalized view angle r based on the view angle α measured by the view angle measurement device 3a or obtained from the focal length based on the integrated model equation 2, and is thus preferable. The pull-in rate h is calculated.

  Next, the baseline length determination unit 6d-1 determines the baseline length b from the calculated pull-in rate h, and the camera control unit 6e controls the baseline length of the camera 1 to the determined baseline length b.

  Further, the imaging control processing unit 6d obtains the subject sensing distance z from the calculated pull-in rate h and the subject actual distance y based on the integrated model equation 1, and obtains the subject sensing distance z and the viewing conditions (m, w). , D are known), and further, parallax information p and parallax are obtained. Here, the convergence angle control unit 6d-2 receives the parallax calculated by the parallax calculation unit 6c based on the video data captured by the camera 1, and maintains the obtained parallax. The convergence angle β is feedback controlled (PID control) by the control of 6e.

  As the actual distance y of the subject, the measurement result of the actual distance measuring device 3b can be used. However, as described in the first control, the actual distance y can be calculated without using the measurement result. it can.

  In the fifth control, after the pull-in rate h is calculated, the process for adjusting the convergence angle β and the process for controlling the baseline length b can be executed asynchronously.

-Sixth control In the sixth control, the value of the horizontal magnification factor s held as 1 in the fifth control is set to a value designated by the user or the like. The set value receiving unit 6g provides the photographing control processing unit 6d with the value of the horizontal width magnification specified by the user or the like. In the fifth control process described above, the imaging control processing unit 6d controls the convergence angle and the like using the value specified here for the horizontal magnification factor s.

[Control based on safety]
The imaging control processing unit 6d can perform the seventh control regarding the safety of the 3D video. The imaging control processing unit 6d uses a baseline length b and a convergence angle β so that a safe parallax range is obtained based on a predetermined safety guideline, for example, a 3D safety guideline (3DC safety guideline (3D consortium, safety guideline group)). To control.

  The shooting condition adjustment unit 6d-3 of the shooting control processing unit 6d specifies the allowable parallax range (p_lower, p_upper) specified in advance by the user or the like and received from the set value receiving unit 6g, and the entire image calculated by the parallax calculation unit 6c. Using the current value of the parallax range (p_min (t), p_max (t)) and the current baseline length b (t) of the camera 1 received from the baseline length measuring device 3c, the tolerance of the baseline length at the next time A range (particularly pointing to the maximum value b_upper (t + 1)) is determined using Equation 18 below. Here, the parallax of the entire image refers to the parallax of all or most of the pixels (pixels) instead of some feature points of the image taken by the camera 1. By obtaining this parallax, The parallax range (p_min (t), p_max (t)) is obtained.

b_upper (t + 1) = b (t) · (p_upper−p_lower) / (p_max (t) −p_min (t))
... (Formula 18)

  As shown in Equation 18, the upper limit b_upper (t + 1) of the baseline length b at the next time (t + 1) is the maximum and minimum values of the baseline length b (t) at the current time (t) and the parallax range of the entire image. And the ratio of the difference between the upper limit and the lower limit of the allowable parallax range.

  Further, the imaging condition adjustment unit 6d-3 specifies the allowable parallax ranges (p_lower, p_upper) designated in advance by the user or the like and received from the set value receiving unit 6g, and the parallax range of the entire image calculated by the parallax calculation unit 6c ( Using the current value of p_min (t), p_max (t)) and the current convergence angle β (t) of the camera 1 stored by the convergence angle calculation unit 6i, the allowable range of the convergence angle at the next time is determined. The following equations 19 and 20 are used for determination.

β_upper (t + 1) = β (t) + δ (p_max (t) <p_upper)
= Β (t) −δ (p_max (t)> p_upper) (Equation 19)
β_lower (t + 1) = β (t) + δ (p_min (t)> p_lower)
= Β (t) −δ (p_min (t) <p_lower) (Equation 20)
Here, the magnitude of δ can be appropriately adjusted according to various factors including the difference between the allowable parallax range and the parallax of the entire image and the time (t). Can be used.

  As can be seen from Equation 19, the upper limit β_upper (t + 1) of the convergence angle at the next time (t + 1) is obtained based on the convergence angle β (t) at the current time (t), and the maximum parallax value ( When p_max (t)) exceeds the upper limit (p_upper) of the allowable parallax range, a value obtained by subtracting δ from β (t) is set to β_upper (t + 1). Conversely, the maximum parallax value (p_max) of the entire image If (t)) is smaller than the upper limit (p_upper) of the allowable parallax range, a value obtained by adding δ to β (t) is set to β_upper (t + 1). Similarly, from Expression 19, when the convergence angle lower limit β_lower (t + 1) at the next time (t + 1) is smaller than the lower limit (p_lower) of the allowable parallax range, the minimum parallax value (p_min (t)) of the entire image , Β (t) minus δ is β_lower (t + 1), and conversely, when the minimum parallax value (p_min (t)) of the entire image exceeds the lower limit (p_lower) of the allowable parallax range, A value obtained by adding δ to β (t) is defined as β_lower (t + 1).

  Also, here, the maximum parallax value (p_max (t)) of the entire image exceeds the upper limit (p_upper) of the allowable parallax range, and the minimum parallax value (p_min (t)) of the entire image is within the allowable parallax range. When smaller than the lower limit (p_lower), b_upper (t + 1) becomes a value smaller than b (t) and the base line length b is adjusted to be smaller based on Expression 18, so that when the time elapses, the entire image The parallax becomes within an allowable range.

  The imaging condition adjustment unit 6d-3 checks the determination content and control content of the baseline length determination unit 6d-1 or the convergence angle control unit 6d-2, and the baseline length b is the upper limit b_upper (t + 1) of the above-described baseline length b. ) Or when the convergence angle β exceeds the range of the convergence angle (β_lower (t + 1) to β_upper (t + 1)), the base line length b and the convergence angle β are adjusted to be within the range. Further, in the 3D video shooting control system 7 according to the embodiment of the present invention, such a function of adjusting the baseline length b and the convergence angle β by the shooting condition adjusting unit 6d-3 is not essential, and is appropriately selected as necessary. Can be incorporated.

  The 3D video shooting control system 7 according to the embodiment of the present invention described above with reference to FIGS. 1 to 5 includes the camera 1 having a configuration including a pair of cameras. A plurality of camera sets (camera pairs) can be used so that 3D imaging of a plurality of patterns can be performed at a time in accordance with viewing conditions, and the control device 6 provides a preferable baseline for each camera set. The length and angle of convergence can be determined and the position and orientation of the corresponding camera set can be controlled.

  For example, if the 3D video shooting control system 7 includes four cameras (camera A, camera B, camera C, camera D), a plurality of camera sets (camera A + camera B) (camera C + camera D), , (Camera A + Camera B) (Camera A + Camera C) (Camera A + Camera D), etc. can be configured.

  Next, with reference to FIG. 6, a configuration example of a computer constituting the control device 6 of the 3D video shooting control system 7 according to an embodiment of the present invention will be described. However, the computer 100 in FIG. 6 only exemplifies a typical configuration of a computer that implements each function of the control device 6.

The computer 100 includes a CPU (Central
Processing Unit) 101, memory 102, imaging device interface 103, actuator interface 104, movement amount measuring device interface 105, display controller 106, display 107, input device interface 108, keyboard 109, mouse 110, external recording medium interface 111, external storage It includes a device 112, a network interface 113, and a bus 114 that connects these components to each other.

  The CPU 101 controls the operation of each component of the computer 100 and executes each function under the control of the OS. For example, control is performed on processing such as the parallax calculation unit 6c and the imaging control processing unit 6d illustrated in FIG.

The memory 102 is usually composed of a RAM (Random Access Memory). The memory 102 is loaded with a program for realizing each function executed by the CPU 101 (functions such as the parallax calculation unit 6c and the imaging control processing unit 6d), and data necessary for the program (for example, current congestion) The angle β and the angle of view α) are temporarily stored.

  The imaging device interface 103 is connected to the camera 1 (each of the left image capturing camera 1-L and the right image capturing camera 1-R), and controls data transmission / reception with the camera 1 to transmit video data and the like from the camera 1. Receive.

  The actuator interface 104 is connected to the actuator 2 and transmits control data for controlling the operation of the actuator to the actuator 2. Further, the movement amount measuring device interface 105 receives measurement data such as an angle and a distance from the movement amount measuring device 3.

The display controller 106 processes drawing data transmitted from the CPU 101 or the like and displays an LCD (Liquid Crystal).
A screen is displayed on a display 107 including a display device including a display) and a touch screen. For example, when a set value is input from the user or the like to the control device 6, a set value input screen is displayed on the display 107.

  The input device interface 108 receives signals from input devices such as a keyboard 109, a mouse 110, and a touch screen that are operated by the user to input to the control device 6, and transmits the signals to the CPU 101. Input data and the like are stored in the memory 102 and the like.

  The external recording medium interface 111 accesses the external recording medium 120 to control data transmission / reception. For example, the optical disk 121 is driven to access the recording surface, and the recorded data is read, or the data stored in the external storage device 112 is written to the optical disk 121. In addition, the portable flash memory 122 is accessed and data is exchanged with the computer 100.

  The external storage device 112 is generally a storage device such as a hard disk. The external storage device 112 stores a program for realizing each function executed by the CPU 101 and stores data used by the program.

  The network interface 113 realizes connection with a network 130 including a LAN and the Internet, and controls data transmission / reception between the computer 100 and the network 130. The network interface 113 allows the computer 100 to transmit and receive data by connecting to a LAN or the Internet. A program executed by the CPU 101 and realizing each function of the present invention can be provided to the computer 100 from the outside via the network interface 113 and the external recording medium interface 111 described above.

  In addition, all or part of a program for realizing each function of the present invention may be made into a chip and distributed in the market.

DESCRIPTION OF SYMBOLS 1 ... Camera, 2 ... Actuator, 3 ... Movement amount measuring device, 4 ... Camera control pan head, 5 ... 3D imaging | photography mechanism, 6 ... Control apparatus, 7 ... 3D Image capturing control system, 8 ... transmitting device, 9 ... receiving device, 10 ... 3D image quality adjusting device, 11 ... 3D projector, 12 ... screen, 13 ... 3D television

Claims (19)

A 3D video imaging control system for performing 3D video imaging control, wherein the 3D video imaging control system includes:
Parallax calculating means for calculating parallax related to the subject from video obtained from a pair of imaging devices that photograph the subject;
Baseline length determining means for determining a baseline length of the pair of imaging devices;
Baseline length control means for controlling to move the pair of imaging devices so as to be the determined baseline length;
The baseline length determination means includes a ratio of a baseline length of the pair of imaging devices to a baseline length of both human eyes, a distance (actual distance) from the baseline of the pair of imaging devices to a subject at the time of shooting, and the 3D video. The base line length of the pair of imaging devices is calculated based on the fact that the ratio (the pull-in rate) to the distance (sensed distance) from the base line of both human eyes to the subject at the time of viewing is related by a predetermined function. A 3D video shooting control system, wherein the 3D video shooting control system determines and determines the sensing distance based on the parallax. The 3D video shooting control system according to claim 1,
Parallax receiving means for receiving the specified parallax;
A convergence angle control means for controlling to move the pair of imaging devices in order to adjust the convergence angle;
The convergence angle control means moves the pair of imaging devices so as to have a convergence angle corresponding to the designated parallax,
The 3D video shooting control system characterized in that the base line length determining means determines the sensing distance based on the designated parallax. The 3D video shooting control system according to claim 1,
A pull-in rate receiving means for receiving a specified pull-in rate;
A convergence angle control means for controlling to move the pair of imaging devices in order to adjust the convergence angle;
The convergence angle control means moves the pair of imaging devices so as to have a convergence angle according to the parallax obtained based on the designated pull-in rate,
The 3D video imaging control system characterized in that the baseline length determining means determines a baseline length of the pair of imaging devices based on the designated pull-in rate. The 3D video shooting control system according to claim 1,
Convergence angle control means for controlling to move the pair of imaging devices in order to adjust the convergence angle,
Based on the relationship that the multiplication of the drawing rate and the angle of view of the pair of imaging devices becomes a constant, the drawing rate is obtained from the angle of view,
The convergence angle control means moves the pair of imaging devices so as to have a convergence angle according to the parallax obtained based on the obtained pull-in rate,
The 3D video imaging control system characterized in that the baseline length determining means determines a baseline length of the pair of imaging devices based on the obtained pull-in rate. The 3D video shooting control system according to claim 4,
The 3D video shooting control system, wherein the angle of view is measured by an angle-of-view measuring unit that measures the angle of view, or is calculated by a focal length and an imaging element width in the pair of imaging devices. In the 3D video imaging control system according to claim 4 or 5,
It further has a width expansion ratio receiving means for receiving the specified width expansion ratio,
The pull-in rate is obtained from the angle of view and the width enlargement magnification based on a relationship in which the multiplication of the drawing rate, the angle of view of the pair of imaging devices, and the designated width enlargement magnification is a constant. 3D video shooting control system. The 3D video shooting control system according to any one of claims 1 to 6,
A depth width ratio receiving means for receiving the specified depth width ratio;
The baseline length determining means is configured to determine whether the ratio between the baseline length of the pair of imaging devices and the baseline length of both human eyes is proportional to the multiplication of the pull-in rate and the depth width ratio. A 3D video shooting control system for determining a base line length of an apparatus. The 3D video shooting control system according to any one of claims 2 to 7,
An allowable parallax range receiving means for receiving the specified allowable parallax range;
Photographing condition adjusting means for adjusting various parameters so that the parallax falls within the allowable parallax range;
The parallax calculating means calculates a parallax range of the entire image in the video obtained from the pair of imaging devices;
The imaging condition adjusting unit determines at least one of an allowable range for the baseline length of the pair of imaging devices and an allowable range for the convergence angle from the calculated parallax range of the entire image and the allowable parallax range. A 3D video shooting control system, wherein the base line length or the convergence angle of the pair of imaging devices is adjusted to the determined allowable range. The 3D video shooting control system according to any one of claims 1 to 8,
The baseline length determination means determines the sensing distance based on the parallax and viewing conditions,
The viewing condition includes a base line length of human binocular eyes, a screen width, and a distance between the base line of human binocular eyes and the screen. The 3D video shooting control system according to any one of claims 1 to 9,
The actual distance is measured by an actual distance measuring unit that measures the actual distance, or is obtained from the parallax, the base line length of the pair of imaging devices, the convergence angle, and the angle of view. . The 3D video shooting control system according to any one of claims 1 to 10,
A 3D video shooting control system that performs shooting control of the 3D video for each of two or more pairs of imaging devices configured of three or more imaging devices for shooting the subject. A 3D video shooting control method for performing shooting control of 3D video, wherein the 3D video shooting control method includes:
A parallax calculation step of calculating parallax related to the subject from video obtained from a pair of imaging devices that shoot the subject;
A baseline length determining step for determining a baseline length of the pair of imaging devices;
A baseline length control step for controlling the pair of imaging devices to move to the determined baseline length, and
In the baseline length determination step, the ratio between the baseline length of the pair of imaging devices and the baseline length of both human eyes is the distance (actual distance) from the baseline of the pair of imaging devices to the subject at the time of shooting and the 3D video The base line length of the pair of imaging devices is calculated based on the fact that the ratio (the pull-in rate) to the distance (sensed distance) from the base line of both human eyes to the subject at the time of viewing is related by a predetermined function. A 3D video shooting control method, comprising: determining and determining the sensing distance based on the parallax. The 3D video shooting control method according to claim 12,
A parallax receiving step for receiving the specified parallax;
A convergence angle control step for controlling to move the pair of imaging devices to adjust a convergence angle,
The convergence angle control step moves the pair of imaging devices so as to have a convergence angle corresponding to the designated parallax,
In the 3D video shooting control method, the base line length determining step obtains the sensing distance based on the designated parallax. The 3D video shooting control method according to claim 12,
A withdrawal rate receiving step for receiving a specified withdrawal rate;
A convergence angle control step for controlling to move the pair of imaging devices to adjust a convergence angle,
The convergence angle control step moves the pair of imaging devices so as to have a convergence angle corresponding to the parallax obtained based on the designated pull-in rate,
The base line length determination step determines a base line length of the pair of imaging devices based on the designated pull-in rate. The 3D video shooting control method according to claim 12,
A convergence angle control method for controlling the movement of the pair of imaging devices to adjust the convergence angle;
Based on the relationship that the multiplication of the drawing rate and the angle of view of the pair of imaging devices becomes a constant, the drawing rate is obtained from the angle of view,
The convergence angle control step moves the pair of imaging devices so that the convergence angle corresponds to the parallax obtained based on the obtained pull-in rate,
The base line length determination step determines a base line length of the pair of imaging devices based on the obtained pull-in rate. On the computer,
Parallax calculating means for calculating a parallax related to the subject from video obtained from a pair of imaging devices that shoot the subject;
Baseline length determining means for determining a baseline length of the pair of imaging devices; and
A program for performing 3D video imaging control for performing 3D video imaging control, which functions as a baseline length control unit that controls to move the pair of imaging devices so as to have the determined baseline length,
The baseline length determination means is configured such that the ratio between the baseline length of the pair of imaging devices and the baseline length of both human eyes is the distance (actual distance) from the baseline of the pair of imaging devices to the subject at the time of shooting and the 3D video The base line length of the pair of imaging devices is calculated based on the fact that the ratio (the pull-in rate) to the distance (sensed distance) from the base line of both human eyes to the subject at the time of viewing is related by a predetermined function. A program for determining and obtaining the sensing distance based on the parallax. The program according to claim 16, wherein the computer includes:
Parallax receiving means for receiving the specified parallax, and
A program that functions as a convergence angle control unit that controls to move the pair of imaging devices in order to adjust a convergence angle;
The convergence angle control means moves the pair of imaging devices so as to have a convergence angle corresponding to the designated parallax,
The base line length determination means obtains the sensing distance based on the designated parallax. The program according to claim 16, wherein the computer includes:
A pull-in rate receiving means for receiving a specified pull-in rate; and
A program that functions as a convergence angle control unit that controls to move the pair of imaging devices in order to adjust a convergence angle;
The convergence angle control means moves the pair of imaging devices so as to have a convergence angle according to the parallax obtained based on the designated pull-in rate,
The base line length determining means determines a base line length of the pair of imaging devices based on the designated pull-in rate. The program according to claim 16, wherein the computer includes:
A program that functions as a convergence angle control unit that controls to move the pair of imaging devices in order to adjust a convergence angle;
Based on the relationship that the multiplication of the drawing rate and the angle of view of the pair of imaging devices becomes a constant, the drawing rate is obtained from the angle of view,
The convergence angle control means moves the pair of imaging devices so as to have a convergence angle according to the parallax obtained based on the obtained pull-in rate,
The base line length determining means determines a base line length of the pair of imaging devices based on the obtained pull-in rate.