JPH07328971A - Manipulator with tv camera - Google Patents

Abstract

PURPOSE:To provide a manipulator with a TV camera capable of automatically moving the manipulator to the proper position for the work and smoothly implementing the work when the work environment is captured by the TV camera. CONSTITUTION:This manipulator with a TV camera is provided with the TV camera 1 having an automatic focusing function, a detector automatically detecting the focal point position, and a detector detecting the zoom position; a tripod head 2 capable of changing the attitude of the TV camera 1; and a means 6 having a detector detecting the position of the tripod head 2 and calculating the proper attitude for the work of a manipulator 4 based on the camera signal 8 such as the automatic focal point position and zoom position and the tripod head position information 9 of the TV camera 1. An operator adjusts the TV camera 1 to the work environment then drives the manipulator 4 to the proper position.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a manipulator equipped with a TV camera.

[0002]

2. Description of the Related Art Manipulators are used in a dangerous and unsuitable environment for humans to work in an environment where humans cannot directly work, such as in nuclear power facilities, or in live power distribution equipment. This type of manipulator includes a master-slave manipulator that operates remotely and a manipulator that operates with a joystick.

When using such a manipulator, an operator takes an image of the work environment with a TV camera and displays it on a monitor, and operates the manipulator or joystick while performing this operation to perform a predetermined work. There is.

In the work using such a manipulator and a TV camera, regarding the cooperation between the manipulator and the TV camera, a method in which no cooperation is performed or the TV camera follows a predetermined portion of the manipulator is adopted. Also,
Even in a system that provides a stereoscopic screen using two TV cameras, the manipulator can be made to follow a predetermined position,
The vergence angle and elevation angle of the two cameras are controlled so as to have predetermined amounts in accordance with the movement of the manipulator.

[0005]

As described above, the conventional T
In the manipulator with a V camera, the TV camera platform and the TV camera move following the movement of the manipulator, but the operator had to move the manipulator after the work environment was captured by the camera.

When the camera is made to follow a predetermined part of the manipulator, the image information may sometimes fluctuate, which makes it difficult to see. Further, in order to obtain work environment information, the image of the TV camera does not necessarily have to follow a predetermined part of the manipulator. On the contrary, when working while grasping the relative position of the work environment and the manipulator,
It is easier to view the TV camera image when it is fixed.

The present invention has been made based on the above circumstances, and when a work environment is captured by a TV camera, the manipulator is automatically moved to a position suitable for the work, and the TV camera can perform the work smoothly. Provide an attached manipulator.

[0008]

A manipulator with a TV camera according to the present invention comprises a TV camera having an autofocus function, a detector for detecting an autofocus position, and a detector for detecting a zoom position, and a posture of the TV camera. It is equipped with a detector that detects the position of the pan head that can be changed, and from the camera signals such as the autofocus position and the zoom position, and the pan head position information of the TV camera, a proper posture for the manipulator to work is determined. It is characterized in that a means for calculating is provided, and after the operator adjusts the TV camera to the work environment, the manipulator is driven to an appropriate position.

Further, the manipulator with a TV camera of the present invention is a TV camera in which the TV camera is stereoscopically viewable by two TV cameras, and the distance between the two cameras, the convergence angle and the elevation angle can be changed respectively. Each of the manipulators is equipped with a position detector that detects the position, and is suitable for the manipulator to work from the position information such as the camera interval, the angle of convergence, and the elevation angle, the autofocus position, the zoom position, and the pan head position information of the TV camera. Using the means to calculate the posture, the operator
It is characterized in that the manipulator is driven to an appropriate position of the manipulator by using the function after adjusting the V camera.

Further, the manipulator with a TV camera of the present invention is provided with a detector for displaying a TV camera image on a monitor, designating a predetermined portion of the image on the monitor by an operator, and detecting the designated portion. The operator uses a means for calculating a proper posture for the manipulator to work from the position information and the position information such as the camera interval, the angle of convergence, the elevation angle, the automatic focus position, the zoom position, and the pan head information of the TV camera. Drives the manipulator to an appropriate position of the manipulator after instructing a designated location including the TV camera in the work environment.

Further, when it is appropriate to move the TV camera platform to see the designated portion of the monitor image for performing work, the TV camera platform is moved to the proper position. The manipulator is driven to the proper position of the manipulator by using the function of moving the TV camera to the TV camera position and calculating the proper posture for the manipulator to work, and after the operator adjusts the TV camera to the work environment. It is characterized by

Further, the manipulator with a TV camera of the present invention has a plurality of TV cameras arranged at different positions and each TV camera is provided with a platform having n degrees of freedom, and the autofocus position and zoom of the TV camera selected appropriately. The operator uses the means to calculate the proper posture for the manipulator to work from the position and the pan head position information of the TV camera so that the operator can use the TV in the work environment.
After manipulating the camera, the manipulator is driven to an appropriate position of the manipulator by using the function.

Furthermore, the manipulator with a TV camera of the present invention has a plurality of cameras arranged at different positions and each TV camera is provided with a platform having n degrees of freedom, and the autofocus position and zoom of the TV camera selected appropriately. Set the proper work environment from the position and the pan head position information of the TV camera, and drive the pan head, auto focus position and zoom position of other TV cameras to the predetermined positions to obtain the proper video information of the work environment. Characterize.

[0014]

In the manipulator with a TV camera of the present invention having the above-mentioned structure, the TV camera and the platform are provided with the above-mentioned functions, and the TV camera captures the work environment, whereby the manipulator is automatically moved to the work place. Therefore, the work efficiency can be improved.

[0015]

Embodiments will be described below with reference to the drawings. FIG. 1 is a system diagram showing the overall configuration of an embodiment of the present invention. In this figure, a TV camera 1 mounted on a platform 2 is mounted on a base 3, and the TV camera 1 can freely set a camera line of sight 7 within the range of the degrees of freedom by the motion of the platform 2 having n degrees of freedom. . A manipulator 4 having n degrees of freedom is mounted on the base 3. The camera signal 8, the platform signal 9, and the manipulator signal 10 are the control device 6
The control device 6 controls the TV camera 1, the platform 2, and the manipulator 4. The operator operates the TV camera 1 and the TV camera 1 which is not shown
When the manipulator and the platform 2 are properly aligned with the work object 5, the manipulator calculates by the control device 6 so as to have an appropriate posture for working, and drives the manipulator 4 to move it to an appropriate position.

FIG. 2 is a motion function diagram of the platform 2 and the manipulator 4 shown in FIG. The platform 2 has two degrees of freedom of a pan 11 and a tilt 12, and the manipulator 4 has a first shaft 13 of the shoulder, a second shaft 14, and a third shaft 13 for extending and contracting the elbow. Fourth axis 16 for panning in the vertical plane of the wrist, fifth axis 7 for tilting in the vertical plane of the wrist, and sixth axis for panning in the horizontal plane of the wrist. 18 and has 6 degrees of freedom. Hereinafter, a case where the tip end portion 19 of the manipulator 4 is moved to a predetermined position will be described. If the platform 2 and the manipulator 4 have a linked structure, respectively,
A rotation vector and a translation vector can be represented between the connected axes. The relationship from the first axis 13 of the manipulator to the tip portion 19 and the pan 11 of the platform to the camera viewpoint 20 can be expressed as a transformation matrix.

In FIG. 3, for simplicity, the fulcrum of the pan 11 of the pan head and the fulcrum of the first axis 13 of the manipulator 4 are the same, and the origin of the world coordinates 23 is also the same. The tip 19 of the manipulator 4 is a vector a25, the camera viewpoint 20 is a vector b26, and the camera viewpoint 20 to the target point 21 is a vector d28.
To the target point 21 are represented by a vector c27.
When the work target 5 is captured by the TV camera 1 by the autofocus function included in the TV camera 1, the distance, which is the scalar amount of the vector d28, can be calculated. Therefore, the in-focus image is used as the focal plane 22 and the work target 5 is used as the focal plane 2.
Assume 2. Furthermore, the focal plane 22 and the vector d28
Shall intersect vertically as indicated by the right angle symbol 24. That is, the vector d28 is defined by defining it as the minimum value from the camera viewpoint 20 to the focal plane 22.
Therefore, vector a25, vector b26, vector d
Since 28 is calculated as a known number, the vector c27 can be calculated. Target point 2 of manipulator tip 19
The movement to 1 may be performed by moving the manipulator 4 by the vector c27 and calculating the movement amount of each joint angle of the manipulator. The manipulator movement calculation may be performed by the method described in Japanese Patent Application No. 3-143374 "Manipulator". However, the manipulator moving method is not limited thereto.

The basic flow of calculation will be described with reference to FIG.
The lens information 29, the platform information 30, and the manipulator position data 31 are input to the arm position calculation unit 32 to calculate a manipulator drive command.

Lens information will be described with reference to FIG.
The distance 1 _f 39 from the autofocus function of the TV camera 1 to the work object 5 is required. 1 _f 39 is calculated in advance by a numerical value of a direct distance or a numerical value calculated jointly by a position detector. Also, the angle of view θ _f ( _f =
0, 1) 36 is decided. In the case of the field angle θ ₁ 36a and the angle of view θ
_The range within the image of the work target 5 is different from the case of 236b. If the data indicating the variable amount of the angle of view 36 is used as the zoom position data, the vertical distance 1 _V
i (i = 0, 1) is determined by the position data and 1 _f 39. In the figure, the vertical distance 1 _{V 2} 37a corresponds to the angle of view θ ₁ 36a, and the vertical distance 1 _{V 2} 37b corresponds to the angle of view θ ₂ 36b. Also vertical distance 1
_A horizontal distance of 1 _h 38 with respect to _V 37 is determined. Therefore, the video signal 35 sent from the TV camera 1 is displayed on the monitor 34 and the work environment can be determined. Since the work environment can be specified by the camera information in this way, the manipulator 4 may be moved to the work environment by using this, as described with reference to FIGS. The moving part of the manipulator comes to the entire surface of the work object 5,
You can switch between them by holding the center of the screen and working. This usage can be carried out by various means such as designation by the operator during work or predetermination of the moving point of the manipulator.

Calculation of the movement target point 40 of the manipulator 4 with respect to the work object 5 will be described with reference to FIG. Although the target point 21 has been described with reference to FIG. 3, the vector e41 up to the movement target point 40 of the manipulator may be determined or may be designated by the operator for each work. After that, as described with reference to FIG. 3, the movement target point 2
The calculation is the same as when 1 is replaced with the movement target point 40. Therefore, the manipulator 4 moves from the manipulator position 4a before the start of movement to the manipulator position 4b after the movement.
Therefore, the manipulator 4 does not collide with the work target 5 after the movement.

A stereoscopic camera will be described with reference to FIG.
In the parallax system using two TV cameras, the right T in FIG.
The distance between the V camera 1 _R and the left TV camera 1 _L is l _m 43
Are placed open. The point where the sight lines of both TV cameras intersect is set as the target point 21, and the angle of intersection is set as the convergence angle α42. A side view seen from the direction of the arrow in (a) is shown in (b). The elevation angle β44 is the angle of intersection of the two cameras in the vertical plane. Convergence angle α42, elevation angle β44 camera interval l according to the focus ratio and zoom ratio in the stereoscopic camera system
_{The m} 43 is appropriately adjusted to generate a stereoscopic image necessary for stereoscopic viewing.

A system for generating a stereoscopic image will be described with reference to FIG. Right video signal 35 _R of the right TV camera 1 _R
And inputs the left image signal 35 _L of the left TV camera 1 _L to the scan converter 45 and the liquid crystal shutter controller 46. In the scan converter 45, the left and right video signals are combined and transmitted to the monitor 34, and the control signal of the liquid crystal shutter controller 46 is sent to the liquid crystal shutter 47 to turn the shutter ON / OFF. The operator wears polarizing glasses 48 and looks at the monitor 34 to enable stereoscopic viewing.

FIG. 9 shows a basic control flow when a stereoscopic camera is used. Right camera lens information 29 _R , left camera lens information 29 _L , convergence angle 42, elevation angle 44,
The camera interval 43 is input to the arm control calculation unit 32, the amount of movement of the manipulator 4 is calculated, and the manipulator drive command 33 is generated. Basically, the movement amount of the manipulator 4 can be calculated as shown in FIG. From the right camera lens information 29 _R and the left camera lens 29 _L , the convergence angle α42, the elevation angle β44, and the camera interval l _m 43 are calculated. Next, convergence angle α42 and elevation angle β44 camera interval l _m 4
3 is used as a parameter to calculate the movement target point 40 of the manipulator 4, and the manipulator 4 can be determined by a predetermined method, a calculation formula set in advance and the calculation formula, or an operator's instruction. Should be driven.

A method of calculating the vector e41 in the stereoscopic view will be described with reference to FIG. The target point 21 is the same as the intersection of the right camera line of sight 7 _R and the left camera line of sight 7 _L with respect to the right TV camera 1 _R and the left TV camera 1 _L. When the manipulator 4 moves to the position of the movement target point 40, the appearance of the depth with respect to the work object varies depending on the values of the convergence angle α42, the elevation angle β44, and the camera interval l _m 43. . The right camera vertical target surface 49 and the left camera vertical target surface 50 depend on the convergence angle α of the camera. The left and right images are displaced as they go in the direction a51 outside the target point 21. Therefore, the manipulator 4 sets the movement target point 40 from the viewpoint that the manipulator 4 moves to a position where it can be seen in three dimensions rather than moving to a position where it is easy to work. In order to calculate the moving target point 40, the convergence angle α42, the elevation angle β44, and the camera interval l _m 4
It depends on whether the vector e41 is determined in advance using 3 and 3 as parameters, or a calculation formula is created and calculated based on the parameters, or designated by the operator.

FIG. 11 shows the monitor 34 with the touch sensor 52 arranged on the screen. Touch sensor 52 is m
_The touch point 54 is represented by a matrix of _f × n _f.
The operator can specify a specific point on the screen. Therefore, it suffices to calculate so as to move a predetermined portion of the manipulator 4 to this point.

A method of calculating the movement target value 40 when the touch point 54 is designated will be described with reference to FIG.
When the deviation angle γ55 is extremely small or neglected,
The vector e41a is sufficient. In the case of considering the shift angle γ55, the vector e41b may be set as the movement target point 40b. The vector e41b may be defined by defining the target point 21 as the shortest distance between the camera line of sight 7 and the focal plane 22. The rest is the same as the case described in FIG.

A method of controlling the manipulator 4 and the platform 2 will be described with reference to FIG. In addition to the description with reference to FIG. 9, the pan head position data 30 is input to the arm / pan head position calculation unit 56 to calculate a manipulator drive command 33 and a pan head drive command 57. Since the control method of the manipulator 4 has been described, it will be omitted. Since the platform 2 also has the same link structure as the manipulator, the same control as that of the manipulator 4 can be performed. The moving target point of the platform 2 may be set in the same manner as in the case of the manipulator 4.

Referring to FIG. 14, a method for obtaining the moving platform moving target point 60 for moving the platform 2 will be described. After calculating the amount of movement of the manipulator 4 with the camera on the focal plane 22a, the amount of movement is determined by a predetermined amount or an amount designated by the operator. Before the movement, the vector f5 from the platform tilt 12 to the focal plane 22a
8 is decided. Next, with respect to the focal plane 22b after the movement, the vector h59 of the movement target point 60 of the tripod head is determined from the tripod head tilt 12. Therefore, the movement amount of the platform can be calculated.

A case where two TV cameras are installed at different positions in FIG. 15 and two cameras are mounted on a platform having one degree of freedom will be described. The work object 5 is captured by the TV camera 1a mounted on the platform 2a. Since moving the manipulator 4 has been described, it will be omitted. Next, the TV camera 1b mounted on the platform 2b was initially in a position where it was difficult to see the work target 5. Therefore, if the platform 2b is driven and moved to the TV camera 1b ', the work target 5 can be easily viewed. Therefore, if the operator adjusts one TV camera to the work target, the other TV cameras also automatically image the work target, so that the number of times the TV camera is matched to the work target can be reduced. TV camera 1
The control method of b may be the same as that described above. The amount of zoom etc. can also be controlled.

A case where the manipulator 4 is moved while avoiding the obstacle 62 will be described with reference to FIG.
In such a work environment, the avoidance route 61 is calculated using the position data of the obstacle 62. The manipulator 4c moves on the avoidance path 61 by CP control, and finally moves to the position of the manipulator 4c. If the work environment is not known in advance, the operator can similarly move by giving the position data of the obstacle 62 to the calculation unit.

The operation flow will be described with reference to FIG. According to the present invention, in step 101, the TV camera is aligned with the work object, the manipulator movement instruction is given in step 102, and the manipulator moves to the target point in step 103. The instruction in step 102 may be given by the end of step 101 or may be given by the operator. In addition, the present invention can be modified in various ways without departing from the scope of the invention.

[0032]

As described above, according to the present invention, when the operator sets the TV camera on the work subject, the manipulator moves to a predetermined work posture, thereby improving work efficiency. Also, in a system equipped with multiple TV cameras,
If one camera is set on the work target, the other cameras will automatically provide the work target image, thus improving work efficiency.

[Brief description of drawings]

FIG. 1 is a configuration diagram according to this embodiment.

FIG. 2 is a motor function diagram according to the present embodiment.

FIG. 3 is a vector representation for calculating a movement amount of a manipulator.

FIG. 4 is a basic control flow chart.

FIG. 5 is a diagram illustrating a method of using an angle of view and a focal length of a TV camera.

FIG. 6 is a diagram for explaining an end point after moving the manipulator.

FIG. 7 is a diagram illustrating a stereoscopic camera.

FIG. 8 is a basic configuration diagram for generating a stereoscopic image.

FIG. 9 is a basic control flow diagram when a stereoscopic camera is used.

FIG. 10 is a diagram illustrating an end point to which the manipulator should move when a stereoscopic camera is used.

FIG. 11 is a diagram illustrating that a touch sensor is incorporated in a monitor.

FIG. 12 is a diagram illustrating an actual end point to be moved at a location designated by a touch sensor.

FIG. 13 is a basic control flow chart when controlling a manipulator and a platform.

FIG. 14 is a diagram illustrating a movement amount of a platform.

FIG. 15 is a diagram for explaining controlling other TV cameras when two TV cameras arranged at different places are used.

FIG. 16 is a diagram illustrating that the manipulator moves while avoiding an obstacle.

FIG. 17 is an operation flow chart.

[Explanation of symbols]

 1 ... TV camera 2 ... Head 4 ... Manipulator 5 ... Work object 8 ... Worker 21 ... Target point 40 ... Moving target point 42 ... Convergence angle 44 ... Elevation angle

Claims (7)

[Claims] 1. A TV camera having an autofocus function, a detector for detecting an autofocus position, and a detector for detecting a zoom position, and a detection for detecting a position of a pan head capable of changing the posture of the TV camera. Equipped with a device for calculating a proper posture for the manipulator to work from camera signals such as autofocus position and zoom position, and pan head position information of the TV camera, so that the operator can operate the TV camera in the working environment. A manipulator with a TV camera, characterized in that the manipulator is driven to an appropriate position after the adjustment. 2. A position detection for detecting the position of each of the two TV cameras, wherein the TV camera is configured so that stereoscopic viewing is possible with the two TV cameras, and the distance between the two cameras and the convergence angle and the elevation angle can be changed respectively. Equipped with a device, camera interval, vergence angle,
Position information such as elevation angle, the automatic focus position, zoom position, T
Using the means for calculating the posture suitable for the manipulator to work from the pan head position information of the V camera, the operator adjusts the TV camera to the work environment and then uses the function to move the manipulator to the proper position of the manipulator. A manipulator with a TV camera characterized by being driven. 3. A TV camera image is displayed on a monitor, an operator designates a predetermined portion of the image on the monitor, and a detector is provided for detecting the designated portion. Information on the designated portion, the camera interval, and the congestion are provided. The operator adjusted the TV camera to the work environment by using the means for calculating the proper posture for the manipulator to work from the position information such as angle and elevation, the autofocus position, the zoom position, and the pan head information of the TV camera. A manipulator with a TV camera, characterized in that the manipulator is driven to an appropriate position of the manipulator after designating a designated position. 4. When it is more appropriate to move the TV camera platform to see the designated portion of the image on the monitor when performing work, the platform of the TV camera is moved to obtain a proper TV. Driving the manipulator to the proper position of the manipulator by using the function after moving the TV camera to the camera position and using the means for calculating the proper posture for the manipulator to work, and after adjusting the TV camera to the work environment by the operator. The manipulator with a TV camera according to claim 3, wherein 5. A plurality of TV cameras are arranged at different positions, and each TV camera is provided with a pan with n degrees of freedom, and the autofocus position and zoom position of the TV camera selected as appropriate and the pan head position information of the TV camera. Therefore, the manipulator drives the manipulator to an appropriate position of the manipulator by using the means for calculating the proper posture for working, and after the operator adjusts the TV camera to the work environment, the function is used. Manipulator with TV camera. 6. A plurality of TV cameras are arranged at different positions, and each TV camera is provided with a pan with n degrees of freedom. From the autofocus position and zoom position of the TV camera and the pan head position information of the TV camera selected appropriately. Set a proper work environment and
A manipulator with a TV camera, characterized in that the platform of the V camera, the automatic focus position and the zoom position are driven to predetermined positions to obtain appropriate video information of the work environment. 7. The manipulator moves from the information of the TV camera to an appropriate position while avoiding obstacles based on the image information and the environmental information that the manipulator has in advance.
The manipulator with a TV camera described in 4, 5, and 6.