JP2007047142A - Position attitude measuring device using image processing and laser beam - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve a problem of a method of using an image processing technology for measuring a position attitude of a movable body wherein the zoom of a camera must be changed when movable body moves near and far, the camera optical axis changes, separation from the actual coordinate system arises, and accurate measurement cannot be performed when the zoom is changed, an instrument frequently cannot be placed in the midway, and there are visual read instrumentations using a laser beam, a scale plate, and a monitor TV to frequently cause an error when the position attitude of an excavating machine in narrow tunnel construction or the like is measured, a gyrocompass used in a curved tunnel is expensive but has low accuracy, can measure only azimuthal angle, and cannot measure the position, and a present instrumentation system using image processing is expensive. <P>SOLUTION: Utilizing an inexpensive and accurate monitor integrated type CCD camera, a free suspension marker or the like, and a laser beam used as a base line, high-speed and highly accurate instrumentation is performed with an image processing device and mathematical processing. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

The present invention relates to an automatic position / orientation measurement system capable of automatically measuring the position / orientation of a surgical machine tool operation arm, a curved type tunnel (tunnel) excavation machine, physical distribution, a transport machine of a transfer system, and the like.

Japanese Patent Application No. 11-35793 Conventionally, it has been difficult to measure the position and orientation of the excavating machine and the construction performance in a surgical operation tool or a curved path or narrow path construction because of the poor environment and its narrowness. For surgical tools, encoders were installed at the joints of the manipulators and calculated from the angular distance, and for tunnel construction, a transit, distance measuring tape, calculator, personal computer, illuminator, etc. were carried all together for visual measurement. However, measurement errors frequently occur due to the inferior environment, and it is necessary to measure frequently especially in the case of curved roads.
The invention of Japanese Patent Application No. 11-35793 of the reference is based on the invention of the present applicant, and includes a plurality of CCD cameras (hereinafter referred to as CCD cameras and the like including C-MOS type cameras, image pickup tube type TV cameras, etc.). The method of automatic surveying of a curved follower excavator using an on-board 3D measuring device is shown. Regarding the method of using a CCD camera or the like, in the case of the present invention, when the distance between the CCD camera and the object to be measured is constant, the zoom and focus of the camera need only be constant.
The present invention is intended to use an inexpensive commercially available CCD camera or the like.
In general, when performing image processing measurement, a fixed focus lens is used and various optical parameters (optical axis, focal length, lens center, lens distortion, etc.) are measured and set before measurement. The principle is that zoom and focus are not changed.

In measurement using image processing, there is always a problem of optical axis movement (coordinate system change) in response to camera zoom changes (except for broadcasting equipment and precision measurement equipment that perform zoom using a high-precision lens feed mechanism). As a general rule, an angle encoder is added to the zoom mechanism, and the optical axis change according to the zoom is fed back to the measured value. However, calibration is necessary for each individual, and it can be expected for bulky and inexpensive commercial products. It's not practical.
In a narrow path, the method of collimating the three-dimensional marker for measurement with a CCD camera or the like may not be able to see all of the plurality of markers because the field of view is narrow.
Conventionally, a single camera system could measure and measure an object two-dimensionally, but a 3D position / orientation cannot be measured, and a 3D position / orientation measurement is performed using two camera systems ( There were restrictions such as a so-called stereo camera system, high cost, and limited installation location.
The unit of measurement of the camera is in units of pixels, and conversion to the actual size uses a cumbersome method such as measuring a solid whose actual size is known in advance, performing calibration, and referring to the conversion coefficient table each time.
Conventionally, when the zoom or focus is changed, the optical parameters are measured and set again, so that it is necessary to repeat the complicated process of moving to a place where the parameter measuring equipment or environment is located and setting the measurement again. .

The measuring method according to claim 1 is a method of measuring a three-dimensional position and orientation of a three-dimensional marker installed on a moving body by mounting a laser emitter, a CCD camera, a zoom lens, etc. on a manual or automatic three-dimensional swivel mounted with a distance meter. This is a method of calculating the position and orientation of (FIG. 1).
The laser emitter is assumed to have a distance measuring function.
The position and orientation refers to three-dimensional coordinates X, Y, Z and a rotation angle ternary with respect to the X, Y, Z axes, so-called pitching, yawing, and rolling.
The laser beam 11 is fixed to a turning reference axis of the swivel base at a fixed or constant angle and swivels at a known optical axis position angle, or is fixed at a point different from the swivel base and camera system like the laser 14, and the light. The position and angle of the shaft are assumed to be known.
In the explanation, it is assumed that the laser is fixed on the turning center.
At this time, if the position angle of the optical axis of the camera coordinate system is known, accurate and quick measurement can be performed, but it is not always necessary to know.
In this state, the marker 12 and the reference laser irradiation image are captured and image-processed, and the position and orientation in the camera optical axis system are calculated from the marker coordinates in the image plane and distance information measured by a distance meter, laser distance meter, or another means. Further, the position and orientation are converted into an absolute coordinate system, and the position and orientation of the moving object are calculated. A method for obtaining the three-dimensional position and orientation of the marker from the position (two-dimensional) in the image plane of the group of three-dimensional markers 12 is a single-lens camera stereoscopic method, a proportional method, or the like.
The single-lens camera stereoscopic viewing method here refers to a three-dimensional solid object having a known shape and a lens having a focal length f, and an actual distance from the lens to the image formation position to the measurement object is calculated. Is the method. In FIG. 3, 30 is the height A of the solid, 31 is the distance L to be obtained, 32 is the focal length f, 33 is the distance d from the lens to the CCD, and 34 is the height a of the CCD image.
In advance, d is obtained by calibration from the focal length, the known A, and the known distance in the manufacturing process. In this way, by performing image processing with an optical system in which d is determined in advance, the unknown distance L is determined from the triangle consisting of the measured values a, D, and A
L = A * d / a can be obtained.
According to this method, not only the distance but also the three-dimensional position and orientation of all solids can be calculated. That is, as in the example of FIG. 3-35, it can be calculated by the single-lens camera stereoscopic method in the previous period on the condition that a known actual triangular pyramid A is imaged a on the CCD.

The proportional method referred to here is a method of calculating the position and orientation of an object from the pixel coefficient of a marker image having a certain actual dimension and a proportional coefficient between the actual dimensions.
In FIGS. 1 and 2, the markers 12a, 12b, and 12c are fixed to the moving body, but the markers 12d and 12e are freely suspended vertically, and the vibrations of the moving body are absorbed by weights and dampers to stably stabilize the vertical. It is held by a mechanism that keeps Reference numerals 12f and 12g are markers that are always kept horizontal by the same mechanism.
Between 12d-12e, a constant length and verticality are maintained regardless of the posture of the moving body, and the crossing angle of 12d-12e and 12f-12g always maintains a right angle, so that the two-dimensional real coordinate value of each marker is calculated using this as a scale. The posture can also be calculated approximately by the distance between 12a, 12b, and 12c and the coordinate value.
When 12a and 12b are substantially perpendicular to the optical axis, the actual distance between 12a and 12b can be used as a scale.
Similarly, if the free hanging markers 12e, 12f, and 12g are suspended from the marker 12c and used as a scale for calculating the actual position of the 12c, more accurate measurement can be performed.
When a large number of markers cannot be installed, three or more plural markers (12h, 12i, 21j) are rigidly contacted three-dimensionally, and the entire number is freely suspended to reduce the number of markers.
When the distance between a plurality of markers and the camera is greatly different, more accurate measurement can be performed by using a plurality of free hanging markers in this way.
When the zooming operation is performed so that the image can be easily processed when the moving object moves far and away and the image becomes difficult to see, the optical axis of the camera changes and deviates from the original coordinate axis, and the relationship with the absolute coordinate system becomes unclear.
Since the position of the marker 12 group in the image plane changes, it cannot be converted into the absolute coordinate system as it is. However, when the method of the present invention is used, the laser irradiation image position is in the absolute coordinate system which is not affected by the zoom ( FIG. 4), the absolute position of the marker is obtained by correcting the marker image position to the relationship with the laser image position.
Here, the individual markers of the marker group 12 may be the same color, but the image processing capability can be further enhanced by color-coding. For example, since the laser beam 13 is normally red, for example, if the marker 12a is green b, c is blue, and the free hanging markers 12d, e, f, g are yellow, etc., the positional relationship between the front, rear, left and right of the marker becomes clear. This is effective for creating image processing software.
Also, when the marker is close, the lens is zoomed, but the parameters of the original optical system change, and the measurement error increases as it is, so measure the distance and image processing before zooming and measure the coordinates of the marker. After zooming, the parameters are recalculated and set based on the known values.

In the second aspect of the invention, an optical system in which a plurality of curved reflectors (including planar reflectors), half mirrors, and screens are arranged to face each other is fixed to a moving body, and a laser beam having a known incident angle is irradiated to the optical system. In this method, the image of the half mirror and the image of the screen are processed by a CCD camera or the like to measure the position and orientation of the moving body.
FIG. 5 shows the above optical system (41, 42, 43. The reflector is arranged so as not to interrupt the camera field of view and the optical path of the laser 52, and its positional relationship is known), a plurality of CCD cameras, etc. Cameras 40, 40a, 40b Cameras outside light receivers 40c), light-receiving devices 50 comprising free-hanging vertical markers 44a, b (two or more or linear markers) and free-hanging horizontal markers 44c, d, CCD cameras, etc. The image 51 which synthesize | combined these images is shown.
To briefly explain the principle, one CCD camera 40, etc., and the reflecting mirrors are all plane mirrors. The incident laser is first incident on the half mirror 41, totally reflected by the plane mirror 42, and finally formed on the screen 43. It shall be. In addition, free hanging vertical markers 44a and 44b and horizontal markers 44c and d are provided in 50 to measure the inclination of 50 with respect to the vertical and as a conversion scale from image pixel units to actual units. How to convert

The same principle as described in detail on the line. (For conversion from pixel unit to real unit, a pre-calibrated conversion coefficient can be used.)
An image 51 shows a reflected or semi-transmitted image halfway through the laser that is incident on the front half mirror 41 of the light receiver 50 and finally forms an image on the screen 43.
The image positions 41, 42, and 43 are transitions from the laser beam that is the reference line, that is, the vertical and horizontal coordinates, and the value obtained by dividing the value by the optical path distance is the direction angle.
That is, the horizontal divergence distance 45 between 41 and 43 is converted into an actual distance, and the horizontal angle (yawing) with respect to the laser optical axis of the light receiver can be calculated from the value divided by the actual distance between 41 and 42.
The vertical divergence distance 46 between 41 and 43 is converted into an actual distance, and the vertical angle (pitching) with respect to the laser optical axis of the light receiver can be calculated from the value obtained by dividing by the actual distance between 41 and 42.
If there is a space only in the laser direction, such as a narrow path, the half mirror 41, camera 40a, and reflecting mirror 42 are half mirrors, and the camera 40a performs image processing of 41 and 42, and the position and orientation are calculated by the previous calculation method. calculate. The cameras 40, 40a, and 40b are installed in the light receiver and need to be corrected due to the influence of the displacement of the position and orientation of the light receiver. However, 40c is separated from the light receiver and is not affected by the displacement of the light receiver. Yes, the correction is simplified.
The camera 42b can cope with a change in the arrangement of the iso-reflecting mirror, the half mirror, the screen, and the camera used when the depth space is limited. In addition, by using a plurality of cameras, the cameras can be installed near the video, and the image resolution can be increased and the measurement accuracy can be improved.
If some of the reflecting mirrors are curved mirrors having a known curvature instead of a plane mirror, or an uneven lens is disposed on the laser optical path, it is possible to deal with a more restricted space.
If the incident laser is red as in the first aspect, the image processing ability can be further enhanced if the markers 44a, b, c, d are color-coded other than red.

Image processing position / orientation measurement and surveying using conventional commercial digital cameras and movie cameras were impossible due to the ambiguity of the optical axis, but the ambiguity of the optical axis was eliminated by combining it with the linearity of the laser beam. It is possible to improve accuracy.
When measuring the shape of a moving solid, it is necessary to measure the position of each point of the solid as accurately as possible. By using the high speed of the image processing technique according to the present invention, it is possible to measure many points instantaneously. I can do it.

Currently, a commercially available digital video camera is purchased, a laser irradiation machine is installed, a prototype of the present invention is manufactured, and image processing software is developed and tested.

Tests using the above prototypes are being conducted at the sewage construction shield work site.

Measurement of the position and direction of moving bodies such as manipulators for medical surgical equipment, machine tool operating equipment, construction work, excavation machinery, transportation and transporting machines. Position and orientation measurement of fixed objects such as structures and work installations.

It is drawing which showed the whole block diagram of Claim 1. [Claim 1] A plane view showing a positional relationship among an optical axis of a CCD camera, a reference laser, and a marker. It is a principle diagram of stereoscopic vision measurement. It is a figure showing the state where the image after zooming deviated from the image before zooming. FIG. 2 is a view showing images of a light receiver and a CCD camera.

Explanation of symbols

10. . . 10. CCD camera etc. . . Reference laser irradiator (with built-in ranging function)
12 . . Three-dimensional marker groups 12a, 12b. . . . . . Three-dimensional marker front marker 12c. . . . Three-dimensional marker rear markers 12d, 12e, 12f, 12g. . . . . Free suspension marker (12f and 12g are perpendicular to 12d and 12e)
12h, 12i, 12j. . . . Free suspended solid marker 13. . . Reference laser irradiation plate / ranging reflection plate 14. . . . . Reference laser irradiator (with built-in ranging function)
20. . . 20. Reference laser image before and after zooming . . 30. Marker image before and after zoom . . . . Actual object dimensions 31. . . . . Actual distance from lens to object 32. . . . . Lens focal length 33. . . . . Distance between lens and CCD 34. . . . . Object image size 35. . . . . Principle of image processing measurement of a three-dimensional object by a stereoscopic method 40, 40a, 40b. . . . . . . . CCD camera 40c. . . . . Fixed camera outside the receiver 41. . . . . Half mirror 42. . . . . Reflector 43. . . . . Screen 44. . . . . Free suspension marker 41a. . . . . Laser image 42a. . . . Laser image 43a when 42 is a half mirror. . . . . In-screen laser images 44a, 44b. . . . . Free suspended vertical markers and images 44c, 44d. . . . . Free suspended horizontal marker and image 45. . . . . 41a, 43a Image horizontal separation (in pixels)
46. . . . . 41a, 43a Image vertical separation (in pixels)
47. . . . . Receiver tilt distance (unit pixel)
48. . . . . 41a, 42a Horizontal separation (unit pixel)
49. . . . . 41a, 42a Vertical separation (unit pixel)
50. . . . . Light receiver 51. . . . . Image state 52. . . . . Laser 53. . . . . lens

Claims (2)

Computer controlled or manually controlled up / down, left / right turning mechanism, variable zoom held by horizontal adjustment mechanism, variable focus function CCD camera, etc. and image processing device and laser beam as a baseline, connected to the CCD camera, etc. Measure the position and orientation of a moving object with three or more multi-colored or three-dimensional markers with multi-color light emitters and a vertical and horizontal marker supported by a free suspension mechanism. how to. A plurality of curved mirrors and lenses that are held by a computer or manually controlled vertical and horizontal turning mechanisms and a horizontal adjustment mechanism, and irradiate a laser beam emitted from an arbitrary angle and a laser beam emitted from a three-dimensional position measuring device using a distance meter. The half mirror and screen are arranged in parallel or at a constant crossing angle, and the laser beam half mirror and the image on the screen that are incident at a known angle and the image of the multicolor marker that is supported by the free suspension mechanism and always maintains vertical and horizontal are displayed. A method of measuring the position and orientation of a moving object using a single or a plurality of CCD cameras or the like and installing a light receiving device for measuring the incident position and direction of a laser beam.

Images