CN113052914A - Container terminal transport vehicle positioning system and method based on binocular stereo vision - Google Patents

Container terminal transport vehicle positioning system and method based on binocular stereo vision Download PDF

Info

Publication number
CN113052914A
CN113052914A CN202110229284.1A CN202110229284A CN113052914A CN 113052914 A CN113052914 A CN 113052914A CN 202110229284 A CN202110229284 A CN 202110229284A CN 113052914 A CN113052914 A CN 113052914A
Authority
CN
China
Prior art keywords
camera
vehicle
coordinate system
diamond
binocular stereo
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110229284.1A
Other languages
Chinese (zh)
Inventor
刘春明
欧阳乐
王全宇
吕刚晓
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Zhenghua Heavy Industries Co Ltd
Original Assignee
Shanghai Zhenghua Heavy Industries Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Zhenghua Heavy Industries Co Ltd filed Critical Shanghai Zhenghua Heavy Industries Co Ltd
Priority to CN202110229284.1A priority Critical patent/CN113052914A/en
Publication of CN113052914A publication Critical patent/CN113052914A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G06T7/85—Stereo camera calibration
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
    • G01C11/02—Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras

Abstract

The invention discloses a container terminal transport vehicle positioning system and method based on binocular stereo vision, which comprises the following steps: the diamond-shaped mark is arranged on the ground; the binocular stereo camera is arranged on the vehicle and comprises a first camera and a second camera, and the binocular stereo camera is used for shooting the diamond-shaped marks; the measuring device is used for measuring the coordinates of the diamond-shaped identifier, the vehicle and the binocular stereo camera under a world coordinate system; and the calculating unit is in communication connection with the binocular stereo camera and the measuring device and is used for positioning the vehicle according to the information sent by the binocular stereo camera and the measuring device. By adopting the technical scheme, the positioning system is high in positioning accuracy and strong in adaptability.

Description

Container terminal transport vehicle positioning system and method based on binocular stereo vision
Technical Field
The invention relates to the technical field of vehicle positioning, in particular to a container terminal transport vehicle positioning system and method based on binocular stereo vision.
Background
The unmanned and intelligent wharf is the core for improving the port transfer efficiency, wherein a positioning system of a transport vehicle at a container wharf is one of the keys for realizing the unmanned wharf. Currently, the main positioning methods used in the existing positioning systems for mobile vehicles include: a positioning method based on differential satellite signals, a positioning method based on ground line drawing and image recognition, a magnetic positioning method based on a ground buried detection body, a positioning method based on laser, ultrasonic or infrared ranging, and the like.
The positioning system based on the differential satellite signals has high cost, limited positioning precision and response speed and low stability, can generate signal blind areas and short-period jumping under the influence of weather, cranes, containers, lighthouses and the like, and has serious limitation on application and popularization of container terminals. The positioning system based on ground line drawing and image recognition cannot meet the requirement of all-weather operation of a storage yard because the reference line of the positioning system is easy to be polluted, and the positioning result of the system is greatly influenced by the posture of a vehicle body, so that the application and popularization of the positioning system are limited. The magnetic positioning system based on the ground buried detection body needs to implement civil engineering on the ground of the storage yard, and the system is also interfered by metal fragments scattered frequently on the ground, so that the requirement on an application scene is higher. Positioning systems based on laser, ultrasonic or infrared ranging need to perform positioning measurement by means of actual reference reflectors, and the systems have high requirements on the reflection capability of the reflectors, are seriously interfered by external light and the like, and sometimes are difficult to accurately reflect the actual positions of equipment.
Therefore, a container terminal transport vehicle positioning system with high positioning accuracy and strong adaptability is urgently needed.
Disclosure of Invention
The invention aims to solve the problems that a positioning system in the prior art is low in positioning accuracy and difficult to adapt to wharf environment.
In order to solve the technical problem, the embodiment of the invention discloses a container terminal transport vehicle positioning system based on binocular stereo vision, which comprises: the diamond-shaped mark is arranged on the ground; the binocular stereo camera is arranged on the vehicle and comprises a first camera and a second camera, and the binocular stereo camera is used for shooting the diamond-shaped marks; the measuring device is used for measuring the coordinates of the diamond-shaped identifier, the vehicle and the binocular stereo camera under a world coordinate system; and the calculating unit is in communication connection with the binocular stereo camera and the measuring device and is used for positioning the vehicle according to the information sent by the binocular stereo camera and the measuring device.
By adopting the technical scheme, the positioning system is high in positioning accuracy and strong in adaptability.
Optionally, the diamond indicates a vertex angle of 60 degrees in the vehicle traveling direction.
Optionally, the sides of the diamond markers range from 50cm to 100 cm.
Optionally, the vehicle is provided with an industrial computer, the industrial computer comprising the computing unit.
Optionally, the vehicle is provided with an electrical cabinet, the industrial computer being disposed within the electrical cabinet.
Optionally, the first camera and the second camera are both connected to the industrial computer through a USB serial data line.
Optionally, the diamond markings are white.
Optionally, the surveying device comprises a total station and a calibration plate.
The embodiment of the invention also discloses a container terminal transport vehicle positioning method based on binocular stereo vision, which is suitable for any one of the positioning systems, and the positioning method comprises the following steps: measuring a first world coordinate of the diamond-shaped mark under a world coordinate system by a measuring device; the measuring device carries out external reference calibration on the binocular stereo camera to obtain a first conversion relation between a camera coordinate system and a vehicle body coordinate system; shooting by a binocular stereo camera to obtain an image of the diamond-shaped mark; the calculation unit positions the vehicle according to the first world coordinate, the first conversion relation and the image.
By adopting the technical scheme, the positioning method has high positioning precision and strong adaptability.
Optionally, the step of locating the vehicle according to the first world coordinate, the first conversion relation and the image by the computing unit includes: the calculation unit obtains a first camera coordinate of the diamond identifier in a camera coordinate system according to the image; the calculation unit obtains a first vehicle body coordinate of the diamond identifier under the vehicle body coordinate system according to the first camera coordinate and the first conversion relation; the calculation unit obtains a second conversion relation between the vehicle body coordinate system and the world coordinate system according to the first world coordinate and the first vehicle body coordinate; and the calculating unit positions the vehicle according to the second conversion relation to obtain the coordinates of the vehicle in the world coordinate system.
Optionally, the positioning method further comprises the following steps: and the calculating unit feeds back the three-dimensional pose information of the vehicle according to the coordinates of the vehicle in the world coordinate system and the second conversion relation.
Optionally, the camera coordinate system includes a first camera coordinate system and a second camera coordinate system, the first conversion relationship includes a first sub-conversion relationship and a second sub-conversion relationship, and the measuring device performs external reference calibration on the binocular stereo camera to obtain the first conversion relationship between the camera coordinate system and the vehicle body coordinate system, including: the measuring device respectively measures a second world coordinate and a third world coordinate of the first camera and the second camera in a world coordinate system; the measuring device measures a fourth world coordinate of the vehicle under a world coordinate system; and obtaining a first sub-conversion relation according to the second world coordinate and the fourth world coordinate, and obtaining a second sub-conversion relation according to the third world coordinate and the fourth world coordinate.
Optionally, the image includes a first image and a second image, the first camera coordinates include first camera sub-coordinates and second camera sub-coordinates, and the step of obtaining, by the computing unit according to the image, first camera coordinates of the diamond identifier in the camera coordinate system includes: the method comprises the steps that a calculation unit obtains a first image and a second image of a diamond mark shot by a first camera and a second camera; the calculation unit obtains first camera sub-coordinates of the diamond identifier under a first camera coordinate system and second camera sub-coordinates of the diamond identifier under a second camera coordinate system according to the first image, the second image and the internal reference data of the binocular stereo camera.
Drawings
FIG. 1 shows a schematic view of a positioning system in an embodiment of the invention;
FIG. 2 illustrates a schematic diagram of diamond shaped markers in one embodiment of the present invention;
FIG. 3 illustrates a flow chart of a positioning method in an embodiment of the invention;
fig. 4 shows a schematic diagram of coordinate system establishment of a positioning system in another embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present invention is provided for illustrative purposes, and other advantages and effects of the present invention will become apparent to those skilled in the art from the present disclosure. While the invention will be described in conjunction with the preferred embodiments, it is not intended that features of the invention be limited to these embodiments. On the contrary, the invention is described in connection with the embodiments for the purpose of covering alternatives or modifications that may be extended based on the claims of the present invention. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. The invention may be practiced without these particulars. Moreover, some of the specific details have been left out of the description in order to avoid obscuring or obscuring the focus of the present invention. It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict.
It should be noted that in this specification, like reference numerals and letters refer to like items in the following drawings, and thus, once an item is defined in one drawing, it need not be further defined and explained in subsequent drawings.
In the description of the embodiments of the present invention, it should be noted that the terms "upper", "lower", "inner", "bottom", and the like, indicate orientations or coordinate relationships based on the orientations or coordinate relationships shown in the drawings or orientations or coordinate relationships conventionally arranged in use of products of the present invention, and are only used for convenience in describing the present invention and simplifying the description, but do not indicate or imply that the referred device or element must have a specific orientation, be constructed in a specific orientation, and be operated, and thus, should not be construed as limiting the present invention.
The terms "first," "second," and the like are used solely to distinguish one from another and are not to be construed as indicating or implying relative importance.
In the description of the embodiments of the present invention, it should be further noted that unless otherwise explicitly stated or limited, the terms "disposed," "connected," and "connected" are to be construed broadly and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meanings of the above terms in the present embodiment can be understood in specific cases by those of ordinary skill in the art.
In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
Referring to fig. 1, an embodiment of the present invention discloses a container terminal transport vehicle 4 positioning system based on binocular stereo vision, including: the diamond mark 1 is arranged on the ground 2; the binocular stereo camera 3 is arranged on the vehicle 4, the binocular stereo camera 3 comprises a first camera 31 and a second camera 32, and the binocular stereo camera 3 is used for shooting the diamond-shaped mark 1; the measuring device 5 is used for measuring coordinates of the diamond-shaped mark 1, the vehicle 4 and the binocular stereo camera 3 in a world coordinate system; and the calculating unit 6 is in communication connection with the binocular stereo camera 3 and the measuring device 5, and is used for positioning the vehicle 4 according to the information sent by the binocular stereo camera 3 and the measuring device 5.
In the present embodiment, the diamond marker 1 is disposed on the ground 2, that is, the diamond marker 1 may be disposed in the working area of the vehicle 4 as a positioning marker, and the diamond marker 1 may be directly disposed on the ground 2, or may be indirectly disposed on the ground 2 through other carrying objects. Compared with positioning by differential satellite signals, the positioning by using the diamond-shaped identification has higher stability, is not easily influenced by environmental factors such as weather, cranes and the like, is more suitable for wharfs and is convenient to popularize; compared with the standard lines such as lane lines and the like as markers, the rhombic markers are not easy to be polluted in the driving process of the vehicle, so that the method is more reliable; compared with a magnetic positioning system for embedding a detection body on the ground, the magnetic positioning system is lower in cost and more stable; compared with laser, ultrasonic or infrared ranging, the method has the advantages that a reflector is not needed, the cost is low, and the influence of the environment is small. The diamond mark 1 has four obvious angular points, is easy to identify, can complete the conversion relation calculation between three-dimensional coordinate systems once through one diamond mark 1, is convenient for setting a positioning system, and has high positioning precision. Meanwhile, the calculation of the conversion relation by the four corner points has redundancy, and the calculation precision can be improved. In addition, the lengths of the four sides of the diamond mark 1 are equal, so that the arrangement is convenient. Preferably, the diamond-shaped mark 1 is fixed on the ground 2 in a spraying or hot melting mode, so that the influence on the working environment is small and the construction is convenient. The size, color, and number of the diamond markers 1 may be determined according to the size, illumination, and camera view of the actual working scene, which are not limited in this embodiment. It can be understood that when the vehicle 4 is a one-way driving vehicle, the binocular stereo camera 3 may be installed only on one side of the vehicle head, and when the vehicle 4 is a two-way driving vehicle, the binocular stereo camera 3 may also be installed on both sides, which is not limited in this embodiment.
The binocular stereo camera 3 is provided on the vehicle 4, the binocular stereo camera 3 includes a first camera 31 and a second camera 32, and the binocular stereo camera 3 is used to photograph the diamond markers 1. That is, in the positioning system disclosed in the present embodiment, the relative positional relationship between the binocular stereo camera 3 and the vehicle 4 is maintained, and the image of the diamond 1 can be captured, and the container terminal transport vehicle 4 can be positioned based on the binocular stereo vision. Binocular stereo vision is an important research field in the field of machine vision, and can acquire spatial geometric information of a target object by calculating a positional deviation (i.e., parallax) between corresponding points of the target object in different images using images of the target object acquired at two different camera positions. The binocular stereo vision is used for positioning the vehicles, has the advantages of low cost, simple system structure, no contact, high efficiency and high precision, is suitable for the online non-contact measurement of mobile equipment, is particularly suitable for positioning container transport vehicles at a wharf, can realize the unmanned and intelligent container transport at the wharf, and further improves the transfer efficiency at the port. And because the image acquisition is accomplished in the twinkling of an eye, consequently can promote the efficiency of positioning process.
Preferably, the binocular stereo camera 3 is fixed to the vehicle 4 by a rigid body firmware (not shown), and the first camera 31 and the second camera 32 are installed right and left in the horizontal direction (X direction shown in fig. 1) of the vehicle 4, so that the stability of the position posture of the cameras can be improved, and the positioning accuracy can be improved.
In the present embodiment, the measurement device 5 is used to measure coordinates of the diamond markers 1, the vehicle 4, and the binocular stereo camera 3 in a world coordinate system. The measuring device 5 may be a measuring instrument such as a total station. And the calculating unit 6 is in communication connection with the binocular stereo camera 3 and the measuring device 5, and is used for positioning the vehicle 4 according to the information sent by the binocular stereo camera 3 and the measuring device 5. The computing unit 6 may be a single chip microcomputer, a microcontroller, a computer, a mobile terminal, a cloud server, or other devices with data processing and computing functions or unit modules therein. In one embodiment, the calculation unit 6 receives the images of the diamond markers 1 sent by the binocular stereo camera 3 and the coordinates of the diamond markers 1, the vehicle 4 and the binocular stereo camera 3 sent by the measurement device 5 in the world coordinate system. It is understood that the calculation unit 6 can obtain the conversion relationship between the vehicle body coordinate system and the camera coordinate system by the coordinates of the vehicle 4 and the binocular stereo camera 3 in the world coordinate system, respectively. In one embodiment, the vehicle body center of the vehicle is set as the origin of the vehicle body coordinate system, the camera center is set as the origin of the camera coordinate system, and the conversion relationship between the vehicle body coordinate system and the camera coordinate system can be obtained by the coordinates of the vehicle body center and the camera center under the world coordinate system and the world coordinate system as the middle coordinate system, wherein the conversion relationship comprises a rotation relationship and a translation relationship, and is represented by a rotation matrix and a translation matrix.
Then, the coordinates of the diamond 1 in the camera coordinate system can be obtained through the image of the diamond 1 by the calculating unit 6, and the coordinates of the diamond 1 in the vehicle body coordinate system are calculated according to the conversion relation obtained in the foregoing. In an embodiment, the calculating unit 6 performs feature extraction and image recognition on the image of the diamond marker 1, can obtain pixel coordinates of four corner points in the imaged image, and can obtain coordinates of the four corner points of the diamond marker 1 in the camera coordinate system by combining the imaging principle and the origin position of the camera coordinate system. And then, combining the coordinates of the diamond identifier 1 in the world coordinate system to obtain a conversion relation between the vehicle body coordinate system and the world coordinate system, further obtaining the coordinates of the vehicle 4 in the world coordinate system, and completing accurate positioning of the vehicle 4. It will be appreciated that the work area may be provided with a plurality of diamond markers 1. Therefore, the continuous positioning of the vehicle 4 can be completed according to the diamond marks 1 at different positions during the running process of the vehicle 4.
The container terminal transport vehicle positioning system disclosed by the embodiment is based on binocular stereoscopic vision and ground rhombus identification, can reduce the automatic upgrading and modification cost of a terminal mobile vehicle, improve the positioning precision and the response speed, reduce the change requirements on application scenes such as a terminal planning layout, infrastructure and operation modes and the like, and is suitable for not only newly-built terminals but also built old terminals. The system combines the advantages of image recognition, stereoscopic vision and reference body ranging, is not easily interfered by the environment, and has strong adaptability and high positioning precision.
Referring to fig. 1 and 2, in another embodiment of the invention, a container terminal transport vehicle positioning system based on binocular stereo vision is disclosed, and the vertex angle of a diamond-shaped logo 1 in the driving direction (the Y direction is shown in fig. 1) of a vehicle 4 is 60 degrees. That is, in the present embodiment, of the four apex angles of the diamond 1, two apex angles in the traveling direction of the vehicle 4 are 60 °, and the other two apex angles are 120 °. At the moment, the imaging height and width of the diamond-shaped mark 1 in the camera are basically consistent, so that the graph can be conveniently identified in the image, and the positioning precision and efficiency are further improved.
Referring to fig. 2, in another embodiment of the invention, a container terminal transport vehicle positioning system based on binocular stereo vision is disclosed, and the side length L of a diamond-shaped mark 1 ranges from 50cm to 100 cm. In the embodiment, the side length of the diamond-shaped mark 1 is set to be 50cm-100cm, so that the situation that when the size is too small, the interference capability of resisting environmental pollution is weak and difficult to recognize can be avoided, and when the size is too large, the situation that the setting cost is high and the shooting difficulty is large can be avoided, and the side length is matched with the size of a vehicle 4 used for container terminal transportation, so that the vehicle 4 can be shot clearly and completely by the binocular stereo camera 3 in the driving process. Preferably, the first camera 31 and the second camera 32 are industrial color cameras with a resolution of 1280 × 720 and a focal length of 8mm, and can more completely and clearly photograph the diamond-shaped mark 1.
Another embodiment of the invention discloses a container terminal transport vehicle positioning system based on binocular stereo vision, the vehicle 4 is provided with an industrial computer (not shown), and the industrial computer comprises a computing unit 6. In the embodiment, the industrial computer completes data processing and calculation in the positioning process of the positioning system, has high response speed, can improve positioning efficiency by being arranged on the vehicle 4, ensures real-time performance and reduces time delay. Preferably, the program development and running environment of the industrial computer is ubuntu18.04, wherein a third-party image processing library can be used, and the positioning efficiency and precision can be further improved.
In another embodiment of the invention, a container terminal transport vehicle positioning system based on binocular stereo vision is disclosed, wherein a vehicle 4 is provided with an electrical cabinet (not shown), and an industrial computer is arranged in the electrical cabinet. In the embodiment, the industrial computer is arranged in the electrical cabinet, so that the industrial computer can be well protected, and the influence of severe weather on the electrical performance of the industrial computer is avoided.
In another embodiment of the invention, a container terminal transport vehicle positioning system based on binocular stereo vision is disclosed, and the first camera 31 and the second camera 32 are both connected to an industrial computer through USB serial port data lines. That is to say, in this embodiment, the first camera 31 and the second camera 32 are connected to the computing unit 6 in a wired manner, and then data transmission is performed, so that reliability and stability of data transmission can be improved, and data transmission is performed through a USB serial data line, which is convenient to set and has low cost.
The invention further discloses a container terminal transport vehicle positioning system based on binocular stereo vision, and the diamond-shaped mark 1 is white. In the embodiment, the diamond mark 1 is white, and can form bright color contrast with the dock working environment, especially with the dock ground 2, so that subsequent image recognition and feature extraction are facilitated.
The invention discloses a container terminal transport vehicle positioning system based on binocular stereo vision, and the measuring device 5 comprises a total station and a calibration plate. In the present embodiment, the calibration plate can be matched with the total station to be disposed on an object to be measured by the total station, for example, the body of the vehicle 4 or the binocular stereo camera 3, so that the coordinates of the object in the world coordinate system can be measured more efficiently and accurately.
Referring to fig. 3, the embodiment of the invention further discloses a container terminal transport vehicle positioning method based on binocular stereo vision, which is applicable to any positioning system in the foregoing embodiments, and the positioning method comprises the following steps: s1: the measuring device 5 measures a first world coordinate of the diamond-shaped mark 1 in a world coordinate system; s2: the measuring device 5 calibrates external parameters of the binocular stereo camera 3 to obtain a first conversion relation between a camera coordinate system and a vehicle body coordinate system; s3: shooting by a binocular stereo camera 3 to obtain an image of the diamond-shaped mark 1; s4: the calculation unit 6 locates the vehicle 4 on the basis of the first world coordinate, the first conversion relation and the image.
In the present embodiment, steps S1, S2, and S3 may be performed before step S4, and there is no restriction on the order among S1, S2, and S3. At S1, the coordinates of the diamond 1 in the world coordinate system, i.e., the first world coordinates, are measured using the measuring device 5 such as a total station. In S2, the measurement device 5 performs external reference calibration on the binocular stereo camera 3 to obtain a conversion relationship between the camera coordinate system and the vehicle body coordinate system, that is, a first conversion relationship. In S3, the binocular stereo camera 3 photographs the diamond 1, and an image of the diamond 1 is obtained. In S4, the calculation unit 6 obtains the conversion relationship between the vehicle body coordinate system and the world coordinate system from the first world coordinate, the first conversion relationship, and the image of the diamond marker 1, thereby completing the positioning of the vehicle 4. It is understood that in some embodiments, steps S1, S2, S3 and S4 are all required for initial positioning, and step S2 is not required when continuous positioning is required during the running of the vehicle 4 after the initial positioning is completed. In some embodiments, the working area of the vehicle 4 is provided with a plurality of diamond markers 1, and the measuring device 5 may measure the coordinates of all diamond markers 1 in the world coordinate system in advance and store the coordinates in the calculating unit 6 for the subsequent positioning process. Therefore, only the steps S3 and S4 need to be repeated in the continuous positioning process, and the positioning efficiency is improved.
The positioning method disclosed by the embodiment is based on binocular stereoscopic vision and ground rhombus identification, can reduce the automatic upgrading and transforming cost of wharf moving vehicles, improve the positioning precision and the response speed, reduce the changing requirements on application scenes such as wharf planning layout, infrastructure, operation modes and the like, and is suitable for not only newly built wharfs but also built old wharfs. The method combines the advantages of image identification, stereoscopic vision and reference body ranging, is not easily interfered by the environment, and has strong adaptability and high positioning precision.
In another embodiment of the present invention, a method for positioning a transportation vehicle at a container terminal based on binocular stereo vision is disclosed, wherein the step S4 of positioning the vehicle 4 by the calculation unit 6 according to the first world coordinate, the first transformation relation and the image comprises: the calculation unit 6 obtains a first camera coordinate of the diamond-shaped mark 1 in a camera coordinate system according to the image; the calculation unit 6 obtains a first vehicle body coordinate of the diamond-shaped mark 1 in the vehicle body coordinate system according to the first camera coordinate and the first conversion relation; the calculating unit 6 obtains a second conversion relation between the vehicle body coordinate system and the world coordinate system according to the first world coordinate and the first vehicle body coordinate; the calculation unit 6 positions the vehicle 4 according to the second conversion relationship to obtain coordinates of the vehicle 4 in a world coordinate system.
In the present embodiment, the calculation unit 6 can calculate the coordinates of the diamond markers 1 in the camera coordinate system, i.e., the first camera coordinates, from the captured image. After the first camera coordinates of the diamond identifier 1 are obtained, the coordinates of the diamond identifier 1 in the vehicle body coordinate system, that is, the first vehicle body coordinates, can be obtained by combining the first conversion relationship. Then, the world coordinates of the diamond 1 in the world coordinate system are combined, so that a conversion relation between the vehicle body coordinate system and the world coordinate system, that is, a second conversion relation, can be obtained. Finally, according to the second conversion relationship, since the coordinates of the vehicle 4 in the body coordinate system are fixed and known, for example, the coordinates may be the origin coordinates, the coordinates of the vehicle 4 in the world coordinate system may be obtained, and the positioning of the vehicle 4 may be completed.
The invention discloses a container terminal transport vehicle positioning method based on binocular stereo vision, which further comprises the following steps: the calculation unit 6 feeds back the three-dimensional pose information of the vehicle 4 according to the coordinates of the vehicle 4 in the world coordinate system and the second conversion relation. In the present embodiment, the three-dimensional pose information of the vehicle 4 is the position information and the posture information of the vehicle 4. In one embodiment, the position information includes three-dimensional coordinates of the origin of the vehicle body coordinate system in the world coordinate system, and the attitude information includes a rotational relationship between the vehicle body coordinate system and the world coordinate system, such as euler angles, quaternions, and the like. In this embodiment, not only can the positioning of the position of the vehicle 4 be completed in time, but also the current attitude angle of the vehicle 4 can be fed back, so that the state information of the vehicle 4 can be grasped in time. The calculation unit 6 can be in communication connection with management units such as a vehicle-mounted terminal, a mobile terminal and a cloud server, so that the three-dimensional pose information of the vehicle 4 can be fed back in real time, and the management and the scheduling of the vehicle 4 of the whole wharf are facilitated.
The invention also discloses a container terminal transport vehicle positioning method based on binocular stereo vision, the camera coordinate system comprises a first camera coordinate system and a second camera coordinate system, the first conversion relation comprises a first sub-conversion relation and a second sub-conversion relation, the measuring device 5 carries out external reference calibration on the binocular stereo camera 3 to obtain the first conversion relation between the camera coordinate system and the vehicle body coordinate system, and the method comprises the following steps: the measuring device 5 measures second world coordinates and third world coordinates of the first camera 31 and the second camera 32 in the world coordinate system, respectively; the measuring device 5 measures a fourth world coordinate of the vehicle 4 in a world coordinate system; and obtaining a first sub-conversion relation according to the second world coordinate and the fourth world coordinate, and obtaining a second sub-conversion relation according to the third world coordinate and the fourth world coordinate. It can be understood that the binocular stereo camera 3 includes the first camera 31 and the second camera 32, and in the present embodiment, in the positioning calculation process, two camera coordinate systems are respectively established for calculation, so that the accuracy of the positioning result can be further improved. Preferably, the same diamond mark 1 is used for completing the calibration of external parameters of the two cameras at the same position, so that the calibration efficiency can be improved.
The invention discloses a container terminal transport vehicle positioning method based on binocular stereo vision, wherein the image comprises a first image and a second image, the first camera coordinate comprises a first camera sub-coordinate and a second camera sub-coordinate, and the calculation unit 6 obtains the first camera coordinate of the diamond-shaped mark 1 under a camera coordinate system according to the image, and the method comprises the following steps: the calculation unit 6 acquires a first image and a second image of the diamond marker 1 shot by the first camera 31 and the second camera 32; the calculation unit 6 obtains a first camera sub-coordinate of the diamond identifier 1 in a first camera coordinate system and a second camera sub-coordinate of the diamond identifier 1 in a second camera coordinate system according to the first image, the second image and the internal reference data of the binocular stereo camera 3.
In the embodiment, when the first camera sub-coordinate and the second camera sub-coordinate are calculated, the accuracy of the result can be improved by combining the internal reference data of the binocular stereo camera 3, that is, the parameter calibration information inside the camera, such as the lens distortion parameter. The internal reference data may be obtained in advance, or may be directly output by the camera at the time of positioning, which is not limited in the present embodiment. Preferably, the internal reference data is acquired before the two cameras are mounted and fixed. In the present embodiment, the first conversion relationship includes a first sub-conversion relationship and a second sub-conversion relationship, and the first vehicle body coordinate of the diamond marker 1 in the vehicle body coordinate system can be calculated according to the first sub-camera coordinate and the first sub-conversion relationship of the diamond marker 1, or the first vehicle body coordinate of the diamond marker 1 in the vehicle body coordinate system can be calculated according to the second sub-camera coordinate and the second sub-conversion relationship of the diamond marker 1. Therefore, certain redundancy can be provided when the first vehicle body coordinate of the diamond-shaped mark 1 under the vehicle body coordinate system is calculated, mutual inspection is performed, and the calculation accuracy is guaranteed.
Referring to fig. 4, in one embodiment,two cameras are arranged left and right, and four coordinate systems are defined in the storage yard area of the wharf. Coordinate system OwXwYwZwIs a world coordinate system. Coordinate system OcXcYcZcIs a vehicle body coordinate system with a center point OcIs the geometric center point, O, of the body upper frame of the vehicle 4cXcThe axis pointing in the longitudinal direction of the upper frame of the vehicle body, OcYcThe axis pointing in the transverse direction of the upper frame of the vehicle body, OcZcThe axis is determined by the right hand rule; coordinate system OlXlYlZlIs the left camera coordinate system in a binocular stereo camera, where OlXlAxis is the line direction of the left camera imaging plane, OlYlThe axis is the column direction of the imaging plane of the left camera, OlZlThe axis is the left camera optical axis and meets the right-hand rule; coordinate system OrXrYrZrIs the right camera coordinate system in a binocular camera, where OrXrAxis is the line direction of the right camera imaging plane, OrYrAxis in the column direction of the right camera imaging plane, OrZrThe axis is the right camera optical axis, satisfying the right hand rule.
The core of positioning the vehicle 4 for container terminal transportation is to obtain a vehicle body coordinate system OcXcYcZcAnd the world coordinate system OwXwYwZwConversion relation between Tc→w. Firstly, the measuring device 5 can be used to measure the four corner points of the diamond-shaped mark 1 on the ground 2 in the world coordinate system OwXwYwZwCoordinates of lower { (x)w,yw,zw)iAnd (i ═ 1,2,3, 4). Secondly, respectively carrying out external reference calibration on the left camera and the right camera, wherein the external reference calibration of the cameras is to obtain a coordinate system O of the left camera and a coordinate system O of the right cameralXlYlZlAnd OrXrYrZrAnd the vehicle body coordinate system OcXcYcZcConversion relation between Tl→cAnd Tr→c(ii) a Then, combining the internal calibration parameter information of the left camera and the right camera,pixel coordinates of the left camera and the right camera in the imaging image according to four corner points of the diamond mark 1 on the ground 2 { (u)l,vl)i1,2,3,4 and { (u) } (i ═ 1,2,3,4)r,vr)i1,2,3,4), and left and right camera coordinate systems OlXlYlZlAnd OrXrYrZrAnd the vehicle body coordinate system OcXcYcZcConversion relation between Tl→cAnd Tr→cCalculating the four corner points of the diamond mark 1 on the ground 2 in a vehicle body coordinate system OcXcYcZcCoordinates of lower { (x)c,yc,zc)iAnd (i ═ 1,2,3, 4). Finally, according to the four corner points of the diamond mark 1 on the ground 2 in the world coordinate system OwXwYwZwAnd a vehicle body coordinate system OcXcYcZcCoordinates of lower { (x)w,yw,zw)i1,2,3,4, and { (x)c,yc,zc)iCalculate a vehicle coordinate system O (i ═ 1,2,3,4)cXcYcZcAnd the world coordinate system OwXwYwZwConversion relation between Tc→wFurther, the coordinates of the vehicle 4 in the world coordinate system can be obtained, and the positioning of the vehicle 4 is completed.
In the drawings, some features of the structures or methods may be shown in a particular arrangement and/or order. However, it is to be understood that such specific arrangement and/or ordering may not be required. Rather, in some embodiments, the features may be arranged in a manner and/or order different from that shown in the illustrative figures. In addition, the inclusion of a structural or methodical feature in a particular figure is not meant to imply that such feature is required in all embodiments, and in some embodiments, may not be included or may be combined with other features.
While the invention has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that the foregoing is a more detailed description of the invention, taken in conjunction with the specific embodiments thereof, and that no limitation of the invention is intended thereby. Various changes in form and detail, including simple deductions or substitutions, may be made by those skilled in the art without departing from the spirit and scope of the invention.

Claims (13)

1. The utility model provides a container terminal haulage vehicle positioning system based on binocular stereo vision which characterized in that includes:
the diamond-shaped mark is arranged on the ground;
the binocular stereo camera is arranged on the vehicle and comprises a first camera and a second camera, and the binocular stereo camera is used for shooting the diamond-shaped identification;
the measuring device is used for measuring the coordinates of the diamond-shaped mark, the vehicle and the binocular stereo camera under a world coordinate system;
and the calculating unit is in communication connection with the binocular stereo camera and the measuring device and is used for positioning the vehicle according to the information sent by the binocular stereo camera and the measuring device.
2. The locator system of claim 1, wherein the diamond shape identifies an apex angle of 60 degrees in the direction of vehicle travel.
3. The positioning system of claim 2, wherein the diamond markers have sides in the range of 50cm to 100 cm.
4. The positioning system of claim 1, wherein the vehicle is provided with an industrial computer, the industrial computer comprising the computing unit.
5. The positioning system of claim 4, wherein the vehicle is provided with an electrical cabinet, the industrial computer being disposed within the electrical cabinet.
6. The positioning system of claim 4, wherein the first camera and the second camera are each connected to the industrial computer by a USB serial data line.
7. The positioning system of claim 1, wherein the diamond shaped markings are white.
8. The positioning system of claim 1, wherein said measuring device comprises a total station and a calibration board.
9. A container terminal transport vehicle positioning method based on binocular stereo vision, which is applied to the positioning system according to any one of claims 1 to 8, and comprises the following steps:
the measuring device measures first world coordinates of the diamond-shaped mark under the world coordinate system;
the measuring device is used for carrying out external reference calibration on the binocular stereo camera to obtain a first conversion relation between a camera coordinate system and a vehicle body coordinate system;
the binocular stereo camera shoots to obtain an image of the diamond-shaped mark;
the computing unit locates the vehicle according to the first world coordinate, the first conversion relationship, and the image.
10. The method of claim 9, wherein the step of the computing unit locating the vehicle based on the first world coordinate, the first transformed relationship, and the image comprises:
the calculation unit obtains a first camera coordinate of the diamond-shaped mark in a camera coordinate system according to the image;
the calculation unit obtains a first vehicle body coordinate of the diamond-shaped mark under a vehicle body coordinate system according to the first camera coordinate and the first conversion relation;
the calculation unit obtains a second conversion relation between the vehicle body coordinate system and the world coordinate system according to the first world coordinate and the first vehicle body coordinate;
and the calculation unit positions the vehicle according to the second conversion relation to obtain the coordinates of the vehicle in the world coordinate system.
11. The positioning method of claim 10, further comprising the steps of:
and the calculation unit feeds back the three-dimensional pose information of the vehicle according to the coordinates of the vehicle in the world coordinate system and the second conversion relation.
12. The positioning method according to claim 10, wherein the camera coordinate system includes a first camera coordinate system and a second camera coordinate system, the first transformation relationship includes a first sub-transformation relationship and a second sub-transformation relationship, and the step of performing the external reference calibration on the binocular stereo camera by the measuring device to obtain the first transformation relationship between the camera coordinate system and the vehicle body coordinate system includes:
the measuring device measures second world coordinates and third world coordinates of the first camera and the second camera in the world coordinate system respectively;
the measuring device measures a fourth world coordinate of the vehicle under the world coordinate system;
and obtaining a first sub-conversion relation according to the second world coordinate and the fourth world coordinate, and obtaining a second sub-conversion relation according to the third world coordinate and the fourth world coordinate.
13. The positioning method according to claim 12, wherein said image comprises a first image and a second image, said first camera coordinates comprise a first camera sub-coordinate and a second camera sub-coordinate, and said step of said calculation unit deriving from said image said first camera coordinates of said diamond shape marker in a camera coordinate system comprises:
the calculation unit acquires the first image and the second image of the diamond-shaped mark shot by the first camera and the second camera;
the calculation unit obtains the first camera sub-coordinate of the diamond-shaped identifier under the first camera coordinate system and the second camera sub-coordinate of the diamond-shaped identifier under the second camera coordinate system according to the first image, the second image and the internal reference data of the binocular stereo camera.
CN202110229284.1A 2021-03-02 2021-03-02 Container terminal transport vehicle positioning system and method based on binocular stereo vision Pending CN113052914A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110229284.1A CN113052914A (en) 2021-03-02 2021-03-02 Container terminal transport vehicle positioning system and method based on binocular stereo vision

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110229284.1A CN113052914A (en) 2021-03-02 2021-03-02 Container terminal transport vehicle positioning system and method based on binocular stereo vision

Publications (1)

Publication Number Publication Date
CN113052914A true CN113052914A (en) 2021-06-29

Family

ID=76509753

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110229284.1A Pending CN113052914A (en) 2021-03-02 2021-03-02 Container terminal transport vehicle positioning system and method based on binocular stereo vision

Country Status (1)

Country Link
CN (1) CN113052914A (en)

Similar Documents

Publication Publication Date Title
CN108571971B (en) AGV visual positioning system and method
CN102944224B (en) Work method for automatic environmental perception systemfor remotely piloted vehicle
CA3027921C (en) Integrated sensor calibration in natural scenes
CN104197928B (en) Multi-camera collaboration-based method for detecting, positioning and tracking unmanned aerial vehicle
CN102967305B (en) Multi-rotor unmanned aerial vehicle pose acquisition method based on markers in shape of large and small square
CN104217439B (en) Indoor visual positioning system and method
JP2020525809A (en) System and method for updating high resolution maps based on binocular images
CN106645205A (en) Unmanned aerial vehicle bridge bottom surface crack detection method and system
CN108594245A (en) A kind of object movement monitoring system and method
CN109737981B (en) Unmanned vehicle target searching device and method based on multiple sensors
CN104200086A (en) Wide-baseline visible light camera pose estimation method
CN109490899A (en) Fire source localization method in a kind of tunnel based on laser radar and infrared thermal imager
CN108733039A (en) The method and apparatus of navigator fix in a kind of robot chamber
CN104990515A (en) Three-dimensional shape measurement system and method for large-size object
CN106019264A (en) Binocular vision based UAV (Unmanned Aerial Vehicle) danger vehicle distance identifying system and method
CN104315995A (en) TOF depth camera three-dimensional coordinate calibration device and method based on virtual multi-cube standard target
TWI754808B (en) Vehicle, vehicle positioning system, and vehicle positioning method
CN104316083A (en) Three-dimensional coordinate calibration device and method of TOF (Time-of-Flight) depth camera based on sphere center positioning of virtual multiple spheres
CN110017852B (en) Navigation positioning error measuring method
CN114413958A (en) Monocular vision distance and speed measurement method of unmanned logistics vehicle
JP2018084492A (en) Self-position estimation method and self-position estimation device
CN113028990A (en) Laser tracking attitude measurement system and method based on weighted least square
CN111724432B (en) Object three-dimensional detection method and device
CN214475141U (en) Container terminal transport vehicle positioning system based on binocular stereo vision
CN117310627A (en) Combined calibration method applied to vehicle-road collaborative road side sensing system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination