CN115937331A - Deep camera external parameter calibration method based on heavy truck battery automatic battery replacement system - Google Patents
Deep camera external parameter calibration method based on heavy truck battery automatic battery replacement system Download PDFInfo
- Publication number
- CN115937331A CN115937331A CN202310061734.XA CN202310061734A CN115937331A CN 115937331 A CN115937331 A CN 115937331A CN 202310061734 A CN202310061734 A CN 202310061734A CN 115937331 A CN115937331 A CN 115937331A
- Authority
- CN
- China
- Prior art keywords
- coordinate system
- battery
- lifting appliance
- camera
- point coordinates
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/60—Other road transportation technologies with climate change mitigation effect
- Y02T10/70—Energy storage systems for electromobility, e.g. batteries
Landscapes
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention provides a depth camera external reference calibration method based on a heavy truck battery automatic battery replacement system, which comprises the following steps: moving the lifting appliance to a position right above the battery pack to align the battery frame, so that the center of the lifting appliance is aligned with the center of the battery pack; after aligning, raising the position to a certain height, and then photographing the battery pack to obtain pixel point coordinates and corresponding depth image data of four key feature points of a battery frame on the image; performing inverse projection transformation by using the pixel point coordinates and the depth values of the feature points to obtain three-dimensional point coordinates of the feature points in a camera coordinate system; acquiring three-dimensional point coordinates of the four characteristic points in a hanger coordinate system; an iterative method is used to solve the translation and rotation transformation from the camera coordinate system to the spreader coordinate system. The method is simple to operate, external parameters can be calibrated only by 4 characteristic points of a single picture, and the method can help realize higher-precision offset calculation value during power replacement of a heavy truck.
Description
Technical Field
The invention relates to the technical field of new energy battery replacement, in particular to a deep camera external reference calibration method based on a heavy truck battery automatic battery replacement system.
Background
An automatic battery replacement system for heavy truck batteries is a top-hung battery replacement mode, and mainly comprises a heavy truck and a battery pack thereof, a movable lifting appliance, a depth camera installed below the lifting appliance and an edge controller as shown in fig. 1. In order to realize accurate positioning and automatic grabbing of the battery by the lifting appliance, the detection and positioning of the heavy-duty battery pack by the camera are required to be carried out so as to judge the moving direction and offset of the lifting appliance.
Because the installation position of the camera is difficult to ensure to be positioned at the center of the lifting appliance and perpendicular to the detection surface of the battery pack, in order to more accurately give the offset amount that the lifting appliance should move, the external reference of the camera, namely the transformation relation between the camera coordinate system and the lifting appliance coordinate system, including translation and rotation transformation, needs to be calibrated.
Disclosure of Invention
The invention aims to provide a depth camera external reference calibration method based on a heavy truck battery automatic battery replacement system, which overcomes the problems or at least partially solves the problems.
In order to achieve the purpose, the technical scheme of the invention is realized as follows:
the invention provides a depth camera external reference calibration method based on an automatic battery replacement system of a heavy truck battery, which comprises the following steps of:
(1) Moving the lifting appliance to the position right above the battery pack to align the battery frame, so that the center of the lifting appliance is aligned with the center of the battery pack, and the corresponding edges are parallel;
(2) After aligning, raising the position to a certain height, and then photographing the battery pack to obtain pixel point coordinates and corresponding depth image data of four key feature points of a battery frame on the image;
(3) Performing inverse projection transformation by using the pixel point coordinates and the depth values of the feature points to obtain three-dimensional point coordinates of the feature points in a camera coordinate system;
(4) Acquiring three-dimensional point coordinates of the four characteristic points in a hanger coordinate system by actually measuring the physical size of the battery;
(5) And solving the translation and rotation transformation from the camera coordinate system to the hanger coordinate system by using an iterative method according to the corresponding relation of the four characteristic points in the camera coordinate system and the hanger coordinate system.
As a further scheme of the present invention, in steps (1) and (2), the method for aligning the battery pack by the lifting appliance specifically comprises:
the method comprises the following steps that firstly, a lifting appliance is lowered in the Z-axis direction and falls above a battery pack, then the position of the lifting appliance is adjusted, and the lifting appliance and the battery pack are aligned;
secondly, ensuring that the center of the lifting appliance and the center of the battery are on the same vertical line and the corresponding edges are parallel by shifting the lifting appliance in the X-axis direction and the Y-axis direction and rotating the lifting appliance by the angle;
and step three, after alignment, lifting the lifting appliance to the initial height, only changing the position of the lifting appliance in the Z-axis direction during lifting, keeping the positions and the angles of the lifting appliance in the X-axis direction and the Y-axis direction unchanged, and then taking a picture of the battery pack.
As a further scheme of the present invention, in step (3), the coordinates of the pixel points of the feature points and the depth values thereof are used to identify four more obvious points on the battery pack on the color picture as four feature points, respectively.
As a further scheme of the invention, the method for identifying the pixel point coordinates adopts one of a deep neural network YOLO detection model and a method for obtaining the pixel point coordinates of four characteristic points in a mode of marking on an image.
As a further scheme of the present invention, in step (4), after obtaining the pixel point coordinates of the feature points and the depth values thereof, obtaining the three-dimensional point coordinates of the feature points in the camera coordinate system through the back projection transformation of the camera;
pixel point coordinates (u, v) of the feature point, depth value d, three-dimensional point coordinates (x) of the camera coordinate system c ,y c ,z c ) The camera's internal parameters include the image distance fx, fy and the offset cx, cy from the center;
the transformation relationship between them is as follows:
as a further aspect of the present invention, in step (4), the x axis and the y axis of the battery pack coordinate system coincide with the x axis and the y axis of the hanger coordinate system, the z axis is parallel, the line connecting the center points is parallel to the z axis, and the three-dimensional point coordinates of the feature points in the hanger coordinate system are obtained by measuring the physical distance of the main feature points on the battery frame.
As a further aspect of the present invention, the four sets of corresponding three-dimensional point pairs obtained through the above steps are respectively three-dimensional point coordinates of the feature points in the camera coordinate system
And the three-dimensional point coordinate under the lifting appliance coordinate system->
As a further aspect of the invention, the rotational and translational transformation of the camera coordinate system to the spreader coordinate system is solved using the Kabsch algorithm, which aims at minimizing the mean square error of the transformed point set P, i.e. P' and Q.
The invention provides a depth camera external reference calibration method based on a heavy truck battery automatic battery replacement system, which has the beneficial effects that:
the method for calibrating the external parameters of the depth camera is suitable for an automatic battery replacement scene of a heavy truck battery;
the calibration method is simple to operate, and the external parameter can be calibrated only by 4 characteristic points of a single picture;
the calibration method provided by the invention can help realize higher-precision offset calculation value when the electricity is reloaded.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the description of the embodiments are briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on the drawings without creative efforts.
Fig. 1 is an automatic battery replacement system for heavy truck batteries.
Fig. 2 is a schematic view of the alignment of the spreader and the battery pack of the present invention.
Fig. 3 is a feature point bitmap of a battery pack according to the present invention.
Fig. 4 is a flow chart of the operation of the present invention.
In the figure: 1. a suspended ceiling; 2. a wire rope; 3. a hanger; 4. a controller; 5. a camera; 6. a battery pack; 7. and (5) heavy truck.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
Referring to fig. 1 to 4, the method for calibrating external parameters of a depth camera based on an automatic battery replacement system for heavy trucks provided by the embodiment of the present invention includes the following steps:
(1) Moving the lifting appliance 3 to a position right above the battery pack 6 to align the battery frame, so that the center of the lifting appliance 3 is aligned with the center of the battery pack 6, and the corresponding edges are parallel;
(2) After the alignment, the image is raised to a certain height position and then photographed against the battery pack 6, and pixel point coordinates and corresponding depth image data of four key feature points of a battery frame on the image are obtained;
(3) Performing inverse projection transformation by using the pixel point coordinates and the depth values of the feature points to obtain three-dimensional point coordinates of the feature points in a camera 5 coordinate system;
(4) Acquiring three-dimensional point coordinates of the four characteristic points in a coordinate system of the lifting appliance 3 by actually measuring the physical size of the battery;
(5) And solving translation and rotation transformation from the camera 5 coordinate system to the hanger 3 coordinate system by using an iterative method according to the corresponding relation of the four characteristic points in the camera 5 coordinate system and the hanger 3 coordinate system.
1. Align battery pack 6 and spreader 3
Because the position of the lifting appliance 3 cannot be seen in the visual field photographed by the camera 5, the coordinate system of the lifting appliance 3 needs to be projected downwards onto the battery pack 6 so as to be detected by the camera 5, and the specific operation is to lower the lifting appliance 3 in the Z-axis direction and fall above the battery pack 6, then adjust the position of the lifting appliance 3, align the lifting appliance 3 and the battery pack 6, and ensure that the center of the lifting appliance 3 and the center of the battery are on the same vertical line and the corresponding edges are parallel by moving the offset of the lifting appliance 3 in the X-axis direction and the Y-axis direction and rotating the angle of the lifting appliance 3, as shown in fig. 2.
After alignment, the spreader 3 is raised to the initial height, only the position in the Z-axis direction is changed during the raising, the positions and angles of the spreader 3 in the X-axis direction and the Y-axis direction are kept unchanged, and then a picture is taken of the battery.
2. Obtaining the pixel point coordinates and the depth of the characteristic points
After aligning the battery pack 6 and the lifting appliance 3, four obvious characteristic points on the battery pack 6 are identified on the color picture, as indicated by the marked points in fig. 3, and the identification method can use a detection model such as a deep neural network YOLO and the like or directly obtain the pixel point coordinates of the four characteristic points in a mode of marking on the image.
After the depth camera 5 acquires the depth map, the depth image is converted into a color image coordinate system through the conversion relation between the depth camera 5 and the color camera 5, and then the depth value of the point can be directly acquired from the depth map through the pixel point coordinates of the feature point.
3. Obtaining three-dimensional point coordinates of the feature points in a camera 5 coordinate system
After the pixel point coordinates and the depth value of the feature point are obtained, the three-dimensional point coordinates of the feature point in the coordinate system of the camera 5, the pixel point coordinates (u, v) of the feature point, the depth value d, and the three-dimensional point coordinates (x) of the coordinate system of the camera 5 can be obtained through the back projection transformation of the camera 5 c ,y c ,z c ) The internal reference of the camera 5 includes the image distances fx, fy and the offsets cx, cy from the center, unit pixels, and the transformation relationship between them is as follows:
4. obtaining three-dimensional point coordinates of the characteristic points under the lifting appliance 3 coordinate system
Since the x-axis and the y-axis of the coordinate system of the battery pack 6 coincide with the x-axis and the y-axis of the coordinate system of the hanger 3 through the alignment operation of the first step, the z-axis is parallel and the connection line of the center point is parallel to the z-axis, three-dimensional point coordinates of the feature point in the coordinate system of the hanger 3 can be obtained by measuring the physical distances of the main feature points on the battery frame as shown by w and h in fig. 3.
The three-dimensional point coordinates of the four feature points as shown in FIG. 3 are (-w/2,h/2,0), (w/2,h/2,0), (w/2, -h/2,0), (-w/2, -h/2,0), respectively (arranged clockwise from the upper left corner).
5. Solving the external parameter transformation
Four groups of corresponding three-dimensional point pairs are obtained through the steps, and the three-dimensional point pairs are respectively three-dimensional point coordinates of the feature points in the camera 5 coordinate system
And three-dimensional point coordinates under the coordinate system of the lifting appliance 3
We solve the rotation and translation transformation of the camera 5 coordinate system to the spreader 3 coordinate system using the Kabsch algorithm, which aims to minimize the mean square error of the transformed point set P, i.e. P' and Q.
The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement or the like made within the spirit and principle of the present application shall be included in the scope of the claims of the present application.
Claims (8)
1. The depth camera external reference calibration method based on the heavy truck battery automatic battery replacement system is characterized by comprising the following steps of:
(1) Moving the lifting appliance (3) to be right above the battery pack (6) to align the battery frame, so that the center of the lifting appliance (3) is aligned with the center of the battery pack (6), and the corresponding edges are parallel;
(2) After being aligned, the image is raised to a certain height position and then photographed against a battery pack (6), and pixel point coordinates and corresponding depth image data of four key feature points of a battery frame on the image are obtained;
(3) Performing inverse projection transformation by using the pixel point coordinates and the depth values of the feature points to obtain three-dimensional point coordinates of the feature points in a camera (5) coordinate system;
(4) Three-dimensional point coordinates of the four characteristic points under a coordinate system of the lifting appliance (3) are obtained through actually measuring the physical size of the battery;
(5) And solving the translation and rotation transformation from the camera (5) coordinate system to the hanger (3) coordinate system by using an iterative method through the corresponding relation of the four characteristic points under the camera (5) coordinate system and the hanger (3) coordinate system.
2. The deep camera external reference calibration method based on the heavy truck battery automatic battery replacement system according to claim 1, wherein in the steps (1) and (2), the method for aligning the hanger (3) with the battery pack (6) specifically comprises the following steps:
the method comprises the following steps that firstly, a lifting appliance (3) is lowered in the Z-axis direction and falls above a battery pack (6), then the position of the lifting appliance (3) is adjusted, and the lifting appliance (3) and the battery pack (6) are aligned;
secondly, ensuring that the center of the lifting appliance (3) and the center of the battery are on the same vertical line and the corresponding edges are parallel by shifting the lifting appliance (3) in the X-axis direction and the Y-axis direction and rotating the lifting appliance (3) by the angle;
and step three, after alignment, lifting the lifting appliance (3) to the initial height, only changing the position of the Z-axis direction during lifting, keeping the position and the angle of the lifting appliance (3) in the X-axis direction and the Y-axis direction unchanged, and then taking a picture of the battery pack (6).
3. The depth camera external reference calibration method based on the heavy-duty battery automatic battery replacement system as claimed in claim 1, wherein in step (3), four points which are more obvious on the battery pack (6) are identified on the color picture as four feature points respectively by using pixel point coordinates of the feature points and depth values thereof.
4. The deep camera external reference calibration method based on the heavy-duty battery automatic battery replacement system as claimed in claim 3, wherein the method for identifying the pixel coordinates adopts one of a deep neural network YOLO detection model and a method for obtaining the pixel coordinates of four feature points in a manner of labeling on an image.
5. The depth camera external reference calibration method based on the heavy-duty battery automatic battery replacement system according to claim 1, wherein in step (4), after the pixel coordinates of the feature points and the depth values thereof are obtained, three-dimensional point coordinates of the feature points in a camera (5) coordinate system are obtained through back projection transformation of the camera (5);
pixel point coordinates (u, v) of feature points, depth value d, three-dimensional point coordinates (x) of camera (5) coordinate system c ,y c ,z c ) The camera (5) references include image distances fx, fy and offsets cx, cy from the center;
the transformation relationship between them is as follows:
6. the depth camera external reference calibration method based on the heavy truck battery automatic battery replacement system is characterized in that in the step (4), the x axis and the y axis of the coordinate system of the battery pack (6) coincide with the x axis and the y axis of the coordinate system of the lifting appliance (3), the z axis is parallel, the connecting line of the central points is parallel to the z axis, and the three-dimensional point coordinates of the characteristic points under the coordinate system of the lifting appliance (3) are obtained by measuring the physical distance of the main characteristic points on the battery frame.
7. The depth camera external reference calibration method based on the heavy-duty battery automatic battery replacement system as claimed in claim 6, wherein the four sets of corresponding three-dimensional point pairs are obtained through the above steps, and are respectively three-dimensional point coordinates of feature points in a camera (5) coordinate system
And three-dimensional point coordinates under the coordinate system of the lifting appliance (3)
8. The depth camera external reference calibration method based on the heavy truck battery automatic battery replacement system according to claim 7, characterized in that the rotation and translation transformation from the camera (5) coordinate system to the hanger (3) coordinate system is solved using a Kabsch algorithm, which aims at minimizing the mean square error of the transformed point set P, i.e. P' and Q.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310061734.XA CN115937331A (en) | 2023-01-13 | 2023-01-13 | Deep camera external parameter calibration method based on heavy truck battery automatic battery replacement system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310061734.XA CN115937331A (en) | 2023-01-13 | 2023-01-13 | Deep camera external parameter calibration method based on heavy truck battery automatic battery replacement system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115937331A true CN115937331A (en) | 2023-04-07 |
Family
ID=86651033
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310061734.XA Pending CN115937331A (en) | 2023-01-13 | 2023-01-13 | Deep camera external parameter calibration method based on heavy truck battery automatic battery replacement system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115937331A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116620098A (en) * | 2023-07-25 | 2023-08-22 | 北京玖行智研交通科技有限公司 | Method for replacing battery box of electric vehicle in power exchange station |
-
2023
- 2023-01-13 CN CN202310061734.XA patent/CN115937331A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116620098A (en) * | 2023-07-25 | 2023-08-22 | 北京玖行智研交通科技有限公司 | Method for replacing battery box of electric vehicle in power exchange station |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104200086B (en) | Wide-baseline visible light camera pose estimation method | |
| CN111473739B (en) | Video monitoring-based surrounding rock deformation real-time monitoring method for tunnel collapse area | |
| CN107462213B (en) | Iron tower inclination angle automatic measurement method based on machine vision system | |
| CN111935412A (en) | Method, system and robot for automatically identifying and tracking inspection target | |
| CN112141894B (en) | Box grabbing alignment method based on fusion of multiple 2D measuring devices | |
| WO2010009570A1 (en) | A hoist-positioning method and intelligent vision hoisting system | |
| CN110595476A (en) | Unmanned aerial vehicle landing navigation method and device based on GPS and image visual fusion | |
| WO2022134490A1 (en) | Gimbal target conversion control method, apparatus, device, and storage medium | |
| CN115937331A (en) | Deep camera external parameter calibration method based on heavy truck battery automatic battery replacement system | |
| WO2024011981A1 (en) | Method for verifying light spot of cylindrical target | |
| CN113043334B (en) | Robot-based photovoltaic cell string positioning method | |
| CN113465522A (en) | Point cloud-based method for measuring distance of hidden danger in power transmission channel | |
| CN108897246B (en) | Stack box control method, device, system and medium | |
| CN115880372A (en) | Unified calibration method and system for external hub positioning camera of automatic crane | |
| CN113879984A (en) | Automatic container grabbing and releasing system based on machine vision and operation method | |
| CN114890280A (en) | Detection alignment method and device for lifting appliance | |
| CN109059769B (en) | Non-contact pantograph lifting pantograph arm rod position relation measuring method | |
| CN110543612B (en) | Card collection positioning method based on monocular vision measurement | |
| CN111372044A (en) | System, method and device for inspecting transmission line in goaf | |
| CN114897981A (en) | Hanger pose identification method based on visual detection | |
| CN111854678B (en) | Pose measurement method based on semantic segmentation and Kalman filtering under monocular vision | |
| CN110717061B (en) | Transformer substation equipment positioning method and system based on camera and associated mapping | |
| CN207198665U (en) | Terrestrial positioning regulation device | |
| CN218847192U (en) | Roadbed filling height self-sensing device and system | |
| CN216006629U (en) | Intelligent mounting system for assembled bridge |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |