CN114596264A - Method, device and system for auxiliary docking, docking method and engineering equipment - Google Patents

Method, device and system for auxiliary docking, docking method and engineering equipment Download PDF

Info

Publication number
CN114596264A
CN114596264A CN202210116783.4A CN202210116783A CN114596264A CN 114596264 A CN114596264 A CN 114596264A CN 202210116783 A CN202210116783 A CN 202210116783A CN 114596264 A CN114596264 A CN 114596264A
Authority
CN
China
Prior art keywords
central axis
slope
coordinate
determining
tangential line
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210116783.4A
Other languages
Chinese (zh)
Inventor
丰生日
李淇阳
高荣芝
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zoomlion Heavy Industry Science and Technology Co Ltd
Original Assignee
Zoomlion Heavy Industry Science and Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zoomlion Heavy Industry Science and Technology Co Ltd filed Critical Zoomlion Heavy Industry Science and Technology Co Ltd
Priority to CN202210116783.4A priority Critical patent/CN114596264A/en
Publication of CN114596264A publication Critical patent/CN114596264A/en
Priority to PCT/CN2022/139042 priority patent/WO2023147743A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G06T3/4007—Interpolation-based scaling, e.g. bilinear interpolation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30164—Workpiece; Machine component

Abstract

The invention relates to the field of engineering equipment, and discloses a method, a device and a system for auxiliary docking, a docking method and engineering equipment, wherein the method for auxiliary docking comprises the following steps: acquiring images of a first object and a second object to be butted; determining a first central axis of the first object along a docking direction and a second central axis of the second object along the docking direction based on the images; determining a tangential line of any of the first object and the second object that is perpendicular to the docking direction; and labeling the first central axis, the second central axis and the tangential line in the image to assist in docking of the first object and the second object. Therefore, the current postures of the first object and the second object are considered in auxiliary docking, and the marked first central axis, second central axis and tangential line can adapt to different postures of the first object and the second object.

Description

Method, device and system for auxiliary docking, docking method and engineering equipment
Technical Field
The invention relates to the field of engineering equipment, in particular to a method, a device and a system for auxiliary docking, a docking method and engineering equipment.
Background
In the field of engineering equipment, vehicle-mounted pumping equipment needs to butt joint a vehicle-mounted material pipeline with other conveying pipelines before conveying materials to a target position area. Because the pipelines for conveying materials are rigid structures, the pipe orifices of the two pipelines must be aligned for the next material conveying. The conventional rear-view reflector is still adopted for pipeline butt joint at the present stage, or other commanders are needed to cooperate to complete the pipeline butt joint, so that the butt joint process is time-consuming.
In order to improve the butt joint efficiency of the material equipment pipeline, an auxiliary butt joint method is provided in the prior art, and a pre-known calibration identifier is loaded according to the position of the equipment pipeline in an image and the steering information of a vehicle, so that a manipulator is assisted to complete pipe joint operation. But no mention is made of the relevant method when the cone is in any attitude. Fig. 1, 2, and 3 are main diagrams of the auxiliary docking method. FIG. 1 is a schematic diagram of a vehicle-mounted pump with a taper pipe and a delivery pipe in butt joint, wherein in actual engineering operation, the taper pipe and the delivery pipe need to be aligned to complete material delivery; FIG. 2 is a schematic view of auxiliary lines when the taper pipe and the delivery pipe are parallel, showing the auxiliary butt lines with the taper pipe mouth as the base point when the taper pipe and the delivery pipe are radially parallel; fig. 3 is a schematic diagram of an auxiliary line when the conical pipe is not parallel to the conveying pipe, when the conveying pipe is not parallel to the conical pipe in the radial direction, the far end of the auxiliary butt line displayed on the display screen marks the steering information of the current vehicle, and the steering auxiliary line is synchronously corrected along with the adjustment of the vehicle posture. After the camera of the auxiliary geminate transistor is fixedly installed on the vehicle, the whole equipment needs to be calibrated. The loaded predicted calibration identification information in the auxiliary docking method is calibrated based on the situation that the attitude of the taper pipe and the vehicle are kept unchanged. However, in actual engineering work, the taper pipe needs to be adjusted to rotate up and down according to the inclined posture of the conveying pipe. As can be seen from fig. 4, the calibration information on the image is different due to different rotation postures and different positions of the pipe orifice surface of the cone. Therefore, the calibration identification information provided in the auxiliary docking method is difficult to adapt to different posture conditions of the taper pipe.
Disclosure of Invention
It is an object of the present invention to provide a method, a device and a system for assisted docking, a docking method and an engineering arrangement, which solve or at least partly solve the above mentioned problems.
To achieve the above object, an aspect of the present invention provides a method for assisting docking, the method comprising: acquiring images of a first object and a second object to be butted; determining a first central axis of the first object along a docking direction and a second central axis of the second object along the docking direction based on the images; determining a tangential line of any of the first object and the second object that is perpendicular to the docking direction; and labeling the first central axis, the second central axis and the tangential line in the image to assist in docking of the first object and the second object.
Optionally, determining the first central axis and/or the second central axis based on the image comprises: determining a first coverage area of the first object in the image and/or a second coverage area of the second object in the image; determining a first skeleton line of the first coverage area and/or a second skeleton line of the second coverage area; and fitting the first skeleton lines into the first central axis and/or fitting the second skeleton lines into the second central axis by using a curve fitting method.
Optionally, the determining a first coverage area of the first object in the image and/or a second coverage area of the second object in the image comprises: determining the first coverage area and/or the second coverage area based on a preset convolutional neural network.
Optionally, the determining a tangential line of any one of the first object and the second object perpendicular to the docking direction comprises: determining a centerline slope for said any one; and determining the tangential line of any one of the lines based on the determined central axis slope and a preset lookup table corresponding to the any one of the lines, wherein the preset lookup table comprises a corresponding relation between the central axis slope and relevant information of the tangential line.
Optionally, the determining the tangent line of any one of the one or more sensors based on the determined mid-axis slope and a predetermined lookup table corresponding to the one or more sensors comprises: under the condition that the determined central axis slope exists in the preset lookup table, finding the first coordinate and the second coordinate corresponding to the determined central axis slope in the preset lookup table; and determining the tangent line of said either one from the found first and second coordinates; and/or under the condition that the determined central axis slope does not exist in the preset lookup table, finding a first approximate central axis slope and a second approximate central axis slope which are closest to the determined central axis slope in the preset lookup table; finding the first coordinate and the second coordinate corresponding to the first approximate central axis slope and the first coordinate and the second coordinate corresponding to the second approximate central axis slope in the preset lookup table; determining the first coordinate and the second coordinate corresponding to the determined central axis slope based on the first coordinate and the second coordinate corresponding to the first approximate central axis slope and the first coordinate and the second coordinate corresponding to the second approximate central axis slope; and determining the tangential line of either one based on the first coordinate and the second coordinate corresponding to the determined mid-axis slope.
Optionally, the tangential line is located at a butt end of either.
Optionally, the either one is one of the first object and the second object whose posture changes less.
Accordingly, another aspect of the present invention provides an apparatus for assisting docking, the apparatus comprising: the image acquisition module is used for acquiring images of a first object and a second object to be butted; a central axis determining module, configured to determine, based on the image, a first central axis of the first object along a docking direction and a second central axis of the second object along the docking direction; a tangential line determination module to determine a tangential line of any one of the first object and the second object that is perpendicular to the docking direction; and the labeling module is used for labeling the first central axis, the second central axis and the tangential line in the image so as to assist the butt joint of the first object and the second object.
Optionally, the central axis determination module determining the first central axis and/or the second central axis based on the image comprises: determining a first coverage area of the first object in the image and/or a second coverage area of the second object in the image; determining a first skeleton line of the first coverage area and/or a second skeleton line of the second coverage area; and fitting the first skeleton line to the first central axis and/or fitting the second skeleton line to the second central axis by using a curve fitting method.
Optionally, the determining a first coverage area of the first object in the image and/or a second coverage area of the second object in the image comprises: determining the first coverage area and/or the second coverage area based on a preset convolutional neural network.
Optionally, the tangential line determination module determining a tangential line of any one of the first object and the second object perpendicular to the docking direction comprises: determining a centerline slope for said any one; and determining the tangential line of any one of the lines based on the determined central axis slope and a preset lookup table corresponding to the any one of the lines, wherein the preset lookup table comprises a corresponding relation between the central axis slope and relevant information of the tangential line.
Optionally, the determining the tangent line of any one of the one or more sensors based on the determined mid-axis slope and a predetermined lookup table corresponding to the one or more sensors comprises: under the condition that the determined central axis slope exists in the preset lookup table, finding the first coordinate and the second coordinate corresponding to the determined central axis slope in the preset lookup table; and determining the tangent line of said either one from the found first and second coordinates; and/or under the condition that the determined central axis slope does not exist in the preset lookup table, finding a first approximate central axis slope and a second approximate central axis slope which are closest to the determined central axis slope in the preset lookup table; finding the first coordinate and the second coordinate corresponding to the first approximate central axis slope and the first coordinate and the second coordinate corresponding to the second approximate central axis slope in the preset lookup table; determining the first coordinate and the second coordinate corresponding to the determined central axis slope based on the first coordinate and the second coordinate corresponding to the first approximate central axis slope and the first coordinate and the second coordinate corresponding to the second approximate central axis slope; and determining the tangential line of either one based on the first coordinate and the second coordinate corresponding to the determined mid-axis slope.
Optionally, the tangential line is located at a butt end of either.
Optionally, the either one is one of the first object and the second object whose posture is less changed.
In addition, another aspect of the present invention also provides a docking method, including: and controlling the docking of the first object and the second object according to the first central axis, the second central axis and the tangential line marked by the method for assisting docking.
In addition, another aspect of the present invention also provides a system for assisting docking, the system comprising: the above-mentioned means for assisting docking; and a display module for displaying the image labeled with the first central axis, the second central axis and the tangential line.
Optionally, the system further comprises: the camera module is used for acquiring videos of a first object and a second object; and a decomposition module for decomposing the acquired video into images.
In addition, the invention also provides engineering equipment, and the engineering equipment comprises the system for assisting the docking.
In addition, another aspect of the present invention also provides a machine-readable storage medium having instructions stored thereon for causing a machine to perform the method for assisted docking or the docking method described above.
According to the technical scheme, the first central axis, the second central axis and the tangential line are determined based on the acquired image, the determined first central axis, second central axis and tangential line are marked in the image for auxiliary butt joint, the first central axis, second central axis and tangential line marked in the image are not preset and are based on the image acquired in real time, and therefore the current postures of the first object and the second object are considered during auxiliary butt joint; according to the current postures of the first object and the second object, a first central axis, a second central axis and a tangential line for assisting in butt joint are marked, and the first central axis, the second central axis and the tangential line marked when the postures are changed are also changed, so that the marked first central axis, second central axis and tangential line can adapt to different postures of the first object and the second object. In addition, the posture of the first object and the posture of the second object are considered during auxiliary docking, so that the docking process is more accurate, and the auxiliary docking effect is better.
Additional features and advantages of the invention will be set forth in the detailed description which follows.
Drawings
The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention and not to limit the invention. In the drawings:
FIG. 1 is a schematic illustration of a prior art docking of a vehicular pump cone and delivery tube;
FIG. 2 is a schematic view of an auxiliary line when a conical tube and a delivery tube are parallel in the prior art;
FIG. 3 is a schematic view of an auxiliary line when a cone and a delivery pipe are not parallel in the prior art;
FIG. 4 is a schematic view of different orientations of the cone;
FIG. 5 is a flow chart of a method for assisting docking provided by an embodiment of the present invention;
FIG. 6 is a schematic illustration of a central axis provided by another embodiment of the present invention;
FIG. 7 is a schematic view of the engagement of the cone and the carrier tube in the presence of a radial offset according to another embodiment of the present invention;
FIG. 8 is a schematic view of two pipes to be butted together according to another embodiment of the present invention;
FIG. 9 is a schematic illustration of the footprint of the two ducts shown in FIG. 8 according to another embodiment of the present invention;
FIG. 10 is a schematic view of the installation of the device in an auxiliary docking provided by another embodiment of the present invention;
FIG. 11 is a schematic view of a central axis and a line tangential to the nozzle provided by another embodiment of the present invention;
FIG. 12 is a top view of a vertebral canal with a calibration tool installed according to another embodiment of the present invention;
FIG. 13 is an elevation view of a spinal canal with a calibration tool installed according to another embodiment of the invention;
FIG. 14 is a schematic view of a spinal canal alignment fixture provided in accordance with another embodiment of the present invention;
FIG. 15 is a schematic view of a spinal canal provided in accordance with another embodiment of the invention in a different position;
FIG. 16 is a flow chart of a method for assisted docking provided by another embodiment of the present invention; and
fig. 17 is a block diagram of an apparatus for assisting docking according to another embodiment of the present invention.
Description of the reference numerals
1 conveying pipe 2 taper pipe
3 camera 4 display
5 vehicle body 6 image acquisition module
7 central axis determining module and 8 tangential line determining module
9 annotating module
Detailed Description
The following detailed description of embodiments of the invention refers to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the preferred embodiment of the invention, are given by way of illustration and explanation only, not limitation.
One aspect of embodiments of the present invention provides a method for assisted docking.
Fig. 5 is a flowchart of a method for assisting in docking according to an embodiment of the present invention. As shown in fig. 5, the method includes the following.
In step S50, images of the first object and the second object to be docked are acquired. Specifically, images of the first object and the second object may be directly captured, for example, captured using a camera; the images of the first object and the second object may also be obtained by capturing videos of the first object and the second object, for example, by using a camera, and performing framing processing on the captured videos. In any manner, the images of the first object and the second object may be acquired.
In step S51, a first central axis of the first object in the docking direction and a second central axis of the second object in the docking direction are determined based on the images. Wherein the docking direction is a direction along which the first object and the second object are docked together. As shown in FIG. 6, a delivery pipe 1 and a taper pipe 2 of the vehicle-mounted pump are butted, the two pipelines are butted along the transverse direction of the pipelines, the butting direction is the transverse direction, and the central axes of the two pipelines are shown by dotted lines in FIG. 6.
In step S52, a tangential line perpendicular to the docking direction of either of the first object and the second object is determined. The tangential line may help to determine the separation of the first object and the second object in a direction perpendicular to the docking direction. As shown in fig. 7, the objects to be butted are a conveying pipe 1 and a conical pipe 2 of a vehicle-mounted pump, the two pipes are butted along the transverse direction of the pipes, the butting direction is transverse, the direction perpendicular to the butting direction is radial, and the tangential line helps to judge the distance between the conveying pipe 1 and the conical pipe 2 in the radial direction. The docking can be better assisted by helping to judge the distance between the first object and the second object in the direction perpendicular to the docking direction. For example, the slope of the central axis of any one of the central axes is determined, wherein the slope of the central axis is the slope of a straight line on which the central axis of any one of the central axes is located. Specifically, a coordinate system may be established, any two points on the central axis may be selected, and the slope of the central axis may be determined based on the two selected points. And determining any tangential line based on the determined central axis slope and any preset lookup table corresponding to the central axis slope, wherein the preset lookup table comprises the corresponding relation between the central axis slope and the tangential line related information. In addition, the first object and the second object may be distinguished based on characteristics of the first object and the second object. For example, during the process of docking the first object and the second object, the first object is still relative to the camera device, the second object moves, and by comparing two adjacent pictures, the first object is determined as the first object without change, and the second object is determined as the second object with change. In particular, it may be compared whether there is a change in the first object and the second object by distinguishing the coverage areas of the first object and the second object, respectively, in the image. And after distinguishing the first object from the second object, determining the tangential line of any one according to the selected preset lookup table corresponding to any one. In addition, the tangential line related information is used to determine the tangential line, for example, the tangential line related information may include coordinates of any two points, or may include a slope and coordinates of one point, and so on. Specifically, the relevant information of the tangential line corresponding to the determined slope of the central axis is determined based on a preset lookup table, and the tangential line of any one of the determined tangential lines is determined according to the determined tangential line information. In addition, the method for establishing the coordinate system when determining the slope of the central axis of any one is the same as the method for establishing the coordinate system when determining the slope of the central axis in the preset lookup table.
In step S53, the first central axis, the second central axis and the tangential line are labeled in the image to assist the docking of the first object and the second object.
According to the technical scheme, the first central axis, the second central axis and the tangential line are determined based on the acquired image, the determined first central axis, second central axis and tangential line are marked in the image for auxiliary butt joint, the first central axis, second central axis and tangential line marked in the image are not preset and are based on the image acquired in real time, and therefore the current postures of the first object and the second object are considered during auxiliary butt joint; according to the current postures of the first object and the second object, a first central axis, a second central axis and a tangential line for assisting in butt joint are marked, and the first central axis, the second central axis and the tangential line marked when the postures are changed are also changed, so that the marked first central axis, second central axis and tangential line can adapt to different postures of the first object and the second object. In addition, the posture of the first object and the posture of the second object are considered during auxiliary docking, so that the docking process is more accurate, and the auxiliary docking effect is better.
There are many ways to determine the first central axis of the first object and/or the second central axis of the second object. Optionally, in an embodiment of the present invention, the first central axis and/or the second central axis may be determined according to the following. A first coverage area of the first object in the image and/or a second coverage area of the second object in the image is determined. As shown in fig. 8, the first object and the second object are two pipes, a delivery pipe 1 and a taper pipe 2; the coverage areas of the two ducts in the image are determined, as shown in white in fig. 9, respectively, the coverage area of the conveying pipe 1 and the coverage area of the cone 2. In the embodiment of the present invention, the first coverage area and/or the second coverage area may be determined by using a preset convolutional neural network. The preset convolutional neural network is a trained convolutional neural network, and the first coverage area and/or the second coverage area can be obtained by inputting the images of the first object and the second object into the preset convolutional neural network. Furthermore, the first coverage area and/or the second coverage area may be determined using image processing algorithms, for example, in combination with computer vision algorithms such as thresholding, watershed, edge extraction, morphological operations, and the like. A first skeleton line of the first coverage area and/or a second skeleton line of the second coverage area are determined. Alternatively, in the embodiment of the present invention, there are many ways to determine the skeleton line, for example, by combining the distance transformation method and the threshold segmentation method to calculate the skeleton line; or, a zhang-suen thinning algorithm can be adopted to extract the skeleton line. And fitting the first skeleton line to the first central axis and/or fitting the second skeleton line to the second central axis by using a curve fitting method. In addition, in the embodiment of the present invention, the image including the coverage area may be a binarized image, or may be an image of another color as long as the coverage area can be distinguished from the non-coverage area.
In addition, in the embodiment of the present invention, the convolutional neural network may be trained according to the following contents to obtain the preset convolutional neural network. 1) Video data of the first object and the second object in the docking process is recorded, for example, by using a camera. For example, the taper pipe and the delivery pipe of the vehicle-mounted pump are butted, video data of the pipe-to-pipe process of the taper pipe and the delivery pipe are recorded, a camera is used for recording, as shown in fig. 10, a camera 3 is installed on a vehicle body 5 of the vehicle-mounted pump, and the process of butting the delivery pipe 1 and the taper pipe 2 is recorded. In addition, when video data in the process of docking the taper pipe and the conveying pipe are recorded, different working conditions can be fully considered, and the video recording is carried out under different working conditions, so that data for training the convolutional neural network are enriched, the processing of the preset neural network is more accurate, and the accuracy of the method for assisting the docking provided by the embodiment of the invention is improved. 2) And (4) screening data. Firstly, the video recorded in the step 1) is subjected to framing processing to obtain images of a first object and a second object. In addition, in order to make the images obtained by framing more effective for training the convolutional neural network and improve the training speed, effective images can be selected from the images obtained by framing for training the convolutional neural network, specifically, the principle of selecting the effective images is that the selected images are different images, and the positions and angles of the first object and/or the second object in the selected images are different. And finally collecting images including the first object and the second object according to the preset value, for example, collecting 10000 images, wherein the larger the number of the collected images is, the better the training effect of the convolutional neural network is. 3) And (6) data annotation. And performing pixel-level semantic annotation on the screened images of the first object and the second object. In the embodiment of the present invention, performing pixel-level semantic annotation is to determine the coverage areas of the first object and the second object. For example, the taper pipe and the delivery pipe of the vehicle-mounted pump are butted, as shown in fig. 8 and 9, pixel level semantic labeling is carried out from fig. 8 to 9, namely coverage areas of the delivery pipe 1 and the taper pipe 2 are determined. 4) And (5) training a model. And dividing the labeled data set into a training set and a test set, and sending images of the training set into a semantic segmentation model based on a convolutional neural network for training. The annotated data set comprises original images of the first object and the second object and annotated images corresponding to the original images, wherein the coverage areas of the first object and the second object are determined, and one original image corresponds to the annotated image. For example, taking the delivery tube and the cone of a vehicle pump docked for example, the annotated data set includes raw images of the delivery tube and the cone (as shown in FIG. 8) and annotated images that define the coverage areas of the delivery tube and the cone (as shown in FIG. 9). The test set is used for testing the trained convolutional neural network. And after the test is finished, obtaining the trained convolutional neural network.
Optionally, in an embodiment of the present invention, the tangential line is located at a fixed position of any one of the two, the tangential line related information includes a first coordinate of a first point and a second coordinate of a second point located on the tangential line, a distance from the first point to the fixed position is a first preset distance, and a distance from the second point to the fixed position is a second preset distance, and determining the tangential line of any one of the two based on the determined slope of the central axis and the preset lookup table may include the following. And comparing the determined central axis slope of any one of the first object and the second object with the central axis slope included in the preset lookup table to judge whether the determined central axis slope of any one of the first object and the second object exists in the preset lookup table. And under the condition that the slope of the central axis of any one of the determined central axes exists in the preset lookup table, finding a first coordinate and a second coordinate corresponding to the slope of the central axis of any one of the determined central axes in the preset lookup table, and further determining a tangential line of any one of the determined central axes according to the found first coordinate and the found second coordinate. And/or, in the case that the determined center axis slope of any one of the center axis slopes is not present in the preset lookup table, finding a first approximate center axis slope and a second approximate center axis slope that are closest to the determined center axis slope of any one of the center axis slopes in the preset lookup table. And finding a first coordinate and a second coordinate corresponding to the slope of the first approximate central axis and a first coordinate and a second coordinate corresponding to the slope of the second approximate central axis in a preset lookup table. And determining a first coordinate and a second coordinate corresponding to the determined central axis slope of any one of the determined central axis slopes based on the first coordinate and the second coordinate corresponding to the first approximate central axis slope and the first coordinate and the second coordinate corresponding to the second approximate central axis slope. Specifically, interpolation may be used to determine the first coordinate and the second coordinate corresponding to the determined center axis slope of any one of the center axis slopes. For example, the midpoint coordinate of the first coordinate corresponding to the first central axis slope and the midpoint coordinate of the first coordinate corresponding to the second central axis slope are determined by an interpolation method as the first coordinate corresponding to the central axis slope of any one of the determined central axis slopes, the midpoint coordinate of the second coordinate corresponding to the first central axis slope and the midpoint coordinate of the second coordinate corresponding to the second central axis slope are determined by the interpolation method as the second coordinate corresponding to the central axis slope of any one of the determined central axis slopes, and thus, the first coordinate and the second coordinate corresponding to the central axis slope of any one of the determined central axis slopes are determined. Determining a tangential line of any one of the determined first and second coordinates corresponding to the slope of the central axis of any one of the determined tangential lines.
Optionally, in an embodiment of the invention, the tangential line is located at the butt end of either. Wherein the docking end is an end of either one that comes into contact with the other one of the first object and the second object when docking with the other one. As shown in fig. 10, a delivery pipe 1 and a conical pipe 2, either of which is the conical pipe 2, are butted. When the conveying pipe 1 is in butt joint with the taper pipe 2, the end A of the taper pipe 2 is in contact with the conveying pipe 1, and the end A is the butt joint end. When the tangential line is positioned at the butt joint end, the distance between the butt joint end of any one object and the butt joint end of the other object can be marked more intuitively, and the butt joint of the first object and the second object is convenient to control. As shown in FIG. 10, the tangential line is located at the end A of the taper pipe 2, and the distance between the end A of the taper pipe 2 and the end B of the conveying pipe 1 can be visually marked, wherein the end B is the butt joint end of the conveying pipe 1.
Optionally, in an embodiment of the present invention, either one of the first object and the second object has a smaller change in posture. For example, a delivery pipe and a taper pipe of a vehicle pump are to be butted, the posture of the taper pipe is less changed, and either one of the taper pipe and the delivery pipe may be the taper pipe. The tangential line can embody the posture of any one, and the butt joint is carried out according to the posture of any one during the butt joint, and when the tangential line is one of the first object and the second object with less posture change, the butt joint can be realized as soon as possible under the condition of less posture change. In addition, if any one of the first object and the second object has less posture change, the data amount in the preset lookup table including the corresponding relationship between the slope of the central axis of any one of the first object and the second object and the first coordinate and the second coordinate is less, so that the data processing is facilitated, and the processing speed is increased.
Optionally, in the embodiment of the present invention, the preset lookup table may be established according to the following contents. The first object corresponds to a preset lookup table, the second object corresponds to a preset lookup table, and the following description will take the preset lookup table corresponding to the first object as an example, and the preset lookup table corresponding to the second object can be established by referring to the following contents. And installing a calibration tool on the first object, wherein the calibration tool is used for representing a tangential line and comprises a straight rod part, and the straight rod part is used for marking the tangential line. For example, the first object is a cone, with the tangential lines at the butt ends of the cone, as shown in FIG. 10. In addition, as shown in fig. 10, in actual use, one end of the taper pipe is connected with the vehicle body 5, the other end of the taper pipe is butted with the conveying pipe 1, and a free nozzle is arranged at the end a, so that the tangential line is positioned at the nozzle of the taper pipe 2, and the tangential line of the taper pipe is the nozzle tangential line, as shown in fig. 11. However, the tool may be mounted only by fixing the tool at a fixed position on the first object. For example, taking the first object and the second object as a taper pipe and a delivery pipe respectively, and the tangential line is positioned at the pipe orifice of the taper pipe as an example, the calibration tool can be designed to be composed of a T-shaped rigid structural component and two anchor points. The T-shaped rigid structural part is composed of two connecting rods which are perpendicular to each other and respectively comprise a long connecting rod and a short connecting rod, wherein the diameter of the short connecting rod is the same as the inner diameter of the pipe orifice of the taper pipe, two anchor points are respectively arranged at two ends of the long connecting rod, the long connecting rod is a straight rod part used for representing a tangential line in the embodiment of the invention, the short connecting rod is used for fixing a calibration tool, and two anchor points are arranged at two ends of the long connecting rod. And (3) inserting the short connecting rod of the calibration tool into the pipe orifice of the taper pipe, rotating the long connecting rod by taking the short connecting rod as an axis as shown in fig. 14, and ensuring that the long connecting rod is parallel to the ground so as to complete the installation of the calibration tool, wherein the installation effect diagram of the calibration tool can be shown in fig. 12 and fig. 13. The short connecting rod part of the calibration tooling can be designed into other structures as long as the calibration tooling can be fixed. The point, which is located on the straight rod part of the calibration tool part and is a first preset distance away from the fixed position where the tangential line is located, is a first point, the point, which is located on the straight rod part and is a second preset distance away from the fixed position where the tangential line is located, is a second point, and the connecting line of the first point and the second point is the tangential line. For example, the two anchor points on the T-shaped rigid structural member are the first point and the second point, respectively, and the tube orifice tangential line can be obtained by connecting the two anchor points. Recording a video of a first object, and adjusting the posture of the first object, wherein the central axis slope of the first object, a first coordinate of a first point representing a current tangential line of the first object, and a second coordinate of a second point are recorded every time the posture of the first object is adjusted, namely, the central axis slope of the first object, the first coordinate and the second coordinate are recorded every time the posture of the first object is changed, so that a preset lookup table corresponding to the first object is obtained, and the corresponding relation between the central axis slope of the first object and the first coordinate and the second coordinate is recorded in the obtained preset lookup table. When the slope of the central axis, the first coordinate and the second coordinate are obtained, the image obtained by framing the recorded video can be used. Obtaining the slope of the central axis may refer to the method described in the above embodiment, determine the central axis of the first object, establish a coordinate system, select any two points on the central axis, and then determine the slope of the central axis according to the selected two points. In addition, there are many methods for obtaining the first coordinates and the second coordinates. For example, after obtaining the image of the first object, a coordinate system may be established, and the first coordinates of the first point and the second coordinates of the second point may be set. In addition, after obtaining the image of the first object, the first point region and the second point region may be extracted, a coordinate system may be established, the center of gravity or the coordinate of the center of mass of the first point region may be calculated to obtain the first coordinate, and the center of gravity or the coordinate of the center of mass of the second point region may be calculated to obtain the second coordinate, the first point region may refer to a region occupied by the first point, and the second point region may refer to a region occupied by the second point. For example, taking the pipe orifice tangential line of the taper pipe as an example, the two anchor points are respectively a first point and a second point, after the image of the taper pipe is obtained, a coordinate system is established, and the coordinates of the anchor points are set; the anchor point area can also be set to be dark color, for example, red, the dark color area is extracted, a coordinate system is established, and the coordinates of the gravity center or the mass center of the dark color area are obtained, namely the coordinates of the anchor point can be obtained. In addition, for subsequent accurate searching, as many central axis slope values as possible and corresponding first coordinates and second coordinates are acquired by adopting a dense sampling method. In addition, in embodiments of the present invention, laser light may be used to characterize tangential lines in addition to using calibration tooling. For example, taking the tangential line on the first object as an example, the laser ray is located at a fixed position of the first object, a point on the laser ray that is a first predetermined distance from the fixed position is a first point, and a point on the laser ray that is a second predetermined distance from the fixed position is a second point. After obtaining the image of the first object, the first coordinates and the second coordinates may be determined with reference to the method described in the above embodiment.
In addition, taking as an example that the conveying pipe and the conical pipe to be butted and the tangential line are located at the nozzle of the conical pipe, when the tangential line of the nozzle of the conical pipe is determined by using a preset lookup table established based on a calibration tool of a "T", the method for determining the tangential line of any one is similar to the method for determining the tangential line described in the above embodiments, specifically, see the following. Determining the slope of the central axis of the taper pipe, and searching the preset search corresponding to the taper pipeAnd finding out whether the determined central axis slope of the taper pipe exists in the table. If the determined central axis slope of the taper pipe exists in the preset lookup table, a first coordinate and a second coordinate corresponding to the determined central axis slope are directly found in the preset lookup table, namely the coordinates corresponding to the two anchor points are found, and the pipe orifice tangential line is determined based on the found coordinates. If the determined mid-axis slope of the taper pipe is not present in the preset lookup table, a first approximate mid-axis slope and a second approximate mid-axis slope that are closest to the determined mid-axis slope of the taper pipe are found in the preset lookup table, for example, K1 and K2, respectively. The coordinates of the two anchor points corresponding to K1, for example, P11、P12The coordinates of the two anchor points corresponding to K2, for example, P, are determined21、P22Wherein, P11And P21Different coordinates, P, for the same anchor point12And P22Is a different coordinate of another anchor point. Calculation of P by interpolation11And P21Coordinate P of the midpoint1，P12And P22Coordinate P of the midpoint2Coordinate P1And P2The connecting lines of the corresponding points are the pipe orifice tangential lines of the current conical pipe posture.
Fig. 16 is a flowchart of a method for assisting docking according to another embodiment of the present invention. As shown in fig. 16, in this embodiment, the method includes the following. In the embodiment, a taper pipe and a conveying pipe of the vehicle-mounted pump are butted, the tangential line is positioned at the pipe orifice of the taper pipe, and the pipe orifice tangential line of the taper pipe is calibrated through a T-shaped calibration tool.
Inputting a target tube image, wherein the target tube comprises a conical tube and a conveying tube. And calling a semantic segmentation model to predict a target pipe area, namely determining the coverage area of the cone pipe in the image and the coverage area of the conveying pipe in the image, wherein the semantic segmentation model is based on a convolutional neural network model and is trained in advance. For example, by comparing two adjacent images input, the coverage area of the cone pipe is unchanged, because the cone pipe is mounted on the vehicle body, as shown in fig. 10, the camera and the cone pipe are relatively static, and the coverage area of the cone pipe in the images is unchanged. And calculating the central axes of the taper pipe and the conveying pipe. And calculating the slope of the central axis of the conical pipe, and calculating the pipe orifice tangential line of the conical pipe according to the slope of the central axis of the conical pipe. And inputting a target tube auxiliary line to the display screen, wherein the target tube auxiliary line comprises a central axis of the conical tube, a central axis of the conveying pipe and a pipe orifice tangential line of the conical tube. In the technical scheme provided by the embodiment of the invention, the auxiliary pipe aligning device can comprise a visual sensor, an intelligent host and a display screen, wherein the visual sensor can be a camera. Wherein, the installation schematic diagram of the pipe fitting is shown in fig. 10. The aim of auxiliary tube alignment is to acquire images and videos of the taper tube and the conveying tube through a visual sensor, identify the positions and the position relation of the taper tube and the conveying tube through an AI algorithm of an intelligent host, and finally display the central axes of the taper tube and the conveying tube and the tangential line of the orifice of the taper tube on a display screen to serve as auxiliary lines for guiding a driver to adjust the posture of the vehicle. Fig. 11 is a schematic diagram of the pair tubes collected by the visual angle of the camera, and it can be known from the pinhole imaging model of the camera that the central axes of the cone tube and the delivery tube are parallel on the image coordinate, which indicates that the central axes of the two target tubes are also parallel in the world coordinate. As can be seen from fig. 7, if there is a large deviation between the front and rear positions (radial directions) of the conical pipe and the conveying pipe, it is difficult to determine the distance between the conical pipe and the conveying pipe in the radial direction only by the positional relationship between the two pipes on the video image without the aid of the pipe orifice tangential line of the conical pipe, so that the conical pipe orifice tangential line is helpful for the driver to accurately determine the radial deviation between the conical pipe and the conveying pipe on the video image, thereby purposefully adjusting the vehicle posture. In a word, in the auxiliary pipe aligning process, the conveying pipe is in a static state under the world coordinate generally, and a driver can adjust the vehicle posture to complete pipe aligning operation according to the parallel relation between the conical pipe on the video image and the central axis of the conveying pipe and the tangential line position relation between the pipe orifice of the conveying pipe and the pipe orifice of the conical pipe. The method for calculating the central axis of the taper pipe and the conveying pipe is realized based on a machine vision technology: (1) firstly, training a semantic segmentation network model; (2) reasoning about the target pipe area; (3) then, calculating a region skeleton line according to the technical region of the target tube region; (4) and finally, calculating the central axis of the target pipe by a curve fitting method according to the framework line of the target pipe area. In addition, when a preset lookup table corresponding to the taper pipe is established, the tangential line of the pipe orifice of the taper pipe is realized by a calibration method: (1) firstly, designing a set of calibration tooling parts; (2) calibrating a tangential line of the pipe orifice of the taper pipe by a calibration tool; (3) and adjusting the postures of the conical pipes, and acquiring pipe orifice tangential line information under different conical pipe postures by using a dense sampling method. In actual pipe aligning operation, the posture of the taper pipe is not fixed, and calibration information based on the posture of the taper pipe cannot be well adapted to other postures of the taper pipe. The technical scheme provided by the embodiment of the invention provides a design of a calibration tool and a pipe orifice tangential line calibration method under different conical pipe postures, and can solve the problem of calculation of calibration information under various conical pipe postures. In the prior art, the taper pipe is calibrated for multiple times, and the same calibration information is used no matter what the taper pipe is. The pipe auxiliary line generated by the method provided by the embodiment of the invention is dynamically adjusted according to the positions of the taper pipe and the conveying pipe in real time, and the flexibility and the multi-working-condition applicability are superior to those of the existing method. In addition, the prior art does not mention how to obtain the posture of the taper pipe, and the technical scheme provided by the embodiment of the invention uses the tangential line of the pipe orifice to represent the posture of the taper pipe.
Accordingly, another aspect of embodiments of the present invention provides an apparatus for assisting docking.
Fig. 17 is a block diagram of an apparatus for assisting in docking according to another embodiment of the present invention, as shown in fig. 17, the apparatus includes an image acquisition module 6, a central axis determination module 7, a tangential line determination module 8, and an annotation module 9. The image acquisition module 6 is configured to acquire images of a first object and a second object to be docked; the central axis determining module 7 is configured to determine a first central axis of the first object in the docking direction and a second central axis of the second object in the docking direction based on the image; the tangential line determination module 8 is for determining a tangential line perpendicular to the docking direction for any one of the first object and the second object; the labeling module 9 is configured to label the first central axis, the second central axis, and the tangential line in the image to assist the docking of the first object and the second object.
Optionally, in an embodiment of the present invention, the determining, by the central axis determining module, the first central axis and/or the second central axis based on the image includes: determining a first coverage area of the first object in the image and/or a second coverage area of the second object in the image; determining a first skeleton line of the first coverage area and/or a second skeleton line of the second coverage area; and fitting the first skeleton line into the first central axis and/or fitting the second skeleton line into the second central axis by using a curve fitting method.
Optionally, in an embodiment of the present invention, determining a first coverage area of the first object in the image and/or a second coverage area of the second object in the image includes: the first coverage area and/or the second coverage area is determined based on a preset convolutional neural network.
Optionally, in an embodiment of the present invention, the determining the tangential line perpendicular to the docking direction of any one of the first object and the second object by the tangential line determination module includes: determining the slope of the central axis of either; and determining any tangential line based on the determined central axis slope and any preset lookup table corresponding to the central axis slope, wherein the preset lookup table comprises a corresponding relation between the central axis slope and the tangential line related information.
Optionally, in an embodiment of the present invention, the tangential line is located at a fixed position of any one of the tangential lines, the tangential line related information includes a first coordinate of a first point and a second coordinate of a second point, the first point is located at a first preset distance from the fixed position, the second point is located at a second preset distance from the fixed position, and determining the tangential line of any one of the tangential lines based on the determined slope of the central axis and a preset lookup table corresponding to any one of the tangential lines includes: under the condition that the determined central axis slope exists in the preset lookup table, finding a first coordinate and a second coordinate corresponding to the determined central axis slope in the preset lookup table; and determining a tangential line to either one from the found first and second coordinates; and/or under the condition that the determined central axis slope does not exist in the preset lookup table, finding a first approximate central axis slope and a second approximate central axis slope which are closest to the determined central axis slope in the preset lookup table; finding a first coordinate and a second coordinate corresponding to the slope of the first approximate central axis and a first coordinate and a second coordinate corresponding to the slope of the second approximate central axis in a preset lookup table; determining a first coordinate and a second coordinate corresponding to the determined central axis slope based on a first coordinate and a second coordinate corresponding to the first approximate central axis slope and a first coordinate and a second coordinate corresponding to the second approximate central axis slope; and determining a tangential line of either one of the first and second coordinates based on the determined first and second coordinates corresponding to the slope of the central axis.
Optionally, in an embodiment of the invention, the tangential line is located at the butt end of either.
Optionally, in an embodiment of the present invention, either one of the first object and the second object has a smaller change in posture.
The specific working principle and benefits of the device for assisting in docking provided by the embodiment of the present invention are similar to those of the method for assisting in docking provided by the embodiment of the present invention, and will not be described herein again.
In addition, another aspect of the embodiments of the present invention further provides a docking method, where the docking method includes: and controlling the docking of the first object and the second object according to the first central axis, the second central axis and the tangential line marked by the method for assisting the docking described in the embodiment.
In addition, another aspect of the embodiments of the present invention further provides a system for assisting docking, where the system includes: the device for assisting docking described in the above embodiments; and a display module for displaying the image labeled with the first central axis, the second central axis and the tangential line.
Optionally, in an embodiment of the present invention, the system further includes: the camera module is used for acquiring videos of a first object and a second object; and a decomposition module for decomposing the acquired video into images.
In addition, another aspect of the embodiments of the present invention further provides an engineering equipment, which includes the system for assisting docking described in the above embodiments.
In addition, another aspect of the embodiments of the present invention also provides a machine-readable storage medium, which stores instructions for causing a machine to execute the method for assisting docking or the docking method described in the above embodiments.
The preferred embodiments of the present invention have been described in detail with reference to the accompanying drawings, however, the present invention is not limited to the specific details of the above embodiments, and various simple modifications can be made to the technical solution of the present invention within the technical idea of the present invention, and these simple modifications are within the protective scope of the present invention.
It should be noted that the various features described in the above embodiments may be combined in any suitable manner without departing from the scope of the invention. The invention is not described in detail in order to avoid unnecessary repetition.
In addition, any combination of the various embodiments of the present invention can be made, and the same should be considered as the disclosure of the present invention as long as the idea of the present invention is not violated.

Claims (19)

1. A method for assisting docking, the method comprising:
acquiring images of a first object and a second object to be butted;
determining, based on the image, a first medial axis of the first object along a docking direction and a second medial axis of the second object along the docking direction;
determining a tangential line of any of the first object and the second object that is perpendicular to the docking direction; and
and marking the first central axis, the second central axis and the tangential line in the image so as to assist the butt joint of the first object and the second object.
2. The method of claim 1, wherein determining the first centerline axis and/or the second centerline axis based on the image comprises:
determining a first coverage area of the first object in the image and/or a second coverage area of the second object in the image;
determining a first skeleton line of the first coverage area and/or a second skeleton line of the second coverage area; and
fitting the first skeleton line to the first central axis and/or fitting the second skeleton line to the second central axis using a curve fitting method.
3. The method of claim 2, wherein the determining a first coverage area of the first object in the image and/or a second coverage area of the second object in the image comprises:
determining the first coverage area and/or the second coverage area based on a preset convolutional neural network.
4. The method of claim 1, wherein the determining a tangential line of any of the first object and the second object that is perpendicular to the docking direction comprises:
determining a centerline slope for said any one; and
and determining the tangential line of any one of the lines based on the determined central axis slope and a preset lookup table corresponding to the any one of the lines, wherein the preset lookup table comprises a corresponding relation between the central axis slope and relevant information of the tangential line.
5. The method of claim 4, wherein the tangential line is located at a fixed location of the any one, wherein the tangential line related information comprises a first coordinate of a first point located on a tangential line at a first predetermined distance from the fixed location and a second coordinate of a second point at a second predetermined distance from the fixed location, wherein determining the tangential line of the any one based on the determined mid-axis slope and a corresponding predetermined lookup table of the any one comprises:
in the case where there is a determined slope of the central axis in the preset lookup table,
finding the first coordinate and the second coordinate corresponding to the determined central axis slope in the preset lookup table; and
determining the tangential line of said either one from the found first and second coordinates; and/or
In the absence of the determined mid-axis slope in the predetermined lookup table,
finding a first approximate central axis slope and a second approximate central axis slope which are closest to the determined central axis slope in the preset lookup table;
finding the first coordinate and the second coordinate corresponding to the first approximate central axis slope and the first coordinate and the second coordinate corresponding to the second approximate central axis slope in the preset lookup table;
determining the first coordinate and the second coordinate corresponding to the determined central axis slope based on the first coordinate and the second coordinate corresponding to the first approximate central axis slope and the first coordinate and the second coordinate corresponding to the second approximate central axis slope; and
determining the tangential line of either one based on the first and second coordinates corresponding to the determined mid-axis slope.
6. The method of any one of claims 1-5, wherein the tangential line is located at a butt end of the any one.
7. The method of any one of claims 1-5, wherein the either one is the one of the first object and the second object having the lesser change in pose.
8. An apparatus for assisting docking, the apparatus comprising:
the image acquisition module is used for acquiring images of a first object and a second object to be butted;
a central axis determining module, configured to determine, based on the image, a first central axis of the first object along a docking direction and a second central axis of the second object along the docking direction;
a tangential line determination module to determine a tangential line of any one of the first object and the second object that is perpendicular to the docking direction; and
and the marking module is used for marking the first central axis, the second central axis and the tangential line in the image so as to assist the butt joint of the first object and the second object.
9. The apparatus of claim 8, wherein the centerline axis determination module determining the first centerline axis and/or the second centerline axis based on the image comprises:
determining a first coverage area of the first object in the image and/or a second coverage area of the second object in the image;
determining a first skeleton line of the first coverage area and/or a second skeleton line of the second coverage area; and
fitting the first skeleton lines into the first central axis and/or fitting the second skeleton lines into the second central axis by using a curve fitting method.
10. The apparatus of claim 9, wherein the determining a first coverage area of the first object in the image and/or a second coverage area of the second object in the image comprises:
determining the first coverage area and/or the second coverage area based on a preset convolutional neural network.
11. The apparatus of claim 8, wherein the tangential line determination module determines a tangential line of any one of the first object and the second object that is perpendicular to the docking direction comprises:
determining a centerline slope for said any one; and
and determining the tangential line of any one of the lines based on the determined central axis slope and a preset lookup table corresponding to the any one of the lines, wherein the preset lookup table comprises a corresponding relation between the central axis slope and relevant information of the tangential line.
12. The apparatus of claim 11, wherein the tangent line is located at a fixed location of the any one, wherein the tangent line related information comprises first coordinates of a first point located on a tangent line at a first predetermined distance from the fixed location and second coordinates of a second point at a second predetermined distance from the fixed location, wherein determining the tangent line of the any one based on the determined mid-axis slope and a corresponding preset lookup table of the any one comprises:
in the case of a determined mid-axis slope in the predetermined look-up table,
finding the first coordinate and the second coordinate corresponding to the determined central axis slope in the preset lookup table; and
determining the tangential line of said either one from the found first and second coordinates; and/or
In the absence of the determined centerline slope in the preset lookup table,
finding a first approximate central axis slope and a second approximate central axis slope which are closest to the determined central axis slope in the preset lookup table;
finding the first coordinate and the second coordinate corresponding to the first approximate central axis slope and the first coordinate and the second coordinate corresponding to the second approximate central axis slope in the preset lookup table;
determining the first coordinate and the second coordinate corresponding to the determined central axis slope based on the first coordinate and the second coordinate corresponding to the first approximate central axis slope and the first coordinate and the second coordinate corresponding to the second approximate central axis slope; and
determining the tangential line of either one based on the first and second coordinates corresponding to the determined mid-axis slope.
13. The device of any one of claims 8-12, wherein the tangential line is located at a butt end of the either.
14. The apparatus of any one of claims 8-12, wherein the either one is the one of the first object and the second object having the lesser change in pose.
15. A docking method, comprising:
the first medial axis, the second medial axis, and the tangential line controls of any of the methods of claims 1-7 docking the first object and the second object.
16. A system for assisting docking, the system comprising:
the device of any one of claims 8-14; and
and the display module is used for displaying the image marked with the first central axis, the second central axis and the tangential line.
17. The system of claim 16, further comprising:
the camera module is used for acquiring videos of a first object and a second object; and
and the decomposition module is used for decomposing the acquired video into images.
18. An engineering equipment, characterized in that it comprises a system according to claim 16 or 17.
19. A machine-readable storage medium having stored thereon instructions for causing a machine to perform the method of any one of claims 1-7 and 15.
CN202210116783.4A 2022-02-07 2022-02-07 Method, device and system for auxiliary docking, docking method and engineering equipment Pending CN114596264A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202210116783.4A CN114596264A (en) 2022-02-07 2022-02-07 Method, device and system for auxiliary docking, docking method and engineering equipment
PCT/CN2022/139042 WO2023147743A1 (en) 2022-02-07 2022-12-14 Docking assisting method, device and system, docking method and engineering equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210116783.4A CN114596264A (en) 2022-02-07 2022-02-07 Method, device and system for auxiliary docking, docking method and engineering equipment

Publications (1)

Publication Number Publication Date
CN114596264A true CN114596264A (en) 2022-06-07

Family

ID=81804733

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210116783.4A Pending CN114596264A (en) 2022-02-07 2022-02-07 Method, device and system for auxiliary docking, docking method and engineering equipment

Country Status (2)

Country Link
CN (1) CN114596264A (en)
WO (1) WO2023147743A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023147743A1 (en) * 2022-02-07 2023-08-10 中联重科股份有限公司 Docking assisting method, device and system, docking method and engineering equipment

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106524946A (en) * 2016-10-27 2017-03-22 西南科技大学 Marine riser automatic connection detection device
US11915603B2 (en) * 2020-06-16 2024-02-27 Honeywell International Inc. Docking guidance display methods and systems
CN112721915A (en) * 2020-12-25 2021-04-30 三一汽车制造有限公司 Parking assist method, parking assist apparatus, vehicle, and storage medium
CN113137067A (en) * 2021-04-12 2021-07-20 中联重科股份有限公司 Auxiliary butt joint device and engineering equipment
CN114596264A (en) * 2022-02-07 2022-06-07 中联重科股份有限公司 Method, device and system for auxiliary docking, docking method and engineering equipment

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023147743A1 (en) * 2022-02-07 2023-08-10 中联重科股份有限公司 Docking assisting method, device and system, docking method and engineering equipment

Also Published As

Publication number Publication date
WO2023147743A1 (en) 2023-08-10

Similar Documents

Publication Publication Date Title
US10634500B2 (en) Aircraft and obstacle avoidance method and system thereof
CN102737236B (en) Method for automatically acquiring vehicle training sample based on multi-modal sensor data
US6752357B2 (en) Distance measuring using passive visual means
CN110246175A (en) Intelligent Mobile Robot image detecting system and method for the panorama camera in conjunction with holder camera
JP2005031085A (en) System and method for analyzing and identifying flaws in manufactured component
CN110962122B (en) Pipeline cleaning robot control system and method
CN114596264A (en) Method, device and system for auxiliary docking, docking method and engineering equipment
CN111570975B (en) Welding deviation measuring method and device based on synchronous drawing and automatic welding system
JP2007113240A (en) Self-location detection method and facility of robot by camera sensor and distance sensor
JP2531488B2 (en) In-pipe measurement method
CN111486797A (en) Automatic extraction method for transverse diameter of subway circular shield tunnel
CN110865366B (en) Intelligent driving radar and image fusion man-machine interaction method
JP3560326B2 (en) Object tracking method and object tracking device
JP2008160635A (en) Camera state detection method
JP2009052907A (en) Foreign matter detecting system
CN105129295B (en) Reversing control method, device, system and rubbish van
JP2002150302A (en) Road surface recognition device
JPH06199297A (en) Parking position measuring method
CN111899512B (en) Vehicle track extraction method and system combined with skyline observation and storage medium
CN107886472A (en) The image mosaic calibration method and image mosaic calibrating installation of panoramic parking system
CN110634136A (en) Pipeline wall damage detection method, device and system
CN113137067A (en) Auxiliary butt joint device and engineering equipment
KR20190071937A (en) Hollow Tube Detection System
CN113554551A (en) Parking method based on parking space calibration
CN111047702A (en) Automatic welding method for flange bent pipe based on binocular vision

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination