WO2023116667A1 - 一种充电设备以及控制机械臂充电的方法 - Google Patents

一种充电设备以及控制机械臂充电的方法 Download PDF

Info

Publication number
WO2023116667A1
WO2023116667A1 PCT/CN2022/140276 CN2022140276W WO2023116667A1 WO 2023116667 A1 WO2023116667 A1 WO 2023116667A1 CN 2022140276 W CN2022140276 W CN 2022140276W WO 2023116667 A1 WO2023116667 A1 WO 2023116667A1
Authority
WO
WIPO (PCT)
Prior art keywords
charging
pose
main body
plug
mechanical arm
Prior art date
Application number
PCT/CN2022/140276
Other languages
English (en)
French (fr)
Inventor
陈辰
楚亚奎
薛景涛
贺亚农
陈晓鹏
黄远灿
张伟民
Original Assignee
华为技术有限公司
北京理工大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司, 北京理工大学 filed Critical 华为技术有限公司
Publication of WO2023116667A1 publication Critical patent/WO2023116667A1/zh

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60L—PROPULSION OF ELECTRICALLY-PROPELLED VEHICLES; SUPPLYING ELECTRIC POWER FOR AUXILIARY EQUIPMENT OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRODYNAMIC BRAKE SYSTEMS FOR VEHICLES IN GENERAL; MAGNETIC SUSPENSION OR LEVITATION FOR VEHICLES; MONITORING OPERATING VARIABLES OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRIC SAFETY DEVICES FOR ELECTRICALLY-PROPELLED VEHICLES
    • B60L53/00—Methods of charging batteries, specially adapted for electric vehicles; Charging stations or on-board charging equipment therefor; Exchange of energy storage elements in electric vehicles
    • B60L53/30—Constructional details of charging stations
    • B60L53/35—Means for automatic or assisted adjustment of the relative position of charging devices and vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60L—PROPULSION OF ELECTRICALLY-PROPELLED VEHICLES; SUPPLYING ELECTRIC POWER FOR AUXILIARY EQUIPMENT OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRODYNAMIC BRAKE SYSTEMS FOR VEHICLES IN GENERAL; MAGNETIC SUSPENSION OR LEVITATION FOR VEHICLES; MONITORING OPERATING VARIABLES OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRIC SAFETY DEVICES FOR ELECTRICALLY-PROPELLED VEHICLES
    • B60L53/00—Methods of charging batteries, specially adapted for electric vehicles; Charging stations or on-board charging equipment therefor; Exchange of energy storage elements in electric vehicles
    • B60L53/10—Methods of charging batteries, specially adapted for electric vehicles; Charging stations or on-board charging equipment therefor; Exchange of energy storage elements in electric vehicles characterised by the energy transfer between the charging station and the vehicle
    • B60L53/14—Conductive energy transfer
    • B60L53/16—Connectors, e.g. plugs or sockets, specially adapted for charging electric vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60L—PROPULSION OF ELECTRICALLY-PROPELLED VEHICLES; SUPPLYING ELECTRIC POWER FOR AUXILIARY EQUIPMENT OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRODYNAMIC BRAKE SYSTEMS FOR VEHICLES IN GENERAL; MAGNETIC SUSPENSION OR LEVITATION FOR VEHICLES; MONITORING OPERATING VARIABLES OF ELECTRICALLY-PROPELLED VEHICLES; ELECTRIC SAFETY DEVICES FOR ELECTRICALLY-PROPELLED VEHICLES
    • B60L53/00—Methods of charging batteries, specially adapted for electric vehicles; Charging stations or on-board charging equipment therefor; Exchange of energy storage elements in electric vehicles
    • B60L53/30—Constructional details of charging stations
    • B60L53/35—Means for automatic or assisted adjustment of the relative position of charging devices and vehicles
    • B60L53/37—Means for automatic or assisted adjustment of the relative position of charging devices and vehicles using optical position determination, e.g. using cameras
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/60—Other road transportation technologies with climate change mitigation effect
    • Y02T10/70—Energy storage systems for electromobility, e.g. batteries
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T10/00—Road transport of goods or passengers
    • Y02T10/60—Other road transportation technologies with climate change mitigation effect
    • Y02T10/7072—Electromobility specific charging systems or methods for batteries, ultracapacitors, supercapacitors or double-layer capacitors

Definitions

  • the present application relates to the technical field of vehicles, in particular to a charging device and a method for controlling charging of a mechanical arm.
  • Mobile charging equipment is embodied in the form of a mobile charging pile, or a mobile chassis equipped with a battery pack.
  • the chassis of the mobile charging car can carry multiple battery packs.
  • the mobile charging device Before the mobile charging device charges the vehicle to be charged, it needs to move to the vicinity of the vehicle to be charged to locate the positioning port area on the vehicle to be charged, and adjust its own posture (the position and the posture of the charging arm) to further Charging port for charging.
  • the current mainstream charging robot solutions all use structured light or binocular or ultrasound combined with robotic arm planning to achieve plugging and unplugging through position control.
  • submillimeter level for example, less than or equal to 2°
  • pose positioning accuracy is required to ensure the smooth insertion of the charging socket into the charging port, and the docking does not damage the connecting device.
  • the solution based on visual feedback requires the visual detection module to meet the requirements of sub-millimeter position recognition and 2° pose recognition.
  • the basic accuracy of the current structured light technology can reach 1-2mm, the pose is about 3°, and it is also affected by the distance. Due to the limitation of field of view, it is difficult to ensure smooth plugging and unplugging. Even if it is plugged in, the existence of pose error will cause a large plugging force, which will easily damage the equipment and even damage the robotic arm. Therefore, it is difficult to achieve the precision requirement for the smooth insertion of the charging socket into the charging port only by performing plugging and unplugging based on the visual feedback technology.
  • the present application provides a manipulator, a charging device, and a method for controlling charging of the manipulator, which can improve the accuracy and reliability of a plugging task.
  • the present application provides a mechanical arm, including: a control structure and a main body of the mechanical arm; a force sensor and a charging plug are arranged on the main body of the mechanical arm, and the charging plug includes a plurality of charging heads, and the force sensor is used for To collect force information on the contact surface between the charging plug and the charging port, the contact surface includes the side surface of at least one charging head among the plurality of charging heads, and the side surface is the axis of the charging head. A surface parallel to the direction; the control structure is used to control the posture of the main body of the mechanical arm according to the force information, so as to insert the charging plug into the charging port.
  • the charging head can be a cylindrical structure, and can include a bottom surface and a side surface.
  • the bottom surface can be the surface that needs to be in contact with the bottom of the charging hole during charging
  • the side surface can be the surface that needs to be in contact with the side wall of the charging hole during charging.
  • the force information includes the force in the radial direction of the charging head.
  • the visual camera can no longer observe the charging port. If only observation at the best observation point is used Plugging and unplugging the charging plug based on the obtained visual pose may cause the situation that it cannot be inserted to the bottom or wedged tightly.
  • This application controls the pose of the robotic arm by obtaining the force information collected by the force sensor. Partially inserted (for example, the side is in contact with the charging port), and the visual information cannot obtain errors, adjust the position and posture according to the force sense, so that the charging plug adapts to the insertion direction of the charging port, thereby improving the accuracy and reliability of the plugging task. sex. It reduces the wear of the charging head due to the extrusion or friction of the side of the charging head and the inner wall of the charging port (or the situation that it is stuck and cannot reach the bottom).
  • the force information includes torque information of the charging port relative to the charging plug.
  • the force information may indicate the 6-dimensional force of the charging port on the charging plug, and the 6-dimensional force may include forces in three-dimensional space (for example, along the x-axis, y-axis, and z-axis directions) and forces in three-dimensional space (such as rotation about the x-axis, y-axis, z-axis). Since the torque information is included, it is possible to more accurately and fine-grainedly identify the blocking direction of the charging port for the charging head, and then provide more precise force control to accurately insert the charging head into the charging hole.
  • the force sensor is fixed on the side of the at least one charging head; or, the mechanical arm includes a joint with a degree of freedom of rotation, and the control structure is used to control the rotation of the joint
  • the force sensor is fixed on the joint; or, the force sensor is fixed between the charging plug and the main body of the mechanical arm.
  • the charging port includes multiple charging holes, and the multiple charging heads are matched one by one with the multiple charging holes; the control structure is specifically used for:
  • the pose adjustment parameters correct the pose of the main body of the mechanical arm from a first pose to a second pose; wherein, the first pose is different from the second pose, and the first pose is different from the second pose, and the first pose is different from the second pose.
  • the second pose is the pose of the main body of the robotic arm when each charging head is aligned with a matching charging hole.
  • the main body of the robotic arm is in a correct pose (for example, the second pose can be a correct pose). For example, when the force information indicates that there is non-axial pressure (or moment) on the charging plug at the charging port, it can be determined that the main body of the manipulator is not in the correct posture.
  • the multiple charging At least one charging head of the heads is not aligned with the matching charging hole, that is, the central axis of the charging head does not coincide with the central axis of the matching charging hole or the deviation is greater than a threshold.
  • the first pose may be the case where the central axis of the charging head does not coincide with the central axis of the matching charging hole or the deviation is greater than a threshold (that is, not in the correct pose as described above).
  • the pose of the main body of the robotic arm can be adjusted based on the force information so that the main body of the robotic arm is in the correct pose.
  • the control mechanism can correct the posture of the main body of the robotic arm (to the second posture, that is, the correct posture) according to the force information when the main body of the robotic arm is in the first posture.
  • the second pose is the pose where the main body of the manipulator can insert the charging plug into the bottom of the charging port in a correct way with less resistance.
  • the first posture is the posture when the bottom end faces of the plurality of charging heads are not in contact with the bottom of the charging hole, and the bottom end faces are the axes of the charging heads. A surface perpendicular to the direction.
  • the alignment includes: the projection range of the charging head along the direction of the matching charging hole is within the range of the matching charging hole, and the central axis of the charging head is in line with the central axis of the matching charging hole The coincidence or deviation is less than the threshold.
  • control structure is further configured to: after correcting the pose of the main body of the robotic arm from the first pose to a second pose, control the pose of the main body of the robotic arm, so that The charging plug moves towards the bottom of the charging hole and along the central axis of the charging head.
  • the resistance of the charging port to the charging plug in the target direction tends to decrease, and the target direction is the axial direction or radial direction of the charging head.
  • the resistance of the charging port to the charging head in the target direction may have a tendency to decrease, and the target direction may be in a radial direction and/or an axial state (for example Can be a radial direction, an axial state, or both a radial direction and an axial direction).
  • the so-called decreasing trend here can be understood as the resistance of the charging port to the charging head in the target direction may not decrease all the time, but there may be a local increase. Resistance in the direction of the target is reduced.
  • the charging plug After the posture of the main body of the manipulator is corrected from the first posture to the second posture, the charging plug is already in the alignment posture with the charging port, in this case, the charging plug can be directed towards the charging port bottom and along the direction of the central axis of the charging head to the bottom.
  • the control structure may indicate that the charging port only exists along the charging plug based on the force information. The resistance in the axial direction of the charging plug is controlled to keep the main body of the mechanical arm in a static state, thereby completing the task of inserting into the charging port.
  • the moment between the charging port and the charging plug tends to decrease.
  • the pose adjustment parameters are specifically used to control a moving direction and a moving speed of the charging plug.
  • control mechanism can obtain in real time the image information collected by the visual sensor for the area of the charging port; according to the position information of the charging port in the image information, obtain the The real-time pose of the main body; according to the real-time pose, the pose of the main body of the robotic arm is controlled so as to guide the charging plug on the main body of the robotic arm to the surroundings of the charging port.
  • the real-time pose of the main body of the manipulator can be obtained based on the image information acquired in real time (the real-time pose can be the distance between the end of the main body of the manipulator and the charging port of the vehicle to be charged).
  • Relative pose because when controlling the pose of the main body of the manipulator, there may be situations such as movement of the base and obstruction of obstacles, resulting in errors in the pose of the main body of the manipulator.
  • This application can be corrected based on the real-time pose
  • the pose of the main body of the robotic arm can improve the recognition accuracy of the pose of the main body of the robotic arm, and then guide the charging plug on the main body of the robotic arm to the surroundings of the charging port more accurately.
  • visual servoing is a closed-loop control method that combines vision with motion.
  • the main function is to realize the guiding movement of the charging gun at the end of the manipulator to the charging port.
  • the general process is to use the visual recognition pose result of the charging gun/charging stand as input, use the visual servo feedback control idea to calculate the driving error, and use the pose-based visual servo to realize the precise guidance of the end of the manipulator to the target pose.
  • the adaptive gain method is adopted to speed up the adjustment speed when the error is small, and realize fast visual servoing.
  • the robotic arm will be at the best observation point, where the most accurate pose of the charging port has been obtained. At this time, the robotic arm will guide the charging socket to the vicinity of the charging port to prepare for the next step of force-controlled insertion.
  • the accuracy of the target observation pose can be guaranteed.
  • the pose of the mobile robot base relative to the charging dock may change continuously.
  • the fast visual servoing of the pose is used to achieve this goal.
  • the idea of visual servoing based on pose feedback can be adopted, and the joint speed control law can be designed based on Li's stability criterion to realize the precise guidance of the end of the manipulator to the target position.
  • the vision sensor is provided on the main body of the robotic arm.
  • the visual sensor may not be arranged on the main body of the robotic arm but may be arranged at a position corresponding to a third-party perspective.
  • the visual sensor when determining the above relative pose, needs to collect image information for the charging port, and determine the relative pose of the charging port relative to the visual sensor based on the image information.
  • the image information includes a 2D image and a 3D point cloud
  • the control mechanism is further configured to: extract the first 3D model corresponding to the charging port according to the 2D image and the preset 3D model of the charging port. - 3D position;
  • the two-dimensional pixel coordinates of the center point of the charging port can be obtained, and the number of each point can be obtained through the mutual position relationship of the center points; according to the standard CAD model of the charging stand, the three-dimensional physical coordinates of the center point of the round hole in the target coordinate system can be obtained, Further, the three-dimensional physical coordinates of each charging hole are obtained.
  • the first 3D position includes 3D positions of M first feature points on the charging port;
  • the second 3D position includes 3D positions of M second feature points on the charging port;
  • the M first feature points are in one-to-one correspondence with the M second feature points;
  • the fusing of the first 3D position and the second 3D position includes: combining the M first feature points
  • the 3D positions of the points are respectively fused with the corresponding 3D positions of the second feature points.
  • the first feature point and the second feature point may be charging holes.
  • This application proposes a method for accurately identifying the position and posture of the charging port that combines 2D image-3D point cloud information.
  • the 2D depth image detection network can be used to perform sub-pixel-level accurate detection of the two-dimensional key points of the charging port, and the three-dimensional key points can be used Detection
  • the three-dimensional key points of the charging port are reliably detected based on the offset of the 3D Hough network.
  • fusion such as using nonlinear optimization
  • the projection error of the fusion of two-dimensional key points and three-dimensional key points is minimized, and then the charging port is obtained.
  • Accurate pose in order to solve the problem of high-precision recognition of the pose of the charging port under the condition of large illumination changes and low target contrast.
  • the main body of the mechanical arm further includes a target interface, and the target interface is used for detachably connecting the charging plug.
  • the target interface may be a component for detachable connection of the cleaning tool based on electromagnetic properties.
  • the magnetic suction structure is used to grab the charging gun to ensure the reliable docking of the charging plug and the mechanical arm, and then proceed to the next charging operation.
  • the target interface is connected to an initial charging plug
  • the control structure is also used to control the posture of the main body of the robot arm and the connection state of the target interface, and remove the initial charging plug from the target interface and replace it with the charging plug;
  • the target charging plug is a plug supporting fast charging, and the charging plug is a plug supporting slow charging; or,
  • the target charging plug is a plug supporting slow charging, and the charging plug is a plug supporting fast charging.
  • the charging device can be compatible with two charging methods of fast charging gun and slow charging gun, can be equipped with a quick-change mechanism, and can select a charging gun according to the user's task requirements.
  • the charging device further includes: a plug storage area, where the plug storage area includes a target storage position for setting the charging plug;
  • the controlling the pose of the main body of the manipulator and the connection state of the target interface, and removing the initial charging plug from the target interface and replacing it with the charging plug includes:
  • the present application provides a charging device, including: a mechanical arm, a force sensor, and a charging plug;
  • the mechanical arm includes a control structure and a main body of the mechanical arm;
  • the main body of the mechanical arm is provided with the force sensor and the charging plug, the charging plug includes a plurality of charging heads, and the force sensor is used to collect the effect of the contact surface between the charging plug and the charging port.
  • Force information the contact surface includes a side surface of at least one charging head among the plurality of charging heads, and the side surface is a surface parallel to the axial direction of the charging head;
  • the control structure is used to control the posture of the main body of the robot arm according to the force information, so as to insert the charging plug into the charging port.
  • the force information includes torque information of the charging port relative to the charging plug.
  • the force sensor is fixed on the side of the at least one charging head; or,
  • the mechanical arm includes a joint with a rotational degree of freedom
  • the control structure is used to control the pose of the main body of the mechanical arm by controlling the rotation of the joint, and the force sensor is fixed on the joint; or,
  • the force sensor is fixed between the charging plug and the main body of the mechanical arm.
  • the charging port includes multiple charging holes, and the multiple charging heads are matched one by one with the multiple charging holes; the control structure is specifically used for:
  • the pose adjustment parameters correct the pose of the main body of the mechanical arm from a first pose to a second pose; wherein, the first pose is different from the second pose, and the first pose is different from the second pose, and the first pose is different from the second pose.
  • the second pose is the pose of the main body of the robotic arm when each charging head is aligned with a matching charging hole.
  • the first posture is the posture when the bottom end faces of the plurality of charging heads are not in contact with the bottom of the charging hole, and the bottom end faces are the axes of the charging heads. A surface perpendicular to the direction.
  • the alignment includes: the projection of the charging head along the direction of the matching charging hole falls within the range of the matching charging hole, and the central axis of the charging head is aligned with the matching charging hole.
  • the central axis coincides or the deviation is less than the threshold.
  • control structure is also used to:
  • the moment between the charging port and the charging plug tends to decrease.
  • the pose adjustment parameters are specifically used to control a moving direction and a moving speed of the charging plug.
  • control structure is specifically used for:
  • the main body of the mechanical arm is controlled to maintain a static state.
  • control mechanism is also used for:
  • the image information collected by the visual sensor for the area of the charging port is obtained in real time;
  • the pose of the main body of the robotic arm is controlled so as to guide the charging plug on the main body of the robotic arm to the periphery of the charging port.
  • the main body of the robotic arm is provided with the vision sensor.
  • the image information includes 2D images and 3D point clouds, and the control mechanism is also used for:
  • the first 3D position and the second 3D position are fused to obtain the pose of the charging port.
  • the first 3D position includes 3D positions of M first feature points on the charging port;
  • the second 3D position includes 3D positions of M second feature points on the charging port;
  • the M first feature points are in one-to-one correspondence with the M second feature points;
  • the fusing the first 3D position and the second 3D position includes:
  • the 3D positions of the M first feature points are respectively fused with the corresponding 3D positions of the second feature points.
  • the main body of the mechanical arm further includes a target interface, and the target interface is used for detachably connecting the charging plug.
  • the target interface is connected to an initial charging plug
  • the control structure is also used to control the posture of the main body of the robot arm and the connection state of the target interface, and remove the initial charging plug from the target interface and replace it with the charging plug;
  • the target charging plug is a plug supporting fast charging, and the charging plug is a plug supporting slow charging; or,
  • the target charging plug is a plug supporting slow charging, and the charging plug is a plug supporting fast charging.
  • the charging device further includes: a plug storage area, where the plug storage area includes a target storage position for setting the charging plug;
  • the controlling the pose of the main body of the manipulator and the connection state of the target interface, and removing the initial charging plug from the target interface and replacing it with the charging plug includes:
  • the charging equipment is a fixed charging pile, a mobile robot or a charging car.
  • the present application provides a method for controlling the charging of a robotic arm, the method is applied to the control structure of the robotic arm, the main body of the robotic arm is provided with a force sensor and a charging plug, and the charging plug includes a plurality of charging heads , the force sensor is used to collect force information on the contact surface between the charging plug and the charging port, the contact surface includes a side surface of at least one charging head among the plurality of charging heads, and the side surface is and A surface parallel to the axial direction of the charging head;
  • the methods include:
  • the pose of the main body of the mechanical arm is controlled so as to insert the charging plug into the charging port.
  • the force information includes torque information of the charging port relative to the charging plug.
  • the force information includes the force in the radial direction of the charging head.
  • the force sensor is fixed on the side of the at least one charging head; or,
  • the mechanical arm includes a joint with a rotational degree of freedom
  • the control structure is used to control the pose of the main body of the mechanical arm by controlling the rotation of the joint, and the force sensor is fixed on the joint; or,
  • the force sensor is fixed between the charging plug and the main body of the mechanical arm.
  • the charging port includes a plurality of charging holes, and the plurality of charging heads are matched with the plurality of charging holes;
  • the pose of the arm body including:
  • the pose adjustment parameters correct the pose of the main body of the mechanical arm from a first pose to a second pose; wherein, the first pose is different from the second pose, and the first pose is different from the second pose, and the first pose is different from the second pose.
  • the second pose is the pose of the main body of the robotic arm when each charging head is aligned with a matching charging hole.
  • the first posture is the posture when the bottom end faces of the plurality of charging heads are not in contact with the bottom of the charging hole, and the bottom end faces are the axes of the charging heads. A surface perpendicular to the direction.
  • the alignment includes: the projection of the charging head along the direction of the matching charging hole falls within the range of the matching charging hole, and the central axis of the charging head is aligned with the matching charging hole.
  • the central axis coincides or the deviation is less than the threshold.
  • the method further includes:
  • the resistance of the charging port to the charging plug in the target direction tends to decrease, and the target direction is the axial direction or radial direction of the charging head.
  • the moment between the charging port and the charging plug tends to decrease.
  • the method also includes:
  • the main body of the mechanical arm is controlled to maintain a static state.
  • the pose adjustment parameters are specifically used to control a moving direction and a moving speed of the charging plug.
  • the method also includes:
  • the image information collected by the visual sensor for the area of the charging port is acquired in real time;
  • the pose of the main body of the robotic arm is controlled so as to guide the charging plug on the main body of the robotic arm to the periphery of the charging port.
  • the main body of the robotic arm is provided with the vision sensor.
  • the image information includes a 2D image and a 3D point cloud
  • the method further includes:
  • the first 3D position and the second 3D position are fused to obtain the pose of the charging port.
  • the first 3D position includes 3D positions of M first feature points on the charging port;
  • the second 3D position includes 3D positions of M second feature points on the charging port;
  • the M first feature points are in one-to-one correspondence with the M second feature points;
  • the fusing the first 3D position and the second 3D position includes:
  • the 3D positions of the M first feature points are respectively fused with the corresponding 3D positions of the second feature points.
  • the main body of the mechanical arm further includes a target interface, and the target interface is used for detachably connecting the charging plug.
  • the target interface is connected to an initial charging plug
  • the method further includes: controlling the posture of the main body of the robot arm and the connection state of the target interface, removing the initial charging plug from the target interface and replacing it with the charging plug; wherein,
  • the target charging plug is a plug supporting fast charging, and the charging plug is a plug supporting slow charging; or,
  • the target charging plug is a plug supporting slow charging, and the charging plug is a plug supporting fast charging.
  • the charging device further includes: a plug storage area, where the plug storage area includes a target storage position for setting the charging plug;
  • the controlling the pose of the main body of the manipulator and the connection state of the target interface, and removing the initial charging plug from the target interface and replacing it with the charging plug includes:
  • the charging equipment is a fixed charging pile, a mobile robot or a charging car.
  • an embodiment of the present application provides a computer-readable storage medium, which is characterized by comprising computer-readable instructions, and when the computer-readable instructions are run on a computer device, the computer device is made to execute the above-mentioned third aspect. and any of its optional methods.
  • the embodiment of the present application provides a computer program product, which is characterized by including computer-readable instructions, and when the computer-readable instructions are run on a computer device, the computer device is made to execute the above-mentioned third aspect and its Either method is optional.
  • the present application provides a chip system
  • the chip system includes a processor, configured to support the executive control mechanism to implement the functions involved in the above aspect, for example, send or process the data involved in the above method; or, information.
  • the system-on-a-chip further includes a memory, and the memory is used for storing necessary program instructions and data of the execution device or the training device.
  • the system-on-a-chip may consist of chips, or may include chips and other discrete devices.
  • the present application provides a device for controlling charging of a robotic arm, the device comprising one or more processors and a memory; wherein, computer-readable instructions are stored in the memory; the one or more processing The computer reads the computer-readable instructions, and executes the above-mentioned third aspect and any optional method thereof.
  • the present application provides a charging device, which is characterized in that it includes: a mechanical arm, a force sensor and a charging plug; the mechanical arm includes a control structure and a main body of the mechanical arm; the main body of the mechanical arm is provided with the force sensor and the The charging plug, the charging plug includes a plurality of charging heads, the force sensor is used to collect force information on the contact surface between the charging plug and the charging port, the contact surface includes the plurality of charging heads The side of at least one charging head, the side is a surface parallel to the axial direction of the charging head; the control structure is used to control the pose of the main body of the mechanical arm according to the force information, so as to The charging plug is inserted into the charging port.
  • the visual camera can no longer observe the charging port. If only observation at the best observation point is used Plugging and unplugging the charging plug based on the obtained visual pose may cause the situation that it cannot be inserted to the bottom or wedged tightly.
  • This application controls the pose of the robotic arm by obtaining the force information collected by the force sensor. Partially inserted (for example, the side is in contact with the charging port), and the visual information cannot obtain errors, adjust the position and posture according to the force sense, so that the charging plug adapts to the insertion direction of the charging port, thereby improving the accuracy and reliability of the plugging task. sex. It reduces the wear of the charging head due to the extrusion or friction of the side of the charging head and the inner wall of the charging port (or the situation that it is stuck and cannot reach the bottom).
  • FIG. 1 is a schematic diagram of a scene provided by an embodiment of the present application.
  • FIG. 2 is a schematic diagram of a scene provided by an embodiment of the present application.
  • FIG. 3 is a schematic diagram of a scene provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a scene provided by an embodiment of the present application.
  • FIG. 5 is a schematic diagram of a scene provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a scene provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a scene provided by an embodiment of the present application.
  • Fig. 8 is a schematic structural diagram of a charging device provided by an embodiment of the present application.
  • FIG. 9 is a schematic diagram of a search method provided by the embodiment of the present application.
  • Figure 10 is a schematic diagram of a test result provided by the embodiment of the present application.
  • Figure 11 is a schematic diagram of a test result provided by the embodiment of the present application.
  • Fig. 12 is a schematic diagram of a pose calculation method provided by the embodiment of the present application.
  • FIG. 13 is a schematic diagram of an information fusion algorithm provided by the embodiment of the present application.
  • Figure 14 is a schematic diagram of a model structure provided by the embodiment of the present application.
  • Figure 15 is a schematic diagram of a test result provided by the embodiment of the present application.
  • FIG. 16 is a schematic diagram of a coordinate system conversion provided by the embodiment of the present application.
  • FIG. 17 is a schematic diagram of 3D information extraction provided by the embodiment of the present application.
  • Figure 18 is a schematic diagram of a method for controlling the charging of a robotic arm provided by an embodiment of the present application.
  • Figure 19 is a schematic diagram of a mechanical arm provided by the embodiment of the present application.
  • Fig. 20 is a schematic diagram of a method for controlling charging of a robotic arm provided in an embodiment of the present application
  • FIG. 21 is a schematic diagram of an RCC device provided in the embodiment of the present application.
  • Figure 22 is a schematic diagram of controlling the charging of a robotic arm provided by the embodiment of the present application.
  • Figure 23 is a force schematic diagram provided by the embodiment of the present application.
  • Figure 24 is a schematic diagram of a device for controlling charging of a robotic arm provided in an embodiment of the present application.
  • FIG. 25 is a schematic structural diagram of a chip provided by an embodiment of the present application.
  • New energy vehicles represent a new direction for the green development of the automotive industry, and have experienced explosive development in my country and the world. With the sharp increase in sales of new energy vehicles, car owners need to "power up” on the road.
  • the existing charging piles in the service area or parking lot are relatively insufficient, which often leads to the phenomenon that a large number of users queue up, and the waiting time in the queue far exceeds the charging time.
  • using mobile charging equipment such as charging robots
  • to automatically charge new energy vehicles and providing "mobile charging treasures" for new energy vehicles can effectively alleviate the seriousness of charging piles in a short period of time. Insufficient problem.
  • the mobile charging device can specifically be in the form of a mobile charging pile, or a mobile chassis equipped with a battery pack.
  • the chassis of the mobile charging car It can be equipped with multiple battery packs.
  • the mobile charging vehicle After receiving the user's request, the mobile charging vehicle will drive to the user's location and provide charging services for the vehicle driven by the user to complete the charging task.
  • the system may include a server 11, at least one vehicle system 12, a mobile charging device 13, a charging pile 14, and the like.
  • each vehicle system 12 may include a user 121 , a terminal device 122 and a vehicle to be charged 123 , as shown in FIG. 3 .
  • the vehicle 123 is the vehicle that the user 121 is driving and needs to be charged, and the owner of the terminal device 122 is the user 121 .
  • the user 121 may be a driver, and when the driver finds that the battery of the vehicle 123 he is driving is insufficient, the driver sends a request message to the server through the terminal device 122, and the request message is used to request to charge the vehicle 123.
  • the user 121 generates a request on the APP of the terminal device 122 , and then sends the request to the server 11 .
  • the terminal device 122 can be a portable device, such as an intelligent terminal, a mobile phone, a notebook computer, a tablet computer, a personal computer (personal computer, PC), a personal digital assistant (personal digital assistant, PDA), a foldable terminal, and a wireless communication function.
  • wearable devices such as smart watches or bracelets
  • user equipment user device
  • UE user equipment
  • augmented reality augmented reality, AR
  • virtual reality virtual reality
  • the vehicle 123 may be an electric vehicle (electric vehicle, EV), and the EV includes a display screen, an on-board processor, and a communication module.
  • the EV may also include other components or units, which are not limited in this embodiment.
  • the above-mentioned vehicle system 12 may not include the terminal device 122, and the functions of the terminal device 122 are realized by the vehicle-machine processor and the communication module in the EV, such as the vehicle-machine processing
  • the server After receiving the instruction triggered by the user, the server sends a request message to the server 11 through the communication module.
  • the server 11 can receive request messages sent by one or more vehicle systems 12, schedule at least one mobile charging device 13 in the system, and dispatch charging tasks to the corresponding mobile charging devices, instructing them to drive to send the request. The location of the user of the message, and charge the requested vehicle. In addition, before scheduling and dispatching charging tasks, the server 11 is also used to obtain road condition information, perform advanced driver assistance systems (advanced driver assistance systems, ADAS) calculations based on the road condition information, and generate scheduling strategies.
  • ADAS advanced driver assistance systems
  • the server 11 is a cloud server, and the cloud server may be a server, or a server cluster composed of multiple servers.
  • the mobile charging device 13 is used to receive the indication signal sent by the server 11, and drive to the position of the vehicle to be charged (the embodiment of the present application may also be referred to as the target vehicle) according to the indication signal, and charge it.
  • the mobile charging device 13 After completing the charging task, return to the charging pile 14 for recharging.
  • the mobile charging device 13 is also used to report the power condition of the current battery pack carried in real time during the process of performing the charging task, and report its own position information to the server 11 in real time during the driving process, so as to serve the server 11. It can grasp the situation of each mobile charging device in the system in real time, and provide a basis for the distribution of subsequent charging tasks and the scheduling of power resources.
  • each mobile charging device 13 includes a vehicle-machine processor, a communication module, a rechargeable battery or a battery pack, and the number of the battery packs may be one or more, and each battery pack carries a certain amount of power.
  • the number of battery packs and the power of each battery pack are not limited.
  • the mobile charging device 13 may be an EV or a fuel vehicle.
  • the embodiment of the present application may be applied to a mobile charging device 13 as shown in FIG. 4 .
  • the mobile charging device 13 may include: a sensor module 110 , a driving device 120 , a charging device 140 and a main control platform 130 .
  • the sensor module 110 may include one or more visual sensors 111 (such as cameras) (this embodiment of the application may also be referred to simply as sensors), for example, it may include ordinary optical cameras, or infrared cameras, structured light sensors or Time-of-flight (ToF) sensors, etc.
  • the sensor module 110 may include an ordinary RGB camera or a red yellow yellow blue (RYYB) camera, and the camera module may also include multiple cameras or sensors to form an RGB-D depth camera solution.
  • the RGB-D depth camera solution may include two RGB cameras to form a binocular solution, may include an RGB camera and a structured light sensor to form a structured light solution, or may include an RGB camera and a ToF sensor A ToF solution is formed, which is not specifically limited in this embodiment of the present application.
  • the visual sensor 111 (for example, a camera) may be a fixed-focus camera or a zoom camera, for example, capable of phase focusing or laser focusing.
  • the visual sensor 111 (such as a camera) may be carried on a motion unit, and the motion unit is used to carry the visual sensor 111 (such as a camera) and drive the visual sensor 111 (such as a camera) to rotate.
  • the motion unit can drive the visual sensor 111 (such as a camera) to generate a two-degree-of-freedom rotation. If the direction pointed by the Z-axis is directly in front of the camera 11, the above-mentioned two-degree-of-freedom rotation can include the visual sensor 111 (such as a camera) ) rotation around the x-axis and rotation around the y-axis of the visual sensor 111 (such as a camera).
  • the motion unit drives the visual sensor 111 (such as a camera) to rotate and can be realized by the rotation of a steering gear or a servo motor.
  • the motion unit can Contains two driving mechanisms driver 1 and driver 2, such as two steering gears or two servo motors, wherein one steering gear 1 or servo motor 1 is used to control the rotation of the visual sensor 111 (such as a camera) with the x-axis as the rotation axis, Another steering gear 2 or servo motor 2 is used to control the rotation of the visual sensor 111 (such as a camera) with the y-axis as the rotation axis.
  • the motion unit can drive the visual sensor 111 (such as a camera) to generate a three-degree-of-freedom rotation, that is, to increase the rotation of the visual sensor 111 (such as a camera) with the z-axis as the rotation axis.
  • the visual sensor 111 such as a camera
  • the motion unit can also include Three driving mechanisms driver 1, driver 2 and driver 3, such as three steering gears or three servo motors, wherein one steering gear 1 or servo motor 1 is used to control the rotation of the visual sensor 111 (such as a camera) with the x-axis as the axis of rotation Rotate, another steering gear 2 or servo motor 2 are used to control the rotation of the visual sensor 111 (such as a camera) with the y-axis as the rotation axis, and another steering gear 3 or servo motor 3 is used to control the visual sensor 111 (such as a camera) to The z axis is the rotation axis.
  • driver 1, driver 2 and driver 3 such as three steering gears or three servo motors
  • the sensor module 110 may also include a motion sensor 112.
  • the motion sensor 112 may be an odometer, an accelerometer, a speedometer, an inertial measurement unit, etc., and is used to collect mileage information of the mobile charging device 13 during driving, such as travel, track, information such as speed.
  • the force sensor 113 may be a sensor for detecting force applied to the end of the robot arm 142 .
  • a pressure sensor capable of detecting force in a uniaxial direction a force sensor or a moment sensor capable of detecting force components in a plurality of axial directions can be used.
  • a six-axis force sensor can be used as the force sensor 113 .
  • the six-axis force sensor detects the magnitude of the force parallel to the three detection axes orthogonal to each other in the inherent sensor coordinate system and the magnitude of the moment around the three detection axes.
  • the force sensor 113 may be disposed at a position other than the end of the mechanical arm 142 , for example, may be disposed on more than one joint of the mechanical arm 142 .
  • the driving device 120 may include components that provide powered motion for the mobile charging device 13 .
  • drive device 120 may include an engine, energy source, transmission, and wheels/tyres.
  • the engine can be an internal combustion engine, an electric motor, an air compression engine or a combination of other types of engines, such as a hybrid engine composed of a gas oil engine and an electric motor, or a hybrid engine composed of an internal combustion engine and an air compression engine.
  • the engine converts the energy source into mechanical energy.
  • Examples of energy sources include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electrical power.
  • the energy source can also provide energy to other systems of the mobile charging device 13 .
  • the transmission transmits mechanical power from the engine to the wheels.
  • the transmission may include a differential and a drive shaft.
  • the transmission may also include other devices, such as clutches.
  • the drive shaft may include one or more axles that may be coupled to one or more wheels.
  • the main control platform 130 is the data processing and control center of the device.
  • the main control platform 130 establishes a communication connection with the charging device 140, the sensor module 110 and the driving device 120, for example, it can receive the image data collected by the sensor module 110 and process the image data. , and send a moving instruction to the driving device 120 .
  • the charging device 140 may include a charging head 141 (or called a charging plug) and a mechanical arm 142 .
  • the main control platform 130 may be an embedded computer platform, including units not limited to computer chips and software systems designed based on the X86 instruction set, ARM instruction set, RISC-V instruction set, or MIPS instruction set.
  • the above-mentioned computer chip may include, for example, a processor 131 and a memory 132, wherein, for example, the processor 131 may include: a central processing unit (central processing unit, CPU), a system chip (system on a chip, SoC), Application processor (application processor, AP), microcontroller (microcontroller), neural network processor (Neural-network Processing Unit, NPU) and/or graphics processing unit (graphics processing unit, GPU) etc.; memory 132 can include, for example Non-volatile memory and volatile memory, non-volatile memory such as flash memory (flash memory), including NAND flash, solid-state hard disk, etc., volatile memory such as synchronous dynamic random-access memory (synchronous dynamic random-access memory) memory, SDRAM), etc.
  • the processor 131 may include: a central processing unit (central processing unit, CPU), a system chip (system on a chip, SoC), Application processor (application processor, AP), microcontroller (microcontroller), neural network processor (Neural-network Processing Unit, NPU
  • the aforementioned software system may include an operating system and program instructions 133 running on the operating system.
  • the processor executes the above-mentioned program instructions, the device shown in FIG. 3 or FIG. 4 is made to execute each step of the method for controlling the charging of the robotic arm provided by the embodiment of the present application.
  • memory 132 may contain program instructions 133 (eg, program logic) executable by processor 131 to perform various functions of mobile charging device 13 , including those described above.
  • Memory 132 may also contain additional instructions, including instructions to send data to, receive data from, interact with, and/or control one or more of actuator 120 , sensor module 110 , control system, and peripheral devices.
  • memory 132 may store data such as road maps, route information, autopilot position, direction, speed, and other such autopilot data, and other information. Such information may be used by mobile charging device 13 during operation of mobile charging device 13 in autonomous, semi-autonomous, and/or manual modes.
  • the wireless communication system 150 may communicate wirelessly with one or more devices (eg, server 11 ), directly or via a communication network.
  • the wireless communication system 150 may use 3G cellular communication, such as code division multiple access (code division multiple access, CDMA), EVD0, global system for mobile communications (global system for mobile communications, GSM)/is a general packet radio service technology (general packet radio service, GPRS), or 4G cellular communication, such as long term evolution (long term evolution, LTE), or 5G cellular communication.
  • the wireless communication system 150 can use WiFi to communicate with a wireless local area network (wireless local area network, WLAN).
  • the wireless communication system 150 may communicate directly with the device using an infrared link, Bluetooth, or ZigBee.
  • Other wireless protocols, such as various autopilot communication systems for example, wireless communication system 150 may include one or more dedicated short range communications (DSRC) devices, which may include autopilot and/or roadside Public and/or private data communication between stations.
  • DSRC dedicated short range communications
  • one or more of these above-mentioned components may be separately installed or associated with the mobile charging device 13 .
  • the memory 132 may exist partially or completely separate from the mobile charging device 13 .
  • the components described above may be communicatively coupled together in a wired and/or wireless manner.
  • FIG. 4 should not be construed as a limitation to the embodiment of the present application.
  • the execution subject of the method for controlling the charging of the robotic arm in the embodiment of the present application can be the mobile charging device 13, or the server 11 on the cloud side.
  • the system shown in FIG. 5 can move the charging device 13 and the server 11,
  • the mobile charging device 13 is communicatively connected to the server 11 (for example, realized by a wireless communication system 150), wherein the mobile charging device 13 can transmit the data collected by the sensor to the server 11, and the server 11 can collect the data collected by the sensor based on the mobile charging device 13.
  • Realize the method for controlling the charging of the robotic arm in the embodiment of the present application and transmit the related processing results of controlling the charging of the robotic arm to the mobile charging device 13 .
  • FIG. 6 There are relatively large differences due to different performances, and can include one or more central processing units (central processing units, CPU) 66 (for example, one or more processors) and memory 632, one or more storage application programs 642 or data 644 storage medium 630 (for example, one or more mass storage devices).
  • the memory 632 and the storage medium 630 may be temporary storage or persistent storage.
  • the program stored in the storage medium 630 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server.
  • the central processing unit 66 may be configured to communicate with the storage medium 630 , and execute a series of instruction operations in the storage medium 630 on the server 600 .
  • the server 600 can also include one or more power supplies 626, one or more wired or wireless network interfaces 650, one or more input and output interfaces 658; or, one or more operating systems 641, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
  • the current mainstream charging robot solutions all use structured light or binocular or ultrasound combined with robotic arm planning to achieve plugging and unplugging through position control.
  • submillimeter level for example, less than or equal to 2°
  • pose positioning accuracy is required to ensure the smooth insertion of the charging socket into the charging port, and the docking does not damage the connecting device.
  • the solution based on visual feedback requires the visual detection module to meet the requirements of sub-millimeter position recognition and 2° pose recognition.
  • the basic accuracy of the current structured light technology can reach 1-2mm, the pose is about 3°, and it is also affected by the distance. Due to the limitation of field of view, it is difficult to ensure smooth plugging and unplugging. Even if it is plugged in, the existence of pose error will cause a large plugging force, which will easily damage the equipment and even damage the robotic arm. Therefore, it is difficult to achieve the precision requirement for the smooth insertion of the charging socket into the charging port only by performing plugging and unplugging based on the visual feedback technology.
  • the robotic arm for charging provided in the embodiment of the present application can realize plugging and unplugging of the charging socket to the charging port with high precision.
  • the charging device in the embodiment of the present application is introduced from the form of the product.
  • FIG. 7 is a schematic diagram of a charging scene of a charging device in an embodiment of the present application, wherein the charging device may include a mechanical arm.
  • the robotic arm can use a multi-axis serial robotic arm to realize the position and pose control of the charging gun.
  • the shape of the robotic arm is not limited to series connection, as long as it can realize the six-dimensional pose guidance of the charging gun at the end.
  • the charging device may include a force sensor (or a torque sensor), and in one possible implementation, it can sense the force and moment (Fx, Fy, Fz, Tx, Ty, Tz) on the end of the gun head, and can sense Lateral force sensors are available.
  • the charging device may include a camera capable of acquiring 2D images and/or 3D point clouds of the charging port for identifying the 6D pose of the charging stand.
  • the charging device may include a charging gun (including a charging plug) and a fixing device, and the charging device may include a mobile platform to expand the moving range of the charging device.
  • Fig. 8 is a schematic diagram of an embodiment of a charging device provided in an embodiment of the present application, wherein the mobile charging device may be a fixed charging pile, a mobile charging vehicle, a mobile charging robot and other products.
  • the charging device may include a mechanical arm 802 , wherein the mechanical arm 802 may include: a control structure and a main body of the mechanical arm 802 .
  • the control mechanism may be the processor 131 described above.
  • control mechanism can control the posture of the main body of the robotic arm 802 to realize the charging task, for example, the charging task can be: inserting the charging plug 803 at the end of the robotic arm 802 into the charging port of the vehicle.
  • the charging device can receive a charging instruction (for example, it can receive a charging instruction sent from a server, or directly receive a charging instruction sent by a user's terminal device ), optionally, the charging command may carry information such as the location of the vehicle to be charged, the model information of the vehicle to be charged, and the charging speed information.
  • the charging device may start the execution of the charging task based on receiving the charging instruction, triggering the start of the moving state.
  • the charging device needs to move to the vicinity of the charging port of the vehicle to be charged before starting the operation of the robotic arm 802, ensuring that the charging port is within the reachable range of the robotic arm 802.
  • This part can use navigation positioning mode or automatic driving Technical realization.
  • the charging device can be compatible with two charging methods: fast charging gun and slow charging gun, so it can be equipped with a quick-change mechanism, and the charging gun can be selected according to the user's task requirements.
  • the magnetic suction structure is used to grab the charging gun , to ensure the reliable docking of the charging gun and the mechanical arm 802, and then proceed to the next charging operation.
  • the task of moving the chassis is basically completed, and the operation of the mechanical arm 802 is about to be performed, so it is necessary to know the charging stand (including the charging port, for the convenience of description in this embodiment, the charging stand is also It can be called the approximate position of the charging port) in order to control the mechanical arm 802 to search for the position of the charging port in this area.
  • the charging stand including the charging port, for the convenience of description in this embodiment, the charging stand is also It can be called the approximate position of the charging port
  • the approximate position of the charging port can be detected according to the vehicle model, or the vehicle model of the charged vehicle can be identified through the charging equipment camera, and the visual information of the charged vehicle can be obtained combined with the vehicle model information to calculate the approximate position of the charging port, and send this position to The robotic arm 802, the robotic arm 802 can proceed to the next step in this area, that is, the search phase.
  • the field of view of the camera such as the field of view (FOV)
  • FOV field of view
  • the search solution can be jointly implemented based on fan-shaped path planning in the kinematics of the robotic arm 802 and a target detection algorithm in visual perception.
  • FIG. 9 For an exemplary flowchart of the search process, reference may be made to FIG. 9 .
  • the search path can be generated by knowing the approximate location information of the charging port (such as the parking position of the vehicle, the height and inclination angle of the charging port, etc.), and the robot arm 802 can be controlled to complete the path following.
  • the target detection and tracking of the charging port are performed simultaneously.
  • the fan-shaped trajectory search in the embodiment of this application is introduced.
  • the height of the charging port of a new energy vehicle is distributed within the range of 70c ⁇ 100cm, so the fan-shaped trajectory search can be performed at this height in Cartesian space combined with the FOV of the camera.
  • the search process is accompanied by a real-time charging port detection algorithm.
  • the robotic arm 802 is driven to move so that the target is in the center of the field of view. This method is conducive to entering the next stage of the visual servoing process.
  • fast and high-precision target detection and tracking of the charging port can be realized based on deep learning.
  • an improved YoloV4 deep learning target detection algorithm combined with templatetracking tracking algorithm can be used as an option.
  • the improved YoloV4 algorithm mainly compresses the number of convolution layers and the number of convolution kernels in the depth of the feature extraction network in the original network structure, and the number of convolution layers of the improved model feature extraction network is reduced (for example It can be reduced from the original 73 layers to 22 layers), so the loading and inference process speed of the model has been greatly improved, and the accuracy has remained almost unchanged.
  • the improved YoloV4 target detection algorithm can accurately detect the charging port area (box coordinates) in the image (x, y, w, h), and pass the image of this area as the target area to the templatetracking tracking algorithm.
  • the tracking algorithm will calculate the area with the highest correlation coefficient with the target area in the image as the tracking result output.
  • the running speed makes up for the problem that the deep learning algorithm cannot be detected in real time due to insufficient computing power on the portable industrial computer, and realizes the real-time detection and tracking of the charging port target in a complex environment.
  • the schematic diagram of the detection result can be referred to as shown in FIG. 10
  • the box area (box) is the output of the detection result
  • the coordinates are (x, y, w, h), representing the pixel coordinates in the image.
  • the tracking effect can be referred to as shown in Figure 11.
  • the box in each frame of image is the tracking area, ensuring that there is a detection result of the charging port in this area.
  • the control mechanism can control the pose of the robotic arm 802 so that the charging port falls within the field of view of the camera (optionally, falls within the central area of the field of view of the camera). Afterwards, the control mechanism can control the pose of the robotic arm 802 based on visual servoing, so as to guide the charging plug 803 on the main body of the robotic arm 802 to the vicinity of the charging port.
  • the pose can include position and orientation
  • the position can be represented by three coordinate values in the three-dimensional coordinate system
  • the pose can be represented by the state specified by the rotation around each coordinate axis.
  • the pose determined based on the information collected by the visual sensor 804 can be the pose of the visual sensor 804 relative to the charging port, and since the position of the visual sensor 804 on the mechanical arm 802 is fixed, that is to say, the visual sensor 804 and the charging plug 803 The relative position relationship between them is fixed, so after knowing the pose of the visual sensor 804 relative to the charging port, it is equivalent to knowing the relative pose between the charging plug 803 and the charging port.
  • the pose of the charging gun in the coordinate system of the manipulator 802 and the pose in the coordinate system of the charging stand represent the same spatial pose.
  • This firstly requires knowing the pose relation of the coordinate system of the charging base relative to the coordinate system of the robot arm 802 . Since the base of the robotic arm 802 may change at any time, this pose relationship also changes dynamically.
  • the charging seat posture obtained by this method and the true value of the charging seat posture have errors, and the insertion of the charging gun cannot usually be guaranteed. Since the true value of the charging seat posture cannot be obtained by other methods, and the force sensor 801 can be installed on the mechanical arm 802 in the embodiment of the present application, the information collected by the force sensor 801 can be used to perform a force-based self-calibration method to Realize the acquisition of the true value of the charging seat, and construct a visually acquired charging seat pose compensation matrix for observation pose compensation.
  • the coordinate system of the charging plug 803 coincides with the coordinate system of the charging port, and the world coordinate system is defined as the charging device base W, the end coordinate system is defined at the end e of the charging plug 803, the charging port coordinate system is O, and the camera coordinate system is C, so it can be:
  • W T e W T 6 6 T C C T e ;
  • the correct insertion posture can be determined by the force-sensing self-calibration method Complete the interpolation pose for the correct ground truth search. Specific methods can be exemplified as follows:
  • the program controls to adjust the pose of the charging plug 803 so that, except for the axis of the charging plug 803, the sum of forces in the other two coordinate directions is the smallest.
  • the end of the robotic arm 802 can be moved to the best observation point.
  • W T e W T 6 ( ⁇ 1 ) 6 T e ;
  • the target charging port can be observed at the best observation point, and the desired charging plug 803 position can be obtained by using the following formula:
  • the expected joint angle after insertion can be solved (In the case where the main body of the robotic arm 802 is composed of multiple joints, the pose of the main body of the robotic arm 802 can be controlled through joint angles).
  • This application proposes a force sense calibration method for the relative pose error between the manipulator 802 and the charging port.
  • the zero position where the charging plug 803 of the manipulator 802 is completely inserted into the charging port is obtained through the force sense self-calibration search technology, and according to the zero position
  • a method for quantitatively compensating the relative pose error between the charging port and the manipulator 802 observed by vision is provided, so as to solve the problem of the relative pose error of the charging port and the manipulator 802 due to the collection of deformation errors of the manipulator 802, visual recognition errors, and hand-eye calibration errors.
  • the absolute pose error of 802 leads to the problem that the charging plug 803 cannot be inserted.
  • the robotic arm 802 may include a base (or called a base) and a main body of the robotic arm 802, and the main body of the robotic arm 802 may be called an arm.
  • the arm can include multiple joints, which are connected sequentially and have rotational degrees of freedom between each other.
  • a force sensor 801 and a charging socket can be installed (for example, they can be installed sequentially).
  • the control mechanism can control the pose of the main body of the robotic arm 802 based on visual servoing, so as to guide the charging plug 803 on the main body of the robotic arm 802 to the vicinity of the charging port.
  • control mechanism can obtain in real time the image information collected by the visual sensor 804 for the area of the charging port; according to the position information of the charging port in the image information, obtain the The real-time pose of the main body of the arm 802 ; according to the real-time pose, the pose of the main body of the robotic arm 802 is controlled so as to guide the charging plug 803 on the main body of the robotic arm 802 to around the charging port.
  • the real-time pose of the main body of the mechanical arm 802 can be obtained based on the image information acquired in real time (the real-time pose can be the distance between the end of the main body of the mechanical arm 802 and the charging port of the vehicle to be charged. relative pose between them), because when controlling the pose of the main body of the robotic arm 802, there may be situations such as base movement and obstruction by obstacles, resulting in errors in the pose of the main body of the robotic arm 802.
  • This application can be based on real-time Correcting the pose of the main body of the robotic arm 802 can improve the pose recognition accuracy of the main body of the robotic arm 802 , and then guide the charging plug 803 on the main body of the robotic arm 802 to the vicinity of the charging port more accurately.
  • fast visual servoing can be performed by controlling the robotic arm 802 to achieve rapid guidance to the charging stand.
  • a stable control law needs to be designed to achieve accurate and fast closed-loop control.
  • the control of the robotic arm 802 and visual recognition results need to be highly integrated from the system.
  • the angle design control algorithm makes the mechanical arm 802 move to ensure that the end charging gun can quickly approach the charging port, ready for the next step of force-controlled plugging and unplugging.
  • the pose recognition (such as 6D pose recognition) of the charging port is the most important link to realize the charging and plugging function, and accurate position recognition is the prerequisite and basis for plugging and unplugging.
  • the algorithm is required to have high robustness, such as for dim conditions in garage scenes.
  • the 6D position mainly refers to the position (x, y, z) and pose angle (Roll, Pitch, Yaw) of the charging port in the camera coordinate system at the end of the robotic arm 802; optionally, an RGBD depth camera combined with a neural network algorithm can be used to realize the 6D pose recognition of the charging port, the recognition accuracy is required to reach ⁇ 1-2mm, and the angular accuracy is within ⁇ 3° (the conversion relationship between coordinate systems can be shown in Figure 16).
  • a combination of three-dimensional key point detection and two-dimensional key point detection network is used to achieve high-precision recognition results in a low-contrast charging port detection environment.
  • visual servoing is a closed-loop control method that combines vision with motion.
  • the main function is to realize the guidance and movement of the charging gun at the end of the robotic arm 802 to the charging port.
  • the general process is to use the visual recognition pose result of the charging gun/charging stand as input, use the visual servo feedback control idea to calculate the driving error, and use the pose-based visual servo to realize the precise guidance of the end of the robotic arm 802 to the target pose.
  • the adaptive gain method is adopted to speed up the adjustment speed when the error is small, and realize fast visual servoing.
  • the robotic arm 802 will be at the best observation point, where the most accurate pose of the charging port has been obtained.
  • the mechanical arm 802 will guide the charging socket to the vicinity of the charging port to prepare for the next step of force-controlled insertion.
  • the accuracy of the target observation pose can be guaranteed.
  • the pose of the mobile robot base relative to the charging dock may change continuously.
  • the fast visual servoing of the pose is used to achieve this goal.
  • the visual servoing idea of pose feedback can be adopted, and the joint speed control law can be designed based on Li's stability criterion, so as to realize the precise guidance of the end of the mechanical arm 802 to the target position.
  • the specific implementation method is that when the system completes the search for the charging stand, the charging stand is in the field of view of the hand-eye camera, and thus the pose matrix from the charging stand to the camera can be obtained. Compared with the pose matrix of the observation point, an error difference is obtained.
  • the current pose vector in Re represents the translation vector of the charging stand coordinate system relative to the camera coordinate system
  • ⁇ u represents the direction vector of the charging coordinate system relative to the camera coordinate system
  • the desired pose vector 0 means that the orientation of the camera and the orientation of the charging coordinate system will be consistent when converged.
  • ) is a function of the error
  • ) can be constructed for fuzzy control, and the parameters of the fuzzy control law can be adjusted through experiments for adaptive gain control.
  • the value of the fuzzy rule satisfies a 1 >a 2 >a 3 >a 4 >a 5 , and the numbers a 1 ⁇ a 5 represent The situation score is performed on the error
  • the error input values are respectively substituted into the membership function to obtain the degree of membership, and defuzzification is carried out according to the principle of the maximum degree of membership. If the error input value has the largest membership degree in a certain set, the corresponding rule value is selected as the control output, and then the attenuation value becomes larger when the error is small, and the attenuation value becomes smaller when the error is large.
  • the value of the pose error differential can be obtained through the exponential decay equation, and the pose error differential can be expressed as the product of the interaction matrix and the speed of the camera:
  • the camera speed can be represented by the product of the Jacobian matrix and the joint speed of the manipulator 802, so that the servo speed of the joint can be calculated according to the pose error
  • obstacle detection can be performed around the mechanical arm 802, and the reaction force obtained according to the distance from the obstacle through the potential field method is used as the deceleration Vo.
  • the potential energy generated by the obstacle will drive the mechanical Arm 802 pushes away.
  • d is the shortest distance between the end of the robotic arm 802 and the environment, the smaller the distance, the greater the potential energy, and ⁇ is a constant. definition:
  • the above formula shows that the acceleration generated by the potential field method will point to the direction of reducing the potential energy of the potential field.
  • the total speed of the mechanical arm 802 joints can be obtained as:
  • the angular velocity can be used as a control input to send a control instruction to the robotic arm 802, thereby forming a closed-loop control, and driving the robotic arm 802 to run a visual servo control cycle.
  • the above-mentioned visual servoing keeps running until it converges, at this time, the mechanical arm 802 drives the camera to move to the best observation point.
  • the visual servoing process may be as shown in FIG. 12 .
  • the present application can employ desired pose compensation as well as compensation calculated by Jacobian in visual servoing.
  • Desired pose compensation When visual servoing converges, what is obtained is the expected observed pose that the actual pose of the camera arrives at.
  • Jacobian matrix Since the actual camera pose is used as the control parameter, the Jacobian matrix can be:
  • the main body of the robotic arm 802 is provided with the visual sensor 804 .
  • the visual sensor 804 may not be disposed on the main body of the mechanical arm 802 but disposed at a position corresponding to a third-party perspective.
  • the visual sensor 804 when determining the above relative pose, needs to collect image information for the charging port, and determine the relative position of the charging port relative to the visual sensor 804 based on the image information. posture.
  • the image information may include a 2D image and a 3D point cloud
  • the control mechanism may extract the first 3D position corresponding to the charging port according to the 2D image and the preset 3D model of the charging port ; extracting a second 3D position corresponding to the charging port according to the 3D point cloud; merging the first 3D position and the second 3D position to obtain a pose of the charging port.
  • the pose of the charging port can be understood as the relative pose of the charging port relative to the visual sensor 804 .
  • the vision sensor 804 on the charging device may be a depth sensor for collecting depth images.
  • the depth sensor can be introduced, for example but not limited to, as follows: a depth camera, a time of flight (time of fight, TOF) camera, or a lidar, a photo scanner, or a lidar detection and ranging (LiDAR) sensor.
  • the control mechanism can collect 3D point cloud through but not limited to the following methods:
  • A. Obtain a 3D point cloud based on the acquired RGB image and depth image.
  • the control mechanism can collect RGB images and depth images through the sensors deployed on the main body of the mechanical arm 802.
  • the RGB images and depth images refer to the surrounding environment that the sensor can perceive at the current moment and current position.
  • RGB image and depth image may be collected by a depth camera deployed on the charging device, or the depth image may be collected by a depth sensor deployed on the charging device, and the depth image may be collected by a depth sensor deployed on the charging device.
  • the camera module (for example, a camera) collects corresponding RGB images.
  • the sensor After the sensor collects the RGB image and depth image of the charging device at the current moment, it first performs instance segmentation on the RGB image. For example, it can use SOLOv2, BlendMask and other algorithms to perform instance segmentation on the RGB image, and output the instance segmentation results. Including the classification category to which each region on the RGB image belongs, the confidence degree (not greater than 1) belonging to the classification category, and the segmentation mask (mask), the instance segmentation result can be defined as Mi. After that, the data belonging to the charging port can be selected, and the RGB image (which can be called the processed RGB image) with the instance segmentation and pixel value redefined (which can be called the processed RGB image) is superimposed on the depth image. Each local area has depth information.
  • the processed RGB image occupies a defined pixel of 1 on the RGB image, and the defined pixel of the rest of the area is 0, then the processed The RGB image is multiplied by the depth image. Since the area with a pixel value of 0 is still 0 after multiplication, the result of the multiplication only retains the area corresponding to each area in the RGB image and the depth information corresponding to each area. Afterwards, the 3D point cloud of the charging port is recovered through the internal reference of the camera.
  • filtering operations can also be performed on the instance segmentation result and the depth image first, such as reducing outliers in the instance segmentation result through morphological filtering, and eliminating outliers in the depth image through depth smoothing. Hole parts, etc.
  • the purpose of filtering is to remove noise.
  • the charging device can collect RGB images and raw laser point cloud data through sensors deployed on the charging device.
  • sensors such as lidar and ordinary cameras can be deployed on the charging device.
  • the original laser point cloud data is collected through the lidar. In this case, it is not necessary to obtain the depth image. It is only necessary to perform instance segmentation on the RGB image according to the above-mentioned similar process, and then correspond to the instance segmentation results to the simultaneously obtained Based on the original laser point cloud data, the 3D point cloud corresponding to the charging port can be obtained.
  • the visual sensor 804 may be used to obtain two-dimensional images and three-dimensional depth point cloud data.
  • the two-dimensional images can be obtained by detecting key points and corresponding to the key points of the same name in the three-dimensional model.
  • the 6D pose of the target but this relies on high-precision detection of two-dimensional key points, and will introduce reconstruction errors when restoring 6D information through 2D-3D correspondence.
  • There are weak light conditions in the garage scene the depth of different charging ports is also different, and there is also a certain color difference.
  • the 3D point cloud can also be matched with the 3D model of the target object to obtain the 6D pose of the target, but the accuracy of the point cloud is slightly worse.
  • the feature extractor composed of layers 1 to 18 can be used to extract feature maps with high representation ability on the image, and then use three fully connected layers to convert the feature map set into a vector with a dimension of 14 .
  • MSE mean square error
  • the three-dimensional coordinates of seven key points in the charging port coordinate system can be obtained as shown in the following table:
  • Table 4 The three-dimensional coordinates of the seven key points of the charging port in the charging port coordinate system
  • the desired key point may not be observed.
  • the direction vector from each existing point to the key point is detected, and The precise coordinates of the key points are obtained indirectly through voting.
  • several standard key points are selected from the 3D point cloud model, which are obtained by the farthest point sampling method.
  • the specific method is to obtain the first point by random sampling for the first time, and select the point farthest from the selected point set from the remaining points each time, so as to obtain N points as standard key points .
  • the training data set is generated from the standard data key points and the artificially generated transformed point cloud for training.
  • the input point cloud is first feature extracted through the PointNet++ network.
  • Loss ⁇ R,t ( ⁇
  • P 2d is the key point of image detection
  • P 3dgt is the three-dimensional point coordinates in the charging port coordinate system
  • R is the transfer matrix from the charging port coordinate system to the camera coordinate system
  • P p3d is the three-dimensional point coordinates in the camera coordinate system
  • the 3D coordinates of the key points detected in the point cloud and the 3D coordinates of the key points in the P p3dgt 3D model.
  • the algorithm can calculate the initial value of R and t through the 2D-3D correspondence of the image, and then use the LM method to obtain the solution of the optimal pose matrix through gradient descent (for example, as shown in Figure 13).
  • the 6D pose (position and attitude) recognition of the charging stand is a key link in the unmanned charging task. Only by ensuring that the 6D pose of the charging port is accurate can the precise guidance of the robotic arm be carried out, and then force control technology is used to carry out Smooth plugging and unplugging. Since the standard slow charging charging port consists of seven holes, the center of the seven holes of the charging port is selected as the key point, and the key point detection is performed on the target area above through the artificial intelligence algorithm. This scheme transforms the detection problem of 7 key points on a two-dimensional image into a regression problem of 14 values, that is, a 14-dimensional vector composed of 7 key point coordinates (x, y).
  • the input of the final network is a color image, and the input is a vector with a dimension of 14, which is the coordinate values of 7 key points on the image.
  • the farthest point sampling method is used to down-sample the point cloud. In addition to the first random sampling, this method selects the point farthest from the selected point set from the remaining points every time.
  • the point after sampling As a down-sampling version of the dense point cloud, and perform three-dimensional registration with the standard point cloud to obtain the corresponding sequence of three-dimensional points, and use the distance between the standard three-dimensional key points and the key points of the target coordinate system after R, t transformation as the three-dimensional objective function ;
  • the improved VGG two-dimensional key point detection network is used to obtain the two-dimensional pixel coordinates of the center point of the charging stand hole, and the number of each point is obtained through the mutual positional relationship of the center points (for example, as shown in Figure 15); according to the charging stand standard
  • the CAD model obtains the three-dimensional physical coordinates of the center point of the circular hole in the target coordinate system.
  • the two-dimensional key pixel points (under the pixel coordinate system) and the three-dimensional physical points (under the target coordinate system) correspond to each other through the numbering sequence.
  • the distance between the two-dimensional key point and the back-projection point is a two-dimensional loss function term;
  • the two functions are weighted by ⁇ and ⁇ according to the errors of two-dimensional detection and three-dimensional detection, respectively, to obtain the total loss function.
  • This application proposes a method for accurately identifying the position and posture of the charging port that combines 2D image-3D point cloud information.
  • the 2D depth image detection network can be used to perform sub-pixel-level accurate detection of the two-dimensional key points of the charging port, and the three-dimensional key points can be used
  • the detection neural network conducts reliable detection of the offset of the three-dimensional key points of the charging port based on the 3D Hough network, and uses nonlinear optimization to minimize the projection error of the fusion of two-dimensional key points and three-dimensional key points to obtain the precise position of the charging port. pose, in order to solve the problem of high-precision recognition of the pose of the charging port under the condition of large illumination changes and low target contrast.
  • the charging plug 803 can be guided to the vicinity of the charging port, for example, the accuracy range is 1 to 2mm, within the error range of 2°. Since the previous planning has a certain error based on vision, and considering the safety of collision, this At this time, the manipulator 802 cannot be in contact with the charging port, so it is necessary to perform a movement according to the visual information, so that the force sensor installed on the main body of the manipulator 802 can sense the contact with the charging port, and then the adaptive adjustment stage based on force perception can be performed (For example, as shown in FIG. 18 can be referred to).
  • This application proposes a visual servo guidance method with adaptive error gain, and proposes the generation of visual servo velocity commands combined with potential field obstacle avoidance.
  • visual servoing needs to interact with the environment, but it cannot collide with the environment and cause damage.
  • a new repulsion speed factor is introduced by combining the potential field obstacle avoidance method, which effectively guarantees the security of visual servoing.
  • the second is to propose a visual servo control law with adaptive gain. By setting the reverse fuzzy relationship between the attenuation index and the error amplitude, a large attenuation coefficient under small errors is realized, which effectively speeds up the convergence speed of visual servoing.
  • the third is to use the visual servo based on the position and orientation.
  • the position and orientation information is used as the feature vector, it contains more and more accurate information than the visual servo based on the image, which can effectively guide the mechanical arm 802 to the desired optimal position.
  • the above-mentioned method solves the problem of non-collision guidance camera staying at the best observation point with a relatively fixed pose when the pose of the robotic arm 802 and the charging port may change constantly.
  • the control mechanism can acquire force information collected by the force sensor 801, the force information can indicate the force (or moment) of the charging port on the charging plug 803, and according to the force information to control the pose of the main body of the robotic arm 802 so as to insert the charging plug 803 into the charging port.
  • the visual camera can no longer observe the charging port. If only the best observation point is used When plugging and unplugging the charging plug 803 based on the observed visual pose, it may not be able to be inserted to the bottom or wedged tightly.
  • the position and posture are adjusted according to the sense of force, so that the charging plug 803 adapts to the insertion direction of the charging port, thereby achieving the accuracy and reliability of the plugging task.
  • the main body of the mechanical arm 802 is provided with a force sensor 801, and the force sensor 801 is used to collect force information on the contact surface between the charging plug and the charging port, and the contact surface It includes a side surface of at least one charging head among the plurality of charging heads, and the side surface is a surface parallel to the axial direction of the charging head.
  • the force information includes the force in the radial direction of the charging head.
  • the force information may include 6-dimensional force information.
  • the force information includes torque information of the charging port relative to the charging plug.
  • the force information may indicate the 6-dimensional force of the charging port on the charging plug, and the 6-dimensional force may include forces in three-dimensional space (for example, along the x-axis, y-axis, and z-axis directions) and forces in three-dimensional space (such as rotation about the x-axis, y-axis, z-axis). Since the torque information is included, it is possible to more accurately and fine-grainedly identify the blocking direction of the charging port for the charging head, and then provide more precise force control to accurately insert the charging head into the charging hole.
  • the force sensor 801 is fixed on the charging plug 803 (it may be on the side of at least one charging head); or, the mechanical arm 802 includes a joint with a degree of freedom of rotation, so
  • the control structure is used to control the pose of the main body of the mechanical arm 802 by controlling the rotation of the joint, and the force sensor 801 is fixed on the joint; or, the force sensor is fixed on the charging plug and the Between the main body of the robot arm.
  • the force sensor 801 may have one or more force detection units, an A-D conversion circuit and a signal processing circuit.
  • the force detection units each include a force detection element and an amplification circuit that amplifies an output signal of the force detection element.
  • the amplification circuit supplies the amplified signal Va to the A-D conversion circuit.
  • the A-D conversion circuit supplies the digitally converted signal Vd to the signal processing circuit.
  • the number of force detection units is set according to the number of detection axes detected by the force sensor 801 and the type of force on each detection axis. For example, the force sensor 801 uses twelve force detection units when detecting the magnitude of force parallel to three detection axes orthogonal to each other and the magnitude of moment around the three detection axes.
  • the force detection element for example, various elements such as piezoelectric, strain gauge or optical can be used.
  • the amplifying circuit is configured as an integrating circuit that integrates the electric charge output from the force detecting element and converts it into a voltage signal Va.
  • the amplifying circuit has an operational amplifier, a capacitor, and a switching element.
  • the negative input terminal of the operational amplifier is connected to the electrode of the force detection element, and the positive input terminal of the operational amplifier is grounded to the ground (reference potential point).
  • the output terminal of the operational amplifier is connected to the A-D conversion circuit.
  • the capacitor is connected between the negative input terminal and the output terminal of the operational amplifier.
  • the switching element is connected in parallel with the capacitor. The switching element performs a switching operation in accordance with an on/off signal supplied from a signal processing circuit.
  • the switch element when the switch element is turned off, the charge output from the force detection element is stored in the capacitor, and its voltage Va is output to the A-D conversion circuit.
  • the switching element when the switching element is turned on, the two terminals of the capacitor are short-circuited. As a result, the charge stored in the capacitor is discharged and becomes zero, and the voltage Va output to the A-D conversion circuit becomes 0 volts.
  • the operation of turning on the switching element corresponds to resetting of the force sensor 801 .
  • the force conversion unit may convert the outputs of the multiple force detection units into F signals (i) representing forces on the multiple detection axes and output them.
  • the force signal F(i) is also simply referred to as "force F(i)".
  • these forces F(i) include forces Fx, Fy, Fz parallel to the three detection axes orthogonal to each other and moments Tx, Ty, Tz around the three detection axes.
  • the force F(i) also includes the force Fx, Fy, Fz parallel to the three detection axes as the magnitude of the force vector Fmag of the three axis direction components and the torque Tx, Ty, Tz around the three detection axes as three The magnitude Tmag of the moment vector of each axial component.
  • the size Fmag of the force vector is the square root of the quadratic sum of the three forces Fx, Fy, Fz, and the size Tmag of the moment vector is the square root of the quadratic sum of the three torques Tx, Ty, Tz (x, y, z axis
  • the schematic diagram can refer to Figure 19).
  • the charging plug 803 includes multiple charging heads, the charging port includes multiple charging holes, and the multiple charging heads match the multiple charging holes one by one;
  • the control structure can determine the pose adjustment parameters according to the force information, and correct the pose of the main body of the mechanical arm 802 from the first pose to the second pose according to the pose adjustment parameters; wherein, the The first pose is different from the second pose, and the second pose is the pose of the main body of the robotic arm 802 when each charging head is aligned with a matching charging hole.
  • the main body of the robotic arm 802 is in a correct pose (for example, the second pose may be a correct pose). For example, when the force information indicates that the charging port exerts pressure in a non-axial direction on the charging plug 803, it may be determined that the main body of the mechanical arm 802 is not in the correct posture.
  • at least one of the multiple charging heads A charging head is not aligned with the matching charging hole, that is, the central axis of the charging head does not coincide with the central axis of the matching charging hole or the deviation is greater than the threshold.
  • a, b, c, and d in FIG. 22 can be referred to.
  • the first posture is that part of the charging plug 803 has been inserted into the charging port, and at least one charging head of the plurality of charging heads is not aligned with the matching charging hole When is the pose of the main body of the robotic arm 802.
  • the first posture is the posture when the bottom end faces of the plurality of charging heads are not in contact with the bottom of the charging hole, and the bottom end faces are the axes of the charging heads. A surface perpendicular to the direction.
  • the alignment includes: the projection range of the charging head to the direction of the charging hole is within the range of the matching charging hole, and the central axis of the charging head coincides with or deviates from the central axis of the matching charging hole less than the threshold.
  • the first pose may be the case where the central axis of the charging head does not coincide with the central axis of the matching charging hole or the deviation is greater than a threshold (that is, not in the correct pose as described above).
  • the first posture is the posture when the bottom end faces of the plurality of charging heads are not in contact with the bottom of the charging hole, and the bottom end faces are the axes of the charging heads. A surface perpendicular to the direction.
  • the pose of the main body of the robotic arm 802 may be adjusted based on the force information, so that the main body of the robotic arm 802 is in the correct pose.
  • the control mechanism can correct the pose of the main body of the robotic arm 802 (to the second pose, that is, the correct position) according to the force information when the main body of the robotic arm 802 is in the first pose. pose), the second pose is the pose where the main body of the robotic arm 802 can insert the charging plug 803 into the bottom of the charging port in a correct way with less resistance.
  • FIG. 22 where a, b, c, and d in FIG. 22 can be regarded as the first pose, and e can be regarded as the second pose.
  • the resistance of the charging port to the charging head in the target direction may have a tendency to decrease, and the target direction may be in a radial direction and/or an axial state (for example Can be a radial direction, an axial state, or both a radial direction and an axial direction).
  • the so-called decreasing trend here can be understood as the resistance of the charging port to the charging head in the target direction may not decrease all the time, but there may be a local increase. Resistance in the direction of the target is reduced.
  • the moment between the charging port and the charging plug tends to decrease.
  • the three curves in Fig. 23 are the resultant forces received by the charging plug 803 in x, y, and z directions. It can be seen from the contact force curve that after using the force control algorithm, the contact force in the x and y directions gradually decreases. , as the insertion process is reduced to near zero, it shows that the adaptive pose adjustment greatly reduces the pressure on the gun body generated by the charging port in the direction of the x and y axes. After using the force control algorithm, the insertion contact force in the z direction is changed from the initial moment 80N gradually decreased to 60N, indicating that after the self-adaptive adjustment stage, the insertion force gradually decreased and remained stable until reliable insertion. It can be seen from the force curve that during the entire insertion process, the charging plug 803 performs rapid adaptive pose adjustment according to force perception, and the resultant force in all directions is in a convergent state, ensuring better compliance characteristics.
  • control mechanism can determine the posture adjustment parameters based on the force information, and the posture adjustment parameters can be specifically used to control the moving direction and/or moving speed of the charging plug 803 (such as moving direction, speed of movement, or direction of movement and speed of movement).
  • control mechanism determines the pose adjustment parameters based on the force information, and controls the pose of the main body of the robotic arm 802 based on the pose adjustment parameters.
  • direct force feedback combined with improved admittance control technology can be used to adjust the compliance parameters by referring to the principle of the remote compliance center equipment, so as to realize six degrees of freedom in the process of inserting and pulling out charging plugs 803 of different types
  • the soft feature ensures a reliable, safe and fast plug-in operation for the charging port.
  • the insertion process can be divided into two stages, the free movement from the docking point to the charging port, and the smooth insertion movement from the charging port to the bottom.
  • the position servo control is used to accurately send the plug into the charging port; then, the admittance control is used to generate the soft motion of the end of the charging plug 803, and the mechanical arm 802 is obtained through force/torque feedback and the designed admittance matrix.
  • the amount of correction to the tool tip velocity thus changing the ideal insertion velocity to produce a compliant insertion behavior.
  • information such as visually perceived pose information, force, moment, position, and speed is fused to achieve reliable insertion and removal.
  • the plugging and unplugging link is the core link of intelligent charging. Since it involves the physical contact between the charging plug 803 and the charging port, there is a change in force, although the vision of the pre-procedure process has given a more accurate The target pose of the charging port, but there is still a certain error that will cause a large contact force. At the same time, the force interaction due to different machining accuracy, fatigue degree, wear degree, etc. during the insertion process is more complicated. Therefore, in order to ensure the safety of the final insertion process To be reliable, active compliance control based on force perception is necessary.
  • the entire force control plugging and unplugging process should take over the pre-order visual information, and judge the contact state in combination with the motion state and force perception, so as to ensure that the active compliance control algorithm can be used to realize the self-adaptive adjustment of the charging plug 803 head after insertion.
  • the active compliant plug-in coordinate system can be defined as follows: the coordinate origin is located at the geometric center of the end face of the charging plug 803, the z-axis is perpendicular to the end face and points to the insertion direction of the charging plug 803, the y-axis is located inside the end face and faces upward, and the x-axis is drawn from the right hand The law is fixed.
  • V ideal [0,0,v z ,0,0,0] T , that is, a linear motion along the z-axis at the speed v z . If the contact force/moment is located, the actual charging plug 803 reference velocity is calculated in Cartesian space according to the following formula:
  • V actual V ideal + A admittance W contact ;
  • V actual is the actual reference speed of the charging plug 803
  • W contact is the contact force/torque
  • a admittance is the admittance matrix.
  • the principle of the charging device implementing the admittance algorithm is shown in Figure 20.
  • Select the admittance matrix A admittance calculate the flexible speed correction value of the charging plug 803 in the Cartesian space according to the contact force/torque measurement given by the force/torque sensor, and add it to the ideal insertion speed to obtain the actual value of the charging device in the Cartesian space Reference speed.
  • the reference velocity of each joint is obtained by multiplying it by the inverse Jacobi matrix, so as to control the charging device to produce compliant insertion motion in terms of velocity representation.
  • the RCC device can adjust the posture of the plug through its own flexibility to achieve alignment with the jack and avoid excessive contact force/torque.
  • a admittance has the form of the admittance matrix of the RCC device, i.e.,
  • k is the stiffness of the three symmetrical springs of standard RCC equipment
  • ⁇ is the angle between the central axis of the regular tetrahedron formed by the three symmetrical springs and the side where the springs are located
  • l is the distance between the far center of the compliance and the end face of the charging plug 803
  • Figure 21 is a schematic diagram of a standard RCC device.
  • the charging plug 803 produces a compliant behavior similar to the RCC device, realizes a compliant insertion process, and avoids excessive contact force, jamming, and blocking.
  • the force perception model and the adaptive adjustment of the admittance control parameters are combined to realize the self-adaptive adjustment of the TCP terminal pose and speed according to the (force and moment) insertion force and friction force, and realize the high tolerance characteristic for position error , improve the success rate of plugging and unplugging, and at the same time, the soft feature realizes the "minimum” plugging force, improves plugging reliability and safety, has strong parameter self-adaptive ability, and can effectively guarantee the completion time of plugging and unplugging; fusion of visual end position perception As well as insertion force perception, it solves the "full insertion" mathematical representation model, effectively judging whether it is fully inserted, and improving task reliability; similarly, the extraction process uses soft force control technology to achieve reliable extraction in reverse, ensuring that the overall insertion task is reliable sex.
  • Figure 22 shows the relationship between the tip of the charging plug 803 and the charging port during the insertion process of the adaptive active pose adjustment.
  • State a is the docking position of the charging plug 803 head given by the visual servo. It can be seen that there is a certain position and position Attitude error (schematic diagram, the actual error value is small), and then through free movement (here, a certain search path design can also be carried out to enhance the contact range and speed) to reach the b state, that is, contact detection. After the contact state detection, it will enter the pose.
  • the self-adaptive adjustment stage is shown in c, and finally reaches the fully reliable insertion state of e.
  • the charging plug 803 is adaptively adjusted according to the contact state and contact, thereby ensuring that the insertion process continuously reduces friction, reduces lateral resistance, and ensures reliability and safety.
  • control structure may control the pose of the main body of the robotic arm 802 after correcting the pose of the main body of the robotic arm 802 from the first pose to the second pose, so that The charging plug 803 moves toward the bottom of the charging hole and along the central axis of the charging head.
  • the charging plug 803 After correcting the posture of the main body of the mechanical arm 802 from the first posture to the second posture, the charging plug 803 is already in the alignment posture with the charging port. In this case, the charging plug 803 can be directed toward the the bottom of the charging hole and move along the direction of the central axis of the charging head to the bottom.
  • the control structure may instruct the charging port to only There is resistance along the axial direction of the charging plug 803 , and the main body of the mechanical arm 802 is controlled to remain in a static state, thereby completing the task of inserting into the charging port.
  • the reverse movement of the insertion process can be performed, and the active flexible technology is also used to adaptively adjust the posture of the charging plug 803 during the extraction process.
  • the mechanical arm 802 will return to its original position according to the settings, ensuring that the mechanical arm 802 shrinks to a smaller space and avoiding collisions with other vehicles and other objects during the movement of the chassis.
  • the plugging and unplugging task is completed. According to the system task setting, the charging device moves to the designated location. Waiting for the next mission command.
  • the main body of the mechanical arm further includes a target interface, and the target interface is used for detachably connecting the charging plug.
  • the target interface may be a component for detachable connection of the cleaning tool based on electromagnetic properties.
  • the magnetic suction structure is used to grab the charging gun to ensure the reliable docking of the charging plug and the mechanical arm, and then proceed to the next charging operation.
  • the target interface is connected to the initial charging plug; the control structure is also used to control the According to the posture of the main body of the manipulator and the connection state of the target interface, the initial charging plug is removed from the target interface and replaced with the charging plug; wherein, the target charging plug supports fast charging A charging plug, the charging plug is a plug supporting slow charging; or, the target charging plug is a plug supporting slow charging, and the charging plug is a plug supporting fast charging.
  • the charging device can be compatible with two charging methods of fast charging gun and slow charging gun, can be equipped with a quick-change mechanism, and can select a charging gun according to the user's task requirements.
  • the charging device further includes: a plug storage area, where the plug storage area includes a target storage position for setting the charging plug; the control mechanism can control the connection state of the target interface, and the The initial charging plug is disassembled from the target interface; controlling the posture of the main body of the mechanical arm, moving the target interface to the target storage position, and controlling the connection status of the target interface; The target interface is connected with the charging plug.
  • the present application provides a charging device, which is characterized in that it includes: a mechanical arm, a force sensor and a charging plug; the mechanical arm includes a control structure and a main body of the mechanical arm; the main body of the mechanical arm is provided with the force sensor and the The charging plug, the charging plug includes a plurality of charging heads, the force sensor is used to collect force information on the contact surface between the charging plug and the charging port, the contact surface includes the plurality of charging heads The side of at least one charging head, the side is a surface parallel to the axial direction of the charging head; the control structure is used to control the pose of the main body of the mechanical arm according to the force information, so as to The charging plug is inserted into the charging port.
  • the visual camera can no longer observe the charging port. If only observation at the best observation point is used Plugging and unplugging the charging plug based on the obtained visual pose may cause the situation that it cannot be inserted to the bottom or wedged tightly.
  • This application controls the pose of the robotic arm by obtaining the force information collected by the force sensor. Partially inserted (for example, the side is in contact with the charging port), and the visual information cannot obtain errors, adjust the position and posture according to the force sense, so that the charging plug adapts to the insertion direction of the charging port, thereby improving the accuracy and reliability of the plugging task. sex. It reduces the wear of the charging head due to the extrusion or friction of the side of the charging head and the inner wall of the charging port (or the situation that it is stuck and cannot reach the bottom).
  • the embodiment of the present application can effectively reduce the requirement for visual positioning accuracy and reduce the cost through fusion of vision and force sense. This is because the force compliance control can adjust the pose of the charging gun according to the force feedback situation. Even if there is an error between the charging gun and the charging stand, the force feedback can be used to identify and adjust the pose of the charging gun to eliminate the identification of the charging stand. Adverse effects of errors. Therefore, the fusion of vision and force avoids the use of expensive structured light cameras, reduces the cost and reduces the volume and weight of the end mechanism. In addition, the embodiment of the present application also overcomes the influence of dragging cables.
  • the method combined with force sense can quantitatively observe the influence of the dragging cable force through identification and apply force compensation during control, effectively overcoming the effect of dragging the cable .
  • the charging port can be quickly found by using the visual method, and the high-precision recognition of the charging seat posture can be realized by fusing 2D-3D information, which reduces the whole process time from the robotic arm 802 to the charging port. At the same time, due to the high visual accuracy, it also reduces the force control search The correct insertion and removal time of the pose greatly improves the speed of the whole process.
  • the embodiment of the present application also provides a method for controlling the charging of the robotic arm.
  • the method can be applied to the control structure of the robotic arm 802 described in the above embodiments.
  • the main body of the robotic arm 802 is provided with a force sensor 801 and A charging plug 803, the charging plug includes a plurality of charging heads, the force sensor is used to collect force information on the contact surface between the charging plug and the charging port, the contact surface includes the plurality of charging heads
  • the side of at least one of the charging heads, the side is a surface parallel to the axial direction of the charging head;
  • the methods include:
  • the pose of the main body of the robotic arm 802 is controlled so as to insert the charging plug 803 into the charging port.
  • the force information includes torque information of the charging port relative to the charging plug.
  • the force information includes the force in the radial direction of the charging head.
  • the force sensor is fixed on the side of the at least one charging head; or,
  • the mechanical arm includes a joint with a rotational degree of freedom
  • the control structure is used to control the pose of the main body of the mechanical arm by controlling the rotation of the joint, and the force sensor is fixed on the joint; or,
  • the force sensor is fixed between the charging plug and the main body of the mechanical arm.
  • the charging port includes a plurality of charging holes, and the plurality of charging heads are matched with the plurality of charging holes;
  • the pose of the arm body including:
  • the pose adjustment parameters correct the pose of the main body of the mechanical arm from a first pose to a second pose; wherein, the first pose is different from the second pose, and the first pose is different from the second pose, and the first pose is different from the second pose.
  • the second pose is the pose of the main body of the robotic arm when each charging head is aligned with a matching charging hole.
  • the first posture is the posture when the bottom end faces of the plurality of charging heads are not in contact with the bottom of the charging hole, and the bottom end faces are the axes of the charging heads. A surface perpendicular to the direction.
  • the alignment includes: the projection of the charging head along the direction of the matching charging hole falls within the range of the matching charging hole, and the central axis of the charging head is aligned with the matching charging hole.
  • the central axes coincide or the deviation is less than the threshold.
  • the method further includes:
  • the pose of the main body of the mechanical arm is controlled so that the charging plug moves toward the bottom of the charging hole and along the central axis of the charging head.
  • the resistance of the charging port to the charging plug in the target direction tends to decrease, and the target direction is the axial direction or radial direction of the charging head.
  • the moment between the charging port and the charging plug tends to decrease.
  • the method also includes:
  • the main body of the mechanical arm is controlled to maintain a static state.
  • the pose adjustment parameters are specifically used to control a moving direction and a moving speed of the charging plug.
  • the method also includes:
  • the image information collected by the visual sensor for the area of the charging port is acquired in real time;
  • the main body of the robotic arm is provided with the vision sensor.
  • the image information includes a 2D image and a 3D point cloud
  • the method further includes:
  • the 2D position and the 3D position are fused to obtain the position information of the charging port.
  • the main body of the mechanical arm further includes a target interface, and the target interface is used for detachably connecting the charging plug.
  • the target interface is connected to an initial charging plug
  • the method further includes: controlling the posture of the main body of the robot arm and the connection state of the target interface, removing the initial charging plug from the target interface and replacing it with the charging plug; wherein,
  • the target charging plug is a plug supporting fast charging, and the charging plug is a plug supporting slow charging; or,
  • the target charging plug is a plug supporting slow charging, and the charging plug is a plug supporting fast charging.
  • the charging device further includes: a plug storage area, where the plug storage area includes a target storage position for setting the charging plug;
  • the controlling the pose of the main body of the manipulator and the connection state of the target interface, and removing the initial charging plug from the target interface and replacing it with the charging plug includes:
  • the charging equipment is a fixed charging pile, a mobile robot or a charging car.
  • control mechanism For the description of the control mechanism, reference may be made to the description of the control structure of the robotic arm 802 in the above-mentioned embodiments, which will not be repeated here.
  • the embodiment of the present application also provides a device for controlling the charging of the robotic arm.
  • the device is applied to the control structure of the robotic arm 802.
  • the main body of the robotic arm is provided with a force sensor and a charging plug, and the charging plug includes a plurality of charging heads.
  • the force sensor is used to collect force information on the contact surface between the charging plug and the charging port, the contact surface includes a side surface of at least one charging head among the plurality of charging heads, and the side surface is and A surface parallel to the axial direction of the charging head;
  • the devices include:
  • the control module is configured to control the pose of the main body of the mechanical arm 802 according to the force information, so as to insert the charging plug 803 into the charging port.
  • the force information includes the force in the radial direction of the charging head.
  • the force information includes torque information of the charging port relative to the charging plug.
  • the force sensor is fixed on the side of the at least one charging head; or,
  • the mechanical arm includes a joint with a rotational degree of freedom
  • the control structure is used to control the pose of the main body of the mechanical arm by controlling the rotation of the joint, and the force sensor is fixed on the joint; or,
  • the force sensor is fixed between the charging plug and the main body of the mechanical arm.
  • the charging port includes multiple charging holes, and the multiple charging heads match the multiple charging holes one by one;
  • the control module is specifically used for:
  • the pose of the main body of the mechanical arm 802 is corrected from the first pose to a second pose; wherein, the first pose is different from the second pose, and the The second pose is the pose of the main body of the robotic arm 802 when each charging head is aligned with a matching charging hole.
  • the first posture is the posture when the bottom end faces of the plurality of charging heads are not in contact with the bottom of the charging hole, and the bottom end faces are the axes of the charging heads. A surface perpendicular to the direction.
  • the alignment includes: the projection of the charging head along the direction of the matching charging hole falls within the range of the matching charging hole, and the central axis of the charging head is aligned with the matching charging hole.
  • the central axis coincides or the deviation is less than the threshold.
  • control module after correcting the pose of the main body of the robotic arm 802 from the first pose to the second pose, the control module is further configured to:
  • the posture of the main body of the mechanical arm 802 is controlled so that the charging plug 803 moves toward the bottom of the charging hole and along the central axis of the charging head.
  • the resistance of the charging port to the charging head in the target direction tends to decrease, and the target direction is the axial direction or radial direction of the charging plug 803 .
  • the moment between the charging port and the charging plug tends to decrease.
  • control module is also used for:
  • the main body of the mechanical arm 802 is controlled to remain in a static state.
  • the pose adjustment parameters are specifically used to control the moving direction and moving speed of the charging plug 803 .
  • the device also includes:
  • An acquisition module configured to acquire in real time the image information collected by the visual sensor 804 for the area of the charging port before controlling the pose of the main body of the robotic arm 802 according to the force information;
  • the control module is also used for:
  • the pose of the main body of the robotic arm 802 is controlled so as to guide the charging plug 803 on the main body of the robotic arm 802 to the vicinity of the charging port.
  • the main body of the robotic arm 802 is provided with the vision sensor 804 .
  • the image information includes 2D images and 3D point clouds
  • the acquisition module is specifically used for:
  • the 2D position and the 3D position are fused to obtain the position information of the charging port.
  • the main body of the mechanical arm further includes a target interface, and the target interface is used for detachably connecting the charging plug.
  • the target interface is connected to an initial charging plug
  • the method further includes: controlling the posture of the main body of the robot arm and the connection state of the target interface, removing the initial charging plug from the target interface and replacing it with the charging plug; wherein,
  • the target charging plug is a plug supporting fast charging, and the charging plug is a plug supporting slow charging; or,
  • the target charging plug is a plug supporting slow charging, and the charging plug is a plug supporting fast charging.
  • the charging device further includes: a plug storage area, where the plug storage area includes a target storage position for setting the charging plug;
  • the controlling the pose of the main body of the manipulator and the connection state of the target interface, and removing the initial charging plug from the target interface and replacing it with the charging plug includes:
  • the charging equipment is a fixed charging pile, a mobile robot or a charging car.
  • the device 2400 for controlling the charging of the robotic arm 802 includes: a receiver 2401, a transmitter 2402, a processor 2403, and a memory 2404 (the number of processors 2403 in the device 2400 for controlling the charging of the robotic arm 802 can be one or more, One processor is taken as an example in FIG. 24 ), where the processor 2403 may include an application processor 24031 and a communication processor 24032 .
  • the receiver 2401 , the transmitter 2402 , the processor 2403 and the memory 2404 may be connected through a bus or in other ways.
  • the memory 2404 may include read-only memory and random-access memory, and provides instructions and data to the processor 2403 .
  • a part of the memory 2404 may also include a non-volatile random access memory (non-volatile random access memory, NVRAM).
  • NVRAM non-volatile random access memory
  • the memory 2404 stores processors and operating instructions, executable modules or data structures, or their subsets, or their extended sets, wherein the operating instructions may include various operating instructions for implementing various operations.
  • the processor 2403 controls the operation of the radar system (including the antenna, receiver 2401 and transmitter 2402).
  • various components of the radar system are coupled together through a bus system, where the bus system may include not only a data bus, but also a power bus, a control bus, and a status signal bus.
  • the various buses are referred to as bus systems in the figures.
  • the method (shown in FIG. 7 ) for controlling the charging of the robotic arm 802 disclosed in the above embodiments of the present application may be applied to the processor 2403 or implemented by the processor 2403 .
  • the processor 2403 may be an integrated circuit chip, which has a signal processing capability.
  • each step of the above-mentioned method may be completed by an integrated logic circuit of hardware in the processor 2403 or instructions in the form of software.
  • the above-mentioned processor 2403 can be a general-purpose processor, a digital signal processor (digital signal processing, DSP), a microprocessor or a microcontroller, and can further include an application-specific integrated circuit (application specific integrated circuit, ASIC), field programmable Field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • the processor 2403 may implement or execute various methods, steps, and logic block diagrams disclosed in the embodiments of the present application.
  • a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
  • the steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor.
  • the software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register.
  • the storage medium is located in the memory 2404, and the processor 2403 reads the information in the memory 2404, and combines its hardware to complete the steps of the method for controlling the charging of the robotic arm 802 provided in the above-mentioned embodiments.
  • the receiver 2401 can be used to receive input digital or character information, and generate signal input related to the settings and function control of the radar system.
  • the transmitter 2402 can be used to output digital or character information through the first interface; the transmitter 2402 can also be used to send instructions to the disk group through the first interface, so as to modify the data in the disk group.
  • the embodiment of the present application also provides a computer program product that, when running on a computer, causes the computer to execute the method for controlling charging of the robotic arm 802 described in the above embodiments.
  • the embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores a program for signal processing, and when it runs on the computer, the computer executes the control as described in the above-mentioned embodiments The method of charging the robot arm 802.
  • the device for controlling the charging of the robotic arm 802 provided in the embodiment of the present application may specifically be a chip.
  • the chip includes: a processing unit and a communication unit.
  • the processing unit may be, for example, a processor, and the communication unit may be, for example, an input/output interface, a pin or circuit etc.
  • the processing unit can execute the computer-executed instructions stored in the storage unit, so that the chip in the execution device executes the image enhancement method described in the above embodiment, or the chip in the training device executes the image enhancement method described in the above embodiment.
  • the storage unit is a storage unit in the chip, such as a register, a cache, etc.
  • the storage unit may also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (read- only memory, ROM) or other types of static storage devices that can store static information and instructions, random access memory (random access memory, RAM), etc.
  • ROM read-only memory
  • RAM random access memory
  • FIG. 25 is a schematic structural diagram of a chip provided by the embodiment of the present application.
  • the chip can be represented as a neural network processor NPU250, and the NPU 250 is mounted to the main CPU (Host CPU) as a coprocessor Above, the tasks are assigned by the Host CPU.
  • the core part of the NPU is the operation circuit 2503, and the operation circuit 2503 is controlled by the controller 2504 to extract matrix data in the memory and perform multiplication operations.
  • the operation circuit 2503 includes multiple processing units (Process Engine, PE).
  • arithmetic circuit 2503 is a two-dimensional systolic array.
  • the arithmetic circuit 2503 may also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition.
  • arithmetic circuitry 2503 is a general purpose matrix processor.
  • the operation circuit fetches the data corresponding to the matrix B from the weight memory 2502, and caches it in each PE in the operation circuit.
  • the operation circuit takes the data of matrix A from the input memory 2501 and performs matrix operation with matrix B, and the obtained partial or final results of the matrix are stored in the accumulator (accumulator) 2508 .
  • the unified memory 2506 is used to store input data and output data.
  • the weight data directly accesses the controller (direct memory access controller, DMAC) 2505 through the storage unit, and the DMAC is transferred to the weight storage 2502.
  • the input data is also transferred to the unified memory 2506 through the DMAC.
  • the BIU is the Bus Interface Unit, that is, the bus interface unit 2510, which is used for the interaction between the AXI bus and the DMAC and the instruction fetch buffer (Instruction Fetch Buffer, IFB) 2509.
  • IFB Instruction Fetch Buffer
  • the bus interface unit 2510 (Bus Interface Unit, BIU for short), is used for the instruction fetch memory 2509 to obtain instructions from the external memory, and is also used for the storage unit access controller 2505 to obtain the original data of the input matrix A or the weight matrix B from the external memory.
  • BIU Bus Interface Unit
  • the DMAC is mainly used to move the input data in the external memory DDR to the unified memory 2506 , to move the weight data to the weight memory 2502 , or to move the input data to the input memory 2501 .
  • the vector computing unit 2507 includes a plurality of computing processing units, and if necessary, further processes the output of the computing circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison and so on. It is mainly used for non-convolutional/fully connected layer network calculations in neural networks, such as Batch Normalization (batch normalization), pixel-level summation, and upsampling of feature planes.
  • Batch Normalization batch normalization
  • pixel-level summation pixel-level summation
  • upsampling of feature planes upsampling of feature planes.
  • vector computation unit 2507 can store the vector of the processed output to unified memory 2506 .
  • the vector calculation unit 2507 can apply a linear function and/or a nonlinear function to the output of the operation circuit 2503, such as performing linear interpolation on the feature plane extracted by the convolutional layer, and for example, a vector of accumulated values to generate an activation value.
  • the vector computation unit 2507 generates normalized values, pixel-level summed values, or both.
  • the vector of processed outputs can be used as an activation input to operational circuitry 2503, eg, for use in subsequent layers in a neural network.
  • An instruction fetch buffer (instruction fetch buffer) 2509 connected to the controller 2504 is used to store instructions used by the controller 2504;
  • the unified memory 2506, the input memory 2501, the weight memory 2502 and the fetch memory 2509 are all On-Chip memories. External memory is private to the NPU hardware architecture.
  • the processor mentioned in any of the above-mentioned places can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more related steps used to control the method for controlling the charging of the mechanical arm 802 described in the above-mentioned embodiments. integrated circuit for program execution.
  • the device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physically separated.
  • a unit can be located in one place, or it can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • the connection relationship between modules indicates that they have communication connections, which can be implemented as one or more communication buses or signal lines.
  • the essence of the technical solution of this application or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product is stored in a readable storage medium, such as a floppy disk of a computer , U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., including several instructions to make a computer device (which can be a personal computer, training device, or network device, etc.) execute the method of each embodiment of the present application .
  • a computer device which can be a personal computer, training device, or network device, etc.
  • all or part of them may be implemented by software, hardware, firmware or any combination thereof.
  • software When implemented using software, it may be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer instructions.
  • the computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable device.
  • the computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be passed from a website site, computer, training device, or data center Wired (eg, coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.) transmission to another website site, computer, training device, or data center.
  • Wired eg, coaxial cable, fiber optic, digital subscriber line (DSL)
  • wireless eg, infrared, wireless, microwave, etc.
  • the computer-readable storage medium may be any available medium that can be stored by a computer, or a data storage device such as a training device or a data center integrated with one or more available media.
  • the available medium may be a magnetic medium (such as a floppy disk, a hard disk, or a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (Solid State Disk, SSD)), etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Power Engineering (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Manipulator (AREA)

Abstract

本申请实施例公开了一种充电设备，包括：控制结构以及设置有力传感器以及充电插头的机械臂主体，力传感器用于在充电插头与充电口接触后采集充电头的侧面和充电口之间的作用力信息；控制结构可以根据作用力信息，控制机械臂主体的位姿，以便将充电插头插入至充电口。本申请通过获取到力传感器采集的作用力信息来进行机械臂的位姿控制，可在充电插头已部分插入，视觉信息无法获取误差的情况下，根据力觉调整位置和位姿，让充电插头适应充电口的插入方向，从而提高了插拔任务的准确性以及可靠性。

Description

一种充电设备以及控制机械臂充电的方法
本申请要求于2021年12月20日提交中国专利局、申请号为202111562748.7、发明名称为“一种充电设备以及控制机械臂充电的方法”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及车辆技术领域，尤其涉及一种充电设备以及控制机械臂充电的方法。
背景技术
随着新能源汽车的发展和规模化的使用，移动充电设备因其具备灵活的充电资源、并且不受环境、车位条件的制约而越来越受到用户的青睐。移动充电设备具体表现为充电桩是可移动的形式，或者可移动底盘搭载电池包形式，比如当移动充电设备是一种移动充电车时，该移动充电车的底盘上可搭载多个电池包，在移动充电车接收到用户的请求后，驶向用户所在地，并为用户驾驶的车辆提供充电服务，完成充电任务。
在移动充电设备为待充电车辆充电之前，需要移动到待充电车辆附近，以定位待充电车辆上的定位口区域，并调整自身的位姿(所处的位置以及充电臂的位姿)进而向充电口充电。
目前主流的充电机器人方案均采用结构光或双目或超声结合机械臂规划方式通过位置控制实现插拔。首先通过结构光相机在其感知范围内对充电口拍照识别，获取充电座相对于相机的位姿，再将位姿坐标转换为机械臂末端充电插口的位姿，即可获取机械臂的控制指令，进行机械臂运动规划完成插拔。
由于为了实现充电插口向充电口的准确插入，要求具备亚毫米级(例如小于等于2°)的位姿定位精度，才可保证充电插口向充电口的顺利插入，且对接不损坏连接器件，因此基于视觉反馈的方案就要求视觉检测模块就要达到位置识别亚毫米级，位姿识别2°的要求，然而当前结构光技术基本精度可以达到1～2mm，位姿3°左右，还会受到距离视场角的限制，本身就很难保证顺利插拔，即使插入，由于位姿误差的存在也会导致较大的插拔力，容易破坏设备，甚至损坏机械臂。因此，仅基于视觉反馈技术进行插拔很难达到充电插口向充电口的顺利插入的精度要求。
发明内容
本申请提供了一种机械臂、充电设备以及控制机械臂充电的方法，可以提高插拔任务的准确性以及可靠性。
第一方面，本申请提供了一种机械臂，包括：控制结构以及机械臂主体；所述机械臂主体上设置有力传感器以及充电插头，所述充电插头包括多个充电头，所述力传感器用于采集所述充电插头和所述充电口之间接触面的作用力信息，所述接触面包括所述多个充电头中至少一个充电头的侧面，所述侧面为和所述充电头的轴向方向平行的表面；所述控制结构用于根据所述作用力信息，控制所述机械臂主体的位姿，以便将所述充电插头插入至 所述充电口。
其中，充电头可以为圆柱状的结构，可以包括底面和侧面，底面可以为充电时需要和充电孔底部相接触的面，侧面可以为充电时需要和充电孔侧壁相接触的面。
在一种可能的实现中，所述作用力信息包括所述充电头的径向方向的力。
在现有的只基于视觉进行位姿控制的方案中，由于视觉本身的误差，而且在充电插头在充电口附近的时候视觉相机已无法继续观测到充电口，如果只采用在最佳观测点观测到的视觉位姿进行充电插头的插拔，可能会发生不能插到底、楔牢的情况，本申请通过获取到力传感器采集的作用力信息来进行机械臂的位姿控制，可在充电插头已部分插入(例如侧面和充电口接触)，视觉信息无法获取误差的情况下，根据力觉调整位置和位姿，让充电插头适应充电口的插入方向，从而提高了插拔任务的准确性以及可靠性。减少了由于充电头侧面和充电口的内壁挤压或者摩擦时对充电头的磨损(或者是卡住而无法达到底部)的情况。
在一种可能的实现中，所述作用力信息包括所述充电口对所述充电插头的力矩信息。
其中，作用力信息可以指示充电口对所述充电插头的6维力，6维力可以包括在三维空间内(例如沿着x轴、y轴、z轴方向)的力以及在三维空间内(例如绕着x轴、y轴、z轴旋转)的力矩。由于其中包括了力矩信息，则可以更精准更细粒度的识别出充电口对于充电头的阻碍方向，进而提供更精准的力控来将充电头准确插入充电孔。
在一种可能的实现中，所述力传感器固定于所述至少一个充电头的侧面；或者，所述机械臂包括具备旋转自由度的关节，所述控制结构用于通过控制所述关节的旋转来控制所述机械臂主体的位姿，所述力传感器固定于所述关节上；或者，所述力传感器固定在所述充电插头和所述机械臂主体之间。
在一种可能的实现中，所述充电口包括多个充电孔，所述多个充电头与所述多个充电孔之间一一匹配；所述控制结构具体用于：
根据所述作用力信息，确定位姿调整参数；
根据所述位姿调整参数，将所述机械臂主体的位姿由第一位姿矫正为第二位姿；其中，所述第一位姿和所述第二位姿不同，且所述第二位姿为每个所述充电头均和相匹配的充电孔对齐时所述机械臂主体的位姿。
在一种可能的实现中，基于作用力信息，可以确定出机械臂主体是否处于正确的位姿(例如第二位姿可以为正确的位姿)。例如，在作用力信息指示充电口对充电插头存在非轴向方向的压力(或者是存在力矩)时，可以确定机械臂主体未处于正确的位姿，在这种情况下，所述多个充电头中的至少一个充电头未和相匹配的充电孔对齐，也就是充电头的中心轴与相匹配的充电孔的中心轴不重合或者偏差大于阈值的情况。
也就是说，第一位姿可以为充电头的中心轴与相匹配的充电孔的中心轴不重合或者偏差大于阈值的情况(也就是上述所述的未处于正确的位姿)。
本申请实施例中，在机械臂主体未处于正确的位姿时，可以基于作用力信息来调整机 械臂主体的位姿，以便机械臂主体处于正确的位姿。
在一种可能的实现中，控制机构可以在基于机械臂主体处于第一位姿时，根据作用力信息对机械臂主体的位姿进行矫正(至第二位姿，也就是正确的位姿)，第二位姿为机械臂主体可以将充电插头以较小阻力的正确方式插入到充电口底部的位姿。
在一种可能的实现中，所述第一位姿为所述多个充电头的底部端面未和所述充电孔的底部接触时的位姿，所述底部端面为和所述充电头的轴向方向垂直的表面。
在一种可能的实现中，所述对齐包括：充电头沿相匹配的充电孔方向的投影范围在相匹配的充电孔的范围内，且充电头的中心轴与相匹配的充电孔的中心轴重合或者偏差小于阈值。
在一种可能的实现中，所述控制结构还用于：在将所述机械臂主体的位姿由第一位姿矫正为第二位姿之后，控制所述机械臂主体的位姿，以便所述充电插头朝向所述充电孔底部且沿着所述充电头的中心轴的方向移动。
在一种可能的实现中，在所述由第一位姿矫正为第二位姿的过程中，所述充电口对所述充电插头在目标方向上的阻力存在降低的趋势，所述目标方向为所述充电头的轴向方向或径向方向。
应理解，在从第一位姿调整至第二位姿时，充电口对所述充电头在目标方向上的阻力可以存在降低的趋势，目标方向可以径向方向和/或轴向状态(例如可以为径向方向、轴向状态、或者径向方向以及轴向方向)。这里所谓的存在降低的趋势，可以理解为充电口对所述充电头在目标方向上的阻力可以不是一直降低，而是可以存在局部的增加，但是从趋势上，充电口对所述充电头在目标方向上的阻力是降低的。
在将所述机械臂主体的位姿由第一位姿矫正为第二位姿之后，充电插头已经处于和充电口的对齐位姿，在这种情况下，可以将充电插头朝向所述充电孔底部且沿着所述充电头的中心轴的方向移动直到底部，在一种可能的实现中，所述控制结构可以基于所述作用力信息指示所述充电口对所述充电插头仅存在沿所述充电插头的轴向方向的阻力，控制所述机械臂主体保持静止状态，进而完成了向充电口的插入任务。
在一种可能的实现中，在所述由第一位姿矫正为第二位姿的过程中，所述充电口对所述充电插头之间的力矩存在降低的趋势。
在一种可能的实现中，所述位姿调整参数具体用于控制所述充电插头的移动方向以及移动速率。
在一种可能的实现中，所述控制机构可以实时获取视觉传感器针对于所述充电口的区域所采集的图像信息；根据所述图像信息中所述充电口的位置信息，得到所述机械臂主体的实时位姿；根据所述实时位姿，控制所述机械臂主体的位姿，以便将所述机械臂主体上 的充电插头引导至所述充电口周围。
由于视觉伺服中采集的图像信息是实时获取的，基于实时获取的图像信息可以求解得到机械臂主体的实时位姿(该实时位姿可以为机械臂主体末端和待充电车辆的充电口之间的相对位姿)，由于在控制所述机械臂主体的位姿时，可能会出现底座移动、障碍物的阻碍等情况，导致机械臂主体的位姿出现误差，本申请可以基于实时位姿来修正机械臂主体的位姿，可以提高机械臂主体的位姿识别精度，进而可以更准确的将机械臂主体上的充电插头引导至所述充电口周围。
在一种可能的实现中，视觉伺服是视觉结合运动进行的闭环控制方法。主要功能是实现机械臂末端充电枪对于充电口的导引运功。大致流程是将充电枪/充电座视觉识别位姿结果作为输入，采用视觉伺服反馈控制思想计算驱动误差，采用基于位姿的视觉伺服实现机械臂末端向目标位姿的精确导引。采用自适应增益方法，加快在小误差时的调整速度，实现快速的视觉伺服。此步骤中伺服结束时机械臂将处于最佳观测点，此处已获取最精准的充电口的位姿。此时机械臂将根据把充电插口导引至充电口附近，为下一步力控插入做准备。
在一种可能的实现中，由于最佳观测点要保证一定的稳定性，才能保证目标观测位姿的精度。然而，移动机器人基座相对充电座的位姿可能会不断地发生变化。为实现在上述变化下，保障最佳观测点相对充电座的位姿一致，采用位姿的快速视觉伺服实现此目标。在一种可能的实现中，可以采用位姿反馈的视觉伺服思想，基于李氏稳定判据设计关节速度控制律，实现机械臂末端向目标位置的精确导引。
可选的，在一种可能的实现中，所述机械臂主体上设置有所述视觉传感器。
可选的，在一种可能的实现中，视觉传感器还可以不设置在机械臂主体上而是设置在第三方视角对应的位置上。
在一种可能的实现中，在进行上述相对位姿的确定时，需要视觉传感器采集到针对于充电口的图像信息，并基于图像信息来确定充电口相对于视觉传感器之间的相对位姿。
在一种可能的实现中，所述图像信息包括2D图像以及3D点云，所述控制机构还用于：根据所述2D图像以及预设的充电口3D模型，提取所述充电口对应的第一3D位置；
根据所述3D点云，提取所述充电口对应的第二3D位置；将所述第一3D位置和所述第二3D位置进行融合，得到所述充电口的位姿。
根据2D图像，可以得到充电口中心点二维像素坐标，通过中心点的相互位置关系得到每个点的编号；根据充电座标准CAD模型得到圆孔中心点在目标坐标系下的三维物理坐标，进而得到各个充电孔的三维物理坐标。
在一种可能的实现中，所述第一3D位置包括所述充电口上M个第一特征点的3D位置；所述第二3D位置包括所述充电口上M个第二特征点的3D位置；所述M个第一特征点和所述M个第二特征点一一对应；所述将所述第一3D位置和所述第二3D位置进行融合，包括：将所述M个第一特征点的3D位置分别和对应的所述第二特征点的3D位置进 行融合。其中，第一特征点和第二特征点可以为充电孔。
本申请提出了一种融合2D图像-3D点云信息的充电口位姿精确识别方法，可以采用2D深度图像检测网络对充电口的二维关键点进行亚像素级的精确检测，采用三维关键点检测对充电口三维关键点进行基于3D霍夫网络的偏移量可靠检测，通过融合(例如采用非线性优化)使得融合二维关键点及三维关键点的投影误差最小，进而求得充电口的精确位姿，以此解决充电口在光照变化大、目标对比度低情况下的位姿高精度识别问题。
在一种可能的实现中，所述机械臂主体还包括目标接口，所述目标接口用于可拆卸连接所述充电插头。可选的，目标接口可以为基于电磁性来进行清扫工具的可拆卸连接的部件。采用磁吸式结构抓取充电枪，保证充电插头和机械臂的可靠对接，然后进行下一步充电操作。
在一种可能的实现中，在所述控制结构根据所述作用力信息，控制所述机械臂主体的位姿之前，所述目标接口与初始充电插头连接；
所述控制结构还用于控制所述机械臂主体的位姿、以及所述目标接口的连接状态，将所述初始充电插头从所述目标接口上进行拆卸并替换为所述充电插头；其中，
所述目标充电插头为支持快充充电的插头，所述充电插头为支持慢充充电的插头；或者，
所述目标充电插头为支持慢充充电的插头，所述充电插头为支持快充充电的插头。
本申请实施例中，充电设备可以兼容快充枪和慢充枪两种充电方式，可以具备快换机构，可根据用户任务需求选取充电枪。
在一种可能的实现中，所述充电设备还包括：插头收纳区域，所述插头收纳区域包括用于设置所述充电插头的目标收纳位置；
所述控制所述机械臂主体的位姿、以及所述目标接口的连接状态，将所述初始充电插头从所述目标接口上进行拆卸并替换为所述充电插头，包括：
控制所述目标接口的连接状态，将所述初始充电插头从所述目标接口上进行拆卸；
控制所述机械臂主体的位姿，将所述目标接口移动至所述目标收纳位置，并控制所述目标接口的连接状态，将所述目标接口与所述充电插头进行连接。
第二方面，本申请提供了一种充电设备，包括：机械臂、力传感器以及充电插头；
所述机械臂包括控制结构以及机械臂主体；
所述机械臂主体上设置有所述力传感器以及所述充电插头，所述充电插头包括多个充电头，所述力传感器用于采集所述充电插头和所述充电口之间接触面的作用力信息，所述接触面包括所述多个充电头中至少一个充电头的侧面，所述侧面为和所述充电头的轴向方向平行的表面；
所述控制结构用于根据所述作用力信息，控制所述机械臂主体的位姿，以便将所述充电插头插入至所述充电口。
在一种可能的实现中，所述作用力信息包括所述充电口对所述充电插头的力矩信息。
在一种可能的实现中，所述力传感器固定于所述至少一个充电头的侧面；或者，
所述机械臂包括具备旋转自由度的关节，所述控制结构用于通过控制所述关节的旋转来控制所述机械臂主体的位姿，所述力传感器固定于所述关节上；或者，
所述力传感器固定在所述充电插头和所述机械臂主体之间。
在一种可能的实现中，所述充电口包括多个充电孔，所述多个充电头与所述多个充电孔之间一一匹配；所述控制结构具体用于：
根据所述作用力信息，确定位姿调整参数；
根据所述位姿调整参数，将所述机械臂主体的位姿由第一位姿矫正为第二位姿；其中，所述第一位姿和所述第二位姿不同，且所述第二位姿为每个所述充电头均和相匹配的充电孔对齐时所述机械臂主体的位姿。
在一种可能的实现中，所述第一位姿为所述多个充电头的底部端面未和所述充电孔的底部接触时的位姿，所述底部端面为和所述充电头的轴向方向垂直的表面。
在一种可能的实现中，所述对齐包括：充电头沿相匹配的充电孔方向的投影落在所述相匹配的充电孔的范围内，且充电头的中心轴与相匹配的充电孔的中心轴重合或者偏差小于阈值。
在一种可能的实现中，所述控制结构还用于：
在将所述机械臂主体的位姿由第一位姿矫正为所述第二位姿之后，控制所述机械臂主体的位姿，以便所述充电插头朝向所述充电孔底部且沿着所述充电头的中心轴的方向移动。
在一种可能的实现中，在所述由第一位姿矫正为第二位姿的过程中，所述充电口对所述充电插头之间的力矩存在降低的趋势。
在一种可能的实现中，所述位姿调整参数具体用于控制所述充电插头的移动方向以及移动速率。
在一种可能的实现中，所述控制结构具体用于：
基于所述作用力信息指示所述充电口对所述充电插头仅存在沿所述充电插头的轴向方向的阻力，控制所述机械臂主体保持静止状态。
在一种可能的实现中，所述控制机构还用于：
根据所述作用力信息，控制所述机械臂主体的位姿之前，实时获取视觉传感器针对于 所述充电口的区域所采集的图像信息；
根据所述图像信息中所述充电口的位置信息，得到所述机械臂主体的实时位姿；
根据所述实时位姿，控制所述机械臂主体的位姿，以便将所述机械臂主体上的充电插头引导至所述充电口周围。
在一种可能的实现中，所述机械臂主体上设置有所述视觉传感器。
在一种可能的实现中，所述图像信息包括2D图像以及3D点云，所述控制机构还用于：
根据所述2D图像以及预设的充电口3D模型，提取所述充电口对应的第一3D位置；
根据所述3D点云，提取所述充电口对应的第二3D位置；
将所述第一3D位置和所述第二3D位置进行融合，得到所述充电口的位姿。
在一种可能的实现中，所述第一3D位置包括所述充电口上M个第一特征点的3D位置；所述第二3D位置包括所述充电口上M个第二特征点的3D位置；所述M个第一特征点和所述M个第二特征点一一对应；
所述将所述第一3D位置和所述第二3D位置进行融合，包括：
将所述M个第一特征点的3D位置分别和对应的所述第二特征点的3D位置进行融合。
在一种可能的实现中，所述机械臂主体还包括目标接口，所述目标接口用于可拆卸连接所述充电插头。
在一种可能的实现中，在所述控制结构根据所述作用力信息，控制所述机械臂主体的位姿之前，所述目标接口与初始充电插头连接；
所述控制结构还用于控制所述机械臂主体的位姿、以及所述目标接口的连接状态，将所述初始充电插头从所述目标接口上进行拆卸并替换为所述充电插头；其中，
所述目标充电插头为支持快充充电的插头，所述充电插头为支持慢充充电的插头；或者，
所述目标充电插头为支持慢充充电的插头，所述充电插头为支持快充充电的插头。
在一种可能的实现中，所述充电设备还包括：插头收纳区域，所述插头收纳区域包括用于设置所述充电插头的目标收纳位置；
所述控制所述机械臂主体的位姿、以及所述目标接口的连接状态，将所述初始充电插头从所述目标接口上进行拆卸并替换为所述充电插头，包括：
控制所述目标接口的连接状态，将所述初始充电插头从所述目标接口上进行拆卸；
控制所述机械臂主体的位姿，将所述目标接口移动至所述目标收纳位置，并控制所述目标接口的连接状态，将所述目标接口与所述充电插头进行连接。
在一种可能的实现中，所述充电设备为固定式充电桩、移动机器人或者充电车。
第三方面，本申请提供了一种控制机器臂充电的方法，所述方法应用于机械臂的控制结构，所述机械臂主体上设置有力传感器以及充电插头，所述充电插头包括多个充电头，所述力传感器用于采集所述充电插头和所述充电口之间接触面的作用力信息，所述接触面包括所述多个充电头中至少一个充电头的侧面，所述侧面为和所述充电头的轴向方向平行的表面；
所述方法包括：
根据所述作用力信息，控制所述机械臂主体的位姿，以便将所述充电插头插入至所述充电口。
在一种可能的实现中，所述作用力信息包括所述充电口对所述充电插头的力矩信息。
在一种可能的实现中，所述作用力信息包括所述充电头的径向方向的力。
在一种可能的实现中，所述力传感器固定于所述至少一个充电头的侧面；或者，
所述机械臂包括具备旋转自由度的关节，所述控制结构用于通过控制所述关节的旋转来控制所述机械臂主体的位姿，所述力传感器固定于所述关节上；或者，
所述力传感器固定在所述充电插头和所述机械臂主体之间。
在一种可能的实现中，所述充电口包括多个充电孔，所述多个充电头与所述多个充电孔之间一一匹配；所述根据所述作用力信息，控制所述机械臂主体的位姿，包括：
根据所述作用力信息，确定位姿调整参数；
根据所述位姿调整参数，将所述机械臂主体的位姿由第一位姿矫正为第二位姿；其中，所述第一位姿和所述第二位姿不同，且所述第二位姿为每个所述充电头均和相匹配的充电孔对齐时所述机械臂主体的位姿。
在一种可能的实现中，所述第一位姿为所述多个充电头的底部端面未和所述充电孔的底部接触时的位姿，所述底部端面为和所述充电头的轴向方向垂直的表面。
在一种可能的实现中，所述对齐包括：充电头沿相匹配的充电孔方向的投影落在所述相匹配的充电孔的范围内，且充电头的中心轴与相匹配的充电孔的中心轴重合或者偏差小于阈值。
在一种可能的实现中，在将所述机械臂主体的位姿由第一位姿矫正为第二位姿之后，所述方法还包括：
控制所述机械臂主体的位姿，以便所述充电插头朝向所述充电孔底部且沿着所述充电 头的中心轴的方向移动。
在一种可能的实现中，在所述由第一位姿矫正为第二位姿的过程中，所述充电口对所述充电插头在目标方向上的阻力存在降低的趋势，所述目标方向为所述充电头的轴向方向或径向方向。
在一种可能的实现中，在所述由第一位姿矫正为第二位姿的过程中，所述充电口对所述充电插头之间的力矩存在降低的趋势。
在一种可能的实现中，所述方法还包括：
基于所述作用力信息指示所述充电口对所述充电插头仅存在沿所述充电插头的轴向方向的阻力，控制所述机械臂主体保持静止状态。
在一种可能的实现中，所述位姿调整参数具体用于控制所述充电插头的移动方向以及移动速率。
在一种可能的实现中，所述方法还包括：
根据所述作用力信息，控制所述机械臂主体的位姿之前，实时获取视觉传感器针对于所述充电口的区域所采集的图像信息；
根据所述图像信息中所述充电口的位置信息，得到所述机械臂主体的实时位姿；
根据所述实时位姿，控制所述机械臂主体的位姿，以便将所述机械臂主体上的充电插头引导至所述充电口周围。
在一种可能的实现中，所述机械臂主体上设置有所述视觉传感器。
在一种可能的实现中，所述图像信息包括2D图像以及3D点云，所述方法还包括：
根据所述2D图像以及预设的充电口3D模型，提取所述充电口对应的第一3D位置；
根据所述3D点云，提取所述充电口对应的第二3D位置；
将所述第一3D位置和所述第二3D位置进行融合，得到所述充电口的位姿。
在一种可能的实现中，所述第一3D位置包括所述充电口上M个第一特征点的3D位置；所述第二3D位置包括所述充电口上M个第二特征点的3D位置；所述M个第一特征点和所述M个第二特征点一一对应；
所述将所述第一3D位置和所述第二3D位置进行融合，包括：
将所述M个第一特征点的3D位置分别和对应的所述第二特征点的3D位置进行融合。
在一种可能的实现中，所述机械臂主体还包括目标接口，所述目标接口用于可拆卸连接所述充电插头。
在一种可能的实现中，在所述控制结构根据所述作用力信息，控制所述机械臂主体的位姿之前，所述目标接口与初始充电插头连接；
所述方法还包括：控制所述机械臂主体的位姿、以及所述目标接口的连接状态，将所述初始充电插头从所述目标接口上进行拆卸并替换为所述充电插头；其中，
所述目标充电插头为支持快充充电的插头，所述充电插头为支持慢充充电的插头；或者，
所述目标充电插头为支持慢充充电的插头，所述充电插头为支持快充充电的插头。
在一种可能的实现中，所述充电设备还包括：插头收纳区域，所述插头收纳区域包括用于设置所述充电插头的目标收纳位置；
所述控制所述机械臂主体的位姿、以及所述目标接口的连接状态，将所述初始充电插头从所述目标接口上进行拆卸并替换为所述充电插头，包括：
控制所述目标接口的连接状态，将所述初始充电插头从所述目标接口上进行拆卸；
控制所述机械臂主体的位姿，将所述目标接口移动至所述目标收纳位置，并控制所述目标接口的连接状态，将所述目标接口与所述充电插头进行连接。
在一种可能的实现中，所述充电设备为固定式充电桩、移动机器人或者充电车。
第四方面，本申请实施例提供了一种计算机可读存储介质，其特征在于，包括计算机可读指令，当该计算机可读指令在计算机设备上运行时，使得该计算机设备执行上述第三方面及其任一可选的方法。
第五方面，本申请实施例提供了一种计算机程序产品，其特征在于，包括计算机可读指令，当该计算机可读指令在计算机设备上运行时，使得该计算机设备执行上述第三方面及其任一可选的方法。
第六方面，本申请提供了一种芯片系统，该芯片系统包括处理器，用于支持执行控制机构实现上述方面中所涉及的功能，例如，发送或处理上述方法中所涉及的数据；或，信息。在一种可能的设计中，该芯片系统还包括存储器，该存储器，用于保存执行设备或训练设备必要的程序指令和数据。该芯片系统，可以由芯片构成，也可以包括芯片和其他分立器件。
第七方面，本申请提供了一种控制机器臂充电的装置，所述装置包括一个或多个处理器和存储器；其中，所述存储器中存储有计算机可读指令；所述一个或多个处理器读取所述计算机可读指令，执行上述第三方面及其任一可选的方法。
本申请提供了一种充电设备，其特征在于，包括：机械臂、力传感器以及充电插头； 所述机械臂包括控制结构以及机械臂主体；所述机械臂主体上设置有所述力传感器以及所述充电插头，所述充电插头包括多个充电头，所述力传感器用于采集所述充电插头和所述充电口之间接触面的作用力信息，所述接触面包括所述多个充电头中至少一个充电头的侧面，所述侧面为和所述充电头的轴向方向平行的表面；所述控制结构用于根据所述作用力信息，控制所述机械臂主体的位姿，以便将所述充电插头插入至所述充电口。在现有的只基于视觉进行位姿控制的方案中，由于视觉本身的误差，而且在充电插头在充电口附近的时候视觉相机已无法继续观测到充电口，如果只采用在最佳观测点观测到的视觉位姿进行充电插头的插拔，可能会发生不能插到底、楔牢的情况，本申请通过获取到力传感器采集的作用力信息来进行机械臂的位姿控制，可在充电插头已部分插入(例如侧面和充电口接触)，视觉信息无法获取误差的情况下，根据力觉调整位置和位姿，让充电插头适应充电口的插入方向，从而提高了插拔任务的准确性以及可靠性。减少了由于充电头侧面和充电口的内壁挤压或者摩擦时对充电头的磨损(或者是卡住而无法达到底部)的情况。
附图说明
图1为本申请实施例提供的场景示意；
图2为本申请实施例提供的场景示意；
图3为本申请实施例提供的场景示意；
图4为本申请实施例提供的场景示意；
图5为本申请实施例提供的场景示意；
图6为本申请实施例提供的场景示意；
图7为本申请实施例提供的场景示意；
图8为本申请实施例提供的一个充电设备的结构示意；
图9为本申请实施例提供的一个搜索方法示意；
图10为本申请实施例提供的一个检测结果示意；
图11为本申请实施例提供的一个检测结果示意；
图12为本申请实施例提供的一个位姿计算方法示意；
图13为本申请实施例提供的一个信息融合算法示意；
图14为本申请实施例提供的一个模型结构示意；
图15为本申请实施例提供的一个检测结果示意；
图16为本申请实施例提供的一个坐标系转换示意；
图17为本申请实施例提供的一个3D信息提取示意；
图18为本申请实施例提供的一个控制机械臂充电的方法示意；
图19为本申请实施例提供的一个机械臂示意；
图20为本申请实施例提供的一个控制机械臂充电的方法示意；
图21为本申请实施例提供的一个RCC设备示意；
图22为本申请实施例提供的一个控制机械臂充电的示意；
图23为本申请实施例提供的一个受力示意；
图24为本申请实施例提供的一个控制机械臂充电的装置示意；
图25为本申请实施例提供的芯片的一种结构示意图。
具体实施方式
下面结合本申请实施例中的附图对本申请实施例进行描述。本申请的实施方式部分使用的术语仅用于对本申请的具体实施例进行解释，而非旨在限定本申请。
本申请的说明书和权利要求书及上述附图中的术语“第一”、第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的术语在适当情况下可以互换，这仅仅是描述本申请的实施例中对相同属性的对象在描述时所采用的区分方式。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，以便包含一系列单元的过程、方法、系统、产品或设备不必限于那些单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它单元。
新能源车代表了汽车行业绿色发展的新方向，在我国和全世界出现了爆发性发展。随着新能源汽车销量陡增，车主们需要在路途中进行“加电”。然而，现有的服务区或停车场充电桩相对严重不足，导致经常出现用户大量排队、排队等待时间远超过充电时间的现象。在短时间无法大量提供充电桩的情况下，采用移动充电设备(例如充电机器人)为新能源车进行自动充电，为新能源车提供”移动充电宝”，可在短时间内有效缓解充电桩严重不足的问题。另一方面，即便停车场里有空闲的充电桩，在停车场里找到电车充电桩也特别费时间，同时快充充电桩普遍存在充电枪和线缆重量重、硬度较高的问题，导致驾驶员人工进行充电操较为困难。充电机器人可以自行导航到车主跟前，并代替人进行充电枪的插拔，会给用户带来更高效的充电服务和更好的充电体验。
如图1所示，移动充电设备具体可以表现为充电桩是可移动的形式，或者可移动底盘搭载电池包形式，比如当移动充电设备是一种移动充电车时，该移动充电车的底盘上可搭载多个电池包，在移动充电车接收到用户的请求后，驶向用户所在地，并为用户驾驶的车辆提供充电服务，完成充电任务。
本申请的技术方案可应用于一种移动充电系统，如图2所示，该系统可以包括服务器11、至少一个车辆系统12、移动充电设备13和充电桩14等。其中，每个车辆系统12中可以包括用户121、终端设备122和待充电车辆123，如图3所示。车辆123为用户121正在驾驶的需要充电的车辆，终端设备122的持有者是用户121。
其中，用户121可以是驾驶员，当驾驶员发现其驾驶的车辆123电量不足时，通过终端设备122向服务器发送一个请求消息，所述请求消息用于请求为车辆123充电。或者，用户121在终端设备122的APP上生成一个请求，然后将该请求发送给服务器11。
终端设备122可以是一种便携式设备，比如智能终端、手机、笔记本电脑、平板电脑、个人计算机(personal computer，PC)、个人数字助理(personal digital assistant，PDA)，可折叠终端、具备无线通讯功能的可穿戴设备(例如智能手表或手环)、用户设备(user device)或用户设备(user equipment，UE)、以及增强现实(augmented reality，AR)或者虚拟现实(virtual reality，VR)设备等，本申请的实施例对终端设备的具体设备形态不做限定。
车辆123可以是一种电动汽车(electric vehicle，EV)，所述EV包括显示屏、车机处理器和通信模块，此外，EV还可以包括其他部件或单元，本实施例对此不予限制。
可选的，在一种可能的实施方式中，上述车辆系统12中可以不包括终端设备122，所述终端设备122的功能由EV中的车机处理器和通信模块来实现，比如车机处理器在接收到用户触发的指令之后，通过通信模块向服务器11发送请求消息。
参见图2，服务器11可接收一个或多个车辆系统12发送的请求消息，对系统中的至少一个移动充电设备13进行调度，并向相应的移动充电设备派发充电任务，指示其驶向发请求消息的用户所在位置，对请求的车辆进行充电。另外，在调度和派发充电任务之前，服务器11还用于获取路况信息，根据路况信息进行高级驾驶辅助系统(advanced driver assistance systems，ADAS)计算，生成调度策略等。
可选的，所述服务器11为云服务器，所述云服务器可以是一个服务器，或者是由多个服务器组成的服务器集群。
移动充电设备13，用于接收服务器11发送的指示信号，并根据指示信号驶向待充电车辆(本申请实施例也可以称之为目标车辆)的位置，并为其充电，当移动充电设备13完成充电任务后，返回充电桩14进行补电。此外，移动充电设备13还用于在执行充电任务的过程中实时地上报当前的所携带的电池包的电量情况，以及在行驶过程中实时地上报自己的位置信息给服务器11，以便为服务器11能够实时地掌握系统中各个移动充电设备的情况，为后续充电任务的派发和电能资源的调度提供依据。
另外，每个移动充电设备13中包括车机处理器、通信模块、充电电池或者电池包，所述电池包的数量可以是一个或者多个，每个电池包携带的一定电量，本实施例对电池包的数量以及每个电池包的电量不予限制。
可选的，所述移动充电设备13可以是一种EV，或者燃油汽车。
接下来介绍移动充电设备13的架构示意。
本申请实施例可以应用于如图4所示的移动充电设备13中。如图4所示，该移动充电设备13可以包括：传感器模块110、驱动装置120、充电装置140和主控平台130。
其中，传感器模块110可以包括一颗或者多颗视觉传感器111(例如摄像头)(本申请实施例也可以简称为传感器)组成，例如可以包括普通的光学摄像头，也可以是红外摄像头、结构光传感器或飞行时间(time-of-flight，ToF)传感器等。示例地，传感器模块110可以包括一颗普通的RGB摄像头或红黄黄蓝(red yellow yellow blue，RYYB)摄像头，摄像装置模块也可以包括多颗摄像头或者传感器组成RGB-D深度摄像头方案。示例地，RGB-D深度摄像头方案可以包括两颗RGB摄像头组成双目方案，可以包括一颗RGB摄像头和一颗结构光传感器组成结构光方案，也可以是包括一颗RGB摄像头和一颗ToF传感器组成ToF方案，本申请实施例对此不做具体限定。另外，视觉传感器111(例如摄像头)可以是定焦摄像头，也可以是变焦摄像头，例如具备相位对焦、激光对焦等能力。
应理解，视觉传感器111(例如摄像头)可以承载于一个运动单元上，运动单元用于承载视觉传感器111(例如摄像头)，并且驱动视觉传感器111(例如摄像头)产生旋转。在一个实施例中，运动单元可以驱动视觉传感器111(例如摄像头)产生两自由度旋转，若Z 轴所指的方向为摄像头11正前方，那么上述两自由度旋转可以包括视觉传感器111(例如摄像头)以x轴为旋转轴的旋转和视觉传感器111(例如摄像头)以y轴为旋转轴的旋转。运动单元驱动视觉传感器111(例如摄像头)产生旋转可以通过舵机或者伺服电机的转动来实现，示例地，当驱动装置用于驱动视觉传感器111(例如摄像头)产生两自由度旋转时，运动单元可以包含两个驱动机构driver 1和driver 2，例如两个舵机或者两个伺服电机，其中一个舵机1或者伺服电机1用于控制视觉传感器111(例如摄像头)以x轴为旋转轴的旋转，另一个舵机2或者伺服电机2用于控制视觉传感器111(例如摄像头)以y轴为旋转轴的旋转。在另一些实施例中，运动单元可以驱动视觉传感器111(例如摄像头)产生三自由度旋转，即增加视觉传感器111(例如摄像头)以z轴为旋转轴的旋转，相应地，运动单元还可以包含三个驱动机构driver 1、driver 2和driver 3，例如三个舵机或者三个伺服电机，其中一个舵机1或者伺服电机1用于控制视觉传感器111(例如摄像头)以x轴为旋转轴的旋转，另一个舵机2或者伺服电机2用于控制视觉传感器111(例如摄像头)以y轴为旋转轴的旋转，另一个舵机3或者伺服电机3用于控制视觉传感器111(例如摄像头)以z轴为旋转轴的旋转。
传感器模块110还可以包括运动传感器112，运动传感器112可以为里程计、加速度计、速度计、惯性测量单元等等，用于采集移动充电设备13在行驶过程中的里程信息，如行程、轨迹、速度等信息。
力传感器113可以为用于检测对机械臂142末端施加力的传感器。作为力传感器113，可以利用可以检测单轴方向的力的压力传感器和可以检测多个轴方向的力的成分的力觉传感器或力矩传感器。在本实施方式中，作为力传感器113，可以使用六轴的力觉传感器。六轴的力觉传感器检测与固有的传感器坐标系中互相正交的三个检测轴平行的力的大小和绕三个检测轴的力矩的大小。需要说明的是，力传感器113可以设置在机械臂142末端的位置以外的位置，例如，可以设置在机械臂142中的一个以上的关节上。
驱动装置120可包括为移动充电设备13提供动力运动的组件。在一个实施例中，驱动装置120可包括引擎、能量源、传动装置和车轮/轮胎。引擎可以是内燃引擎、电动机、空气压缩引擎或其他类型的引擎组合，例如气油发动机和电动机组成的混动引擎，内燃引擎和空气压缩引擎组成的混动引擎。引擎将能量源转换成机械能量。
能量源的示例包括汽油、柴油、其他基于石油的燃料、丙烷、其他基于压缩气体的燃料、乙醇、太阳能电池板、电池和其他电力来源。能量源也可以为移动充电设备13的其他系统提供能量。
传动装置可以将来自引擎的机械动力传送到车轮。传动装置可包括差速器和驱动轴。在一个实施例中，传动装置还可以包括其他器件，比如离合器。其中，驱动轴可包括可耦合到一个或多个车轮的一个或多个轴。
主控平台130是该装置的数据处理和控制中心，主控平台130与充电装置140、传感器模块110和驱动装置120建立通信连接，例如能够接收传感器模块110采集的图像数据，对图像数据进行处理，以及向驱动装置120发送移动的指令等。其中，充电装置140可以包括充电头141(或者称之为充电插头)和机械臂142。在一些实施例中，主控平台130可 以是嵌入式的计算机平台，包括单元不限于基于X86指令集、ARM指令集、RISC-V指令集或者MIPS指令集等设计的计算机芯片和软件系统等。
在一个实施例中，上述计算机芯片例如可以包括处理器131和存储器132，其中，处理器131例如可以包括：中央处理器(central processing unit，CPU)、系统芯片(system on a chip，SoC）、应用处理器(application processor，AP)、微控制器(microcontroller)、神经网络处理器(Neural-network Processing Unit，NPU)和/或图形处理器(graphics processing unit，GPU)等；存储器132例如可以包括非易失性存储器和易失性存储器，非易失性存储器例如快闪存储器(flash memory)，包括NAND flash、固态硬盘等，易失性存储器例如同步动态随机存取內存(synchronous dynamic random-access memory，SDRAM)等。
在一个实施例中，上述软件系统可以包括操作系统以及运行在该操作系统中的程序指令133。当处理器执行上述程序指令时，使得图3或图4所示的装置执行本申请实施例提供的控制机械臂充电的方法的各个步骤。
在一些实施例中，存储器132可包含程序指令133(例如，程序逻辑)，程序指令133可被处理器131执行来执行移动充电设备13的各种功能，包括以上描述的那些功能。存储器132也可包含额外的指令，包括向驱动装置120、传感器模块110、控制系统和外围设备中的一个或多个发送数据、从其接收数据、与其交互和/或对其进行控制的指令。
除了程序指令133以外，存储器132还可存储数据，例如道路地图、路线信息，自动驾驶装置的位置、方向、速度以及其它这样的自动驾驶装置数据，以及其他信息。这种信息可在移动充电设备13在自主、半自主和/或手动模式中操作期间被移动充电设备13使用。
无线通信系统150可以直接地或者经由通信网络来与一个或多个设备(例如服务器11)无线通信。例如，无线通信系统150可使用3G蜂窝通信，例如码分多址(code division multiple access，CDMA)、EVD0、全球移动通信系统(global system for mobile communications，GSM)/是通用分组无线服务技术(general packet radio service，GPRS)，或者4G蜂窝通信，例如长期演进(long term evolution，LTE)，或者5G蜂窝通信。无线通信系统150可利用WiFi与无线局域网(wireless local area network，WLAN)通信。在一些实施例中，无线通信系统150可利用红外链路、蓝牙或ZigBee与设备直接通信。其他无线协议，例如各种自动驾驶装置通信系统，例如，无线通信系统150可包括一个或多个专用短程通信(dedicated short range communications，DSRC)设备，这些设备可包括自动驾驶装置和/或路边台站之间的公共和/或私有数据通信。
可选地，上述这些组件中的一个或多个可与移动充电设备13分开安装或关联。例如，存储器132可以部分或完全地与移动充电设备13分开存在。上述组件可以按有线和/或无线方式来通信地耦合在一起。
可选地，上述组件只是一个示例，实际应用中，上述各个模块中的组件有可能根据实际需要增添或者删除，图4不应理解为对本申请实施例的限制。
本申请实施例中控制机械臂充电的方法的执行主体可以为移动充电设备13，也可以为云侧的服务器11，参照图5，图5示出的系统中可以移动充电设备13和服务器11，移动充电设备13和服务器11通信连接(例如通过无线通信系统150实现)，其中，移动充电设备 13可以将传感器采集的数据传递至服务器11，由服务器11基于移动充电设备13将传感器采集的数据来实现本申请实施例中的控制机械臂充电的方法，并将控制机械臂充电的相关处理结果传递至移动充电设备13。
接下来介绍服务器11的架构示意。
本申请实施例还提供了一种服务器，请参阅图6，图6是本申请实施例提供的服务器一种结构示意图，具体的，服务器600由一个或多个服务器实现，服务器600可因配置或性能不同而产生比较大的差异，可以包括一个或一个以上中央处理器(central processing units，CPU)66(例如，一个或一个以上处理器)和存储器632，一个或一个以上存储应用程序642或数据644的存储介质630(例如一个或一个以上海量存储设备)。其中，存储器632和存储介质630可以是短暂存储或持久存储。存储在存储介质630的程序可以包括一个或一个以上模块(图示没标出)，每个模块可以包括对服务器中的一系列指令操作。更进一步地，中央处理器66可以设置为与存储介质630通信，在服务器600上执行存储介质630中的一系列指令操作。
服务器600还可以包括一个或一个以上电源626，一个或一个以上有线或无线网络接口650，一个或一个以上输入输出接口658；或，一个或一个以上操作系统641，例如Windows ServerTM，Mac OS XTM，UnixTM,LinuxTM，FreeBSDTM等等。
目前主流的充电机器人方案均采用结构光或双目或超声结合机械臂规划方式通过位置控制实现插拔。首先通过结构光相机在其感知范围内对充电口拍照识别，获取充电座相对于相机的位姿，再将位姿坐标转换为机械臂末端充电插口的位姿，即可获取机械臂的控制指令，进行机械臂运动规划完成插拔。
由于为了实现充电插口向充电口的准确插入，要求具备亚毫米级(例如小于等于2°)的位姿定位精度，才可保证充电插口向充电口的顺利插入，且对接不损坏连接器件，因此基于视觉反馈的方案就要求视觉检测模块就要达到位置识别亚毫米级，位姿识别2°的要求，然而当前结构光技术基本精度可以达到1～2mm，位姿3°左右，还会受到距离视场角的限制，本身就很难保证顺利插拔，即使插入，由于位姿误差的存在也会导致较大的插拔力，容易破坏设备，甚至损坏机械臂。因此，仅基于视觉反馈技术进行插拔很难达到充电插口向充电口的顺利插入的精度要求。
本申请实施例提供的用于进行充电的机械臂可以高精度的实现充电插口向充电口的插拔。
接下来从产品的形态上介绍本申请实施例中的充电设备。
参照图7，图7为本申请实施例中的充电设备的充电场景示意，其中，充电设备可以包括机械臂。机械臂可以采用多轴串联型机械臂，实现对于充电枪的位置和位姿控制，机械臂形态不限于串联，只要可以实现末端充电枪六维位姿导的引即可。充电设备可以包括力传感器(或者可以为力矩传感器)，在一种可能的实现中，其能够感知枪头末端所受的力和力矩(Fx，Fy，Fz，Tx，Ty，Tz)，能够感知侧向力传感器均可。充电设备可以包括摄像头，能够获取充电口的2D图像和/或3D点云，用于识别充电座的6D位姿。充电设备可以包括充电枪(包括充电插头)及固定装置，充电设备可以包括移动平台，扩展充电装置的移 动范围。
下面结合本申请实施例中的附图对本申请实施例进行描述。本申请的实施方式部分使用的术语仅用于对本申请的具体实施例进行解释，而非旨在限定本申请。
参照图8，图8为本申请实施例提供的一种充电设备的实施例示意图，其中，移动充电设备可以为固定式充电桩、移动充电车辆、移动充电机器人等产品。
如图8示出的那样，本申请实施例提供的充电装置可以包括机械臂802，其中机械臂802可以包括：控制结构以及机械臂802主体。其中，控制机构可以为上述介绍的处理器131。
在一种可能的实现中，控制机构可以控制机械臂802主体的位姿，来实现充电任务，充电任务例如可以为：将机械臂802末端的充电插头803插入至车辆的充电口中。
在一种可能的实现中，在充电任务的开始阶段，可选的，充电设备可以接收充电指令(例如可以接收到来自服务器发送的充电指令，或者是直接接收到用户的终端设备发送的充电指令)，可选的，充电指令可以携带有被充电车的位置、被充电车辆的车型信息、快慢充信息等信息。充电设备可以基于接收到充电指令而开启充电任务的执行，触发开启移动状态。
在一种可能的实现中，充电设备需要移动至被充电车辆充电口附近才能开始机械臂802的操作，保证充电口在机械臂802可达空间范围内，此部分可以采用导航定位模式或者自动驾驶技术实现。
在一种可能的实现中，充电设备可以兼容快充枪和慢充枪两种充电方式，因此可以具备快换机构，可根据用户任务需求选取充电枪，这里采用磁吸式结构抓取充电枪，保证充电枪和机械臂802的可靠对接，然后进行下一步充电操作。
在一种可能的实现中，充电设备选择停靠点之后，移动底盘的任务基本达成，即将进行机械臂802的操作，那么需要知道充电座(包括充电口，本实施例为了方便描述，充电座也可以称之为充电口)的大概位置，才能控制机械臂802在该区域进行充电口的位置搜索。充电口的大致位置可以根据车型检测，也可以通过充电设备相机对被充电车辆进行车型识别，获取到被充电车辆的视觉信息结合车辆模型信息即可解算出充电口大致位置，将此位置发送给机械臂802，机械臂802即可在此区域进行下一步流程，即搜索阶段。
在一种可能的实现中，由于移动平台的停靠位置存在位置误差，相机视野，例如视场角(field of view，FOV)也会存在限制。很难保证在机械臂802末端的相机范围内保证充电口可见，因此有必要进行充电口的“搜索”策略，进行充电口的初步定位或者为粗定位。可选的，搜索方案可以基于机械臂802运动学中的扇形路径规划以及视觉感知中目标检测算法来共同实现。
示例性的，搜索过程的流程图可以参照图9。具体的，由于充电车停靠位置存在偏差，并且待充电车辆品牌、型号及停靠位置等的不同，加之机械臂802末端camera视角存在限制，因此为了保证充电口可见首先控制机械臂802进行指定区域的搜索。可以通过已知充电口的大致位置信息(如车辆停靠位置以及充电口高度、倾角等)生成搜索路径，控制机 械臂802完成路径跟随。搜索过程中同步进行充电口的目标检测与跟踪。
接下来介绍本申请实施例中的扇形轨迹搜索，通常情况下，新能源汽车的充电口高度分布在70c～100cm范围内，因此可在笛卡尔空间内结合相机FOV在此高度进行扇形轨迹搜索，搜索过程伴随实时充电口检测算法，检测到目标则驱动机械臂802运动使得目标处于视野中心，这样的方式有利于进入下一阶段的视觉伺服过程。
在一种可能的实现中，可以基于深度学习来实现充电口的目标快速和高精度检测与跟踪。为了提升任务快速性，高效率实现充电口搜索，可选的，可以采用改进的YoloV4深度学习目标检测算法结合templatetracking跟踪算法的方法。
其中，改进的YoloV4算法主要是将原网络结构中的特征提取网络在深度上分别对卷积层数和卷积核数进行压缩，改进后的模型特征提取网络的卷积层数下降了(例如可以由原本的73层下降至22层)，因此模型的加载和推理过程速度都有很大提升，并且精度几乎保持不变。
表1：改进的YoloV4算法与YoloV4算法对比
Figure PCTCN2022140276-appb-000001
改进的YoloV4目标检测算法可准确地检测出图像中充电口区域(box坐标)即(x,y,w,h)，将此区域图像作为目标区域传递给templatetracking跟踪算法，当下一帧图像到来时，跟踪算法将计算图像中与目标区域相关系数最高的区域作为跟踪结果输出。
传统的图像检测算法无法应对未知且昏暗的地下车库与品种繁多的车辆品牌、型号，而深度学习则不同，深度学习受海量数据驱动，具有极强的鲁棒性和泛化能力，但同时，深度学习算法的预测推理过程会进行大量计算，需要硬件设备(显卡)的算力支持。因此单一的深度学习算法或传统图像检测算法均无法得到我们预期的效果。而此方案利用深度学习算法对环境变化的鲁棒性与对陌生环境的泛化能力，确保复杂环境中可以成功检测到充电口目标，同时对原有算法模型进行了精简和压缩，提高了算法运行速度，并且结合了传统算法中的template tracking算法弥补了深度学习算法在便携式的工控机上因算力不足导致的无法实时检测的问题，实现了复杂环境中充电口目标的实时检测与跟踪。其中，检测结果的示意可以参照图10所示，方块区域(box)为检测结果输出，坐标为(x,y,w,h)，代表图像中的像素坐标。追踪效果可以参照图11所示，每一帧图像中在box为追踪区域，确保该区域存在充电口的检测结果。
通过上述针对于充电口的搜索，控制机构可以控制机械臂802的位姿，以便将充电口落到相机的视野范围内(可选的，落到相机的视野中心区域)。之后，控制机构可以可以基于视觉伺服的方式控制机械臂802的位姿，以便将机械臂802主体上的充电插头803引导至所述充电口周围。
需要说明的是，位姿(position and orientation)可以包括位置和位姿，位置可以可以由三维坐标系中的三个坐标值表示，位姿可以由绕各坐标轴旋转所规定的状态来表示。
在将机械臂802主体上的充电插头803引导至所述充电口周围时，需要知晓充电口相对于机械臂802主体(具体可以为机械臂802主体上设置充电插口的位置)的相对位姿，基于视觉传感器804采集的信息确定的位姿可以为视觉传感器804相对于充电口的位姿，而由于视觉传感器804在机械臂802上的位置是固定的，也就是说视觉传感器804和充电插头803之间的相对位置关系是固定的，因此在知晓视觉传感器804相对于充电口的位姿之后，相当于知道了充电插头803和充电口之间的相对位姿。
应理解，在获取充电口相对于视觉传感器804的相对位姿时，由于标定误差的存在，可能并不等同于充电座相对于机械臂802的相对位姿，这之间还会存在系统误差，且随着机械臂802基座与充电座相对位姿不同而发生变化，本申请可以通过标定解决两个坐标系之间存在的系统误差。
具体的，充电机器人要完成充电枪的插拔，应满足在完全插入时，充电枪在机械臂802坐标系下的位姿与在充电座坐标系下的位姿代表同一空间位姿。这首先要求知道充电座坐标系相对于机械臂802坐标系的位姿关系。由于机械臂802基座可能随时发生变化，因此这个位姿关系也是动态发生变化的。我们可以通过视觉感知的方式得到充电座相对于相机坐标系的位姿，然后通过手眼参数变换到机械臂802末端坐标系，再通过正运动学得到充电座相对于机械臂802坐标系的位姿。由于机械臂802系统误差、手眼标定误差和视觉识别误差的客观存在，通过这种方法得到的充电座位姿，和充电座位姿的真值是存在误差的，通常不能保证充电枪的插入。由于充电座位姿的真值无法通过其他方法获得，而本申请实施例中的机械臂802上可以安装有力传感器801，因此可以通过力传感器801采集的信息来进行基于力觉的自标定方法，来实现充电座真值的获取，并构建视觉获取的充电座位姿补偿矩阵进行观测位姿补偿。
在一种可能的实现中，假设充电插头803和充电口没有间隙，由于充电连接件为过渡配合，因此假设成立，认为充电插头803坐标系与充电口坐标系重合，定义世界坐标系为充电设备的基座W，末端坐标系定义在充电插头803的最末端e，充电口坐标系为O，相机坐标系为C，于是可以有：
Figure PCTCN2022140276-appb-000002
WT e＝ WT 6 6T C CT e；
第一步，在对接成功时，可得到：
Figure PCTCN2022140276-appb-000003
Figure PCTCN2022140276-appb-000004
利用正运动学的计算方法：
Figure PCTCN2022140276-appb-000005
可见：
Figure PCTCN2022140276-appb-000006
为允许一定的容错性，充电口和充电插头803之间可以存在一定的但非常小的误差，可以通过力觉自标定方法来确定正确的插入位姿
Figure PCTCN2022140276-appb-000007
完成对正确的真值插入位姿
Figure PCTCN2022140276-appb-000008
的搜索。具体办法可以示例性的为：
(1)利用充电插头803插入充电口到底。
(2)利用柔性调整充电插头803的位姿，使得所受力矩最小。
(3)程序控制调整充电插头803的位姿，使得除充电插头803轴向外，其他两个坐标方向受力和最小。
(4)在垂直于充电插头803轴向的两个方向(x，y方向)进行平移，得到x，y方向的最小值和最大值
(5)平移机械臂802末端充电插头803，使其停止在x，y方向的中值附近。
这样，通过力反馈可以找到真值插入位姿
Figure PCTCN2022140276-appb-000009
在第二步，在机械臂802基座与第一步保持一致的前提下，可以让机械臂802末端移动到最佳观测点处。
在最佳观测点处，可得到正运动学计算的末端位姿：
WT e＝ WT 6(θ 1) 6T e；
通过在最佳观测点观测到充电口位姿
Figure PCTCN2022140276-appb-000010
通过视觉得到的末端位姿：
Figure PCTCN2022140276-appb-000011
其中
Figure PCTCN2022140276-appb-000012
是视觉测量误差，
Figure PCTCN2022140276-appb-000013
是手眼参数的误差，
Figure PCTCN2022140276-appb-000014
是手眼标定的测量值，令上面两个公式相等，可以得到补偿量：
Figure PCTCN2022140276-appb-000015
得到补偿量之后，可以在最佳观测点观测到目标充电口，用如下公式来求取期望充电插头803位姿：
Figure PCTCN2022140276-appb-000016
结合公式：
Figure PCTCN2022140276-appb-000017
令上述两式相等，可得：
Figure PCTCN2022140276-appb-000018
进而可以求解出插入后的期望关节角
Figure PCTCN2022140276-appb-000019
(在机械臂802主体由多个关节组成的情况下，可以通过关节角来控制机械臂802主体的位姿)。
本申请提出了一种机械臂802与充电口相对位姿误差的力觉标定方法，通过力觉自标定搜索的技术得到机械臂802充电插头803与充电口完全插入的零位，并根据该零位给出定量补偿视觉观测到的充电口与机械臂802相对位姿误的误差的方法，以此解决机械臂802形变误差、视觉识别误差、手眼标定误差等汇集起来的的充电口相对机械臂802的绝对位姿误差导致不能插入充电插头803的问题。
在一种可能的实现中，机械臂802可以包括基台(或者称之为基座)以及机械臂802主体，机械臂802主体可以称之为手臂。手臂可以包括多个关节，关节之间依次连接，且相互之间具备旋转自由度。在机械臂802主体的末端(远离基座的一端)上，可以安装有力传感器801以及充电插口(例如可以顺序安装)。本申请实施例中，控制机构可以可以基于视觉伺服的方式控制机械臂802主体的位姿，以便将机械臂802主体上的充电插头803引导至所述充电口周围。
在一种可能的实现中，所述控制机构可以实时获取视觉传感器804针对于所述充电口的区域所采集的图像信息；根据所述图像信息中所述充电口的位置信息，得到所述机械臂802主体的实时位姿；根据所述实时位姿，控制所述机械臂802主体的位姿，以便将所述机械臂802主体上的充电插头803引导至所述充电口周围。
由于视觉伺服中采集的图像信息是实时获取的，基于实时获取的图像信息可以求解得到机械臂802主体的实时位姿(该实时位姿可以为机械臂802主体末端和待充电车辆的充电口之间的相对位姿)，由于在控制所述机械臂802主体的位姿时，可能会出现底座移动、障碍物的阻碍等情况，导致机械臂802主体的位姿出现误差，本申请可以基于实时位姿来修正机械臂802主体的位姿，可以提高机械臂802主体的位姿识别精度，进而可以更准确的将机械臂802主体上的充电插头803引导至所述充电口周围。
其中，可以通过控制机械臂802进行快速视觉伺服，实现对于充电座的快速导引，这里需要设计稳定的控制律实现精确而快速的闭环控制，机械臂802控制和视觉识别结果需要高度融合从系统角度设计控制算法，使得机械臂802运动进而保证末端充电枪能够快速逼近充电口，为下一步力控插拔做好准备。
在一种可能的实现中，充电口的位姿识别(例如6D位姿识别)是实现充电插拔功能 的最重要环节，精确的位置识别是进行插拔的前提条件和基础。同时要求算法还需要具有较高的鲁棒性，比如针对车库场景的昏暗条件。6D位置主要是指充电口在机械臂802末端camera坐标系中的位置(x，y，z)和位姿角度(Roll，Pitch，Yaw)；可选的，可以采用RGBD深度相机结合神经网络算法实现充电口的6D位姿识别，识别精度要求达到±1～2mm，角度精度±3°以内(坐标系之间的转换关系可以参照图16所示)。
在一种可能的实现中，采用三维关键点检测结合二维关键点检测网络的方法，实现低对比度充电口检测环境中获取高精度识别结果。
其中，视觉伺服是视觉结合运动进行的闭环控制方法。主要功能是实现机械臂802末端充电枪对于充电口的导引运功。大致流程是将充电枪/充电座视觉识别位姿结果作为输入，采用视觉伺服反馈控制思想计算驱动误差，采用基于位姿的视觉伺服实现机械臂802末端向目标位姿的精确导引。采用自适应增益方法，加快在小误差时的调整速度，实现快速的视觉伺服。此步骤中伺服结束时机械臂802将处于最佳观测点，此处已获取最精准的充电口的位姿。此时机械臂802将根据把充电插口导引至充电口附近，为下一步力控插入做准备。
更具体的，由于最佳观测点要保证一定的稳定性，才能保证目标观测位姿的精度。然而，移动机器人基座相对充电座的位姿可能会不断地发生变化。为实现在上述变化下，保障最佳观测点相对充电座的位姿一致，采用位姿的快速视觉伺服实现此目标。在一种可能的实现中，可以采用位姿反馈的视觉伺服思想，基于李氏稳定判据设计关节速度控制律，实现机械臂802末端向目标位置的精确导引。
具体实现方法为，系统在完成对充电座的搜索时，充电座在手眼相机的视野中，由此可以得到充电座到相机的位姿矩阵，这个位姿矩阵与期望的所设定的最佳观测点的位姿矩阵相比较，得到了一个误差差值。假设当前位姿向量
Figure PCTCN2022140276-appb-000020
其中
Figure PCTCN2022140276-appb-000021
表示充电座坐标系相对于相机坐标系的平移向量，θu表示充电坐标系相对相机坐标系的方向矢量，期望位姿向量
Figure PCTCN2022140276-appb-000022
0表示相机的方向和充电坐标系的方向在收敛时将保持一致。则定义误差值：e＝s *-s。
通过构建误差的微分与误差成-λ比例关系的方程，保证误差以指数形式衰减。这是较为简单的视觉伺服控制律：
Figure PCTCN2022140276-appb-000023
这种控制律通常会收敛速度太慢，在误差比较小的时候，可以设定自适应增益的伺服控制律：
Figure PCTCN2022140276-appb-000024
其中λ(||e||)为误差||e||的函数。如表2所示，可以构建正态型模糊隶属函数A(||e||)进行模糊控制，通过实验调整模糊控制律的参数进行自适应增益控制。依据所制定的模糊集合从极小、较小、中等、较大、极大的变化，其模糊规则数值满足a 1>a 2>a 3>a 4>a 5，数字a 1～a 5表示对误差||e||进行态势评分，误差越小评分越高。将误差输入值分别代入隶属函数求解隶属度，并按照隶属度最大原则进行反模糊化。若误差输入值在某一集合中隶 属度最大，则选择相应的规则值作为控制输出量，进而实现在误差小的时候衰减值变大，误差大的时候衰减值变小。
表2模糊隶属函数及模糊规则
Figure PCTCN2022140276-appb-000025
在每个伺服周期内，可以根据视觉感知的位姿误差，通过指数衰减方程得到位姿误差微分的值，位姿误差微分可表达为交互矩阵与相机的速度的乘积：
则相机速度与位姿向量的误差关系可表达为：其中
Figure PCTCN2022140276-appb-000026
为相机速度，
Figure PCTCN2022140276-appb-000027
为相机角速度：
Figure PCTCN2022140276-appb-000028
而相机速度又可以由雅可比矩阵与机械臂802关节速度的乘积表示，这样，就可以根据位姿误差求解处关节的伺服速度
Figure PCTCN2022140276-appb-000029
Figure PCTCN2022140276-appb-000030
可选的，可以对机械臂802周围进行障碍物检测，并通过势场法根据与障碍物的距离得到的反作用力作为减速度Vo，在有障碍物的时候，障碍物产生的势能将把机械臂802推开。定义：
Figure PCTCN2022140276-appb-000031
其中d是机械臂802末端到环境之间的最短距离，距离越小，势能越大，μ是常数。定义：
Figure PCTCN2022140276-appb-000032
上述公式表明势场法产生的加速度将指向降低势场势能方向。机械臂802关节总速度可求得为：
Figure PCTCN2022140276-appb-000033
即可将角速度作为控制输入给机械臂802发送控制指令，进而形成闭环控制，驱动机械臂802运行一个视觉伺服控制周期。
上述视觉伺服一直不断运行直到收敛，这时机械臂802带动相机移动到最佳观测点。
可选的，视觉伺服流程可以如图12所示。
本申请可以在视觉伺服中采用期望位姿补偿以及雅可比计算的补偿。
(1)期望位姿补偿。视觉伺服收敛的时候，得到的是相机实际位姿到达的期望观测位姿。即：
Figure PCTCN2022140276-appb-000034
此处
Figure PCTCN2022140276-appb-000035
代表相机的测量位姿，和理想相机c位姿存在补偿量
Figure PCTCN2022140276-appb-000036
因此期望值的设定根据补偿量进行补偿后作为实际期望值。
Figure PCTCN2022140276-appb-000037
(2)雅可比计算的补偿。由于采用的是实际相机位姿作为控制参数，因此雅可比矩阵可以为：
Figure PCTCN2022140276-appb-000038
抵达最佳观测点之后的观测补偿已在上述实施例中说明，这里不再赘述。
可选的，在一种可能的实现中，所述机械臂802主体上设置有所述视觉传感器804。
可选的，在一种可能的实现中，视觉传感器804还可以不设置在机械臂802主体上而是设置在第三方视角对应的位置上。
在一种可能的实现中，在进行上述相对位姿的确定时，需要视觉传感器804采集到针对于充电口的图像信息，并基于图像信息来确定充电口相对于视觉传感器804之间的相对位姿。
在一种可能的实现中，所述图像信息可以包括2D图像以及3D点云，进而控制机构可以根据所述2D图像以及预设的充电口3D模型，提取所述充电口对应的第一3D位置；根据所述3D点云，提取所述充电口对应的第二3D位置；将所述第一3D位置和所述第二3D位置进行融合，得到所述充电口的位姿。其中，充电口的位姿可以理解为充电口相对于视觉传感器804之间的相对位姿。
在一种可能的实现中，充电设备上的视觉传感器804可以为用于采集深度图像的深度传感器。深度传感器可以例如但不限于如下介绍：深度相机、飞行时间(time of fight，TOF)相机、或激光雷达、照相式扫描仪、或激光雷达探测与测量(light detection and ranging，LiDAR)传感器。控制机构采集3D点云可以通过但不限于如下方式：
A、基于获取的RGB图像和深度图像得到3D点云。
在本申请实施例中，控制机构可以通过部署于该机械臂802主体上的传感器采集RGB图像和深度图像，RGB图像和深度图像就是指传感器在当前时刻、当前位置能感知到的周 围环境形成的RGB图像和深度图像。例如，可以是通过部署于该充电设备上的深度相机采集得到RGB图像和对应的深度图像，也可以是通过部署于该充电设备上的深度传感器采集深度图像，并通过部署于该充电设备上的摄像模块(如，摄像头)采集到对应的RGB图像。
传感器采集到充电设备在当前时刻的RGB图像和深度图像之后，先对该RGB图像进行实例分割，如，可采用SOLOv2、BlendMask等算法对该RGB图像进行实例分割，输出实例分割结果，实例分割结果包括该RGB图像上每个区域所属的分类类别、属于该分类类别的置信度(不大于1)以及分割的掩膜(mask)，可将实例分割结果定义为Mi。之后，可以选择属于充电口的数据，并将实现了实例分割且重新定义了像素值的RGB图像(可称为处理后的RGB图像)与深度图像进行叠加，该处理后的RGB图像中的每个局部区域就具有了深度信息，具体地，由于处理后的RGB图像每个局部区域对应的mask在该RGB图像上占据的区域定义像素为1，其余区域定义像素为0，那么将该处理后的RGB图像与深度图像相乘，由于像素值为0的区域相乘后依然是0，因此相乘后的结果只保留了RGB图像中每个区域对应的区域以及每个区域对应的深度信息。之后，再通过相机内参恢复出充电口的3D点云。
需要说明的是，在本申请的一些实施方式中，还可以对实例分割结果以及深度图像分别先进行滤波操作，如，通过形态学滤波降低实例分割结果的外点、通过深度平滑消除深度图像的空洞部分等，总之，滤波的目的是为了去除噪声。
B、基于获取的RGB图像和原始激光点云数据得到多个3D点云。
在本申请实施例中，充电设备可以通过部署于该充电设备上的传感器采集、RGB图像和原始激光点云数据，例如，充电设备上可部署有激光雷达、普通的摄像头等传感器，通过摄像头采集RGB图像，同时通过该激光雷达采集原始激光点云数据，在这种情况下，则不需要获取深度图像，只需按照上述类似过程将RGB图像进行实例分割，然后将实例分割结果对应到同时获取的原始激光点云数据上，从而得到充电口对应的3D点云。
在一种可能的实现中，可有采用视觉传感器804来获取到二维图像和三维深度点云数据，二维图像可以通过检测关键点并通过与三维模型的同名关键点进行对应，可以求取目标的6D位姿，但这依赖对二维关键点的高精度检测，同时在通过2D-3D对应恢复6D信息的时候会引入重建误差。车库场景存在弱光线条件，不同充电口深度也有所不同，还存在一定色差。同时，三维点云也可以与目标物体的三维模型进行匹配得到目标6D位姿，但点云的精度略微差一些，需要采用融合二维图像和三维点云各自的重建信息，实现更高精度的定位。为实现对充电口的精确插入，需要得到充电口在机械臂802坐标系中的精确位姿，这可以通过传感器观测到充电口相对于传感器的位姿，再根据传感器安装位置由传感器相对于机械臂802末端的位姿折算到机械臂802末端，再由机械臂802末端通过正运动学折算到相对于机械臂802基座的相对位姿关系。其中，卷积神经网络结构的示意可以参照图14所示。
表3 2D关键点及序号检测卷积神经网络结构
Figure PCTCN2022140276-appb-000039
例如，可以通过第1到18层构成的特征提取器在图像上提取出高表征能力的特征图集合(feature maps)，而后利用三层全连接层，将特征图集合转换为维度为14的向量。计算该向量与图像上2D关键点真值组成的向量之间的均方差(MSE)作为网络模型的loss函数，不断输入数据、迭代模型直至loss收敛。最终可以获得一个高精度的关键点检测算法模型。
示例性的，可以得到七个关键点在充电口坐标系下的三维坐标见下表所示：
表4充电口七个关键点充电口坐标系下的三维坐标
Figure PCTCN2022140276-appb-000040
针对于3D点云关键点检测网络，在实际观测到的三维点云中，所期望的关键点可能并没有观测到，通过关键点检测网络，检测现有各点到关键点的方向向量，并通过投票的方式间接求得关键点的精确坐标。首先，从三维点云模型中选取若干个标准关键点，这通过最远点采样法求得。具体办法是，第一次通过随机采样的方式得到第一个点，其他每次都在剩余的点中选取与已选点集的距离最远的点，这样求取N个点作为标准关键点。其次，得到了关键点之后，由标准数据关键点以及人工生成变换后的点云生成训练数据集进行训练。输入的点云首先通过PointNet++网络进行特征提取，针对每一个点，有一个多层的全连接神经网络用以计算每个点到K个关键点的偏差，然后根据这些偏差采用3D霍夫投票网络进行投票聚类，并由聚类的点集求解得到精确的三维关键点(具体流程可以参照图17所示)。在进行模型训练时，可以把两个函数根据二维检测和三维检测的误差分别以α，β来进行加权，得到总的损失函数如下：
Loss＝∑ R,t(α||P 2d-proj(RP 3dgt+t)|| 2+β||P p3d-(RP p3dgt+t)|| 2)；
其中，P 2d是图像检测的关键点，P 3dgt是充电口坐标系下的三维点坐标，R，t是从充电口坐标系到相机坐标系的转移矩阵，P p3d是相机坐标系下的三维点云中检测的关键点三维坐标，P p3dgt三维模型中的关键点三维坐标。算法可以通过图像的2D-3D对应计算R，t初值，然后采用L-M法通过梯度下降求取最优位姿矩阵的解(例如可以参照图13所示)。
在一种可能的实现中，充电座的6Dpose(位置和姿态)识别是无人充电任务的关键环节，确保充电口的6D位姿准确才能进行机械臂的精确导引，进而采用力控技术进行柔顺 插拔。由于标准慢充充电口由七个孔组成，因此选取充电口的七个孔的圆心作为关键点，通过人工智能算法对上文中的目标区域进行关键点检测。此方案将二维图像上的7个关键点检测问题转化为14个数值的回归问题，即7个关键点坐标(x，y)构成的14维向量。基于VGG16网络结构，修改网络结构中的最后三层全连接层参数，使最后一层输出维度为14，将原本的softmax loss改为均方差loss。最终网络的输入为一张彩色图像，输入为维度为14的向量，即为图像上7个关键点的坐标值。
采用最远点采样法对点云进行降采样，该方法除了第一次随机采样之外，其他每次都在剩余的点中选取与已选点集的距离最远的点，采样后的点作为稠密点云的降采样版本，并和标准点云进行三维配准，得到三维点对应序列，并以标准三维关键点与目标坐标系关键点的通过R,t变换之后的距离为三维目标函数；
采用改进的VGG二维关键点检测网络，得到充电座圆孔中心点二维像素坐标，通过中心点的相互位置关系得到每个点的编号(例如可以参照图15所示)；根据充电座标准CAD模型得到圆孔中心点在目标坐标系下的三维物理坐标。二维关键像素点(像素坐标系下)、三维物理点(目标坐标系下)通过编号顺序一一对应，我们根据三维物理点反投影到像素坐标系下得到对应的反投影像素点，并比对二维关键点与反投影点之间的距离为二维损失函数项；
把两个函数根据二维检测和三维检测的误差分别以α，β来进行加权，得到总的损失函数。
本申请提出了一种融合2D图像-3D点云信息的充电口位姿精确识别方法，可以采用2D深度图像检测网络对充电口的二维关键点进行亚像素级的精确检测，采用三维关键点检测神经网络对充电口三维关键点进行基于3D霍夫网络的偏移量可靠检测，采用非线性优化使得融合二维关键点及三维关键点的投影误差最小的技术来求得充电口的精确位姿，以此解决充电口在光照变化大、目标对比度低情况下的位姿高精度识别问题。
通过上述视觉伺服的方式，可以将充电插头803引导至充电口附近，例如精度范围在1至2mm，2°的误差范围内，由于前一段规划基于视觉存在一定误差，同时考虑碰撞安全性，此时控制机械臂802不能与充电口接触，因此需要根据视觉信息进行一段运动，能让机械臂802主体上安装的力觉传感器感知到与充电口接触，然后可以进行基于力感知的自适应调整阶段(例如可以参照图18所示)。
本申请提出了一种自适应误差增益的视觉伺服导引方法，提出了结合势场避障的视觉伺服速度指令生成。视觉伺服在实际应用中要和环境交互，但又不能与环境发生碰撞导致破坏，采用结合势场避障的方法引入新的排斥速度因子，有效地保障了视觉伺服的安全性。二是提出了自适应增益的视觉伺服控制律，通过设定衰减指数与误差幅值之间的反向模糊关系，实现了在小误差下的大衰减系数，有效加快了视觉伺服的收敛速度。三是采用了基于位姿的视觉伺服，由于把位置和位姿信息作为特征向量，比基于图像的视觉伺服包含了更多更精确的信息，可有效地导引机械臂802到期望的最佳观测点处，通过上述方法解决了机械臂802与充电口位姿可能不断变化的情况下无碰撞导引相机停留在位姿相对固定的最佳观测点的问题。
在充电插头803接触到充电口之后，控制机构可以获取到力传感器801采集的作用力信息，该作用力信息可以指示充电口对充电插头803的作用力(或者力矩)，并根据所述作用力信息，控制所述机械臂802主体的位姿，以便将所述充电插头803插入至所述充电口。
在现有的只基于视觉进行位姿控制的方案中，由于视觉本身的误差，而且在充电插头803在充电口附近的时候视觉相机已无法继续观测到充电口，如果只采用在最佳观测点观测到的视觉位姿进行充电插头803的插拔，可能会发生不能插到底、楔牢的情况，本申请通过获取到力传感器801采集的作用力信息来进行机械臂802的位姿控制，可在充电插头803已部分插入，视觉信息无法获取误差的情况下，根据力觉调整位置和位姿，让充电插头803适应充电口的插入方向，从而实现了插拔任务的准确性以及可靠性。
接下来介绍本申请实施例中的力传感器801。
在一种可能的实现中，所述机械臂802主体上设置有力传感器801，所述力传感器801用于采集所述充电插头和所述充电口之间接触面的作用力信息，所述接触面包括所述多个充电头中至少一个充电头的侧面，所述侧面为和所述充电头的轴向方向平行的表面。
在一种可能的实现中，所述作用力信息包括所述充电头的径向方向的力。例如作用力信息可以包括6维的作用力信息。
在一种可能的实现中，所述作用力信息包括所述充电口对所述充电插头的力矩信息。
其中，作用力信息可以指示充电口对所述充电插头的6维力，6维力可以包括在三维空间内(例如沿着x轴、y轴、z轴方向)的力以及在三维空间内(例如绕着x轴、y轴、z轴旋转)的力矩。由于其中包括了力矩信息，则可以更精准更细粒度的识别出充电口对于充电头的阻碍方向，进而提供更精准的力控来将充电头准确插入充电孔。
在一种可能的实现中，所述力传感器801固定于所述充电插头803上(具有可以在至少一个充电头的侧面上)；或者，所述机械臂802包括具备旋转自由度的关节，所述控制结构用于通过控制所述关节的旋转来控制所述机械臂802主体的位姿，所述力传感器801固定于所述关节上；或者，所述力传感器固定在所述充电插头和所述机械臂主体之间。
在一种可能的实现中，力传感器801可以具有一个或多个力检测单元、A-D转换电路和信号处理电路。力检测单元各自包含力检测元件和放大力检测元件的输出信号的放大电路。放大电路将放大后的信号Va向A-D转换电路供给。A-D转换电路将数字转换后的信号Vd向信号处理电路供给。力检测单元的个数根据由力传感器801检测的检测轴的数量以及关于各检测轴的力的种类设定。例如，力传感器801在检测与互相正交的三个检测轴平行的力的大小和绕三个检测轴的力矩的大小时，使用十二个力检测单元。
在一种可能的实现中，作为力检测元件，例如可以使用压电式、应变片式或光学式等各种元件。
在一种可能的实现中，放大电路构成为作为将从力检测元件输出的电荷积分并转换成电压信号Va的积分电路。具体地，放大电路具有运算放大器、电容器和开关元件。运算放大器的负极输入端子与力检测元件的电极连接，运算放大器的正极输入端子与地(基准电位点)接地。而且，运算放大器的输出端子与A-D转换电路连接。电容器连接在运算放大器的负极输入端子与输出端子之间。开关元件与电容器并列连接。开关元件根据从信号处理电 路供给的接通/断开信号执行切换操作。
在一种可能的实现中，开关元件是断开时，从力检测元件输出的电荷在电容器中存储，其电压Va向A-D转换电路输出。另一方面，开关元件成为接通时，电容器的两端子间短路。其结果是电容器中存储的电荷被放电并成为零，向A-D转换电路输出的电压Va成为0伏特。将开关元件接通的操作相当于力传感器801的重置。
在一种可能的实现中，力转换部可以将多个力检测单元的输出转换为表示关于多个检测轴的力的F信号(i)并输出。需要说明的是，也将力信号F(i)简单地称为“力F(i)”。可选的，这些力F(i)包含与互相正交的三个检测轴平行的力Fx、Fy、Fz和绕三个检测轴的力矩Tx、Ty、Tz。力F(i)此外也包含将与三个检测轴平行的力Fx、Fy、Fz作为三个轴方向成分的力向量的大小Fmag和将绕三个检测轴的力矩Tx、Ty、Tz作为三个轴方向成分的力矩向量的大小Tmag。力向量的大小Fmag是三个力Fx、Fy、Fz的二次方和的平方根，力矩向量的大小Tmag是三个力矩Tx、Ty、Tz的二次方和的平方根(x、y、z轴的示意可以参照图19所示)。
在一种可能的实现中，所述充电插头803包括多个充电头，所述充电口包括多个充电孔，所述多个充电头与所述多个充电孔之间一一匹配；所述控制结构可以根据所述作用力信息，确定位姿调整参数，并根据所述位姿调整参数，将所述机械臂802主体的位姿由第一位姿矫正为第二位姿；其中，所述第一位姿和所述第二位姿不同，且所述第二位姿为每个所述充电头均和相匹配的充电孔对齐时所述机械臂802主体的位姿。
在一种可能的实现中，基于作用力信息，可以确定出机械臂802主体是否处于正确的位姿(例如第二位姿可以为正确的位姿)。例如，在作用力信息指示充电口对充电插头803存在非轴向方向的压力时，可以确定机械臂802主体未处于正确的位姿，在这种情况下，所述多个充电头中的至少一个充电头未和相匹配的充电孔对齐，也就是充电头的中心轴与相匹配的充电孔的中心轴不重合或者偏差大于阈值的情况。例如可以参照图22中的a、b、c、d。
在一种可能的实现中，所述第一位姿为部分所述充电插头803已插入至所述充电口，且所述多个充电头中的至少一个充电头未和相匹配的充电孔对齐时所述机械臂802主体的位姿。
在一种可能的实现中，所述第一位姿为所述多个充电头的底部端面未和所述充电孔的底部接触时的位姿，所述底部端面为和所述充电头的轴向方向垂直的表面。
在一种可能的实现中，所述对齐包括：充电头向充电孔方向的投影范围在相匹配的充电孔的范围内，且充电头的中心轴与相匹配的充电孔的中心轴重合或者偏差小于阈值。
也就是说，第一位姿可以为充电头的中心轴与相匹配的充电孔的中心轴不重合或者偏差大于阈值的情况(也就是上述所述的未处于正确的位姿)。
在一种可能的实现中，所述第一位姿为所述多个充电头的底部端面未和所述充电孔的底部接触时的位姿，所述底部端面为和所述充电头的轴向方向垂直的表面。
本申请实施例中，在机械臂802主体未处于正确的位姿时，可以基于作用力信息来调整机械臂802主体的位姿，以便机械臂802主体处于正确的位姿。
在一种可能的实现中，控制机构可以在基于机械臂802主体处于第一位姿时，根据作用 力信息对机械臂802主体的位姿进行矫正(至第二位姿，也就是正确的位姿)，第二位姿为机械臂802主体可以将充电插头803以较小阻力的正确方式插入到充电口底部的位姿。
示例性的，可以参照图22，图22中的a、b、c、d可以认为是第一位姿，e可以认为的第二位姿。
应理解，在从第一位姿调整至第二位姿时，充电口对所述充电头在目标方向上的阻力可以存在降低的趋势，目标方向可以径向方向和/或轴向状态(例如可以为径向方向、轴向状态、或者径向方向以及轴向方向)。这里所谓的存在降低的趋势，可以理解为充电口对所述充电头在目标方向上的阻力可以不是一直降低，而是可以存在局部的增加，但是从趋势上，充电口对所述充电头在目标方向上的阻力是降低的。
在一种可能的实现中，在所述由第一位姿矫正为第二位姿的过程中，所述充电口对所述充电插头之间的力矩存在降低的趋势。
参照图23，图23中三条曲线分别为充电插头803x，y，z三个方向上受到的合力，由接触力曲线图可以看出，使用力控算法后，x，y方向接触力逐渐减小，随着插入过程缩减到零点附近，说明自适应位姿调整大大降低了x，y轴方向方向充电口对枪体产生的压力，使用力控算法后，z方向插入接触力由从起始时刻的80N逐渐降低至60N，说明自适应调整阶段之后插拔力逐渐减小并保持稳定，直至可靠插入。从受力曲线可以看出，整个插入过程充电插头803根据力感知进行了快速的自适应位姿调整，并且各个方向合力处于收敛状态，保证具备较好的柔顺特性。
在一种可能的实现中，控制机构可以基于作用力信息来确定位姿调整参数，该位姿调整参数具体可以用于控制所述充电插头803的移动方向和/或移动速率(例如移动方向、移动速率、或移动方向以及移动速率)。
接下来结合一个具体示例介绍控制机构如何基于作用力信息来确定位姿调整参数，并基于位姿调整参数来控制机械臂802主体的位姿。
在一种可能的实现中，可以采用直接力反馈结合改进型导纳控制技术，参考远柔顺中心设备的原理进行柔顺参数调节，实现不同型号充电插头803在插入和拔出过程中的六自由度柔顺特性，保证对于充电口的可靠安全快速插拔操作。其中，可以将插入过程分为二个阶段，停泊点到充电口端口的自由运动，充电口端口到底部的柔顺插入运动。在自由运动阶段，利用位置伺服控制精确地将插头送入充电口端口内；然后，利用导纳控制产生充电插头803末端的柔顺运动，通过力/力矩反馈和设计的导纳矩阵获取机械臂802工具末端速度的修正量，从而改变理想的插入速度产生柔顺插入行为。整个插入过程，融合视觉感知到的位姿信息、力、力矩、位置、速度等信息，实现可靠插拔。
在一种可能的实现中，插拔环节是智能充电的核心环节，由于涉及到充电插头803和充电口的物理接触，因此就存在力的变化，尽管前序流程视觉已经给出了较为精准的充电口目标位姿，但是依然存在一定的误差会造成较大的接触力，同时插入过程中由于不同加工精度，疲劳程度，磨损程度等产生的力交互更为复杂，因此为了保证最后插入过程安全可靠，有必要进行基于力感知的主动柔顺控制。整个力控插拔流程要承接前序视觉信息，并要结合运动状态和力感知进行接触状态判断，保证可以进行插入后采用主动柔顺控制算 法实现充电插头803头的自适应调整。
示例性的，主动柔顺插拔坐标系可以定义为，坐标原点位于充电插头803端面的几何中心，z轴垂直于端面并指向充电插头803插入方向，y轴位于端面内朝上，x轴由右手法则确定。
导纳控制的基本原理为，设充电插头803的插入理想速度为V ideal＝[0,0,v z,0,0,0] T，即，以速度v z沿z轴做直线运动。如果接触力/力矩部位，则按下列公式在Cartesian空间计算实际的充电插头803参考速度：
V actual＝V ideal+A admittanceW contact；
式中，V actual是实际的充电插头803参考速度，W contact是接触力/力矩，A admittance是导纳矩阵。
充电设备实现导纳算法的原理如图20所示。选定导纳矩阵A admittance，依据由力/力矩传感器给出接触力/力矩测量计算出充电插头803在Cartesian空间的柔性速度修正量，与理想插入速度相加后得到充电设备在Cartesian空间的实际参考速度。然后，乘以逆Jacobi矩阵获得每一个关节的参考速度，从而在速度表征上控制充电设备产生柔顺插入运动。
当存在接触力/力矩时，RCC设备可以通过自身柔性调整插头位姿，实现与插孔对准，避免产生过大的接触力/力矩。通过选择A admittance具有RCC设备的导纳矩阵的形式，即，
Figure PCTCN2022140276-appb-000041
式中，k是标准RCC设备三个对称弹簧的刚度，α是三个对称弹簧形成的正四面体的中心轴线与弹簧所在边的夹角，l是柔顺远中心与充电插头803端面的距离，
Figure PCTCN2022140276-appb-000042
分别为绕位于柔顺远中心上与主动柔顺插拔坐标系平行的RCC设备坐标系x轴、y轴、z轴的旋转刚度。图21是标准RCC设备示意图。
因此，通过采用上述改进型导纳控制算法，充电插头803产生与RCC设备类似的柔顺行为，实现柔顺的插入过程，避免产生过大的接触力以及卡死、阻塞等现象。
本申请实施例中，结合力感知模型以及自适应调整导纳控制参数，实现根据(力和力矩)插拔力摩擦力自适应调整TCP末端位姿和速度，实现对于位置误差的高容限特性，提高插拔成功率，同时柔顺特性实现“最小”插拔力，提升插拔可靠性和安全性，具有较强的参数自适应能力，能够有效保障插拔完成时间；融合视觉末段位置感知以及插拔力感知，解算“完全插入”数学表征模型，有效判断是否完全插入，提升任务可靠性；同理拔出过程采用柔顺力控技反向实现可靠拔出，保证整体插拔任务可靠性。
图22展示了自适应主动位姿调整插入过程中，充电插头803枪头和充电口的关系，a状态为视觉伺服给出的充电插头803头停靠位置，可看出，存在一定的位置和位姿误差(示意图，实际误差值较小)，然后通过自由运动(此处也可以进行一定的搜索路径设计增强接触范围和速度)到达b状态，即接触检测，接触状态检测之后即将进入位姿的自适应调整阶段如图c，最终达到e的完全可靠插入状态，此过程中c和d流程将持续循环运行，直至满足可靠插入条件完成任务。因此整个插入过程中充电插头803是随着接触状态和接触了进行自适应调整的，进而保证插入过程持续减小摩擦力，降低侧向阻力保证可靠性安全性。
在一种可能的实现中，所述控制结构可以在将所述机械臂802主体的位姿由第一位姿矫正为第二位姿之后，控制所述机械臂802主体的位姿，以便所述充电插头803朝向所述充电孔底部且沿着所述充电头的中心轴的方向移动。
在将所述机械臂802主体的位姿由第一位姿矫正为第二位姿之后，充电插头803已经处于和充电口的对齐位姿，在这种情况下，可以将充电插头803朝向所述充电孔底部且沿着所述充电头的中心轴的方向移动直到底部，在一种可能的实现中，所述控制结构可以基于所述作用力信息指示所述充电口对所述充电头仅存在沿所述充电插头803的轴向方向的阻力，控制所述机械臂802主体保持静止状态，进而完成了向充电口的插入任务。
完成充电后，可以进行插入过程的逆向运动，同样采用主动柔性技术，自适应调整拔出过程中充电插头803的位姿。拔出完成后，机械臂802将根据设定完成归位，保证机械臂802收缩到较小的空间，避免底盘移动过程中对其他车辆等物体产生碰撞。机械臂802收回后即完成插拔任务，根据系统任务设定，充电充电设备移动至指定地点。等待下一次任务指令。
在一种可能的实现中，所述机械臂主体还包括目标接口，所述目标接口用于可拆卸连接所述充电插头。可选的，目标接口可以为基于电磁性来进行清扫工具的可拆卸连接的部件。采用磁吸式结构抓取充电枪，保证充电插头和机械臂的可靠对接，然后进行下一步充电操作。
在一种可能的实现中，在所述控制结构根据所述作用力信息，控制所述机械臂主体的位姿之前，所述目标接口与初始充电插头连接；所述控制结构还用于控制所述机械臂主体的位姿、以及所述目标接口的连接状态，将所述初始充电插头从所述目标接口上进行拆卸并替换为所述充电插头；其中，所述目标充电插头为支持快充充电的插头，所述充电插头为支持慢充充电的插头；或者，所述目标充电插头为支持慢充充电的插头，所述充电插头 为支持快充充电的插头。
本申请实施例中，充电设备可以兼容快充枪和慢充枪两种充电方式，可以具备快换机构，可根据用户任务需求选取充电枪。
在一种可能的实现中，所述充电设备还包括：插头收纳区域，所述插头收纳区域包括用于设置所述充电插头的目标收纳位置；控制机构可以控制所述目标接口的连接状态，将所述初始充电插头从所述目标接口上进行拆卸；控制所述机械臂主体的位姿，将所述目标接口移动至所述目标收纳位置，并控制所述目标接口的连接状态，将所述目标接口与所述充电插头进行连接。
本申请提供了一种充电设备，其特征在于，包括：机械臂、力传感器以及充电插头；所述机械臂包括控制结构以及机械臂主体；所述机械臂主体上设置有所述力传感器以及所述充电插头，所述充电插头包括多个充电头，所述力传感器用于采集所述充电插头和所述充电口之间接触面的作用力信息，所述接触面包括所述多个充电头中至少一个充电头的侧面，所述侧面为和所述充电头的轴向方向平行的表面；所述控制结构用于根据所述作用力信息，控制所述机械臂主体的位姿，以便将所述充电插头插入至所述充电口。在现有的只基于视觉进行位姿控制的方案中，由于视觉本身的误差，而且在充电插头在充电口附近的时候视觉相机已无法继续观测到充电口，如果只采用在最佳观测点观测到的视觉位姿进行充电插头的插拔，可能会发生不能插到底、楔牢的情况，本申请通过获取到力传感器采集的作用力信息来进行机械臂的位姿控制，可在充电插头已部分插入(例如侧面和充电口接触)，视觉信息无法获取误差的情况下，根据力觉调整位置和位姿，让充电插头适应充电口的插入方向，从而提高了插拔任务的准确性以及可靠性。减少了由于充电头侧面和充电口的内壁挤压或者摩擦时对充电头的磨损(或者是卡住而无法达到底部)的情况。
本申请实施例通过视觉和力觉融合，可有效降低对视觉定位精度的要求，降低了成本。这是因为力柔顺控制可对充电枪的位姿根据力反馈的状况进行调整，即便充电枪和充电座存在误差，从力反馈可进行辨识并通过调整充电枪的位姿，消除了充电座识别误差带来的不利影响。因此，视觉力觉的融合避免采用昂贵的结构光相机，降低成本同时降低末端机构的体积和重量。此外，本申请实施例还克服了拖拽线缆的影响。由于拖拽线缆会对充电枪的位姿造成影响，结合力觉的方法能通过辨识定量观测到拖拽线缆力的影响并在控制的时候施加力补偿，有效克服拖拽线缆的影响。采用视觉方法可快速找到充电口并通过融合2D-3D信息实现充电座位姿的高精度识别，降低了机械臂802到充电口的全过程时间，同时由于视觉精度较高，也减少了力控搜索正确插拔位姿的时间，使得全流程速度有较大提升。
此外，本申请实施例还提供了一种控制机器臂充电的方法，所述方法可以应用于上述实施例中所描述的机械臂802的控制结构，所述机械臂802主体上设置有力传感器801以及充电插头803，所述充电插头包括多个充电头，所述力传感器用于采集所述充电插头和所述充电口之间接触面的作用力信息，所述接触面包括所述多个充电头中至少一个充电头的侧面，所述侧面为和所述充电头的轴向方向平行的表面；
所述方法包括：
根据所述作用力信息，控制所述机械臂802主体的位姿，以便将所述充电插头803插 入至所述充电口。
在一种可能的实现中，所述作用力信息包括所述充电口对所述充电插头的力矩信息。
在一种可能的实现中，所述作用力信息包括所述充电头的径向方向的力。
在一种可能的实现中，所述力传感器固定于所述至少一个充电头的侧面；或者，
所述机械臂包括具备旋转自由度的关节，所述控制结构用于通过控制所述关节的旋转来控制所述机械臂主体的位姿，所述力传感器固定于所述关节上；或者，
所述力传感器固定在所述充电插头和所述机械臂主体之间。
在一种可能的实现中，所述充电口包括多个充电孔，所述多个充电头与所述多个充电孔之间一一匹配；所述根据所述作用力信息，控制所述机械臂主体的位姿，包括：
根据所述作用力信息，确定位姿调整参数；
根据所述位姿调整参数，将所述机械臂主体的位姿由第一位姿矫正为第二位姿；其中，所述第一位姿和所述第二位姿不同，且所述第二位姿为每个所述充电头均和相匹配的充电孔对齐时所述机械臂主体的位姿。
在一种可能的实现中，所述第一位姿为所述多个充电头的底部端面未和所述充电孔的底部接触时的位姿，所述底部端面为和所述充电头的轴向方向垂直的表面。
在一种可能的实现中，所述对齐包括：充电头沿相匹配的充电孔方向的投影落在所述相匹配的充电孔的范围内，且充电头的中心轴与相匹配的充电孔的中心轴重合或者偏差小于阈值。
在一种可能的实现中，在将所述机械臂主体的位姿由第一位姿矫正为第二位姿之后，所述方法还包括：
控制所述机械臂主体的位姿，以便所述充电插头朝向所述充电孔底部且沿着所述充电头的中心轴的方向移动。
在一种可能的实现中，在所述由第一位姿矫正为第二位姿的过程中，所述充电口对所述充电插头在目标方向上的阻力存在降低的趋势，所述目标方向为所述充电头的轴向方向或径向方向。
在一种可能的实现中，在所述由第一位姿矫正为第二位姿的过程中，所述充电口对所述充电插头之间的力矩存在降低的趋势。
在一种可能的实现中，所述方法还包括：
基于所述作用力信息指示所述充电口对所述充电插头仅存在沿所述充电插头的轴向方向的阻力，控制所述机械臂主体保持静止状态。
在一种可能的实现中，所述位姿调整参数具体用于控制所述充电插头的移动方向以及移动速率。
在一种可能的实现中，所述方法还包括：
根据所述作用力信息，控制所述机械臂主体的位姿之前，实时获取视觉传感器针对于所述充电口的区域所采集的图像信息；
根据所述图像信息中所述充电口的位置信息，得到所述机械臂主体的实时位姿；
根据所述实时位姿，控制所述机械臂主体的位姿，以便将所述机械臂主体上的充电插 头引导至所述充电口周围。
在一种可能的实现中，所述机械臂主体上设置有所述视觉传感器。
在一种可能的实现中，所述图像信息包括2D图像以及3D点云，所述方法还包括：
根据所述2D图像，提取所述充电口对应的2D位置；
根据所述3D点云，提取所述充电口对应的3D位置；
将所述2D位置和所述3D位置进行融合，得到所述充电口的位置信息。
在一种可能的实现中，所述机械臂主体还包括目标接口，所述目标接口用于可拆卸连接所述充电插头。
在一种可能的实现中，在所述控制结构根据所述作用力信息，控制所述机械臂主体的位姿之前，所述目标接口与初始充电插头连接；
所述方法还包括：控制所述机械臂主体的位姿、以及所述目标接口的连接状态，将所述初始充电插头从所述目标接口上进行拆卸并替换为所述充电插头；其中，
所述目标充电插头为支持快充充电的插头，所述充电插头为支持慢充充电的插头；或者，
所述目标充电插头为支持慢充充电的插头，所述充电插头为支持快充充电的插头。
在一种可能的实现中，所述充电设备还包括：插头收纳区域，所述插头收纳区域包括用于设置所述充电插头的目标收纳位置；
所述控制所述机械臂主体的位姿、以及所述目标接口的连接状态，将所述初始充电插头从所述目标接口上进行拆卸并替换为所述充电插头，包括：
控制所述目标接口的连接状态，将所述初始充电插头从所述目标接口上进行拆卸；
控制所述机械臂主体的位姿，将所述目标接口移动至所述目标收纳位置，并控制所述目标接口的连接状态，将所述目标接口与所述充电插头进行连接。
在一种可能的实现中，所述充电设备为固定式充电桩、移动机器人或者充电车。
关于控制机构的描述，可以参照上述实施例中关于机械臂802的控制结构的描述，这里不再赘述。
本申请实施例还提供了一种控制机器臂充电的装置，所述装置应用于机械臂802的控制结构，所述机械臂主体上设置有力传感器以及充电插头，所述充电插头包括多个充电头，所述力传感器用于采集所述充电插头和所述充电口之间接触面的作用力信息，所述接触面包括所述多个充电头中至少一个充电头的侧面，所述侧面为和所述充电头的轴向方向平行的表面；
所述装置包括：
控制模块，用于根据所述作用力信息，控制所述机械臂802主体的位姿，以便将所述充电插头803插入至所述充电口。
在一种可能的实现中，所述作用力信息包括所述充电头的径向方向的力。
在一种可能的实现中，所述作用力信息包括所述充电口对所述充电插头的力矩信息。
在一种可能的实现中，所述力传感器固定于所述至少一个充电头的侧面；或者，
所述机械臂包括具备旋转自由度的关节，所述控制结构用于通过控制所述关节的旋转 来控制所述机械臂主体的位姿，所述力传感器固定于所述关节上；或者，
所述力传感器固定在所述充电插头和所述机械臂主体之间。
在一种可能的实现中，所述充电口包括多个充电孔，所述多个充电头与所述多个充电孔之间一一匹配；所述控制模块，具体用于：
根据所述作用力信息，确定位姿调整参数；
根据所述位姿调整参数，将所述机械臂802主体的位姿由第一位姿矫正为第二位姿；其中，所述第一位姿和所述第二位姿不同，且所述第二位姿为每个所述充电头均和相匹配的充电孔对齐时所述机械臂802主体的位姿。
在一种可能的实现中，所述第一位姿为所述多个充电头的底部端面未和所述充电孔的底部接触时的位姿，所述底部端面为和所述充电头的轴向方向垂直的表面。
在一种可能的实现中，所述对齐包括：充电头沿相匹配的充电孔方向的投影落在所述相匹配的充电孔的范围内，且充电头的中心轴与相匹配的充电孔的中心轴重合或者偏差小于阈值。
在一种可能的实现中，在将所述机械臂802主体的位姿由第一位姿矫正为第二位姿之后，所述控制模块，还用于：
控制所述机械臂802主体的位姿，以便所述充电插头803朝向所述充电孔底部且沿着所述充电头的中心轴的方向移动。
在一种可能的实现中，在所述由第一位姿矫正为第二位姿的过程中，所述充电口对所述充电头在目标方向上的阻力存在降低的趋势，所述目标方向为所述充电插头803的轴向方向或径向方向。
在一种可能的实现中，在所述由第一位姿矫正为第二位姿的过程中，所述充电口对所述充电插头之间的力矩存在降低的趋势。
在一种可能的实现中，所述控制模块，还用于：
基于所述作用力信息指示所述充电口对所述充电头仅存在沿所述充电插头803的轴向方向的阻力，控制所述机械臂802主体保持静止状态。
在一种可能的实现中，所述位姿调整参数具体用于控制所述充电插头803的移动方向以及移动速率。
在一种可能的实现中，所述装置还包括：
获取模块，用于根据所述作用力信息，控制所述机械臂802主体的位姿之前，实时获取视觉传感器804针对于所述充电口的区域所采集的图像信息；
根据所述图像信息中所述充电口的位置信息，得到所述机械臂802主体的实时位姿；
所述控制模块，还用于：
根据所述实时位姿，控制所述机械臂802主体的位姿，以便将所述机械臂802主体上的充电插头803引导至所述充电口周围。
在一种可能的实现中，所述机械臂802主体上设置有所述视觉传感器804。
在一种可能的实现中，所述图像信息包括2D图像以及3D点云，所述获取模块，具体用于：
根据所述2D图像，提取所述充电口对应的2D位置；
根据所述3D点云，提取所述充电口对应的3D位置；
将所述2D位置和所述3D位置进行融合，得到所述充电口的位置信息。
在一种可能的实现中，所述机械臂主体还包括目标接口，所述目标接口用于可拆卸连接所述充电插头。
在一种可能的实现中，在所述控制结构根据所述作用力信息，控制所述机械臂主体的位姿之前，所述目标接口与初始充电插头连接；
所述方法还包括：控制所述机械臂主体的位姿、以及所述目标接口的连接状态，将所述初始充电插头从所述目标接口上进行拆卸并替换为所述充电插头；其中，
所述目标充电插头为支持快充充电的插头，所述充电插头为支持慢充充电的插头；或者，
所述目标充电插头为支持慢充充电的插头，所述充电插头为支持快充充电的插头。
在一种可能的实现中，所述充电设备还包括：插头收纳区域，所述插头收纳区域包括用于设置所述充电插头的目标收纳位置；
所述控制所述机械臂主体的位姿、以及所述目标接口的连接状态，将所述初始充电插头从所述目标接口上进行拆卸并替换为所述充电插头，包括：
控制所述目标接口的连接状态，将所述初始充电插头从所述目标接口上进行拆卸；
控制所述机械臂主体的位姿，将所述目标接口移动至所述目标收纳位置，并控制所述目标接口的连接状态，将所述目标接口与所述充电插头进行连接。
在一种可能的实现中，所述充电设备为固定式充电桩、移动机器人或者充电车。
接下来介绍本申请实施例提供的一种控制机械臂802充电的装置，请参阅图24，图24为本申请实施例提供的控制机械臂802充电的装置的一种结构示意图。具体的，控制机械臂802充电的装置2400包括：接收器2401、发射器2402、处理器2403和存储器2404(其中控制机械臂802充电的装置2400中的处理器2403的数量可以一个或多个，图24中以一个处理器为例)，其中，处理器2403可以包括应用处理器24031和通信处理器24032。在本申请的一些实施例中，接收器2401、发射器2402、处理器2403和存储器2404可通过总线或其它方式连接。
存储器2404可以包括只读存储器和随机存取存储器，并向处理器2403提供指令和数据。存储器2404的一部分还可以包括非易失性随机存取存储器(non-volatile random access memory，NVRAM)。存储器2404存储有处理器和操作指令、可执行模块或者数据结构，或者它们的子集，或者它们的扩展集，其中，操作指令可包括各种操作指令，用于实现各种操作。
处理器2403控制雷达系统(包括天线、接收器2401以及发射器2402)的操作。具体的应用中，雷达系统的各个组件通过总线系统耦合在一起，其中总线系统除包括数据总线之外，还可以包括电源总线、控制总线和状态信号总线等。但是为了清楚说明起见，在图中将各种总线都称为总线系统。
上述本申请实施例揭示的控制机械臂802充电的方法(图7所示的)可以应用于处理器2403中，或者由处理器2403实现。处理器2403可以是一种集成电路芯片，具有信号的处理能力。在实现过程中，上述方法的各步骤可以通过处理器2403中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器2403可以是通用处理器、数字信号处理器(digital signal processing，DSP)、微处理器或微控制器，还可进一步包括专用集成电路(application specific integrated circuit，ASIC)、现场可编程门阵列(field-programmable gate array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。该处理器2403可以实现或者执行本申请实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器2404，处理器2403读取存储器2404中的信息，结合其硬件完成上述实施例提供的控制机械臂802充电的方法的步骤。
接收器2401可用于接收输入的数字或字符信息，以及产生与雷达系统的相关设置以及功能控制有关的信号输入。发射器2402可用于通过第一接口输出数字或字符信息；发射器2402还可用于通过第一接口向磁盘组发送指令，以修改磁盘组中的数据。
本申请实施例中还提供一种包括计算机程序产品，当其在计算机上运行时，使得计算机执行上述实施例中描述的控制机械臂802充电的方法。
本申请实施例中还提供一种计算机可读存储介质，该计算机可读存储介质中存储有用于进行信号处理的程序，当其在计算机上运行时，使得计算机执行如上述实施例中描述的控制机械臂802充电的方法。
本申请实施例提供的控制机械臂802充电的装置具体可以为芯片，芯片包括：处理单元和通信单元，该处理单元例如可以是处理器，该通信单元例如可以是输入/输出接口、管脚或电路等。该处理单元可执行存储单元存储的计算机执行指令，以使执行设备内的芯片执行上述实施例描述的图像增强方法，或者，以使训练设备内的芯片执行上述实施例描述的图像增强方法。可选地，该存储单元为该芯片内的存储单元，如寄存器、缓存等，该存储单元还可以是该无线接入设备端内的位于该芯片外部的存储单元，如只读存储器(read-only memory，ROM)或可存储静态信息和指令的其他类型的静态存储设备，随机存取存储器(random access memory，RAM)等。
具体的，请参阅图25，图25为本申请实施例提供的芯片的一种结构示意图，该芯片可以表现为神经网络处理器NPU250，NPU 250作为协处理器挂载到主CPU(Host CPU)上，由Host CPU分配任务。NPU的核心部分为运算电路2503，通过控制器2504控制运算电路2503提取存储器中的矩阵数据并进行乘法运算。
在一些实现中，运算电路2503内部包括多个处理单元(Process Engine,PE)。在一些实现中，运算电路2503是二维脉动阵列。运算电路2503还可以是一维脉动阵列或者能够执行例如乘法和加法这样的数学运算的其它电子线路。在一些实现中，运算电路2503是通 用的矩阵处理器。
举例来说，假设有输入矩阵A，权重矩阵B，输出矩阵C。运算电路从权重存储器2502中取矩阵B相应的数据，并缓存在运算电路中每一个PE上。运算电路从输入存储器2501中取矩阵A数据与矩阵B进行矩阵运算，得到的矩阵的部分结果或最终结果，保存在累加器(accumulator)2508中。
统一存储器2506用于存放输入数据以及输出数据。权重数据直接通过存储单元访问控制器(direct memory access controller，DMAC)2505，DMAC被搬运到权重存储器2502中。输入数据也通过DMAC被搬运到统一存储器2506中。
BIU为Bus Interface Unit即，总线接口单元2510，用于AXI总线与DMAC和取指存储器(Instruction Fetch Buffer，IFB)2509的交互。
总线接口单元2510(Bus Interface Unit，简称BIU)，用于取指存储器2509从外部存储器获取指令，还用于存储单元访问控制器2505从外部存储器获取输入矩阵A或者权重矩阵B的原数据。
DMAC主要用于将外部存储器DDR中的输入数据搬运到统一存储器2506或将权重数据搬运到权重存储器2502中或将输入数据数据搬运到输入存储器2501中。
向量计算单元2507包括多个运算处理单元，在需要的情况下，对运算电路的输出做进一步处理，如向量乘，向量加，指数运算，对数运算，大小比较等等。主要用于神经网络中非卷积/全连接层网络计算，如Batch Normalization(批归一化)，像素级求和，对特征平面进行上采样等。
在一些实现中，向量计算单元2507能将经处理的输出的向量存储到统一存储器2506。例如，向量计算单元2507可以将线性函数和/或非线性函数应用到运算电路2503的输出，例如对卷积层提取的特征平面进行线性插值，再例如累加值的向量，用以生成激活值。在一些实现中，向量计算单元2507生成归一化的值、像素级求和的值，或二者均有。在一些实现中，处理过的输出的向量能够用作到运算电路2503的激活输入，例如用于在神经网络中的后续层中的使用。
控制器2504连接的取指存储器(instruction fetch buffer)2509，用于存储控制器2504使用的指令；
统一存储器2506，输入存储器2501，权重存储器2502以及取指存储器2509均为On-Chip存储器。外部存储器私有于该NPU硬件架构。
其中，上述任一处提到的处理器，可以是一个通用中央处理器，微处理器，ASIC，或一个或多个用于控制上述实施例中描述的控制机械臂802充电的方法相关步骤的程序执行的集成电路。
另外需说明的是，以上所描述的装置实施例仅仅是示意性的，其中该作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。另外，本申请提供的装置实施例附图中，模块之间的连接关系表示它们之间具有通信连接，具体可以实现为一条或 多条通信总线或信号线。
通过以上的实施方式的描述，所属领域的技术人员可以清楚地了解到本申请可借助软件加必需的通用硬件的方式来实现，当然也可以通过专用硬件包括专用集成电路、专用CPU、专用存储器、专用元器件等来实现。一般情况下，凡由计算机程序完成的功能都可以很容易地用相应的硬件来实现，而且，用来实现同一功能的具体硬件结构也可以是多种多样的，例如模拟电路、数字电路或专用电路等。但是，对本申请而言更多情况下软件程序实现是更佳的实施方式。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品存储在可读取的存储介质中，如计算机的软盘、U盘、移动硬盘、ROM、RAM、磁碟或者光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，训练设备，或者网络设备等)执行本申请各个实施例该的方法。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。
该计算机程序产品包括一个或多个计算机指令。在计算机上加载和执行该计算机程序指令时，全部或部分地产生按照本申请实施例该的流程或功能。该计算机可以是通用计算机、专用计算机、计算机网络、或者其他可编程装置。该计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一计算机可读存储介质传输，例如，该计算机指令可以从一个网站站点、计算机、训练设备或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、训练设备或数据中心进行传输。该计算机可读存储介质可以是计算机能够存储的任何可用介质或者是包含一个或多个可用介质集成的训练设备、数据中心等数据存储设备。该可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘(Solid State Disk，SSD))等。

Claims (29)

  1. 一种充电设备，其特征在于，包括：机械臂、力传感器以及充电插头；
    所述机械臂包括控制结构以及机械臂主体；
    所述机械臂主体上设置有所述力传感器以及所述充电插头，所述充电插头包括多个充电头，所述力传感器用于采集所述充电插头和所述充电口之间接触面的作用力信息，所述接触面包括所述多个充电头中至少一个充电头的侧面，所述侧面为和所述充电头的轴向方向平行的表面；
    所述控制结构用于根据所述作用力信息，控制所述机械臂主体的位姿，以便将所述充电插头插入至所述充电口。
  2. 根据权利要求1所述的充电设备，其特征在于，所述作用力信息包括所述充电口对所述充电插头的力矩信息。
  3. 根据权利要求1或2所述的充电设备，其特征在于，
    所述力传感器固定于所述至少一个充电头的侧面；或者，
    所述机械臂包括具备旋转自由度的关节，所述控制结构用于通过控制所述关节的旋转来控制所述机械臂主体的位姿，所述力传感器固定于所述关节上；或者，
    所述力传感器固定在所述充电插头和所述机械臂主体之间。
  4. 根据权利要求1至3任一所述的充电设备，其特征在于，所述充电口包括多个充电孔，所述多个充电头与所述多个充电孔之间一一匹配；所述控制结构具体用于：
    根据所述作用力信息，确定位姿调整参数；
    根据所述位姿调整参数，将所述机械臂主体的位姿由第一位姿矫正为第二位姿；其中，所述第一位姿和所述第二位姿不同，且所述第二位姿为每个所述充电头均和相匹配的充电孔对齐时所述机械臂主体的位姿。
  5. 根据权利要求4所述的充电设备，其特征在于，所述第一位姿为所述多个充电头的底部端面未和所述充电孔的底部接触时的位姿，所述底部端面为和所述充电头的轴向方向垂直的表面。
  6. 根据权利要求4或5所述的充电设备，其特征在于，所述对齐包括：充电头沿相匹配的充电孔方向的投影落在所述相匹配的充电孔的范围内，且充电头的中心轴与相匹配的充电孔的中心轴重合或者偏差小于阈值。
  7. 根据权利要求4至6任一所述的充电设备，其特征在于，所述控制结构还用于：
    在将所述机械臂主体的位姿由第一位姿矫正为所述第二位姿之后，控制所述机械臂主体的位姿，以便所述充电插头朝向所述充电孔底部且沿着所述充电头的中心轴的方向移动。
  8. 根据权利要求4至7任一所述的充电设备，其特征在于，在所述由第一位姿矫正为第二位姿的过程中，所述充电口对所述充电插头之间的力矩存在降低的趋势。
  9. 根据权利要求4至8任一所述的充电设备，其特征在于，所述位姿调整参数具体用于控制所述充电插头的移动方向以及移动速率。
  10. 根据权利要求1至9任一所述的充电设备，其特征在于，所述控制结构具体用于：
    基于所述作用力信息指示所述充电口对所述充电插头仅存在沿所述充电插头的轴向方向的阻力，控制所述机械臂主体保持静止状态。
  11. 根据权利要求1至10任一所述的充电设备，其特征在于，所述控制机构还用于：
    根据所述作用力信息，控制所述机械臂主体的位姿之前，实时获取视觉传感器针对于所述充电口的区域所采集的图像信息；
    根据所述图像信息中所述充电口的位置信息，得到所述机械臂主体的实时位姿；
    根据所述实时位姿，控制所述机械臂主体的位姿，以便将所述机械臂主体上的充电插头引导至所述充电口周围。
  12. 根据权利要求11所述的充电设备，其特征在于，所述机械臂主体上设置有所述视觉传感器。
  13. 根据权利要求11或12所述的充电设备，其特征在于，所述图像信息包括2D图像以及3D点云，所述控制机构还用于：
    根据所述2D图像以及预设的充电口3D模型，提取所述充电口对应的第一3D位置；
    根据所述3D点云，提取所述充电口对应的第二3D位置；
    将所述第一3D位置和所述第二3D位置进行融合，得到所述充电口的位姿。
  14. 根据权利要求13所述的充电设备，其特征在于，所述第一3D位置包括所述充电口上M个第一特征点的3D位置；所述第二3D位置包括所述充电口上M个第二特征点的3D位置；所述M个第一特征点和所述M个第二特征点一一对应；
    所述将所述第一3D位置和所述第二3D位置进行融合，包括：
    将所述M个第一特征点的3D位置分别和对应的所述第二特征点的3D位置进行融合。
  15. 根据权利要求1至14任一所述的充电设备，其特征在于，所述机械臂主体还包括目标接口，所述目标接口用于可拆卸连接所述充电插头。
  16. 根据权利要求15所述的充电设备，其特征在于，在所述控制结构根据所述作用力 信息，控制所述机械臂主体的位姿之前，所述目标接口与初始充电插头连接；
    所述控制结构还用于控制所述机械臂主体的位姿、以及所述目标接口的连接状态，将所述初始充电插头从所述目标接口上进行拆卸并替换为所述充电插头；其中，
    所述目标充电插头为支持快充充电的插头，所述充电插头为支持慢充充电的插头；或者，
    所述目标充电插头为支持慢充充电的插头，所述充电插头为支持快充充电的插头。
  17. 根据权利要求16所述的充电设备，其特征在于，所述充电设备还包括：插头收纳区域，所述插头收纳区域包括用于设置所述充电插头的目标收纳位置；
    所述控制所述机械臂主体的位姿、以及所述目标接口的连接状态，将所述初始充电插头从所述目标接口上进行拆卸并替换为所述充电插头，包括：
    控制所述目标接口的连接状态，将所述初始充电插头从所述目标接口上进行拆卸；
    控制所述机械臂主体的位姿，将所述目标接口移动至所述目标收纳位置，并控制所述目标接口的连接状态，将所述目标接口与所述充电插头进行连接。
  18. 根据权利要求1至17任一所述的充电设备，其特征在于，所述充电设备为固定式充电桩、移动机器人或者充电车。
  19. 一种控制机器臂充电的方法，其特征在于，所述方法应用于机械臂的控制结构，所述机械臂主体上设置有力传感器以及充电插头，所述充电插头包括多个充电头，所述力传感器用于采集所述充电插头和所述充电口之间接触面的作用力信息，所述接触面包括所述多个充电头中至少一个充电头的侧面，所述侧面为和所述充电头的轴向方向平行的表面；
    所述方法包括：
    根据所述作用力信息，控制所述机械臂主体的位姿，以便将所述充电插头插入至所述充电口。
  20. 根据权利要求19所述的方法，其特征在于，所述作用力信息包括所述充电口对所述充电插头的力矩信息。
  21. 根据权利要求19或20所述的方法，其特征在于，所述充电口包括多个充电孔，所述多个充电头与所述多个充电孔之间一一匹配；所述根据所述作用力信息，控制所述机械臂主体的位姿，包括：
    根据所述作用力信息，确定位姿调整参数；
    根据所述位姿调整参数，将所述机械臂主体的位姿由第一位姿矫正为第二位姿；其中，所述第一位姿和所述第二位姿不同，且所述第二位姿为每个所述充电头均和相匹配的充电孔对齐时所述机械臂主体的位姿。
  22. 根据权利要求21所述的方法，其特征在于，所述第一位姿为所述多个充电头的底部端面未和所述充电孔的底部接触时的位姿，所述底部端面为和所述充电头的轴向方向垂直的表面。
  23. 根据权利要求21或22所述的方法，其特征在于，在将所述机械臂主体的位姿由第一位姿矫正为第二位姿之后，所述方法还包括：
    控制所述机械臂主体的位姿，以便所述充电插头朝向所述充电孔底部且沿着所述充电头的中心轴的方向移动。
  24. 根据权利要求19至23任一所述的方法，其特征在于，所述方法还包括：
    基于所述作用力信息指示所述充电口对所述充电插头仅存在沿所述充电插头的轴向方向的阻力，控制所述机械臂主体保持静止状态。
  25. 根据权利要求19至24任一所述的方法，其特征在于，所述方法还包括：
    根据所述作用力信息，控制所述机械臂主体的位姿之前，实时获取视觉传感器针对于所述充电口的区域所采集的图像信息；
    根据所述图像信息中所述充电口的位置信息，得到所述机械臂主体的实时位姿；
    根据所述实时位姿，控制所述机械臂主体的位姿，以便将所述机械臂主体上的充电插头引导至所述充电口周围。
  26. 根据权利要求25所述的方法，其特征在于，所述图像信息包括2D图像以及3D点云，所述方法还包括：
    根据所述2D图像以及预设的充电口3D模型，提取所述充电口对应的第一3D位置；
    根据所述3D点云，提取所述充电口对应的第二3D位置；
    将所述第一3D位置和所述第二3D位置进行融合，得到所述充电口的位姿。
  27. 根据权利要求26所述的方法，其特征在于，所述第一3D位置包括所述充电口上M个第一特征点的3D位置；所述第二3D位置包括所述充电口上M个第二特征点的3D位置；所述M个第一特征点和所述M个第二特征点一一对应；
    所述将所述第一3D位置和所述第二3D位置进行融合，包括：
    将所述M个第一特征点的3D位置分别和对应的所述第二特征点的3D位置进行融合。
  28. 一种计算机可读存储介质，其特征在于，包括计算机可读指令，当所述计算机可读指令在计算机设备上运行时，使得所述计算机设备执行权利要求19至27任一项所述的方法。
  29. 一种计算机程序产品，其特征在于，包括计算机可读指令，当所述计算机可读指令在计算机设备上运行时，使得所述计算机设备执行如权利要求19至27任一所述的方法。
PCT/CN2022/140276 2021-12-20 2022-12-20 一种充电设备以及控制机械臂充电的方法 WO2023116667A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111562748.7A CN116278880A (zh) 2021-12-20 2021-12-20 一种充电设备以及控制机械臂充电的方法
CN202111562748.7 2021-12-20

Publications (1)

Publication Number Publication Date
WO2023116667A1 true WO2023116667A1 (zh) 2023-06-29

Family

ID=86811783

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/140276 WO2023116667A1 (zh) 2021-12-20 2022-12-20 一种充电设备以及控制机械臂充电的方法

Country Status (2)

Country Link
CN (1) CN116278880A (zh)
WO (1) WO2023116667A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117245651A (zh) * 2023-09-12 2023-12-19 北京小米机器人技术有限公司 机械臂插拔控制方法、装置、设备及存储介质
CN117718986A (zh) * 2024-02-08 2024-03-19 宁德时代新能源科技股份有限公司 电池测试系统及电池测试方法

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117584138B (zh) * 2024-01-18 2024-04-09 河南新科起重机股份有限公司 一种基于换电站三维定位抓取的智能运动控制调节系统
CN117697769B (zh) * 2024-02-06 2024-04-30 成都威世通智能科技有限公司 一种基于深度学习的机器人控制系统和方法
CN117841751B (zh) * 2024-03-08 2024-05-17 沧州智慧城科技有限公司 基于意图识别的充电枪系统及充电方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104520134A (zh) * 2012-07-27 2015-04-15 库卡罗伯特有限公司 用于对机动车辆充电的充电系统和方法
CN107618396A (zh) * 2017-10-19 2018-01-23 环球车享汽车租赁有限公司 自动充电系统及方法
CN108146264A (zh) * 2016-12-05 2018-06-12 库卡罗伯特有限公司 借助于机器人系统给机动车补给
CN110920450A (zh) * 2019-10-25 2020-03-27 北京华商三优新能源科技有限公司 一种电动汽车全自动充电系统
WO2020237642A1 (en) * 2019-05-31 2020-12-03 Abb Schweiz Ag Apparatus and method for charging electric vehicle, and method for calibrating apparatus for charging electric vehicle
CN112070835A (zh) * 2020-08-21 2020-12-11 达闼机器人有限公司 机械臂位姿预测方法、装置、存储介质及电子设备
CN215070651U (zh) * 2021-05-19 2021-12-07 广州小鹏汽车科技有限公司 充电枪及充电桩

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104520134A (zh) * 2012-07-27 2015-04-15 库卡罗伯特有限公司 用于对机动车辆充电的充电系统和方法
CN108146264A (zh) * 2016-12-05 2018-06-12 库卡罗伯特有限公司 借助于机器人系统给机动车补给
CN107618396A (zh) * 2017-10-19 2018-01-23 环球车享汽车租赁有限公司 自动充电系统及方法
WO2020237642A1 (en) * 2019-05-31 2020-12-03 Abb Schweiz Ag Apparatus and method for charging electric vehicle, and method for calibrating apparatus for charging electric vehicle
CN110920450A (zh) * 2019-10-25 2020-03-27 北京华商三优新能源科技有限公司 一种电动汽车全自动充电系统
CN112070835A (zh) * 2020-08-21 2020-12-11 达闼机器人有限公司 机械臂位姿预测方法、装置、存储介质及电子设备
CN215070651U (zh) * 2021-05-19 2021-12-07 广州小鹏汽车科技有限公司 充电枪及充电桩

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117245651A (zh) * 2023-09-12 2023-12-19 北京小米机器人技术有限公司 机械臂插拔控制方法、装置、设备及存储介质
CN117718986A (zh) * 2024-02-08 2024-03-19 宁德时代新能源科技股份有限公司 电池测试系统及电池测试方法

Also Published As

Publication number Publication date
CN116278880A (zh) 2023-06-23

Similar Documents

Publication Publication Date Title
WO2023116667A1 (zh) 一种充电设备以及控制机械臂充电的方法
US11132780B2 (en) Target detection method, training method, electronic device, and computer-readable medium
US20230247015A1 (en) Pixelwise Filterable Depth Maps for Robots
US11580724B2 (en) Virtual teach and repeat mobile manipulation system
CN112734765B (zh) 基于实例分割与多传感器融合的移动机器人定位方法、系统及介质
CN113591518B (zh) 一种图像的处理方法、网络的训练方法以及相关设备
US11587302B2 (en) Shared dense network with robot task-specific heads
US11769269B2 (en) Fusing multiple depth sensing modalities
US20220355495A1 (en) Robot Docking Station Identification Surface
US20240153314A1 (en) Engagement Detection and Attention Estimation for Human-Robot Interaction
CN113961013A (zh) 一种基于rgb-d slam的无人机路径规划方法
CN114240769A (zh) 一种图像处理方法以及装置
US11440196B1 (en) Object association using machine learning models
US11656923B2 (en) Systems and methods for inter-process communication within a robot
EP3842888A1 (en) Pixelwise filterable depth maps for robots
US20220168909A1 (en) Fusing a Static Large Field of View and High Fidelity Moveable Sensors for a Robot Platform
CN113066124A (zh) 一种神经网络的训练方法以及相关设备
CN116704391A (zh) 一种对接控制方法及其装置
US20220268939A1 (en) Label transfer between data from multiple sensors
EP4053804A1 (en) Joint training of a narrow field of view sensor with a global map for broader context
RU2819023C1 (ru) Способ и устройство для обнаружения препятствий, самоходный робот и носитель для хранения данных
CN116701586A (zh) 一种数据处理方法及其相关装置
CN117077073A (zh) 一种多模态数据的处理方法及相关装置
Shirai Dept. of Computer-Controlled Machinery, Faculty of Engineering, Osaka University Yamadaoka, Suita, Osaka, 565, Japan

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22909979

Country of ref document: EP

Kind code of ref document: A1