CN117428788B

CN117428788B - Equipment control method and device, electronic equipment and storage medium

Info

Publication number: CN117428788B
Application number: CN202311713195.XA
Authority: CN
Inventors: 王春茂; 夏若桓; 王彪; 陈祖文
Original assignee: Hangzhou Hikrobot Co Ltd
Current assignee: Hangzhou Hikrobot Co Ltd
Priority date: 2023-12-13
Filing date: 2023-12-13
Publication date: 2024-04-05
Anticipated expiration: 2043-12-13
Also published as: CN117428788A

Abstract

The application provides an equipment control method, an equipment control device, electronic equipment and a storage medium, relates to the field of industrial automation, and is used for solving the problem that grabbing equipment grabs articles with different placement postures. The method comprises the following steps: acquiring first position information of a plurality of preset positioning points on a target object, and determining a first pose of the target object according to the first position information of the plurality of preset positioning points; determining grabbing control information based on the first pose and grabbing transition information, wherein the grabbing transition information characterizes the relation between the equipment pose of the grabbing equipment and the pose of the characteristic object when the grabbing equipment grabs the characteristic object; the target object is any one of the characteristic objects; the grabbing control information comprises the pose of first target equipment when the grabbing equipment grabs the target object; and controlling the grabbing device to grab the target object in the pose of the first target device based on the grabbing control information.

Description

Equipment control method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of industrial automation, and in particular, to a device control method, an apparatus, an electronic device, and a storage medium.

Background

With the development of industrial automation, more and more automatic machines are widely used. For example, automatic gripping devices that grip articles play an important role in warehousing, logistics, and industrial assembly.

In a scene where an automatic gripping device such as a robot arm grips a workpiece, since the workpiece is generally large, the placement position is not easily controlled accurately, and thus gripping cannot be performed in the same manner each time. And because the shapes of different workpieces are different, accurate pose estimation is difficult to carry out on different workpieces, so that the conventional grabbing method is not suitable for the workpieces.

Therefore, how to enable the grabbing device to grab articles with different placement postures becomes a technical problem to be solved.

Disclosure of Invention

The application provides an equipment control method, an equipment control device, electronic equipment and a storage medium, which are used for solving the problem that grabbing equipment grabs articles with different placement postures.

In a first aspect, the present application provides an apparatus control method applied to a gripping apparatus, the method including: acquiring first position information of a plurality of preset positioning points on a target object, and determining a first pose of the target object according to the first position information of the plurality of preset positioning points; determining grabbing control information based on the first pose and grabbing transition information, wherein the grabbing transition information characterizes the relation between the equipment pose of the grabbing equipment and the pose of the characteristic object when the grabbing equipment grabs the characteristic object; the target object is any one of the characteristic objects; the grabbing control information comprises the pose of first target equipment when the grabbing equipment grabs the target object; and controlling the grabbing device to grab the target object in the pose of the first target device based on the grabbing control information.

According to the technical scheme, the equipment control method of the application firstly obtains the position information of the preset positioning point on the target object, so that the first pose of the target object is determined according to the first position information. Since the preset positioning point is a point of the preset position on the characteristic object, and the target object is used as one of the characteristic objects, the relative relation between the gesture of the target object and the preset positioning point can be known, and the relative relation between the gesture of any other characteristic object and the preset positioning point is the same. Therefore, the first pose of the current target object can be determined by presetting the first position information of the positioning point. When the gripping device grips the characteristic object in the same way, the pose of the gripping device gripping the characteristic object has a constant relative relation with the pose of the characteristic object, so that gripping transition information representing the constant relative relation between the pose of the gripping device gripping the characteristic object and the pose of the characteristic object and the first pose of the target object can be used to determine gripping control information when the gripping device grips the target object in the same way. Thereby controlling the gripping device to grip the target object according to the gripping control information. Therefore, the method can grasp the target object with different poses according to the real poses of the target object because the grasping control information of the method is determined according to the first poses of the target object which are actually obtained. In addition, the pose of the target object is determined according to the first position information of the preset positioning point of the target object, and the method is independent of whether the color and texture of the target object are rich and whether the geometric details are prominent, so that the method can be widely applied to various objects.

In one possible implementation manner, obtaining first position information of a plurality of preset positioning points on the target object, and determining a first pose of the target object according to the first position information of the plurality of preset positioning points includes: acquiring first position information of a plurality of preset positioning points on a target object, determining fitting center points of the plurality of preset positioning points and position information of the fitting center points according to the first position information of the plurality of preset positioning points on the target object, and taking the position information of the fitting center points as the position information of the target object; determining the attitude information of the target object based on the first position information of each preset positioning point; the first pose of the target object includes pose information and position information of the target object.

In one possible implementation manner, determining pose information of the target object based on the first position information of each preset positioning point includes: performing plane fitting on at least three preset positioning points according to first position information of the at least three preset positioning points to obtain a fitting plane; an angle of rotation of the first direction relative to a first coordinate axis of a base coordinate system of the grabbing device is used as first posture information of the target object; the first direction is the normal direction of the fitting plane; determining a second direction according to projection information of each preset positioning point on the fitting plane, and taking the rotation angle of the second direction relative to a second coordinate axis of the base coordinate system as second posture information of the target object; the second direction is perpendicular to the first direction; taking the angle of rotation of the third direction relative to the third coordinate axis of the base coordinate system as third posture information of the target object; the third direction is perpendicular to the second direction and the third direction; the pose information of the target object includes first pose information, second pose information, and third pose information.

In one possible implementation manner, determining the second direction according to the projection information of each preset positioning point on the fitting plane includes: determining projection vectors corresponding to all preset positioning points according to projection information of all preset positioning points on a fitting plane, wherein the projection vector corresponding to one preset positioning point points to the projection point of one preset positioning point on the fitting plane from the projection point of the fitting center point on the fitting plane; and taking the direction corresponding to the target vector in the projection vector as an initial second direction, and correcting the initial second direction according to the directions corresponding to the rest projection vectors to obtain a second direction.

In one possible implementation manner, correcting the initial second direction according to the directions corresponding to the rest of the projection vectors to obtain a second direction includes: taking a projection point of the fitting center point on the fitting plane as an origin, and establishing a two-dimensional target coordinate system in the fitting plane; rotating the projection vector outside the target quadrant to the target quadrant according to a preset rule to obtain a corresponding rotation vector; the target quadrant is the quadrant to which the target vector belongs in the target coordinate system; the preset rule comprises a rotation direction and a rotation angle, wherein the rotation direction is anticlockwise or clockwise, and the rotation angle is determined according to the quadrant of the projection vector in the target coordinate system; adjusting the included angle between the initial second direction and the target coordinate axis by the included angle between the vector in the target quadrant and the target coordinate axis to obtain a second direction; the vectors in the target quadrant comprise projection vectors corresponding to at least one projection point and/or rotation vectors corresponding to at least one projection vector, and the target coordinate axis is any coordinate axis forming the target quadrant.

In one possible implementation, the capture transition information is a capture transition matrixThe method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>To characterize the device to be graspedPose matrix of pose of captured characteristic object, < +.>To characterize a pose matrix of a device pose of a gripping device when the gripping device grips a feature object.

In one possible implementation, before determining the capture control information based on the first pose and the capture transition information, the device control method further includes: acquiring second position information of a preset positioning point on the characteristic object, and determining a second pose of the characteristic object; the grabbing equipment is controlled to grab the characteristic object, and the equipment pose when the grabbing equipment grabs the characteristic object is determined; and determining grabbing transition information according to the relation between the second pose of the characteristic object and the pose of the equipment when the grabbing equipment grabs the characteristic object.

In one possible implementation, the device control method further includes: under the condition that the grabbing equipment in the grabbing state operates to a preset pose, determining third position information of a preset locating point of the target object; the grabbing state represents a state that grabbing equipment is in grabbing a target object; determining first deviation information according to third position information of a preset positioning point and preset standard position information corresponding to the preset positioning point; acquiring position information of a target placement bit, and determining second deviation information according to the position information of the target placement bit and the position information of a preset placement bit; determining target placement control information based on the first deviation information, the second deviation information and preset placement control information, wherein the preset placement control information is control information when the grabbing device is in a grabbing state and is used for placing the characteristic object at a preset placement position under the condition that the grabbing device is operated to a preset pose and the position information of a preset positioning point on the characteristic object is preset standard position information; and controlling the grabbing equipment to place the target object to the target placing position according to the target placing control information.

In one possible implementation manner, before determining the third position information of the preset positioning point of the target object in the case that the gripping device in the gripping state is operated to the preset pose, the device control method further includes: determining the position information of the upper preset positioning point of the characteristic object captured by the capturing device with the preset gesture as preset standard position information; and controlling the grabbing equipment to place the characteristic object on a preset placement position, and determining preset placement control information.

In a second aspect, the present application provides an apparatus control device, comprising: the pose determining module is used for acquiring first position information of a plurality of preset positioning points on the target object and determining a first pose of the target object according to the first position information of the plurality of preset positioning points; the control information determining module is used for determining grabbing control information based on the first pose and grabbing transition information, wherein the grabbing transition information characterizes the relation between the pose of equipment of the grabbing equipment and the pose of the characteristic object when the grabbing equipment grabs the characteristic object; the target object is any one of the characteristic objects; the grabbing control information comprises the pose of first target equipment when the grabbing equipment grabs the target object; and the control module is used for controlling the grabbing device to grab the target object according to the pose of the first target device based on the grabbing control information.

In one possible implementation, the pose determining module includes a position information determining unit and a pose information determining unit. The position information determining unit is used for acquiring first position information of a plurality of preset positioning points on the target object, determining fitting center points of the plurality of preset positioning points and position information of the fitting center points according to the first position information of the plurality of preset positioning points on the target object, and taking the position information of the fitting center points as the position information of the target object; a gesture information determining unit for determining gesture information of the target object based on the first position information of each preset positioning point; the first pose of the target object includes pose information and position information of the target object.

In one possible implementation, the pose information determining unit includes a plane fitting subunit, a first pose information determining subunit, a second pose information determining subunit, and a third pose information determining subunit. The plane fitting subunit is used for carrying out plane fitting on at least three preset positioning points according to the first position information of the at least three preset positioning points to obtain a fitting plane; a first posture information determination subunit configured to use, as first posture information of the target object, an angle by which the first direction is rotated with respect to a first coordinate axis of the base coordinate system of the grasping apparatus; the first direction is the normal direction of the fitting plane; the second attitude information determining subunit is used for determining a second direction according to the projection information of each preset positioning point on the fitting plane, and taking the rotation angle of the second direction relative to a second coordinate axis of the base coordinate system as second attitude information of the target object; a third posture information determination subunit configured to have the second direction perpendicular to the first direction; taking the angle of rotation of the third direction relative to the third coordinate axis of the base coordinate system as third posture information of the target object; the third direction, the second direction and the first direction are mutually perpendicular; the pose information of the target object includes first pose information, second pose information, and third pose information.

In one possible implementation manner, the second gesture information determining subunit is specifically configured to determine, according to projection information of each preset positioning point on the fitting plane, a projection vector corresponding to each preset positioning point, where the projection vector corresponding to one preset positioning point points to a projection point of one preset positioning point on the fitting plane from a projection point of the fitting center point on the fitting plane; and taking the direction corresponding to the target vector in the projection vector as an initial second direction, and correcting the initial second direction according to the directions corresponding to the rest projection vectors to obtain a second direction.

In one possible implementation manner, the second gesture information determining subunit is specifically configured to establish a two-dimensional target coordinate system in the fitting plane with a projection point of the fitting center point on the fitting plane as an origin; rotating the projection vector outside the target quadrant to the target quadrant according to a preset rule to obtain a corresponding rotation vector; the target quadrant is the quadrant to which the target vector belongs in the target coordinate system; the preset rule comprises a rotation direction and a rotation angle, wherein the rotation direction is anticlockwise or clockwise, and the rotation angle is determined according to the quadrant of the projection vector in the target coordinate system; adjusting the included angle between the initial second direction and the target coordinate axis by the included angle between the vector in the target quadrant and the target coordinate axis to obtain a second direction; the vectors in the target quadrant comprise projection vectors corresponding to at least one projection point and/or rotation vectors corresponding to at least one projection vector, and the target coordinate axis is any coordinate axis forming the target quadrant.

In one possible implementation, the capture transition information is a capture transition matrixThe method comprises the steps of carrying out a first treatment on the surface of the Wherein (1)>For a pose matrix representing the pose of a feature object to be grasped by a grasping device, a +.>To characterize a pose matrix of a device pose of a gripping device when the gripping device grips a feature object.

In one possible implementation manner, the device control apparatus further includes a capture transition information determining module, configured to obtain second position information of a preset positioning point on the feature object and determine a second pose of the feature object before determining capture control information based on the first pose and the capture transition information; the grabbing equipment is controlled to grab the characteristic object, and the equipment pose when the grabbing equipment grabs the characteristic object is determined; and determining grabbing transition information according to the relation between the second pose of the characteristic object and the pose of the equipment when the grabbing equipment grabs the characteristic object.

In one possible implementation manner, the device control apparatus further includes a deviation information determining unit and a target placement control information determining unit. The pose determining module is further used for determining preset third position information of the target object under the condition that the grabbing equipment in the grabbing state moves to a preset pose; the grabbing state represents a state that grabbing equipment is in grabbing a target object; the deviation information determining unit is used for determining first deviation information according to third position information of a preset positioning point and preset standard position information corresponding to the preset positioning point; the deviation information determining unit is further used for obtaining the position information of the target placement bit and determining second deviation information according to the position information of the target placement bit and the position information of the preset placement bit; the target placement control information determining unit is used for determining target placement control information based on the first deviation information, the second deviation information and preset placement control information, wherein the preset placement control information is control information when the grabbing device in the grabbing state places the feature object in a preset placement position under the condition that the grabbing device in the grabbing state moves to a preset pose and the position information of a preset positioning point on the feature object is preset standard position information; and the control module is also used for controlling the grabbing equipment to place the target object to the target placement position according to the target placement control information.

In a possible implementation manner, the device control apparatus further includes a preset placement control information determining module, configured to, before determining the third position information of the preset positioning point of the target object in a case that the gripping device in the gripping state is operated to the preset pose, further include: determining the position information of the upper preset positioning point of the characteristic object captured by the capturing device with the preset gesture as preset standard position information; and controlling the grabbing equipment to place the characteristic object on a preset placement position, and determining preset placement control information.

In a third aspect, an embodiment of the present application provides an electronic device, including: a memory and a processor; the memory is coupled to the processor; the memory is for storing computer program code, the computer program code comprising computer instructions; wherein the computer instructions, when executed by the processor, cause the electronic device to perform the device control method as in the first aspect and any one of its possible designs.

In a fourth aspect, the present application provides a computer-readable storage medium comprising: computer software instructions; the computer software instructions, when run in the device control apparatus, cause the device control apparatus to implement the method of the first aspect described above.

In a fifth aspect, the present application provides a computer program product for, when run on a device control apparatus, causing the device control apparatus to perform the steps of the related method described in the first aspect above, to carry out the method of the first aspect above.

Advantageous effects of the second aspect to the fifth aspect described above may refer to corresponding descriptions of the first aspect, and are not repeated.

Drawings

FIG. 1 is a schematic diagram of a system for device control provided herein;

FIG. 2 is a schematic flow chart of a method for controlling a device provided in the present application;

fig. 3 is a second schematic flow chart of a device control method provided in the present application;

fig. 4 is a schematic flow chart III of a device control method provided in the present application;

FIG. 5 is a schematic view of projection vectors in a target coordinate system of an apparatus control method provided in the present application;

FIG. 6 is a schematic diagram of projection vectors and rotation vectors in a target coordinate system of an apparatus control method provided in the present application;

fig. 7 is a flow chart diagram of a device control method provided in the present application;

FIG. 8 is a schematic diagram of a device control apparatus according to the present application;

fig. 9 is a schematic diagram of a device second of the device control apparatus provided in the present application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.

It should be noted that, in the embodiments of the present application, words such as "exemplary" or "such as" are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "such as" is intended to present related concepts in a concrete fashion.

In order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, the terms "first", "second", and the like are used to distinguish the same item or similar items having substantially the same function and effect, and those skilled in the art will understand that the terms "first", "second", and the like are not limited in number and execution order.

For ease of understanding, related concepts related to the present application will first be briefly described.

Pose (pose)

The pose description and the coordinate transformation of the robot are the basis for the kinematic and dynamic analysis of the industrial robot. The pose represents a position and a posture, and any one rigid body can accurately and uniquely represent the position state by the position and the posture in a space coordinate system (OXYZ). Any one point in space can be specified explicitly with the six parameters described above, i.e., (x, y, z, rx, ry, rz), even though (x, y, z) are all the same, (rx, ry, rz) different represent robots in different poses to reach the same point.

Euler angle

The euler angle is a triplet of independent angle parameters for uniquely determining the fixed point rotational rigid body position, for example, the euler angle may be (rx, ry, rz) in a spatial coordinate system (ozz), where the rotation angle around the X axis is rx, the rotation angle around the Y axis is ry, and the rotation angle around the Z axis is rz. The Euler angles are transformed to obtain corresponding rotation matrices.

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

As known from the background art, with the development of industrial automation, more and more automatic machines are widely used. For example, gripping devices that grip articles play an important role in warehousing, logistics, and industrial assembly.

In the case of automatic gripping articles such as a robot arm, such as a gripping robot arm, which takes a workpiece, since the workpiece is generally large, the placement position is not easily controlled accurately, and thus gripping cannot be performed in the same manner each time. And because the shapes of different workpieces are different, accurate pose estimation is difficult to carry out on different workpieces, so that the conventional grabbing method is not suitable for the workpieces.

In view of the above, the present application provides a device control method, an apparatus, an electronic device, and a storage medium.

In some possible embodiments, an apparatus control method provided in the embodiments of the present application may be applied to an apparatus control system shown in fig. 1, where the apparatus control system includes at least a gripping apparatus 11 and a target object 12.

The grabbing device 11 may acquire first position information of a plurality of preset positioning points on the target object 12, and determine a first pose of the target object 12 according to the first position information of the plurality of preset positioning points; determining grabbing control information based on the first pose and grabbing transition information, wherein the grabbing transition information characterizes the relation between the equipment pose of the grabbing equipment 11 and the pose of the characteristic object when the grabbing equipment 11 grabs the characteristic object; the target object 12 is any one of the feature objects; the grabbing control information comprises a first target device pose when the grabbing device 11 grabs the target object 12; based on the gripping control information, the gripping apparatus 11 is controlled to grip the target object 12 in the first target apparatus pose.

In a possible implementation manner, the capturing device 11 may further obtain first position information of at least three preset positioning points on the target object 12, determine fitting center points of the at least three preset positioning points and position information of the fitting center points according to the first position information of the at least three preset positioning points on the target object 12, and use the position information of the fitting center points as the position information of the target object 12; determining the pose information of the target object 12 from the first position information of each preset positioning point, wherein the first pose of the target object 12 comprises the pose information and the position information of the target object 12.

In one possible implementation manner, the grabbing device 11 may further perform plane fitting on at least three preset positioning points according to the first position information of the at least three preset positioning points to obtain a fitted plane; an angle by which the first direction is rotated with respect to the first coordinate axis of the base coordinate system of the grasping apparatus 11 is taken as first posture information of the target object 12; the first direction is the normal direction of the fitting plane; determining a second direction according to projection information of each preset positioning point on the fitting plane, and taking the rotation angle of the second direction relative to a second coordinate axis of the base coordinate system as second posture information of the target object 12; the second direction is perpendicular to the first direction; an angle by which the third direction is rotated with respect to the third coordinate axis of the base coordinate system is taken as third posture information of the target object 12; the third direction, the second direction and the first direction are mutually perpendicular; the posture information of the target object 12 includes first posture information, second posture information, and third posture information.

In a possible implementation manner, the capturing device 11 may further determine a projection vector corresponding to each preset positioning point according to the projection information of each preset positioning point on the fitting plane, where the projection vector corresponding to one preset positioning point points to the projection point of one preset positioning point on the fitting plane from the projection point of the fitting center point on the fitting plane; and taking the direction corresponding to the target vector in the projection vector as an initial second direction, and correcting the initial second direction according to the directions corresponding to the rest projection vectors to obtain a second direction.

In one possible implementation manner, the capturing device 11 may further set up a two-dimensional target coordinate system in the fitting plane with a projection point of the fitting center point on the fitting plane as an origin; rotating the projection vector outside the target quadrant to the target quadrant according to a preset rule to obtain a corresponding rotation vector; the target quadrant is the quadrant to which the target vector belongs in the target coordinate system; the preset rule comprises a rotation direction and a rotation angle, wherein the rotation direction is anticlockwise or clockwise, and the rotation angle is determined according to the quadrant of the projection vector in the target coordinate system; adjusting the included angle between the initial second direction and the target coordinate axis by the included angle between the vector in the target quadrant and the target coordinate axis to obtain a second direction; the vectors in the target quadrant comprise projection vectors corresponding to at least one projection point and/or rotation vectors corresponding to at least one projection vector, and the target coordinate axis is any coordinate axis forming the target quadrant.

In a possible implementation manner, the capturing device 11 may further acquire second position information of a preset positioning point on the feature object and determine the second pose of the feature object before determining the capturing control information based on the first pose and the capturing transition information; the grabbing device 11 is controlled to grab the characteristic object, and the device pose when the grabbing device 11 grabs the characteristic object is determined; the gripping transition information is determined from the relationship between the second pose of the feature object and the pose of the device when the gripping device 11 grips the feature object.

In one possible implementation manner, the capturing device 11 may further determine third position information of a preset positioning point of the target object 12 in a case where the capturing device 11 in the capturing state is operated to a preset pose; the gripping state characterizes the state in which the gripping apparatus 11 is gripping the target object 12; determining first deviation information according to third position information of a preset positioning point and preset standard position information corresponding to the preset positioning point; acquiring position information of a target placement bit, and determining second deviation information according to the position information of the target placement bit and the position information of a preset placement bit; determining target placement control information based on the first deviation information, the second deviation information and preset placement control information, wherein the preset placement control information is control information when the gripping device 11 in the gripping state is placed on a preset placement position under the condition that the gripping device 11 in the gripping state operates to a preset pose and the position information of a preset positioning point on the characteristic object is preset standard position information; the gripping device 11 is controlled to place the target object 12 to the target placement position based on the target placement control information.

In one possible implementation manner, the capturing device 11 may further determine, as the preset standard position information, the position information of the upper preset positioning point of the feature object captured by the capturing device 11 in the preset posture before determining the third position information of the preset positioning point of the target object 12 in the case that the capturing device 11 in the capturing state is operated to the preset pose; the control grasping apparatus 11 places the feature object at a preset placement position, and determines preset placement control information.

The gripping device 11 in the embodiment of the present application may be a gripping device such as a robot arm gripper, a suction cup gripping device, or the like, which is not limited in the embodiment of the present application. It should be understood that the robotic arm gripper is shown in fig. 1 by way of example only and is not intended to be limiting in practice.

The target object 12 in the embodiment of the present application is an apparatus to be gripped by the gripping apparatus 11, such as a workpiece, a warehouse cargo, or the like, which can be gripped by the gripping apparatus, and the present application does not limit this.

In some possible embodiments, as shown in fig. 2, the device control method provided in the embodiments of the present application may include S101 to S103. The device control method provided in the embodiment of the present application may be applied to the gripping device 11 in fig. 1, for example.

S101, acquiring first position information of a plurality of preset positioning points on a target object, and determining a first pose of the target object according to the first position information of the plurality of preset positioning points.

S102, based on the first pose and the grabbing transition information, grabbing control information is determined.

The grabbing transition information characterizes the relation between the equipment pose of the grabbing equipment and the pose of the characteristic object when the grabbing equipment grabs the characteristic object. The grabbing control information comprises the pose of the first target device when the grabbing device grabs the target object. The target object is any one of the feature objects.

It will be appreciated that the feature object should be some object of similar or identical shape and feature, and that the gripping device may grip the feature object in the same way. For an object having the same shape characteristics, for example, for a workpiece object whose characteristic object is a door, the gripping device may grip the first door and the second door in the same manner, although the colors of the first door and the second door are different. For another example, the radians of the four corner edges of the third door and the fourth door may be different for objects having similar shape characteristics, but the third door and the fourth door have similar shape characteristics (e.g., the doors are similar in size and the overall frame shape is similar), and the radians of the four corner edges of the third door and the fourth door do not affect the gripping manner of the gripping device for the gripping device, so the gripping device can grip the third door and the fourth door in the same manner. It can be seen that the third door and the fourth door described above are also similar feature objects that can be grasped by the grasping apparatus in the same manner.

In addition, the target object is taken as any one of the characteristic objects, the target object is provided with a preset positioning point, the characteristic object is also provided with a preset positioning point, and the relative positions of the preset positioning point and the characteristic object (including the target object) are relatively determined. And because the number of the preset positioning points is multiple, the position of the characteristic object and the gesture of the characteristic object can be determined through the preset positioning points. That is, if the position information of the preset positioning point on the feature object is known, the current pose of the feature object can be relatively determined. Similarly, the method can determine the first pose of the target object through the first position information by acquiring the first position information of a plurality of preset positioning points on the target object.

Of course, the first position information of the plurality of preset positioning points on the target object may be obtained through shooting identification of the camera, or may be obtained through other positioning modes, which is not particularly limited in the method.

Further, in the method, the grabbing control information is determined according to the first pose of the target object and the grabbing transition information. It will be appreciated that if the gripping device grips the feature object in the same gripping manner, when the gripping device grips the target object in the feature object, the pose information of the gripping device and the pose information of the target object should have a constant relative relationship or difference relationship (such as a difference in position and a relative relationship and/or a difference and a relative relationship in pose), and because the pose of the target object is similar to that before being gripped by the gripping device, the pose information of the target object when not being gripped should have a constant relative relationship or difference. Based on the above, the method determines the relation between the equipment pose of the grabbing equipment and the pose of the characteristic object as grabbing transition information when the grabbing equipment grabs the characteristic object, and determines grabbing control information by using the unchanged relation and the pose of the target object. The capture control information should include a first target device pose that the capture device can capture the target object. Also, since the feature objects can be grabbed in the same manner, the feature objects (including the target object) can use the same grabbing transition information.

S103, based on the grabbing control information, controlling the grabbing device to grab the target object in the pose of the first target device.

As can be seen from the foregoing embodiments, in the device control method of the present application, position information of a preset positioning point on a target object is first obtained, so as to determine a first pose of the target object according to the first position information. Since the preset positioning point is a point of the preset position on the characteristic object, and the target object is used as one of the characteristic objects, the relative relation between the gesture of the target object and the preset positioning point can be known, and the relative relation between the gesture of any other characteristic object and the preset positioning point is the same. Therefore, the first pose of the current target object can be determined by presetting the first position information of the positioning point. When the gripping device grips the characteristic object in the same way, the pose of the gripping device gripping the characteristic object has a constant relative relation with the pose of the characteristic object, so that gripping transition information representing the constant relative relation between the pose of the gripping device gripping the characteristic object and the pose of the characteristic object and the first pose of the target object can be used to determine gripping control information when the gripping device grips the target object in the same way. Thereby controlling the gripping device to grip the target object according to the gripping control information. Therefore, the method can grasp the target object with different poses according to the real poses of the target object because the grasping control information of the method is determined according to the first poses of the target object which are actually obtained. In addition, the pose of the target object is determined according to the first position information of the preset positioning point of the target object, and the method is independent of whether the color and texture of the target object are rich and whether the geometric details are prominent, so that the method can be widely applied to various objects.

In some possible embodiments, as shown in fig. 3, first position information of a plurality of preset positioning points on the target object is acquired, and a first pose of the target object is determined according to the first position information of the plurality of preset positioning points, including S1011 to S1012 described below.

S1011, acquiring first position information of a plurality of preset positioning points on the target object, determining fitting center points of the plurality of preset positioning points and position information of the fitting center points according to the first position information of the plurality of preset positioning points on the target object, and taking the position information of the fitting center points as the position information of the target object.

It should be understood that since the preset positioning point is a point with a fixed relative position on the target object, the fitting center point of the preset positioning point is also a point with a fixed relative position to the target object. And, because the fitting center point is obtained by fitting the first position information of a plurality of preset positioning points, the position information determined by the fitting center point has smaller error than the position information determined by a single preset positioning point.

For example, the coordinates of a plurality of preset positioning points may be calculated by a summation average, and the coordinates obtained by the summation average are used as the position information of the fitting center point.

S1012, determining the gesture information of the target object based on the first position information of each preset positioning point.

Wherein the first pose of the target object includes pose information and position information of the target object.

Because the preset positioning point is a point with a fixed relative position on the target object, the gesture information of the target object is determined based on the relative relation between the actual gesture of the target object and the position information of the preset positioning point.

It should be appreciated that when describing the pose in terms of six parameters such as (x, y, z, rx, ry, rz), its positional information may be characterized as (x, y, z) and its pose information may be characterized as (rx, ry, rz).

For example, for a target object having four preset positioning points and where the four preset positioning points are co-located on a plane, if the relative relationship between the four preset positioning points has identifiable characteristics, the first preset positioning point, the second preset positioning point, the third preset positioning point and the fourth preset positioning point in the four preset positioning points can be distinguished according to the relative relationship between the preset positioning points, and then the gesture information of the target object can be determined according to the pointing directions of the two points. For example, the direction a of the first preset positioning point pointing to the second preset positioning point is perpendicular to the direction B of the first preset positioning point pointing to the third preset positioning point, the rotation angle of the direction a relative to the x axis of the base coordinate system of the gripping device can be determined as rx in the gesture information, the rotation angle of the direction B relative to the y axis of the base coordinate system of the gripping device can be determined as ry in the gesture information, and then rz is determined according to the fact that rz is perpendicular to rx and ry.

As can be seen from the foregoing embodiments, the device control method provided in the embodiments of the present application determines, based on the first position information of the preset positioning point, the fitting center point as the position information of the capturing device and the pose information of the target object, thereby determining the pose of the target object. Because the preset positioning point is a point with fixed relative position on the target object, the obtained position information and gesture information have a preset relative relation with the target object, and therefore, the method can determine the actual pose of the target object when the position information of the preset positioning point of the target object is the first position information.

In another possible embodiment, obtaining first position information of a plurality of preset positioning points on the target object, and determining a first pose of the target object according to the first position information of the plurality of preset positioning points includes: first position information of a plurality of preset positioning points on a target object is obtained. And determining the first pose of the target object based on the corresponding relation between the first position information and the positioning point on the three-dimensional model of the target object.

It should be understood that the three-dimensional model of the target object is a model constructed according to the target object, and the positioning points on the target object have corresponding relations with the positioning points on the three-dimensional model of the target object.

Pose estimation based on correspondence relates to a method for finding correspondence between observed input data and an object model. The pose information can be calculated through a Perrective-n-Point (PnP) algorithm according to the corresponding relation between the pixel position of the locating Point on the three-dimensional model of the target object and the position of the preset locating Point in the image of the target object.

Of course, the first pose of the target object may also be determined by other methods based on the first position information of a plurality of preset positioning points on the target object. For example, the three-dimensional point cloud of the target object can be constructed according to the first position information of the plurality of preset positioning points, and the pose of the object can be further predicted based on the least square method. The present application is not particularly limited thereto.

In some possible embodiments, as shown in fig. 4, pose information of the target object is determined based on the first position information of each preset anchor point, including S201 to S204 described below.

S201, performing plane fitting on at least three preset positioning points according to first position information of the at least three preset positioning points to obtain a fitting plane.

The gesture information of the target object comprises first gesture information, second gesture information and third gesture information.

It should be appreciated that since at least three points can determine one plane, the preset positioning points are at least three. And because more than three preset positioning points may not be on the same plane, the method determines the fitting plane by a plane fitting mode. For example, plane fitting of a plurality of points may be performed by a method such as a least square method or a lagrange multiplier method, which is not particularly limited in this application.

S202, an angle by which the first direction is rotated with respect to the first coordinate axis of the base coordinate system of the grasping apparatus is taken as first posture information of the target object.

The first direction is the normal direction of the fitting plane.

It should be appreciated that, although the normal direction of a plane may have two opposite directions, the normal direction of the first posture information of the target object at this time is only required to be guaranteed in the method, and the direction of the first posture information is determined by other characteristic objects. Namely, the characteristic objects all determine the attitude information of the characteristic objects by the same method.

S203, determining a second direction according to projection information of each preset positioning point on the fitting plane, and taking the rotation angle of the second direction relative to a second coordinate axis of the base coordinate system as second posture information of the target object.

Wherein the second direction is perpendicular to the first direction.

In one possible implementation manner, determining the second direction according to the projection information of each preset positioning point on the fitting plane includes: and determining the projection of the fitting center point on the fitting plane to point to the direction of a target preset locating point in the preset locating points as a second direction.

And S204, taking the rotation angle of the third direction relative to the third coordinate axis of the base coordinate system as third posture information of the target object.

Wherein the third direction, the second direction and the first direction are perpendicular to each other.

It should be understood that when describing the pose of the rigid body, the pose information includes first pose information, second pose information and third pose, and the three pose information should point to three angles perpendicular to each other, so that the third pose information can be obtained after the first pose information and the second pose information are known.

In some possible embodiments, determining the second direction according to projection information of each preset positioning point on the fitting plane includes: and determining projection vectors corresponding to all preset positioning points according to the projection information of all preset positioning points on the fitting plane, wherein the projection vector corresponding to one preset positioning point points to the projection point of one preset positioning point on the fitting plane from the projection point of the fitting center point on the fitting plane. And taking the direction corresponding to the target vector in the projection vector as an initial second direction, and correcting the initial second direction according to the directions corresponding to the rest projection vectors to obtain a second direction.

It should be understood that, since the preset positioning point is a point with a fixed relative position on the target object, the obtained fitting plane and the projection vector corresponding to the preset positioning point on the fitting plane have a predetermined relative position relationship with the target object. One of the target vectors may be selected as the initial second direction. And because the positioning of the preset positioning point possibly generates errors, the target vector is influenced to generate errors, the initial second direction of the target vector can be corrected according to a plurality of vectors in the projection vector, and the second direction with smaller errors is obtained.

In some possible embodiments, correcting the initial second direction according to the directions corresponding to the rest of the projection vectors to obtain a second direction includes:

step one: and taking a projection point of the fitting center point on the fitting plane as an origin, and establishing a two-dimensional target coordinate system in the fitting plane.

It should be understood that, because the projection vectors corresponding to the preset positioning points are all the projection points of the fitting center point on the fitting plane and point to the projection point of one preset positioning point on the fitting plane, the projection point of the fitting center point on the fitting plane is taken as the origin, namely the departure point of each projection vector on the plane. Because the fitting center point is the center of the preset positioning point, and the fitting plane is obtained by fitting the preset positioning point, the preset positioning points are generally distributed in each quadrant of the target coordinate system in the two-dimensional target coordinate system which is built by taking the projection point of the fitting center point on the fitting plane as the origin in the fitting plane.

For example, the projection of the x-axis direction of the base coordinate system of the grabbing device can be determined as the direction of the x-axis in the two-dimensional target coordinate system, and the projection point of the fitting center point on the fitting plane is determined as the origin of the target coordinate system, so that the direction of the y-axis in the target coordinate system can be determined perpendicular to the x-axis at the origin according to the y-axis, and the two-dimensional target coordinate system on the fitting plane is established.

Step two: and rotating the projection vector positioned outside the target quadrant to the target quadrant according to a preset rule to obtain a corresponding rotation vector.

The target quadrant is the quadrant to which the target vector belongs in the target coordinate system. The preset rule comprises a rotation direction and a rotation angle, wherein the rotation direction is anticlockwise or clockwise, and the rotation angle is determined according to the quadrant of the projection vector in the target coordinate system.

As shown in fig. 5, the target object has four preset positioning points, and projection vectors corresponding to the four preset positioning points are distributed in four directions of the target coordinate system. If the projection vector corresponding to the first quadrant is the target vector, the projection vector corresponding to the second quadrant is the second vector, the projection vector corresponding to the third quadrant is the third vector, and the projection vector corresponding to the fourth quadrant is the fourth vector. As shown in fig. 6, taking the rotation direction in the preset rule as an example of clockwise rotation, the second vector needs to be rotated 90 ° clockwise, the third vector needs to be rotated 180 ° clockwise, the fourth vector needs to be rotated 270 ° clockwise, and finally three rotation vectors located in the first quadrant are obtained, the second rotation vector corresponds to the second vector, the third rotation vector corresponds to the third vector, and the fourth rotation vector corresponds to the fourth vector.

In addition, the rotation angle of each vector in the preset rule may also be related to the number of preset points.

Of course, the preset rules may have other ways to achieve similar effects, which are not specifically limited in this application.

Step three: and adjusting the included angle between the initial second direction and the target coordinate axis by using the included angle between the vector in the target quadrant and the target coordinate axis to obtain the second direction.

The vectors in the target quadrant comprise projection vectors corresponding to at least one projection point and/or rotation vectors corresponding to at least one projection vector, and the target coordinate axis is any coordinate axis forming the target quadrant.

For example, in the above embodiment, the angles between the four vectors (three rotation vectors and one target vector) in the target quadrant and the x-axis as the target coordinate axis are used, if the angle between the rotation vector and the x-axis is larger than the angle between the target vector coordinate and the x-axis, the angle between the initial second direction and the target coordinate axis is increased, and if the angle between the rotation vector and the x-axis is smaller than the angle between the target vector coordinate and the x-axis, the angle between the initial second direction and the target coordinate axis is decreased.

For another example, the deviation between the initial second direction and the second direction may be determined from an average value of angles between four vectors (three rotation vectors and one target vector) in the target quadrant and the x-axis as the target coordinate axis.

Of course, the second direction may be determined in other similar manners, as long as the feature objects all determine the posture information thereof by the same method, which is not particularly limited in this application.

In some possible embodiments, before determining the grip control information based on the first pose and the grip transition information, the method further comprises: and acquiring second position information of a preset positioning point on the characteristic object, and determining a second pose of the characteristic object. And controlling the grabbing equipment to grab the characteristic object, and determining the equipment pose when the grabbing equipment grabs the characteristic object. And determining grabbing transition information according to the relation between the second pose of the characteristic object and the pose of the equipment when the grabbing equipment grabs the characteristic object.

The grabbing transition information characterizes the relation between the equipment pose of the grabbing equipment and the pose of the characteristic object when the grabbing equipment grabs the characteristic object, so that the grabbing transition information can be determined through the relation between the actual equipment pose of the grabbing equipment and the pose of the characteristic object, namely the data obtained in actual operation.

In some possible embodiments, the grab transition information is a grab transition matrix。

Wherein,for a pose matrix representing the pose of a feature object to be grasped by a grasping device, a +. >To characterize a pose matrix of a device pose of a gripping device when the gripping device grips a feature object.

It should be understood that when the gripping device grips the feature object, the device pose of the gripping device and the pose of the feature object gripped by the device to be gripped may convert the spatial coordinates (x, y, z, rx, ry, rz) in the base coordinate system of the gripping device into a three-dimensional matrix for representation, so as to perform the above calculation. In addition, since the pose of the feature object to be grabbed by the grabbing device may be acquired by means of positioning camera recognition and the like, the directly obtained used coordinate system for representing the pose information of the feature object may not be consistent with the coordinate system used by the pose of the grabbing device when the grabbing device grabs the feature object, for example, the pose of the feature object is under the coordinate system of the positioning camera, and the pose of the grabbing device is under the basic coordinate system of the grabbing device. Before the calculation of the grabbing transition matrix is performed, the method should be used forAnd->And converting the coordinate system into the same coordinate system and then calculating.

In some possible embodiments, before determining the third position information of the preset positioning point of the target object in a case that the gripping device in the gripping state is operated to the preset pose, the method further includes: and determining the position information of the upper preset positioning point of the characteristic object captured by the capturing device with the preset gesture as preset standard position information. And controlling the grabbing equipment to place the characteristic object on a preset placement position, and determining preset placement control information.

For example, the pose matrix may beThe preset pose of the grabbing device is arranged in a matrix +.>Grabbing equipment for representing preset gesturesWhen the characteristic object is grabbed, the position information of the locating point is preset on the characteristic object so as to enable the characteristic object to be +.>And representing preset standard position information. Placing the characteristic object on a preset placement position by the control grabbing device, and determining the pose of the characteristic object when the characteristic object is placed in the preset placement control information to be contained so as to be +.>And the pose of the characteristic object when the characteristic object is placed is represented. Then preset the placement control information +.>。

In some possible embodiments, as illustrated in fig. 7, the device control method further includes S301 to S305.

S301, determining third position information of a preset positioning point of the target object under the condition that the grabbing device in the grabbing state moves to a preset pose.

The grabbing state characterizes that the grabbing device is in a state of grabbing the target object.

It will be appreciated that due to the gripping design of the gripping device, and the placement of the target object, the target object may be subject to an offset when the gripping device grips the target object. Therefore, the third positional relationship of the preset positioning point of the target object can be retrieved.

S302, determining first deviation information according to third position information of a preset positioning point and preset standard position information corresponding to the preset positioning point.

S303, acquiring the position information of the target placement bit, and determining second deviation information according to the position information of the target placement bit and the position information of the preset placement bit.

Since the target placement position may be changed or the target placement position is on another object and the placement of the other object is shifted, there occurs a case where the position information of the target placement position is inconsistent with the position information of the preset placement position, the position information of the target placement position can be retrieved and the second deviation information between the two can be determined.

S304, determining target placement control information based on the first deviation information, the second deviation information and preset placement control information.

The preset placement control information is control information when the grabbing device in the grabbing state moves to a preset pose and the position information of a preset locating point on the characteristic object is preset standard position information.

S305, controlling the grabbing device to place the target object on the target placement position according to the target placement control information.

As can be seen from the foregoing embodiments, since the preset placement control information may characterize the control information when the gripping device in the gripping state moves to the preset pose and the position information of the preset positioning point on the feature object is the preset standard position information, the device control method provided in the embodiment of the present application may determine, based on the preset placement control information, the target placement control information for controlling the gripping device to place the target object on the target placement position after obtaining the first deviation information between the third position information of the preset positioning point and the preset standard position information corresponding to the preset positioning point and the second deviation information between the position information of the target placement position and the position information of the preset placement position. That is, new placement control information is determined according to deviation between preset standard position information corresponding to a preset positioning point and position information of a preset placement position in preset placement control information, and third position information of a current preset positioning point and position information of a target placement position. Therefore, the method can place the target object on the corresponding placement position according to the actual position information of the target object and the placement information of the target placement position.

The above embodiment is extended to position matrixThe third position information of the preset positioning point of the target object is represented by a matrix rtrThird position information of preset anchor point +.>Preset standard position information corresponding to preset positioning pointFirst deviation information +.>. In a position matrix->Position information characterizing the target placement bits in matrix +.>Representing the position information of the preset position bit, representing the position information of the target position bit by matrix rtp, second deviation information of the position information of the preset position bit +.>. In the target placement control information, a pose matrix representing the placement of the grabbing device is +.>。

In one possible embodiment, as shown in fig. 8, the present application provides an apparatus control device. The device control apparatus includes a pose determination module 401, a control information determination module 402, and a control module 403.

The pose determining module 401 is configured to obtain first position information of a plurality of preset positioning points on the target object, and determine a first pose of the target object according to the first position information of the plurality of preset positioning points.

A control information determining module 402, configured to determine capture control information based on the first pose and capture transition information, where the capture transition information characterizes a relationship between a pose of a device of the capture device and a pose of the feature object when the capture device captures the feature object; the target object is any one of the characteristic objects; the grabbing control information comprises the pose of the first target device when the grabbing device grabs the target object.

The control module 403 is configured to control the grabbing device to grab the target object with the pose of the first target device based on the grabbing control information.

In one possible embodiment, the pose determination module 401 includes a position information determination unit and a pose information determination unit. The position information determining unit is used for acquiring first position information of a plurality of preset positioning points on the target object, determining fitting center points of the plurality of preset positioning points and position information of the fitting center points according to the first position information of the plurality of preset positioning points on the target object, and taking the position information of the fitting center points as the position information of the target object; a gesture information determining unit for determining gesture information of the target object based on the first position information of each preset positioning point; the first pose of the target object includes pose information and position information of the target object.

In one possible embodiment, the pose information determination unit comprises a plane fitting subunit, a first pose information determination subunit, a second pose information determination subunit, and a third pose information determination subunit. The plane fitting subunit is used for carrying out plane fitting on at least three preset positioning points according to the first position information of the at least three preset positioning points to obtain a fitting plane; a first posture information determination subunit configured to use, as first posture information of the target object, an angle by which the first direction is rotated with respect to a first coordinate axis of the base coordinate system of the grasping apparatus; the first direction is the normal direction of the fitting plane; the second attitude information determining subunit is used for determining a second direction according to the projection information of each preset positioning point on the fitting plane, and taking the rotation angle of the second direction relative to a second coordinate axis of the base coordinate system as second attitude information of the target object; the second direction is perpendicular to the first direction; a third posture information determining subunit configured to use, as third posture information of the target object, an angle by which the third direction rotates with respect to the third coordinate axis of the base coordinate system; the third direction, the second direction and the first direction are mutually perpendicular; the pose information of the target object includes first pose information, second pose information, and third pose information.

In a possible embodiment, the second gesture information determining subunit is specifically configured to determine, according to projection information of each preset positioning point on the fitting plane, a projection vector corresponding to each preset positioning point, where the projection vector corresponding to one preset positioning point points from a projection point of the fitting center point on the fitting plane to a projection point of one preset positioning point on the fitting plane; and taking the direction corresponding to the target vector in the projection vector as an initial second direction, and correcting the initial second direction according to the directions corresponding to the rest projection vectors to obtain a second direction.

In a possible embodiment, the second pose information determining subunit is specifically configured to establish a two-dimensional target coordinate system in the fitting plane with a projection point of the fitting center point on the fitting plane as an origin; rotating the projection vector outside the target quadrant to the target quadrant according to a preset rule to obtain a corresponding rotation vector; the target quadrant is the quadrant to which the target vector belongs in the target coordinate system; the preset rule comprises a rotation direction and a rotation angle, wherein the rotation direction is anticlockwise or clockwise, and the rotation angle is determined according to the quadrant of the projection vector in the target coordinate system; adjusting the included angle between the initial second direction and the target coordinate axis by the included angle between the vector in the target quadrant and the target coordinate axis to obtain a second direction; the vectors in the target quadrant comprise projection vectors corresponding to at least one projection point and/or rotation vectors corresponding to at least one projection vector, and the target coordinate axis is any coordinate axis forming the target quadrant.

In one possible embodiment, the grabbing transition information is a grabbing transition matrixThe method comprises the steps of carrying out a first treatment on the surface of the Wherein,for a pose matrix representing the pose of a feature object to be grasped by a grasping device, a +.>To characterize the gripping of the characteristic object by the gripping deviceAnd grabbing a pose matrix of the equipment pose of the equipment.

In a possible embodiment, the device control apparatus further includes a capture transition information determining module, configured to obtain second position information of a preset positioning point on the feature object and determine a second pose of the feature object before determining capture control information based on the first pose and the capture transition information; the grabbing equipment is controlled to grab the characteristic object, and the equipment pose when the grabbing equipment grabs the characteristic object is determined; and determining grabbing transition information according to the relation between the second pose of the characteristic object and the pose of the equipment when the grabbing equipment grabs the characteristic object.

In one possible embodiment, the device control apparatus further includes a deviation information determination unit and a target placement control information determination unit. The pose determining module 401 is further configured to determine preset third position information of the target object when the gripping device in the gripping state is operated to a preset pose; the grabbing state represents a state that grabbing equipment is in grabbing a target object; the deviation information determining unit is used for determining first deviation information according to third position information of a preset positioning point and preset standard position information corresponding to the preset positioning point; the deviation information determining unit is further used for obtaining the position information of the target placement bit and determining second deviation information according to the position information of the target placement bit and the position information of the preset placement bit; the target placement control information determining unit is used for determining target placement control information based on the first deviation information, the second deviation information and preset placement control information, wherein the preset placement control information is control information when the grabbing device in the grabbing state places the feature object in a preset placement position under the condition that the grabbing device in the grabbing state moves to a preset pose and the position information of a preset positioning point on the feature object is preset standard position information; the control module 403 is further configured to control the gripping device to place the target object on the target placement location according to the target placement control information.

In a possible embodiment, the device control apparatus further includes a preset placement control information determining module 402, configured to, before determining the third position information of the preset positioning point of the target object in a case that the gripping device in the gripping state is operated to the preset pose, further include: determining the position information of the upper preset positioning point of the characteristic object captured by the capturing device with the preset gesture as preset standard position information; and controlling the grabbing equipment to place the characteristic object on a preset placement position, and determining preset placement control information.

In the case of implementing the functions of the integrated units in the form of hardware, the embodiment of the application provides a schematic diagram of hardware composition of a device control apparatus, as shown in fig. 9, where the device control apparatus includes: a processor 502, a communication interface 503, and a bus 504. In one possible implementation, the device control apparatus may further include a memory 501.

The processor 502, which may be a processor, may be implemented or executed with the various exemplary logic blocks, modules, and circuits described in connection with the present disclosure. The processor 502 may be a central processor, a general purpose processor, a digital signal processor, an application specific integrated circuit, a field programmable gate array or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. Which may implement or perform the various exemplary logic blocks, modules, and circuits described in connection with this disclosure. The processor 502 may also be a combination of computing functions, e.g., comprising one or more microprocessors, a combination of a DSP and a microprocessor, etc.

A communication interface 503 for connecting with other devices via a communication network. The communication network may be an ethernet, a radio access network, a wireless local area network (wireless local area networks, WLAN), etc.

Memory 501, which may be, but is not limited to, a read-only memory (ROM) or other type of static storage device that may store static information and instructions, a random access memory (random access memory, RAM) or other type of dynamic storage device that may store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), magnetic disk storage or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

As a possible implementation, the memory 501 may exist separately from the processor 502, and the memory 501 may be connected to the processor 502 through the bus 504 for storing instructions or program codes. The processor 502, when calling and executing instructions or program code stored in the memory 501, is capable of implementing the device control method provided in the embodiments of the present application.

In another possible implementation, the memory 501 may also be integrated with the processor 502.

Bus 504, which may be an extended industry standard architecture (extended industry standard architecture, EISA) bus or the like. The bus 504 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in fig. 9, but not only one bus or one type of bus.

From the foregoing description of the embodiments, it will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-described division of functional modules is illustrated, and in practical application, the above-described functional allocation may be performed by different functional modules according to needs, i.e. the internal structure of the device control apparatus is divided into different functional modules to perform all or part of the functions described above.

Embodiments of the present application also provide a computer-readable storage medium. All or part of the flow in the above method embodiments may be implemented by computer instructions to instruct related hardware, and the program may be stored in the above computer readable storage medium, and the program may include the flow in the above method embodiments when executed. The computer readable storage medium may be any of the foregoing embodiments or memory. The computer readable storage medium may be an external storage device of the device control apparatus, for example, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash card (flash card) or the like provided in the device control apparatus. Further, the computer-readable storage medium may include both the internal storage unit and the external storage device of the device control apparatus. The computer-readable storage medium is used for storing the computer program and other programs and data required by the device control apparatus. The above-described computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

The present application also provides a computer program product comprising a computer program which, when run on a computer, causes the computer to perform any one of the apparatus control methods provided in the above embodiments.

Although the present application has been described herein in connection with various embodiments, other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed application, from a review of the figures, the disclosure, and the appended claims. In the claims, the word "Comprising" does not exclude other elements or steps, and the "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Although the present application has been described in connection with specific features and embodiments thereof, it will be apparent that various modifications and combinations can be made without departing from the spirit and scope of the application. Accordingly, the specification and drawings are merely exemplary illustrations of the present application as defined in the appended claims and are considered to cover any and all modifications, variations, combinations, or equivalents that fall within the scope of the present application. It will be apparent to those skilled in the art that various modifications and variations can be made in the present application without departing from the spirit or scope of the application. Thus, if such modifications and variations of the present application fall within the scope of the claims and the equivalents thereof, the present application is intended to cover such modifications and variations.

The foregoing is merely a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present disclosure should be covered in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An apparatus control method, applied to a gripping apparatus, comprising:

acquiring first position information of a plurality of preset positioning points on a target object, determining fitting center points of a plurality of preset positioning points and position information of the fitting center points according to the first position information of the plurality of preset positioning points on the target object, and taking the position information of the fitting center points as the position information of the target object;

performing plane fitting on at least three preset positioning points according to the first position information of the at least three preset positioning points to obtain fitted planes;

an angle of rotation of a first direction relative to a first coordinate axis of a base coordinate system of the grabbing device is used as first posture information of the target object; the first direction is the normal direction of the fitting plane;

Determining a second direction according to projection information of each preset positioning point on the fitting plane, and taking the angle of rotation of the second direction relative to a second coordinate axis of the base coordinate system as second posture information of the target object; the second direction is perpendicular to the first direction;

taking the angle of rotation of a third direction relative to a third coordinate axis of the base coordinate system as third posture information of the target object; the third direction, the second direction and the first direction are perpendicular to each other;

determining grabbing control information based on a first pose of the target object and grabbing transition information, wherein the first pose of the target object comprises pose information of the target object and position information of the target object, the pose information of the target object comprises the first pose information, the second pose information and the third pose information, and the grabbing transition information represents a relation between equipment pose of the grabbing equipment and pose of the characteristic object when the grabbing equipment grabs the characteristic object; the target object is any one of characteristic objects; the grabbing control information comprises a first target equipment pose when the grabbing equipment grabs the target object;

And controlling the grabbing device to grab the target object according to the pose of the first target device based on the grabbing control information.

2. The apparatus control method according to claim 1, wherein the determining the second direction based on the projection information of each of the preset positioning points on the fitting plane includes:

determining projection vectors corresponding to all preset positioning points according to projection information of all preset positioning points on the fitting plane, wherein the projection vector corresponding to one preset positioning point points at the projection points of one preset positioning point on the fitting plane from the projection point of the fitting center point on the fitting plane;

and correcting the initial second direction by taking the direction corresponding to the target vector in the projection vector as the initial second direction according to the directions corresponding to the rest projection vectors to obtain the second direction.

3. The apparatus control method according to claim 2, wherein the correcting the initial second direction according to the directions corresponding to the remaining projection vectors to obtain the second direction includes:

establishing a two-dimensional target coordinate system in the fitting plane by taking a projection point of the fitting center point on the fitting plane as an origin;

Rotating the projection vector positioned outside the target quadrant to the target quadrant according to a preset rule to obtain a corresponding rotation vector; the target quadrant is a quadrant to which the target vector belongs in the target coordinate system; the preset rule comprises a rotation direction and a rotation angle, wherein the rotation direction is anticlockwise or clockwise, and the rotation angle is determined according to a quadrant of the projection vector in the target coordinate system;

adjusting the included angle between the initial second direction and the target coordinate axis by using the included angle between the vector in the target quadrant and the target coordinate axis to obtain the second direction; the vectors in the target quadrant comprise projection vectors corresponding to at least one projection point and/or rotation vectors corresponding to at least one projection vector, and the target coordinate axis is any coordinate axis forming the target quadrant.

4. The apparatus control method according to claim 1, wherein the grab transition information is a grab transition matrix；

Wherein,for the representation of the pose matrix of the pose of the feature object to be gripped by the gripping device +.>To characterize a pose matrix of a device pose of the gripping device when the gripping device grips a feature object.

5. The apparatus control method according to any one of claims 1 to 4, characterized in that before the determination of the grip control information based on the first pose of the target object and grip transition information, the method further comprises:

acquiring second position information of the preset positioning point on the characteristic object, and determining a second pose of the characteristic object;

controlling the grabbing device to grab the characteristic object, and determining the equipment pose when the grabbing device grabs the characteristic object;

and determining the grabbing transition information according to the relation between the second pose of the characteristic object and the equipment pose when the grabbing equipment grabs the characteristic object.

6. The apparatus control method according to any one of claims 1 to 4, characterized in that the method further comprises:

under the condition that the grabbing equipment in the grabbing state is operated to a preset pose, determining third position information of the preset locating point of the target object; the grabbing state characterizes that the grabbing device is in a state of grabbing the target object;

determining first deviation information according to the third position information of the preset positioning point and preset standard position information corresponding to the preset positioning point;

Acquiring position information of a target placement bit, and determining second deviation information according to the position information of the target placement bit and the position information of a preset placement bit;

determining target placement control information based on the first deviation information, the second deviation information and preset placement control information, wherein the preset placement control information is control information when the grabbing device in a grabbing state moves to the preset pose and the position information of a preset locating point on the characteristic object is the preset standard position information;

and controlling the grabbing equipment to place the target object on the target placing position according to the target placing control information.

7. The apparatus control method according to claim 6, wherein before determining third position information of the preset positioning point of the target object in a case where the gripping apparatus in the gripping state is operated to a preset pose, the method further comprises:

determining the position information of the preset positioning point on the characteristic object captured by the capturing device of the preset pose as the preset standard position information;

And controlling the grabbing equipment to place the characteristic object on the preset placing position, and determining the preset placing control information.

8. A device control apparatus, characterized in that the apparatus comprises:

the pose determining module is used for acquiring first position information of a plurality of preset positioning points on a target object, determining fitting center points of the plurality of preset positioning points and position information of the fitting center points according to the first position information of the plurality of preset positioning points on the target object, and taking the position information of the fitting center points as the position information of the target object;

the pose determining module is further configured to perform plane fitting on at least three preset positioning points according to the first position information of the at least three preset positioning points, so as to obtain a fitted plane;

the pose determining module is further configured to use an angle of rotation of a first direction relative to a first coordinate axis of a base coordinate system of the capturing device as first pose information of the target object; the first direction is the normal direction of the fitting plane;

the pose determining module is further configured to determine a second direction according to projection information of each preset positioning point on the fitting plane, and use an angle of rotation of the second direction relative to a second coordinate axis of the base coordinate system as second pose information of the target object; the second direction is perpendicular to the first direction;

The pose determining module is further configured to use an angle of rotation of a third direction relative to a third coordinate axis of the base coordinate system as third pose information of the target object; the third direction, the second direction and the first direction are perpendicular to each other;

a control information determining module, configured to determine capture control information based on a first pose of the target object and capture transition information, where the first pose of the target object includes pose information of the target object and position information of the target object, and the pose information of the target object includes the first pose information, the second pose information, and the third pose information, and the capture transition information characterizes a relationship between a device pose of the capturing device and a pose of the feature object when the capturing device captures the feature object; the target object is any one of characteristic objects; the grabbing control information comprises a first target equipment pose when the grabbing equipment grabs the target object;

and the control module is used for controlling the grabbing device to grab the target object according to the pose of the first target device based on the grabbing control information.

9. An electronic device comprising a processor and a memory, the memory for storing computer instructions, the processor for invoking and executing the computer instructions from the memory, implementing the device control method of any of claims 1-7.

10. A computer-readable storage medium, the computer-readable storage medium comprising: computer software instructions; when executed in a device control apparatus, causes the device control apparatus to implement the device control method as claimed in any one of claims 1 to 7.