CN115582829A - Method and device for determining position of mechanical arm, electronic equipment and storage medium - Google Patents

Method and device for determining position of mechanical arm, electronic equipment and storage medium Download PDF

Info

Publication number
CN115582829A
CN115582829A CN202110757391.1A CN202110757391A CN115582829A CN 115582829 A CN115582829 A CN 115582829A CN 202110757391 A CN202110757391 A CN 202110757391A CN 115582829 A CN115582829 A CN 115582829A
Authority
CN
China
Prior art keywords
determining
mechanical arm
calibration
relation
image acquisition
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110757391.1A
Other languages
Chinese (zh)
Inventor
赵龙飞
迟万超
蒋俊南
王帅
杨思成
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN202110757391.1A priority Critical patent/CN115582829A/en
Publication of CN115582829A publication Critical patent/CN115582829A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00Programme-controlled manipulators
    • B25J9/16Programme controls
    • B25J9/1602Programme controls characterised by the control system, structure, architecture
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B25HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25JMANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00Programme-controlled manipulators
    • B25J9/16Programme controls
    • B25J9/1602Programme controls characterised by the control system, structure, architecture
    • B25J9/161Hardware, e.g. neural networks, fuzzy logic, interfaces, processor

Landscapes

  • Engineering & Computer Science (AREA)
  • Automation & Control Theory (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Fuzzy Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Manipulator (AREA)

Abstract

The invention provides a method for determining the position of a mechanical arm, which comprises the following steps: determining a first position relation between the mechanical arm and an intermediate coordinate system based on the position parameters of the mechanical arm; acquiring position parameters of the image acquisition device; determining a second positional relationship between the image acquisition device and an intermediate coordinate system based on the position parameters of the image acquisition device; performing position conversion processing on the first position relation and the second position relation to obtain a first position relation and a second position relation conversion relation parameter; and determining the position of the mechanical arm based on the first position relation and the second position relation conversion relation parameter. The invention also provides a position determining device of the mechanical arm, electronic equipment, a storage medium and a robot. The invention can accurately determine the position of the mechanical arm, reduce accumulated errors and simultaneously improve the accuracy of a single posture acquisition effect so as to provide more convenient use of the mechanical arm for a user.

Description

Method and device for determining position of mechanical arm, electronic equipment and storage medium
Technical Field
The present invention relates to a visual information processing technology, and in particular, to a method and an apparatus for determining a position of a robot arm, an electronic device, a storage medium, and a robot.
Background
Aiming at the operation of a mechanical arm based on a visual algorithm, the requirement of obtaining a coordinate system transformation relation between a visual sensing device and the mechanical arm is usually met, a target position obtained by the visual sensing device can be converted into a mechanical arm coordinate system through the transformation relation of the coordinate system, because the mechanical arm can reach infinite number of point positions in a motion space, the existing positioning precision detection device can only detect some special point positions or limited point positions, and when a large number of point positions need to be detected, the workload is very huge; in the conventional technology, when the hands and eyes are calibrated, the mechanical arm grasps the calibration plate, the mechanical arm is moved for many times, and the camera acquires the position of the calibration plate to finish calibration. The method relies on the accuracy of obtaining the position of the calibration plate by multiple cameras, accumulated errors exist, and meanwhile, the accuracy of obtaining the effect by a single posture is poor.
Disclosure of Invention
In view of this, embodiments of the present invention provide a method and an apparatus for determining a position of a robot arm, an electronic device, a storage medium, and a robot, which can accurately determine a position of the robot arm, reduce an accumulated error, and improve accuracy of a single-gesture obtaining effect, so that a user can use the robot arm more conveniently.
The technical scheme of the embodiment of the invention is realized as follows:
the embodiment of the invention provides a method for determining the position of a mechanical arm, which comprises the following steps:
acquiring position parameters of the mechanical arm;
determining a first position relation between the mechanical arm and an intermediate coordinate system based on the position parameters of the mechanical arm;
acquiring position parameters of the image acquisition device;
determining a second positional relationship between the image acquisition device and an intermediate coordinate system based on the positional parameters of the image acquisition device;
performing position conversion processing on the first position relation and the second position relation to obtain a first position relation and a second position relation conversion relation parameter;
and determining the position of the mechanical arm based on the first position relation and the second position relation conversion relation parameter.
An embodiment of the present invention further provides a device for determining a position of a robot arm, including:
the information transmission module is used for acquiring the position parameters of the mechanical arm;
the information processing module is used for determining a first position relation between the mechanical arm and an intermediate coordinate system based on the position parameters of the mechanical arm;
the information transmission module is used for acquiring the position parameters of the image acquisition device;
the information processing module is used for determining a second position relation between the image acquisition device and the middle coordinate system based on the position parameters of the image acquisition device;
the information processing module is configured to perform position transformation processing on the first position relationship and the second position relationship to obtain a first position relationship and a second position relationship conversion relationship parameter;
and the information processing module is used for determining the position of the mechanical arm based on the first position relation and the second position relation conversion relation parameter.
In the above-mentioned scheme, the first step of the method,
the information processing module is used for configuring a calibration tip at the tail end of the mechanical arm based on the position parameters of the mechanical arm;
the information processing module is used for configuring a first calibration point, a second calibration point, a third calibration point and a fourth calibration point in a calibration plate corresponding to the mechanical arm;
the information processing module is configured to determine a set of first calibration point coordinates of the first calibration point, the second calibration point, the third calibration point, and the fourth calibration point in a calibration plate coordinate system of the calibration plate;
the information processing module is used for adjusting the calibration tip to the corresponding positions of different calibration points of the calibration plate;
the information processing module is configured to determine the first calibration point, the second calibration point, the third calibration point, the fourth calibration point, and a set of coordinates of the second calibration point in a mechanical arm base coordinate system corresponding to the mechanical arm according to corresponding positions of different calibration points of the calibration plate;
the information processing module is configured to determine a transformation matrix corresponding to the first position relationship based on the set of first calibration point coordinates and the set of second calibration point coordinates;
and the information processing module is used for determining the first position relation between the mechanical arm and the middle coordinate system through the transformation matrix corresponding to the first position relation.
In the above-mentioned scheme, the first step of the method,
the information processing module is used for determining a first three-dimensional center point and a second three-dimensional center point of different calibration point matrixes based on the set of the first calibration point coordinates and the set of the second calibration point coordinates;
the information processing module is used for carrying out secondary centralization processing on the calibration points in the calibration plate to obtain a secondary centralization processing result;
the information processing module is used for carrying out singular value decomposition processing on the secondary centralization processing result to obtain a rotation matrix;
the information processing module is used for determining a transfer matrix according to the first three-dimensional central point, the second three-dimensional central point and the rotation matrix;
and the information processing module is used for carrying out matrix combination processing on the rotation matrix and the transfer matrix to obtain a transformation matrix corresponding to the first position relation.
In the above-mentioned scheme, the first step of the method,
the information processing module is used for determining the type of the calibration plate;
the information processing module is used for determining the side length parameter of a unit calibration square in the type of the calibration plate based on the type of the calibration plate;
and the information processing module is used for adjusting the set of the first calibration point coordinates according to the side length parameter of the unit calibration square block.
In the above-mentioned scheme, the first step of the method,
the information processing module is used for acquiring images in the use environment of the mechanical equipment through the image acquisition device;
the information processing module is used for determining the side length parameter of a unit calibration square block in the type of the calibration plate;
the information processing module is used for determining the spatial posture of the calibration plate based on the image in the use environment of the mechanical equipment and the pixel difference value of the side length parameter of the unit calibration square;
the information processing module is used for determining the position parameters of the image acquisition device according to the space posture of the calibration plate;
and the information processing module is used for determining an image acquisition matrix according to the space posture of the calibration plate and the position parameters of the image acquisition device, and determining a second position relation between the image acquisition device and the middle coordinate system according to the image acquisition matrix.
In the above-mentioned scheme, the first step of the method,
the information processing module is used for acquiring the target position of the mechanical arm;
the information processing module is used for comparing the position of the mechanical arm with the target position when the position of the mechanical arm is determined based on the first position relation and the second position relation conversion relation parameter;
the information processing module is used for adjusting the position of the mechanical arm when the coordinate difference value between the position of the mechanical arm and the target position exceeds a threshold value.
An embodiment of the present invention further provides an electronic device, including:
a memory for storing executable instructions;
and the processor is used for realizing the preorder position determining method of the mechanical arm or realizing the preorder position determining method of the mechanical arm when the processor runs the executable instructions stored in the memory.
An embodiment of the present invention further provides a computer-readable storage medium storing executable instructions, where the executable instructions, when executed by a processor, implement a method for determining a position of a preceding robot arm, or implement a method for determining a position of a preceding robot arm.
An embodiment of the present invention further provides a robot, including:
the mechanical equipment comprises a mechanical arm and an image acquisition device, is used for receiving instructions of a processor and realizes the preorder position determination method of the mechanical arm
A memory for storing executable instructions;
and the processor is used for realizing the preorder position determining method of the mechanical arm or realizing the preorder position determining method of the mechanical arm when the executable instructions stored in the memory are run.
The embodiment of the invention has the following beneficial effects:
according to the invention, the position parameters of the mechanical arm are obtained; determining a first position relation between the mechanical arm and an intermediate coordinate system based on the position parameters of the mechanical arm; acquiring position parameters of the image acquisition device; determining a second positional relationship between the image acquisition device and an intermediate coordinate system based on the positional parameters of the image acquisition device; performing position conversion processing on the first position relation and the second position relation to obtain a first position relation and a second position relation conversion relation parameter; the position of the mechanical arm is determined based on the first position relation and the second position relation conversion relation parameter, so that the position of the mechanical arm can be accurately determined, accumulated errors are reduced, and meanwhile, the accuracy of a single posture obtaining effect is improved, so that a user can use the mechanical arm more conveniently.
Drawings
Fig. 1 is a schematic usage environment diagram of a position determination method for a robot arm according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of an electronic device according to an embodiment of the present invention;
fig. 3 is a schematic flow chart of an alternative method for determining the position of a robot arm according to an embodiment of the present invention;
FIG. 4 is a schematic flow chart illustrating an alternative method for determining the position of a robotic arm according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a calibration plate processing procedure according to an embodiment of the present invention;
fig. 6 is a schematic flow chart of an alternative method for determining the position of the robot arm according to the embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail with reference to the accompanying drawings, the described embodiments should not be construed as limiting the present invention, and all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
In the following description, reference is made to "some embodiments" which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict.
Before further detailed description of the embodiments of the present invention, terms and expressions referred to in the embodiments of the present invention are described, and the terms and expressions referred to in the embodiments of the present invention are applicable to the following explanations.
1) In response to the condition or state indicating that the executed operation depends on, one or more of the executed operations may be in real-time or may have a set delay when the dependent condition or state is satisfied; there is no restriction on the order of execution of the operations performed unless otherwise specified.
2) The Calibration plate (Calibration Target) is used for correcting lens distortion in machine vision, image measurement, photogrammetry, three-dimensional reconstruction and other applications; determining a conversion relation between the physical size and the pixel; and determining the mutual relation between the three-dimensional geometric position of a certain point on the surface of the space object and the corresponding point in the image, wherein a geometric model imaged by a camera needs to be established.
The camera shoots the array flat plate with the fixed-spacing pattern, and a geometric model of the camera can be obtained through calculation of a calibration algorithm, so that high-precision measurement and reconstruction results are obtained. And a flat plate with an array of fixed pitch patterns is a calibration plate.
3) The client, the carrier in the terminal implementing the specific function, e.g. the mobile client (APP), is the carrier of the specific function in the mobile terminal, e.g. the program performing the function of user gesture recognition.
4) Singular Value Decomposition (SVD) is an important matrix Decomposition in linear algebra, and is implemented to be insensitive to disturbance of the matrix.
In the method for determining the position of the mechanical arm, the implementation environment shown in fig. 1 is combined, and the defects in the conventional technology are briefly introduced, wherein in the conventional technology, a calibration plate is fixed at the tail end of the mechanical arm, a camera is used for shooting the calibration plate for multiple times to obtain the position from the camera to the calibration plate, meanwhile, an internal interface of the mechanical arm is used for obtaining the position conversion relationship from a base of the mechanical arm to the tail end of the mechanical arm, and finally, the position conversion relationship from the camera to the mechanical arm is obtained through calculation. However, the method for acquiring the position of the calibration plate by using the camera has errors caused by inaccurate internal reference calibration of the camera, external illumination change, the calibration plate not being in the center of the image, and the like, thereby causing the calibration effect of the whole system to be poor.
Fig. 1 is a schematic usage scenario of a method for determining a position of a robot arm provided in an embodiment of the present invention, where a mechanical device in a production line may be assisted by the method for determining a position of a robot arm provided in the present application to perform different tasks through the robot arm, and referring to fig. 1, a terminal (including a terminal 10-1 and a terminal 10-2) is provided with corresponding clients capable of performing different functions, where the clients are terminals (including the terminal 10-1 and the terminal 10-2) that acquire different target positions from corresponding servers 200 through a network 300 and that control the mechanical device 400 to perform different tasks, the terminals are connected to the servers 200 through the network 300, the network 300 may be a wide area network or a local area network, or a combination of the two, and data transmission is implemented using a wireless link, where types of the corresponding tasks acquired by the terminals (including the terminal 10-1 and the terminal 10-2) from the corresponding servers 200 through the network 300 may be the same or different, for example: the terminals (including the terminal 10-1 and the terminal 10-2) can acquire the cargo carrying task from the corresponding server 200 through the network 300, control the robot to complete the preset standard grabbing and placing action group, and realize the functions of grabbing and placing objects by the robot. The camera is installed at the part outside the robot arm, is relatively fixed with the base (world coordinate system) of the robot, and does not move along with the movement of the robot, so that the position determining method of the robot arm provided by the application can be implemented according to the following steps: acquiring position parameters of the mechanical arm; determining a first position relation between the mechanical arm and an intermediate coordinate system based on the position parameters of the mechanical arm; acquiring position parameters of the image acquisition device; determining a second positional relationship between the image acquisition device and an intermediate coordinate system based on the position parameters of the image acquisition device; performing position conversion processing on the first position relation and the second position relation to obtain a first position relation and a second position relation conversion relation parameter; and determining the position of the mechanical arm based on the first position relation and the second position relation conversion relation parameter.
The embodiment of the present invention may be implemented by combining a Cloud technology, where the Cloud technology (Cloud technology) is a hosting technology for unifying series resources such as hardware, software, and a network in a wide area network or a local area network to implement calculation, storage, processing, and sharing of data, and may also be understood as a generic term of a network technology, an information technology, an integration technology, a management platform technology, an application technology, and the like applied based on a Cloud computing business model. Background services of the technical network system require a large amount of computing and storage resources, such as video websites, photo-like websites and more portal websites, so cloud technology needs to be supported by cloud computing.
It should be noted that cloud computing is a computing mode, and distributes computing tasks on a resource pool formed by a large number of computers, so that various application systems can obtain computing power, storage space and information services as required. The network that provides the resources is referred to as the "cloud". Resources in the "cloud" appear to the user as being infinitely expandable and available at any time, available on demand, expandable at any time, and paid for on-demand. As a basic capability provider of cloud computing, a cloud computing resource pool platform, which is called an Infrastructure as a Service (IaaS) for short, is established, and multiple types of virtual resources are deployed in a resource pool and are used by external clients selectively. The cloud computing resource pool mainly comprises: a computing device (which may be a virtualized machine, including an operating system), a storage device, and a network device.
With reference to the embodiment shown in fig. 1, the target object determining method provided in the embodiment of the present invention may be implemented by corresponding cloud devices, for example: the terminals (including the terminal 10-1 and the terminal 10-2) are connected to the server 200 located at the cloud end through a network 300, and the network 300 may be a wide area network or a local area network, or a combination of the two. It should be noted that the server 200 may be a physical device or a virtualization device.
The cloud equipment can also store the position parameters of the image acquisition device, the first position relation and the second position relation conversion relation parameters and the position of the mechanical arm, so that the position parameters stored by the cloud equipment can be conveniently and timely called in different application environments, and the adjustment time of the mechanical arm is saved.
Specifically, as shown in fig. 1 in the preamble embodiment, the server 200 may be an independent physical server, may also be a server cluster or a distributed system formed by a plurality of physical servers, and may also be a cloud server that provides basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a network service, cloud communication, a middleware service, a domain name service, a security service, a CDN, and a big data and artificial intelligence platform. The terminal may be, but is not limited to, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and the like. The terminal and the server may be directly or indirectly connected through wired or wireless communication, and the application is not limited herein.
As will be described in detail below, the electronic device according to the embodiment of the present invention may be implemented in various forms, such as a dedicated robot with a position adjustment function, or a robot arm with a position adjustment function, for example, the server 200 in fig. 1. Fig. 2 is a schematic diagram of a composition structure of an electronic device according to an embodiment of the present invention, and it is understood that fig. 2 only shows an exemplary structure of the electronic device, and not a whole structure, and a part of the structure or the whole structure shown in fig. 2 may be implemented as needed.
The electronic equipment provided by the embodiment of the invention comprises: at least one processor 201, memory 202, user interface 203, and at least one network interface 204. The various components in the electronic device 20 are coupled together by a bus system 205. It will be appreciated that the bus system 205 is used to enable communications among the components. The bus system 205 includes a power bus, a control bus, and a status signal bus in addition to a data bus. For clarity of illustration, however, the various buses are labeled as bus system 205 in fig. 2.
The user interface 203 may include, among other things, a display, a keyboard, a mouse, a trackball, a click wheel, a key, a button, a touch pad, or a touch screen.
It will be appreciated that the memory 202 can be either volatile memory or nonvolatile memory, and can include both volatile and nonvolatile memory. The memory 202 in the embodiments of the present invention is capable of storing data to support the operation of the terminal (e.g., 10-1). Examples of such data include: any computer program, such as an operating system and application programs, for operating on a terminal (e.g., 10-1). The operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, and the like, and is used for implementing various basic services and processing hardware-based tasks. The application program may include various application programs.
In some embodiments, the position determining apparatus of the robot arm provided by the embodiments of the present invention may be implemented by a combination of hardware and software, and by way of example, the position determining apparatus of the robot arm provided by the embodiments of the present invention may be a processor in the form of a hardware decoding processor, which is programmed to execute the position determining method of the robot arm provided by the embodiments of the present invention. For example, a processor in the form of a hardware decode processor may employ one or more Application Specific Integrated Circuits (ASICs), DS ps, programmable Logic Devices (PLDs), complex Programmable Logic Devices (CPLDs), field Programmable Gate arrays (FPGs), or other electronic components.
As an example of the implementation of the position determining apparatus of the robot arm provided by the embodiment of the present invention by using a combination of hardware and software, the position determining apparatus of the robot arm provided by the embodiment of the present invention may be directly embodied as a combination of software modules executed by the processor 201, the software modules may be located in a storage medium, the storage medium is located in the memory 202, the processor 201 reads executable instructions included in the software modules in the memory 202, and the method for determining the position of the robot arm provided by the embodiment of the present invention is completed in combination with necessary hardware (for example, including the processor 201 and other components connected to the bus 205).
By way of example, the Processor 201 may be an integrated circuit chip having Signal processing capabilities, such as a general purpose Processor, a Digital Signal Processor (DSP), or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or the like, wherein the general purpose Processor may be a microprocessor or any conventional Processor, or the like.
As an example of the position determining apparatus of the robot arm provided by the embodiment of the present invention implemented by hardware, the apparatus provided by the embodiment of the present invention may be implemented by directly using the processor 201 in the form of a hardware decoding processor, for example, the position determining method for implementing the robot arm provided by the embodiment of the present invention may be implemented by one or more Application Specific Integrated Circuits (ASICs), DSPs, programmable Logic Devices (PLDs), complex Programmable Logic Devices (CPLDs), field Programmable Gate Arrays (FPGAs), or other electronic components.
The memory 202 in embodiments of the present invention is used to store various types of data to support the operation of the electronic device 20. Examples of such data include: any executable instructions for operating on the electronic device 20, such as executable instructions, may be included in the executable instructions, and the program implementing the method for determining a position of a slave arm according to an embodiment of the present invention may be included in the executable instructions.
In other embodiments, the position determining apparatus of the robot arm provided by the embodiment of the present invention may be implemented by software, and fig. 2 shows the position determining apparatus 2020 of the robot arm stored in the memory 202, which may be software in the form of programs and plug-ins, and includes a series of modules, and as an example of the programs stored in the memory 202, may include the position determining apparatus 2020 of the robot arm, and the position determining apparatus 2020 of the robot arm includes the following software modules:
and the information transmission module 2081 is used for acquiring the position parameters of the mechanical arm.
The information processing module 2082 is configured to determine a first positional relationship between the mechanical arm and the intermediate coordinate system based on the position parameter of the mechanical arm.
The information transmission module 2081 is used for acquiring the position parameters of the image acquisition device.
The information processing module 2082 is configured to determine a second positional relationship between the image capturing device and the intermediate coordinate system based on the position parameter of the image capturing device.
The information processing module 2082, configured to perform position transformation processing on the first position relationship and the second position relationship to obtain a first position relationship and a second position relationship transformation relationship parameter;
and the information processing module is used for determining the position of the mechanical arm based on the first position relation and the second position relation conversion relation parameter.
Referring to fig. 3, fig. 3 is an optional flowchart of the method for determining the position of the mechanical arm according to the embodiment of the present invention, and it can be understood that the method for determining the position of the mechanical arm shown in fig. 3 may be applied to the field of robot-assisted recognition to determine the position of the mechanical arm, where the steps shown in fig. 3 may be executed by various electronic devices operating the device for determining the position of the mechanical arm, such as a dedicated robot with a position adjusting function, and a medical robot arm with a human body examination function. The following is a description of the steps shown in fig. 3.
Step 301: a position determining device of the mechanical arm acquires position parameters of the mechanical arm.
Step 302: a position determining device of the robot arm determines a first positional relationship between the robot arm and an intermediate coordinate system based on the position parameter of the robot arm.
In some embodiments of the present invention, referring to fig. 4, fig. 4 is an optional schematic flowchart of a method for determining a position of a mechanical arm according to an embodiment of the present invention, so as to obtain a first position relationship between the mechanical arm and an intermediate coordinate system, where the method specifically includes the following steps:
step 401: and configuring a calibration tip at the tail end of the mechanical arm based on the position parameters of the mechanical arm.
Step 402: and configuring a first calibration point, a second calibration point, a third calibration point and a fourth calibration point in a calibration plate corresponding to the mechanical arm.
Referring to fig. 5, fig. 5 is a schematic diagram of a processing procedure of a calibration board in an embodiment of the present invention, specifically, a first calibration point, a second calibration point, a third calibration point, and a fourth calibration point are a, B, C, and D, respectively.
Step 403: determining a set of first calibration point coordinates in a calibration plate coordinate system of the calibration plate for the first calibration point, the second calibration point, the third calibration point, and the fourth calibration point.
Step 404: and adjusting the calibration tip to the corresponding position of different calibration points of the calibration plate.
Step 405: and determining a set of coordinates of the first calibration point, the second calibration point, the third calibration point and the fourth calibration point according to corresponding positions of different calibration points of the calibration plate, and the second calibration point in a mechanical arm base coordinate system corresponding to the mechanical arm.
As shown in fig. 5, in the calibration plate coordinate system, assuming that the side length of each square is 0.03m, the coordinates of four points a, B, C, and D are known, corresponding to (0,0,0), (0.3,0,0), (0.21,0,0), (0.3,0.21,0). Moving the calibrated tip of the end of the mechanical arm to the four points can acquire the coordinates of the end of the mechanical arm relative to the coordinates of the base of the mechanical arm. Thus, after the points of the four known points in the two coordinate systems are determined, the goal is to determine a transformation matrix of the coordinate system where the four points a, B, C, D are located (where the transformation matrix is composed of a rotation matrix R and a transfer matrix T), where the set of first calibration point coordinates of the four points a, B, C, D is denoted as a (a) 1 、a 2 、a 3 、a 4 ) And the set of the second calibration point coordinates is marked as B (B) 1 、b 2 、b 3 、b 4 ) The determination process of the transformation matrix refers to formula 1:
Figure BDA0003148298260000122
and T = Bc-R Ac, bc is an average value of the set of second calibration point coordinates obtained by the mean function, ac is an average value of the set of first calibration point coordinates obtained by the mean function, and when B = R a + T is satisfied, the rotation matrix R and the transfer matrix T form a transformation matrix.
Step 406: and determining a transformation matrix corresponding to the first position relation based on the set of the first calibration point coordinates and the set of the second calibration point coordinates.
In some embodiments of the present invention, determining a transformation matrix corresponding to the first position relationship based on the set of first calibration point coordinates and the set of second calibration point coordinates may be implemented by:
determining a first three-dimensional center point and a second three-dimensional center point of different calibration point matrixes based on the set of first calibration point coordinates and the set of second calibration point coordinates; performing secondary centralization treatment on the calibration points in the calibration plate to obtain a secondary centralization treatment result; performing singular value decomposition processing on the secondary centralization processing result to obtain a rotation matrix; determining a transfer matrix according to the first three-dimensional central point, the second three-dimensional central point and the rotation matrix; and carrying out matrix combination processing on the rotation matrix and the transfer matrix to obtain a transformation matrix corresponding to the first position relation. The method comprises the steps of calculating a first three-dimensional central point Ac and a second three-dimensional central point Bc of an A matrix and a B matrix, then calculating an R matrix, solving the R matrix by adopting an optimization mode, recentering the A and B point sets to generate sets A and B in order to eliminate the influence of a transfer matrix T, and obtaining u, s and v of the matrices by SVD decomposition, so that a rotation matrix R can be obtained, and further calculating to obtain the transfer matrix T according to the rotation matrix R in combination with the first three-dimensional central point Ac and the second three-dimensional central point Bc.
Step 407: and determining the first position relation between the mechanical arm and the middle coordinate system through the transformation matrix corresponding to the first position relation.
Step 303: and the position determining device of the mechanical arm acquires the position parameters of the image acquisition device.
Step 304: and the position determining device of the mechanical arm determines a second position relation between the image acquisition device and the middle coordinate system based on the position parameters of the image acquisition device.
In some embodiments of the present invention, determining the second positional relationship of the image capturing device to the intermediate coordinate system based on the position parameter of the image capturing device may be implemented by:
acquiring an image in the use environment of the mechanical equipment through the image acquisition device; determining the side length parameter of a unit calibration square block in the type of the calibration plate; determining the space posture of the calibration plate based on the image in the use environment of the mechanical equipment and the pixel difference value of the side length parameter of the unit calibration square block; determining the position parameters of the image acquisition device according to the space posture of the calibration plate; and determining an image acquisition matrix according to the spatial attitude of the calibration plate and the position parameters of the image acquisition device, and determining a second position relation between the image acquisition device and the middle coordinate system according to the image acquisition matrix.
Step 305: and the position determining device of the mechanical arm performs position conversion processing on the first position relation and the second position relation to obtain conversion relation parameters of the first position relation and the second position relation.
Wherein, the middle coordinate system is world, the coordinate system of the image acquisition device is Camera, and the coordinate system of the tail end of the mechanical arm is Robot, then the transformation formula of the mechanical arm under the image acquisition device refers to formula 2:
Figure BDA0003148298260000131
therefore, the coordinate system of the image acquisition device and the coordinate system of the tail end of the mechanical arm can be converted through the formula 2.
In some embodiments of the invention, the type of the calibration plate may also be determined; determining a side length parameter of a unit calibration square in the type of the calibration plate based on the type of the calibration plate; and adjusting the set of the coordinates of the first calibration point according to the side length parameter of the unit calibration square, so that the method can adapt to different mechanical arm use environments.
Step 306: the position determining device of the robot arm determines the position of the robot arm based on the first positional relationship and the second positional relationship conversion relationship parameter.
In some embodiments of the invention, a target position of the robotic arm may also be obtained; comparing the position of the robot arm with the target position when determining the position of the robot arm based on the first positional relationship and the second positional relationship conversion relationship parameter; when the coordinate difference value between the position of the mechanical arm and the target position exceeds a threshold value, the position of the mechanical arm is adjusted, and therefore a user can use the mechanical arm more conveniently.
Continuing to describe the method for determining the position of the mechanical arm according to the embodiment of the present invention with reference to fig. 2 and fig. 6, an optional flowchart of the method for determining the position of the mechanical arm according to the embodiment of the present invention is shown, and it can be understood that the method for determining the position of the mechanical arm shown in fig. 6 can be applied to the field of medical image processing to detect a lesion of a patient through the mechanical arm, for example, an ultrasound image of renal bleeding and an ultrasound image of a tumor of a gallbladder, where the steps shown in fig. 6 can be executed by various electronic devices operating the device for determining the position of the mechanical arm, for example, a special medical robot with a position adjusting function, or a medical robotic arm with a human body examination function. The following is a description of the steps shown in fig. 6.
Step 601: and determining a first position relation between the mechanical arm and an intermediate coordinate system based on the position parameters of the mechanical arm.
Step 602: and determining a second position relation between the image acquisition device and the intermediate coordinate system based on the position parameters of the image acquisition device.
Step 603: and performing position conversion processing on the first position relation and the second position relation to obtain a first position relation and a second position relation conversion relation parameter.
Step 604: and determining the position of the mechanical arm based on the first position relation and the second position relation conversion relation parameter.
Step 605: and comparing with the focus detection position.
The medical image obtained by the method for determining the position of the mechanical arm provided by the application comprises a set of medical images to be segmented of the same target region of a target object, wherein the set of medical images to be segmented can be at least one CT image or endoscope image of a focus of the same patient. By adjusting the position of the mechanical arm, the medical image can be actively turned, rotated, zoomed and contrast enhanced. Wherein, the type of the input medical image can be a two-dimensional plane image or a three-dimensional image; for example: computed Tomography (CT) magnetic Resonance Imaging (mrimaginal Resonance Imaging) ultrasound, X-ray, electrocardiogram, electroencephalogram, optical photography, and the like, images generated by medical instruments.
Step 606: and when the coordinate difference value between the position of the mechanical arm and the focus detection position exceeds a threshold value, adjusting the position of the mechanical arm.
When the coordinate difference between the position of the mechanical arm of the ultrasonic image of renal bleeding and the focus detection position is determined to be greater than or equal to 1cm of a threshold value, adjusting the position of the mechanical arm; when the coordinate difference between the position of the ultrasonic image mechanical arm for determining the gallbladder tumor and the focus detection position is greater than or equal to 0.5cm, the position of the mechanical arm is adjusted, so that the focus is accurately imaged by accurately determining the position of the mechanical arm.
The beneficial technical effects are as follows:
according to the embodiment of the invention, the position parameters of the mechanical arm are obtained; determining a first position relation between the mechanical arm and an intermediate coordinate system based on the position parameters of the mechanical arm; acquiring position parameters of the image acquisition device; determining a second positional relationship between the image acquisition device and an intermediate coordinate system based on the position parameters of the image acquisition device; performing position conversion processing on the first position relation and the second position relation to obtain a first position relation conversion relation parameter and a second position relation conversion relation parameter; the position of the mechanical arm is determined based on the first position relation and the second position relation conversion relation parameter, so that the position of the mechanical arm can be accurately determined, accumulated errors are reduced, and meanwhile, the accuracy of a single-posture obtaining effect is improved, so that a user can use the mechanical arm more conveniently.
The above description is only exemplary of the present invention and should not be taken as limiting the scope of the present invention, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (10)

1. A method for determining the position of a mechanical arm, which is used in a mechanical device consisting of the mechanical arm and an image acquisition device, is characterized by comprising the following steps:
acquiring position parameters of the mechanical arm;
determining a first position relation between the mechanical arm and an intermediate coordinate system based on the position parameters of the mechanical arm;
acquiring position parameters of the image acquisition device;
determining a second positional relationship between the image acquisition device and an intermediate coordinate system based on the position parameters of the image acquisition device;
performing position conversion processing on the first position relation and the second position relation to obtain a first position relation and a second position relation conversion relation parameter;
and determining the position of the mechanical arm based on the first position relation and the second position relation conversion relation parameter.
2. The method of claim 1, wherein determining the first positional relationship of the robotic arm to an intermediate coordinate system based on the positional parameters of the robotic arm comprises:
configuring a calibration tip at the tail end of the mechanical arm based on the position parameters of the mechanical arm;
a first calibration point, a second calibration point, a third calibration point and a fourth calibration point are configured in the calibration plate corresponding to the mechanical arm;
determining a set of first calibration point coordinates in a calibration plate coordinate system of the calibration plate for the first calibration point, the second calibration point, the third calibration point, and the fourth calibration point;
adjusting the calibration tip to the corresponding position of different calibration points of the calibration plate;
determining a set of coordinates of the first calibration point, the second calibration point, the third calibration point and the fourth calibration point according to corresponding positions of different calibration points of the calibration plate, and the second calibration point in a base coordinate system of the mechanical arm corresponding to the mechanical arm;
determining a transformation matrix corresponding to the first position relation based on the set of the first calibration point coordinates and the set of the second calibration point coordinates;
and determining the first position relation between the mechanical arm and the middle coordinate system through the transformation matrix corresponding to the first position relation.
3. The method according to claim 2, wherein determining a transformation matrix corresponding to the first location relationship based on the first set of scaled point coordinates and the second set of scaled point coordinates comprises:
determining a first three-dimensional center point and a second three-dimensional center point of different calibration point matrixes based on the set of first calibration point coordinates and the set of second calibration point coordinates;
performing secondary centralization treatment on the calibration points in the calibration plate to obtain a secondary centralization treatment result;
performing singular value decomposition processing on the secondary centralization processing result to obtain a rotation matrix;
determining a transfer matrix according to the first three-dimensional central point, the second three-dimensional central point and the rotation matrix;
and carrying out matrix combination processing on the rotation matrix and the transfer matrix to obtain a transformation matrix corresponding to the first position relation.
4. The method of claim 2, further comprising:
determining the type of the calibration plate;
determining a side length parameter of a unit calibration square in the type of the calibration plate based on the type of the calibration plate;
and adjusting the set of the coordinates of the first calibration point according to the side length parameter of the unit calibration square.
5. The method of claim 1, wherein determining the second positional relationship of the image capture device to an intermediate coordinate system based on the positional parameters of the image capture device comprises:
acquiring an image in the use environment of the mechanical equipment through the image acquisition device;
determining the side length parameter of a unit calibration square block in the type of the calibration plate;
determining the space posture of the calibration plate based on the image in the use environment of the mechanical equipment and the pixel difference value of the side length parameter of the unit calibration square block;
determining the position parameters of the image acquisition device according to the space posture of the calibration plate;
and determining an image acquisition matrix according to the spatial attitude of the calibration plate and the position parameters of the image acquisition device, and determining a second position relation between the image acquisition device and the middle coordinate system according to the image acquisition matrix.
6. The method of claim 1, further comprising:
acquiring a target position of a mechanical arm;
comparing the position of the robot arm with the target position when determining the position of the robot arm based on the first positional relationship and the second positional relationship conversion relationship parameter;
and when the coordinate difference value between the position of the mechanical arm and the target position exceeds a threshold value, adjusting the position of the mechanical arm.
7. A position determining apparatus of a robot arm, characterized by comprising:
the information transmission module is used for acquiring the position parameters of the mechanical arm;
the information processing module is used for determining a first position relation between the mechanical arm and an intermediate coordinate system based on the position parameters of the mechanical arm;
the information transmission module is used for acquiring the position parameters of the image acquisition device;
the information processing module is used for determining a second position relation between the image acquisition device and the middle coordinate system based on the position parameters of the image acquisition device;
the information processing module is configured to perform position transformation processing on the first position relationship and the second position relationship to obtain a first position relationship and a second position relationship conversion relationship parameter;
and the information processing module is used for determining the position of the mechanical arm based on the first position relation and the second position relation conversion relation parameter.
8. An electronic device, characterized in that the electronic device comprises:
a memory for storing executable instructions;
a processor for executing the executable instructions stored in the memory to implement the method of determining the position of a robotic arm of any of claims 1 to 6.
9. A computer readable storage medium storing executable instructions, wherein the executable instructions when executed by a processor implement the method of determining the position of a robotic arm of any of claims 1 to 6.
10. A robot, characterized in that the robot comprises:
the mechanical equipment comprises a mechanical arm and an image acquisition device, and is used for receiving instructions of a processor and realizing the position determination method of the mechanical arm in any one of claims 1 to 6;
a memory for storing executable instructions;
a processor for executing the executable instructions stored in the memory to implement the method of determining the position of a robotic arm of any of claims 1 to 6.
CN202110757391.1A 2021-07-05 2021-07-05 Method and device for determining position of mechanical arm, electronic equipment and storage medium Pending CN115582829A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110757391.1A CN115582829A (en) 2021-07-05 2021-07-05 Method and device for determining position of mechanical arm, electronic equipment and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110757391.1A CN115582829A (en) 2021-07-05 2021-07-05 Method and device for determining position of mechanical arm, electronic equipment and storage medium

Publications (1)

Publication Number Publication Date
CN115582829A true CN115582829A (en) 2023-01-10

Family

ID=84772491

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110757391.1A Pending CN115582829A (en) 2021-07-05 2021-07-05 Method and device for determining position of mechanical arm, electronic equipment and storage medium

Country Status (1)

Country Link
CN (1) CN115582829A (en)

Citations (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102294695A (en) * 2010-06-25 2011-12-28 鸿富锦精密工业(深圳)有限公司 Robot calibration method and calibration system
CN105509671A (en) * 2015-12-01 2016-04-20 中南大学 Method for calibrating central point of robot tool through employing plane calibration plate
CN106272444A (en) * 2016-08-31 2017-01-04 山东中清智能科技有限公司 A kind of realize trick relation and method that dual robot relation is demarcated simultaneously
CN106393174A (en) * 2016-10-09 2017-02-15 华中科技大学 Method for calibrating structural parameters of robot using double-ball-bar
CN107450885A (en) * 2017-07-21 2017-12-08 上海交通大学 A kind of coordinate transform method for solving of industrial robot and three-dimension sensor
CN109658460A (en) * 2018-12-11 2019-04-19 北京无线电测量研究所 A kind of mechanical arm tail end camera hand and eye calibrating method and system
CN109807937A (en) * 2018-12-28 2019-05-28 北京信息科技大学 A kind of Robotic Hand-Eye Calibration method based on natural scene
CN110276799A (en) * 2019-05-29 2019-09-24 季华实验室 A kind of coordinate scaling method, calibration system and mechanical arm
CN110842919A (en) * 2019-11-05 2020-02-28 四川长虹智能制造技术有限公司 Visual guide method for screwing of robot
CN111015657A (en) * 2019-12-19 2020-04-17 佛山科学技术学院 Adaptive control method, device and system of industrial robot
CN111482964A (en) * 2020-04-17 2020-08-04 上海智殷自动化科技有限公司 Novel robot hand-eye calibration method
CN111515950A (en) * 2020-04-28 2020-08-11 腾讯科技(深圳)有限公司 Method, device and equipment for determining transformation relation of robot coordinate system and storage medium
CN111890355A (en) * 2020-06-29 2020-11-06 北京大学 Robot calibration method, device and system
CN111968185A (en) * 2020-09-01 2020-11-20 深圳辰视智能科技有限公司 Calibration board, nine-point calibration object grabbing method and system based on code definition
CN112171666A (en) * 2020-09-15 2021-01-05 深圳市商汤科技有限公司 Pose calibration method and device for visual robot, visual robot and medium
CN113021328A (en) * 2019-12-09 2021-06-25 广东博智林机器人有限公司 Hand-eye calibration method, device, equipment and medium

Patent Citations (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102294695A (en) * 2010-06-25 2011-12-28 鸿富锦精密工业(深圳)有限公司 Robot calibration method and calibration system
CN105509671A (en) * 2015-12-01 2016-04-20 中南大学 Method for calibrating central point of robot tool through employing plane calibration plate
CN106272444A (en) * 2016-08-31 2017-01-04 山东中清智能科技有限公司 A kind of realize trick relation and method that dual robot relation is demarcated simultaneously
CN106393174A (en) * 2016-10-09 2017-02-15 华中科技大学 Method for calibrating structural parameters of robot using double-ball-bar
CN107450885A (en) * 2017-07-21 2017-12-08 上海交通大学 A kind of coordinate transform method for solving of industrial robot and three-dimension sensor
CN109658460A (en) * 2018-12-11 2019-04-19 北京无线电测量研究所 A kind of mechanical arm tail end camera hand and eye calibrating method and system
CN109807937A (en) * 2018-12-28 2019-05-28 北京信息科技大学 A kind of Robotic Hand-Eye Calibration method based on natural scene
CN110276799A (en) * 2019-05-29 2019-09-24 季华实验室 A kind of coordinate scaling method, calibration system and mechanical arm
CN110842919A (en) * 2019-11-05 2020-02-28 四川长虹智能制造技术有限公司 Visual guide method for screwing of robot
CN113021328A (en) * 2019-12-09 2021-06-25 广东博智林机器人有限公司 Hand-eye calibration method, device, equipment and medium
CN111015657A (en) * 2019-12-19 2020-04-17 佛山科学技术学院 Adaptive control method, device and system of industrial robot
CN111482964A (en) * 2020-04-17 2020-08-04 上海智殷自动化科技有限公司 Novel robot hand-eye calibration method
CN111515950A (en) * 2020-04-28 2020-08-11 腾讯科技(深圳)有限公司 Method, device and equipment for determining transformation relation of robot coordinate system and storage medium
CN111890355A (en) * 2020-06-29 2020-11-06 北京大学 Robot calibration method, device and system
CN111968185A (en) * 2020-09-01 2020-11-20 深圳辰视智能科技有限公司 Calibration board, nine-point calibration object grabbing method and system based on code definition
CN112171666A (en) * 2020-09-15 2021-01-05 深圳市商汤科技有限公司 Pose calibration method and device for visual robot, visual robot and medium

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
新机器视觉: "深入浅出地理解机器人手眼标定", 微信公众号, 23 May 2020 (2020-05-23), pages 1 - 5 *

Similar Documents

Publication Publication Date Title
CN107274400B (en) Space positioning device, positioning processing method and device, and virtual reality system
EP3872764B1 (en) Method and apparatus for constructing map
US20220319050A1 (en) Calibration method and apparatus, processor, electronic device, and storage medium
CN111445533B (en) Binocular camera calibration method, device, equipment and medium
WO2020057121A1 (en) Data processing method and apparatus, electronic device and storage medium
US20240046557A1 (en) Method, device, and non-transitory computer-readable storage medium for reconstructing a three-dimensional model
CN115984371A (en) Scanning head posture detection method, device, equipment and medium
CN113256718A (en) Positioning method and device, equipment and storage medium
CN113362445B (en) Method and device for reconstructing object based on point cloud data
CN107146243B (en) Method for registering images and device
WO2021128731A1 (en) Image processing method and apparatus, image processing device, and storage medium
CN115582829A (en) Method and device for determining position of mechanical arm, electronic equipment and storage medium
CN112102417A (en) Method and device for determining world coordinates and external reference calibration method for vehicle-road cooperative roadside camera
CN116258752A (en) Registration method, registration apparatus, electronic device, and computer-readable storage medium
CN113262048B (en) Spatial registration method and device, terminal equipment and intraoperative navigation system
CN113570659B (en) Shooting device pose estimation method, device, computer equipment and storage medium
CN114926542A (en) Mixed reality fixed reference system calibration method based on optical positioning system
CN111870346B (en) Space registration method and device for robot and image equipment and electronic equipment
CN113052915A (en) Camera external parameter calibration method and device, augmented reality system, terminal device and storage medium
CN111080734B (en) Method and terminal for processing Positron Emission Tomography (PET) data
US8755819B1 (en) Device location determination using images
CN112132909A (en) Parameter acquisition method and device, media data processing method and storage medium
CN112880675B (en) Pose smoothing method and device for visual positioning, terminal and mobile robot
CN112767484B (en) Fusion method of positioning model, positioning method and electronic device
CN116100564B (en) High-precision calibration method and device for calibrating manipulator

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination