WO2023207186A1 - 目标定位方法、装置、电子设备及存储介质 - Google Patents

目标定位方法、装置、电子设备及存储介质 Download PDF

Info

Publication number
WO2023207186A1
WO2023207186A1 PCT/CN2022/142695 CN2022142695W WO2023207186A1 WO 2023207186 A1 WO2023207186 A1 WO 2023207186A1 CN 2022142695 W CN2022142695 W CN 2022142695W WO 2023207186 A1 WO2023207186 A1 WO 2023207186A1
Authority
WO
WIPO (PCT)
Prior art keywords
cameras
target object
information
hand
pose
Prior art date
Application number
PCT/CN2022/142695
Other languages
English (en)
French (fr)
Inventor
吴文峰
姜德志
李小娟
Original Assignee
博众精工科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 博众精工科技股份有限公司 filed Critical 博众精工科技股份有限公司
Publication of WO2023207186A1 publication Critical patent/WO2023207186A1/zh

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1628—Programme controls characterised by the control loop
    • B25J9/1653—Programme controls characterised by the control loop parameters identification, estimation, stiffness, accuracy, error analysis
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1602—Programme controls characterised by the control system, structure, architecture
    • B25J9/161—Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
    • B25J9/1661—Programme controls characterised by programming, planning systems for manipulators characterised by task planning, object-oriented languages
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/16—Programme controls
    • B25J9/1694—Programme controls characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
    • B25J9/1697—Vision controlled systems

Definitions

  • This application relates to the field of machine vision technology, for example, to a target positioning method, device, electronic equipment and storage medium.
  • Machine vision is widely used in industries such as automated production and intelligent equipment manufacturing.
  • Robotic arms often have insufficient positioning accuracy when grabbing objects.
  • This application provides a target positioning method, device, electronic equipment and storage medium to solve the problem of insufficient positioning accuracy of a robotic arm.
  • a target positioning method including:
  • the hand-eye distance corresponding to the robot hand and at least two cameras, and obtain the pose information corresponding to the at least two cameras and the pose information of the target object, wherein the hand-eye distance is the distance between the robot hand and the at least two cameras. The distance between each of the cameras;
  • the positioning information of the target object is obtained by performing an average compensation process based on the hand-eye distance corresponding to the manipulator and the at least two cameras respectively and the posture deviation information corresponding to the at least two cameras and the target object.
  • a target positioning device including:
  • the information acquisition module is configured to obtain the hand-eye distance corresponding to the robot hand and at least two cameras, and to obtain the pose information corresponding to the at least two cameras and the pose information of the target object, wherein the hand-eye distance is the the distance between the robot hand and each of the at least two cameras;
  • a deviation information determination module configured to determine the pose deviation information of the at least two cameras corresponding to the target object based on the pose information corresponding to the at least two cameras and the pose information of the target object;
  • the compensation processing module is configured to perform average compensation processing based on the hand-eye distance corresponding to the manipulator and the at least two cameras respectively and the posture deviation information corresponding to the at least two cameras and the target object to obtain the target object. Positioning information.
  • an electronic device including:
  • the memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor, so that the at least one processor can execute the method described in any embodiment of the present application. Targeting methods.
  • a computer-readable storage medium stores computer instructions, and the computer instructions are used to implement any of the embodiments of the present application when executed by a processor. target positioning method.
  • Figure 1 is a flow chart of a target positioning method provided according to Embodiment 1 of the present application.
  • Figure 2 is a schematic diagram of hand-eye distance determination provided according to Embodiment 1 of the present application.
  • Figure 3 is a flow chart of a target positioning method provided according to Embodiment 2 of the present application.
  • Figure 4 is a schematic structural diagram of a target positioning device provided according to Embodiment 3 of the present application.
  • FIG. 5 is a schematic structural diagram of an electronic device that implements the target positioning method according to the embodiment of the present application.
  • Figure 1 is a flow chart of a target positioning method provided in Embodiment 1 of the present application. This embodiment can be applied to the situation where a robotic arm automatically positions a captured target object.
  • This method can be executed by a target positioning device.
  • the target positioning device can be implemented in the form of hardware and/or software, and the target positioning device can be configured in an electronic device.
  • the electronic device may be a terminal and/or a server. As shown in Figure 1, the method includes the following steps.
  • the hand-eye distance is the distance between the robot hand and each of the at least two cameras.
  • the camera can be fixedly installed on the gripper of the manipulator.
  • the manipulator can include multiple grippers, that is, the manipulator uses multiple grippers to grasp the target object, and the camera can follow the movement of the manipulator.
  • the hand-eye distance refers to the distance from the center of the manipulator's flange to the center of the camera's field of view. When the number of cameras is multiple, the number of obtained hand-eye distances is also multiple.
  • the pose information of the camera refers to the standard position information and standard attitude information in the image captured by the camera
  • the pose information of the target object refers to the actual position information and actual attitude information of the target object in the image captured by the camera.
  • obtaining the hand-eye distance corresponding to the manipulator and at least two cameras includes: obtaining the position information of the rotation center of the manipulator; obtaining the calibration position information of the at least two cameras respectively, based on the at least two cameras.
  • the calibration position information corresponding to each camera and the position information of the rotation center of the manipulator determine the hand-eye distance corresponding to the at least two cameras.
  • the position information of the manipulator rotation center refers to the position coordinates of the manipulator flange center.
  • Calibration position information refers to the position coordinates of the center of the camera's field of view.
  • the position coordinates of the robot flange center can be (X 1 , Y 1 ); as shown in Figure 2 , the number of cameras can be two, as shown by the dotted line on the left side in Figure 2, the distance from the position coordinates of the center of the first camera's field of view to the position coordinates of the center of the manipulator flange can be the hand-eye distance of the first camera; on the right side of Figure 2
  • the dotted line on the side, the distance between the position coordinates of the second camera's field of view center and the position coordinates of the manipulator flange center can be the hand-eye distance of the second camera.
  • the calibration position information can be determined by moving the manipulator and pricking with a sharp needle, where the position of the sharp needle pricking can be the eye position, and the eye position can be determined as the calibration position information.
  • the pose information of each camera and the pose information of the target object can be obtained by capturing images with the cameras.
  • the feature points of the target object in the image captured by the camera can be extracted, and the position information and attitude information of the feature points can be used as the pose information of the target object.
  • the feature point can be a mark point or center of mass of the target object, etc., which is not limited here.
  • the pose information of the camera may be the pose information of the reference point in the image captured by the camera, where the reference point may be a marker point or a center point of the image, etc., which is not limited here.
  • the pose deviation information refers to the position deviation value or angle deviation value between the standard pose information in the image captured by the camera and the pose information of the actual target object.
  • the pose deviation information of each camera and the target object is determined through the pose information of each camera and the pose information of the target object, thereby realizing the detection of the pose deviation of the target object.
  • the pose deviation information can be used to compensate for the positioning deviation. This enables precise grasping of the target object even if it is offset.
  • the positioning information of the target object refers to the position of the target object to be grasped by the manipulator.
  • Average compensation processing refers to adding and averaging the hand-eye distance and pose deviation information corresponding to multiple cameras, so that the obtained positioning information of the target object is more accurate and the positioning accuracy of the manipulator is improved.
  • the method before obtaining the hand-eye distances corresponding to the robot hand and the at least two cameras, the method further includes: obtaining the actual coordinate information corresponding to the robot hand and the pixel coordinate information corresponding to the at least two cameras; The coordinate transformation relationship is determined based on the actual coordinate information corresponding to the manipulator and the pixel coordinate information corresponding to at least two cameras.
  • the actual coordinate information may be robot coordinate information.
  • Pixel coordinate information refers to camera coordinate information.
  • the actual coordinate information and the pixel coordinate information belong to two different position coordinate systems, and the two cannot be directly calculated. Therefore, the coordinate conversion relationship between the actual coordinate information corresponding to the manipulator and the pixel coordinate information corresponding to each camera can be established to realize the coordinate system conversion.
  • the coordinate transformation relationship can be determined through the nine-point calibration method.
  • the method further includes: based on the coordinate transformation relationship, The pixel coordinate information corresponding to the at least two cameras is converted to obtain actual coordinate information corresponding to each pixel coordinate information.
  • the coordinate conversion relationship can be used as a conversion rule.
  • the corresponding pixel coordinate information of at least two cameras can be converted to obtain the corresponding actual coordinate information and realize multiple parameters.
  • the coordinate system is unified to facilitate operations between multiple parameters.
  • the technical solution of the embodiment of the present application realizes the hand-eye distance and multiple camera information by acquiring the hand-eye distance between the robot hand and at least two cameras, and acquiring the corresponding pose information of the at least two cameras and the pose information of the target object.
  • the hand-eye distance between the camera and the manipulator and the detected pose deviation information are averagely compensated, making the positioning information of the target object more accurate and solving the problem of insufficient positioning accuracy of the current manipulator.
  • Figure 3 is a flow chart of a target positioning method provided in Embodiment 2 of the present application.
  • the target positioning method in this embodiment can be combined with multiple optional solutions of the target positioning method provided in the above embodiment.
  • the target positioning method provided in this embodiment is explained.
  • the pose information of the camera includes a center pixel position and a standard photographing angle
  • the pose information of the target object includes a characteristic pixel position and a characteristic angle
  • the pose information is based on the corresponding poses of the at least two cameras.
  • information and the pose information of the target object to determine the pose deviation information of the at least two cameras from the target object, including: determining the position of the target object based on the characteristic pixel position of the target object and the center pixel position corresponding to the at least two cameras respectively.
  • Position deviation information corresponding to the at least two cameras determining angle deviation information corresponding to the at least two cameras based on the characteristic angle of the target object and the standard photographing angle corresponding to the at least two cameras; Performing an average compensation process on the hand-eye distance corresponding to the robot hand and the at least two cameras and the posture deviation information of the at least two cameras and the target object respectively, to obtain the positioning information of the target object, including: based on the robot hand The hand-eye distance, the position deviation information and the angle deviation information respectively corresponding to the at least two cameras are averaged and compensated to obtain the positioning information of the target object.
  • the method includes the following steps.
  • the center pixel position refers to the pixel center point of the image captured by the camera, which is the standard position information.
  • the standard photographing angle refers to the standard placement angle of the target object in the image captured by the camera, that is, the standard posture information, such as placing the target object horizontally.
  • the feature pixel position refers to the actual position of the target object feature point in the image captured by the camera, that is, the actual position information.
  • the feature angle refers to the actual angle of the target object feature point in the image captured by the camera, that is, the actual attitude information.
  • the position deviation information corresponding to at least two cameras can be obtained by making a difference between the characteristic pixel position of the target object and the center pixel positions respectively corresponding to at least two cameras.
  • the characteristic angle of the target object can be differed from the standard photographing angles corresponding to at least two cameras to obtain the angle deviation information corresponding to at least two cameras.
  • S240 Perform average compensation processing based on the hand-eye distances, the position deviation information, and the angle deviation information corresponding to the manipulator and the at least two cameras, respectively, to obtain positioning information of the target object.
  • the camera may include a first camera and a second camera. Based on the hand-eye distance, position deviation information and angular deviation information of at least two cameras and the manipulator, average compensation processing is performed to obtain the positioning information of the target object, including:
  • _ represents the hand-eye distance of the second camera
  • (d x1 , d y1 ) represents the position deviation information of the first camera
  • (d x2 , d y2 ) represents the position deviation information of the second camera
  • d a1 represents the angular deviation of the first camera information
  • d a2 represents the angular deviation information of the second camera.
  • the target object may be a large workpiece or product that is difficult to capture completely with a single camera.
  • the target object is positioned based on the partial images of the multiple target objects. While avoiding the use of large-resolution cameras, it also improves the positioning accuracy of target objects.
  • the technical solution of the embodiment of the present application is to obtain the hand-eye distance between the robot hand and at least two cameras, and obtain the central pixel position and standard photographing angle corresponding to the at least two cameras and the characteristic pixel position and characteristic angle of the target object; based on Based on the characteristic pixel position of the target object and the center pixel positions corresponding to at least two cameras, the position deviation information corresponding to at least two cameras is determined, and based on the characteristic angle of the target object and the standard photographing angle corresponding to at least two cameras, the position deviation information is determined.
  • the angle deviation information corresponding to at least two cameras realizes the detection of the position deviation and angle deviation of the target object, and average compensation processing is performed based on the hand-eye distance, position deviation information and angle deviation information of at least two cameras and the manipulator, so that The positioning information of the target object obtained is more accurate, which solves the problem of insufficient positioning accuracy of the robotic arm.
  • FIG 4 is a schematic structural diagram of a target positioning device provided in Embodiment 3 of the present application. As shown in Figure 4, the device includes the following modules.
  • the information acquisition module 310 is configured to obtain the hand-eye distance corresponding to the robot hand and at least two cameras, and to obtain the pose information corresponding to the at least two cameras and the pose information of the target object, wherein the hand-eye distance is the the distance between the robot hand and each of the at least two cameras;
  • the deviation information determination module 320 is configured to determine the pose deviation information of the at least two cameras corresponding to the target object based on the pose information corresponding to the at least two cameras and the pose information of the target object;
  • the compensation processing module 330 is configured to perform an average compensation process based on the hand-eye distance corresponding to the manipulator and the at least two cameras respectively and the posture deviation information corresponding to the at least two cameras and the target object to obtain the target object. positioning information.
  • the technical solution of the embodiment of the present application realizes the hand-eye distance and multiple cameras by obtaining the hand-eye distance corresponding to the robot hand and at least two cameras, and obtaining the posture information corresponding to the at least two cameras and the posture information of the target object. Acquisition of information; determining the pose deviation information of at least two cameras and the target object based on the corresponding pose information of at least two cameras and the pose information of the target object, realizing the detection of the pose deviation of the target object, based on at least The hand-eye distance between the two cameras and the manipulator and the detected pose deviation information are averagely compensated to make the positioning information of the target object more accurate and solve the problem of insufficient positioning accuracy of the manipulator.
  • the information acquisition module 310 is set to:
  • Calibration position information of at least two cameras is obtained respectively, and hand-eye distances corresponding to the at least two cameras are determined based on the calibration position information corresponding to the at least two cameras and the position information of the rotation center of the manipulator.
  • the pose information of the camera includes the center pixel position and the standard photographing angle
  • the pose information of the target object includes the characteristic pixel position and the characteristic angle
  • the deviation information determination module 320 includes:
  • the position deviation information determination unit is configured to determine the position deviation information corresponding to at least two cameras based on the characteristic pixel position of the target object and the center pixel positions respectively corresponding to the at least two cameras;
  • the angle deviation information determination unit is configured to determine the angle deviation information corresponding to at least two cameras based on the characteristic angle of the target object and the standard photographing angles respectively corresponding to at least two cameras;
  • the compensation processing module 330 includes:
  • the positioning information determination unit is configured to perform an average compensation process based on the hand-eye distance, the position deviation information and the angle deviation information respectively corresponding to the manipulator and the at least two cameras to obtain the positioning information of the target object.
  • the position deviation information determination unit is set to:
  • the characteristic pixel position of the target object is compared with the center pixel positions corresponding to at least two cameras to obtain position deviation information corresponding to at least two cameras.
  • the angle deviation information determination unit is set to:
  • the camera includes a first camera and a second camera
  • the positioning information determination unit is configured to:
  • _ represents the hand-eye distance of the second camera
  • (d x1 , d y1 ) represents the position deviation information of the first camera
  • (d x2 , d y2 ) represents the position deviation information of the second camera
  • d a1 represents the angular deviation of the first camera information
  • d a2 represents the angular deviation information of the second camera.
  • the device also includes:
  • the coordinate information acquisition module is configured to obtain the actual coordinate information corresponding to the manipulator and the pixel coordinate information corresponding to at least two cameras;
  • a conversion relationship determination module configured to determine a coordinate conversion relationship based on the actual coordinate information corresponding to the manipulator and the pixel coordinate information corresponding to at least two cameras;
  • the pixel coordinate conversion module is configured to convert the pixel coordinate information corresponding to the at least two cameras based on the coordinate conversion relationship to obtain actual coordinate information corresponding to each pixel coordinate information.
  • the target positioning device provided by the embodiments of this application can execute the target positioning method provided by any embodiment of this application, and has corresponding functional modules and effects for executing the method.
  • FIG. 5 shows a schematic structural diagram of an electronic device 10 that can be used to implement embodiments of the present application.
  • Electronic device 10 is intended to represent many forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers.
  • Electronic devices may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (eg, helmets, glasses, watches, etc.), and other similar computing devices.
  • the components shown herein, their connections and relationships, and their functions are examples only and are not intended to limit the implementation of the present application as described and/or claimed herein.
  • the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (Read-Only Memory, ROM) 12, a random access memory (Random Access Memory, RAM) 13 and so on, wherein the memory stores a computer program that can be executed by at least one processor 11.
  • the processor 11 can execute multiple functions according to the computer program stored in the ROM 12 or the computer program loaded from the storage unit 18 into the RAM 13. appropriate actions and treatments.
  • various programs and data required for the operation of the electronic device 10 can also be stored.
  • the processor 11, the ROM 12 and the RAM 13 are connected to each other via the bus 14.
  • An input/output (I/O) interface 15 is also connected to the bus 14 .
  • the I/O interface 15 Multiple components in the electronic device 10 are connected to the I/O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc. etc.; and communication unit 19, such as network card, modem, wireless communication transceiver, etc.
  • the communication unit 19 allows the electronic device 10 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunications networks.
  • Processor 11 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (Central Processing Unit, CPU), a graphics processing unit (GPU), a variety of dedicated artificial intelligence (Artificial Intelligence, AI) computing chips, a variety of running Machine learning model algorithm processor, digital signal processor (Digital Signal Processor, DSP), and any appropriate processor, controller, microcontroller, etc.
  • the processor 11 performs multiple methods and processes described above, such as a target positioning method, including:
  • the positioning information of the target object is obtained by performing an average compensation process based on the hand-eye distance corresponding to the manipulator and the at least two cameras respectively and the posture deviation information corresponding to the at least two cameras and the target object.
  • the target positioning method may be implemented as a computer program, which is tangibly embodied in a computer-readable storage medium, such as the storage unit 18 .
  • part or all of the computer program may be loaded and/or installed onto the electronic device 10 via the ROM 12 and/or the communication unit 19.
  • the processor 11 may be configured to perform the target positioning method in any other suitable manner (eg, by means of firmware).
  • FPGAs Field Programmable Gate Arrays
  • ASICs Application Specific Integrated Circuits
  • ASSP Application Specific Standard Parts
  • SOC System on Chip
  • CPLD Complex Programmable Logic Device
  • These various embodiments may include implementation in one or more computer programs executable and/or interpreted on a programmable system including at least one programmable processor, the programmable processor
  • the processor which may be a special purpose or general purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
  • An output device may be a special purpose or general purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
  • An output device may be a special purpose or general purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
  • Computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that the computer program, when executed by the processor, causes the functions/operations specified in the flowcharts and/or block diagrams to be implemented.
  • a computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
  • a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device.
  • Computer-readable storage media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any suitable combination of the foregoing.
  • the computer-readable storage medium may be a machine-readable signal medium.
  • Machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard drives, RAM, ROM, Erasable Programmable Read-Only Memory (EPROM), flash memory, fiber optics , portable compact disk read-only memory (Compact Disc Read Only Memory, CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • the storage medium may be a non-transitory storage medium.
  • the systems and techniques described herein may be implemented on an electronic device having a display device (e.g., a cathode ray tube (CRT) or liquid crystal) for displaying information to the user.
  • a display device e.g., a cathode ray tube (CRT) or liquid crystal
  • a display Liquid Crystal Display, LCD monitor
  • a keyboard and pointing device e.g., a mouse or a trackball
  • Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and may be provided in any form, including Acoustic input, voice input or tactile input) to receive input from the user.
  • the systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., A user's computer having a graphical user interface or web browser through which the user can interact with implementations of the systems and technologies described herein), or including such backend components, middleware components, or any combination of front-end components in a computing system.
  • the components of the system may be interconnected by any form or medium of digital data communication (eg, a communications network). Examples of communication networks include: Local Area Network (LAN), Wide Area Network (WAN), blockchain network, and the Internet.
  • Computing systems may include clients and servers.
  • Clients and servers are generally remote from each other and typically interact over a communications network.
  • the relationship of client and server is created by computer programs running on corresponding computers and having a client-server relationship with each other.
  • the server can be a cloud server, also known as cloud computing server or cloud host. It is a host product in the cloud computing service system to solve the problems that exist in traditional physical host and virtual private server (VPS) services. It has the disadvantages of difficult management and weak business scalability.
  • VPN virtual private server
  • Steps can be reordered, added, or removed using various forms of the process shown above.
  • multiple steps described in this application can be executed in parallel, sequentially, or in different orders.
  • the desired results of the technical solution of this application can be achieved, there is no limitation here.

Landscapes

  • Engineering & Computer Science (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Automation & Control Theory (AREA)
  • Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Fuzzy Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

一种目标定位方法，包括：获取机械手与至少两个相机分别对应的手眼距离，以及获取至少两个相机分别对应的位姿信息和目标物体的位姿信息，其中，手眼距离为机械手与至少两个相机中的每一个之间的距离；基于至少两个相机分别对应的位姿信息和目标物体的位姿信息确定至少两个相机分别与目标物体对应的位姿偏差信息；基于机械手与至少两个相机分别对应的手眼距离和至少两个相机分别与目标物体对应的位姿偏差信息进行平均补偿处理，得到目标物体的定位信息。该方法可以提高目标物体的定位精度。还提供一种目标定位装置、电子设备及存储介质。

Description

目标定位方法、装置、电子设备及存储介质
本申请要求在2022年04月27日提交中国专利局、申请号为202210457393.3的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本申请涉及机器视觉技术领域，例如涉及一种目标定位方法、装置、电子设备及存储介质。
背景技术
机器视觉广泛应用于自动化生产、以及智能设备制造等行业。
将机器视觉技术和机械臂技术相结合，利用机器视觉的定位功能使机械臂具有自己的“眼睛”来获取工件的位置等环境信息，引导机械臂完成抓取、以及搬运等工作，对提高生产线的效率和扩展机械臂的应用范围都具有重要的意义。
机械臂对于物品的抓取往往会出现定位精度不够的情况。
发明内容
本申请提供了一种目标定位方法、装置、电子设备及存储介质，以解决机械臂定位精度不足的问题。
根据本申请的一方面，提供了一种目标定位方法，包括：
获取机械手与至少两个相机分别对应的手眼距离，以及获取所述至少两个相机分别对应的位姿信息和目标物体的位姿信息，其中，所述手眼距离为所述机械手与所述至少两个相机中的每一个之间的距离；
基于所述至少两个相机分别对应的位姿信息和目标物体的位姿信息确定所述至少两个相机分别与目标物体对应的位姿偏差信息；
基于所述机械手与所述至少两个相机分别对应的手眼距离和所述至少两个相机分别与目标物体对应的位姿偏差信息进行平均补偿处理，得到所述目标物体的定位信息。
根据本申请的另一方面，提供了一种目标定位装置，包括：
信息获取模块，设置为获取机械手与至少两个相机分别对应的手眼距离，以及获取所述至少两个相机分别对应的位姿信息和目标物体的位姿信息，其 中，所述手眼距离为所述机械手与所述至少两个相机中的每一个之间的距离；
偏差信息确定模块，设置为基于所述至少两个相机分别对应的位姿信息和目标物体的位姿信息确定所述至少两个相机分别与目标物体对应的位姿偏差信息；
补偿处理模块，设置为基于所述机械手与所述至少两个相机分别对应的手眼距离和所述至少两个相机分别与目标物体对应的位姿偏差信息进行平均补偿处理，得到所述目标物体的定位信息。
根据本申请的另一方面，提供了一种电子设备，所述电子设备包括：
至少一个处理器；以及
与所述至少一个处理器通信连接的存储器；其中，
所述存储器存储有可被所述至少一个处理器执行的计算机程序，所述计算机程序被所述至少一个处理器执行，以使所述至少一个处理器能够执行本申请任一实施例所述的目标定位方法。
根据本申请的另一方面，提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机指令，所述计算机指令用于使处理器执行时实现本申请任一实施例所述的目标定位方法。
附图说明
图1是根据本申请实施例一提供的一种目标定位方法的流程图；
图2是根据本申请实施例一提供的一种手眼距离确定示意图；
图3是根据本申请实施例二提供的一种目标定位方法的流程图；
图4是根据本申请实施例三提供的一种目标定位装置的结构示意图；
图5是实现本申请实施例的目标定位方法的电子设备的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行描述，所描述的实施例仅仅是本申请一部分的实施例。
本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。这样使用的数据在适当情况下可以互换，以便这里描述的本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步 骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。
实施例一
图1为本申请实施例一提供的一种目标定位方法的流程图，本实施例可适用于机械臂对所抓取的目标物体进行自动定位的情况，该方法可以由目标定位装置来执行，该目标定位装置可以采用硬件和/或软件的形式实现，该目标定位装置可配置于电子设备中。例如，电子设备可以是终端和/或服务器。如图1所示，该方法包括以下步骤。
S110、获取机械手与至少两个相机分别对应的手眼距离，以及获取所述至少两个相机分别对应的位姿信息和目标物体的位姿信息。
所述手眼距离为所述机械手与所述至少两个相机中的每一个之间的距离。相机可以固定设置在机械手的抓手上，机械手可以包括多个抓手，即机械手通过多个抓手完成对目标物体的抓取，并且相机可以跟随机械手移动。手眼距离指的是机械手的法兰中心到相机的视野中心的距离。当相机的数量为多个时，获取的手眼距离的数量也为多个。相机的位姿信息指的是相机拍摄图像中标准位置信息和标准姿态信息，目标物体的位姿信息指的是相机拍摄图像中目标物体的实际位置信息和实际姿态信息。
在上述多个实施例的基础上，获取机械手与至少两个相机分别对应的手眼距离，包括：获取机械手的旋转中心的位置信息；分别获取至少两个相机的标定位置信息，基于所述至少两个相机分别对应的标定位置信息和机械手的旋转中心的位置信息确定所述至少两个相机分别对应的手眼距离。
机械手旋转中心的位置信息指的是机械手法兰中心的位置坐标。标定位置信息指的是相机视野中心的位置坐标。
示例性的，建立以法兰为旋转中心的机械坐标系，并确定机械手法兰中心的位置坐标，例如，机械手法兰中心的位置坐标可以为(X 1,Y 1)；如图2所示，相机的数量可以为两个，如图2中左侧的虚线，第一相机视野中心的位置坐标到机械手法兰中心的位置坐标的距离可以为第一相机的手眼距离；如图2中右侧的虚线，第二相机视野中心的位置坐标到机械手法兰中心的位置坐标的距离可以为第二相机的手眼距离。
在一些实施例中，标定位置信息可以通过移动机械手并使用尖针扎取确定，其中，尖针扎取的位置可以为眼睛位置，可以将该眼睛位置确定为标定位置信息。
在一些实施例中，每个相机的位姿信息和目标物体的位姿信息可以通过相机拍摄图像得到。可以提取相机拍摄图像中目标物体的特征点，并将特征点的位置信息和姿态信息作为目标物体的位姿信息。其中，特征点可以是目标物体的标记点或者质心等，在此不做限定。相机的位姿信息可以是相机拍摄图像中参考点的位姿信息，其中，参考点可以是图像的标记点或中心点等，在此不做限定。
S120、基于所述至少两个相机分别对应的位姿信息和目标物体的位姿信息确定所述至少两个相机分别与目标物体对应的位姿偏差信息。
位姿偏差信息指的是相机拍摄图像中的标准位姿信息与实际目标物体的位姿信息的位置偏差值或者角度偏差值。
通过每个相机的位姿信息和目标物体的位姿信息确定每个相机与目标物体的位姿偏差信息，实现了对目标物体位姿偏移的检测，位姿偏差信息可用于弥补定位偏差，使目标物体在发生偏移的情况下，实现精准抓取。
S130、基于所述机械手与所述至少两个相机分别对应的手眼距离和所述至少两个相机分别与目标物体对应的位姿偏差信息进行平均补偿处理，得到所述目标物体的定位信息。
目标物体的定位信息指的是机械手所要抓取的目标物体的位置。平均补偿处理指的是对多个相机对应的手眼距离和位姿偏差信息进行相加并进行平均，使得到的目标物体的定位信息更加准确，提高了机械手的定位精度。
在上述多个实施例的基础上，在获取机械手与至少两个相机分别对应的手眼距离之前，所述方法还包括：获取机械手对应的实际坐标信息以及至少两个相机分别对应的像素坐标信息；基于所述机械手对应的实际坐标信息以及至少两个相机分别对应的像素坐标信息确定坐标转换关系。
实际坐标信息可以是机械手坐标信息。像素坐标信息指的是相机坐标信息。实际坐标信息与像素坐标信息属于两种不同的位置坐标系，两者无法直接运算，因此，可以建立机械手对应的实际坐标信息与每个相机对应的像素坐标信息之间的坐标转换关系，实现坐标系的转换。可以通过九点标定方法确定坐标转换关系。
在上述多个实施例的基础上，在基于所述机械手对应的实际坐标信息以及至少两个相机分别对应的像素坐标信息确定坐标转换关系之后，所述方法还包括：基于所述坐标转换关系，对所述至少两个相机分别对应的像素坐标信息进行转换，得到每个像素坐标信息对应的实际坐标信息。
坐标转换关系一经确立，可将坐标转换关系作为转换规则，在相机采集 到像素坐标信息之后，可以将至少两个相机分别对应的像素坐标信息进行转换，得到对应的实际坐标信息，实现多个参数坐标系的统一，以便多个参数之间进行运算。
本申请实施例的技术方案，通过获取机械手与至少两个相机的手眼距离，以及获取至少两个相机分别对应的位姿信息和目标物体的位姿信息，实现了手眼距离和多个相机信息的获取；基于至少两个相机分别对应的位姿信息和目标物体的位姿信息确定至少两个相机与目标物体的位姿偏差信息，实现了对目标物体位姿偏移的检测，基于至少两个相机与机械手的手眼距离和检测得到的位姿偏差信息进行平均补偿处理，使得到的目标物体的定位信息更加准确，解决了目前机械臂定位精度不足的问题。
实施例二
图3为本申请实施例二提供的一种目标定位方法的流程图，本实施例的目标定位方法与上述实施例中提供的目标定位方法中多个可选方案可以结合。本实施例提供的目标定位方法进行了说明。可选的，所述相机的位姿信息包括中心像素位置和标准拍照角度，所述目标物体的位姿信息包括特征像素位置和特征角度，所述基于所述至少两个相机分别对应的位姿信息和目标物体的位姿信息确定所述至少两个相机分别与目标物体的位姿偏差信息，包括：基于目标物体的特征像素位置和所述至少两个相机分别对应的中心像素位置，确定所述至少两个相机分别对应的位置偏差信息；基于目标物体的特征角度和所述至少两个相机分别对应的标准拍照角度，确定所述至少两个相机分别对应的角度偏差信息；所述基于所述机械手与所述至少两个相机分别对应的手眼距离和所述至少两个相机分别与目标物体的位姿偏差信息进行平均补偿处理，得到所述目标物体的定位信息，包括：基于所述机械手与所述至少两个相机分别对应的手眼距离、所述位置偏差信息和所述角度偏差信息进行平均补偿处理，得到目标物体的定位信息。
如图3所示，该方法包括以下步骤。
S210、获取机械手与至少两个相机分别对应的手眼距离，以及获取所述至少两个相机分别对应的中心像素位置和标准拍照角度与目标物体的特征像素位置和特征角度。
中心像素位置指的是相机拍摄图像的像素中心点，即标准位置信息。标准拍照角度指的是相机拍摄图像中目标物体的标准摆放角度，即标准姿态信息，例如水平摆放目标物体。特征像素位置指的是相机拍摄图像中目标物体 特征点的实际位置，即实际位置信息。特征角度指的是相机拍摄图像中目标物体特征点的实际角度，即实际姿态信息。
S220、基于目标物体的特征像素位置和所述至少两个相机分别对应的中心像素位置，确定所述至少两个相机分别对应的位置偏差信息。
示例性的，可以将目标物体的特征像素位置与至少两个相机分别对应的中心像素位置作差，得到至少两个相机分别对应的位置偏差信息。
S230、基于目标物体的特征角度和所述至少两个相机分别对应的标准拍照角度，确定所述至少两个相机分别对应的角度偏差信息。
示例性的，可以将目标物体的特征角度与至少两个相机分别对应的标准拍照角度作差，得到至少两个相机分别对应的角度偏差信息。
S240、基于所述机械手与所述至少两个相机分别对应的手眼距离、所述位置偏差信息和所述角度偏差信息进行平均补偿处理，得到目标物体的定位信息。
示例性的，相机可以包括第一相机和第二相机，基于至少两个相机与机械手的手眼距离、位置偏差信息和角度偏差信息进行平均补偿处理，得到目标物体的定位信息，包括：
Figure PCTCN2022142695-appb-000001
Figure PCTCN2022142695-appb-000002
Figure PCTCN2022142695-appb-000003
其中，X表示定位信息的水平方向坐标，Y表示定位信息的竖直方向坐标，A表示定位信息中的角度；(D x1,D y1)表示第一相机的手眼距离；(D x2,D y2)表示第二相机的手眼距离；(d x1,d y1)表示第一相机的位置偏差信息,(d x2,d y2)表示第二相机的位置偏差信息；d a1表示第一相机的角度偏差信息，d a2表示第二相机的角度偏差信息。通过上述运算，实现了双相机的目标物体的定位，并且将第一相机和第二相机的平均值作为目标物体的定位信息，使定位信息 更加准确，从而提高了定位信息的精度。
在一些实施例中，目标物体可以是大工件或产品，通过单个相机难以拍摄完整，通过设置多个相机，分别拍摄目标物体的局部图像，根据多个目标物体的局部图像对目标物体进行定位，在避免使用大分辨率相机的同时，也提高了对目标物体的定位精度。
本申请实施例的技术方案，通过获取机械手与至少两个相机的手眼距离，以及获取所述至少两个相机分别对应的中心像素位置和标准拍照角度和目标物体的特征像素位置和特征角度；基于目标物体的特征像素位置和至少两个相机分别对应的中心像素位置，确定至少两个相机分别对应的位置偏差信息，以及基于目标物体的特征角度和至少两个相机分别对应的标准拍照角度，确定至少两个相机分别对应的角度偏差信息，实现了对目标物体位置偏移和角度偏移的检测，基于至少两个相机与机械手的手眼距离、位置偏差信息和角度偏差信息进行平均补偿处理，使得到的目标物体的定位信息更加准确，解决了机械臂定位精度不足的问题。
实施例三
图4为本申请实施例三提供的一种目标定位装置的结构示意图。如图4所示，该装置包括以下模块。
信息获取模块310，设置为获取机械手与至少两个相机分别对应的手眼距离，以及获取所述至少两个相机分别对应的位姿信息和目标物体的位姿信息，其中，所述手眼距离为所述机械手与所述至少两个相机中的每一个之间的距离；
偏差信息确定模块320，设置为基于所述至少两个相机分别对应的位姿信息和目标物体的位姿信息确定所述至少两个相机分别与目标物体对应的位姿偏差信息；
补偿处理模块330，设置为基于所述机械手与所述至少两个相机分别对应的手眼距离和所述至少两个相机分别与目标物体对应的位姿偏差信息进行平均补偿处理，得到所述目标物体的定位信息。
本申请实施例的技术方案，通过获取机械手与至少两个相机分别对应的手眼距离，以及获取至少两个相机分别对应的位姿信息和目标物体的位姿信息，实现了手眼距离和多个相机信息的获取；基于至少两个相机分别对应的位姿信息和目标物体的位姿信息确定至少两个相机与目标物体的位姿偏差信 息，实现了对目标物体位姿偏移的检测，基于至少两个相机与机械手的手眼距离和检测得到的位姿偏差信息进行平均补偿处理，使得到的目标物体的定位信息更加准确，解决了机械臂定位精度不足的问题。
可选的，信息获取模块310，是设置为：
获取机械手的旋转中心的位置信息；
分别获取至少两个相机的标定位置信息，基于所述至少两个相机分别对应的标定位置信息和机械手的旋转中心的位置信息确定所述至少两个相机分别对应的手眼距离。
可选的，所述相机的位姿信息包括中心像素位置和标准拍照角度，所述目标物体的位姿信息包括特征像素位置和特征角度，偏差信息确定模块320，包括：
位置偏差信息确定单元，设置为基于目标物体的特征像素位置和至少两个相机分别对应的中心像素位置，确定至少两个相机分别对应的位置偏差信息；
角度偏差信息确定单元，设置为基于目标物体的特征角度和至少两个相机分别对应的标准拍照角度，确定至少两个相机分别对应的角度偏差信息；
补偿处理模块330，包括：
定位信息确定单元，设置为基于所述机械手与所述至少两个相机分别对应的手眼距离、所述位置偏差信息和所述角度偏差信息进行平均补偿处理，得到目标物体的定位信息。
可选的，所述位置偏差信息确定单元，是设置为：
将目标物体的特征像素位置与至少两个相机分别对应的中心像素位置作差，得到至少两个相机分别对应的位置偏差信息。
可选的，所述角度偏差信息确定单元，是设置为：
将目标物体的特征角度与至少两个相机分别对应的标准拍照角度作差，得到至少两个相机分别对应的角度偏差信息。
可选的，所述相机包括第一相机和第二相机，所述定位信息确定单元，是设置为：
Figure PCTCN2022142695-appb-000004
Figure PCTCN2022142695-appb-000005
Figure PCTCN2022142695-appb-000006
其中，X表示定位信息的水平方向坐标，Y表示定位信息的竖直方向坐标，A表示定位信息中的角度；(D x1,D y1)表示第一相机的手眼距离；(D x2,D y2)表示第二相机的手眼距离；(d x1,d y1)表示第一相机的位置偏差信息,(d x2,d y2)表示第二相机的位置偏差信息；d a1表示第一相机的角度偏差信息，d a2表示第二相机的角度偏差信息。
可选的，所述装置还包括：
坐标信息获取模块，设置为获取机械手对应的实际坐标信息以及至少两个相机分别对应的像素坐标信息；
转换关系确定模块，设置为基于所述机械手对应的实际坐标信息以及至少两个相机分别对应的像素坐标信息确定坐标转换关系；
像素坐标转换模块，设置为基于所述坐标转换关系，对所述至少两个相机分别对应的像素坐标信息进行转换，得到每个像素坐标信息对应的实际坐标信息。
本申请实施例所提供的目标定位装置可执行本申请任意实施例所提供的目标定位方法，具备执行方法相应的功能模块和效果。
实施例四
图5示出了可以用来实施本申请的实施例的电子设备10的结构示意图。电子设备10旨在表示多种形式的数字计算机，诸如，膝上型计算机、台式计算机、工作台、个人数字助理、服务器、刀片式服务器、大型计算机、和其它适合的计算机。电子设备还可以表示多种形式的移动装置，诸如，个人数字处理、蜂窝电话、智能电话、可穿戴设备(如头盔、眼镜、手表等)和其它类似的计算装置。本文所示的部件、它们的连接和关系、以及它们的功能仅仅作为示例，并且不意在限制本文中描述的和/或者要求的本申请的实现。
如图5所示，电子设备10包括至少一个处理器11，以及与至少一个处 理器11通信连接的存储器，如只读存储器(Read-Only Memory，ROM)12、随机访问存储器(Random Access Memory，RAM)13等，其中，存储器存储有可被至少一个处理器11执行的计算机程序，处理器11可以根据存储在ROM12中的计算机程序或者从存储单元18加载到RAM13中的计算机程序，来执行多种适当的动作和处理。在RAM 13中，还可存储电子设备10操作所需的多种程序和数据。处理器11、ROM 12以及RAM 13通过总线14彼此相连。输入/输出(Input/Output，I/O)接口15也连接至总线14。
电子设备10中的多个部件连接至I/O接口15，包括：输入单元16，例如键盘、鼠标等；输出单元17，例如多种类型的显示器、扬声器等；存储单元18，例如磁盘、光盘等；以及通信单元19，例如网卡、调制解调器、无线通信收发机等。通信单元19允许电子设备10通过诸如因特网的计算机网络和/或多种电信网络与其他设备交换信息/数据。
处理器11可以是多种具有处理和计算能力的通用和/或专用处理组件。处理器11的一些示例包括但不限于中央处理单元(Central Processing Unit，CPU)、图形处理单元(Graphics Processing Unit，GPU)、多种专用的人工智能(Artificial Intelligence，AI)计算芯片、多种运行机器学习模型算法的处理器、数字信号处理器(Digital Signal Processor，DSP)、以及任何适当的处理器、控制器、微控制器等。处理器11执行上文所描述的多个方法和处理，例如一种目标定位方法，包括：
获取机械手与至少两个相机分别对应的手眼距离，以及获取至少两个相机分别对应的位姿信息和目标物体的位姿信息；
基于至少两个相机分别对应的位姿信息和目标物体的位姿信息确定至少两个相机分别与目标物体对应的位姿偏差信息；
基于所述机械手与所述至少两个相机分别对应的手眼距离和所述至少两个相机分别与目标物体对应的位姿偏差信息进行平均补偿处理，得到所述目标物体的定位信息。
在一些实施例中，目标定位方法可被实现为计算机程序，其被有形地包含于计算机可读存储介质，例如存储单元18。在一些实施例中，计算机程序的部分或者全部可以经由ROM 12和/或通信单元19而被载入和/或安装到电子设备10上。当计算机程序加载到RAM 13并由处理器11执行时，可以执行上文描述的目标定位方法的一个或多个步骤。备选地，在其他实施例中，处理器11可以通过其他任何适当的方式(例如，借助于固件)而被配置为执行目标定位方法。
本文中以上描述的系统和技术的多种实施方式可以在数字电子电路系统、集成电路系统、现场可编程门阵列(Field Programmable Gate Array，FPGA)、专用集成电路(Application Specific Integrated Circuit，ASIC)、专用标准产品(Application Specific Standard Parts，ASSP)、芯片上系统的系统(System on Chip，SOC)、复杂可编程逻辑设备(Complex Programmable Logic Device，CPLD)、计算机硬件、固件、软件、和/或它们的组合中实现。这些多种实施方式可以包括：实施在一个或者多个计算机程序中，该一个或者多个计算机程序可在包括至少一个可编程处理器的可编程系统上执行和/或解释，该可编程处理器可以是专用或者通用可编程处理器，可以从存储系统、至少一个输入装置、和至少一个输出装置接收数据和指令，并且将数据和指令传输至该存储系统、该至少一个输入装置、和该至少一个输出装置。
用于实施本申请的方法的计算机程序可以采用一个或多个编程语言的任何组合来编写。这些计算机程序可以提供给通用计算机、专用计算机或其他可编程数据处理装置的处理器，使得计算机程序当由处理器执行时使流程图和/或框图中所规定的功能/操作被实施。计算机程序可以完全在机器上执行、部分地在机器上执行，作为独立软件包部分地在机器上执行且部分地在远程机器上执行或完全在远程机器或服务器上执行。
在本申请的上下文中，计算机可读存储介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的计算机程序。计算机可读存储介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。备选地，计算机可读存储介质可以是机器可读信号介质。机器可读存储介质包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、RAM、ROM、可擦除可编程只读存储器(Erasable Programmable Read-Only Memory，EPROM)、快闪存储器、光纤、便捷式紧凑盘只读存储器(Compact Disc Read Only Memory，CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。存储介质可以是非暂态(non-transitory)存储介质。
为了提供与用户的交互，可以在电子设备上实施此处描述的系统和技术，该电子设备具有：用于向用户显示信息的显示装置(例如，阴极射线管(Cathode Ray Tube，CRT)或者液晶显示器(Liquid Crystal Display，LCD)监视器)；以及键盘和指向装置(例如，鼠标或者轨迹球)，用户可以通过该键盘和该指向装置来将输入提供给电子设备。其它种类的装置还可以用于提供与用户的交互；例如，提供给用户的反馈可以是任何形式的传感反馈(例如，视觉反馈、听觉反馈、或者触觉反馈)；并且可以用任何形式(包括声 输入、语音输入或者、触觉输入)来接收来自用户的输入。
可以将此处描述的系统和技术实施在包括后台部件的计算系统(例如，作为数据服务器)、或者包括中间件部件的计算系统(例如，应用服务器)、或者包括前端部件的计算系统(例如，具有图形用户界面或者网络浏览器的用户计算机，用户可以通过该图形用户界面或者该网络浏览器来与此处描述的系统和技术的实施方式交互)、或者包括这种后台部件、中间件部件、或者前端部件的任何组合的计算系统中。可以通过任何形式或者介质的数字数据通信(例如，通信网络)来将系统的部件相互连接。通信网络的示例包括：局域网(Local Area Network，LAN)、广域网(Wide Area Network，WAN)、区块链网络和互联网。
计算系统可以包括客户端和服务器。客户端和服务器一般远离彼此并且通常通过通信网络进行交互。通过在相应的计算机上运行并且彼此具有客户端-服务器关系的计算机程序来产生客户端和服务器的关系。服务器可以是云服务器，又称为云计算服务器或云主机，是云计算服务体系中的一项主机产品，以解决了传统物理主机与虚拟专用服务器(Virtual Private Server，VPS)服务中，存在的管理难度大，业务扩展性弱的缺陷。
可以使用上面所示的多种形式的流程，重新排序、增加或删除步骤。例如，本申请中记载的多个步骤可以并行地执行也可以顺序地执行也可以不同的次序执行，只要能够实现本申请的技术方案所期望的结果，本文在此不进行限制。
上述实施方式，并不构成对本申请保护范围的限制。根据设计要求和其他因素，可以进行多种修改、组合、子组合和替代。

Claims (10)

  1. 一种目标定位方法，包括：
    获取机械手与至少两个相机分别对应的手眼距离，以及获取所述至少两个相机分别对应的位姿信息和目标物体的位姿信息，其中，所述手眼距离为所述机械手与所述至少两个相机中的每一个之间的距离；
    基于所述至少两个相机分别对应的位姿信息和所述目标物体的位姿信息确定所述至少两个相机分别与所述目标物体对应的位姿偏差信息；
    基于所述机械手与所述至少两个相机分别对应的手眼距离和所述至少两个相机分别与所述目标物体对应的位姿偏差信息进行平均补偿处理，得到所述目标物体的定位信息。
  2. 根据权利要求1所述的方法，其中，所述获取机械手与至少两个相机分别对应的手眼距离，包括：
    获取所述机械手的旋转中心的位置信息；
    分别获取所述至少两个相机的标定位置信息，基于所述至少两个相机分别对应的标定位置信息和所述机械手的旋转中心的位置信息确定所述至少两个相机分别对应的手眼距离。
  3. 根据权利要求1所述的方法，其中，所述至少两个相机分别对应的位姿信息包括中心像素位置和标准拍照角度，所述目标物体的位姿信息包括特征像素位置和特征角度，所述基于所述至少两个相机分别对应的位姿信息和所述目标物体的位姿信息确定所述至少两个相机分别与所述目标物体对应的位姿偏差信息，包括：
    基于所述目标物体的特征像素位置和所述至少两个相机分别对应的中心像素位置，确定所述至少两个相机分别对应的位置偏差信息；
    基于所述目标物体的特征角度和所述至少两个相机分别对应的标准拍照角度，确定所述至少两个相机分别对应的角度偏差信息；
    所述基于所述机械手与所述至少两个相机分别对应的手眼距离和所述至少两个相机分别与所述目标物体对应的位姿偏差信息进行平均补偿处理，得到所述目标物体的定位信息，包括：
    基于所述机械手与所述至少两个相机分别对应的手眼距离、所述至少两个相机分别与所述目标物体对应的位置偏差信息和所述至少两个相机分别与所述目标物体对应的角度偏差信息进行平均补偿处理，得到所述目标物体的定位信息。
  4. 根据权利要求3所述的方法，其中，所述基于所述目标物体的特征像素 位置和所述至少两个相机分别对应的中心像素位置，确定所述至少两个相机分别对应的位置偏差信息，包括：
    将所述目标物体的特征像素位置与所述至少两个相机分别对应的中心像素位置作差，得到所述至少两个相机分别对应的位置偏差信息。
  5. 根据权利要求3所述的方法，其中，所述基于所述目标物体的特征角度和所述至少两个相机分别对应的标准拍照角度，确定所述至少两个相机分别对应的角度偏差信息，包括：
    将所述目标物体的特征角度与所述至少两个相机分别对应的标准拍照角度作差，得到所述至少两个相机分别对应的角度偏差信息。
  6. 根据权利要求3所述的方法，其中，所述相机包括第一相机和第二相机，所述基于所述机械手与所述至少两个相机分别对应的手眼距离、所述至少两个相机分别与所述目标物体对应的位置偏差信息和所述至少两个相机分别与所述目标物体对应的角度偏差信息进行平均补偿处理，得到所述目标物体的定位信息，包括：
    Figure PCTCN2022142695-appb-100001
    Figure PCTCN2022142695-appb-100002
    Figure PCTCN2022142695-appb-100003
    其中，X表示所述定位信息的水平方向坐标，Y表示所述定位信息的竖直方向坐标，A表示所述定位信息中的角度；(D x1,D y1)表示所述第一相机的手眼距离；(D x2,D y2)表示所述第二相机的手眼距离；(d x1,d y1)表示所述第一相机的位置偏差信息,(d x2,d y2)表示所述第二相机的位置偏差信息；d a1表示所述第一相机的角度偏差信息，d a2表示所述第二相机的角度偏差信息。
  7. 根据权利要求1所述的方法，其中，在所述获取机械手与至少两个相机分别对应的手眼距离之前，所述方法还包括：
    获取所述机械手对应的实际坐标信息以及所述至少两个相机分别对应的像素坐标信息；
    基于所述机械手对应的实际坐标信息以及所述至少两个相机分别对应的像素坐标信息确定坐标转换关系；
    基于所述坐标转换关系，对所述至少两个相机分别对应的像素坐标信息进行转换，得到每个像素坐标信息对应的实际坐标信息。
  8. 一种目标定位装置，包括：
    信息获取模块，设置为获取机械手与至少两个相机分别对应的手眼距离，以及获取所述至少两个相机分别对应的位姿信息和目标物体的位姿信息，其中，所述手眼距离为所述机械手与所述至少两个相机中的每一个之间的距离；
    偏差信息确定模块，设置为基于所述至少两个相机分别对应的位姿信息和所述目标物体的位姿信息确定所述至少两个相机分别与所述目标物体对应的位姿偏差信息；
    补偿处理模块，设置为基于所述机械手与所述至少两个相机分别对应的手眼距离和所述至少两个相机分别与所述目标物体对应的位姿偏差信息进行平均补偿处理，得到所述目标物体的定位信息。
  9. 一种电子设备，包括：
    至少一个处理器；以及
    与所述至少一个处理器通信连接的存储器；其中，
    所述存储器存储有可被所述至少一个处理器执行的计算机程序，所述计算机程序被所述至少一个处理器执行，以使所述至少一个处理器能够执行权利要求1-7中任一项所述的目标定位方法。
  10. 一种计算机可读存储介质，其中，所述计算机可读存储介质存储有计算机指令，所述计算机指令用于使处理器执行时实现权利要求1-7中任一项所述的目标定位方法。
PCT/CN2022/142695 2022-04-27 2022-12-28 目标定位方法、装置、电子设备及存储介质 WO2023207186A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202210457393.3 2022-04-27
CN202210457393.3A CN114734444B (zh) 2022-04-27 2022-04-27 一种目标定位方法、装置、电子设备及存储介质

Publications (1)

Publication Number Publication Date
WO2023207186A1 true WO2023207186A1 (zh) 2023-11-02

Family

ID=82282860

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/142695 WO2023207186A1 (zh) 2022-04-27 2022-12-28 目标定位方法、装置、电子设备及存储介质

Country Status (2)

Country Link
CN (1) CN114734444B (zh)
WO (1) WO2023207186A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114734444B (zh) * 2022-04-27 2023-06-27 博众精工科技股份有限公司 一种目标定位方法、装置、电子设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108876855A (zh) * 2018-05-28 2018-11-23 哈尔滨工程大学 一种基于深度学习的海参检测和双目视觉定位方法
US20200230818A1 (en) * 2019-01-22 2020-07-23 Samsung Electronics Co., Ltd. Hand-eye calibration method and system
CN112258589A (zh) * 2020-11-16 2021-01-22 北京如影智能科技有限公司 一种手眼标定的方法及装置
CN113043334A (zh) * 2021-02-23 2021-06-29 上海埃奇机器人技术有限公司 一种基于机器人的光伏电池串定位方法
CN114734444A (zh) * 2022-04-27 2022-07-12 博众精工科技股份有限公司 一种目标定位方法、装置、电子设备及存储介质

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006003263A (ja) * 2004-06-18 2006-01-05 Hitachi Ltd 視覚情報処理装置および適用システム
CN106426172B (zh) * 2016-10-27 2019-04-16 深圳元启智能技术有限公司 一种工业机器人工具坐标系的标定方法与系统
JP6527178B2 (ja) * 2017-01-12 2019-06-05 ファナック株式会社 視覚センサのキャリブレーション装置、方法及びプログラム
CN107218930B (zh) * 2017-05-05 2020-01-03 山东大学 基于单目手眼系统的空间圆六维位置-姿态主动式测量方法
US10967507B2 (en) * 2018-05-02 2021-04-06 X Development Llc Positioning a robot sensor for object classification
CN111070199A (zh) * 2018-10-18 2020-04-28 杭州海康威视数字技术股份有限公司 一种手眼标定评估方法及机器人
CN109794938B (zh) * 2019-02-01 2021-08-06 南京航空航天大学 一种适用于曲面结构的机器人制孔误差补偿装置及其方法
CN110605714B (zh) * 2019-08-06 2021-08-03 华中科技大学 一种基于人眼凝视点的手眼协调抓取方法
CN111015655B (zh) * 2019-12-18 2022-02-22 深圳市优必选科技股份有限公司 机械臂抓取方法、装置、计算机可读存储介质及机器人
CN111300422B (zh) * 2020-03-17 2021-04-23 浙江大学 基于视觉图像的机器人工件抓取位姿误差补偿方法
CN112975973B (zh) * 2021-03-02 2022-07-26 中山大学 一种应用于柔性机器人的混合标定方法及装置
CN113084808B (zh) * 2021-04-02 2023-09-22 上海智能制造功能平台有限公司 一种基于单目视觉的移动机械臂2d平面抓取方法
CN114012731B (zh) * 2021-11-23 2023-03-31 深圳市如本科技有限公司 手眼标定方法、装置、计算机设备和存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108876855A (zh) * 2018-05-28 2018-11-23 哈尔滨工程大学 一种基于深度学习的海参检测和双目视觉定位方法
US20200230818A1 (en) * 2019-01-22 2020-07-23 Samsung Electronics Co., Ltd. Hand-eye calibration method and system
CN112258589A (zh) * 2020-11-16 2021-01-22 北京如影智能科技有限公司 一种手眼标定的方法及装置
CN113043334A (zh) * 2021-02-23 2021-06-29 上海埃奇机器人技术有限公司 一种基于机器人的光伏电池串定位方法
CN114734444A (zh) * 2022-04-27 2022-07-12 博众精工科技股份有限公司 一种目标定位方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN114734444A (zh) 2022-07-12
CN114734444B (zh) 2023-06-27

Similar Documents

Publication Publication Date Title
CN111754515B (zh) 堆叠物品的顺序抓取方法和装置
CN107767423B (zh) 一种基于双目视觉的机械臂目标定位抓取方法
WO2019114339A1 (zh) 机械臂运动的校正方法和装置
WO2020228643A1 (zh) 交互控制方法、装置、电子设备及存储介质
US9652660B2 (en) Control system, robot system, and control method
WO2023207186A1 (zh) 目标定位方法、装置、电子设备及存储介质
WO2020190166A1 (ru) Способ и система захвата объекта с помощью роботизированного устройства
WO2024016965A1 (zh) 坐标系校准方法、装置、机器人和存储介质
CN113997295B (zh) 机械臂的手眼标定方法、装置、电子设备及存储介质
CN115213896A (zh) 基于机械臂的物体抓取方法、系统、设备及存储介质
CN112518748B (zh) 面向运动物品的视觉机械臂自动抓取方法与系统
CN115781673A (zh) 一种零件抓取方法、装置、设备及介质
WO2022217667A1 (zh) 人体生理样本采集方法、装置、电子设备和存储介质
WO2023221963A1 (zh) 跟随误差确定方法、装置、设备及存储介质
WO2022021156A1 (zh) 用于机器人抓取三维物体的方法和装置
WO2024055470A1 (zh) 旋转中心的标定方法、装置、计算机设备和存储介质
CN116000966A (zh) 一种工件抓取方法、装置、设备和存储介质
CN116749198A (zh) 一种基于双目立体视觉引导机械臂抓取方法
US20220351495A1 (en) Method for matching image feature point, electronic device and storage medium
CN115383740A (zh) 一种基于双目视觉的机械臂目标物体抓取方法
CN115592666A (zh) 组件定位方法、组件定位装置、组件定位系统和机器人
CN115070757A (zh) 物体抓取方法、装置、机器人及存储介质
CN115049810A (zh) 固态激光雷达点云的上色方法、装置、设备及存储介质
CN117359642A (zh) 一种机械臂抓取零件方法、装置、设备及存储介质
CN115781665B (zh) 一种基于单目相机的机械臂控制方法、装置及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22939992

Country of ref document: EP

Kind code of ref document: A1