CN114445548A

CN114445548A - Method, device and system for determining three-dimensional model of object to be measured

Info

Publication number: CN114445548A
Application number: CN202011189488.9A
Authority: CN
Inventors: 贺银增; 陈颀潇
Original assignee: Siemens Ltd China
Current assignee: Siemens Ltd China
Priority date: 2020-10-30
Filing date: 2020-10-30
Publication date: 2022-05-06

Abstract

The embodiment of the invention discloses a method, a device and a system for determining a three-dimensional model of an object to be measured. The method comprises the following steps: based on adjusting the position of an end effector of the robot, enabling an elastic detection element arranged on the end effector to keep a predetermined contact force or contact torque moving on each surface of the object to be measured; determining profile data for each surface based on the amount of position adjustment of the end effector during movement of the elastic detection element over each surface; a three-dimensional model of the object to be measured is determined based on the profile data for each surface. The embodiment of the invention can determine the three-dimensional model of the object to be detected without adopting laser scanning equipment, saves the cost and is also suitable for large-size objects to be detected. Furthermore, the embodiment of the invention has no special requirement on the light sensitivity of the object to be measured.

Description

Method, device and system for determining three-dimensional model of object to be measured

Technical Field

The invention relates to the technical field of robots, in particular to a method, a device and a system for determining a three-dimensional model of an object to be measured.

Background

Three-dimensional (3D) models are polygonal representations of objects that are typically displayed with a computer or other video device. The displayed object may be a real-world entity or may be an imaginary object. Three-dimensional models are now used in a variety of different fields. They are used in the medical industry to make accurate models of organs; the film industry uses them for moving characters, objects, and real films; the video game industry uses them as a resource in computers and video games; they are used in the scientific field as accurate models of compounds; the construction industry uses them to display proposed building or landscape presentations; the engineering community uses them for designing new equipment, vehicles, structures, and other application areas.

Conventionally, a laser scanning apparatus is used to perform laser scanning on an object to be measured to capture a three-dimensional model of the object to be measured. However, laser scanning equipment is expensive, leading to cost problems. In addition, in laser scanning, the size of a scanning object needs to be limited to be smaller than a specified value, and thus the method cannot be applied to a large-sized object. Moreover, the laser scanning method has strict requirements on the surface of the object, and is not suitable for objects sensitive to light.

Disclosure of Invention

The invention mainly aims to provide a method, a device and a system for determining a three-dimensional model of an object to be measured.

The technical scheme of the embodiment of the invention is realized as follows:

a method of determining a three-dimensional model of an object to be measured, comprising:

based on adjusting the position of an end effector of the robot, enabling an elastic detection element arranged on the end effector to keep a predetermined contact force or contact torque moving on each surface of the object to be measured;

determining profile data for each surface based on the amount of position adjustment of the end effector during movement of the elastic detection element over each surface;

a three-dimensional model of the object to be measured is determined based on the profile data for each surface.

Therefore, the embodiment of the invention can determine the three-dimensional model of the object to be measured without adopting laser scanning equipment, saves the cost and is also suitable for large-size objects to be measured. Furthermore, the embodiment of the invention has no special requirement on the light sensitivity of the object to be measured.

In one embodiment, the elastic detecting element moving on each surface of the object to be measured with a predetermined contact force or contact torque includes:

the elastic detection element keeps a preset contact force or contact torque to move on each surface of the object to be detected line by line;

the elastic detecting element maintains a predetermined contact force or contact torque to move on each surface of the object to be measured line by line.

Therefore, the embodiment of the invention can have various moving modes on the surface of the object to be measured.

acquiring a three-dimensional model of an object to be detected shot by using a shooting component;

based on adjusting the position of an end effector of a robot, enabling an elastic detection element arranged on the end effector to keep a predetermined contact force or contact torque moving on a target surface area of the object to be measured;

determining profile data for the target surface area based on the amount of position adjustment of the end effector during movement of the elastic detection element over the target surface area;

adjusting the three-dimensional model based on the contour data of the target surface area.

Therefore, in the embodiment of the invention, the three-dimensional model of the object to be detected is shot by the shooting component, so that the three-dimensional model of the object to be detected can be quickly established. Furthermore, based on the position of the end effector of the robot being adjusted so that the elastic detection element mounted on the end effector maintains a predetermined contact force or contact torque on the target surface area during movement, the contour data of the target surface area can be determined based on the position adjustment amount of the end effector, and the corresponding area in the three-dimensional model of the object to be measured can be updated using the contour data of the target surface area, so that the contour accuracy of the target surface area as a key area can also be ensured.

In one embodiment, the elastic detection element maintaining a predetermined contact force or contact torque to move on the target surface area of the object to be measured includes:

the elastic detection element keeps a preset contact force or contact torque to move line by line on a target surface area of the object to be detected;

the elastic detection element maintains a predetermined contact force or contact torque to move column by column on the target surface area of the object to be measured.

It can be seen that embodiments of the present invention can have a variety of movement patterns on the target surface area.

In one embodiment, the target surface area comprises at least one of:

a key zone determined based on zone priority; a region having a difference from the contour feature of the peripheral region; a surface area determined based on a user instruction.

It can be seen that embodiments of the present invention can determine the target surface area in a variety of ways.

An apparatus for determining a three-dimensional model of an object to be measured, comprising:

a position adjusting module for enabling an elastic detecting element arranged on an end effector of the robot to maintain a predetermined contact force or contact torque to move on each surface of an object to be measured based on adjusting a position of the end effector;

a profile data determination module for determining profile data for each surface based on the amount of position adjustment of the end effector during movement of the elastic detection element over each surface;

and the three-dimensional model determining module is used for determining the three-dimensional model of the object to be measured based on the contour data of each surface.

Therefore, the embodiment of the invention can determine the three-dimensional model of the object to be detected without adopting laser scanning equipment, saves the cost and is also suitable for large-size objects to be detected. Furthermore, the embodiment of the invention has no special requirement on the light sensitivity of the object to be measured.

In one embodiment, the position adjustment module is configured to enable an elastic detection element disposed on the end effector to maintain a predetermined contact force or contact torque to move line by line on each surface of the object to be measured; or to enable elastic detection elements arranged on said end-effector to maintain a predetermined contact force or contact torque moving on each surface of the object to be measured, column by column.

the acquisition module is used for acquiring a three-dimensional model of the object to be detected shot by the shooting component;

a position adjusting module for enabling an elastic detection element arranged on an end effector of the robot to maintain a predetermined contact force or contact torque to move on a target surface area of the object to be measured based on adjusting a position of the end effector;

a contour data acquisition module for determining contour data of the target surface area based on a position adjustment amount of the end effector during movement of the elastic detection element in the target surface area;

a model adjustment module to adjust the three-dimensional model based on the contour data of the target surface area.

In one embodiment, the position adjustment module is configured to enable an elastic detection element disposed on the end effector to maintain a predetermined contact force or contact torque to move line by line on a target surface area of an object to be measured; or to enable elastic detection elements arranged on said end-effector to maintain a predetermined contact force or contact torque moving column by column over a target surface area of the object to be measured.

In one embodiment, the target surface area comprises at least one of:

A system for determining a three-dimensional model of an object to be measured, comprising:

a robot including an end effector;

a force and torque sensor mounted on the end effector;

an elastic detection element coupled with the force and moment sensor;

a controller for adjusting a position of the end effector based on detection values of the force and torque sensors, enabling the elastic detection element to maintain a predetermined contact force or contact torque to move on each surface of the object to be measured; determining profile data for each surface based on the amount of position adjustment of the end effector during movement of the elastic detection element over each surface; a three-dimensional model of the object to be measured is determined based on the profile data for each surface.

the shooting component is used for shooting a three-dimensional model of the object to be detected;

a robot comprising an end effector;

a force and torque sensor mounted on the end effector;

an elastic detection element coupled with the force and torque sensor;

a controller for adjusting a position of the end effector based on detection values of the force and torque sensors, enabling the elastic detection element to maintain a predetermined contact force or contact torque to move on a target surface area of the object to be measured; determining profile data for the target surface area based on the amount of position adjustment of the end effector during movement of the elastic detection element over the target surface area; adjusting the three-dimensional model based on the contour data of the target surface area.

Therefore, in the embodiment of the invention, the three-dimensional model of the object to be measured can be quickly established by shooting the three-dimensional model of the object to be measured by the shooting component. Furthermore, based on the position of the end effector of the robot being adjusted so that the elastic detection element mounted on the end effector maintains a predetermined contact force or contact torque on the target surface area during movement, the contour data of the target surface area can be determined based on the position adjustment amount of the end effector, and the corresponding area in the three-dimensional model of the object to be measured can be updated using the contour data of the target surface area, so that the contour accuracy of the target surface area as a key area can also be ensured.

In one embodiment, the camera assembly comprises:

at least one three-dimensional camera; or

At least two-dimensional cameras and a processor, wherein each two-dimensional camera is respectively arranged at a predetermined position on the periphery of the object to be measured; the processor is used for synthesizing at least two-dimensional images of an object to be detected shot by the at least two-dimensional cameras into a three-dimensional model of the object to be detected, wherein the depth of field adopted in the synthesis is the depth of field of any one two-dimensional image of the at least two-dimensional images; or

The system comprises at least one two-dimensional camera, at least one depth sensor and a processor, wherein the at least one two-dimensional camera and the at least one depth sensor are arranged at the same position; the processor is used for generating a three-dimensional model of the object to be measured by utilizing at least one depth of field provided by the at least one depth of field sensor and at least one two-dimensional picture of the object to be measured provided by the at least one two-dimensional camera.

It can be seen that the shooting assembly has multiple embodiments and is suitable for various application scenarios.

An apparatus for determining a three-dimensional model of an object to be measured, comprising: a memory; a processor; wherein the memory has stored therein an application executable by the processor for causing the processor to perform the method of determining a three-dimensional model of an object to be measured as described in any one of the above.

A computer-readable storage medium having stored thereon a computer program for implementing a method of determining a three-dimensional model of an object to be measured as described in any one of the preceding claims when being executed by a processor.

Drawings

FIG. 1 is a flow chart of a first exemplary method for determining a three-dimensional model of an object under test according to an embodiment of the present invention.

FIG. 2 is a block diagram of a first exemplary system for determining a three-dimensional model of an object under test according to an embodiment of the present invention.

FIG. 3 is a block diagram of a first exemplary apparatus for determining a three-dimensional model of an object under test according to an embodiment of the present invention.

FIG. 4 is a flowchart illustrating a second exemplary method for determining a three-dimensional model of an object under test according to an embodiment of the present invention.

FIG. 5 is a block diagram of a second exemplary system for determining a three-dimensional model of an object under test according to an embodiment of the present invention.

FIG. 6 is a block diagram of a second exemplary apparatus for determining a three-dimensional model of an object under test according to an embodiment of the present invention.

FIG. 7 is a flow chart illustrating force and torque control according to an embodiment of the present invention.

FIG. 8 is a block diagram of an apparatus for determining a three-dimensional model of an object to be measured with a memory-processor architecture in accordance with an embodiment of the present invention.

Wherein the reference numbers are as follows:

Detailed Description

In order to make the technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the detailed description and specific examples, while indicating the scope of the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention.

For simplicity and clarity of description, the invention will be described below by describing several representative embodiments. Numerous details of the embodiments are set forth to provide an understanding of the principles of the invention. It will be apparent, however, that the invention may be practiced without these specific details. Some embodiments are not described in detail, but rather are merely provided as frameworks, in order to avoid unnecessarily obscuring aspects of the invention. Hereinafter, "comprising" means "including but not limited to", "according to … …" means "at least according to … …, but not limited to … … only". In view of the language convention of chinese, the following description, when it does not specifically state the number of a component, means that the component may be one or more, or may be understood as at least one.

A Robot (Robot) is a machine device capable of automatically performing work. It can accept human command, run the program programmed in advance, and also can make a plan according to the principle made by artificial intelligence technology. The robot may include an industrial robot, an agricultural robot, a household robot, a medical robot, a service robot, a space robot, an underwater robot, a military robot, a rescue and relief robot, an educational and teaching robot, an entertainment robot, and the like. Industrial robots are multi-joint manipulators or multi-degree-of-freedom machine devices oriented to the industrial field, which can automatically perform work and realize various functions by means of self power and control capability.

In the embodiment of the invention, the position of the end effector of the robot is adjusted, so that the elastic detection element arranged on the end effector keeps a preset contact force or contact torque on the surface of the object to be measured in the moving process, thereby determining the profile data of the surface based on the position adjustment amount of the end effector, and determining the three-dimensional model of the object to be measured by using the profile data of each surface, so that a laser scanning device is not needed, the cost is saved, and the method is also suitable for large-size objects to be measured. Furthermore, the embodiments of the present invention do not have any special requirements for the light sensitivity of the object to be measured.

As shown in fig. 1, the method includes:

step 101: based on adjusting the position of the end effector of the robot, an elastic detection element arranged on the end effector is enabled to keep a predetermined contact force or contact torque moving on each surface of the object to be measured.

Here, the end effector means any tool having a certain function connected to the edge (joint) of the robot. This may include robotic grippers, robotic tool quick-change devices, robotic collision sensors, robotic rotary connectors, robotic pressure tools, compliant devices, robotic spray guns, robotic burr cleaning tools, robotic arc welding torches, robotic electric welding torches, and the like. A robot end-effector is generally considered to be a peripheral device of a robot, an attachment of a robot, a robot tool, an end-of-arm tool. The mechanical gripping type end effector used in the industrial robot is mostly of a double-finger claw type, and can be classified into a translation type and a rotation type according to the movement of a finger. The mechanical clamping method may be classified into an outer clamping type and an inner supporting type, and the mechanical clamping method may be classified into an electric (electromagnetic) type, a hydraulic type and a pneumatic type, and a combination thereof.

Here, a resilient probe element (e.g., a spring-loaded probe) on the end effector may be moved line by line on each surface of the object to be measured, maintaining a predetermined contact force or contact torque. Alternatively, the elastic detection element may also be moved column by column on each surface of the object to be measured, maintaining a predetermined contact force or contact torque.

Here, the predetermined contact force or contact torque may be manually specified in advance. When there are variations (e.g., protrusions or depressions) on the surface of the object to be measured, the position of the end effector needs to be adjusted accordingly in order for the elastic detecting element to maintain a predetermined contact force or contact torque to move on each surface of the object to be measured. Therefore, the position adjustment amount of the end effector can reflect the amount of change on the surface of the object to be measured.

Step 102: determining profile data for each surface based on the amount of position adjustment of the end effector during movement of the elastic probe element over each surface.

Based on the amount of positional adjustment of the end effector by the elastic detection elements during the movement of each surface, profile data for each surface can be determined. Specifically, the method comprises the following steps: the profile data may include: (1) whether the surface is raised or depressed; (2) the specific shape of the protrusion or depression; (3) the height of the protrusion or the depth of the depression when the protrusion or depression occurs on the surface, and the like.

Step 103: a three-dimensional model of the object to be measured is determined based on the profile data for each surface.

Here, the shape of each surface can be determined based on the profile data of each surface, and a three-dimensional model of the object to be measured can be determined from the shapes of the respective surfaces. In the process of determining the three-dimensional model, the overall contour information of the three-dimensional model can be determined by referring to the overall contour information of the three-dimensional model observed manually or by utilizing the moving process of the elastic detection element on the overall surface of the object to be detected.

As shown in fig. 2, a system 20 for determining a three-dimensional model of an object to be measured comprises:

a robot 21, the robot 21 including an end effector 22;

a force and torque sensor 23 mounted on the end effector 22;

an elastic detection element 24 coupled to the force and torque sensor 23;

a controller 25 for adjusting the position of the end effector 22 based on the detection values of the force and moment sensor 23, enabling the elastic detection element 24 to maintain a predetermined contact force or contact moment to move on each surface of the object to be measured; determining profile data for each surface based on the amount of position adjustment of the end effector 22 during movement of the elastic detection element 24 over each surface; a three-dimensional model of the object to be measured is determined based on the profile data for each surface.

During the execution of the motion control commands by the robot 21, the force and torque sensor 23 may acquire the contact force and/or the contact torque between the elastic detection element 24 and the work piece.

In one embodiment, the elastic detection element 24 is coupled to the force and torque sensor 23 via a first flange, and the force and torque sensor 23 is further coupled to the end effector 22 via a second flange. The controller 25 of the robot 21 may control the robot 21 such that the elastic detection element 24 on the end effector 22 of the robot 21 moves row by row or column by column on each surface of the object to be measured, wherein the force and torque sensor 23 detects a current value of a contact force or a current value of a contact torque between the elastic detection element 24 and the surface during the movement. The controller 25 controls the position of the end effector 22 of the robot in a feedback control manner based on the difference between the current value of the contact force or the current value of the contact torque and the predetermined contact force or the predetermined contact torque so that the elastic detection element 24 on the end effector 22 maintains the predetermined contact force or the predetermined contact torque in contact with the surface of the object to be measured as much as possible. When there is a change (e.g., a protrusion or a depression) on the surface of the object to be measured, the position of the end effector 22 needs to be adjusted accordingly in order to maintain the elastic detection element 24 at a predetermined contact force or contact torque. The amount of positional adjustment of end effector 22 may reflect the amount of change on the surface of the object to be measured. Accordingly, controller 25 may determine profile data for each surface based on the amount of positional adjustment of end effector 22 during movement of elastic detection element 24 over each surface, and determine a three-dimensional model of the object to be measured based on the profile data for each surface.

As shown in fig. 3, the apparatus 200 for determining a three-dimensional model of an object to be measured includes:

a position adjusting module 201 for enabling an elastic detecting element disposed on an end effector of the robot to maintain a predetermined contact force or contact torque to move on each surface of an object to be measured based on adjusting a position of the end effector;

a profile data determination module 202 for determining profile data for each surface based on the amount of position adjustment of the end effector during movement of the elastic detection element over each surface;

a three-dimensional model determining module 203 for determining a three-dimensional model of the object to be measured based on the profile data of each surface.

In one embodiment, the position adjusting module 201 is configured to enable an elastic detecting element disposed on the end effector to maintain a predetermined contact force or contact torque to move line by line on each surface of the object to be measured; or to enable elastic detection elements arranged on said end-effector to maintain a predetermined contact force or contact torque moving on each surface of the object to be measured, column by column.

As shown in fig. 4, the method includes:

step 301: and acquiring a three-dimensional model of the object to be detected shot by the shooting component.

Here, the photographing component may be implemented as:

(1) at least one three-dimensional camera:

in this embodiment, a three-dimensional model of the object to be measured can be imaged with the at least one three-dimensional camera.

(2) At least two-dimensional cameras and a processor, wherein each two-dimensional camera is respectively arranged at a predetermined position on the periphery of the object to be measured:

in this embodiment, the processor is configured to synthesize at least two-dimensional images of the object to be measured captured by the at least two-dimensional cameras into the three-dimensional model of the object to be measured, wherein a depth of field employed in the synthesis is a depth of field of any one of the at least two-dimensional images.

(3) The system comprises at least one two-dimensional camera, at least one depth sensor and a processor, wherein the at least one two-dimensional camera and the at least one depth sensor are arranged at the same position:

in this embodiment, the processor is configured to generate a three-dimensional model of the object to be measured using the at least one depth of field provided by the at least one depth of field sensor in combination with the at least one two-dimensional photograph of the object to be measured provided by the at least one two-dimensional camera.

Step 302: based on adjusting the position of an end effector of a robot, an elastic detection element arranged on the end effector is enabled to keep a predetermined contact force or contact torque moving on a target surface area of the object to be measured.

Here, the end effector means any tool having a certain function connected to the edge (joint) of the robot. This may include robotic grippers, robotic tool quick-change devices, robotic collision sensors, robotic rotary connectors, robotic pressure tools, compliant devices, robotic spray guns, robotic burr cleaning tools, robotic arc welding torches, robotic electric welding torches, and the like. A robot end-effector is generally considered to be a peripheral device of a robot, an attachment of a robot, a robot tool, an end-of-arm tool. The mechanical clamping type end effector used in the industrial robot is mostly of a double-finger claw type, and if the mechanical clamping type end effector is divided into a translation type and a rotation type according to the movement of a finger. The mechanical clamping method may be classified into an outer clamping type and an inner supporting type, and the mechanical clamping method may be classified into an electric (electromagnetic) type, a hydraulic type and a pneumatic type, and a combination thereof.

Here, an elastic detecting element (e.g., a spring-type probe) may move line by line on a target surface area of the object to be measured while maintaining a predetermined contact force or contact torque. Alternatively, the elastic detection element may also be moved column by column over the target surface area of the object to be measured, maintaining a predetermined contact force or contact torque.

The target surface area may be implemented as:

(1) key zone determined based on zone priority

Examples are: in an application of making an accurate model of an organ, a key organ with a high priority may be designated in advance as a key region.

(2) Regions having a difference from the contour feature of the peripheral region

Examples are: a characteristic region that is significantly different from the peripheral region in the three-dimensional model of the object to be measured photographed by the photographing component may be set as the target surface region.

(3) Surface area determined based on user instruction

Here, the region specified in the three-dimensional model of the object photographed by the photographing component may be determined as the target surface region based on a selection operation by the user.

While the above exemplary description describes a typical example of determining a target surface area, those skilled in the art will appreciate that this description is exemplary only and is not intended to limit the scope of embodiments of the present invention.

Here, the predetermined contact force or contact torque may be manually specified in advance. When there is a change in the surface of the target surface area (e.g., a bump or a depression), the position of the end effector needs to be adjusted accordingly in order for the elastic detection element to maintain a predetermined contact force or contact torque moving across the target surface area. Thus, the amount of position adjustment of the end effector may reflect the amount of change in the target surface area.

Step 303: determining profile data for the target surface area based on the amount of position adjustment of the end effector by the elastic detection element during movement of the target surface area.

Based on the amount of position adjustment of the end effector during movement of the elastic detection element over the target surface area, profile data for the target surface area may be determined. Specifically, the method comprises the following steps: the profile data may include: (1) whether the surface is raised or depressed; (2) the specific shape of the protrusion or depression; (3) the height of the protrusion or the depth of the depression when the protrusion or depression occurs on the surface, and the like.

Step 304: adjusting the three-dimensional model based on the contour data of the target surface area.

Here, the target surface area in the three-dimensional model is adjusted using the contour data of the target surface area, enabling fine analysis and modeling for the target surface area.

Therefore, in the embodiment of the invention, the three-dimensional model of the object to be detected is shot by the shooting component, so that the three-dimensional model of the object to be detected can be quickly established. Furthermore, based on the position of the end effector of the robot, the elastic detection element mounted on the end effector can determine the contour data of the target surface area based on the position adjustment amount of the end effector during the movement process of maintaining the predetermined contact force or contact torque on the target surface area, and the corresponding area in the three-dimensional model of the object to be measured is updated by using the contour data of the target surface area, so that the contour accuracy of the target surface area as a key area can also be ensured.

As shown in fig. 5, the system 30 for determining a three-dimensional model of an object to be measured comprises:

a photographing component 37 for photographing a three-dimensional model of an object to be measured;

a robot 31, the robot 31 including an end effector 32;

a force and torque sensor 33 mounted on the end effector 32;

an elastic detection element 34 coupled with the force and torque sensor 33;

a controller 35 for adjusting the position of the end effector 32 based on the detection value of the force and moment sensor 33, enabling the elastic detection element 34 to maintain a predetermined contact force or contact moment to move on a target surface area of the object to be measured; determining profile data for the target surface area based on the amount of positional adjustment of the end effector 32 by the elastic detection element 34 during movement of the target surface area; adjusting the three-dimensional model based on the contour data of the target surface area.

During the execution of the motion control commands by the robot 31, the force and torque sensor 33 may acquire the contact force and/or the contact torque between the elastic detection element 34 and the work piece.

In one embodiment, the camera assembly 37 includes: at least one three-dimensional camera; or, at least two-dimensional cameras and a processor, wherein each two-dimensional camera is respectively arranged at a predetermined position of the periphery of the object to be measured; the processor is used for synthesizing at least two-dimensional images of an object to be detected shot by the at least two-dimensional cameras into a three-dimensional model of the object to be detected, wherein the depth of field adopted in the synthesis is the depth of field of any one two-dimensional image of the at least two-dimensional images; or, at least one two-dimensional camera, at least one depth of field sensor and a processor, the at least one two-dimensional camera and the at least one depth of field sensor being mounted at the same location; the processor is used for generating a three-dimensional model of the object to be measured by utilizing at least one depth of field provided by the at least one depth of field sensor and at least one two-dimensional picture of the object to be measured provided by the at least one two-dimensional camera.

In one embodiment, the elastic detection element 34 is coupled to the force and torque sensor 33 via a first flange, and the force and torque sensor 33 is further coupled to the end effector 32 via a second flange.

The target surface area may be specified in the human machine interface of the controller 35. Alternatively, the controller 35 automatically determines a region having a difference from the contour feature of the peripheral region as the target surface region based on image analysis of the three-dimensional model provided by the photographing component 37. The controller 35 of the robot 31 may control the robot 31 such that the elastic detection element 34 on the end effector 32 moves row by row or column by column over the target surface area, wherein the force and torque sensor 33 detects a current value of a contact force or a current value of a contact torque between the elastic detection element 34 and the surface of the target surface area. The controller 35 controls the position of the robot's end-effector 32 in a feedback control manner based on the difference between the current value of the contact force or contact torque and the predetermined contact force or contact torque to try to maintain the predetermined contact force or contact torque on the end-effector 32 in contact with the target surface area. When there is a change in the target surface area (e.g., a bump or a depression), the position of the end effector 32 needs to be adjusted accordingly in order for the elastic detection element 34 to maintain a predetermined contact force or contact torque. Therefore, the position adjustment amount of the end effector 32 can reflect the amount of change on the surface of the object to be measured. Accordingly, the controller 35 may determine profile data of the target surface area based on the position adjustment amount of the end effector 32 during the movement of the elastic detection element 34 in the target surface area, and update the corresponding area in the three-dimensional model of the object to be measured based on the profile data of the target surface area.

As shown in fig. 6, the apparatus 400 includes:

an obtaining module 401, configured to obtain a three-dimensional model of an object to be detected captured by a capturing component;

a position adjusting module 402 for enabling an elastic detecting element arranged on an end effector of the robot to maintain a predetermined contact force or contact torque to move on a target surface area of the object to be measured based on adjusting a position of the end effector;

a contour data acquisition module 403, configured to determine contour data of the target surface area based on a position adjustment amount of the end effector during movement of the elastic detection element in the target surface area;

a model adjustment module 404 for adjusting the three-dimensional model based on the contour data of the target surface area.

In one embodiment, the position adjusting module 402 is configured to enable an elastic detecting element disposed on the end effector to maintain a predetermined contact force or contact torque to move line by line on a target surface area of an object to be measured; or to enable an elastic detection element arranged on the end effector to maintain a predetermined contact force or contact torque to move column by column on a target surface area of the object to be measured.

In one embodiment, the target surface area comprises at least one of: a key zone determined based on zone priority; a region having a difference from the contour feature of the peripheral region; a surface area determined based on user instructions, and so on.

With respect to the system architectures shown in fig. 2 and 5, in the embodiment of the present invention, the robot may be controlled based on the forces and/or moments detected by the force and moment sensors. FIG. 7 is a flow chart illustrating force and torque control according to an embodiment of the present invention.

As shown in fig. 7, a predetermined contact force and a desired Value of the contact torque (Value1) are input to the operator 701. Further, the actual Value (Value2) of the contact force and/or the contact torque between the end effector and the workpiece acquired by the force and torque sensor 705 (corresponding to the force and torque sensor 23 in fig. 2 and the force and torque sensor 33 in fig. 5) disposed at the end effector is inputted to the arithmetic unit 701. In the operator 701, a difference value between the expected value and the actual value is calculated, and the difference value is input to the PID adjusting module 702 to perform PID adjustment. The kinematics conversion module 703 performs a kinematics conversion (kinematics conversion) operation on the PID adjustment result output by the PID adjustment module 702. The position and orientation adjustment module 704 adjusts the position and orientation of the robot based on the operation result of the kinematics conversion module 703, thereby changing the contact force and/or the contact torque detected by the force and torque sensor 705.

FIG. 8 is a block diagram of an apparatus for determining a three-dimensional model of an object to be tested with a memory-architecture according to an embodiment of the present invention.

As shown in fig. 8, the apparatus 800 includes a processor 801 and a memory 802;

the memory 802 has stored therein an application executable by the processor 801 for causing the processor 801 to perform the first exemplary method 100 of determining a three-dimensional model of an object under test as shown in fig. 1 or the second exemplary method 300 of determining a three-dimensional model of an object under test as shown in fig. 3.

The memory 802 may be embodied as various storage media such as an Electrically Erasable Programmable Read Only Memory (EEPROM), a Flash memory (Flash memory), and a Programmable Read Only Memory (PROM). The processor 801 may be implemented to include one or more central processors or one or more field programmable gate arrays that integrate one or more central processor cores. In particular, the central processor or central processor core may be implemented as a CPU or MCU.

It should be noted that not all steps and modules in the above flows and structures are necessary, and some steps or modules may be omitted according to actual needs. The execution order of the steps is not fixed and can be adjusted as required. The division of each module is only for convenience of describing adopted functional division, and in actual implementation, one module may be divided into multiple modules, and the functions of multiple modules may also be implemented by the same module, and these modules may be located in the same device or in different devices.

The hardware modules in the various embodiments may be implemented mechanically or electronically. For example, a hardware module may comprise a specially designed non-volatile circuit or logic device (e.g., a special-purpose processor such as an FPGA or an ASIC) for performing certain operations. A hardware module may also include programmable logic devices or circuits (e.g., including a general-purpose processor or other programmable processor) that are temporarily configured by software to perform certain operations. The implementation of the hardware module in a mechanical manner, or in a dedicated permanent circuit, or in a temporarily configured circuit (e.g., configured by software), may be determined based on cost and time considerations.

The present invention also provides a machine-readable storage medium storing instructions for causing a machine to perform a method as described herein. Specifically, a system or an apparatus equipped with a storage medium on which a software program code that realizes the functions of any of the embodiments described above is stored may be provided, and a computer (or a CPU or MPU) of the system or the apparatus is caused to read out and execute the program code stored in the storage medium. Further, part or all of the actual operations may be performed by an operating system or the like operating on the computer by instructions based on the program code. The functions of any of the above-described embodiments may also be implemented by writing the program code read out from the storage medium to a memory provided in an expansion board inserted into the computer or to a memory provided in an expansion unit connected to the computer, and then causing a CPU or the like mounted on the expansion board or the expansion unit to perform part or all of the actual operations based on the instructions of the program code.

Examples of the storage medium for supplying the program code include floppy disks, hard disks, magneto-optical disks, optical disks (e.g., CD-ROMs, CD-R, CD-RWs, DVD-ROMs, DVD-RAMs, DVD-RWs, DVD + RWs), magnetic tapes, nonvolatile memory cards, and ROMs. Alternatively, the program code may be downloaded from a server computer or the cloud by a communication network.

The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method (100) of determining a three-dimensional model of an object to be measured, comprising:

-based on adjusting the position of an end-effector of the robot, enabling an elastic detection element arranged on said end-effector to move (101) on each surface of the object to be measured, maintaining a predetermined contact force or contact torque;

determining profile data (102) for each surface based on the amount of position adjustment of the end effector during movement of the elastic detection element over each surface;

a three-dimensional model (103) of the object to be measured is determined based on the contour data of each surface.

2. Method (100) for determining a three-dimensional model of an object to be measured according to claim 1, wherein the elastic detection element keeping a predetermined contact force or contact moment moving on each surface of the object to be measured comprises:

the elastic detection element keeps a preset contact force or contact torque to move on each surface of the object to be detected line by line; or

3. A method (300) of determining a three-dimensional model of an object to be measured, comprising:

acquiring a three-dimensional model (301) of an object to be detected shot by a shooting component;

based on adjusting the position of an end effector of a robot, enabling an elastic detection element arranged on the end effector to move (302) on a target surface area of the object to be measured keeping a predetermined contact force or contact torque;

determining profile data (303) of the target surface area based on the amount of position adjustment of the end effector by the elastic detection element during movement of the target surface area;

the three-dimensional model is adjusted (304) based on the contour data of the target surface area.

4. The method (300) for determining a three-dimensional model of an object to be measured according to claim 3,

the elastic detecting element maintaining a predetermined contact force or contact torque to move on a target surface area of an object to be measured includes:

the elastic detection element keeps a preset contact force or contact torque to move line by line on a target surface area of the object to be detected; or

5. A method (300) for determining a three-dimensional model of an object to be measured according to claim 3, characterized in that the target surface area comprises at least one of the following:

6. An apparatus (200) for determining a three-dimensional model of an object to be measured, comprising:

a position adjustment module (201) for enabling an elastic detection element arranged on an end effector of the robot to maintain a predetermined contact force or contact torque to move on each surface of an object to be measured, based on adjusting a position of the end effector;

a profile data determination module (202) for determining profile data for each surface based on the amount of position adjustment of the end effector during movement of the elastic detection element over each surface;

a three-dimensional model determination module (203) for determining a three-dimensional model of the object to be measured based on the contour data of each surface.

7. Apparatus (200) for determining a three-dimensional model of an object to be measured according to claim 6,

the position adjusting module (201) is used for enabling an elastic detection element arranged on the end effector to keep a preset contact force or contact torque to move on each surface of the object to be measured line by line; or to enable elastic detection elements arranged on said end-effector to maintain a predetermined contact force or contact torque moving on each surface of the object to be measured, column by column.

8. An apparatus (400) for determining a three-dimensional model of an object to be measured, comprising:

the acquisition module (401) is used for acquiring a three-dimensional model of the object to be detected shot by the shooting component;

a position adjustment module (402) for enabling an elastic detection element arranged on an end effector of an adjustment robot to maintain a predetermined contact force or contact torque to move on a target surface area of the object to be measured, based on adjusting a position of the end effector;

a contour data acquisition module (403) for determining contour data of the target surface area based on a position adjustment amount of the end effector during movement of the elastic detection element in the target surface area;

a model adjustment module (404) for adjusting the three-dimensional model based on the contour data of the target surface area.

9. Apparatus (400) for determining a three-dimensional model of an object to be measured according to claim 8,

the position adjusting module (402) is used for enabling an elastic detection element arranged on the end effector to keep a preset contact force or contact torque to move line by line on a target surface area of an object to be measured; or to enable an elastic detection element arranged on the end effector to maintain a predetermined contact force or contact torque to move column by column on a target surface area of the object to be measured.

10. The apparatus (400) for determining a three-dimensional model of an object to be measured according to claim 8, wherein the target surface area comprises at least one of:

11. A system (20) for determining a three-dimensional model of an object to be measured, comprising:

a robot (21), the robot (21) comprising an end effector (22);

a force and torque sensor (23) mounted on the end effector (22);

an elastic detection element (24) coupled with the force and torque sensor (23);

a controller (25) for adjusting the position of the end effector (22) based on the detection value of the force and torque sensor (23) so that the elastic detection element (24) maintains a predetermined contact force or contact torque to move on each surface of the object to be measured; determining profile data for each surface based on the amount of positional adjustment of the end effector (22) by the elastic detection element (24) during movement of each surface; a three-dimensional model of the object to be measured is determined based on the profile data for each surface.

12. A system (30) for determining a three-dimensional model of an object to be measured, comprising:

a photographing component (37) for photographing a three-dimensional model of an object to be measured;

a robot (31), the robot (31) comprising an end effector (32);

a force and torque sensor (33) mounted on the end effector (32);

an elastic detection element (34) coupled with the force and torque sensor (33);

a controller (35) for adjusting the position of the end effector (32) based on the detection value of the force and torque sensor (33) so that the elastic detection element (34) maintains a predetermined contact force or contact torque to move on the target surface area of the object to be measured; determining profile data for the target surface area based on a position adjustment of the end effector (32) during movement of the elastic detection element (34) over the target surface area; adjusting the three-dimensional model based on the contour data of the target surface area.

13. System (30) for determining a three-dimensional model of an object to be measured according to claim 12,

the photographing assembly (37) includes:

at least one three-dimensional camera; or

14. An apparatus (800) for determining a three-dimensional model of an object to be measured, comprising: a memory (801); a processor (802); wherein the memory (801) has stored therein an application executable by the processor (802) for causing the processor (802) to perform the method (100) of determining a three-dimensional model of an object under test according to any one of claims 1 to 2 or the method (300) of determining a three-dimensional model of an object under test according to any one of claims 3 to 5.

15. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, implements the method (100) of determining a three-dimensional model of an object to be measured according to any one of claims 1 to 2 or the method (300) of determining a three-dimensional model of an object to be measured according to any one of claims 3 to 5.