CN118114524A - Three-dimensional contact force field measuring method of visual touch sensor based on inverse finite element analysis - Google Patents
Three-dimensional contact force field measuring method of visual touch sensor based on inverse finite element analysis Download PDFInfo
- Publication number
- CN118114524A CN118114524A CN202410277133.7A CN202410277133A CN118114524A CN 118114524 A CN118114524 A CN 118114524A CN 202410277133 A CN202410277133 A CN 202410277133A CN 118114524 A CN118114524 A CN 118114524A
- Authority
- CN
- China
- Prior art keywords
- dimensional
- contact force
- force field
- finite element
- field
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 59
- 230000000007 visual effect Effects 0.000 title claims abstract description 57
- 238000004458 analytical method Methods 0.000 title claims abstract description 44
- 238000005259 measurement Methods 0.000 claims abstract description 26
- 230000006870 function Effects 0.000 claims abstract description 20
- 238000013507 mapping Methods 0.000 claims abstract description 18
- 239000011159 matrix material Substances 0.000 claims abstract description 16
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 10
- 238000012546 transfer Methods 0.000 claims abstract description 4
- 238000006073 displacement reaction Methods 0.000 claims description 47
- VYPSYNLAJGMNEJ-UHFFFAOYSA-N Silicium dioxide Chemical compound O=[Si]=O VYPSYNLAJGMNEJ-UHFFFAOYSA-N 0.000 claims description 42
- 239000000741 silica gel Substances 0.000 claims description 38
- 229910002027 silica gel Inorganic materials 0.000 claims description 38
- 230000015654 memory Effects 0.000 claims description 15
- 238000004590 computer program Methods 0.000 claims description 14
- 238000001514 detection method Methods 0.000 claims description 11
- NIXOWILDQLNWCW-UHFFFAOYSA-N acrylic acid group Chemical group C(C=C)(=O)O NIXOWILDQLNWCW-UHFFFAOYSA-N 0.000 claims description 10
- 238000003860 storage Methods 0.000 claims description 7
- 239000000463 material Substances 0.000 claims description 6
- 238000000691 measurement method Methods 0.000 claims description 5
- 230000009466 transformation Effects 0.000 claims description 5
- 230000008878 coupling Effects 0.000 claims description 4
- 238000010168 coupling process Methods 0.000 claims description 4
- 238000005859 coupling reaction Methods 0.000 claims description 4
- 230000006399 behavior Effects 0.000 claims description 2
- 238000001914 filtration Methods 0.000 claims description 2
- 239000013598 vector Substances 0.000 description 12
- 238000004364 calculation method Methods 0.000 description 7
- 230000008569 process Effects 0.000 description 7
- 238000005286 illumination Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 4
- 238000009826 distribution Methods 0.000 description 4
- 239000003550 marker Substances 0.000 description 4
- 230000003993 interaction Effects 0.000 description 3
- 238000004519 manufacturing process Methods 0.000 description 3
- 230000008447 perception Effects 0.000 description 3
- NAWXUBYGYWOOIX-SFHVURJKSA-N (2s)-2-[[4-[2-(2,4-diaminoquinazolin-6-yl)ethyl]benzoyl]amino]-4-methylidenepentanedioic acid Chemical compound C1=CC2=NC(N)=NC(N)=C2C=C1CCC1=CC=C(C(=O)N[C@@H](CC(=C)C(O)=O)C(O)=O)C=C1 NAWXUBYGYWOOIX-SFHVURJKSA-N 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 208000017604 Hodgkin disease Diseases 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- -1 area is uneven Chemical compound 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 230000008094 contradictory effect Effects 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000005489 elastic deformation Effects 0.000 description 1
- 229920001971 elastomer Polymers 0.000 description 1
- 239000000806 elastomer Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 230000002427 irreversible effect Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 239000007769 metal material Substances 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 238000013433 optimization analysis Methods 0.000 description 1
- 239000002861 polymer material Substances 0.000 description 1
- 229920001296 polysiloxane Polymers 0.000 description 1
- 238000002310 reflectometry Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012916 structural analysis Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000001356 surgical procedure Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F30/00—Computer-aided design [CAD]
- G06F30/20—Design optimisation, verification or simulation
- G06F30/23—Design optimisation, verification or simulation using finite element methods [FEM] or finite difference methods [FDM]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2119/00—Details relating to the type or aim of the analysis or the optimisation
- G06F2119/14—Force analysis or force optimisation, e.g. static or dynamic forces
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Hardware Design (AREA)
- Evolutionary Computation (AREA)
- Geometry (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Force Measurement Appropriate To Specific Purposes (AREA)
Abstract
The invention discloses a three-dimensional contact force field measuring method of a visual touch sensor based on inverse finite element analysis, which relates to the technical field of force measurement of sensors and comprises the following steps of: modeling a sensing link of a three-dimensional contact force field of the visual touch sensor, and obtaining a simplified physical model and boundary conditions; performing structural discretization on the visual touch sensing unit based on a high-performance model and parameter combination; constructing a forward mapping relation between a three-dimensional contact force field and a three-dimensional deformation field of the flexible sensing layer according to a linear finite element theory; comprehensively analyzing to obtain an overall rigidity matrix; acquiring a three-dimensional deformation field of the surface of the flexible sensing layer based on a photometric stereo method and an image tracking algorithm; and acquiring transfer functions from the three-dimensional deformation field to the three-dimensional contact force field through multi-layer inverse finite element analysis, and acquiring the three-dimensional contact force field according to the three-dimensional deformation field and the overall rigidity matrix which are obtained in advance.
Description
Technical Field
The invention relates to the technical field of force measurement of sensors, in particular to a three-dimensional contact force field measurement method of a visual touch sensor based on inverse finite element analysis.
Background
Robots are increasingly assuming complex and elaborate operational tasks such as precision part assembly, assistance to surgeons surgery, and on-orbit space services. For these operational tasks, open loop or vision-guided robotic control is not accurate enough. Force sensing and tactile perception are vital to accurate control and environment interaction of the robot, wherein accurate and real-time three-dimensional contact force field measurement provides an important reference for closed loop feedback control of the robot, so that the accuracy and robustness of robot operation are remarkably improved, and the method has a wide application prospect.
However, the complete measurement of three-dimensional contact force fields, including tangential forces, still presents significant challenges. The touch sensor adopting the voltage or electromagnetic principle can only measure resultant force or normal pressure field, but distributed tangential force plays an important role in judging friction, slippage and other conditions between the sensor and a contacted object, and under the condition that the tangential force is unknown, the sensor cannot completely reflect the contact relation between the sensor and the object, and although in recent years, along with continuous improvement of a flexible electronic manufacturing process, a learner embeds a uniquely designed three-dimensional structure in flexible electronic skin, so that measurement of the three-dimensional contact force field is primarily realized, the flexible electronic touch sensor has the advantages of complex manufacturing process, lower spatial resolution and high cost, and has great difficulty in inputting practical application in industry.
Compared with a flexible electronic touch sensor, the visual touch sensor has advantages in the aspects of manufacturing process, resolution and the like, and consists of a flexible sensing layer and an embedded camera. When the silica gel contacts with an object and deforms, the built-in camera captures an image, and a deformation field of the silica gel is reconstructed by utilizing a stereoscopic vision theory and an image processing algorithm. Such sensors have high spatial resolution and multi-modal sensing capabilities; however, the use of such a sensor to detect three-dimensional contact force is still in the start phase, and there are the following problems:
1. The mapping relation between the deformation field information of the sensor and the contact force is fitted GelSight by using a machine learning method through manual annotation data, but the measured three-dimensional resultant force has larger noise and is influenced by the material of the contact object.
2. A contact force and torque estimation method using natural helmholtz-hodgkin's decomposition (nHHD) was developed based on the visual tactile sensor FingerVision, which establishes a mapping from the decomposed components of nHHD to the contact force and torque estimates, but the estimation of the contact force is limited to the resultant force level and cannot be finely measured on the three-dimensional contact force field.
3. Based on the labeling data provided by machine learning and finite element simulation, a method for estimating the contact force distribution is provided, but the experimental result is not ideal enough. Thus, there is still a need for further research and innovation in achieving fine measurements of three-dimensional contact force fields.
In view of the above, the invention provides a three-dimensional contact force field measuring method of a visual touch sensor based on inverse finite element analysis, so as to improve the sensing and perception capabilities of a robot.
Disclosure of Invention
The invention aims to provide a three-dimensional contact force field measuring method of a visual touch sensor based on inverse finite element analysis, which aims to solve the defects in the background technology.
According to one aspect of the present invention, there is provided a method for measuring a three-dimensional contact force field of a visual tactile sensor based on inverse finite element analysis, comprising the steps of:
step S1: modeling a sensing link of a three-dimensional contact force field of the visual touch sensor, and obtaining a simplified physical model and boundary conditions;
step S2: performing structural discretization on the visual touch sensing unit based on a high-performance model and parameter combination;
Step S3: constructing a forward mapping relation between a three-dimensional contact force field and a three-dimensional deformation field of the flexible sensing layer according to a linear finite element theory;
Step S4: acquiring an overall stiffness matrix based on comprehensive analysis of the steps S1-S3;
Step S5: acquiring a three-dimensional deformation field of the surface of the flexible sensing layer based on a photometric stereo method and an image tracking algorithm;
Step S6: and acquiring transfer functions from the three-dimensional deformation field to the three-dimensional contact force field through multi-layer inverse finite element analysis, and acquiring the three-dimensional contact force field according to the three-dimensional deformation field and the overall rigidity matrix which are obtained in advance.
As a preferred technical solution of the present invention, the acquisition logic of the physical model and the boundary conditions is simplified:
Capturing a tactile image of the contact surface in real time based on the visual tactile sensor; extracting a central rectangular region according to preset conditions; the preset condition is a=1.5r; wherein: a represents the side length of the central rectangular area, and R represents the radius of the flexible sensing layer;
The bottom surface of the flexible sensing layer is tightly attached to the acrylic prism, and the acrylic prism is used as a fixed support of the flexible sensing layer;
When the flexible sensing layer is in contact with an external object, a visual and tactile image can be obtained, and a three-dimensional deformation field of the surface of the sensing unit is obtained through a photometric stereo method; the force boundary condition of the intermediate layer of the sensor unit is zero.
As a preferred embodiment of the present invention, the high-performance model and parameter combination analysis includes: the constitutive model discretizes the unit type and unit size used by the flexible sensing layer;
the constitutive model is based on a flexible sensing layer made of flexible materials, performs mechanical analysis on the contact behavior of the flexible sensing layer and external objects based on a linear elastic model,
The unit type is that the flexible sensing layer is discretized by using six-node octahedral grid units;
and (3) under the premise of ensuring that the unit order is unchanged, the unit size is ensured, the result is approximated to the correct solution by gradually encrypting the grid, and the thickness and the in-plane direction unit size are determined.
As a preferable technical scheme of the invention, a mapping relation between a contact force field and a deformation field of silica gel is established;
establishing basic equations of three-dimensional contact force field measurement based on linear finite element theory, wherein the basic equations comprise balance equations, geometric equations, physical equations, stress boundary conditions and displacement boundary conditions based on elastic mechanics principle;
And constructing a forward mapping relation between the three-dimensional contact force field and the three-dimensional deformation field of the flexible sensing layer, and representing the unit displacement, strain and stress information by using the node displacement.
As a preferable technical scheme of the invention, an overall rigidity matrix is established as a measurement basis of three-dimensional contact force;
And obtaining the unit stress through a basic equation, selecting an isoparametric unit mode to determine the coordinate transformation and displacement mode of the three-dimensional contact force field, and determining and obtaining the integral rigidity matrix through a mapping relation.
As a preferable technical scheme of the invention, the visual touch image comprises depth information of an object and position information of a mark point on the surface of the sensing unit; characterizing normal deformation of the surface of the sensing unit based on depth information of the object; characterizing tangential deformation of the surface of the sensing unit based on the position information of the marking points on the surface of the sensing unit; and coupling the normal deformation and the tangential deformation to obtain a three-dimensional deformation field on the surface of the sensing unit.
As a preferable technical scheme of the invention, the logic for acquiring the position information of the mark points on the surface of the sensing unit:
acquiring an initial image frame of an initial state of the surface of a sensing unit;
the initial background image is obtained after the initial image frame is processed by low-pass Gaussian filtering;
acquiring a sensing unit image frame when the sensor contacts the cylinder, subtracting an initial background image from the sensing unit image frame to obtain a detection contact area, and marking the detection contact area as a detection contact image;
The mark mask after thresholding the detected contact image detects the position of each mark in the contact area, thereby acquiring the horizontal deformation information of the sensing unit.
As a preferable technical scheme of the invention, the detection contact area is a central area of the silica gel pad, wherein the touch image is captured by a camera;
Modeling the detection contact area based on a linear finite element theory, and calculating displacement and stress strain conditions of the silica gel in the central area of the silica gel pad and the force boundary condition;
The three-dimensional contact force field condition of the contact area is reversely deduced through the displacement and force boundary condition of the central area of the silica gel pad and the displacement and stress strain condition of the interior of the silica gel.
According to still another aspect of the present invention, there is provided an electronic apparatus including: a processor and a memory, wherein the memory stores a computer program for the processor to call;
The processor executes the three-dimensional contact force field measuring method of the visual touch sensor based on inverse finite element analysis by calling a computer program stored in the memory.
According to a further aspect of the present invention there is provided a computer readable storage medium storing instructions that when run on a computer cause the computer to perform the method of three-dimensional contact force field measurement of a visual tactile sensor based on inverse finite element analysis described above.
In the technical scheme, the invention has the technical effects and advantages that:
Firstly, modeling a sensing link of a three-dimensional contact force field based on the structural design and working principle of a visual touch sensor; reconstructing a three-dimensional deformation field of the surface of the flexible sensing layer based on a photometric stereo method and an image tracking algorithm; and finally, establishing a mapping relation from the three-dimensional deformation field to the three-dimensional contact force field by adopting a multi-layer inverse finite element method. In addition, the invention systematically analyzes the influence of the size and the type of the constitutive model and the finite element of the flexible sensing layer of the visual touch sensor on the reconstruction result of the three-dimensional contact force field, so as to obtain a high-performance model and parameter combination and increase the accuracy of the measurement result of the three-dimensional contact force field; the method provides reliable feedback for the three-dimensional contact force field for the visual touch sensor, and lays a solid foundation for intelligent perception and operation of the robot.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments described in the present application, and other drawings may be obtained according to these drawings for a person having ordinary skill in the art.
FIG. 1 is a flow chart of measurement of three-dimensional contact force field of a visual touch sensor based on multilayer inverse finite element optimization analysis in the method of the invention;
FIG. 2 is a simplified physical model of a three-dimensional touch force field measurement of a tactile sensor in accordance with the method of the present invention;
FIG. 3 is a grid discretization schematic of a flexible sensing layer of the present invention;
FIG. 4 is a depth map of threads reconstructed based on photometric stereo method according to the present invention;
FIG. 5 is an exemplary diagram of a process for deriving a tangential displacement of a marker from a tactile image in accordance with the present invention;
FIG. 6 is a graph of the measurement results of the three-dimensional contact force field of the present invention;
Fig. 7 is a graph comparing the measurement results of three-dimensional contact force measured by the ATI force sensor according to the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Embodiment 1, please refer to fig. 1, the three-dimensional touch force field measuring method of the visual touch sensor based on inverse finite element analysis according to the embodiment comprises the following steps;
step S1: modeling a sensing link of a three-dimensional contact force field of the visual touch sensor, and obtaining a simplified physical model and boundary conditions;
It should be noted that: the visual touch sensor comprises a flexible sensing layer and an acrylic prism for fixing the flexible sensing layer; the flexible sensing layer is provided with uniformly arranged mark points, as shown in fig. 2.
Simplified physical model and boundary condition acquisition logic:
Capturing a tactile image of the contact surface in real time based on the visual tactile sensor; extracting a central rectangular region according to preset conditions; the preset condition is a=1.5r; wherein: a represents the side length of the central rectangular area, and R represents the radius of the flexible sensing layer;
the purpose of the preset condition is to ensure that the number of marker points in the central rectangular region is sufficiently large for inverse finite element analysis.
The bottom surface of the flexible sensing layer is tightly attached to an acrylic prism for providing an optical path of the visual touch sensor, and the rigidity of the flexible sensing layer is far smaller than that of the acrylic prism, so that the acrylic prism can be simplified into a fixed support capable of providing the flexible sensing layer, namely
In order to obtain a three-dimensional deformation field of the surface of the sensing unit, the embodiment adopts a method of combining a photometric stereo method and an image tracking algorithm;
Specifically, when the flexible sensing layer is in contact with an external object, a camera built in the visual touch sensor can capture visual touch images, and the images comprise depth information of the object and position information of marking points on the surface of the sensing unit; depth information of the object, namely normal deformation of the surface of the sensing unit, can be obtained through a photometric stereo method; the horizontal position information of the mark point, namely tangential deformation of the surface of the sensing unit, can be obtained through an image tracking algorithm; coupling the normal deformation and the tangential deformation to obtain a three-dimensional deformation field of the surface of the sensing unit, namely
No external force can be applied inside the sensing unit, so that the force boundary condition of the intermediate layer is zero, namely
It should be noted that: the touch image captured by the camera arranged in the visual touch sensor is mainly concentrated in the central part of the silica gel pad, and the illumination distribution of the silica gel edge area is uneven, and a large measurement error exists, so that the three-dimensional touch force field can be reconstructed only aiming at the central rectangular area captured by the camera when the inverse finite element is used; and extracting the deformation of the central rectangular area.
Step S2: performing structural discretization on the visual touch sensing unit based on a high-performance model and parameter combination;
The invention performs specific parameter analysis on the flexible sensing layer of the visual touch sensor to obtain a high-performance model and parameter combination, and increases the accuracy of three-dimensional contact force field measurement. Specific parameter analysis includes: the model is composed of a unit type, a unit layer number and an in-plane size used by the discrete flexible sensing layer.
The constitutive model: the sensing unit of the visual touch sensor is a piece of flexible silica gel, and the mechanical properties of the high polymer material are greatly different from those of metal materials, such as large elastic deformation, incompressibility and the like. The modeling analysis is carried out on the silica gel material by selecting a proper model, which is the basis for accurately measuring the three-dimensional contact force field of the silica gel surface and is also an important premise for acquiring external information when the sensor interacts with the outside.
Cell type and size: one of the core problems of the finite element method is to perform reasonable discretization on the system, and in theory, when the number of nodes tends to infinity (i.e. the cell size tends to zero) or the order of the cell interpolation displacement tends to infinity, the solution of the finite element can approach the true solution infinitely. But due to the operating principle of the visual tactile sensor: the requirements of the photometric stereo method used in normal displacement measurement and the tracking algorithm used in horizontal displacement measurement on the duty ratio of the mark point are contradictory, and the measurement accuracy of the normal displacement is reduced due to undersize of the unit. In addition, the convergence rate and the calculation efficiency are greatly reduced due to the excessively dense grids and the excessively high cell orders, so that the accuracy of normal displacement measurement, tangential displacement measurement and inverse finite element reconstruction of a contact force field is considered, and proper cell types and sizes are selected.
(1) High performance model and parameter combination acquisition
The flexible sensing layer is made of flexible materials such as silica gel, so that a super-elastic model is often used for mechanical analysis of the flexible sensing layer. But by structural analysis of gelslim3.0 it was found that: the bottom of the silica gel pad corresponding to the flexible sensing layer is tightly adhered with the acrylic prism. Because the hardness of acrylic is far greater than that of silica gel, the deformation of the silica gel is limited, and the maximum deformation degree of the silica gel cannot exceed 30% of the thickness of the silica gel. Meanwhile, the working principle of the visual touch sensor determines that the visual touch sensor has the characteristic of high sensitivity, and the camera can better capture the touch image under the condition that the sensing unit only needs to slightly deform (within about 10 percent) under normal conditions. For the consideration of calculation efficiency, a linear elasticity assumption can be used here to establish a mapping relationship between the contact force field and the deformation field of the silica gel.
In terms of the space finite element problem, the cell types currently in common use include both tetrahedrons and hexahedrons. At the same order, the accuracy of the hexahedral mesh is theoretically higher than that of the tetrahedral mesh. Taking a first-order unit as an example, the first-order tetrahedron unit belongs to a constant strain unit, and the stress strain of any point in the unit is the same, so that gradient change is difficult to embody. The first order hexahedron unit is a gradient unit, and the strain and the stress inside the unit change linearly, so that the gradient change area can be described more accurately. Besides the advantages in precision, under the same size, the number of total nodes which are discretely generated by using hexahedral units is much smaller than that of tetrahedrons, so that the time of finite element calculation can be saved. The six-node octahedral mesh is used here to discretize the silica gel.
On the premise of ensuring that the unit order is unchanged, the result can be approximated to the correct solution by encrypting the grid step by step. The method does not adopt a higher order polynomial as a basis function, has good stability and numerical reliability, and can reach the precision of general engineering. However, when the number of grids reaches a certain degree, the effect of continuously encrypting the grids on improving the precision is not obvious, but the calculation efficiency is greatly reduced, so that the precision and the calculation efficiency need to be comprehensively considered in the process of selecting the unit size. It is assumed here that the thickness of the flexible sensor layer is hmm, and the cell size set in the thickness direction is h/2 to h/6; meanwhile, under the condition of small deformation, the in-plane dimension of the setting unit is consistent with the thickness direction, when normal deformation is large, the in-plane dimension of the setting unit can be twice as large as the thickness direction unit, and when tangential deformation is large, the in-plane dimension can be set smaller, namely about 1/2 of the thickness direction unit dimension.
(2) Structure discretization
The discretization of the structure is the first step of analysis by using a finite element method, namely, the original continuous structure is replaced by a limited number of discrete units arranged according to a certain rule, and all the units can meet a certain relation on nodes and boundaries. In view of comprehensive consideration of calculation accuracy and calculation efficiency, the invention uses the obtained high-performance model and parameter combination to discretize the flexible sensing layer: in the linear elastic constitutive model, the dimension of a six-node octahedral unit is about 1/2-1/6 of the thickness, and the in-plane dimension of the unit is 1/2-2 times of the dimension of the unit in the thickness direction in consideration of different application scenes, as shown in fig. 3.
Step S3: constructing a forward mapping relation between a three-dimensional contact force field and a three-dimensional deformation field of the flexible sensing layer according to a linear finite element theory;
(1) Basic equation for establishing three-dimensional contact force field measurement based on linear finite element theory
Under the condition of small deformation, the current configuration and the initial configuration of the silica gel are slightly different and can be ignored, and the following basic equation can be obtained according to the elastic mechanics principle: ;
Balance equation:
Geometric equation: epsilon=a T u; (2)
The physical equation: sigma=dε; (3)
Stress boundary conditions:
Displacement boundary conditions:
Wherein: sigma represents a stress vector, epsilon represents a strain vector, G represents a volumetric force vector, u represents a displacement vector, Representing the displacement vector of the elastomer on a geometric boundary,/>An area force vector representing an area per unit area of the elastic body on the force boundary; n represents a direction cosine vector of the external normal of the force boundary; a represents a differential operator; d represents the elastic matrix, which is entirely dependent on the elastic modulus and poisson's ratio.
(2) Constructing a forward mapping relation between a three-dimensional contact force field and a three-dimensional deformation field of the flexible sensing layer, and representing unit displacement, strain and stress information by using node displacement;
In order to be able to represent cell displacement, strain and stress information by node displacement, it is necessary to assume a displacement distribution (displacement pattern) of the silica gel cell. Generally, a reasonable displacement pattern is required to make the displacement vectors of the units conform to a certain coordination relationship. In a given displacement mode, the displacement of any point within a cell can be expressed approximately in terms of the cell's node displacement:
u＝N·ue； (7)
Wherein: u is the displacement of any point in the unit, N is a shape function, and u e corresponds to the unit node displacement vector.
Step S4: acquiring an overall stiffness matrix based on comprehensive analysis of the steps S1-S3;
In order to adapt to the irregular boundary of the silica gel pad and enable the analysis result to have higher precision, the parameter unit modes are selected, and the shape functions used by the coordinate transformation mode and the displacement mode have the same form. Meanwhile, in response to the unit selected during the previous grid division, parameter units such as eight-node hexahedron and the like are selected, and the corresponding shape functions are as follows:
Wherein: n 1～N8 respectively represents parameter units corresponding to the hexahedrons of the eight nodes, and ζ, eta and ζ represent local natural coordinates of the corresponding parameter units;
the coordinate transformation mode and the displacement mode are respectively
x(ξ,η,ζ)＝N(ξ,η,ζ)xe； (9)
u(x,y,z)＝u(ξ,η,ζ)＝N(ξ,η,ζ)ue； (10)
Wherein: n is a shape function, x is the position of any point in a unit, x e corresponds to a unit node position vector, u is the displacement of any point in the unit, u e corresponds to a unit node displacement vector, xi, eta and zeta respectively represent local natural coordinates of the unit, and x, y and z respectively represent whole rectangular coordinates.
Substituting the formula (10) into the geometric equation formula (2) to obtain the strain of any point in the silica gel unit:
And substituting the cell strain formula (11) into the physical equation formula (3) to obtain the cell stress. The variation of the formula (10) and the formula (11) is substituted into the virtual work equation to obtain
Where K e is the cell stiffness matrix,For volumetric force equivalent force of unit,/>For the equivalent force of the external force on the surface of the unit,Is the equivalent node force of the interaction between the units.
Equation (12) reflects the balance of only one discrete silica gel unit. But due toThe functions between adjacent silica gel units are mutually equal and opposite, so that when a plurality of units are assembled, the interaction force can be exactly counteracted. From the above analysis, summing all silica gel units can yield the overall system of equations as follows:
KU＝F； (13)
Wherein K is an overall stiffness matrix, F is an equivalent node force (namely a three-dimensional contact force field) caused by contact, U is a node displacement vector (namely a three-dimensional deformation field) of a discrete silica gel system, and thus, the forward mapping relation between the deformation field of the silica gel and the external contact force field of the silica gel is successfully established.
Step S5: acquiring a three-dimensional deformation field of the surface of the flexible sensing layer based on a photometric stereo method and an image tracking algorithm;
The three-dimensional deformation field of the flexible sensing layer of the visual touch sensor mainly comes from the coupling of two modes: normal deformation reconstruction based on photometric stereo method and tangential deformation reconstruction based on image tracking algorithm.
(1) Normal deformation reconstruction
Photometric stereo is a technique in computer vision, formally proposed for the first time by Robert j.woodham in 1979, whose core idea is to reconstruct the surface normal of an object by observing the object under different lighting conditions. The use of photometric stereo requires the following three assumptions:
the image captured by the camera is an orthogonal projection of the sensor surface, with each point in the image corresponding to a point on the sensor surface. Based on this assumption, the gradient of the sensor surface deformation can be defined as the partial derivative of the point seen in the image.
The color of each point in the image is a function of the sensor contact surface normal, which means that drop shadows or internal reflections are ignored.
The albedo (the proportion of incident light reflected by the contact surface) is constant over the sensor contact surface.
In order to reconstruct the normal of the contact surface, normal deformation information of the flexible silica gel is acquired, and a mapping relationship between the light intensity and the reflection function at each pixel needs to be determined. The specific process is as follows:
modeling the contact surface of a sensor using the height function z=f (x, y), the surface normal being noted as The surface normal is noted as this form because the plane is a zero order surface of scalar field phi (x, y, z) =f (x, y) -z, its surface normalIf the illumination and surface reflection are uniformly distributed, the illumination intensity at (x, y) can be calculated as/>, under a single light source, by assuming that (2) the color at each pixel in the image depends only on the local surface normalThe reflection function R can be obtained by modeling the illumination environment and the surface reflectivity. Considering that the light source is from multiple directions, when the illumination colors are different, it can be inferred that different channels in Red Green Blue (RGB) image I, i.e
Wherein ：I(x,y)＝(I1(x,y),I2(x,y),I3(x,y));R(p,q)＝(R1(x,y),R2(x,y),R3(x,y));I1(x,y)、I2(x,y) and I 3 (x, y) correspond to projection images under light from a single direction, respectively.
Assuming that the reflection function R is uniform for the same sensor, R is equal toNonlinear relationship. In order to establish the observed light intensity to surface normal mapping, the inverse of the reflection function R, i.e. R -1, needs to be obtained. Theoretically, since many sets of surface gradients will correspond to the same set of intensities, the reflection function R is irreversible, where R -1 is obtained by constructing a look-up table: the illumination intensity is measured using an object of a known shape (with a known surface gradient), such as a sphere of a known radius, and then a three-dimensional look-up table is generated based on the known gradient and the measured intensity data.
Obtained by R -1 The height field z=f (x.y) can then be obtained by integrating the surface normals, as shown in fig. 4. The integral is converted into poisson equation/>Solving, wherein/> A fast poisson solver with Discrete Sine Transformation (DST) can be used to solve poisson's equations to achieve fast reconstruction of the surface height field and to be able to run on-line.
(2) Tangential deformation reconstruction
In addition to obtaining a height map of the contact surface (i.e. normal deformation of the flexible silicone), the reconstruction of the three-dimensional deformation field also includes obtaining horizontal tangential deformation information of the sensing unit. The approximate estimation of the shear deformation is achieved by simply tracking the motion of the marker array from the image, the core idea of which is to identify and compare the positions of the marker points at the initial moment and the current moment.
The specific process is shown in fig. 5: first, the contact area is detected. Since the fabric texture of the contact areas is highlighted, the Canny filter is used to detect edges and several morphological filters are used to group the edges to form the contact areas. Then the actual displacement field is calculated. Black marks are selected by setting a threshold value, a simple spot detection function (SimpleBlobDetector) is deployed to detect the position of each mark in a contact area, and the displacement of the mark point is calculated by matching and comparing the relative positions of the mark points in a reference frame (initial time) and a current frame, namely, the horizontal deformation information of the sensing unit is obtained. Exemplary representations are: fig. 5 (a) shows an initial frame Frm0; FIG. 5 (b) shows a low-pass Gaussian filtered image of the original frame Frm0, leaving only the original background I0 of the color background; FIG. 5 (c) shows an image frame Frm when the sensor contacts a cylinder; FIG. 5 (d) shows the image dI after subtracting the initial background I0 from the current frame Frm (color normalized for ease of display); fig. 5 (e) illustrates a mark mask after thresholding the image dI, thereby acquiring horizontal deformation information of the sensing unit.
Step S6: and acquiring transfer functions from the three-dimensional deformation field to the three-dimensional contact force field through multi-layer inverse finite element analysis, and acquiring the three-dimensional contact force field according to the three-dimensional deformation field and the overall rigidity matrix which are obtained in advance.
In the case of effectively modeling the positive problem of silica gel contact based on linear finite element theory, as shown in fig. 6, i.e., given the known silica gel material parameters and boundary conditions, the displacement and stress strain conditions inside the silica gel can be calculated from any given external load. After a further accurate three-dimensional deformation field is obtained, the three-dimensional contact force field situation can be deduced back from the deformation information, as shown in fig. 7, wherein: the three graphs in fig. 7 (a), 7 (b) and 7 (c) respectively show the comparison result graphs in the x, y and z directions, and the change trends of the two curves displayed in each graph are basically consistent, which indicates that the three-dimensional contact force measuring method provided by the invention has higher accuracy.
Since the tactile image captured by the camera built in the sensor is mainly concentrated in the center portion of the silica gel pad, a finite element model of a rectangular region at the center of the silica gel is used when solving the inverse problem. Assuming that the discrete silica gel system has n e layers of units, n p layers of nodes (n p＝ne+1,ne is larger than or equal to 2), the node position, the node displacement and the external load of the system can be respectively expressed as:
the three-dimensional surface contact force to be solved is actually F 1, and the formula (13) can be rewritten as
According to previously obtained boundary conditions For the case of only two layers of cells (n e =2), substituting the above conditions into equation (15) yields the equation as follows:
Converting the solution three-dimensional contact force field (F 1) into solution intermediate layer node displacement distribution (U 2), and simplifying to obtain a three-dimensional contact force field result
For systems with more than two layers of cells, the representation of the intermediate layer displacement profile can be simplified by introducing a merge variable U 2 Corresponding matrix/> AndMay be defined as K 1,2、K2,1 'and K 2,2'; similarly, the three-dimensional contact standpoint results are:
Example 2
In this embodiment, which is not described in detail in embodiment 1, an electronic device is provided, including: a processor and a memory, wherein the memory stores a computer program for the processor to call;
The processor executes the three-dimensional contact force field measuring method of the visual touch sensor based on inverse finite element analysis by calling a computer program stored in the memory.
The electronic device may vary greatly in configuration or performance, and can include one or more processors (Central Processing Units, CPU) and one or more memories, where the memories store at least one computer program that is loaded and executed by the processors to implement the inverse finite element analysis-based visual touch sensor three-dimensional touch force field measurement method provided by the above-described method embodiments. The electronic device can also include other components for implementing the functions of the device, for example, the electronic device can also have a wired or wireless network interface, an input-output interface, and the like, for input-output. The embodiments of the present application are not described herein.
The present embodiment provides a computer-readable storage medium having stored thereon an erasable computer program;
The computer program, when run on a computer device, causes the computer device to perform the method of three-dimensional contact force field measurement of a visual tactile sensor based on inverse finite element analysis described above.
In an exemplary embodiment, a computer readable storage medium is also provided, for example a memory comprising at least one computer program executable by a processor to perform the method of three-dimensional contact force field measurement of a visual tactile sensor based on inverse finite element analysis in the above embodiment. For example, the computer readable storage medium can be Read-Only Memory (ROM), random access Memory (Random Access Memory, RAM), compact disk Read-Only Memory (Compact Disc Read-Only Memory, CD-ROM), magnetic tape, floppy disk, optical data storage device, and the like.
In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program comprising one or more program codes, the one or more program codes being stored in a computer readable storage medium. The one or more processors of the electronic device are capable of reading the one or more program codes from the computer-readable storage medium, the one or more processors executing the one or more program codes such that the electronic device is capable of performing the above-described inverse finite element analysis-based visual touch sensor three-dimensional touch force field measurement method.
It should be understood that, in various embodiments of the present application, the sequence numbers of the foregoing processes do not mean the order of execution, and the order of execution of the processes should be determined by the functions and internal logic thereof, and should not constitute any limitation on the implementation process of the embodiments of the present application.
It should be understood that determining B from a does not mean determining B from a alone, but can also determine B from a and/or other information.
The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that variations or substitutions are within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (10)
1. The three-dimensional contact force field measuring method of the visual touch sensor based on inverse finite element analysis is characterized by comprising the following steps of:
step S1: modeling a sensing link of a three-dimensional contact force field of the visual touch sensor, and obtaining a simplified physical model and boundary conditions;
step S2: performing structural discretization on the visual touch sensing unit based on a high-performance model and parameter combination;
Step S3: constructing a forward mapping relation between a three-dimensional contact force field and a three-dimensional deformation field of the flexible sensing layer according to a linear finite element theory;
Step S4: acquiring an overall stiffness matrix based on comprehensive analysis of the steps S1-S3;
Step S5: acquiring a three-dimensional deformation field of the surface of the flexible sensing layer based on a photometric stereo method and an image tracking algorithm;
Step S6: and acquiring transfer functions from the three-dimensional deformation field to the three-dimensional contact force field through multi-layer inverse finite element analysis, and acquiring the three-dimensional contact force field according to the three-dimensional deformation field and the overall rigidity matrix which are obtained in advance.
2. The method for measuring the three-dimensional contact force field of the visual touch sensor based on inverse finite element analysis according to claim 1, wherein the method comprises the following steps of: simplified physical model and boundary condition acquisition logic:
Capturing a tactile image of the contact surface in real time based on the visual tactile sensor; extracting a central rectangular region according to preset conditions; the preset condition is a=1.5r; wherein: a represents the side length of the central rectangular area, and R represents the radius of the flexible sensing layer;
The bottom surface of the flexible sensing layer is tightly attached to the acrylic prism, and the acrylic prism is used as a fixed support of the flexible sensing layer;
When the flexible sensing layer is in contact with an external object, a visual and tactile image can be obtained, and a three-dimensional deformation field of the surface of the sensing unit is obtained through a photometric stereo method; the force boundary condition of the intermediate layer of the sensor unit is zero.
3. The method for measuring the three-dimensional contact force field of the visual touch sensor based on inverse finite element analysis according to claim 2, wherein the method comprises the following steps of: the high performance model and parameter combination analysis includes: the constitutive model discretizes the unit type and unit size used by the flexible sensing layer;
the constitutive model is based on a flexible sensing layer made of flexible materials, performs mechanical analysis on the contact behavior of the flexible sensing layer and external objects based on a linear elastic model,
The unit type is that the flexible sensing layer is discretized by using six-node octahedral grid units;
and (3) under the premise of ensuring that the unit order is unchanged, the unit size is ensured, the result is approximated to the correct solution by gradually encrypting the grid, and the thickness and the in-plane direction unit size are determined.
4. A visual touch sensor three-dimensional touch force field measurement method based on inverse finite element analysis according to claim 3, characterized in that: establishing a mapping relation between a contact force field and a deformation field of silica gel;
establishing basic equations of three-dimensional contact force field measurement based on linear finite element theory, wherein the basic equations comprise balance equations, geometric equations, physical equations, stress boundary conditions and displacement boundary conditions based on elastic mechanics principle;
And constructing a forward mapping relation between the three-dimensional contact force field and the three-dimensional deformation field of the flexible sensing layer, and representing the unit displacement, strain and stress information by using the node displacement.
5. The method for measuring the three-dimensional contact force field of the visual touch sensor based on inverse finite element analysis according to claim 4, wherein the method comprises the following steps of: establishing an overall stiffness matrix as a measurement basis of three-dimensional contact force;
And obtaining the unit stress through a basic equation, selecting an isoparametric unit mode to determine the coordinate transformation and displacement mode of the three-dimensional contact force field, and determining and obtaining the integral rigidity matrix through a mapping relation.
6. The method for measuring the three-dimensional contact force field of the visual touch sensor based on inverse finite element analysis according to claim 5, wherein the method comprises the following steps of: the visual and tactile image comprises depth information of an object and position information of a sensing unit surface mark point; characterizing normal deformation of the surface of the sensing unit based on depth information of the object; characterizing tangential deformation of the surface of the sensing unit based on the position information of the marking points on the surface of the sensing unit; and coupling the normal deformation and the tangential deformation to obtain a three-dimensional deformation field on the surface of the sensing unit.
7. The method for measuring the three-dimensional contact force field of the visual touch sensor based on inverse finite element analysis according to claim 6, wherein the method comprises the following steps of: the logic for acquiring the position information of the mark points on the surface of the sensing unit comprises the following logic:
acquiring an initial image frame of an initial state of the surface of a sensing unit;
the initial background image is obtained after the initial image frame is processed by low-pass Gaussian filtering;
acquiring a sensing unit image frame when the sensor contacts the cylinder, subtracting an initial background image from the sensing unit image frame to obtain a detection contact area, and marking the detection contact area as a detection contact image;
The mark mask after thresholding the detected contact image detects the position of each mark in the contact area, thereby acquiring the horizontal deformation information of the sensing unit.
8. The method for measuring the three-dimensional contact force field of the visual touch sensor based on inverse finite element analysis according to claim 7, wherein the method comprises the following steps of: the detection contact area is a central area of the silica gel pad, wherein the touch image is captured by a camera;
Modeling a detection contact area based on a linear finite element theory, and calculating displacement and force boundary conditions of a central area of the silica gel pad;
And reversely pushing out the three-dimensional contact force field condition of the contact area through the displacement of the central area of the silica gel pad and the force boundary condition.
9. An electronic device, characterized in that: comprising the following steps: a processor and a memory, wherein the memory stores a computer program for the processor to call;
The processor performs the visual tactile sensor three-dimensional contact force field measurement method based on inverse finite element analysis according to any one of claims 1-8 by invoking a computer program stored in the memory.
10. A computer-readable storage medium, characterized by: instructions stored thereon which, when executed on a computer, cause the computer to perform the method for measuring a three-dimensional contact force field of a visual tactile sensor based on inverse finite element analysis according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410277133.7A CN118114524A (en) | 2024-03-12 | 2024-03-12 | Three-dimensional contact force field measuring method of visual touch sensor based on inverse finite element analysis |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410277133.7A CN118114524A (en) | 2024-03-12 | 2024-03-12 | Three-dimensional contact force field measuring method of visual touch sensor based on inverse finite element analysis |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN118114524A true CN118114524A (en) | 2024-05-31 |
Family
ID=91215055
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410277133.7A Pending CN118114524A (en) | 2024-03-12 | 2024-03-12 | Three-dimensional contact force field measuring method of visual touch sensor based on inverse finite element analysis |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN118114524A (en) |
-
2024
- 2024-03-12 CN CN202410277133.7A patent/CN118114524A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ma et al. | Dense tactile force estimation using GelSlim and inverse FEM | |
| CN108549873B (en) | Three-dimensional face recognition method and three-dimensional face recognition system | |
| CN114661169B (en) | Vision-based touch measurement method, device, equipment and storage medium | |
| Ferrier et al. | Reconstructing the shape of a deformable membrane from image data | |
| JP6635690B2 (en) | Information processing apparatus, information processing method and program | |
| JP4206057B2 (en) | Force vector reconstruction method using optical tactile sensor | |
| Zhang et al. | Motion of an uncalibrated stereo rig: Self-calibration and metric reconstruction | |
| Daney et al. | Interval method for calibration of parallel robots: Vision-based experiments | |
| Khalfaoui et al. | An efficient method for fully automatic 3D digitization of unknown objects | |
| CN113330486A (en) | Depth estimation | |
| Wu et al. | Pseudo-siamese graph matching network for textureless objects’6-d pose estimation | |
| WO2021067339A1 (en) | Systems and methods for determining pose of objects held by flexible end effectors | |
| CN116129037B (en) | Visual touch sensor, three-dimensional reconstruction method, system, equipment and storage medium thereof | |
| Cui et al. | Self-supervised contact geometry learning by GelStereo visuotactile sensing | |
| Grieve et al. | Force prediction by fingernail imaging using active appearance models | |
| Wiemann et al. | An extended evaluation of open source surface reconstruction software for robotic applications | |
| Ma et al. | Dense tactile force distribution estimation using gelslim and inverse fem | |
| Yamaura et al. | Shape reconstruction from a normal map in terms of uniform bi-quadratic B-spline surfaces | |
| CN115049744A (en) | Robot hand-eye coordinate conversion method and device, computer equipment and storage medium | |
| Du et al. | 3D contact point cloud reconstruction from vision-based tactile flow | |
| CN118114524A (en) | Three-dimensional contact force field measuring method of visual touch sensor based on inverse finite element analysis | |
| Lu et al. | Tac2Structure: Object Surface Reconstruction Only Through Multi Times Touch | |
| Li et al. | Automatic view planning with self-termination in 3D object reconstructions | |
| CN210802390U (en) | Tactile sensor, robot, and elastic body | |
| Zou et al. | Area-preserving surface flattening using lie advection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |