CN117152259A - Micro-assembly positioning acceleration method and system based on multichannel microscopic vision guidance - Google Patents
Micro-assembly positioning acceleration method and system based on multichannel microscopic vision guidance Download PDFInfo
- Publication number
- CN117152259A CN117152259A CN202311434234.2A CN202311434234A CN117152259A CN 117152259 A CN117152259 A CN 117152259A CN 202311434234 A CN202311434234 A CN 202311434234A CN 117152259 A CN117152259 A CN 117152259A
- Authority
- CN
- China
- Prior art keywords
- matrix
- positioning
- micro
- algorithm
- microscopic vision
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 68
- 230000001133 acceleration Effects 0.000 title claims abstract description 52
- 239000011159 matrix material Substances 0.000 claims abstract description 88
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 55
- 238000006243 chemical reaction Methods 0.000 claims abstract description 19
- 230000008569 process Effects 0.000 claims abstract description 16
- 238000012804 iterative process Methods 0.000 claims abstract description 9
- HPTJABJPZMULFH-UHFFFAOYSA-N 12-[(Cyclohexylcarbamoyl)amino]dodecanoic acid Chemical compound OC(=O)CCCCCCCCCCCNC(=O)NC1CCCCC1 HPTJABJPZMULFH-UHFFFAOYSA-N 0.000 claims abstract 7
- 230000006870 function Effects 0.000 claims description 77
- 238000004364 calculation method Methods 0.000 claims description 32
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 11
- 238000013519 translation Methods 0.000 claims description 11
- 238000005315 distribution function Methods 0.000 claims description 7
- 238000010276 construction Methods 0.000 claims description 6
- 230000009467 reduction Effects 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 6
- 239000003550 marker Substances 0.000 claims description 5
- 238000004590 computer program Methods 0.000 claims description 4
- 238000000354 decomposition reaction Methods 0.000 claims description 4
- 230000004807 localization Effects 0.000 claims description 4
- 230000001131 transforming effect Effects 0.000 claims 1
- 230000000007 visual effect Effects 0.000 abstract description 14
- 238000005457 optimization Methods 0.000 abstract description 4
- 238000012545 processing Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000004088 simulation Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000002401 inhibitory effect Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5016—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals the resource being the memory
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5011—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resources being hardware resources other than CPUs, Servers and Terminals
- G06F9/5022—Mechanisms to release resources
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5005—Allocation of resources, e.g. of the central processing unit [CPU] to service a request
- G06F9/5027—Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/0014—Image feed-back for automatic industrial control, e.g. robot with camera
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/60—Memory management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2209/00—Indexing scheme relating to G06F9/00
- G06F2209/50—Indexing scheme relating to G06F9/50
- G06F2209/5018—Thread allocation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10056—Microscopic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
- G06T2207/30164—Workpiece; Machine component
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Robotics (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a micro-assembly positioning acceleration method and a system based on multichannel microscopic vision guidance, wherein the method comprises the following steps: constructing a multichannel microscopic vision positioning objective function; reducing the dimension of the algorithm space complexity through Cronecker product and column vector conversion; in the iterative solving process, the combination matrix is converted into a constant matrix, so that the calculated amount of intermediate variables in the iterative process is reduced. And realizing multi-channel image acquisition by utilizing a plurality of cameras, and carrying out algorithm optimization and hardware parallel acceleration on micro-assembly real-time positioning facing microscopic vision guidance. Firstly, improving the visual positioning algorithm level, and reducing the space complexity of the algorithm; and then, parallel computation of the program is realized through a CUDA architecture of the GPU display card, so that the running efficiency of the micro-assembly real-time positioning method is further improved.
Description
Technical Field
The invention belongs to the technical field of micro-assembly and micro-operation system positioning, and relates to a micro-assembly positioning acceleration method and system based on multichannel microscopic vision guidance.
Background
The main characteristics of the micro-assembly and micro-operation technology are that the operation object is tiny, so that the micro-assembly and micro-operation equipment is required to have high positioning precision, more operation functions and degrees of freedom. If manual operation is used, the micro-machine assembly can be completed only by means of proper observation and operation assistance means, and a lot of time is spent and high requirements are put on operators. At present, micro-assembly and micro-operation techniques are still in progress. Existing micro-operating systems rely primarily on operator skill, but are limited by the operator themselves and cannot handle tiny devices and tools. If a visual positioning function is added to the micro-system, guiding information can be provided for operation, so that the accuracy and efficiency of the operation can be greatly improved.
Pose information calculation based on an objective function is a common visual positioning algorithm, and is well developed and applied in the field of micro-assembly. Although current visual localization techniques are developed to a high level, the implementation efficiency of algorithms and the combined multi-channel visual localization acceleration techniques still need to be further improved. Firstly, the complexity of the algorithm can be further reduced, the calculated amount of intermediate variables in the iterative process is reduced, and the speed is improved from the software level. On the other hand, computer vision and image processing algorithms are computationally intensive, and the resolution of images is becoming increasingly large, requiring powerful hardware systems to process these large images in real time. GPUs have many parallel processing units, and these cores are made up of a grid of streaming multiprocessors. While the CUDA architecture program is executed as a series of multithreading running in parallel. Currently, the CUDA architecture only supports NVIDIA GPUs, which can be viewed as a combination of blocks, each of which can execute multiple threads, each of which is bound to a different streaming multiprocessor on the GPU. Therefore, the CUDA architecture based on the GPU can well realize hardware acceleration of the visual positioning method. Hardware acceleration is mainly to promote the mathematical operation efficiency of the algorithm, i.e., the ratio of mathematical operation and memory access operation. But the best mathematical operation efficiency and the best performance can be obtained by not maximizing the operation amount of each thread and minimizing the access time. There are three bottlenecks to common kernel execution: on the computational bottleneck, on the memory and on the delay mask. For a particular core, if it is computationally, it should be considered that some computations are equivalently translated into memory, e.g., some operations may attempt to translate into memory look-up tables; the card is on the memory, so that some memory can be converted into corresponding calculation, for example, some data is not reloaded but is directly calculated.
Disclosure of Invention
The invention aims to provide a micro-assembly positioning acceleration method and system based on multi-channel microscopic vision guidance, which realize multi-channel image acquisition by utilizing a plurality of cameras and perform algorithm optimization and hardware parallel acceleration on micro-assembly real-time positioning facing the microscopic vision guidance. Firstly, improving the visual positioning algorithm level, and reducing the space complexity of the algorithm; and then, parallel computation of the program is realized through a CUDA architecture of the GPU display card, so that the running efficiency of the micro-assembly real-time positioning method is further improved.
The technical solution for realizing the purpose of the invention is as follows:
a micro-assembly positioning acceleration method based on multichannel microscopic vision guidance comprises the following steps:
s01: constructing a multichannel microscopic vision positioning objective function;
s02: reducing the dimension of the algorithm space complexity through Cronecker product and column vector conversion;
s03: in the iterative solving process, the combination matrix is converted into a constant matrix, so that the calculated amount of intermediate variables in the iterative process is reduced.
In a preferred embodiment, the step S03 further includes hardware acceleration, including:
designing a main function, and distributing the memory for calculating data at the equipment end through a CUDA dynamic memory distribution function;
designing a GPU kernel function, realizing parallel computation of an algorithm, copying data of the kernel function from the memory of the equipment to the memory of the host, and completing the copying by a CUDA memory copy function;
initializing the CUDA environment parameters of the equipment end, setting thread blocks and the number of corresponding threads, importing space point data for positioning to the GPU end, and completing the configuration of the data; starting the designed kernel function to perform multi-channel calculation, wherein the hardware acceleration device end and the algorithm host end are in data interconnection and intercommunication, and after the calculation is completed, the device end performs memory release and ends all threads.
In a preferred technical solution, the step S01 of constructing a multi-channel microscopic vision positioning objective function includes:
incorporating affine matrixDescribing the uncertainty of the ellipse after the projection of the circular marker points, the objective function is:
wherein,representing the number of the camera, ">，/>Is the number of channels of the vision system; />，/>Rotation matrix and translation vector, respectively; />Is an identity matrix; />Is a sight line direction projection matrix and acts on object points; />Is spatial point at->Normalized coordinates in the individual cameras.
In a preferred embodiment, the method for performing the kronecker product-column vector conversion in step S02 includes:
according to a vectorization algorithm of the matrix, and through conversion of the Cronecker product, the equivalent transformation is carried out on the objective function:
wherein the symbols areThe representation is:the method comprises the steps of carrying out a first treatment on the surface of the Sign->Is an operator of the kronecker product; />Representing an operation to convert the multi-dimensional matrix into a column vector;
further convert into:
wherein,，/>is a constant.
In a preferred embodiment, the step S03 of converting the combining matrix into the constant matrix includes:
in the iterative solving process, the translation vector is used forSeen as an intermediate quantity, does not need to output +.>According to the objective function, the following holds:
from the properties of the kronecker product, we get:
since the sum of the centroid coordinates of all spatial points is 0 before starting the iteration, the identity matrix is eliminatedGet about->The acceleration expression of (2) is:
based on introducing disturbance affine termsAn average of the projected point coordinates along the line of sight is obtained:
wherein,representing the number of iterations;
in order to iteratively find the optimal solution using absolute orientation, an intermediate matrix needs to be calculated：
Further improving the algorithm structure to makeI.e. a vectorized representation of the intermediate matrix, is further expressed as:
at this time, it is obtained by singular value decomposition，/>，/>Is an orthogonal matrix, < >>Is a diagonal matrix; then update +.>Combination matrix in formula->Expressed as:
due toTherefore, there are:
the matrix obtained from the above equation is a constant matrix.
The invention discloses a micro-assembly positioning acceleration system based on multichannel microscopic vision guidance, which comprises:
the objective function construction module is used for constructing a multichannel microscopic vision positioning objective function;
the dimension reduction module is used for reducing the dimension of the algorithm space complexity through Cronecker sum and column vector conversion;
and the iterative calculation module is used for converting the combination matrix into a constant matrix in the iterative solving process, so that the calculated amount of intermediate variables in the iterative process is reduced.
In the preferred technical scheme, the system also comprises a hardware acceleration module, and the hardware acceleration method comprises the following steps:
designing a main function, and distributing the memory for calculating data at the equipment end through a CUDA dynamic memory distribution function;
designing a GPU kernel function, realizing parallel computation of an algorithm, copying data of the kernel function from the memory of the equipment to the memory of the host, and completing the copying by a CUDA memory copy function;
initializing the CUDA environment parameters of the equipment end, setting thread blocks and the number of corresponding threads, importing space point data for positioning to the GPU end, and completing the configuration of the data; starting the designed kernel function to perform multi-channel calculation, wherein the hardware acceleration device end and the algorithm host end are in data interconnection and intercommunication, and after the calculation is completed, the device end performs memory release and ends all threads.
In a preferred technical scheme, the construction of the multi-channel microscopic vision positioning objective function in the objective function construction module includes:
incorporating affine matrixDescribing the uncertainty of the ellipse after the projection of the circular marker points, the objective function is:
wherein,representing the number of the camera, ">，/>Is the number of channels of the vision system; />，/>Rotation matrix and translation vector, respectively; />Is an identity matrix; />Is a sight line direction projection matrix and acts on object points; />Is spatial point at->Normalized coordinates in the individual cameras.
In a preferred embodiment, the method for performing the conversion between the kronecker product and the column vector in the dimension reduction module includes:
according to a vectorization algorithm of the matrix, and through conversion of the Cronecker product, the equivalent transformation is carried out on the objective function:
wherein the symbols areThe representation is:the method comprises the steps of carrying out a first treatment on the surface of the Sign->Is an operator of the kronecker product; />Representing an operation to convert the multi-dimensional matrix into a column vector;
further convert into:
wherein,，/>is a constant.
The invention also discloses a computer storage medium, on which a computer program is stored, which when executed realizes the micro-assembly positioning acceleration method based on the multi-channel microscopic vision guidance.
Compared with the prior art, the invention has the remarkable advantages that:
aiming at the acceleration problem of a microscopic visual positioning method in micro assembly, the invention constructs a multichannel microscopic visual positioning objective function, improves an algorithm structure, reduces the space complexity of a visual positioning algorithm, reduces the calculation amount of intermediate variables in the iterative process, and improves the resolving speed of pose information from a software level. On the other hand, considering that computer vision and image processing algorithms belong to intensive calculation, the image data volume becomes larger and a powerful hardware system is required to process the large images in real time, so that parallel operation of a vision positioning algorithm is realized through a CUDA (compute unified device architecture) of a GPU (graphics processing unit) display card, the rapidity of vision guiding information calculation in micro assembly is improved through software and hardware cooperation, and the micro assembly and micro operating system are promoted to develop towards autonomous and intelligent directions.
Drawings
FIG. 1 is a flow chart of a preferred embodiment of a method for accelerating positioning of a micro-assembly based on multi-channel microscopic vision guidance;
FIG. 2 is a diagram of a software algorithm optimized acceleration and hardware acceleration architecture;
FIG. 3 is a graph comparing mean error curves of attitude estimation under different elliptical uncertainties;
FIG. 4 is a graph comparing mean error curves of position estimation under different elliptical uncertainties;
FIG. 5 is a graph comparing the mean error curves of the gesture estimation under different feature point numbers;
FIG. 6 is a graph comparing mean error curves of position estimation under different feature point numbers.
Detailed Description
The principle of the invention is as follows: and realizing multi-channel image acquisition by utilizing a plurality of cameras, and carrying out algorithm optimization and hardware parallel acceleration on micro-assembly real-time positioning facing microscopic vision guidance. Firstly, improving the visual positioning algorithm level, and reducing the space complexity of the algorithm; and then, parallel computation of the program is realized through a CUDA architecture of the GPU display card, so that the running efficiency of the micro-assembly real-time positioning method is further improved.
Example 1:
as shown in fig. 1, a micro-assembly positioning acceleration method based on multi-channel microscopic vision guidance comprises the following steps:
s01: constructing a multichannel microscopic vision positioning objective function;
s02: reducing the dimension of the algorithm space complexity through Cronecker product and column vector conversion;
s03: in the iterative solving process, the combination matrix is converted into a constant matrix, so that the calculated amount of intermediate variables in the iterative process is reduced.
In a preferred embodiment, step S03 further includes hardware acceleration, including:
designing a main function, and distributing the memory for calculating data at the equipment end through a CUDA dynamic memory distribution function;
designing a GPU kernel function, realizing parallel computation of an algorithm, copying data of the kernel function from the memory of the equipment to the memory of the host, and completing the copying by a CUDA memory copy function;
initializing the CUDA environment parameters of the equipment end, setting thread blocks and the number of corresponding threads, importing space point data for positioning to the GPU end, and completing the configuration of the data; starting the designed kernel function to perform multi-channel calculation, wherein the hardware acceleration device end and the algorithm host end are in data interconnection and intercommunication, and after the calculation is completed, the device end performs memory release and ends all threads.
In a preferred embodiment, step S01 of constructing the multi-channel microscopic visual localization objective function includes:
incorporating affine matrixDescribing the uncertainty of the ellipse after the projection of the circular marker points, the objective function is:
wherein,representing the number of the camera, ">，/>Is the number of channels of the vision system; />，/>Rotation matrix and translation vector, respectively; />Is an identity matrix; />Is a sight line direction projection matrix and acts on object points; />Is spatial point at->Normalized coordinates in the individual cameras.
In a preferred embodiment, the method of the kronecker product-column vector conversion in step S02 includes:
according to a vectorization algorithm of the matrix, and through conversion of the Cronecker product, the equivalent transformation is carried out on the objective function:
wherein the symbols areThe representation is:the method comprises the steps of carrying out a first treatment on the surface of the Sign->Is an operator of the kronecker product; />Representing an operation to convert the multi-dimensional matrix into a column vector;
further convert into:
wherein,，/>is a constant.
In a preferred embodiment, the converting the combining matrix into the constant matrix in step S03 includes:
in the iterative solving process, the translation vector is used forSeen as an intermediate quantity, does not need to output +.>According to the objective function, the following holds:
from the properties of the kronecker product, we get:
since the sum of the centroid coordinates of all spatial points is 0 before starting the iteration, the identity matrix is eliminatedGet about->The acceleration expression of (2) is:
based on introducing disturbance affine termsAn average of the projected point coordinates along the line of sight is obtained:
wherein,representing the number of iterations;
in order to iteratively find the optimal solution using absolute orientation, an intermediate matrix needs to be calculated：
Further improving the algorithm structure to makeI.e. a vectorized representation of the intermediate matrix, is further expressed as:
at this time, it is obtained by singular value decomposition，/>，/>Is an orthogonal matrix, < >>Is a diagonal matrix; then update +.>Combination matrix in formula->Expressed as:
due toTherefore, there are:
the matrix obtained from the above equation is a constant matrix.
In another embodiment, a computer storage medium has a computer program stored thereon, which when executed implements the above-described method for positioning and accelerating micro-assembly based on multi-channel microscopic visual guidance.
In yet another embodiment, a microfabricated positioning acceleration system based on multi-channel microscopic vision guidance, comprising:
the objective function construction module is used for constructing a multichannel microscopic vision positioning objective function;
the dimension reduction module is used for reducing the dimension of the algorithm space complexity through Cronecker sum and column vector conversion;
and the iterative calculation module is used for converting the combination matrix into a constant matrix in the iterative solving process, so that the calculated amount of intermediate variables in the iterative process is reduced.
The hardware acceleration module is further included, and the hardware acceleration method comprises the following steps:
designing a main function, and distributing the memory for calculating data at the equipment end through a CUDA dynamic memory distribution function;
designing a GPU kernel function, realizing parallel computation of an algorithm, copying data of the kernel function from the memory of the equipment to the memory of the host, and completing the copying by a CUDA memory copy function;
initializing the CUDA environment parameters of the equipment end, setting thread blocks and the number of corresponding threads, importing space point data for positioning to the GPU end, and completing the configuration of the data; starting the designed kernel function to perform multi-channel calculation, wherein the hardware acceleration device end and the algorithm host end are in data interconnection and intercommunication, and after the calculation is completed, the device end performs memory release and ends all threads.
Specifically, the following description will be given by taking a preferred embodiment as an example for the workflow of the micro-assembly positioning acceleration system based on the multi-channel microscopic vision guidance:
the overall execution flow of the spatial low-complexity positioning algorithm and the hardware acceleration proposed in the present embodiment is shown in fig. 2. On the basis of constructing a pose solving objective function, the dimension reduction of the algorithm space complexity is realized through the conversion of the Cronecker product and the column vector. The method comprises the following specific steps:
step one: space complexity optimization of visual pose information solving algorithm
Step 11: establishing a multi-channel stereo microscopic vision minimum objective function shown in the formula (1), wherein the objective function introduces an affine matrixThe uncertainty of the ellipse after the projection of the circular marker point can be described:
(1)
wherein,representing the number of the camera, ">，/>Is the number of channels of the vision system; />，/>Rotation matrix and translation vector, respectively; />Is an identity matrix; />Is a sight line direction projection matrix and acts on object points; />Is spatial point at->Normalized coordinates in the individual cameras.
Step 12: according to the vectorization algorithm of the matrix and the conversion by the kronecker product, the equivalent transformation of the equation (1) can be performed:
(2)
wherein, for convenience of description, symbols are givenThe representation is:the method comprises the steps of carrying out a first treatment on the surface of the Sign->Is an operator of the kronecker product; />Representing the operation of converting the multidimensional matrix into a column vector.
Equation (2) can be further converted into:
(3)
wherein, for convenience of expression, let. Can see +.>Is a constant, so the spatial complexity through the conversion sub (3) is reduced to +.>。
Step 13: in the iterative solving process, the method canSeen as an intermediate quantity, so that no output per iteration is required>Is a value of (2). According to equation (1), the following holds:
(4)
depending on the nature of the kronecker product, formula (4) can be written as:
(5)
since the sum of the centroid coordinates of all spatial points is 0 before starting the iteration. Therefore, the identity matrix can be eliminatedGet about->The acceleration expression of (2) is:
(6)
then based on the introduction of disturbance affine termThe average of the projected point coordinates along the line of sight can be obtained:
(7)/>
wherein,representing the number of iterations.
Step 14: in order to iteratively find the optimal solution using absolute orientation, an intermediate matrix needs to be calculated：
(8)
Further improving the algorithm structure to makeI.e., a vectorized representation of the intermediate matrix, equation (8) may be expressed as:
(9)
at this time, it is obtained by singular value decomposition，/>，/>Is a 3 x 3 orthogonal matrix,>is a 3 x 3 diagonal matrix; then update +.>. The combination matrix in formula (9)>Can be expressed as:
(10)
due toCan be written as +.>Therefore, there are:
(11)
the matrix can be seen from the formula (11) to be a constant matrix, so that iteration solution round by round is not needed, the calculated amount in the iteration process is reduced, and the iteration efficiency is improved.
Thus at the maximum number of iterationsWithin the range, update->Variation of the minimum error function of the next iterationThe calculation amount is reduced when +.>Under the condition, the iteration can be terminated, wherein +.>Is the iteration stop judgment threshold.
Step two: kernel function and main function design based on CUDA architecture
Step 21: in main function design, first, the main function is distributed on the equipment through CUDA dynamic memory distribution function、、/>Memory of the data. And (3) designing a GPU kernel function, wherein the GPU kernel function comprises parallel computation of the formulas (6) - (9), and data of the kernel function must be copied from the memory of the device to the memory of the host computer and is completed by a CUDA memory copy function. And the memory is optimized on the whole, the memory is uniformly distributed and released, the data copying operation between the equipment end and the host end is reduced, and the data transmission delay is reduced.
Step 22: in the iterative algorithm solving process, a hardware acceleration cooperative technology is accompanied, namely, data interaction exists between a host side and a device side. In the aspect of hardware acceleration, firstly, the device side CUDA environment parameters are initialized, the thread blocks (blocks) and the number of corresponding threads are set, and then the space point data for positioning is imported to the GPU side, so that the data configuration is completed. And then starting the designed kernel function to perform multi-channel calculation, so that a plurality of iterative calculation tasks can be completed in parallel in the same time. In the process, the hardware acceleration equipment end and the algorithm host end have data interconnection and intercommunication, and after the calculation is completed, the equipment end releases the memory and ends all threads.
In order to further verify the effectiveness of the method of the invention, verification is performed by simulation experiments and actual experiments. Simulation experiments were performed under Matlab2022b development environment. Since affine terms are introduced into the objective function, the elliptical uncertainty of the feature point projection is considered, so that the magnitude of the elliptical uncertainty is changed through simulation data, the distance is increased from 1 to 15, and the interval is 1. The rotation error and the translation error obtained by the method are respectively compared with four other common resolving methods, and the results are shown in fig. 3 and 4. As can be seen from fig. 3 and 4, as the elliptical uncertainty increases, the errors of the five pose solutions all show an increasing trend, but the translational and rotational calculation errors of the method of the present invention are minimal. For the calculation of rotation information, the error rising speed of the method is obviously lower than that of other four methods for comparison, and the method has a good effect of inhibiting projection ellipse uncertainty.
Because the vision positioning algorithm needs the support of the characteristic point data, the increase of the number of the characteristic points is beneficial to improving the accuracy of vision positioning in a certain range. However, many times there are not enough feature points. The number of calculation points was increased from 4 to 30 through experimental tests, and the rotation average error and the translation average error are shown in fig. 5 and 6. The comparison shows that the positioning accuracy of the five methods is continuously improved along with the increase of the number of the feature points. Compared with other three traditional methods, the scale weighting method has better effect, and the pose calculated by the method has highest precision, so that the improvement on the algorithm space complexity does not reduce the calculation precision of the algorithm, and has good comprehensive performance under the condition of same ellipse uncertainty and operation point number.
And finally, verifying the acceleration effect of the algorithm, performing operation by setting different numbers of space points, and adopting parallel calculation under CPU serial and CUDA architecture, wherein the result is shown in table 1.
TABLE 1 time consuming comparison of spatial point coordinate calculations
/>
As can be seen from Table 1, as the number of points increases, the calculation speed ratio is larger, which shows that CUDA is suitable for large-number calculation. When the number of points is increased to 300, the acceleration ratio is stabilized to be about 12.1, so that the invention realizes a better acceleration ratio from the aspect of soft and hard cooperative technology, and can meet the requirement of real-time positioning of micro assembly.
The foregoing examples are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the foregoing examples, and any other changes, modifications, substitutions, combinations, and simplifications that do not depart from the spirit and principles of the present invention should be made therein and are intended to be equivalent substitutes within the scope of the present invention.
Claims (10)
1. The micro-assembly positioning acceleration method based on multichannel microscopic vision guidance is characterized by comprising the following steps of:
s01: constructing a multichannel microscopic vision positioning objective function;
s02: reducing the dimension of the algorithm space complexity through Cronecker product and column vector conversion;
s03: in the iterative solving process, the combination matrix is converted into a constant matrix, so that the calculated amount of intermediate variables in the iterative process is reduced.
2. The method for positioning and accelerating the micro-assembly based on the multi-channel microscopic vision guidance according to claim 1, wherein the step S03 further comprises hardware acceleration, comprising:
designing a main function, and distributing the memory for calculating data at the equipment end through a CUDA dynamic memory distribution function;
designing a GPU kernel function, realizing parallel computation of an algorithm, copying data of the kernel function from the memory of the equipment to the memory of the host, and completing the copying by a CUDA memory copy function;
initializing the CUDA environment parameters of the equipment end, setting thread blocks and the number of corresponding threads, importing space point data for positioning to the GPU end, and completing the configuration of the data; starting the designed kernel function to perform multi-channel calculation, wherein the hardware acceleration device end and the algorithm host end are in data interconnection and intercommunication, and after the calculation is completed, the device end performs memory release and ends all threads.
3. The method for accelerating the positioning of the micro-assembly based on the multi-channel microscopic vision guidance according to claim 1, wherein the constructing the multi-channel microscopic vision positioning objective function in the step S01 includes:
incorporating affine matrixDescribing the uncertainty of the ellipse after the projection of the circular marker points, the objective function is:
，
wherein,representing the number of the camera, ">，/>Is the number of channels of the vision system; />，/>Rotation matrix and translation vector, respectively; />Is an identity matrix; />Is a sight line direction projection matrix and acts on object points; />Is spatial point at->Normalized coordinates in the individual cameras.
4. The method for accelerating the positioning of the micro-assembly based on the multi-channel microscopic vision guidance according to claim 3, wherein the method for transforming the product of the kronecker and the column vector in the step S02 comprises the following steps:
according to a vectorization algorithm of the matrix, and through conversion of the Cronecker product, the equivalent transformation is carried out on the objective function:
，
wherein the symbols areThe representation is: />The method comprises the steps of carrying out a first treatment on the surface of the Sign->Is an operator of the kronecker product; />Representing an operation to convert the multi-dimensional matrix into a column vector;
further convert into:
，
wherein,，/>is a constant.
5. The method for positioning and accelerating the micro-assembly based on the multi-channel microscopic vision guidance according to claim 4, wherein the step S03 of converting the combination matrix into a constant matrix comprises:
in the iterative solving process, the translation vector is used forSeen as an intermediate quantity, does not need to output +.>According to the objective function, the following holds:
，
from the properties of the kronecker product, we get:
，
since the sum of the centroid coordinates of all spatial points is 0 before starting the iteration, the identity matrix is eliminatedGet about->The acceleration expression of (2) is:
，
based on introducing disturbance affine termsAn average of the projected point coordinates along the line of sight is obtained:
，
wherein,representing the number of iterations;
in order to iteratively find the optimal solution using absolute orientation, an intermediate matrix needs to be calculated：
，
Further improving algorithmStructure, orderI.e. a vectorized representation of the intermediate matrix, is further expressed as:
，
at this time, it is obtained by singular value decomposition，/>，/>Is an orthogonal matrix, < >>Is a diagonal matrix; then update +.>Combination matrix in formula->Expressed as:
，
due toTherefore, there are:
，
the matrix obtained from the above equation is a constant matrix.
6. A micro-assembly positioning acceleration system based on multi-channel microscopic vision guidance, comprising:
the objective function construction module is used for constructing a multichannel microscopic vision positioning objective function;
the dimension reduction module is used for reducing the dimension of the algorithm space complexity through Cronecker sum and column vector conversion;
and the iterative calculation module is used for converting the combination matrix into a constant matrix in the iterative solving process, so that the calculated amount of intermediate variables in the iterative process is reduced.
7. The micro-assembly positioning acceleration system based on multi-channel microscopic vision guidance according to claim 6, further comprising a hardware acceleration module, wherein the hardware acceleration method comprises:
designing a main function, and distributing the memory for calculating data at the equipment end through a CUDA dynamic memory distribution function;
designing a GPU kernel function, realizing parallel computation of an algorithm, copying data of the kernel function from the memory of the equipment to the memory of the host, and completing the copying by a CUDA memory copy function;
initializing the CUDA environment parameters of the equipment end, setting thread blocks and the number of corresponding threads, importing space point data for positioning to the GPU end, and completing the configuration of the data; starting the designed kernel function to perform multi-channel calculation, wherein the hardware acceleration device end and the algorithm host end are in data interconnection and intercommunication, and after the calculation is completed, the device end performs memory release and ends all threads.
8. The multi-channel microscopic vision guidance based micro-assembly positioning acceleration system of claim 6, wherein constructing the multi-channel microscopic vision positioning objective function in the objective function construction module comprises:
incorporating affine matrixFor the uncertainty of ellipse after the projection of the circular mark pointDescribing, the objective function is:
，
wherein,representing the number of the camera, ">，/>Is the number of channels of the vision system; />，/>Rotation matrix and translation vector, respectively; />Is an identity matrix; />Is a sight line direction projection matrix and acts on object points; />Is spatial point at->Normalized coordinates in the individual cameras.
9. The multi-channel microscopic vision guidance based micro-assembly positioning acceleration system of claim 8, wherein the method of the dimension reduction module through kronecker product-column vector conversion comprises:
according to a vectorization algorithm of the matrix, and through conversion of the Cronecker product, the equivalent transformation is carried out on the objective function:
，
wherein the symbols areThe representation is: />The method comprises the steps of carrying out a first treatment on the surface of the Sign->Is an operator of the kronecker product; />Representing an operation to convert the multi-dimensional matrix into a column vector;
further convert into:
，
wherein,，/>is a constant.
10. A computer storage medium having stored thereon a computer program, characterized in that the computer program, when executed, implements the multi-channel microscopic vision guidance based micro-assembly localization acceleration method of any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311434234.2A CN117152259A (en) | 2023-11-01 | 2023-11-01 | Micro-assembly positioning acceleration method and system based on multichannel microscopic vision guidance |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311434234.2A CN117152259A (en) | 2023-11-01 | 2023-11-01 | Micro-assembly positioning acceleration method and system based on multichannel microscopic vision guidance |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117152259A true CN117152259A (en) | 2023-12-01 |
Family
ID=88903198
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311434234.2A Pending CN117152259A (en) | 2023-11-01 | 2023-11-01 | Micro-assembly positioning acceleration method and system based on multichannel microscopic vision guidance |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117152259A (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112986944A (en) * | 2021-03-04 | 2021-06-18 | 西安电子科技大学 | CUDA heterogeneous parallel acceleration-based radar MTI and MTD implementation method |
| CN113409404A (en) * | 2021-06-29 | 2021-09-17 | 常熟理工学院 | CUDA architecture parallel optimization three-dimensional deformation measurement method based on novel correlation function constraint |
-
2023
- 2023-11-01 CN CN202311434234.2A patent/CN117152259A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112986944A (en) * | 2021-03-04 | 2021-06-18 | 西安电子科技大学 | CUDA heterogeneous parallel acceleration-based radar MTI and MTD implementation method |
| CN113409404A (en) * | 2021-06-29 | 2021-09-17 | 常熟理工学院 | CUDA architecture parallel optimization three-dimensional deformation measurement method based on novel correlation function constraint |
Non-Patent Citations (1)
| Title |
|---|
| GUIYANG ZHANG等: "Expedited Pose Estimation Algorithm Involving Perturbance Affine Term Based on Projection Vector for Space Target", 《IEEE》, pages 1 - 16 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN104461466B (en) | The method for improving calculating speed based on MPI and OpenMP Hybrid paradigm parallel computations | |
| CN110135569A (en) | Heterogeneous platform neuron positioning three-level flow parallel method, system and medium | |
| CN111707262B (en) | Point cloud matching method, medium, terminal and device based on closest point vector projection | |
| Park et al. | Parallel motion planning using poisson-disk sampling | |
| CN117152259A (en) | Micro-assembly positioning acceleration method and system based on multichannel microscopic vision guidance | |
| CN116038720B (en) | Hand-eye calibration method, device and equipment based on point cloud registration | |
| US20100070243A1 (en) | Wire Harness Unfolding | |
| JP2019046196A (en) | Matrix simplification device, program, and matrix simplification method | |
| CN114117896B (en) | Binary protocol optimization implementation method and system for ultra-long SIMD pipeline | |
| CN116227615A (en) | Quantum search simulation method and system for super computing | |
| CN110502975A (en) | A kind of batch processing system that pedestrian identifies again | |
| CN114119684B (en) | Marker point registration method based on tetrahedral structure | |
| CN104092467A (en) | Method for compressing matrixes through dual quaternions | |
| CN112116640B (en) | Binocular stereo matching method based on OpenCL | |
| CN110516194B (en) | Heterogeneous many-core processor-based grid point quantum color dynamics parallel acceleration method | |
| CN113297860A (en) | Method, system, electronic device and storage medium for optimizing machine translation model | |
| CN111125619B (en) | Method for determining strain along curved surface of curve | |
| Shimokawabe et al. | A high-productivity framework for multi-GPU computation of mesh-based applications | |
| CN104615583A (en) | Data processing method and device based on GPU platform | |
| CN110276803B (en) | Formalization method and device for camera pose estimation, electronic equipment and storage medium | |
| Wang et al. | Research on EBE-FEM realized by CUDA applying to electromagnetic field analysis | |
| Chen et al. | A simple algorithm for camera pose estimation | |
| CN111951336B (en) | Three-dimensional large coordinate processing method based on GPU | |
| Kolpakov et al. | Software for heterogeneous computer systems and structures of data processing systems with increased performance | |
| Shrestha et al. | Multi-level parallel algorithm to solve the Eikonal equation with the fast sweeping method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20231201 |