CN116452655B

CN116452655B - Laminating and positioning method, device, equipment and medium applied to MPIS industrial control main board

Info

Publication number: CN116452655B
Application number: CN202310415140.4A
Authority: CN
Inventors: 吴丰义; 吴世杰
Original assignee: Shenzhen Lingyi Technology Co ltd
Current assignee: Shenzhen Lingyi Technology Co ltd
Priority date: 2023-04-18
Filing date: 2023-04-18
Publication date: 2023-11-21
Anticipated expiration: 2043-04-18
Also published as: CN116452655A

Abstract

The invention relates to an artificial intelligence technology, and discloses a fitting and positioning method, a fitting and positioning device, equipment and a medium applied to an MPIS industrial control main board, wherein the method comprises the following steps: performing multi-stage focusing on the depth camera by using the primary distance interval obtained by distance measurement, and performing multi-focus shooting on the MPIS industrial control main board by using the focused depth camera to obtain a primary main board depth atlas; performing multi-stage definition operation on the primary main board depth atlas to obtain a standard main board depth image; sequentially carrying out image segmentation and multi-type edge fitting operation on the standard main plate depth image to obtain a fitted main plate depth image, and carrying out fitting positioning labeling on the fitted main plate depth image to obtain a labeled main plate depth image; and carrying out coordinate extraction on the attaching marking areas in the depth pictures of the marking main boards one by one, and attaching the MPIS industrial control main boards. The invention also provides laminating positioning equipment and medium applied to the MPIS industrial control main board. The invention can improve the accuracy of positioning and attaching the industrial control main board.

Description

Laminating and positioning method, device, equipment and medium applied to MPIS industrial control main board

Technical Field

The invention relates to the technical field of artificial intelligence, in particular to a fitting and positioning method, device, equipment and medium applied to an MPIS industrial control main board.

Background

Industrial control main boards of the MPIS framework are widely applied to a plurality of intelligent control fields by virtue of low power consumption and stability, in order to meet the increasing demands of markets on the industrial control main boards of the MPIS, the yield of the industrial control main boards of the MPIS needs to be improved, and in the process of producing the industrial control main boards of the MPIS, an intelligent robot is generally required to carry out positioning and laminating on devices such as capacitors and the like on the designed MPIS printed circuit boards.

The existing positioning and laminating methods of the MPIS industrial control main board are mainly positioning and laminating methods based on simple machine vision, for example, main board pictures are collected, a laminating area and a datum point in the main board pictures are positioned by using a convolutional neural network model, three-dimensional coordinates of the laminating area are determined according to the position relation between the laminating area and the datum point, and a robot is used for laminating according to the three-dimensional coordinates.

Disclosure of Invention

The invention provides a laminating and positioning method, a laminating and positioning device, equipment and a medium applied to an MPIS industrial control main board, and mainly aims to solve the problem of lower accuracy in positioning and laminating the industrial control main board.

In order to achieve the above object, the present invention provides a fitting and positioning method applied to an MPIS industrial control motherboard, including:

performing ultrasonic ranging on an MPIS industrial control main board on a bonding workbench to obtain a primary distance interval, performing multi-stage focusing on a depth camera by using the primary distance interval, and performing multi-focus shooting on the MPIS industrial control main board by using the focused depth camera to obtain a primary main board depth atlas;

performing distortion calibration correction on the primary main board depth atlas to obtain a secondary main board depth atlas, and performing multi-type image blocking operation on the secondary main board depth atlas to obtain a secondary main board depth image block set;

screening a standard secondary depth image block set from the secondary main board depth image block set by using a preset mean value definition algorithm, splicing the standard secondary depth image block set into a spliced main board depth image, and sequentially performing multistage filtering and depth equalization operation on the spliced main board depth image to obtain a standard main board depth image, wherein the screening the standard secondary depth image block set from the secondary main board depth image block set by using the preset mean value definition algorithm comprises the following steps: selecting secondary main board depth block groups in the secondary main board depth block group one by one as target depth block groups, and selecting depth blocks in the target depth block groups one by one as target depth blocks; filtering and smoothing the target depth image block to obtain a target filtering image block, and generating a target gray image block according to the target depth image block and the target filtering image block; calculating the definition of each target gray scale tile by using the following mean definition algorithm:

Wherein L refers to the definition, M refers to the total number of horizontal pixels of the target gray tile, N refers to the total number of vertical pixels of the target gray tile, i refers to the ith horizontal pixel, j refers to the jth vertical pixel, α is a preset horizontal sampling length, β is a preset vertical sampling length, |·| is an absolute value function, g () is a gray value function, g (i+α, j+β) refers to the gray value of the pixel with coordinates (i+α, j+β) in the target gray tile, g (i, j) refers to the gray value of the pixel with coordinates (i, j) in the target gray tile, e is a natural logarithmic sign, o is an imaginary sign, u, v is a transform coefficient of a fast fourier formula; selecting a target depth image block corresponding to a target gray image block with the maximum definition from each target depth image block group as a standard secondary depth image block, and collecting all the standard secondary depth image blocks into a standard secondary depth image block set;

sequentially carrying out image segmentation and multi-type edge fitting operation on the standard main plate depth image to obtain a fitted main plate depth image, and carrying out fitting positioning labeling on the fitted main plate depth image by using a clustering matching algorithm to obtain a labeled main plate depth image;

And selecting the laminating marking areas in the marking mainboard depth pictures one by one as target laminating marking areas, reconstructing the target laminating marking areas to obtain target laminating marking point clouds, extracting target laminating coordinates from the target laminating marking point clouds, and laminating the MPIS industrial control mainboard by utilizing the target laminating coordinates.

Optionally, the ultrasonic ranging is performed on the MPIS industrial control main board on the bonding workbench to obtain a primary distance interval, including:

transmitting main board ranging ultrasonic waves to an MPIS industrial control main board on the bonding workbench by utilizing an ultrasonic generator, and acquiring main board ranging reflected waves of the main board ranging ultrasonic waves by utilizing an ultrasonic receiver;

calculating a main board distance interval and a main board distance measurement angle interval between the ultrasonic generator and the MPIS industrial control main board according to the angle of the main board distance measurement reflected wave and the time difference between the main board distance measurement ultrasonic wave and the main board distance measurement reflected wave;

transmitting camera ranging ultrasonic waves to a depth camera by using the ultrasonic generator, and acquiring camera ranging reflected waves of the camera ranging ultrasonic waves by using the ultrasonic receiver;

Calculating the shortest camera distance and the shortest camera ranging angle between the ultrasonic generator and the depth camera according to the angle of the camera ranging reflected wave and the time difference between the camera ranging emitted ultrasonic wave and the camera ranging reflected wave;

and calculating a camera main board included angle interval according to the main board ranging angle interval and the shortest camera ranging angle, and calculating a primary distance interval between the depth camera and the MPIS industrial control main board according to the camera main board included angle interval, the main board distance interval and the shortest camera distance.

Optionally, the performing distortion calibration correction on the primary main board depth atlas to obtain a secondary main board depth atlas includes:

selecting pictures in the primary main board depth map set one by one as target primary depth pictures, and calculating target secondary depth pictures corresponding to the target primary depth pictures by using the following calibration correction algorithm:

wherein,is the pixel abscissa of the target secondary depth picture,/i>Refers to the pixel ordinate of the target secondary depth picture, x refers to the pixel abscissa of the target primary depth picture, y refers to the pixel ordinate of the target primary depth picture, (x, y) refers to the pixel with the coordinate point (x, y) in the target primary depth picture, delta refers to the coordinate value range of the target primary depth picture, h ₁ Is the first term of Taylor series expansion, h ₂ The second term of Taylor series expansion, d refers to the calibrated internal reference of the depth camera, k ₁ Is the first calibration parameter, k, of the calibration correction algorithm ₂ The second calibration parameter of the calibration correction algorithm is referred to;

and collecting all the target secondary depth pictures into a secondary main board depth atlas.

Optionally, the performing the multi-type image blocking operation on the secondary motherboard depth atlas to obtain a secondary motherboard depth atlas group set includes:

selecting pictures in the secondary main board depth map set one by one as target secondary depth pictures, and dividing the target secondary depth pictures into a plurality of secondary depth image blocks by using a quadtree;

acquiring the picture sizes of the secondary depth image blocks one by one, converging the secondary depth image blocks with the picture sizes larger than a preset size threshold into a large-size depth image block group, and converging the secondary depth image blocks with the picture sizes smaller than or equal to the size threshold into a small-size depth image block group;

performing quadtree segmentation, trigeminal tree segmentation and binary tree segmentation on the secondary depth image blocks in the large-size depth image block group in sequence to obtain an upper depth image block group;

Sequentially performing binary tree segmentation, trigeminal tree segmentation and quadtree segmentation on secondary depth tiles in the small-size depth tile group to obtain a lower depth tile group;

sequentially converging the upper depth tile group and the lower depth tile group into a target depth tile sequence, and converging all the target depth tile sequences into a secondary depth tile sequence set;

and collecting all the depth tiles with the same sequence numbers in the secondary depth tile sequence set into secondary main board depth tile groups, and collecting all the secondary main board depth tile groups into secondary main board depth tile group sets.

Optionally, the sequentially performing multistage filtering and depth equalization operations on the split motherboard depth image to obtain a standard motherboard depth image, including:

carrying out Gaussian smoothing filtering on the spliced mainboard depth picture to obtain a Gaussian filtering depth picture;

carrying out bilateral filtering on the Gaussian filtering depth picture to obtain a bilateral filtering depth picture;

and generating a depth histogram of the bilateral filtering depth picture, and performing depth equalization operation on the bilateral filtering depth picture by using the depth histogram to obtain a standard mainboard depth picture.

Optionally, the sequentially performing image segmentation and multi-type edge fitting operations on the standard main board depth image to obtain a fitted main board depth image, including:

performing picture segmentation operation on the standard main board depth picture to obtain a segmented main board depth picture;

performing edge detection on the segmented main board depth picture to obtain a main board edge set;

performing multi-type convolution filtering on the main board edge set, and splitting the main board edge set into a linear edge set and a curve edge set according to a filtering result of the multi-type convolution filtering;

and performing linear fitting on each linear edge in the linear edge set by using the following weighted fitting algorithm to obtain a fitted linear edge set:

wherein epsilon refers to a fitting error, R refers to the R-th pixel point corresponding to the straight line edge, R refers to the total number of the pixel points corresponding to the straight line edge, and w _r Refer to the clipping weight of the r pixel point, ρ and σ are the slope coefficient of the linear edge after fitting, the displacement coefficient of the linear edge after γ fitting, and a _r Refers to the abscissa, b, of the (r) th pixel point corresponding to the straight line edge _r The ordinate of the r pixel point corresponding to the straight line edge is referred, and lambda refers to the fitting coefficient of the weighted fitting algorithm;

Performing curve fitting on each curve edge in the curve edge set by using a least square method to obtain a fitted curve edge set;

and collecting the fitting straight line edge set and the fitting curve edge set into a fitting edge set, and carrying out edge updating on the segmentation master plate depth picture according to the fitting edge set to obtain a fitting master plate depth picture.

Optionally, the fitting, positioning and labeling the fitted main board depth image by using a cluster matching algorithm to obtain a labeled main board depth image, including:

extracting fitting features of the fitting main board depth pictures to obtain fitting feature sets;

calculating the clustering distance between each bonding feature in the bonding feature set by using the following clustering distance algorithm, and carrying out feature clustering according to the clustering distance to obtain a bonding feature class set:

wherein G meansThe clustering distance, T, refers to the total number of features in the fit features, T refers to the T-th feature in the fit features, arccos is an arccosine function symbol, Q refers to the Q-th dimension feature, Q refers to the total number of feature dimensions of each feature in the fit features,the q-th dimension characteristic of the t-th characteristic of the fitting characteristic S is +. >A q-th dimension feature of a t-th feature of the fitting feature P;

selecting the laminating feature types in the laminating feature type set one by one as target laminating feature types, and taking the clustering center of the target laminating feature types as a target laminating center feature;

and matching the labeling features corresponding to the target laminating center features from a preset labeling feature library to serve as target labeling features of the target laminating feature class, and carrying out laminating, positioning and labeling on the fitting main board depth picture by utilizing all the target labeling features to obtain a labeling main board depth picture.

In order to solve the above problems, the present invention further provides a bonding and positioning device applied to an MPIS industrial control motherboard, the device includes:

the depth shooting module is used for carrying out ultrasonic ranging on the MPIS industrial control main board on the laminating workbench to obtain a primary distance interval, carrying out multi-stage focusing on a depth camera by utilizing the primary distance interval, and carrying out multi-focus shooting on the MPIS industrial control main board by utilizing the focused depth camera to obtain a primary main board depth atlas;

the image blocking module is used for carrying out distortion calibration correction on the primary main board depth atlas to obtain a secondary main board depth atlas, and carrying out multi-type image blocking operation on the secondary main board depth atlas to obtain a secondary main board depth image block set;

The block screening module is configured to screen a standard secondary depth block set from the secondary main board depth block set by using a preset mean value definition algorithm, splice the standard secondary depth block set into a spliced main board depth picture, and sequentially perform multistage filtering and depth equalization operation on the spliced main board depth picture to obtain a standard main board depth picture, where the screen the standard secondary depth block set from the secondary main board depth block set by using the preset mean value definition algorithm includes: selecting secondary main board depth block groups in the secondary main board depth block group one by one as target depth block groups, and selecting depth blocks in the target depth block groups one by one as target depth blocks; filtering and smoothing the target depth image block to obtain a target filtering image block, and generating a target gray image block according to the target depth image block and the target filtering image block; calculating the definition of each target gray scale tile by using the following mean definition algorithm:

The fitting labeling module is used for sequentially carrying out picture segmentation and multi-type edge fitting operation on the standard main board depth picture to obtain a fitting main board depth picture, and carrying out fitting positioning labeling on the fitting main board depth picture by using a clustering matching algorithm to obtain a labeling main board depth picture;

and the positioning and laminating module is used for selecting laminating marking areas in the depth pictures of the marking main board one by one as target laminating marking areas, carrying out point cloud reconstruction on the target laminating marking areas to obtain target laminating marking point clouds, extracting target laminating coordinates from the target laminating marking point clouds, and utilizing the target laminating coordinates to laminate the MPIS industrial control main board.

In order to solve the above problems, the present invention also provides an apparatus comprising:

at least one processor; the method comprises the steps of,

a memory communicatively coupled to the at least one processor; wherein,

the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor, so that the at least one processor can execute the above bonding positioning method applied to the MPIS industrial control main board.

In order to solve the above-mentioned problems, the present invention further provides a medium, in which at least one computer program is stored, where the at least one computer program is executed by a processor in a device to implement the above-mentioned lamination positioning method applied to an MPIS industrial control motherboard.

According to the embodiment of the invention, the primary distance interval is utilized to carry out multi-stage focusing on the depth camera, the focused depth camera is utilized to carry out multi-focus shooting on the MPIS industrial control mainboard to obtain a primary mainboard depth image set, a mainboard depth image covering the whole focal section of the MPIS industrial control mainboard can be obtained, the clear images of all parts of the MPIS industrial control mainboard can be obtained after the subsequent image blocks are spliced, the primary mainboard depth image set is subjected to distortion calibration correction to obtain a secondary mainboard depth image set, imaging errors caused by the distortion of a depth camera lens can be reduced, a more accurate depth image is obtained, the secondary mainboard depth image set is subjected to multi-type image blocking operation to obtain a secondary mainboard depth image set, the mainboard depth image can be split into an image set of a plurality of areas, image subdivision is realized, the picture subdivision is also convenient, the picture with the clearest area is conveniently selected to splice, the standard secondary depth image set is spliced into a spliced depth image, the areas in the spliced depth image are all in the focus center, the depth image can be sequentially improved, the depth image can be subjected to depth equalization operation on the mainboard depth image, the depth image can be subjected to depth equalization, and the depth image of the mainboard can be removed, and the depth image depth of the depth image of the standard is balanced;

Through carrying out picture segmentation and multi-type edge fitting operation to standard mainboard depth picture, obtain the mainboard depth picture of fitting, can get rid of in the mainboard depth picture because the edge blurring phenomenon that illumination reflection caused, improve the degree of accuracy of laminating location, through utilizing cluster matching algorithm to the mainboard depth picture of fitting carries out laminating location mark, obtains annotating mainboard depth picture, can mark laminating kind and the degree of depth position information that need laminate the position in the MPIS industrial control mainboard to make things convenient for follow-up location laminating, through follow target laminating mark point cloud draws the target laminating coordinate, and utilize the target laminating coordinate to laminate in the MPIS industrial control mainboard, can directly rely on the position coordinate and the laminating type of the regional that need laminate everywhere on the point cloud data acquisition MPIS industrial control mainboard, can improve the efficiency of laminating. Therefore, the bonding and positioning method, the bonding and positioning device, the bonding and positioning equipment and the medium applied to the MPIS industrial control main board can solve the problem of lower accuracy in bonding and positioning of the MPIS industrial control main board.

Drawings

Fig. 1 is a flow chart of a bonding positioning method applied to an MPIS industrial control motherboard according to an embodiment of the present invention;

FIG. 2 is a flowchart illustrating a multi-type image segmentation according to an embodiment of the present application;

fig. 3 is a flowchart illustrating a process of generating a standard motherboard depth image according to an embodiment of the present application;

FIG. 4 is a functional block diagram of a bonding and positioning device applied to an MPIS industrial control motherboard according to an embodiment of the present application;

fig. 5 is a schematic structural diagram of an apparatus for implementing the bonding and positioning method applied to an MPIS industrial control motherboard according to an embodiment of the present application.

The achievement of the objects, functional features and advantages of the present application will be further described with reference to the accompanying drawings, in conjunction with the embodiments.

Detailed Description

It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

The embodiment of the application provides a fitting and positioning method applied to an MPIS industrial control main board. The execution main body of the laminating and positioning method applied to the MPIS industrial control main board comprises at least one of equipment which can be configured to execute the method provided by the embodiment of the application, such as a server side, a terminal and the like. In other words, the attaching and positioning method applied to the MPIS industrial control main board may be executed by software or hardware installed in a terminal device or a server device, where the software may be a blockchain platform. The service end includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like. The server may be an independent server, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

Referring to fig. 1, a flow chart of a bonding and positioning method applied to an MPIS industrial control motherboard according to an embodiment of the present invention is shown. In this embodiment, the attaching and positioning method applied to the MPIS industrial control motherboard includes:

s1, performing ultrasonic ranging on an MPIS industrial control main board on a bonding workbench to obtain a primary distance section, performing multi-stage focusing on a depth camera by using the primary distance section, and performing multi-focus shooting on the MPIS industrial control main board by using the focused depth camera to obtain a primary main board depth map set.

In the embodiment of the invention, the bonding workbench is an operation workbench for positioning and bonding a main board, the bonding workbench can fix the main board to be bonded when the main board is bonded, the MPIS industrial control main board is an industrial control main board adopting a microprocessor (Microprocessor without interlocked piped stages architecture, MPIS for short) framework without an internal interlocking pipeline level, wherein the MPIS framework is a processor framework adopting a simplified instruction set and comprises a large number of registers, instruction numbers, characters and visible pipeline delay time slots, the characteristics enable the MIPS framework to provide the highest performance per square millimeter and the lowest energy consumption in the current System On Chip (SOC) design, and the industrial control main board is a main board applied to industrial occasions, is adopted by industrial computers, can adapt to a wide-temperature environment according to requirements, can adapt to severe environments, can work under high load for a long time and the like.

In the embodiment of the present invention, the ultrasonic ranging is performed on the MPIS industrial control motherboard on the bonding workbench to obtain a primary distance interval, including:

In detail, the types of the ultrasonic generator and the ultrasonic receiver may be ST-822 ultrasonic distance meter or MS6450 ultrasonic distance meter, the main board distance interval refers to a distance interval between the closest edge and the farthest edge of the ultrasonic generator to the MPIS industrial control main board, the main board distance measurement angle interval refers to an angle interval between the angle of the closest edge and the farthest edge of the ultrasonic generator to the MPIS industrial control main board, the shortest camera distance refers to a shortest distance between the ultrasonic generator and the depth camera, the shortest camera distance measurement angle refers to an angle corresponding to the shortest distance between the ultrasonic generator and the depth camera, the camera main board angle interval refers to an angle interval between the two edges of the depth camera to the MPIS industrial control main board, and the primary distance interval refers to a distance interval between the shortest distance and the longest distance between the depth camera to the MPIS industrial control main board, wherein the depth camera may be a binocular depth camera.

In detail, because the area of the MPIS industrial control main board is large, the noise part is out of focus in the process of taking the depth picture of the main board, so that the shortest distance and the nearest distance between the depth camera and the MPIS industrial control main board need to be determined, and the MPIS industrial control main board is taken in full focus Duan Fugai.

In an embodiment of the present invention, the performing multi-stage focusing on the depth camera by using the primary distance interval includes:

acquiring a focal length parameter of a depth camera, and calculating a primary image distance interval of the depth camera according to the focal length parameter and the primary distance interval;

extracting a primary image distance set from the primary image distance interval by using a preset image distance window;

and selecting primary image distances in the primary image distance set one by one as real-time image distances of the depth camera for focusing.

In detail, the focal length parameter refers to a default focal length of the depth camera, the calculating the primary image distance interval of the depth camera according to the focal length parameter and the primary distance interval refers to subtracting the reciprocal of the primary distance interval from the reciprocal of the focal length parameter to obtain a primary image distance reciprocal interval, and taking the reciprocal of the primary image distance reciprocal interval as the primary image distance interval.

Specifically, the extracting the primary image distance set from the primary image distance interval by using the preset image distance window refers to extracting a primary image distance from the primary image distance interval every other image distance window length to form the primary image distance set.

In detail, performing multi-focus shooting on the MPIS industrial control main board by using the depth camera after focusing, and obtaining a primary main board depth atlas refers to performing multi-stage focusing on the depth camera according to the primary image distance set and shooting a primary main board depth atlas corresponding to each primary image distance in the primary image distance set.

In the embodiment of the invention, the primary distance interval is utilized to carry out multi-stage focusing on the depth camera, and the focused depth camera is utilized to carry out multi-focus shooting on the MPIS industrial control mainboard, so that a primary mainboard depth atlas is obtained, a mainboard depth picture covering the whole focal section of the MPIS industrial control mainboard can be obtained, and the clear picture of each part of the MPIS industrial control mainboard can be obtained after the subsequent picture blocks are spliced.

S2, performing distortion calibration correction on the primary main board depth atlas to obtain a secondary main board depth atlas, and performing multi-type image blocking operation on the secondary main board depth atlas to obtain a secondary main board depth image block set.

In the embodiment of the invention, in the process of shooting the depth picture, the picture distortion caused by the curvature of the lens of the depth camera exists in the depth camera, so that the primary main board depth atlas is required to be subjected to distortion calibration and correction, and the accuracy of subsequent positioning and laminating is improved.

In the embodiment of the present invention, the performing distortion calibration correction on the primary main board depth atlas to obtain a secondary main board depth atlas includes:

In detail, the first calibration parameter and the second calibration parameter are obtained by calibrating the depth camera by using a calibration plate in advance, and the calibration internal reference is a lens distortion internal reference of the depth camera.

In the embodiment of the invention, the target secondary depth picture corresponding to the target primary depth picture is calculated by using the calibration correction algorithm, so that the distortion of the main board depth picture caused by the distortion of the lens of the depth camera can be calibrated, and the main board depth picture with more accurate position information can be obtained.

In detail, referring to fig. 2, the performing the multi-type image blocking operation on the secondary motherboard depth atlas to obtain a secondary motherboard depth atlas set includes:

s21, selecting pictures in the secondary main board depth map set one by one as target secondary depth pictures, and dividing the target secondary depth pictures into a plurality of secondary depth image blocks by using a quadtree;

s22, acquiring the picture sizes of the secondary depth image blocks one by one, converging the secondary depth image blocks with the picture sizes larger than a preset size threshold into a large-size depth image block group, and converging the secondary depth image blocks with the picture sizes smaller than or equal to the size threshold into a small-size depth image block group;

s23, performing quadtree segmentation, trigeminal tree segmentation and binary tree segmentation on the secondary depth image blocks in the large-size depth image block group in sequence to obtain an upper depth image block group;

s24, sequentially carrying out binary tree segmentation, trigeminal tree segmentation and quadtree segmentation on the secondary depth image blocks in the small-size depth image block group to obtain a lower depth image block group;

s25, sequentially converging the upper-level depth block group and the lower-level depth block group into a target depth block sequence, and converging all the target depth block sequences into a secondary depth block sequence set;

S26, collecting all depth tiles with the same sequence numbers in the secondary depth tile sequence set into secondary main board depth tile groups, and collecting all secondary main board depth tile groups into secondary main board depth tile group sets.

Specifically, the quadtree segmentation refers to the segmentation of an image into four regions of the same size, the trigeminal tree segmentation refers to the segmentation of an image into three regions of the same size, and the binary tree segmentation refers to the segmentation of an image into two regions of the same size.

In the embodiment of the invention, the secondary main board depth atlas is obtained by carrying out distortion calibration correction on the primary main board depth atlas, imaging errors caused by distortion of a depth camera lens can be reduced, a more accurate depth image is obtained, and a secondary main board depth image block set is obtained by carrying out multi-type image blocking operation on the secondary main board depth atlas, so that a main board depth image can be split into image block sets of a plurality of areas, image subdivision is realized, and the most clear images of all areas can be conveniently selected for splicing.

S3, screening a standard secondary depth image block set from the secondary main board depth image block set by using a preset mean value definition algorithm, splicing the standard secondary depth image block set into a spliced main board depth image, and sequentially performing multistage filtering and depth equalization operation on the spliced main board depth image to obtain a standard main board depth image.

In this embodiment, the standard word set depth tile set is a graph set formed by the most clear secondary main board depth tiles in each secondary main board depth tile set in the secondary main board depth tile set.

In the embodiment of the present invention, the screening the standard secondary depth tile set from the secondary main board depth tile set by using a preset mean value definition algorithm includes:

selecting secondary main board depth block groups in the secondary main board depth block group one by one as target depth block groups, and selecting depth blocks in the target depth block groups one by one as target depth blocks;

filtering and smoothing the target depth image block to obtain a target filtering image block, and generating a target gray image block according to the target depth image block and the target filtering image block;

calculating the definition of each target gray scale tile by using the following mean definition algorithm:

wherein L refers to the definition, M refers to the total number of horizontal pixels of the target gray tile, N refers to the total number of vertical pixels of the target gray tile, i refers to the ith horizontal pixel, j refers to the jth vertical pixel, α is a preset horizontal sampling length, β is a preset vertical sampling length, |·| is an absolute value function, g () is a gray value function, g (i+α, j+β) refers to the gray value of the pixel with coordinates (i+α, j+β) in the target gray tile, g (i, j) refers to the gray value of the pixel with coordinates (i, j) in the target gray tile, e is a natural logarithmic sign, o is an imaginary sign, u, v is a transform coefficient of a fast fourier formula;

And selecting a target depth image block corresponding to the target gray image block with the maximum definition from each target depth image block group as a standard secondary depth image block, and collecting all the standard secondary depth image blocks into a standard secondary depth image block set.

In detail, by calculating the definition of each target gray scale block by using the mean value definition algorithm, edge detection can be performed in the transverse and longitudinal directions to obtain the gray scale edge sharpening condition of each block, and the definition of each block is further determined by combining the Fourier spectrum of each block, so that the accuracy of the definition is improved.

In detail, the splicing the standard secondary depth tile sets into the spliced main board depth picture refers to performing multi-type image blocking operation on the secondary main board depth tile sets according to the above-mentioned step S2, and performing the splicing in the reverse direction of the method for obtaining the secondary main board depth tile sets to obtain the spliced main board depth picture.

Specifically, referring to fig. 3, the sequentially performing multistage filtering and depth equalization operations on the split main board depth image to obtain a standard main board depth image includes:

S31, carrying out Gaussian smoothing filtering on the spliced mainboard depth pictures to obtain Gaussian filtering depth pictures;

s32, carrying out bilateral filtering on the Gaussian filter depth picture to obtain a bilateral filter depth picture;

and S33, generating a depth histogram of the bilateral filtering depth picture, and performing depth equalization operation on the bilateral filtering depth picture by using the depth histogram to obtain a standard main board depth picture.

In detail, the gaussian filter depth picture may be subjected to bilateral filtering by using a Gabor filter to obtain a bilateral filter depth picture, and the depth histogram is used to perform a depth equalization operation on the bilateral filter depth picture, so as to obtain a standard motherboard depth picture, which means performing a probability equalization operation on a depth level of the bilateral filter depth picture.

In the embodiment of the invention, the standard secondary depth image block set is spliced into the spliced main board depth image, so that all areas of the main board in the spliced main board depth image are ensured to be in the focusing center, the details of the image are improved, the standard main board depth image is obtained by sequentially carrying out multistage filtering and depth equalization operation on the spliced main board depth image, the noise of the depth image can be removed, the depth data of the MPIS industrial control main board is ensured to be in a depth plane, and the attaching accuracy is improved.

And S4, sequentially carrying out image segmentation and multi-type edge fitting operation on the standard main plate depth image to obtain a fitted main plate depth image, and carrying out fitting, positioning and labeling on the fitted main plate depth image by using a clustering matching algorithm to obtain a labeled main plate depth image.

In the embodiment of the present invention, the sequentially performing image segmentation and multi-type edge fitting operations on the standard main board depth image to obtain a fitted main board depth image includes:

In detail, the weighted fitting algorithm is utilized to perform straight line fitting on each straight line edge in the straight line edge set to obtain a fitted straight line edge set, so that the problem of overlarge fitting error caused by the distance relation of edge pixel points in the fitting process can be reduced, and the fitting accuracy is improved.

Specifically, the fitting main board depth image is subjected to fitting, positioning and labeling by using a cluster matching algorithm to obtain a labeled main board depth image, and the method comprises the following steps:

wherein G is the clustering distance, T is the total number of features in the fit features, T is the T-th feature in the fit features, arccos is an arccos function symbol, Q is the Q-th dimension feature, Q is the total number of feature dimensions of each feature in the fit features,the q-th dimension characteristic of the t-th characteristic of the fitting characteristic S is +.>A q-th dimension feature of a t-th feature of the fitting feature P;

Specifically, the fitting of the depth pictures of the main board to obtain the fitting feature set means that the fitting of the depth pictures of the main board is convolved by using a plurality of convolution layers, the convolved results are gathered into the fitting feature set, and the K-means clustering algorithm can be used for carrying out feature clustering according to the clustering distance to obtain the fitting feature class set.

In detail, the clustering distance algorithm is utilized to calculate the clustering distance between each fitting feature in the fitting feature set, so that the matching of a plurality of features in multiple dimensions can be realized, the characterization of the clustering distance is improved, and the accuracy of fitting positioning labeling is further improved.

According to the embodiment of the invention, the fitting main board depth image is obtained by carrying out image segmentation and multi-type edge fitting operation on the standard main board depth image, the edge blurring phenomenon caused by illumination reflection in the main board depth image can be removed, the fitting positioning accuracy is improved, the fitting positioning marking is carried out on the fitting main board depth image by utilizing a clustering matching algorithm to obtain the marked main board depth image, and the fitting type and depth position information of the part needing fitting in the MPIS industrial control main board can be marked, so that the subsequent positioning fitting is facilitated.

S5, selecting the attaching marking areas in the depth pictures of the marking main board one by one as target attaching marking areas, reconstructing the target attaching marking areas to obtain target attaching marking point clouds, extracting target attaching coordinates from the target attaching marking point clouds, and attaching the MPIS industrial control main board by utilizing the target attaching coordinates.

In the embodiment of the invention, the step of reconstructing the point cloud of the target attaching and marking area to obtain the target attaching and marking point cloud refers to the step of converting the target attaching and marking area into a primary attaching and marking point cloud according to the camera internal parameter of the depth camera and the depth value of each pixel point of the target attaching and marking area, and the step of converting the primary attaching and marking point cloud into a world coordinate system according to the external parameter of the depth camera to obtain the target attaching and marking point cloud.

In detail, the applying the target laminating coordinate to the MPIS industrial control main board means that the laminating robot is controlled to laminate according to the target laminating coordinate.

According to the embodiment of the invention, the target laminating coordinate is extracted from the target laminating marking point cloud, and the MPIS industrial control main board is laminated by utilizing the target laminating coordinate, so that the position coordinates and the laminating types of areas needing to be laminated on each part of the MPIS industrial control main board can be directly obtained by means of point cloud data, and the laminating efficiency can be improved.

Through carrying out picture segmentation and multi-type edge fitting operation to standard mainboard depth picture, obtain the mainboard depth picture of fitting, can get rid of in the mainboard depth picture because the edge blurring phenomenon that illumination reflection caused, improve the degree of accuracy of laminating location, through utilizing cluster matching algorithm to the mainboard depth picture of fitting carries out laminating location mark, obtains annotating mainboard depth picture, can mark laminating kind and the degree of depth position information that need laminate the position in the MPIS industrial control mainboard to make things convenient for follow-up location laminating, through follow target laminating mark point cloud draws the target laminating coordinate, and utilize the target laminating coordinate to laminate in the MPIS industrial control mainboard, can directly rely on the position coordinate and the laminating type of the regional that need laminate everywhere on the point cloud data acquisition MPIS industrial control mainboard, can improve the efficiency of laminating. Therefore, the attaching and positioning method applied to the MPIS industrial control main board can solve the problem of lower accuracy in the process of attaching and positioning the MPIS industrial control main board.

Fig. 4 is a functional block diagram of a bonding positioning device applied to an MPIS industrial control motherboard according to an embodiment of the present invention.

The attaching and positioning device 100 applied to the MPIS industrial control main board can be installed in equipment. According to the realized functions, the fitting and positioning device 100 applied to the MPIS industrial control motherboard may include a depth shooting module 101, an image blocking module 102, a tile screening module 103, a fitting and labeling module 104, and a positioning and fitting module 105. The module of the invention, which may also be referred to as a unit, refers to a series of computer program segments, which are stored in the memory of the device, capable of being executed by the processor of the device and of performing a fixed function.

In the present embodiment, the functions concerning the respective modules/units are as follows:

the depth shooting module 101 is configured to perform ultrasonic ranging on an MPIS industrial control motherboard on a bonding workbench to obtain a primary distance interval, perform multi-stage focusing on a depth camera by using the primary distance interval, and perform multi-focus shooting on the MPIS industrial control motherboard by using the focused depth camera to obtain a primary motherboard depth atlas;

the image blocking module 102 is configured to perform distortion calibration correction on the primary main board depth atlas to obtain a secondary main board depth atlas, and perform multi-type image blocking operation on the secondary main board depth atlas to obtain a secondary main board depth image block set;

The tile screening module 103 is configured to screen a standard secondary depth tile set from the secondary main board depth tile set by using a preset mean value definition algorithm, splice the standard secondary depth tile set into a spliced main board depth image, and sequentially perform multistage filtering and depth equalization operations on the spliced main board depth image to obtain a standard main board depth image, where the screen the standard secondary depth tile set from the secondary main board depth tile set by using the preset mean value definition algorithm includes: selecting secondary main board depth block groups in the secondary main board depth block group one by one as target depth block groups, and selecting depth blocks in the target depth block groups one by one as target depth blocks; filtering and smoothing the target depth image block to obtain a target filtering image block, and generating a target gray image block according to the target depth image block and the target filtering image block; calculating the definition of each target gray scale tile by using the following mean definition algorithm:

The fitting and labeling module 104 is configured to sequentially perform image segmentation and multi-type edge fitting operations on the standard main board depth image to obtain a fitted main board depth image, and perform fitting, positioning and labeling on the fitted main board depth image by using a cluster matching algorithm to obtain a labeled main board depth image;

the positioning and laminating module 105 is configured to select a laminating labeling area in the labeling motherboard depth image one by one as a target laminating labeling area, reconstruct a point cloud of the target laminating labeling area to obtain a target laminating labeling point cloud, extract target laminating coordinates from the target laminating labeling point cloud, and utilize the target laminating coordinates to laminate the MPIS industrial control motherboard.

In detail, each module in the bonding and positioning device 100 applied to the MPIS industrial control motherboard in the embodiment of the present invention adopts the same technical means as the bonding and positioning method applied to the MPIS industrial control motherboard in the above-mentioned fig. 1 to 3, and can produce the same technical effects, which are not repeated here.

Fig. 5 is a schematic structural diagram of an apparatus for implementing a bonding positioning method applied to an MPIS industrial control motherboard according to an embodiment of the present invention.

The device 1 may comprise a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may further comprise a computer program stored in the memory 11 and executable on the processor 10, such as a laminating and positioning program applied to an MPIS industrial control motherboard.

The processor 10 may be formed by an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be formed by a plurality of integrated circuits packaged with the same function or different functions, including one or more central processing units (Central Processing unit, CPU), a microprocessor, a digital processing chip, a graphics processor, a combination of various control chips, and so on. The processor 10 is a Control Unit (Control Unit) of the apparatus, connects various parts of the entire apparatus using various interfaces and lines, and executes various functions of the apparatus and processes data by running or executing programs or modules stored in the memory 11 (for example, executing a bonding positioning program applied to an MPIS industrial Control main board, etc.), and calling data stored in the memory 11.

The memory 11 includes at least one type of medium including flash memory, a removable hard disk, a multimedia card, a card type memory (e.g., SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, etc. The memory 11 may in some embodiments be an internal storage unit of the device, such as a removable hard disk of the device. The memory 11 may in other embodiments also be an external storage device of the device, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card) or the like, which are provided on the device. Further, the memory 11 may also include both an internal storage unit and an external storage device of the device. The memory 11 may be used to store not only application software installed in the device and various data, such as codes of a bonding positioning program applied to an MPIS industrial control motherboard, but also temporarily store data that has been output or is to be output.

The communication bus 12 may be a peripheral component interconnect standard (peripheral component interconnect, PCI) bus, or an extended industry standard architecture (extended industry standard architecture, EISA) bus, among others. The bus may be classified as an address bus, a data bus, a control bus, etc. The bus is arranged to enable a connection communication between the memory 11 and at least one processor 10 etc.

The communication interface 13 is used for communication between the above-mentioned devices and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and/or a wireless interface (e.g., WI-FI interface, bluetooth interface, etc.), typically used to establish a communication connection between the device and other devices. The user interface may be a Display (Display), an input unit such as a Keyboard (Keyboard), or alternatively a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, or the like. The display may also be referred to as a display screen or display unit, as appropriate, for displaying information processed in the device and for displaying a visual user interface.

Only devices having components are shown, and it will be understood by those skilled in the art that the structures shown in the figures are not limiting of the devices and may include fewer or more components than shown, or some combination of components, or a different arrangement of components.

For example, although not shown, the apparatus may further include a power source (such as a battery) for supplying power to the respective components, and preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that functions of charge management, discharge management, power consumption management, etc. are implemented through the power management device. The power supply may also include one or more of any of a direct current or alternating current power supply, recharging device, power failure detection circuit, power converter or inverter, power status indicator, etc. The device may also include various sensors, bluetooth modules, wi-Fi modules, etc., which are not described in detail herein.

It should be understood that the embodiments described are for illustrative purposes only and are not limited to this configuration in the scope of the patent application.

The attaching and positioning program stored in the memory 11 of the device 1 and applied to the MPIS industrial control motherboard is a combination of a plurality of instructions, and when running in the processor 10, it can be implemented:

In particular, the specific implementation method of the above instructions by the processor 10 may refer to the description of the relevant steps in the corresponding embodiment of the drawings, which is not repeated herein.

Further, the modules/units integrated in the device 1 may be stored in a medium if implemented in the form of software functional units and sold or used as separate products. The medium may be volatile or nonvolatile. For example, the medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM).

The invention also provides a medium storing a computer program which, when executed by a processor of a device, may implement:

In the several embodiments provided in the present invention, it should be understood that the disclosed apparatus, device and method may be implemented in other manners. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be other manners of division when actually implemented.

The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional module in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units can be realized in a form of hardware or a form of hardware and a form of software functional modules.

It will be evident to those skilled in the art that the application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.

The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.

The embodiment of the application can acquire and process the related data based on the artificial intelligence technology. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results.

Furthermore, it is evident that the word "comprising" does not exclude other elements or steps, and that the singular does not exclude a plurality. Multiple units or means as set forth in the system embodiments may also be implemented by one unit or means in software or hardware. The terms first, second, etc. are used to denote a name, but not any particular order.

Finally, it should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made to the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. The attaching and positioning method applied to the MPIS industrial control main board is characterized by comprising the following steps of:

s1: performing ultrasonic ranging on an MPIS industrial control main board on a bonding workbench to obtain a primary distance interval, performing multi-stage focusing on a depth camera by using the primary distance interval, and performing multi-focus shooting on the MPIS industrial control main board by using the focused depth camera to obtain a primary main board depth atlas;

s2: performing distortion calibration correction on the primary main board depth atlas to obtain a secondary main board depth atlas, and performing multi-type image blocking operation on the secondary main board depth atlas to obtain a secondary main board depth image block set;

s3: screening a standard secondary depth image block set from the secondary main board depth image block set by using a preset mean value definition algorithm, splicing the standard secondary depth image block set into a spliced main board depth image, and sequentially performing multistage filtering and depth equalization operation on the spliced main board depth image to obtain a standard main board depth image, wherein the screening the standard secondary depth image block set from the secondary main board depth image block set by using the preset mean value definition algorithm comprises the following steps:

S31: selecting secondary main board depth block groups in the secondary main board depth block group one by one as target depth block groups, and selecting depth blocks in the target depth block groups one by one as target depth blocks;

s32: filtering and smoothing the target depth image block to obtain a target filtering image block, and generating a target gray image block according to the target depth image block and the target filtering image block;

s33: calculating the definition of each target gray scale tile by using the following mean definition algorithm:

S34: selecting a target depth image block corresponding to a target gray image block with the maximum definition from each target depth image block group as a standard secondary depth image block, and collecting all the standard secondary depth image blocks into a standard secondary depth image block set;

s4: sequentially carrying out image segmentation and multi-type edge fitting operation on the standard main plate depth image to obtain a fitted main plate depth image, and carrying out fitting positioning labeling on the fitted main plate depth image by using a clustering matching algorithm to obtain a labeled main plate depth image;

s5: selecting a fitting marking area in the marking mainboard depth picture one by one as a target fitting marking area, reconstructing point cloud of the target fitting marking area to obtain target fitting marking point cloud, extracting target fitting coordinates from the target fitting marking point cloud, and fitting the MPIS industrial control mainboard by utilizing the target fitting coordinates;

the step of sequentially carrying out image segmentation and multi-type edge fitting operation on the standard main board depth image to obtain a fitting main board depth image comprises the following steps:

collecting the fitting straight line edge set and the fitting curve edge set into a fitting edge set, and carrying out edge updating on the segmentation master plate depth picture according to the fitting edge set to obtain a fitting master plate depth picture;

The fitting main board depth image is subjected to fitting, positioning and labeling by using a clustering matching algorithm to obtain a labeled main board depth image, and the method comprises the following steps:

wherein G is the clustering distance, T is the total number of features in the fit features, T is the T-th feature in the fit features, arccos is an arccos function symbol, Q is the Q-th dimension feature, Q is the total number of feature dimensions of each feature in the fit features,refers to the q-th dimension characteristic of the t-th characteristic of the lamination characteristic S, P _t ^q A q-th dimension feature of a t-th feature of the fitting feature P;

2. The bonding positioning method applied to the MPIS industrial control main board according to claim 1, wherein the ultrasonic ranging is performed on the MPIS industrial control main board on the bonding workbench to obtain a primary distance interval, and the method comprises the following steps:

3. The fitting and positioning method applied to an MPIS industrial control motherboard according to claim 1, wherein the performing distortion calibration correction on the primary motherboard depth atlas to obtain a secondary motherboard depth atlas includes:

wherein,is the pixel abscissa of the target secondary depth picture,/i>Refers to the pixel ordinate of the target secondary depth picture, _x refers to the pixel abscissa of the target primary depth picture, y refers to the pixel ordinate of the target primary depth picture, (x, y) refers to the pixel with the coordinate point (x, y) in the target primary depth picture, _δ refers to the coordinate value range of the target primary depth picture, h ₁ Is the first term of Taylor series expansion, h ₂ The second term of Taylor series expansion, d refers to the calibrated internal reference of the depth camera, k ₁ Is the first calibration parameter, k, of the calibration correction algorithm ₂ The second calibration parameter of the calibration correction algorithm is referred to;

4. The fitting and positioning method applied to an MPIS industrial control motherboard according to claim 1, wherein the performing a multi-type image blocking operation on the secondary motherboard depth atlas to obtain a secondary motherboard depth atlas set includes:

5. The fitting and positioning method applied to an MPIS industrial control motherboard according to claim 1, wherein the sequentially performing multistage filtering and depth equalization operations on the split motherboard depth image to obtain a standard motherboard depth image includes:

6. A bonding positioning device applied to an MPIS industrial control motherboard, for implementing the bonding positioning method applied to an MPIS industrial control motherboard according to any one of claims 1 to 5, wherein the device comprises:

7. An electronic device, the electronic device comprising:

at least one processor; the method comprises the steps of,

a memory communicatively coupled to the at least one processor; wherein,

the memory stores a computer program executable by the at least one processor, the computer program being executable by the at least one processor to enable the at least one processor to perform the bonding positioning method applied to the MPIS industrial control motherboard according to any one of claims 1 to 5.

8. A computer readable storage medium storing a computer program, wherein the computer program when executed by a processor implements the bonding positioning method applied to an MPIS industrial control motherboard according to any one of claims 1 to 5.