CN117649409A

CN117649409A - Automatic limiting system, method, device and medium for sliding table based on machine vision

Info

Publication number: CN117649409A
Application number: CN202410120035.2A
Authority: CN
Inventors: 张贺兴; 陈权; 施锐; 蔡碧良
Original assignee: Ruan Electronics Shenzhen Co ltd
Current assignee: Ruan Electronics Shenzhen Co ltd
Priority date: 2024-01-29
Filing date: 2024-01-29
Publication date: 2024-03-05
Anticipated expiration: 2044-01-29
Also published as: CN117649409B

Abstract

The invention relates to the technical field of slipway limiting, and discloses a slipway independent limiting system, a slipway independent limiting method, a slipway independent limiting device and a slipway independent limiting medium based on machine vision, wherein the slipway independent limiting system comprises a point cloud reconstruction module, a semantic segmentation module, a route identification module, a motion monitoring module and a pre-aiming limiting module, wherein the point cloud reconstruction module comprises a point cloud detection module, a semantic segmentation module and a pre-aiming limiting module, wherein the point cloud detection module comprises a point cloud detection module, a point cloud detection module and a pre-aiming limiting module, and the point cloud detection module comprises a point cloud detection module and a point cloud detection module comprises a point cloud detection module, and a point cloud detection module comprises the point cloud detection module: the point cloud reconstruction module is used for collecting a sliding table system drawing set and a sliding table system point cloud set and constructing a sliding table system model; the semantic segmentation module is used for extracting a sliding table part block group set and a sliding table part semantic group from the sliding table system diagram set; the route identification module is used for extracting a sliding table route model from the sliding table system model according to the sliding table component block assembly set; the motion monitoring module is used for detecting a moving target and removing noise from the real-time sliding table graph sequence set to obtain a moving part graph block sequence set; and the pre-aiming limiting module is used for mapping and pre-aiming the sliding table route model and limiting the sliding table according to the moving part block sequence group set. The invention can improve the accuracy of limiting the sliding table.

Description

Automatic limiting system, method, device and medium for sliding table based on machine vision

Technical Field

The invention relates to the technical field of slipway limiting, in particular to an autonomous slipway limiting system, method, device and medium based on machine vision.

Background

The sliding table is a mechanical structure, is a sliding platform or a carrier which is generally parallel to one direction, is designed to translate or slide along a track or a guide rail in a motion system, is widely applied to the fields of industrial automation, numerical control machine tools, semiconductor manufacturing, laser processing and the like, and is required to be limited in order to prevent the sliding table from exceeding a specified working range, avoid accidents, damage to equipment or product quality problems.

The existing sliding table limiting system is mainly a sliding table limiting system based on a photoelectric sensor, even if the photoelectric sensor detects the position of the sliding table, a stop signal is triggered when the sliding table reaches a set position, the sliding table is limited according to the stop signal, in practical application, the sliding table limiting system based on the photoelectric sensor needs to perform sliding table motion simulation in advance, the optimal installation site of the photoelectric sensor is determined, the installation efficiency is possibly poor, the limiting application scene of the photoelectric sensor is single, the requirements of complex limiting scenes such as motion collision limiting are difficult to meet, meanwhile, the data of the photoelectric sensor are greatly influenced by the environment, and the accuracy of the sliding table during limiting is possibly low.

Disclosure of Invention

The invention provides a machine vision-based automatic limiting system, method, device and medium for a sliding table, and mainly aims to solve the problem of low accuracy in the process of limiting the sliding table.

In order to achieve the above purpose, the invention provides a slipway autonomous limiting system based on machine vision, which comprises a point cloud reconstruction module, a semantic segmentation module, a route identification module, a motion monitoring module and a pre-aiming limiting module, wherein:

the point cloud reconstruction module is used for carrying out multi-view data acquisition on the sliding table system to obtain a sliding table system drawing set and a sliding table system point cloud set, and constructing a sliding table system model according to the sliding table system point cloud set, wherein the point cloud reconstruction module is specifically used for when constructing the sliding table system model according to the sliding table system point cloud set: world coordinate transformation is carried out on the sliding table system point cloud set to obtain a primary sliding table point cloud set; extracting a point cloud characteristic set from the primary slipway point cloud set, and performing point cloud splicing reconstruction on the primary slipway point cloud set according to the point Yun Te characteristic set to obtain reconstructed slipway point cloud; and performing point cloud denoising on the reconstructed slipway point cloud by using the following neighborhood oscillation filtering algorithm to obtain a standard slipway point cloud: Wherein (1)>Means the point cloud +.>Coordinate values of>For the reconstruction of the point cloud in the slipway point cloud +.>Neighborhood point cloud->，/>Is the reconstruction slipway point Yun Zhongdian cloud->Neighborhood of->、/>、/>Is a preset weight coefficient, +.>Is of circumference rate>Is a standard deviation coefficient>Is an exponential function sign>Means that the point cloud in the reconstructed slipway point cloud +.>And the neighborhood point cloud->Distance between->Is cosine function symbol>Refers to the neighborhood point cloud->Coordinate values of (2); performing three-dimensional reconstruction on the standard slipway point cloud to obtain a slipway system model;

the semantic segmentation module is used for segmenting the sliding table system diagram set into sliding table component block group sets by utilizing a sub-pixel assembly segmentation method, and extracting sliding table component semantic groups from the sliding table component block group sets;

the route identification module is used for carrying out component mapping and point cloud correction on the sliding table system model according to the sliding table component block set to obtain a correction system model, and extracting a sliding table route model from the correction system model by utilizing the sliding table component semantic group and the sliding table component block set;

the motion monitoring module is used for acquiring real-time data of the sliding table system to obtain a real-time sliding table diagram sequence set, and detecting a moving target and removing noise from the real-time sliding table diagram sequence set to obtain a moving part block sequence set;

The pre-aiming limiting module is used for mapping, matching and moving the sliding table route model according to the moving part block sequence group set to obtain a pre-aiming movement result, and limiting the sliding table of the sliding table system according to the pre-aiming movement result.

Optionally, the point cloud reconstruction module is specifically configured to, when performing point cloud stitching reconstruction on the primary slipway point cloud set according to the point Yun Te collection to obtain reconstructed slipway point cloud:

performing feature matching on the point cloud feature set to obtain a matched point cloud feature set;

extracting a matched primary point cloud group set from the primary slipway point cloud set according to the matched point cloud characteristic group set;

selecting the matched primary point cloud groups in the matched primary point cloud group set one by one as target point cloud groups, and generating a target transformation matrix of the target point cloud groups;

screening target point cloud pairs from the target point cloud groups, and calculating a point cloud transformation relation of the target point cloud pairs according to the target transformation matrix;

performing coordinate transformation on the target point cloud pair according to the point cloud transformation relation to obtain a target transformation point cloud pair;

repeatedly iterating the target transformation point cloud pair to obtain a target matching point cloud pair, and taking the point cloud transformation relationship of the target matching point cloud pair as a target matching point cloud transformation relationship;

And performing point cloud splicing on the target point cloud group by utilizing the target matching point cloud transformation relationship to obtain target splicing point clouds, and splicing all the target splicing point clouds into reconstructed slipway point clouds.

Optionally, the semantic segmentation module is specifically configured to, when the sliding table system diagram set is segmented into the sliding table component block group set by using a sub-pixel component segmentation method:

performing image graying, gray balance and smooth denoising operations on the sliding table system atlas to obtain a standard sliding table atlas;

selecting standard sliding table pictures in the standard sliding table picture set one by one as target standard sliding table pictures, and performing edge detection on the target standard sliding table pictures to obtain a primary sliding table component profile group;

and carrying out subpixel interpolation on the primary slipway component contour group by using the following slipway subpixel contour interpolation algorithm to obtain a subpixel slipway component contour group:wherein (1)>For the sub-pixel slipway component profile group the pixel coordinates are +.>Pixel value of>For the primary slipway component profile group the pixel coordinates are +.>Pixel value of>For pixel coordinates +.>Lateral interpolation weights corresponding to pixels of +.>For pixel coordinates +. >Longitudinal interpolation weights for pixels of (2),/l>The interpolation coefficient is preset;

performing edge suppression and edge connection operation on the sub-pixel sliding table component profile group to obtain a standard sub-pixel component profile group;

dividing the target standard sliding table picture into sliding table part block groups according to the standard sub-pixel part contour groups;

and collecting the slipway component block groups of all the target standard slipway pictures in the slipway system drawing set into a slipway component block group set.

Optionally, when performing edge suppression and edge connection operations on the sub-pixel sliding table component profile group, the semantic segmentation module is specifically configured to:

selecting the sub-pixel sliding table component contours in the sub-pixel sliding table component contour group one by one as target component contours, and calculating gradient amplitude values and gradient directions corresponding to the target component contours;

performing non-maximum suppression on the target component profile according to the gradient amplitude and the gradient direction to obtain a target suppression profile;

performing edge fitting on the target inhibition profile by using a least square method to obtain a target fitting profile;

performing fine granularity sampling and edge smoothing operation on the target fitting contour to obtain a target standard sub-pixel component contour;

And collecting all target standard sub-pixel component contours of the target component contours in the sub-pixel sliding table component contour group into a standard sub-pixel component contour group.

Optionally, the route identifying module is specifically configured to, when performing component mapping and point cloud correction on the sliding table system model according to the sliding table component block set to obtain a corrected system model:

selecting the sliding table component block groups in the sliding table component block groups one by one as target sliding table component block groups, and extracting target shooting parameters corresponding to the target sliding table component block groups;

carrying out model projection on the sliding table system model according to the target shooting parameters to obtain target sliding table projection;

performing component matching and mapping segmentation on the target sliding table projection by using the target sliding table component block group to obtain a matching sliding table projection group;

performing edge mapping correction on the sliding table system model by using the matched sliding table projection group to obtain a primary correction model;

and taking the primary correction model at the moment as a correction system model until the target sliding table component block group is the last sliding table component block group in the sliding table component block group.

Optionally, the motion monitoring module is specifically configured to, when performing motion target detection and motion denoising on the real-time sliding table graph sequence set to obtain a motion component block sequence set:

selecting the real-time sliding table graph sequences in the real-time sliding table graph sequence set one by one as target sliding table graph sequences, and extracting target sliding table graph groups from the target sliding table graph sequences according to the sequential sequence of the combination of every two sliding table graph sequences;

performing inter-frame motion detection and motion frame segmentation operation on the target sliding table graph group to obtain a target inter-frame component graph block group;

denoising the target inter-frame component block group into a target denoising component block group by using a preset iterative component denoising algorithm;

performing part edge segmentation operation on the target denoising part block group to obtain a target standard part block group;

collecting the image blocks of all the target standard component image block groups in the target sliding table image sequence into a target component image block sequence group according to the sequence order;

and collecting all target component block sequence groups of the target sliding table graph sequences in the real-time sliding table graph sequence set into a moving component block sequence group set.

Optionally, the motion monitoring module is specifically configured to, when denoising the target inter-frame component tile group into a target denoising component tile group using a preset iterative component denoising algorithm:

Selecting the blocks in the target inter-frame component block group one by one as target inter-frame blocks, and calculating the dynamic sharpness of the target inter-frame blocks by using the following inter-frame dynamic sharpness algorithm:wherein (1)>Means the dynamic sharpness, +.>Is a predetermined inter-frame sharpness coefficient, +.>Is the pixel length of the target inter tile,is the pixel width of the target inter tile, < >>Means the lateral first +_in the target inter block>Individual pixels +.>Refers to the longitudinal first ∈in the target inter block>Individual pixels +.>Refers to the side length of a sampling frame preset by the target inter-frame image block, < >>Is a gray symbol +.>Means that the coordinate point in the target inter-frame picture block is +.>Gray value of the pixel of +.>Means that the coordinate point in the target inter-frame picture block is +.>Gray value of the pixel of +.>Means that the coordinate point in the target inter-frame picture block is +.>Gray value of the pixel of +.>Means that the coordinate point in the target inter-frame picture block is +.>Gray value of the pixel of +.>Means that the coordinate point in the target inter-frame image block isGray values of pixels of (a);

judging whether the dynamic sharpness is larger than a preset sharpness threshold value or not;

if yes, performing motion denoising on the target inter-frame image block according to the dynamic sharpness, and returning to the step of calculating the dynamic sharpness of the target inter-frame image block by using the following inter-frame dynamic sharpness algorithm;

If not, the target inter-frame image block is taken as a target denoising image block, and the target denoising image blocks of all the target inter-frame image blocks in the target inter-frame component image block group are collected into a target denoising component image block group.

In order to solve the problems, the invention also provides a self-limiting method of the sliding table based on machine vision, which comprises the following steps:

carrying out multi-view data acquisition on a sliding table system to obtain a sliding table system drawing set and a sliding table system point cloud set, constructing a sliding table system model according to the sliding table system point cloud set, wherein the constructing of the sliding table system model according to the sliding table system point cloud set comprises the following steps: world coordinate transformation is carried out on the sliding table system point cloud set to obtain a primary sliding table point cloud set; extracting a point cloud characteristic set from the primary slipway point cloud set, and performing point cloud splicing reconstruction on the primary slipway point cloud set according to the point Yun Te characteristic set to obtain reconstructed slipway point cloud; and performing point cloud denoising on the reconstructed slipway point cloud by using the following neighborhood oscillation filtering algorithm to obtain a standard slipway point cloud:wherein (1)>Means the point cloud +.>Coordinate values of>For the reconstruction of the point cloud in the slipway point cloud +. >Neighborhood point cloud->，/>Is the reconstruction slidePoint cloud>Neighborhood of->、/>、/>Is a preset weight coefficient, +.>Is of circumference rate>Is a standard deviation coefficient>Is an exponential function sign>Means that the point cloud in the reconstructed slipway point cloud +.>And the neighborhood point cloud->Distance between->Is cosine function symbol>Refers to the neighborhood point cloud->Coordinate values of (2); performing three-dimensional reconstruction on the standard slipway point cloud to obtain a slipway system model;

dividing the sliding table system diagram set into sliding table component block group sets by utilizing a sub-pixel assembly dividing method, and extracting a sliding table component semantic group from the sliding table component block group sets;

performing component mapping and point cloud correction on the sliding table system model according to the sliding table component block group set to obtain a correction system model, and extracting a sliding table route model from the correction system model by utilizing the sliding table component semantic group and the sliding table component block group set;

the method comprises the steps of collecting real-time data of the sliding table system to obtain a real-time sliding table diagram sequence set, and detecting a moving target and removing noise from the real-time sliding table diagram sequence set to obtain a moving part block sequence set;

and carrying out mapping matching and movement pre-aiming on the sliding table route model according to the moving part block sequence set to obtain a pre-aiming movement result, and carrying out sliding table limiting on the sliding table system according to the pre-aiming movement result.

In order to solve the problems, the invention also provides a self-limiting device of the sliding table based on machine vision, which comprises camera equipment, point cloud acquisition equipment and a processor, wherein:

the camera equipment is used for shooting the sliding table system at multiple angles to obtain a sliding table system drawing set;

the point cloud acquisition equipment is used for carrying out multi-angle point cloud acquisition on the sliding table system to obtain a point cloud set of the sliding table system;

the processor is used for limiting the sliding table in the sliding table system according to the automatic sliding table limiting method based on machine vision, the sliding table system atlas and the sliding table point cloud set.

In order to solve the above-mentioned problems, the present invention further provides a computer readable storage medium, in which at least one computer program is stored, the at least one computer program being executed by a processor in an electronic device to implement the above-mentioned machine vision-based slipway autonomous limiting method.

According to the invention, the multi-view data acquisition is carried out on the sliding table system to obtain the sliding table system atlas and the sliding table system point cloud atlas, the picture information and the point cloud information of the sliding table system can be acquired, the modeling accuracy of the subsequent sliding table system and the limiting accuracy of the sliding table are improved, the sliding table system atlas is divided into the sliding table component image block sets by utilizing a sub-pixel assembly dividing method, the dividing accuracy of each sliding table component in the sliding table system atlas can be improved, so that a three-dimensional model of the sliding table system can be conveniently and accurately obtained later, the semantic analysis of sliding table components can be realized by extracting the sliding table component image block sets from the sliding table component image block sets, the subsequent sliding table route detection and the limiting judgment can be conveniently carried out, the point cloud correction can be obtained by carrying out component mapping and point cloud correction on the sliding table system model according to the sliding table component image block sets, the further correction can be carried out according to the sub-pixel image dividing result, the accuracy of the correction system model is improved, the subsequent limiting accuracy of the sliding table can be improved, the subsequent sliding table can be conveniently and the automatic limiting data acquisition of the sliding table model can be conveniently realized by extracting the sliding table route from the sliding table system model by utilizing the sliding table component image block sets and the sliding table component image block sets.

The moving object detection and the movement denoising are carried out on the real-time sliding table diagram sequence set to obtain a moving part block sequence set, a part in a moving state in a sliding table system can be monitored in real time, the part is subjected to clear operation, the accuracy of modeling sliding table movement and the accuracy of limiting a follow-up sliding table are improved, mapping matching and movement pre-aiming are carried out on the sliding table route model according to the moving part block sequence set to obtain a pre-aiming movement result, the movement track and the object collision condition of the sliding table in a short time in the future in a sliding table route can be rapidly predicted, the sliding table system is limited according to the pre-aiming movement result, the accurate monitoring of sliding table movement can be realized, and the limiting accuracy of the sliding table is improved. Therefore, the automatic limiting system, method, device and medium for the sliding table based on the machine vision can solve the problem of lower accuracy when the sliding table is limited.

Drawings

FIG. 1 is a functional block diagram of a machine vision-based autonomous sliding table limiting system according to an embodiment of the present invention;

FIG. 2 is a flow chart of extracting standard subpixel component profile groups according to one embodiment of the present invention;

FIG. 3 is a flow chart of generating a model of a corrective system in accordance with an embodiment of the present invention;

fig. 4 is a schematic flow chart of a sliding table automatic limiting method based on machine vision according to an embodiment of the present invention;

the achievement of the objects, functional features and advantages of the present invention will be further described with reference to the accompanying drawings, in conjunction with the embodiments.

Detailed Description

It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.

The embodiment of the application provides a slipway autonomous limiting system based on machine vision. The execution main body of the automatic limit system of the sliding table based on machine vision comprises at least one of electronic equipment, such as a server side, a terminal and the like, which can be configured to execute the system provided by the embodiment of the application. In other words, the machine vision-based slipway autonomous limiting system may be implemented by software or hardware installed in a terminal device or a server device. The service end includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like. The server may be an independent server, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

Fig. 1 is a functional block diagram of a machine vision-based autonomous sliding table limiting system according to an embodiment of the present invention.

The machine vision-based slipway autonomous limiting system 100 of the present invention may be installed in an electronic device. According to the implemented functions, the machine vision-based slipway autonomous limiting system 100 may include a point cloud reconstruction module 101, a semantic segmentation module 102, a route identification module 103, a motion monitoring module 104, and a pre-aiming limiting module 105. The module of the invention, which may also be referred to as a unit, refers to a series of computer program segments, which are stored in the memory of the electronic device, capable of being executed by the processor of the electronic device and of performing a fixed function.

In the present embodiment, the functions concerning the respective modules/units are as follows:

the point cloud reconstruction module 101 is configured to perform multi-view data acquisition on a sliding table system, obtain a sliding table system atlas and a sliding table system point cloud set, and construct a sliding table system model according to the sliding table system point cloud set.

In the embodiment of the invention, the traditional sliding table limiting system mainly uses the autonomous measurement and analysis of a user to determine the installation position and the installation direction of the sliding table limiting sensor, and a large installation error possibly exists, so that repeated debugging is required, and the point cloud reconstruction module 101 can obtain a relatively accurate sliding table system model by respectively acquiring the picture data and the point cloud data of the sliding table system, thereby facilitating the user to perform the simulative accurate limiting analysis.

In the embodiment of the invention, each sliding table system picture in the sliding table system picture set corresponds to each sliding table system point cloud in the sliding table system point cloud set, and has the same visual angle and shooting position.

In the embodiment of the present invention, the point cloud reconstruction module 101 performs multi-view data acquisition on a sliding table system to obtain a sliding table system atlas and a sliding table system point cloud atlas, that is, mounting a plurality of point cloud acquisition devices such as an imaging device and a laser radar or a structured light three-dimensional scanner on a plurality of positions of the sliding table system, adjusting the imaging device and the point cloud acquisition device on the same position to the same direction, performing data acquisition on the sliding table system, collecting sliding table system pictures obtained by all the imaging devices into the sliding table system atlas, and collecting sliding table system point clouds obtained by all the point cloud acquisition devices into the sliding table system point cloud atlas.

In the embodiment of the present invention, when the point cloud reconstruction module 101 constructs a slipway system model according to the slipway system point cloud set, the point cloud reconstruction module is specifically configured to:

world coordinate transformation is carried out on the sliding table system point cloud set to obtain a primary sliding table point cloud set;

extracting a point cloud characteristic set from the primary slipway point cloud set, and performing point cloud splicing reconstruction on the primary slipway point cloud set according to the point Yun Te characteristic set to obtain reconstructed slipway point cloud;

Performing point cloud denoising on the reconstructed slipway point cloud to obtain a standard slipway point cloud;

and carrying out three-dimensional reconstruction on the standard slipway point cloud to obtain a slipway system model.

Specifically, the reconstructed slip table point cloud can be subjected to point cloud denoising by using the following neighborhood oscillation filtering algorithm to obtain a standard slip table point cloud:wherein,means the point cloud +.>Coordinate values of>For reconstructing the point cloud in the slipway point cloudNeighborhood point cloud->，/>Is the reconstruction slipway point Yun Zhongdian cloud->Neighborhood of->、/>、/>Is a preset weight coefficient, +.>Is of circumference rate>Is a standard deviation coefficient>Is an exponential function sign>Means that the point cloud in the reconstructed slipway point cloud +.>And the neighborhood point cloud->Distance between->Is cosine function symbol>Refers to the neighborhood point cloud->Coordinate values of (2); performing three-dimensional reconstruction on the standard slipway point cloud to obtain a slipway system model;

in detail, the obtaining the primary sliding table point cloud set by the point cloud reconstruction module 101 refers to extracting a point cloud collection parameter set from the sliding table system point cloud set, and performing world coordinate transformation on the sliding table system point cloud set according to the point cloud collection parameter set to obtain the primary sliding table point cloud set, where the point cloud collection parameter set includes parameters such as a rotation direction and a gravity direction of a point cloud, and the point cloud reconstruction module 101 may extract a point cloud feature set from the primary sliding table point cloud set by using a convolution pooling method.

In detail, the point cloud reconstruction module 101 can combine gaussian filtering to realize noise removal filtering of the point cloud by using a neighborhood oscillation filtering algorithm, so that the filtering algorithm is more flexible and can adapt to more diversified point cloud data characteristics.

Specifically, when the point cloud reconstruction module 101 performs point cloud stitching reconstruction on the primary slipway point cloud set according to the point Yun Te collection to obtain reconstructed slipway point cloud, the point cloud reconstruction module is specifically configured to:

In detail, the point cloud reconstruction module 101 may perform feature matching on the point cloud feature set by using a cosine similarity algorithm to obtain a matched point cloud feature set, where each matched primary point cloud set in the matched primary point cloud set is composed of point clouds corresponding to each matched point cloud feature set in the primary slipway point cloud set.

Specifically, the point cloud reconstruction module 101 may calculate the point cloud transformation relationship of the target point cloud pair according to the target transformation matrix by using a least square method, iterate the target transformation point cloud pair by using an ICP algorithm to obtain a target matching point cloud pair, and the point cloud reconstruction module 101 may perform three-dimensional reconstruction on the standard slipway point cloud by using a voxelized or curved surface reconstruction algorithm to obtain a slipway system model.

In the embodiment of the invention, the point cloud reconstruction module 101 acquires the drawing set of the sliding table system and the point cloud set of the sliding table system by carrying out multi-view data acquisition on the sliding table system, so that the picture information and the point cloud information of the sliding table system can be acquired, and further the modeling accuracy of the subsequent sliding table system and the limiting accuracy of the sliding table are improved.

The semantic segmentation module 102 is configured to segment the sliding table system diagram set into a sliding table component block set by using a sub-pixel component segmentation method, and extract a sliding table component semantic group from the sliding table component block set.

In the embodiment of the invention, the traditional sliding table limiting system generally adopts a manual calibration method to determine the guide rail and the sliding table component of the sliding table system, and further determines the limiting method, but the sliding table limiting system of the invention can automatically identify and identify different components of the sliding table system by utilizing the semantic segmentation module 102 to segment the picture component of the sliding table system atlas and identify the semantic recognition, thereby improving the limiting efficiency of the sliding table, and each sliding table component block group in the sliding table component block group corresponds to the block of all components of one sliding table system picture in the sliding table system atlas.

In the embodiment of the present invention, when the semantic segmentation module 102 segments the sliding table system diagram set into the sliding table component block set by using the sub-pixel component segmentation method, the semantic segmentation module is specifically configured to:

sub-pixel interpolation is carried out on the primary sliding table component profile group, so that a sub-pixel sliding table component profile group is obtained;

Specifically, the following sliding table sub-pixel contour interpolation algorithm may be used to perform sub-pixel interpolation on the primary sliding table component contour set, so as to obtain a sub-pixel sliding table component contour set:wherein (1)>For the sub-pixel slipway component profile group the pixel coordinates are +.>Pixel value of>For the primary slipway component profile group the pixel coordinates are +.>Pixel value of>For pixel coordinates +.>Lateral interpolation weights corresponding to pixels of +. >For pixel coordinates +.>Longitudinal interpolation weights for pixels of (2),/l>The interpolation coefficient is preset;

in detail, the semantic segmentation module 102 may perform edge detection on the target standard sliding table picture by using Sobel, prewitt and Canny edge detection algorithm to obtain a primary sliding table component profile group, and the sliding table sub-pixel profile interpolation algorithm may implement sub-pixel interpolation of the profile according to the position relationship and the pixel value of 16 pixel points adjacent to each pixel in the primary sliding table component profile group.

Specifically, referring to fig. 2, when performing edge suppression and edge connection operations on the sub-pixel slipway component profile group to obtain a standard sub-pixel component profile group, the semantic segmentation module 102 is specifically configured to:

s21, selecting the sub-pixel sliding table component contours in the sub-pixel sliding table component contour group one by one as target component contours, and calculating gradient amplitude values and gradient directions corresponding to the target component contours;

s22, performing non-maximum suppression on the target component profile according to the gradient amplitude and the gradient direction to obtain a target suppression profile;

s23, performing edge fitting on the target inhibition profile by using a least square method to obtain a target fitting profile;

S24, carrying out fine-granularity sampling and edge smoothing operation on the target fitting contour to obtain a target standard sub-pixel component contour;

s25, collecting all target standard sub-pixel component outlines of the target component outlines in the sub-pixel sliding table component outline group into a standard sub-pixel component outline group.

Specifically, the semantic segmentation module 102 may calculate the gradient magnitude and the gradient direction corresponding to the target component contour by using an operator such as Sobel, prewitt, and the semantic segmentation module 102 may extract the sliding table component semantic group from the sliding table component block group set by using a convolutional neural network model or a model such as VGG-16 after being trained in advance.

In the embodiment of the invention, the semantic segmentation module 102 segments the sliding table system diagram set into the sliding table component block set by using the sub-pixel component segmentation method, so that the segmentation accuracy of each sliding table component in the sliding table system diagram set can be improved, a more accurate three-dimensional model of the sliding table system can be conveniently obtained later, and the semantic analysis of the sliding table component can be realized by extracting the sliding table component semantic group from the sliding table component block set, so that the subsequent sliding table route detection and limit judgment are convenient.

The route identification module 103 is configured to perform component mapping and point cloud correction on the sliding table system model according to the sliding table component block set, obtain a correction system model, and extract a sliding table route model from the correction system model by using the sliding table component semantic group and the sliding table component block set.

In the embodiment of the invention, the traditional sliding table limiting system generally adopts a manual calibration method to determine the guide rail and the sliding table component of the sliding table system, so as to determine the limiting method.

In the embodiment of the present invention, referring to fig. 3, when the route identifying module 103 performs component mapping and point cloud correction on the sliding table system model according to the sliding table component block set to obtain a corrected system model, the route identifying module is specifically configured to:

s31, selecting the sliding table component block groups in the sliding table component block groups one by one as target sliding table component block groups, and extracting target shooting parameters corresponding to the target sliding table component block groups;

S32, carrying out model projection on the sliding table system model according to the target shooting parameters to obtain target sliding table projection;

s33, performing component matching and mapping segmentation on the target sliding table projection by using the target sliding table component block group to obtain a matched sliding table projection group;

s34, carrying out edge mapping correction on the sliding table system model by utilizing the matched sliding table projection group to obtain a primary correction model;

and S35, taking the primary correction model at the moment as a correction system model until the target sliding table component block group is the last sliding table component block group in the sliding table component block group.

Specifically, the target shooting parameter refers to information such as a position and an angle of the image capturing device when shooting a picture corresponding to the target sliding table component block group, the route identifying module 103 may perform component matching and mapping segmentation on the target sliding table projection by using a nearest neighbor matching algorithm and the target sliding table component block group to obtain a matching sliding table projection group, and the route identifying module 103 may perform edge mapping correction on the sliding table system model according to the matching sliding table projection group by using an affine transformation or perspective transformation algorithm to obtain a primary correction model.

In detail, the route recognition module 103 is specifically configured to, when extracting a slip route model from the correction system model using the slip component semantic group and the slip component tile group set: extracting a sliding table route semantic group from the sliding table component semantic group; assembling the sliding table part block groups corresponding to the sliding table route semantic groups into sliding table route part block groups; mapping the sliding table route component block group into the correction system model to obtain a sliding table route model.

Specifically, the sliding table route semantic group refers to a combination of component semantics related to a sliding table route in the sliding table component semantic group, such as sliding table semantics, guide rail semantics and the like.

In the embodiment of the present invention, the route identification module 103 performs component mapping and point cloud correction on the sliding table system model according to the sliding table component block set to obtain a correction system model, and may further correct the point cloud model according to the sub-pixel level image segmentation result, thereby improving the accuracy of the correction system model, and further improving the accuracy of the subsequent autonomous sliding table limiting.

The motion monitoring module 104 is configured to perform real-time data acquisition on the sliding table system to obtain a real-time sliding table graph sequence set, and perform moving object detection and motion denoising on the real-time sliding table graph sequence set to obtain a moving component block sequence set.

In the embodiment of the invention, in order to improve the real-time performance of system limiting, the sliding table limiting system needs to realize rapid motion calculation, so that the motion monitoring module 104 is utilized to acquire real-time data of the sliding table system to obtain a real-time sliding table graph sequence set, and the real-time sliding table graph sequence set is subjected to motion target detection and motion denoising to obtain a moving part block sequence set, so that only picture data during motion can be acquired, the data processing quantity is reduced, the motion calculation efficiency is improved, and the limiting real-time performance is improved.

In the embodiment of the present invention, the motion monitoring module 104 performs real-time data collection on the sliding table system to obtain a real-time sliding table graph sequence set, that is, a shooting device is used to rapidly perform multi-angle shooting on the sliding table system according to a preset shooting interval, obtain a plurality of real-time sliding table graph sequences with different shooting angles, and collect each real-time sliding table graph sequence into a real-time sliding table graph sequence set.

In the embodiment of the present invention, when the motion monitoring module 104 performs motion target detection and motion denoising on the real-time sliding table graph sequence set to obtain a motion component block sequence set, the motion monitoring module is specifically configured to:

In detail, the motion monitoring module 104 extracts the target sliding table image group from the target sliding table image sequence according to the sequential sequence of two-by-two combinations, that is, one sliding table image in the target sliding table image sequence is selected as a target sliding table image, the sliding table image located behind the target sliding table image in the target sliding table image sequence is used as a target adjacent sliding table image, the target sliding table image and the target adjacent sliding table image are collected into the target sliding table image group, and the motion monitoring module 104 can perform the inter-frame motion detection and the motion frame segmentation operation on the target sliding table image group by using an optical flow method, a block matching method or a difference image method to obtain a target inter-frame component image group.

Specifically, the motion monitoring module 104 is specifically configured to, when denoising the target inter-frame component tile group into a target denoising component tile group using a preset iterative component denoising algorithm:

selecting the image blocks in the image block group of the target inter-frame component one by one as target inter-frame image blocks, and calculating the dynamic sharpness of the target inter-frame image blocks;

Specifically, the dynamic sharpness of the target inter-frame tile may be calculated using the following inter-frame dynamic sharpness algorithm:wherein (1)>Means the dynamic sharpness, +.>Is a predetermined inter-frame sharpness coefficient, +.>Is the pixel length of the target inter tile, < >>Is the pixel width of the target inter tile, < > >Means the lateral first +_in the target inter block>Individual pixels +.>Refers to the longitudinal first ∈in the target inter block>Individual pixels +.>Refers to the side length of a sampling frame preset by the target inter-frame image block, < >>Is a gray symbol +.>Means that the coordinate point in the target inter-frame picture block is +.>Gray value of the pixel of +.>Means that the coordinate point in the target inter-frame picture block is +.>Is used for the gray-scale value of the pixel of (c),means that the coordinate point in the target inter-frame picture block is +.>Gray value of the pixel of +.>Means that the coordinate point in the target inter-frame picture block is +.>Gray value of the pixel of +.>Means that the coordinate point in the target inter-frame picture block is +.>Gray values of pixels of (a);

in detail, the motion monitoring module 104 calculates the dynamic sharpness of the target inter-frame image block by using the following inter-frame dynamic sharpness algorithm, so as to determine the sharpness of the image block according to the gradient difference value and the distribution condition of the neighborhood pixels of the motion image block, thereby implementing image block motion denoising, and the motion monitoring module 104 may perform motion denoising on the target inter-frame image block according to the dynamic sharpness by using wavelet denoising and inter-frame difference method.

In the embodiment of the invention, the motion monitoring module 104 obtains the moving component block sequence set by detecting the moving target and removing the noise of the real-time sliding table graph sequence set, so that the components in the moving state in the sliding table system can be monitored in real time and subjected to the clear operation, and the accuracy of modeling the sliding table motion and the accuracy of limiting the follow-up sliding table are improved.

The pre-aiming limiting module 105 is configured to map, match and move the sliding table route model according to the moving part block sequence set, obtain a pre-aiming movement result, and limit the sliding table according to the pre-aiming movement result.

In the embodiment of the invention, the existing sliding table limiting system is mainly based on a limiting method of a real-time photoelectric sensor, so that the accuracy of judging the moving speed and the direction of the sliding table is low, and the random foreign matter collision event cannot be monitored in real time, so that the accuracy of limiting the sliding table is low.

In the embodiment of the present invention, when mapping, matching and moving the sliding table route model according to the moving part block sequence set, the pre-aiming limiting module 105 is specifically configured to:

mapping the moving part block sequence set into the sliding table route model to obtain a sliding table moving model sequence;

carrying out semantic analysis on the moving parts of the sliding table moving model sequence to obtain a moving semantic group;

screening a sliding table part model sequence and a foreign body part model sequence group from the sliding table motion model sequence according to the motion semantic group;

Moving the sliding table part model sequence to obtain a pre-aiming sliding table track;

performing motion pre-aiming on the foreign part model sequence group to obtain a pre-aiming motion track group;

performing collision detection on the pre-aiming sliding table track and the pre-aiming movement track group to obtain a pre-aiming movement result;

and integrating the track of the pre-aiming sliding table and the pre-aiming movement result into a pre-aiming movement result.

Specifically, the sliding table part model sequence is a model sequence formed by sliding table parts in the sliding table operation model sequence, and the foreign part model sequence group is a model sequence formed by the rest of moving parts except the sliding table parts in the sliding table movement model sequence.

In detail, the pre-aiming limiting module 105 may perform motion pre-aiming by using an interpolation algorithm, wherein the motion pre-aiming refers to calculating a motion track in a short time in the future, and the collision detection is performed on the pre-aiming sliding table track and the pre-aiming motion track group, so as to obtain a pre-aiming motion result, which refers to judging whether the pre-aiming sliding table track and the pre-aiming motion track group cross-collide at the same time.

Specifically, the pre-aiming limiting module 105 performs sliding table limiting on the sliding table system according to the pre-aiming movement result, that is, performs sliding table limiting according to pre-aiming time when collision occurs in the pre-aiming movement result or the track of the pre-aiming sliding table reaches a predetermined limiting position.

In the embodiment of the present invention, the pretightening limiting module 105 performs mapping matching and motion pretightening on the sliding table route model according to the moving component block sequence set to obtain a pretightening motion result, so that a motion track and an object collision condition of the sliding table in a short time in the future in the sliding table route can be rapidly predicted, and accurate monitoring on the sliding table motion can be realized and the accuracy of the sliding table limiting can be improved by performing sliding table limiting on the sliding table system according to the pretightening motion result.

The moving object detection and the movement denoising are carried out on the real-time sliding table diagram sequence set to obtain a moving part block sequence set, a part in a moving state in a sliding table system can be monitored in real time, the part is subjected to clear operation, the accuracy of modeling sliding table movement and the accuracy of limiting a follow-up sliding table are improved, mapping matching and movement pre-aiming are carried out on the sliding table route model according to the moving part block sequence set to obtain a pre-aiming movement result, the movement track and the object collision condition of the sliding table in a short time in the future in a sliding table route can be rapidly predicted, the sliding table system is limited according to the pre-aiming movement result, the accurate monitoring of sliding table movement can be realized, and the limiting accuracy of the sliding table is improved. Therefore, the automatic limiting system for the sliding table based on the machine vision can solve the problem of lower accuracy when the sliding table is limited.

Referring to fig. 4, a flow chart of a method for automatically limiting a sliding table based on machine vision according to an embodiment of the present invention is shown. In this embodiment, the automatic limit method for the sliding table based on machine vision includes:

S1, carrying out multi-view data acquisition on a sliding table system to obtain a sliding table system drawing set and a sliding table system point cloud set, constructing a sliding table system model according to the sliding table system point cloud set, wherein the constructing of the sliding table system model according to the sliding table system point cloud set comprises the following steps: world coordinate transformation is carried out on the sliding table system point cloud set to obtain a primaryA sliding table point cloud set; extracting a point cloud characteristic set from the primary slipway point cloud set, and performing point cloud splicing reconstruction on the primary slipway point cloud set according to the point Yun Te characteristic set to obtain reconstructed slipway point cloud; and performing point cloud denoising on the reconstructed slipway point cloud by using the following neighborhood oscillation filtering algorithm to obtain a standard slipway point cloud:wherein (1)>Means the point cloud +.>Coordinate values of>For the reconstruction of the point cloud in the slipway point cloud +.>Neighborhood point cloud->，/>Is the reconstruction slipway point Yun Zhongdian cloud->Neighborhood of->、/>、/>Is a preset weight coefficient, +.>Is of circumference rate>Is a standard deviation coefficient>Is an exponential function sign>Means that the point cloud in the reconstructed slipway point cloud +.>And the neighborhood point cloud->Distance between->Is cosine function symbol>Refers to the neighborhood point cloud- >Coordinate values of (2); performing three-dimensional reconstruction on the standard slipway point cloud to obtain a slipway system model;

s2, dividing the sliding table system diagram set into sliding table component block group sets by utilizing a sub-pixel assembly dividing method, and extracting a sliding table component semantic group from the sliding table component block group sets.

S3, carrying out component mapping and point cloud correction on the sliding table system model according to the sliding table component block set to obtain a correction system model, and extracting a sliding table route model from the correction system model by utilizing the sliding table component semantic group and the sliding table component block set.

S4, carrying out real-time data acquisition on the sliding table system to obtain a real-time sliding table diagram sequence set, and carrying out moving object detection and movement denoising on the real-time sliding table diagram sequence set to obtain a moving part block sequence set.

And S5, mapping and matching the sliding table route model and pre-aiming the movement according to the moving part block sequence set to obtain a pre-aiming movement result, and limiting the sliding table of the sliding table system according to the pre-aiming movement result.

In detail, the automatic limit method for the sliding table based on machine vision in the embodiment of the present invention adopts the same technical means as the automatic limit system for the sliding table based on machine vision described in fig. 1, and can produce the same technical effects, which are not repeated here.

The invention also provides a sliding table independent limiting device based on machine vision, which comprises camera equipment, point cloud acquisition equipment and a processor, wherein:

The present invention also provides a computer readable storage medium storing a computer program which, when executed by a processor of an electronic device, can implement:

carrying out multi-view data acquisition on a sliding table system to obtain a sliding table system drawing set and a sliding table system point cloud set, and constructing a sliding table system model according to the sliding table system point cloud set, wherein the point cloud reconstruction module is specifically used for constructing the sliding table system model according to the sliding table system point cloud set: world coordinate transformation is carried out on the sliding table system point cloud set to obtain a primary sliding table point cloud set; extracting a point cloud characteristic set from the primary slipway point cloud set, and performing point cloud splicing reconstruction on the primary slipway point cloud set according to the point Yun Te characteristic set to obtain reconstructed slipway point cloud; and performing point cloud denoising on the reconstructed slipway point cloud by using the following neighborhood oscillation filtering algorithm to obtain a standard slipway point cloud: Wherein (1)>Means the point cloud +.>Coordinate values of>For the reconstruction of the point cloud in the slipway point cloud +.>Neighborhood point cloud->，/>Is the reconstruction slipway point Yun Zhongdian cloud->Neighborhood of->、/>、/>Is a preset weight coefficient, +.>Is of circumference rate>Is a standard deviation coefficient>Is an exponential function sign>Means that the point cloud in the reconstructed slipway point cloud +.>And the neighborhood point cloudDistance between->Is cosine function symbol>Refers to the neighborhood point cloud->Coordinate values of (2); performing three-dimensional reconstruction on the standard slipway point cloud to obtain a slipway system model;

In the several embodiments provided by the present invention, it should be understood that the disclosed apparatus, system, and system may be implemented in other manners. For example, the system embodiments described above are merely illustrative, e.g., the division of the modules is merely a logical function division, and other manners of division may be implemented in practice.

The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical units, may be located in one place, or may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional module in the embodiments of the present invention may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units can be realized in a form of hardware or a form of hardware and a form of software functional modules.

It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof.

The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.

The embodiment of the application can acquire and process the related data based on the artificial intelligence technology. Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, system, technique, and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend, and extend human intelligence, sense the environment, acquire knowledge, and use knowledge to obtain optimal results.

Furthermore, it is evident that the word "comprising" does not exclude other elements or steps, and that the singular does not exclude a plurality. Multiple units or systems set forth in the system embodiments may also be implemented by one unit or system in software or hardware. The terms first, second, etc. are used to denote a name, but not any particular order.

Finally, it should be noted that the above-mentioned embodiments are merely for illustrating the technical solution of the present invention and not for limiting the same, and although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications and equivalents may be made to the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. The utility model provides a slip table is from spacing system independently based on machine vision, its characterized in that, the system includes point cloud rebuilding module, semantic segmentation module, route identification module, motion monitoring module and pre-aiming spacing module, wherein:

2. The machine vision-based slipway autonomous limiting system of claim 1, wherein the point cloud reconstruction module is configured to, when performing point cloud stitching reconstruction on the primary slipway point cloud set according to the point Yun Te collection, obtain reconstructed slipway point cloud:

3. The machine vision-based slip autonomous limiting system of claim 1, wherein the semantic segmentation module is configured to, when segmenting the slip system atlas into slip component part atlas sets using a subpixel assembly segmentation method:

and carrying out subpixel interpolation on the primary slipway component contour group by using the following slipway subpixel contour interpolation algorithm to obtain a subpixel slipway component contour group: Wherein (1)>For the sub-pixel slipway component profile group the pixel coordinates are +.>Pixel value of>For the primary slipway component profile group the pixel coordinates are +.>Pixel value of>For pixel coordinates +.>Is used to determine the corresponding lateral interpolation weights for the pixels of (c),for pixel coordinates +.>Longitudinal interpolation weights for pixels of (2),/l>The interpolation coefficient is preset;

4. The machine vision-based slipway autonomous limiting system of claim 3, wherein the semantic segmentation module is configured to, when performing edge suppression and edge connection operations on the sub-pixel slipway component profile set to obtain a standard sub-pixel component profile set:

5. The machine vision-based slip autonomous limiting system of claim 1, wherein the route identification module is configured to, when performing component mapping and point cloud correction on the slip system model according to the slip component block set, obtain a corrected system model:

6. The machine vision-based slip autonomous limiting system of claim 1, wherein the motion monitoring module is configured to, when performing moving object detection and motion denoising on the real-time slip graph sequence set to obtain a moving part block sequence set:

7. The machine vision-based slip autonomous limiting system of claim 1, wherein the motion monitoring module is configured to, when denoising the target inter-frame component tile group into a target denoising component tile group using a preset iterative component denoising algorithm:

selecting the blocks in the target inter-frame component block group one by one as target inter-frame blocks, and calculating the dynamic sharpness of the target inter-frame blocks by using the following inter-frame dynamic sharpness algorithm: wherein (1)>Means the dynamic sharpness, +.>Is a predetermined inter-frame sharpness coefficient, +.>Is the pixel length of the target inter tile, < >>Is the pixel width of the target inter tile, < >>Means the lateral first +_in the target inter block >Individual pixels +.>Refers to the longitudinal first ∈in the target inter block>Individual pixels +.>Refers to the side length of a sampling frame preset by the target inter-frame image block, < >>Is a gray symbol +.>Means that the coordinate point in the target inter-frame picture block is +.>Is used for the gray-scale value of the pixel of (c),means that the coordinate point in the target inter-frame picture block is +.>Gray value of the pixel of +.>Means that the coordinate point in the target inter-frame picture block is +.>Gray value of the pixel of +.>Means that the coordinate point in the target inter-frame picture block is +.>Gray value of the pixel of +.>Means that the coordinate point in the target inter-frame picture block is +.>Gray values of pixels of (a);

8. The automatic limiting method for the sliding table based on the machine vision is characterized by comprising the following steps of:

Carrying out multi-view data acquisition on a sliding table system to obtain a sliding table system drawing set and a sliding table system point cloud set, constructing a sliding table system model according to the sliding table system point cloud set, wherein the constructing of the sliding table system model according to the sliding table system point cloud set comprises the following steps: world coordinate transformation is carried out on the sliding table system point cloud set to obtain a primary sliding table point cloud set; extracting a point cloud characteristic set from the primary slipway point cloud set, and performing point cloud splicing reconstruction on the primary slipway point cloud set according to the point Yun Te characteristic set to obtain reconstructed slipway point cloud; and performing point cloud denoising on the reconstructed slipway point cloud by using the following neighborhood oscillation filtering algorithm to obtain a standard slipway point cloud:wherein (1)>Means the point cloud +.>Coordinate values of>For the reconstruction of the point cloud in the slipway point cloud +.>Neighborhood point cloud->，/>Is the reconstruction slipway point Yun Zhongdian cloud->Neighborhood of->、/>、/>Is a preset weight coefficient, +.>Is of circumference rate>Is a standard deviation coefficient>Is an exponential function sign>Means that the point cloud in the reconstructed slipway point cloud +.>And the neighborhood point cloud->Distance between->Is cosine function symbol>Refers to the neighborhood point cloud Coordinate values of (2); performing three-dimensional reconstruction on the standard slipway point cloud to obtain a slipway system model;

9. Automatic stop device of slip table based on machine vision, its characterized in that includes camera equipment, point cloud collection equipment and treater, wherein:

the processor is configured to limit a slipway in a slipway system according to the autonomous slipway limiting method based on machine vision, the slipway system atlas, and the slipway point cloud set in claim 8.

10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the machine vision-based slip autonomous positioning method as set forth in claim 8.