CA3121440C - Assembly body change detection method, device and medium based on attention mechanism - Google Patents
Assembly body change detection method, device and medium based on attention mechanism Download PDFInfo
- Publication number
- CA3121440C CA3121440C CA3121440A CA3121440A CA3121440C CA 3121440 C CA3121440 C CA 3121440C CA 3121440 A CA3121440 A CA 3121440A CA 3121440 A CA3121440 A CA 3121440A CA 3121440 C CA3121440 C CA 3121440C
- Authority
- CA
- Canada
- Prior art keywords
- feature
- assembly
- attention mechanism
- attention
- change
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000008859 change Effects 0.000 title claims abstract description 87
- 238000001514 detection method Methods 0.000 title claims abstract description 54
- 230000007246 mechanism Effects 0.000 title claims abstract description 43
- 230000004927 fusion Effects 0.000 claims abstract description 34
- 238000012549 training Methods 0.000 claims abstract description 34
- 238000000034 method Methods 0.000 claims abstract description 26
- 230000008569 process Effects 0.000 claims abstract description 24
- 238000000605 extraction Methods 0.000 claims abstract description 17
- 238000012545 processing Methods 0.000 claims abstract description 4
- 230000006870 function Effects 0.000 claims description 10
- 238000004590 computer program Methods 0.000 claims description 9
- 238000013507 mapping Methods 0.000 claims description 8
- 230000011218 segmentation Effects 0.000 claims description 7
- 239000013598 vector Substances 0.000 claims description 7
- 238000005070 sampling Methods 0.000 claims description 3
- 230000001131 transforming effect Effects 0.000 claims description 3
- 238000003384 imaging method Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 6
- 238000013135 deep learning Methods 0.000 description 4
- 238000004519 manufacturing process Methods 0.000 description 4
- 230000015572 biosynthetic process Effects 0.000 description 3
- 238000012544 monitoring process Methods 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007500 overflow downdraw method Methods 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/757—Matching configurations of points or features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F18/2148—Generating training patterns; Bootstrap methods, e.g. bagging or boosting characterised by the process organisation or structure, e.g. boosting cascade
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/217—Validation; Performance evaluation; Active pattern learning techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/64—Three-dimensional objects
- G06V20/647—Three-dimensional objects by matching two-dimensional images to three-dimensional objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/06—Recognition of objects for industrial automation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Abstract
An assembly change detection method based on an attention mechanism includes establishing a three-dimensional model of an assembly body, adding a tag to each part in the three-dimensional model, setting several assembly nodes, obtaining depth images of the three- dimensional model under each assembly node in different viewing angles, and obtaining a change tag image of a newly added part at each assembly node; selecting two depth images at front and back moments in different viewing angles as training samples; performing semantic fusion, feature extraction, attention mechanism processing and metric learning sequentially on the training samples, training a detection model, continuously selecting training samples to train the detection model, saving model parameters with the optimal similarity during the training process, and completing the training; and obtaining depth images of two successive assembly nodes in the assembly process of the assembly body to be detected, inputting the depth images into the trained detection model, and outputting the change image of the newly added part of the assembly body in the assembly process.
Description
ASSEMBLY BODY CHANGE DETECTION METHOD, DEVICE AND MEDIUM BASED
ON ATTENTION MECHANISM
Field of the Invention The present invention relates to an assembly body multi-view change detection method based on an attention mechanism, belonging to the technical field of computer vision and intelligent manufacturing.
Background of the Invention Computer vision is of great significance to the upgrade of intelligent manufacturing industry, especially the emergence of deep learning networks has promoted the development of modern industry. In the process of large-scale customized production and assembly, continuous changes in product types will increase the difficulty in assembling products. In the assembly process of complex assembly body parts, if correct or un-correct assembly of the newly assembled parts is not detected in time, the quality and assembly efficiency of mechanical products will be affected.
Therefore, during the assembly process of mechanical assembly bodies, the detection of new assembly parts in each assembly step from multiple perspectives will help to obtain relevant information of the mechanical assembly process, and find errors in time, thereby achieving rapid positioning of the errors, improving the production efficiency of mechanical products, guaranteeing the quality level of mechanical products, and having important research value for the intelligent detection of the assembly process of the mechanical assembly bodies.
Image change detection can determine the difference in image state based on two images at different times and different viewing angles. The image change detection is currently mainly used in satellite image and aerial image research, and has important practical value in agricultural survey, Date Recue/Date Received 2021-06-08 natural disaster detection, forest resource monitoring, etc. The traditional change detection method mainly includes three steps: 1) inputting two images that need to be detected and analyzed; 2) obtaining a difference image by using a difference operator or other methods;
and 3) analyzing data difference of the difference image to obtain a change region. With regard to the traditional change detection method, the performance depends on the formation of the difference image, and the formation of the difference image has greater noise interference, so higher accuracy cannot be obtained.
At present, with the vigorous development of deep learning technology, deep learning has been introduced into change detection, and many change detection algorithms based on deep learning have been proposed and achieved good results. However, these methods mainly aim at the detection of changes in similar viewing angles between remote sensing satellite images and aerial images. At present, there are few researches on multi-view change detection of mechanical assembly bodies. This is mainly because, compared with satellite images, mechanical parts have the characteristics of complicated structures, severe occlusion, large changes in viewing angles, single color and texture information of parts, etc., it is difficult to detect changes in their assembly process, and there is a lack of corresponding data sets.
Summary of the Invention In order to solve the above-mentioned problems in the prior art, the present invention proposes an assembly change detection method, device and medium based on an attention mechanism, which can detect changes in the assembly process of an assembly body.
The technical solution of the present invention is as follows:
ON ATTENTION MECHANISM
Field of the Invention The present invention relates to an assembly body multi-view change detection method based on an attention mechanism, belonging to the technical field of computer vision and intelligent manufacturing.
Background of the Invention Computer vision is of great significance to the upgrade of intelligent manufacturing industry, especially the emergence of deep learning networks has promoted the development of modern industry. In the process of large-scale customized production and assembly, continuous changes in product types will increase the difficulty in assembling products. In the assembly process of complex assembly body parts, if correct or un-correct assembly of the newly assembled parts is not detected in time, the quality and assembly efficiency of mechanical products will be affected.
Therefore, during the assembly process of mechanical assembly bodies, the detection of new assembly parts in each assembly step from multiple perspectives will help to obtain relevant information of the mechanical assembly process, and find errors in time, thereby achieving rapid positioning of the errors, improving the production efficiency of mechanical products, guaranteeing the quality level of mechanical products, and having important research value for the intelligent detection of the assembly process of the mechanical assembly bodies.
Image change detection can determine the difference in image state based on two images at different times and different viewing angles. The image change detection is currently mainly used in satellite image and aerial image research, and has important practical value in agricultural survey, Date Recue/Date Received 2021-06-08 natural disaster detection, forest resource monitoring, etc. The traditional change detection method mainly includes three steps: 1) inputting two images that need to be detected and analyzed; 2) obtaining a difference image by using a difference operator or other methods;
and 3) analyzing data difference of the difference image to obtain a change region. With regard to the traditional change detection method, the performance depends on the formation of the difference image, and the formation of the difference image has greater noise interference, so higher accuracy cannot be obtained.
At present, with the vigorous development of deep learning technology, deep learning has been introduced into change detection, and many change detection algorithms based on deep learning have been proposed and achieved good results. However, these methods mainly aim at the detection of changes in similar viewing angles between remote sensing satellite images and aerial images. At present, there are few researches on multi-view change detection of mechanical assembly bodies. This is mainly because, compared with satellite images, mechanical parts have the characteristics of complicated structures, severe occlusion, large changes in viewing angles, single color and texture information of parts, etc., it is difficult to detect changes in their assembly process, and there is a lack of corresponding data sets.
Summary of the Invention In order to solve the above-mentioned problems in the prior art, the present invention proposes an assembly change detection method, device and medium based on an attention mechanism, which can detect changes in the assembly process of an assembly body.
The technical solution of the present invention is as follows:
2 Date Recue/Date Received 2021-06-08 Technical solution one:
An assembly body change detection method based on an attention mechanism, including the following steps:
establishing a data set; establishing a three-dimensional model of an assembly body, adding a tag to each part in the three-dimensional model, determining several assembly nodes according to given assembly steps of the assembly body, performing imaging processing on the three-dimensional model under each assembly node to obtain depth images of the three-dimensional model under each assembly node in different viewing angles, and obtaining a change tag image of a newly added part at each assembly node according to the tag of the part;
training a detection model; selecting two depth images of the three-dimensional model of two successive adjacent assembly nodes in different viewing angles as training samples; performing semantic segmentation on the two depth images respectively to obtain semantic images, and performing pixel fusion on the two semantic images and the corresponding depth images to obtain two fusion images; performing feature extraction on the two fusion images respectively to obtain two corresponding initial feature pictures; collecting context information of the two initial feature pictures through the attention mechanism, and capturing global position dependencies of the initial feature pictures according to the context information to obtain two corresponding attention feature pictures; calculating a Euclidean distance between the two attention feature pictures, generating a feature distance picture, and outputting a change image according to the feature distance picture;
defining a loss function to calculate a similarity between the change image and the change tag image, continuously selecting training samples to train the detection model, saving model
An assembly body change detection method based on an attention mechanism, including the following steps:
establishing a data set; establishing a three-dimensional model of an assembly body, adding a tag to each part in the three-dimensional model, determining several assembly nodes according to given assembly steps of the assembly body, performing imaging processing on the three-dimensional model under each assembly node to obtain depth images of the three-dimensional model under each assembly node in different viewing angles, and obtaining a change tag image of a newly added part at each assembly node according to the tag of the part;
training a detection model; selecting two depth images of the three-dimensional model of two successive adjacent assembly nodes in different viewing angles as training samples; performing semantic segmentation on the two depth images respectively to obtain semantic images, and performing pixel fusion on the two semantic images and the corresponding depth images to obtain two fusion images; performing feature extraction on the two fusion images respectively to obtain two corresponding initial feature pictures; collecting context information of the two initial feature pictures through the attention mechanism, and capturing global position dependencies of the initial feature pictures according to the context information to obtain two corresponding attention feature pictures; calculating a Euclidean distance between the two attention feature pictures, generating a feature distance picture, and outputting a change image according to the feature distance picture;
defining a loss function to calculate a similarity between the change image and the change tag image, continuously selecting training samples to train the detection model, saving model
3 Date Recue/Date Received 2021-06-08 parameters with the optimal similarity during the training process, and completing the training; and performing change detection; obtaining the depth images of the two successive assembly nodes in the assembly process of the assembly body to be detected, inputting the depth images into the trained detection model, and outputting the change image of the newly added part of the assembly body in the assembly process.
Further, in the step of performing feature extraction on the two fusion images to obtain two corresponding initial feature pictures, the feature extraction method is specifically as follows:
performing 3 x3 convolution on the inputted fusion images, and extracting features of the inputted images by using four stages of a RepVGG classification network;
embedding the attention mechanism after each stage of the RepVGG
classification network to obtain four groups of feature maps respectively, performing 1 x 1 convolution on the four groups of feature maps, and transforming channel sizes of the feature maps to P1 which is set as an average channel size of the first two groups of feature maps; and performing unified up-sampling on feature picture sizes of the last three stages of feature maps as feature picture sizes of the first stage, performing tandem fusion of channels on the four groups of feature pictures obtained, and performing 3 x3 convolution and 1 x 1 convolution sequentially on the fused feature pictures to obtain the initial feature pictures.
Further, the attention mechanism embedded in the RepVGG classification network is the same as the attention mechanism used in the step of collecting context information of the two initial feature pictures, and the attention mechanism includes two stages:
in the first stage, while saving original feature information, collecting context information of
Further, in the step of performing feature extraction on the two fusion images to obtain two corresponding initial feature pictures, the feature extraction method is specifically as follows:
performing 3 x3 convolution on the inputted fusion images, and extracting features of the inputted images by using four stages of a RepVGG classification network;
embedding the attention mechanism after each stage of the RepVGG
classification network to obtain four groups of feature maps respectively, performing 1 x 1 convolution on the four groups of feature maps, and transforming channel sizes of the feature maps to P1 which is set as an average channel size of the first two groups of feature maps; and performing unified up-sampling on feature picture sizes of the last three stages of feature maps as feature picture sizes of the first stage, performing tandem fusion of channels on the four groups of feature pictures obtained, and performing 3 x3 convolution and 1 x 1 convolution sequentially on the fused feature pictures to obtain the initial feature pictures.
Further, the attention mechanism embedded in the RepVGG classification network is the same as the attention mechanism used in the step of collecting context information of the two initial feature pictures, and the attention mechanism includes two stages:
in the first stage, while saving original feature information, collecting context information of
4 Date Recue/Date Received 2021-06-08 the feature information in horizontal and vertical directions through a self-attention algorithm to obtain horizontal and vertical cross paths of the original feature information; and in the second stage, obtaining other context information from the horizontal and vertical cross paths through the self-attention algorithm again to obtain the global position dependencies of the feature pictures.
Further, the step of calculating a Euclidean distance between the two attention feature pictures, generating a feature distance picture, and outputting a change image according to the feature distance picture is specific as follows:
firstly, adjusting sizes of the two inputted attention feature pictures to the same sizes as the initial inputted depth images by bilinear interpolation;
calculating the Euclidean distance between the two attention feature pictures after the size adjustment, and generating the feature distance picture; and setting a change threshold according to the distance information in the feature distance picture, dividing a depth image region into a changed region and an unchanged region through the change threshold and the distance information, and outputting the change image of the subsequent assembly node relative to the previous assembly node accordingly.
Further, the loss function is specifically:
N
L = ¨2N n=1 yd2 + (1¨ y) max(margin ¨ d, 0)2;
where d =II an ¨ bn 112 represents a Euclidean distance between mapping vectors of two training samples, y is a tag value obtained through the change tag image, y=1 represents similarity,
Further, the step of calculating a Euclidean distance between the two attention feature pictures, generating a feature distance picture, and outputting a change image according to the feature distance picture is specific as follows:
firstly, adjusting sizes of the two inputted attention feature pictures to the same sizes as the initial inputted depth images by bilinear interpolation;
calculating the Euclidean distance between the two attention feature pictures after the size adjustment, and generating the feature distance picture; and setting a change threshold according to the distance information in the feature distance picture, dividing a depth image region into a changed region and an unchanged region through the change threshold and the distance information, and outputting the change image of the subsequent assembly node relative to the previous assembly node accordingly.
Further, the loss function is specifically:
N
L = ¨2N n=1 yd2 + (1¨ y) max(margin ¨ d, 0)2;
where d =II an ¨ bn 112 represents a Euclidean distance between mapping vectors of two training samples, y is a tag value obtained through the change tag image, y=1 represents similarity,
5 Date Recue/Date Received 2021-06-08 y=0 represents dissimilarity, N is a number of pixels of the training samples, and margin is a threshold set artificially.
Technical solution two:
An assembly body change detection device based on an attention mechanism, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the assembly body change detection method based on an attention mechanism according to any embodiment of the present invention.
Technical solution three:
An assembly body change detection medium based on an attention mechanism, storing a computer program thereon, characterized in that the computer program is executed by a processor to implement the assembly body change detection method based on an attention mechanism according to any embodiment of the present invention.
The present invention has the following beneficial effects:
The present invention designs a detection model including a semantic fusion module, a feature extraction module, an attention module, and a metric learning module. The semantic fusion module is configured to fuse semantic images and depth images to obtain fusion images, thereby enriching image feature information; the feature extraction module extracts high-dimensional semantic information and low-dimensional spatial information of the fusion images to obtain initial feature pictures; the attention module quickly focuses on key information of a current task from the initial feature pictures, and then obtains final feature pictures; and the metric learning module calculates
Technical solution two:
An assembly body change detection device based on an attention mechanism, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the assembly body change detection method based on an attention mechanism according to any embodiment of the present invention.
Technical solution three:
An assembly body change detection medium based on an attention mechanism, storing a computer program thereon, characterized in that the computer program is executed by a processor to implement the assembly body change detection method based on an attention mechanism according to any embodiment of the present invention.
The present invention has the following beneficial effects:
The present invention designs a detection model including a semantic fusion module, a feature extraction module, an attention module, and a metric learning module. The semantic fusion module is configured to fuse semantic images and depth images to obtain fusion images, thereby enriching image feature information; the feature extraction module extracts high-dimensional semantic information and low-dimensional spatial information of the fusion images to obtain initial feature pictures; the attention module quickly focuses on key information of a current task from the initial feature pictures, and then obtains final feature pictures; and the metric learning module calculates
6 Date Recue/Date Received 2021-06-08 distances between features according to the feature information of the final feature pictures to obtain a change region, and outputs change images of successive assembly nodes of the assembly body through the change region, wherein the change images can be used for monitoring missing assembly, wrong assembly, and assembly steps in the assembly process of an assembly body.
Brief Description of the Drawings Fig. 1 is a flowchart of an embodiment of the present invention;
Fig. 2 is a schematic diagram of a detection model according to an embodiment of the present invention;
Fig. 3 is an illustrative diagram of a feature extraction module according to an embodiment of the present invention;
Fig. 4 is an illustrative diagram of an attention module according to an embodiment of the present invention;
Fig. 5 is an illustrative diagram of training samples according to an embodiment of the present invention;
Fig. 6 is an illustrative diagram of a change image outputted by the model and a real change tag image according to an embodiment of the present invention.
Detailed Description of the Embodiments The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
Embodiment 1:
Brief Description of the Drawings Fig. 1 is a flowchart of an embodiment of the present invention;
Fig. 2 is a schematic diagram of a detection model according to an embodiment of the present invention;
Fig. 3 is an illustrative diagram of a feature extraction module according to an embodiment of the present invention;
Fig. 4 is an illustrative diagram of an attention module according to an embodiment of the present invention;
Fig. 5 is an illustrative diagram of training samples according to an embodiment of the present invention;
Fig. 6 is an illustrative diagram of a change image outputted by the model and a real change tag image according to an embodiment of the present invention.
Detailed Description of the Embodiments The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
Embodiment 1:
7 Date Recue/Date Received 2021-06-08 Referring to Fig. 1, an assembly body change detection method based on an attention mechanism includes the following steps:
Establishing a data set; establishing a three-dimensional model of a mechanical assembly body through SolidWorks software, and adding a tag to each part in the three-dimensional model, in this embodiment, the tags added to the parts being color marks, setting m assembly nodes and m-1 assembly steps, assembling one part in each step, then loading the three-dimensional model of the mechanical assembly body into depth image and color image formation software, setting a virtual camera to image different angles of each assembly node, obtaining depth images and color images of the three-dimensional model under each assembly node in different viewing angles, and generating a change tag image of a newly assembled part at each assembly node by using the color marks of the color images;
training a detection model; selecting the depth image of the three-dimensional model of the previous assembly node in one viewing angle and the depth images of the three-dimensional model of the subsequent assembly node in different viewing angles as a set of training samples;
referring to Fig. 2, in this embodiment, the detection model including four modules: a semantic fusion module, a feature extraction module, an attention module, and a metric learning module;
firstly, inputting Training samples to the semantic fusion module, the semantic fusion module including a semantic segmentation network and a pixel fusion network, first performing semantic segmentation on two depth images through the semantic segmentation network to obtain two corresponding semantic images, and then performing pixel fusion on the two semantic images with
Establishing a data set; establishing a three-dimensional model of a mechanical assembly body through SolidWorks software, and adding a tag to each part in the three-dimensional model, in this embodiment, the tags added to the parts being color marks, setting m assembly nodes and m-1 assembly steps, assembling one part in each step, then loading the three-dimensional model of the mechanical assembly body into depth image and color image formation software, setting a virtual camera to image different angles of each assembly node, obtaining depth images and color images of the three-dimensional model under each assembly node in different viewing angles, and generating a change tag image of a newly assembled part at each assembly node by using the color marks of the color images;
training a detection model; selecting the depth image of the three-dimensional model of the previous assembly node in one viewing angle and the depth images of the three-dimensional model of the subsequent assembly node in different viewing angles as a set of training samples;
referring to Fig. 2, in this embodiment, the detection model including four modules: a semantic fusion module, a feature extraction module, an attention module, and a metric learning module;
firstly, inputting Training samples to the semantic fusion module, the semantic fusion module including a semantic segmentation network and a pixel fusion network, first performing semantic segmentation on two depth images through the semantic segmentation network to obtain two corresponding semantic images, and then performing pixel fusion on the two semantic images with
8 Date Recue/Date Received 2021-06-08 their corresponding depth images through the pixel fusion network to obtain two fusion images, in this embodiment, the semantic segmentation network (e.g., FCN, U-Net, etc.) configured to segment assembly parts, and input depth images in different viewing angles to the semantic segmentation network to obtain semantic images. The pixel fusion network fuses the semantic images and the depth images according to a certain pixel ratio by using a pixel-level image fusion method, thereby saving depth information as much as possible while fusing the semantic information, and increasing the amount of image feature information.
The feature extraction module performs feature extraction on the two fusion images respectively. In this embodiment, a multi-scale feature extraction and fusion mechanism is designed .. to fuse high-dimensional semantic information and low-dimensional spatial information, thereby increasing the amount of feature information and obtaining two corresponding initial feature picture;
collecting context information of the two initial feature pictures respectively by using the attention mechanism, and capturing global position dependencies of the initial feature pictures according to the context information bythe attention module to obtain two corresponding attention feature pictures;
by the metric module, firstly, calculating a Euclidean distance between the two attention feature pictures and generating a feature distance picture, and outputting a change image according to the feature distance picture; defining a loss function to calculate a similarity between the change image and the change tag image of the subsequent assembly node, continuously selecting training samples to train the detection model, saving model parameters with the optimal similarity during
The feature extraction module performs feature extraction on the two fusion images respectively. In this embodiment, a multi-scale feature extraction and fusion mechanism is designed .. to fuse high-dimensional semantic information and low-dimensional spatial information, thereby increasing the amount of feature information and obtaining two corresponding initial feature picture;
collecting context information of the two initial feature pictures respectively by using the attention mechanism, and capturing global position dependencies of the initial feature pictures according to the context information bythe attention module to obtain two corresponding attention feature pictures;
by the metric module, firstly, calculating a Euclidean distance between the two attention feature pictures and generating a feature distance picture, and outputting a change image according to the feature distance picture; defining a loss function to calculate a similarity between the change image and the change tag image of the subsequent assembly node, continuously selecting training samples to train the detection model, saving model parameters with the optimal similarity during
9 Date Recue/Date Received 2021-06-08 the training process, and completing the training;
performing change detection; obtaining depth images of two successive assembly nodes in the assembly process of the assembly body to be detected, and inputing depth images into the trained detection model to output the change image of the newly added part of the assembly body in the assembly process.
In this embodiment, a detection model is designed to include a semantic fusion module, a feature extraction module, an attention module, and a metric learning module.
The semantic fusion module is configured to fuse semantic images and depth images to obtain fusion images, thereby enriching image feature information; the feature extraction module extracts high-dimensional semantic information and low-dimensional spatial information of the fusion images to obtain initial feature pictures; the attention module quickly focuses on key information of a current task from the initial feature pictures to obtain final feature pictures; and the metric learning module calculates distances between features according to the feature information of the final feature pictures to obtain a change region, and outputs change images of successive assembly nodes of the assembly through the change region, monitoring missing assembly, wrong assembly, and whether the assembly steps are correct during the assembly process of an assembly body by using the change images.
Embodiment 2:
Referring to Fig. 3, this embodiment proposes a specific implementation of the feature extraction module:
firstly, performing 3 x3 convolution on the inputted fusion images, and extracting features of Date Recue/Date Received 2021-06-08 the inputted images by using four stages of a RepVGG classification network;
embedding the attention mechanism after each stage of the RepVGG
classification network to obtain four groups of feature maps respectively, performing 1 x 1 convolution on the four groups of feature maps, and then transforming channel sizes of the feature maps to P1 which is set as an average channel size of the first two groups of feature maps;
performing unified up-sampling on feature picture sizes of the last three stages of feature maps as feature picture sizes of the first stage, performing tandem fusion of channels on the four groups of feature pictures obtained, and performing 3 x3 convolution and 1 x 1 convolution sequentially on the fused feature pictures to obtain the initial feature pictures. Due to adopting the multi-scale feature fusion mechanism, the fused feature pictures have a large amount of information, which is not conducive to the improvement of module speed and accuracy. The two convolutional layers can effectively reduce channel dimensions and make full use of the fused feature information to generate more distinguishing feature quantities.
Further, referring to Fig. 4, the attention mechanism embedded in the RepVGG
classification network is the same as the attention mechanism used by the attention module.
This embodiment proposes a specific implementation of the attention mechanism. The mechanism includes two stages:
in the first stage Self-attention 1, while original feature information is saved, context information of the feature information in horizontal and vertical directions is collected through a Self-attention algorithm to obtain initial attention feature pictures including horizontal and vertical cross paths of the original feature information;
Date Recue/Date Received 2021-06-08 in the second stage Self-attention 2, the generated initial attention feature pictures are fed to Self-attention 2, and other context information is obtained from the horizontal and vertical cross paths through the Self-attention algorithm again to obtain global position dependencies of the feature pictures. The parameter value of each feature vector is shared in the Self-attention 1 stage and the Self-attention 2 stage. The global position dependencies can be captured by only two times of Self-attention algorithm, and information can be collected from all pixels for each position in the attention feature picture finally outputted. The attention module helps to quickly focus on key information of a current task in mass inputted information, reduce irrelevant information, and significantly improve the processing efficiency and accuracy of the task.
Further, this embodiment proposes a specific implementation of the metric learning module:
this module learns nonlinear transformation from inputted data to a mapping space, mapping vectors of similar samples tend to be closer, and mapping vectors of different samples tend to be far away from each other. This, this patent uses a contrast loss function to define a smaller distance and a larger distance of mapping vectors to realize each change in the mapping space. The contrast loss function L is:
1 c-d\I
L = ¨2N n=1 yd2 + (1 ¨ y) max(margin ¨ d, 0)2 (1) Where d =II an ¨ bn 112 represents a Euclidean distance between two sample mapping vectors, y is a tag value obtained through the change tag image, y=1 represents that pixel values of pixels corresponding to the predicted change image and the real change tag image are similar or match, y=0 represents mismatch, N is a number of pixels of the training samples, and margin is a set threshold, which is set to 1 here. The loss function can be understood as two steps. The first Date Recue/Date Received 2021-06-08 step is to calculate distance information of two training samples, that is, feature difference information of two successive images. The second step is to compare the actual difference information between the predicted change image and the real change tag image according to the difference information.
During the specific process, dual-phase feature images at different times and different viewing angles are first given, and the size of each feature picture is adjusted to the same size as the inputted image by bilinear interpolation. Then the Euclidean distance between the feature pictures after the size adjustment is calculated to generate a feature distance picture D E
Rw0x110, where Wo and Ho are respectively the height and width of the inputted image. In the training stage, network parameters are learned and continuously updated through the comparison loss function defined above, a change mask is generated, and the optimal module is saved. In the testing stage, a change region B is obtained according to the optimal model saved by the training network by setting a fixed threshold:
Di,j > 0 Bi,j 1 = (2) 0 else Where i, j(1 i W0, 1 j Ho ) represent width and height indexes of the feature distance picture D, and 0 is a fixed threshold set for a separation change region, which is set to 1 here.
A scene change image of the assembly process is outputted based on the change region B. As shown in Fig. 5 and Fig. 6, Fig. 5 shows a set of training samples, that is, an illustrative diagram of depth images of two adjacent assembly nodes of the three-dimensional model of the assembly body at the front and back moments; in Fig. 6, the left is a change image outputted according to the two Date Recue/Date Received 2021-06-08 depth images in Fig. 5, and the right is an actual change tag image obtained according to the assembly nodes; a similarity is calculated through the loss function, and training samples are continuously selected for iterative training of the model, so that the change image outputted by the model gradually matches the change tag image.
Embodiment 3:
An assembly body change detection device based on an attention mechanism includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the assembly body change detection method based on an attention mechanism according to any embodiment of the present invention.
Embodiment 4:
An assembly body change detection medium based on an attention mechanism stores a computer program thereon, characterized in that the computer program is executed by a processor to implement the assembly body change detection method based on an attention mechanism according to any embodiment of the present invention.
The foregoing descriptions are merely embodiments of the present invention, and do not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation using the description of the present invention and the accompanying drawings, or directly or indirectly applied to other related technologies shall fall within the protection scope of the present invention.
Date Recue/Date Received 2021-06-08
performing change detection; obtaining depth images of two successive assembly nodes in the assembly process of the assembly body to be detected, and inputing depth images into the trained detection model to output the change image of the newly added part of the assembly body in the assembly process.
In this embodiment, a detection model is designed to include a semantic fusion module, a feature extraction module, an attention module, and a metric learning module.
The semantic fusion module is configured to fuse semantic images and depth images to obtain fusion images, thereby enriching image feature information; the feature extraction module extracts high-dimensional semantic information and low-dimensional spatial information of the fusion images to obtain initial feature pictures; the attention module quickly focuses on key information of a current task from the initial feature pictures to obtain final feature pictures; and the metric learning module calculates distances between features according to the feature information of the final feature pictures to obtain a change region, and outputs change images of successive assembly nodes of the assembly through the change region, monitoring missing assembly, wrong assembly, and whether the assembly steps are correct during the assembly process of an assembly body by using the change images.
Embodiment 2:
Referring to Fig. 3, this embodiment proposes a specific implementation of the feature extraction module:
firstly, performing 3 x3 convolution on the inputted fusion images, and extracting features of Date Recue/Date Received 2021-06-08 the inputted images by using four stages of a RepVGG classification network;
embedding the attention mechanism after each stage of the RepVGG
classification network to obtain four groups of feature maps respectively, performing 1 x 1 convolution on the four groups of feature maps, and then transforming channel sizes of the feature maps to P1 which is set as an average channel size of the first two groups of feature maps;
performing unified up-sampling on feature picture sizes of the last three stages of feature maps as feature picture sizes of the first stage, performing tandem fusion of channels on the four groups of feature pictures obtained, and performing 3 x3 convolution and 1 x 1 convolution sequentially on the fused feature pictures to obtain the initial feature pictures. Due to adopting the multi-scale feature fusion mechanism, the fused feature pictures have a large amount of information, which is not conducive to the improvement of module speed and accuracy. The two convolutional layers can effectively reduce channel dimensions and make full use of the fused feature information to generate more distinguishing feature quantities.
Further, referring to Fig. 4, the attention mechanism embedded in the RepVGG
classification network is the same as the attention mechanism used by the attention module.
This embodiment proposes a specific implementation of the attention mechanism. The mechanism includes two stages:
in the first stage Self-attention 1, while original feature information is saved, context information of the feature information in horizontal and vertical directions is collected through a Self-attention algorithm to obtain initial attention feature pictures including horizontal and vertical cross paths of the original feature information;
Date Recue/Date Received 2021-06-08 in the second stage Self-attention 2, the generated initial attention feature pictures are fed to Self-attention 2, and other context information is obtained from the horizontal and vertical cross paths through the Self-attention algorithm again to obtain global position dependencies of the feature pictures. The parameter value of each feature vector is shared in the Self-attention 1 stage and the Self-attention 2 stage. The global position dependencies can be captured by only two times of Self-attention algorithm, and information can be collected from all pixels for each position in the attention feature picture finally outputted. The attention module helps to quickly focus on key information of a current task in mass inputted information, reduce irrelevant information, and significantly improve the processing efficiency and accuracy of the task.
Further, this embodiment proposes a specific implementation of the metric learning module:
this module learns nonlinear transformation from inputted data to a mapping space, mapping vectors of similar samples tend to be closer, and mapping vectors of different samples tend to be far away from each other. This, this patent uses a contrast loss function to define a smaller distance and a larger distance of mapping vectors to realize each change in the mapping space. The contrast loss function L is:
1 c-d\I
L = ¨2N n=1 yd2 + (1 ¨ y) max(margin ¨ d, 0)2 (1) Where d =II an ¨ bn 112 represents a Euclidean distance between two sample mapping vectors, y is a tag value obtained through the change tag image, y=1 represents that pixel values of pixels corresponding to the predicted change image and the real change tag image are similar or match, y=0 represents mismatch, N is a number of pixels of the training samples, and margin is a set threshold, which is set to 1 here. The loss function can be understood as two steps. The first Date Recue/Date Received 2021-06-08 step is to calculate distance information of two training samples, that is, feature difference information of two successive images. The second step is to compare the actual difference information between the predicted change image and the real change tag image according to the difference information.
During the specific process, dual-phase feature images at different times and different viewing angles are first given, and the size of each feature picture is adjusted to the same size as the inputted image by bilinear interpolation. Then the Euclidean distance between the feature pictures after the size adjustment is calculated to generate a feature distance picture D E
Rw0x110, where Wo and Ho are respectively the height and width of the inputted image. In the training stage, network parameters are learned and continuously updated through the comparison loss function defined above, a change mask is generated, and the optimal module is saved. In the testing stage, a change region B is obtained according to the optimal model saved by the training network by setting a fixed threshold:
Di,j > 0 Bi,j 1 = (2) 0 else Where i, j(1 i W0, 1 j Ho ) represent width and height indexes of the feature distance picture D, and 0 is a fixed threshold set for a separation change region, which is set to 1 here.
A scene change image of the assembly process is outputted based on the change region B. As shown in Fig. 5 and Fig. 6, Fig. 5 shows a set of training samples, that is, an illustrative diagram of depth images of two adjacent assembly nodes of the three-dimensional model of the assembly body at the front and back moments; in Fig. 6, the left is a change image outputted according to the two Date Recue/Date Received 2021-06-08 depth images in Fig. 5, and the right is an actual change tag image obtained according to the assembly nodes; a similarity is calculated through the loss function, and training samples are continuously selected for iterative training of the model, so that the change image outputted by the model gradually matches the change tag image.
Embodiment 3:
An assembly body change detection device based on an attention mechanism includes a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the assembly body change detection method based on an attention mechanism according to any embodiment of the present invention.
Embodiment 4:
An assembly body change detection medium based on an attention mechanism stores a computer program thereon, characterized in that the computer program is executed by a processor to implement the assembly body change detection method based on an attention mechanism according to any embodiment of the present invention.
The foregoing descriptions are merely embodiments of the present invention, and do not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation using the description of the present invention and the accompanying drawings, or directly or indirectly applied to other related technologies shall fall within the protection scope of the present invention.
Date Recue/Date Received 2021-06-08
Claims (7)
1. An assembly body change detection method based on an attention mechanism, comprising the following steps:
establishing a data set; establishing a three-dimensional model of an assembly body, adding a tag to each part in the three-dimensional model, determining several assembly nodes according to given assembly steps of the assembly body, perfoiming imaging processing on the three-dimensional model under each assembly node to obtain depth images of the three-dimensional model under each assembly node in different viewing angles, and obtaining a change tag image of a newly added part at each assembly node according to the tag of the part;
training a detection model; selecting two depth images of the three-dimensional model of two successive adjacent assembly nodes in different viewing angles as training samples;
perfonning semantic segmentation on the two depth images respectively to obtain semantic images, and performing pixel fusion on the two semantic images and the corresponding depth images to obtain two fusion images; performing feature extraction on the two fusion images respectively to obtain two corresponding initial feature pictures; collecting context information of the two initial feature pictures through the attention mechanism, and capturing global position dependencies of the initial feature pictures according to the context information to obtain two corresponding attention feature pictures; calculating a Euclidean distance between the two attention feature pictures, generating a feature distance picture, and outputting a change image according to the feature distance picture; defining a loss function to calculate a similarity between the change image and the change tag image, continuously selecting training samples to train the detection model, saving model parameters with the optimal similarity during the training process, and completing the training; and performing change detection; obtaining the depth images of the two successive assembly nodes in the assembly process of the assembly body to be detected, inputting the depth images into the trained detection model, and outputting the change image of the newly added part of the assembly body in the assembly process.
Date Recue/Date Received 2023-01-16
establishing a data set; establishing a three-dimensional model of an assembly body, adding a tag to each part in the three-dimensional model, determining several assembly nodes according to given assembly steps of the assembly body, perfoiming imaging processing on the three-dimensional model under each assembly node to obtain depth images of the three-dimensional model under each assembly node in different viewing angles, and obtaining a change tag image of a newly added part at each assembly node according to the tag of the part;
training a detection model; selecting two depth images of the three-dimensional model of two successive adjacent assembly nodes in different viewing angles as training samples;
perfonning semantic segmentation on the two depth images respectively to obtain semantic images, and performing pixel fusion on the two semantic images and the corresponding depth images to obtain two fusion images; performing feature extraction on the two fusion images respectively to obtain two corresponding initial feature pictures; collecting context information of the two initial feature pictures through the attention mechanism, and capturing global position dependencies of the initial feature pictures according to the context information to obtain two corresponding attention feature pictures; calculating a Euclidean distance between the two attention feature pictures, generating a feature distance picture, and outputting a change image according to the feature distance picture; defining a loss function to calculate a similarity between the change image and the change tag image, continuously selecting training samples to train the detection model, saving model parameters with the optimal similarity during the training process, and completing the training; and performing change detection; obtaining the depth images of the two successive assembly nodes in the assembly process of the assembly body to be detected, inputting the depth images into the trained detection model, and outputting the change image of the newly added part of the assembly body in the assembly process.
Date Recue/Date Received 2023-01-16
2. The assembly body change detection method based on an attention mechanism according to claim 1, characterized in that in the step of performing feature extraction on the two fusion images to obtain two corresponding initial feature maps, the feature extraction method is specifically as follows:
performing 3x3 convolution on the inputted fusion images, and extracting features of the inputted images by using four stages of a RepVGG classification network;
embedding the attention mechanism after each stage of the RepVGG
classification network to obtain four groups of feature maps respectively, performing 1x 1 convolution on the four groups of feature maps, and transforming channel sizes of the feature maps to PI which is set as an average channel size of the first two groups of feature maps; and performing unified up-sampling on feature picture sizes of the last three stages of feature maps as feature picture sizes of the first stage, performing tandem fusion of channels on the four groups of feature pictures obtained, and performing 3x3 convolution and 1x1 convolution sequentially on the fused feature maps to obtain the initial feature pictures.
performing 3x3 convolution on the inputted fusion images, and extracting features of the inputted images by using four stages of a RepVGG classification network;
embedding the attention mechanism after each stage of the RepVGG
classification network to obtain four groups of feature maps respectively, performing 1x 1 convolution on the four groups of feature maps, and transforming channel sizes of the feature maps to PI which is set as an average channel size of the first two groups of feature maps; and performing unified up-sampling on feature picture sizes of the last three stages of feature maps as feature picture sizes of the first stage, performing tandem fusion of channels on the four groups of feature pictures obtained, and performing 3x3 convolution and 1x1 convolution sequentially on the fused feature maps to obtain the initial feature pictures.
3. The assembly body change detection method based on an attention mechanism according to claim 2, characterized in that the attention mechanism embedded in the RepVGG classification network is the same as the attention mechanism used in the step of collecting context information of the two initial feature maps, and the attention mechanism comprises two stages:
in the first stage, while saving original feature information, collecting context information of the feature information in horizontal and vertical directions through a self-attention algorithm to obtain horizontal and vertical cross paths of the original feature information; and in the second stage, obtaining other context information from the horizontal and vertical cross paths through the self-attention algorithm again to obtain the global position dependencies of the feature pictures.
Date Recue/Date Received 2023-01-16
in the first stage, while saving original feature information, collecting context information of the feature information in horizontal and vertical directions through a self-attention algorithm to obtain horizontal and vertical cross paths of the original feature information; and in the second stage, obtaining other context information from the horizontal and vertical cross paths through the self-attention algorithm again to obtain the global position dependencies of the feature pictures.
Date Recue/Date Received 2023-01-16
4. The assembly body change detection method based on an attention mechanism according to claim 1, characterized in that the specific step of calculating a Euclidean distance between the two attention feature pictures, generating a feature distance picture, and outputting a change image according to the feature distance picture is specifically as follows:
first, adjusting sizes of the two inputted attention feature pictures to the same sizes as the initial inputted depth images by bilinear interpolation;
calculating the Euclidean distance between the two attention feature pictures after the size adjustment, and generating the feature distance picture; and setting a change threshold according to the distance information in the feature distance picture, dividing a depth image region into a changed region and an unchanged region through the change threshold and the distance information, and outputting the change image of the subsequent assembly node relative to the previous assembly node accordingly.
first, adjusting sizes of the two inputted attention feature pictures to the same sizes as the initial inputted depth images by bilinear interpolation;
calculating the Euclidean distance between the two attention feature pictures after the size adjustment, and generating the feature distance picture; and setting a change threshold according to the distance information in the feature distance picture, dividing a depth image region into a changed region and an unchanged region through the change threshold and the distance information, and outputting the change image of the subsequent assembly node relative to the previous assembly node accordingly.
5. The assembly body change detection method based on an attention mechanism according to claim 1, characterized the loss function is specifically:
where d= an¨bnif 2 represents a Euclidean distance between mapping vectors of two training samples, y is a tag value obtained through the change tag image, y=1 represents similarity, y=0 represents dissimilarity, N is the quantity of pixels of the training samples, and margin is a threshold set artificially.
where d= an¨bnif 2 represents a Euclidean distance between mapping vectors of two training samples, y is a tag value obtained through the change tag image, y=1 represents similarity, y=0 represents dissimilarity, N is the quantity of pixels of the training samples, and margin is a threshold set artificially.
6. An assembly body change detection device based on an attention mechanism, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the program to implement the assembly body change detection method based on an attention mechanism according to any one of claims 1 to 5.
Date Recue/Date Received 2023-01-16
Date Recue/Date Received 2023-01-16
7. An assembly body change detection medium based on an attention mechanism, storing a computer program thereon, characterized in that the computer program is executed by a processor to implement the assembly body change detection method based on an attention mechanism according to any one of claims 1 to 5.
Date Recue/Date Received 2023-01-16
Date Recue/Date Received 2023-01-16
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110507300.9A CN113269237B (en) | 2021-05-10 | 2021-05-10 | Assembly change detection method, device and medium based on attention mechanism |
| CN202110507300.9 | 2021-05-10 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CA3121440A1 CA3121440A1 (en) | 2021-11-16 |
| CA3121440C true CA3121440C (en) | 2023-11-07 |
Family
ID=77230275
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA3121440A Active CA3121440C (en) | 2021-05-10 | 2021-06-08 | Assembly body change detection method, device and medium based on attention mechanism |
Country Status (3)
| Country | Link |
|---|---|
| US (1) | US11630972B2 (en) |
| CN (1) | CN113269237B (en) |
| CA (1) | CA3121440C (en) |
Families Citing this family (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020255227A1 (en) * | 2019-06-17 | 2020-12-24 | 日本電信電話株式会社 | Learning device, search device, learning method, search method, learning program, and search program |
| CN115019578A (en) * | 2022-05-18 | 2022-09-06 | 湖南铁道职业技术学院 | Component dismounting training method and system |
| CN115033998B (en) * | 2022-07-13 | 2023-02-21 | 北京航空航天大学 | Personalized 2D data set construction method for mechanical parts |
| CN115239515B (en) * | 2022-07-28 | 2023-04-07 | 德玛克(长兴)精密机械有限公司 | Precise intelligent processing and manufacturing system for mechanical parts and manufacturing method thereof |
| CN115761815B (en) * | 2023-01-10 | 2023-04-25 | 合肥中科类脑智能技术有限公司 | Training method of human body detection model, human body detection method, device and medium |
| CN116205856B (en) * | 2023-02-01 | 2023-09-08 | 哈尔滨市科佳通用机电股份有限公司 | Method and system for detecting breaking fault of shaft chain of human brake based on deep learning |
| CN116052007B (en) * | 2023-03-30 | 2023-08-11 | 山东锋士信息技术有限公司 | Remote sensing image change detection method integrating time and space information |
| CN116343045B (en) * | 2023-03-30 | 2024-03-19 | 南京理工大学 | Lightweight SAR image ship target detection method based on YOLO v5 |
| CN116091848B (en) * | 2023-04-03 | 2023-07-04 | 青岛创新奇智科技集团股份有限公司 | Test tube classification method, device, equipment and storage medium |
| CN116452983B (en) * | 2023-06-12 | 2023-10-10 | 合肥工业大学 | Quick discovering method for land landform change based on unmanned aerial vehicle aerial image |
| CN116665019B (en) * | 2023-07-31 | 2023-09-29 | 山东交通学院 | Multi-axis interaction multi-dimensional attention network for vehicle re-identification |
| CN117036756B (en) * | 2023-08-08 | 2024-04-05 | 重庆市地理信息和遥感应用中心(重庆市测绘产品质量检验测试中心) | Remote sensing image matching method and system based on variation automatic encoder |
| CN116881704B (en) * | 2023-09-06 | 2023-11-14 | 北京新亚盛创电气技术有限公司 | Early warning method and system for power grid running state |
| CN116958907B (en) * | 2023-09-18 | 2023-12-26 | 四川泓宝润业工程技术有限公司 | Method and system for inspecting surrounding hidden danger targets of gas pipeline |
| CN117671432A (en) * | 2024-01-31 | 2024-03-08 | 苏州元脑智能科技有限公司 | Method and device for training change analysis model, electronic equipment and storage medium |
Family Cites Families (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3304423A1 (en) * | 2015-06-05 | 2018-04-11 | Siemens Aktiengesellschaft | Method and system for simultaneous scene parsing and model fusion for endoscopic and laparoscopic navigation |
| US10452960B1 (en) * | 2018-10-01 | 2019-10-22 | Texas Instruments Incorporated | Image classification |
| US10755128B2 (en) * | 2018-12-18 | 2020-08-25 | Slyce Acquisition Inc. | Scene and user-input context aided visual search |
| US11361470B2 (en) * | 2019-05-09 | 2022-06-14 | Sri International | Semantically-aware image-based visual localization |
| CN110705457B (en) * | 2019-09-29 | 2024-01-19 | 核工业北京地质研究院 | Remote sensing image building change detection method |
| EP4058983A1 (en) * | 2019-11-12 | 2022-09-21 | Geomagical Labs, Inc. | Method and system for scene image modification |
| CN111127493A (en) * | 2019-11-12 | 2020-05-08 | 中国矿业大学 | Remote sensing image semantic segmentation method based on attention multi-scale feature fusion |
| CN111160311B (en) * | 2020-01-02 | 2022-05-17 | 西北工业大学 | Yellow river ice semantic segmentation method based on multi-attention machine system double-flow fusion network |
| CN111274892B (en) * | 2020-01-14 | 2020-12-18 | 北京科技大学 | Robust remote sensing image change detection method and system |
| CN111476714B (en) * | 2020-03-30 | 2022-10-28 | 清华大学 | Cross-scale image splicing method and device based on PSV neural network |
| CN111445474B (en) * | 2020-05-25 | 2023-06-27 | 南京信息工程大学 | Kidney CT image segmentation method based on bidirectional re-attention depth network |
| CN111797779A (en) * | 2020-07-08 | 2020-10-20 | 兰州交通大学 | Remote sensing image semantic segmentation method based on regional attention multi-scale feature fusion |
| CN112288750B (en) * | 2020-11-20 | 2022-09-20 | 青岛理工大学 | Mechanical assembly image segmentation method and device based on deep learning network |
| CN112488025B (en) * | 2020-12-10 | 2022-06-14 | 武汉大学 | Double-temporal remote sensing image semantic change detection method based on multi-modal feature fusion |
| CN112668494A (en) * | 2020-12-31 | 2021-04-16 | 西安电子科技大学 | Small sample change detection method based on multi-scale feature extraction |
-
2021
- 2021-05-10 CN CN202110507300.9A patent/CN113269237B/en active Active
- 2021-06-08 CA CA3121440A patent/CA3121440C/en active Active
- 2021-06-09 US US17/342,592 patent/US11630972B2/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| US20220358334A1 (en) | 2022-11-10 |
| CN113269237A (en) | 2021-08-17 |
| CN113269237B (en) | 2022-12-27 |
| US11630972B2 (en) | 2023-04-18 |
| CA3121440A1 (en) | 2021-11-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA3121440C (en) | Assembly body change detection method, device and medium based on attention mechanism | |
| CN110163953B (en) | Three-dimensional face reconstruction method and device, storage medium and electronic device | |
| US11763433B2 (en) | Depth image generation method and device | |
| WO2022166412A1 (en) | Self-supervised three-dimensional reconstruction method and system based on collaborative segmentation and data enhancement | |
| CN111563415B (en) | Binocular vision-based three-dimensional target detection system and method | |
| JP6902122B2 (en) | Double viewing angle Image calibration and image processing methods, equipment, storage media and electronics | |
| CN110381268B (en) | Method, device, storage medium and electronic equipment for generating video | |
| CN109525786B (en) | Video processing method and device, terminal equipment and storage medium | |
| US11651581B2 (en) | System and method for correspondence map determination | |
| CN113838191A (en) | Three-dimensional reconstruction method based on attention mechanism and monocular multi-view | |
| CN112734822A (en) | Stereo matching algorithm based on infrared and visible light images | |
| CN114140623A (en) | Image feature point extraction method and system | |
| CN115035235A (en) | Three-dimensional reconstruction method and device | |
| CN114663686A (en) | Object feature point matching method and device, and training method and device | |
| CN111105451B (en) | Driving scene binocular depth estimation method for overcoming occlusion effect | |
| CN110443228B (en) | Pedestrian matching method and device, electronic equipment and storage medium | |
| CN110120012B (en) | Video stitching method for synchronous key frame extraction based on binocular camera | |
| Koch et al. | Comparison of monocular depth estimation methods using geometrically relevant metrics on the IBims-1 dataset | |
| Huang et al. | ES-Net: An efficient stereo matching network | |
| CN116823816B (en) | Detection equipment and detection method based on security monitoring static memory | |
| Nouduri et al. | Deep realistic novel view generation for city-scale aerial images | |
| Liu et al. | Geometrized Transformer for Self-Supervised Homography Estimation | |
| CN114022458A (en) | Skeleton detection method and device, electronic equipment and computer readable storage medium | |
| Lu | Bird-View 3D Reconstruction for Crops with Repeated Textures | |
| Choi et al. | Online 3D edge reconstruction of wiry structures from monocular image sequences |