CN115205543A - Intelligent manufacturing method and system of stainless steel cabinet - Google Patents

Intelligent manufacturing method and system of stainless steel cabinet Download PDF

Info

Publication number
CN115205543A
CN115205543A CN202210849271.9A CN202210849271A CN115205543A CN 115205543 A CN115205543 A CN 115205543A CN 202210849271 A CN202210849271 A CN 202210849271A CN 115205543 A CN115205543 A CN 115205543A
Authority
CN
China
Prior art keywords
feature
global
map
feature map
view
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210849271.9A
Other languages
Chinese (zh)
Inventor
顾冰程
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ningbo Xikaman Kitchenware Co ltd
Original Assignee
Ningbo Xikaman Kitchenware Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ningbo Xikaman Kitchenware Co ltd filed Critical Ningbo Xikaman Kitchenware Co ltd
Priority to CN202210849271.9A priority Critical patent/CN115205543A/en
Publication of CN115205543A publication Critical patent/CN115205543A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/16—Image acquisition using multiple overlapping images; Image stitching
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G06V10/765—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects using rules for classification or partitioning the feature space
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Image Analysis (AREA)

Abstract

The application relates to the field of intelligent manufacturing detection, and particularly discloses an intelligent manufacturing method and system of a stainless steel cabinet, wherein a convolutional neural network model based on a deep learning technology is adopted to extract implicit associated features from a plurality of visual angle images of the assembled stainless steel cabinet collected from a plurality of visual angles through local and global aspects, and homography alignment of hierarchical scene depth streams based on vector differential expression is performed by using a method of hierarchical depth homography alignment fusion during feature fusion, so that the classification effect of the fused features is improved, and further, the assembly quality of the assembled stainless steel cabinet can be accurately detected.

Description

Intelligent manufacturing method and system of stainless steel cabinet
Technical Field
The present invention relates to the field of intelligent manufacturing detection, and more particularly, to an intelligent manufacturing method of a stainless steel cabinet and a system thereof.
Background
The requirements of people on living environment and quality are continuously improved, and the metamorphosis from a living room to a bedroom to a kitchen shows the requirements of people on the living quality. For the whole kitchen, the stainless steel cabinet is taken as a main body, and is matched with modern products, so that the taste of life can be reflected. Due to the particularity of the kitchen, it is not possible to change the kitchen freely, so the quality of the cabinet becomes very important, and the quality inspection of the stainless steel cabinet after the manufacturing and assembling is also very important.
However, most of the existing stainless steel cabinets need to be inspected by personal inspection equipment after being assembled, which not only takes time and costs, but also results in poor inspection accuracy. Therefore, an optimized method for testing the assembly quality of stainless steel cabinets is desired.
Disclosure of Invention
The present application is proposed to solve the above-mentioned technical problems. The embodiment of the application provides an intelligent manufacturing method and system of a stainless steel cabinet, wherein a convolutional neural network model based on a deep learning technology is adopted to extract implicit associated features from a plurality of visual angle images of the assembled stainless steel cabinet collected from a plurality of visual angles in local and global aspects, and homography alignment of hierarchical scene depth streams based on vector differential expression is performed by using a hierarchical depth homography alignment fusion method during feature fusion, so that the classification effect of the fused features is improved, and further, the assembly quality of the assembled stainless steel cabinet can be accurately detected.
According to an aspect of the present application, there is provided a method of smart manufacturing of a stainless steel cabinet, comprising:
acquiring a plurality of perspective images of the assembled stainless steel cabinet from a plurality of perspectives by a camera;
respectively passing each view image in the plurality of view images through a first convolutional neural network serving as a feature extractor to obtain a plurality of local view feature maps;
arranging the plurality of local view angle characteristic graphs into a three-dimensional input tensor, and then obtaining an inter-view angle association characteristic graph by using a second convolution neural network of a three-dimensional convolution kernel;
carrying out panoramic stitching on the plurality of local view angle characteristic graphs to obtain a panoramic characteristic graph;
passing the panoramic feature map through a non-local neural network to obtain a global association feature map;
respectively expanding the inter-view association feature map and the global association feature map into vectors to obtain an inter-view association feature vector and a global association feature vector;
fusing the inter-view association feature vector and the global association feature vector to obtain a classification feature vector; and
and passing the classified feature vector through a classifier to obtain a classification result, wherein the classification result is used for indicating whether the assembling quality of the assembled stainless steel cabinet meets a preset requirement or not.
In the above intelligent manufacturing method of a stainless steel cabinet, passing each of the plurality of perspective images through a first convolution neural network as a feature extractor to obtain a plurality of local perspective feature maps, respectively, includes: each layer of the first convolutional neural network performs input data in forward transmission of the layer respectively: performing convolution processing on input data to obtain a convolution characteristic diagram; pooling the convolution characteristic map to obtain a pooled characteristic map; and performing nonlinear activation on the pooled feature map to obtain an activated feature map; wherein, the output of the last layer of the first convolutional neural network is the plurality of local view feature maps, and the input of the first layer of the first convolutional neural network is each view image in the plurality of view images.
In the above intelligent manufacturing method of a stainless steel cabinet, after arranging the plurality of local perspective feature maps as a three-dimensional input tensor, obtaining an inter-perspective correlation feature map by using a second convolution neural network of a three-dimensional convolution kernel, the method includes: the second convolutional neural network using the three-dimensional convolutional kernel performs, in forward transfer of layers, respectively: performing three-dimensional convolution processing on the input data based on the three-dimensional convolution kernel to obtain a convolution characteristic diagram; pooling the convolution characteristic map to obtain a pooled characteristic map; and performing nonlinear activation on the pooled feature map to obtain an activated feature map; wherein the output of the last layer of the second convolutional neural network is the inter-view correlation feature map, and the input of the first layer of the second convolutional neural network is the three-dimensional input tensor.
In the above intelligent manufacturing method of a stainless steel cabinet, the panoramic characteristic map is obtained by panoramic stitching the plurality of local viewing angle characteristic maps, and the method includes: and splicing the plurality of local view angle characteristic graphs along the height dimension or the width dimension to obtain the panoramic characteristic graph.
In the above intelligent manufacturing method of stainless steel cabinet, the passing the panoramic characteristic map through a non-local neural network to obtain a global correlation characteristic map includes: inputting the panoramic characteristic map into a first point convolutional layer, a second point convolutional layer and a third point convolutional layer of the non-local neural network respectively to obtain a first characteristic map, a second characteristic map and a third characteristic map; calculating a position-weighted sum of the first feature map and the second feature map to obtain an intermediate fused feature map; inputting the intermediate fusion feature map into a Softmax function to normalize the feature values of the positions in the intermediate fusion feature map to obtain a normalized intermediate fusion feature map; calculating a position-weighted sum of the normalized intermediate fused feature map and the third feature map to obtain a re-fused feature map; calculating the similarity among the feature values of all positions in the re-fused feature map by embedding a Gaussian similarity function into the re-fused feature map to obtain a global perception feature map; passing the global perception feature map through a fourth convolution layer of the non-local neural network to obtain a channel adjustment global perception feature map; and calculating a position-weighted sum of the channel adjustment global perception feature map and the high-dimensional association local feature map to obtain the global association feature map.
In the above intelligent manufacturing method of a stainless steel cabinet, the expanding the inter-view correlation feature map and the global correlation feature map into vectors to obtain an inter-view correlation feature vector and a global correlation feature vector respectively includes: dividing each feature matrix in the inter-view associated feature map and each feature matrix in the global associated feature map by taking a row vector as a dividing unit to obtain a plurality of first row vectors corresponding to the inter-view associated feature map and a plurality of second row vectors corresponding to the global associated feature map; splicing the plurality of first line vectors to obtain the associated characteristic vectors among the visual angles; and splicing the second line vectors to obtain the global correlation characteristic vector.
In the above intelligent manufacturing method of a stainless steel cabinet, fusing the inter-view correlation feature vector and the global correlation feature vector to obtain a classification feature vector, including: calculating the position-based difference between the inter-view associated feature vector and the global associated feature vector to obtain a difference feature vector; calculating the sum of the inter-view associated feature vectors and the global associated feature vector according to positions to obtain a sum feature vector; calculating a full-field Jing Shanying correlation matrix between the inter-view correlation feature vector and the global correlation feature vector; performing a logarithm operation on the added feature vector to obtain a logarithm added feature vector, wherein the logarithm operation on the added feature vector represents calculating a logarithm function value of a feature value of each position in the added feature vector; calculating a norm of the differential feature vector as a hierarchical depth characteristic value of the differential feature vector; calculating the Frobenius norm of the full-scene homographic incidence matrix as the depth perception value of the full-field Jing Shanying incidence matrix; weighting the logarithmic summation feature vector according to the position by taking the hierarchical depth characteristic value as a weighting coefficient to obtain a weighted feature vector; and calculating the sum of the depth perception value and the feature value of each position in the weighted feature vector by taking the depth perception value as a bias to obtain the classified feature vector.
In the above intelligent manufacturing method of a stainless steel cabinet, calculating a full-field Jing Shanying correlation matrix between the inter-view correlation feature vector and the global correlation feature vector includes: calculating a product between the transposed vector of the inter-view relevance eigenvector and the global relevance eigenvector to obtain the full-field Jing Shanying relevance matrix.
In the intelligent manufacturing method of the stainless steel cabinet, the classified feature vector is obtained by passing through a classifierTo the classification result, including: processing the classification feature vector using the classifier to obtain the classification result with the following formula: softmax { (W) n ,B n ):…:(W 1 ,B 1 ) | X }, wherein W 1 To W n As a weight matrix, B 1 To B n Is a bias vector and X is the classification feature vector.
According to another aspect of the present application, there is provided an intelligent manufacturing system of a stainless steel cabinet, comprising:
an image data acquisition unit for acquiring a plurality of perspective images of the assembled stainless steel cabinet from a plurality of perspectives by a camera;
the local feature extraction unit is used for enabling each visual angle image in the multiple visual angle images to pass through a first convolution neural network serving as a feature extractor respectively to obtain multiple local visual angle feature maps;
the three-dimensional convolution unit is used for arranging the plurality of local view angle characteristic graphs into a three-dimensional input tensor and then obtaining an inter-view angle correlation characteristic graph through a second convolution neural network using a three-dimensional convolution kernel;
the panorama splicing unit is used for carrying out panorama splicing on the plurality of local visual angle characteristic graphs to obtain a panorama characteristic graph;
the global feature extraction unit is used for enabling the panoramic feature map to pass through a non-local neural network to obtain a global associated feature map;
the dimension reduction unit is used for respectively expanding the inter-view association feature map and the global association feature map into vectors so as to obtain an inter-view association feature vector and a global association feature vector;
the feature fusion unit is used for fusing the inter-view associated feature vector and the global associated feature vector to obtain a classified feature vector; and
and the classification unit is used for enabling the classification characteristic vectors to pass through a classifier to obtain a classification result, and the classification result is used for indicating whether the assembly quality of the assembled stainless steel cabinet meets a preset requirement or not.
In the above intelligent manufacturing system for stainless steel cabinets, the local feature extraction unit is further configured to: each layer of the first convolutional neural network performs input data in forward transmission of the layer respectively: performing convolution processing on input data to obtain a convolution characteristic diagram; pooling the convolution characteristic map to obtain a pooled characteristic map; and performing nonlinear activation on the pooled feature map to obtain an activated feature map; wherein, the output of the last layer of the first convolutional neural network is the plurality of local view angle feature maps, and the input of the first layer of the first convolutional neural network is each view angle image in the plurality of view angle images.
In the above intelligent manufacturing system for stainless steel cabinets, the three-dimensional convolution unit is further configured to: the second convolutional neural network using the three-dimensional convolutional kernel performs, in forward transfer of layers, respectively: performing three-dimensional convolution processing on the input data based on the three-dimensional convolution kernel to obtain a convolution characteristic diagram; pooling the convolution characteristic map to obtain a pooled characteristic map; and performing nonlinear activation on the pooled feature map to obtain an activated feature map; wherein the output of the last layer of the second convolutional neural network is the inter-view correlation feature map, and the input of the first layer of the second convolutional neural network is the three-dimensional input tensor.
In the above intelligent manufacturing system for stainless steel cabinets, the panoramic stitching unit is further configured to: and splicing the plurality of local view angle characteristic graphs along the height dimension or the width dimension to obtain the panoramic characteristic graph.
In the above intelligent manufacturing system for stainless steel cabinets, the global feature extraction unit is further configured to: inputting the panoramic characteristic map into a first point convolutional layer, a second point convolutional layer and a third point convolutional layer of the non-local neural network respectively to obtain a first characteristic map, a second characteristic map and a third characteristic map; calculating a position-weighted sum of the first feature map and the second feature map to obtain an intermediate fused feature map; inputting the intermediate fusion feature map into a Softmax function to normalize the feature values of the positions in the intermediate fusion feature map to obtain a normalized intermediate fusion feature map; calculating a position-weighted sum of the normalized intermediate fused feature map and the third feature map to obtain a re-fused feature map; calculating the similarity among the feature values of all positions in the re-fused feature map by embedding a Gaussian similarity function into the re-fused feature map to obtain a global perception feature map; passing the global perception feature map through a fourth convolution layer of the non-local neural network to obtain a channel adjustment global perception feature map; and calculating a position-weighted sum of the channel adjustment global perception feature map and the high-dimensional association local feature map to obtain the global association feature map.
In the above intelligent manufacturing system for stainless steel cabinets, the dimension reduction unit is further configured to: dividing each feature matrix in the inter-view associated feature map and each feature matrix in the global associated feature map by taking a row vector as a dividing unit to obtain a plurality of first row vectors corresponding to the inter-view associated feature map and a plurality of second row vectors corresponding to the global associated feature map; splicing the plurality of first line vectors to obtain the associated characteristic vectors among the visual angles; and splicing the second line vectors to obtain the global correlation characteristic vector.
In the above-mentioned intelligent manufacturing system of stainless steel cabinet, the feature fusion unit includes: the difference subunit is used for calculating the position-based difference between the inter-view correlation feature vector and the global correlation feature vector to obtain a difference feature vector; the summing subunit is used for calculating the sum of the inter-view associated feature vectors and the global associated feature vector according to the positions to obtain a summed feature vector; the incidence matrix subunit is used for calculating a full-field Jing Shanying incidence matrix between the inter-view incidence characteristic vector and the global incidence characteristic vector; a logarithm calculation subunit, configured to perform a logarithm operation on the summed feature vector to obtain a logarithm summed feature vector, where the performing of the logarithm operation on the summed feature vector indicates calculating a logarithm function value of a feature value of each position in the summed feature vector; the characteristic value operator unit is used for calculating a norm of the differential characteristic vector as a hierarchical depth characteristic value of the differential characteristic vector; the depth perception value operator unit is used for calculating the Frobenius norm of the full-scene homographic incidence matrix as the depth perception value of the full-field Jing Shanying incidence matrix; the weighting subunit is used for weighting the logarithm addition characteristic vector according to the position by taking the hierarchical depth characteristic value as a weighting coefficient to obtain a weighted characteristic vector; and the fusion subunit is used for calculating the sum of the depth perception value and the feature value of each position in the weighted feature vector by taking the depth perception value as a bias so as to obtain the classified feature vector.
In the above intelligent manufacturing system for stainless steel cabinets, the correlation matrix subunit is further configured to: calculating a product between the transposed vector of the inter-view relevance eigenvector and the global relevance eigenvector to obtain the full-field Jing Shanying relevance matrix.
In the above intelligent manufacturing system for stainless steel cabinets, the sorting unit is further configured to: processing the classification feature vector using the classifier to obtain the classification result with the following formula: softmax { (W) n ,B n ):…:(W 1 ,B 1 ) I X }, wherein W 1 To W n As a weight matrix, B 1 To B n X is the classified feature vector.
According to yet another aspect of the present application, there is provided a computer readable medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform the intelligent manufacturing method of stainless steel cabinets as described above.
Compared with the prior art, the intelligent manufacturing method and system for the stainless steel cabinet, provided by the application, adopt the convolutional neural network model based on the deep learning technology to extract implicit associated features from a plurality of visual angle images of the assembled stainless steel cabinet collected from a plurality of visual angles through local and global aspects, and use the method of hierarchical depth homography alignment fusion to perform homography alignment of hierarchical scene depth streams based on vector differential expression during feature fusion, so that the classification effect of the fused features is improved, and further, the assembly quality of the assembled stainless steel cabinet can be accurately detected.
Drawings
The above and other objects, features and advantages of the present application will become more apparent by describing in more detail embodiments of the present application with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings, like reference numbers generally represent like parts or steps.
Fig. 1 is an application scenario diagram of an intelligent manufacturing method of a stainless steel cabinet according to an embodiment of the present application;
FIG. 2 is a flow chart of a method of smart manufacturing of stainless steel cabinets according to an embodiment of the present application;
fig. 3 is a system architecture diagram of a method for intelligent manufacturing of stainless steel cabinets according to an embodiment of the present application;
fig. 4 is a flowchart of fusing the inter-view correlation feature vector and the global correlation feature vector to obtain a classification feature vector in the intelligent manufacturing method of a stainless steel cabinet according to an embodiment of the present application;
FIG. 5 is a block diagram of an intelligent manufacturing system for stainless steel cabinets according to an embodiment of the present application;
fig. 6 is a block diagram of a feature fusion unit in an intelligent manufacturing system for stainless steel cabinets according to an embodiment of the present application.
Detailed Description
Hereinafter, example embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be understood that the described embodiments are only some embodiments of the present application and not all embodiments of the present application, and that the present application is not limited by the example embodiments described herein.
Overview of a scene
As mentioned above, the requirements of people on living environment and quality are continuously improved, and the metamorphosis from the living room to the bedroom to the kitchen shows up the requirements of people on living quality everywhere. For the whole kitchen, the stainless steel cabinet is used as a main body and is matched with modern products, so that the taste of life can be reflected. Due to the particularity of the kitchen, the cabinet cannot be replaced freely, so that the quality of the cabinet becomes very important, and the quality inspection of the stainless steel cabinet after the manufacturing and assembling is also very important.
However, most of the existing stainless steel cabinets need to be inspected by personal inspection equipment after being assembled, which not only takes time and costs, but also results in poor inspection accuracy. Therefore, an optimized method for testing the assembly quality of stainless steel cabinets is desired.
Accordingly, the present inventors have considered that quality inspection of cabinet assembly is attempted only through images without performing structural strength check and the like through various inspection equipment, with the development of deep neural networks, reducing quality inspection costs.
Based on this, in the technical scheme of this application, first, gather the multiple perspective images of the stainless steel cupboard that the completion was assembled from multiple perspectives through the camera. Then, the images of the multiple visual angles are subjected to feature extraction processing in a convolutional neural network model with excellent performance in image feature extraction, so that local implicit feature distribution information in the images of the multiple visual angles is extracted, and a plurality of local visual angle feature maps are obtained.
It should be understood that, since the perspective images of the assembled stainless steel cabinet have the characteristic information associated with each other, in order to better extract the characteristic information about the assembly quality of the stainless steel cabinet from the perspective images, the local perspective characteristic maps are further arranged as a three-dimensional input tensor, and then a second convolution neural network of a three-dimensional convolution kernel is used to obtain the inter-perspective associated characteristic map. Here, the second convolutional neural network can deeply mine spatial feature information of the assembled stainless steel cabinet using a three-dimensional convolutional kernel.
Moreover, considering that convolution is a typical local operation, for each pixel point in the multiple perspective images of the assembled stainless steel cabinet, the pixel points are not isolated, and the relevance between the pixels generates a foreground object. Therefore, in the technical solution of the present application, in order to extract the relevance between a certain pixel point and all the remaining pixel points of the multiple perspective images of the assembled stainless steel cabinet, a non-local neural network is used to further extract the features of the images. Namely, the plurality of local view angle feature maps are subjected to panoramic stitching to obtain a panoramic feature map, and the panoramic feature map is passed through a non-local neural network to obtain a global association feature map. Particularly, the non-local neural network captures remote dependence information by calculating the similarity of all pixel points of the image, further models context features, enables the network to pay attention to the whole content of the image, and further improves the extraction capability of the main network features in classification and detection tasks.
Further, when the inter-view correlation feature map and the global correlation feature map are fused, considering that the inter-view correlation feature map itself performs feature extraction through a first convolutional neural network and a second convolutional neural network which are cascaded, and the global correlation feature map further increases the layer depth through the convolution operation of a non-local neural network on the basis of the inter-view correlation feature map, in the technical scheme of the application, when the inter-view correlation feature map and the global correlation feature map are fused, the hierarchical depth homography alignment fusion is performed.
Specifically, the inter-view association feature map and the global association feature map are first expanded into an inter-view association feature vector V 1 And global associated feature vector V 2 Then, hierarchical depth homography alignment fusion of the feature vectors is performed, which is expressed as:
Figure BDA0003752653930000091
wherein V 1 Representing the associated feature vector, V, between said views 2 Represents the global associated feature vector, V 3 Represents the classification feature vector, | · | non-woven phosphor 1 Represents a norm of the vector, and | · |. Non-woven counting F Representing the Frobenius norm of the matrix,
Figure BDA0003752653930000092
and
Figure BDA0003752653930000093
respectively represent subtraction and addition by position, and |, indicates dot-by-position.
Here, the hierarchical depth homography alignment fusion performs homography alignment of hierarchical scene depth streams based on vector differential expression according to the hierarchical depth characteristics based on feature fusion of vector characterization, and uses the depth perception of the full-field Jing Shanying incidence matrix between vectors as the bias of superposition, so that the hierarchical depth homography alignment of features is effectively performed on the basis of distribution dislocation caused by different hierarchical depth features between the feature distributions of the inter-view incidence feature map and the global incidence feature map, thereby improving the classification feature vector V after fusion 3 To improve the accuracy of the assembly quality detection of the assembled stainless steel cabinet.
Based on this, the present application proposes an intelligent manufacturing method of a stainless steel cabinet, which includes: acquiring a plurality of visual angle images of the assembled stainless steel cabinet from a plurality of visual angles through a camera; respectively passing each view image in the plurality of view images through a first convolutional neural network serving as a feature extractor to obtain a plurality of local view feature maps; arranging the plurality of local view angle characteristic graphs into a three-dimensional input tensor, and then obtaining an inter-view angle association characteristic graph by using a second convolution neural network of a three-dimensional convolution kernel; carrying out panoramic stitching on the plurality of local view angle characteristic graphs to obtain a panoramic characteristic graph; passing the panoramic feature map through a non-local neural network to obtain a global association feature map; respectively expanding the inter-view association feature map and the global association feature map into vectors to obtain an inter-view association feature vector and a global association feature vector; fusing the inter-view association feature vector and the global association feature vector to obtain a classification feature vector; and passing the classification feature vector through a classifier to obtain a classification result, wherein the classification result is used for indicating whether the assembly quality of the assembled stainless steel cabinet meets a preset requirement or not.
Fig. 1 illustrates an application scenario of an intelligent manufacturing method of a stainless steel cabinet according to an embodiment of the present application. As shown in fig. 1, in this application scenario, first, a plurality of perspective images of the assembled stainless steel cabinet (e.g., T as illustrated in fig. 1) are acquired from a plurality of perspectives by a camera (e.g., C as illustrated in fig. 1). Then, the obtained multiple perspective images of the stainless steel cabinet are input into a server (e.g., S as illustrated in fig. 1) deployed with an intelligent manufacturing algorithm of the stainless steel cabinet, wherein the server can process the multiple perspective images of the stainless steel cabinet with the intelligent manufacturing algorithm of the stainless steel cabinet to generate a classification result indicating whether the assembly quality of the assembled stainless steel cabinet meets a predetermined requirement.
Having described the basic principles of the present application, various non-limiting embodiments of the present application will now be described with reference to the accompanying drawings.
Exemplary method
Fig. 2 illustrates a flow chart of a method of intelligent manufacturing of stainless steel cabinets. As shown in fig. 2, the intelligent manufacturing method of a stainless steel cabinet according to the embodiment of the present application includes: s110, collecting a plurality of visual angle images of the assembled stainless steel cabinet from a plurality of visual angles through a camera; s120, enabling each view image in the multiple view images to pass through a first convolution neural network serving as a feature extractor to obtain multiple local view feature maps; s130, arranging the plurality of local view angle characteristic graphs into a three-dimensional input tensor, and then obtaining an inter-view angle association characteristic graph by using a second convolution neural network of a three-dimensional convolution kernel; s140, carrying out panoramic stitching on the plurality of local view angle feature maps to obtain a panoramic feature map; s150, passing the panoramic feature map through a non-local neural network to obtain a global association feature map; s160, respectively expanding the inter-view association feature map and the global association feature map into vectors to obtain an inter-view association feature vector and a global association feature vector; s170, fusing the inter-view association feature vector and the global association feature vector to obtain a classification feature vector; and S180, passing the classified feature vectors through a classifier to obtain a classification result, wherein the classification result is used for indicating whether the assembly quality of the assembled stainless steel cabinet meets a preset requirement or not.
Fig. 3 illustrates an architectural schematic of a method of smart manufacturing of stainless steel cabinets according to an embodiment of the application. As shown IN fig. 3, IN the network architecture of the intelligent manufacturing method of the stainless steel cabinet, first, each of the obtained plurality of perspective images (e.g., IN0 as illustrated IN fig. 3) is respectively passed through a first convolution neural network (e.g., CNN1 as illustrated IN fig. 3) as a feature extractor to obtain a plurality of local perspective feature maps (e.g., F1 as illustrated IN fig. 3); then, arranging the plurality of local perspective feature maps into a three-dimensional input tensor (e.g., T as illustrated in fig. 3) and then obtaining an inter-perspective correlation feature map (e.g., F2 as illustrated in fig. 3) by using a second convolution neural network (e.g., CNN2 as illustrated in fig. 3) of a three-dimensional convolution kernel; then, performing panorama stitching on the plurality of local view angle feature maps to obtain a panorama feature map (e.g., F3 as illustrated in fig. 3); then, passing the panoramic feature map through a non-local neural network (e.g., CNN3 as illustrated in fig. 3) to obtain a global associated feature map (e.g., F4 as illustrated in fig. 3); then, expanding the inter-view correlation feature map and the global correlation feature map into vectors to obtain an inter-view correlation feature vector (e.g., VF1 as illustrated in fig. 3) and a global correlation feature vector (e.g., VF2 as illustrated in fig. 3), respectively; then, fusing the inter-view associated feature vector and the global associated feature vector to obtain a classified feature vector (e.g., VF as illustrated in fig. 3); and, finally, passing the classified feature vector through a classifier (e.g., circle S as illustrated in fig. 3) to obtain a classification result, which is used to indicate whether the assembly quality of the assembled stainless steel cabinet meets a predetermined requirement.
Acquiring a plurality of view angle images of the assembled stainless steel cabinet from a plurality of view angles by a camera in steps S110 and S120; and respectively passing each of the plurality of perspective images through a first convolution neural network serving as a feature extractor to obtain a plurality of local perspective feature maps. As described above, in the technical solution of the present application, it is desirable to perform quality inspection of cabinet assembly only by means of images without performing inspection such as structural strength verification by means of various inspection apparatuses, so as to reduce the cost of quality inspection. Specifically, in the technical solution of the present application, first, a plurality of perspective images of the assembled stainless steel cabinet are collected by a camera from a plurality of perspectives. Then, the images of the multiple visual angles are subjected to feature extraction processing in a convolutional neural network model with excellent performance in image feature extraction so as to extract local implicit feature distribution information in the images of the multiple visual angles, and therefore multiple local visual angle feature maps are obtained.
Specifically, in this embodiment of the present application, the process of obtaining a plurality of local perspective feature maps by passing each of the plurality of perspective images through a first convolutional neural network as a feature extractor includes: each layer of the first convolutional neural network performs input data in forward transmission of the layer respectively: performing convolution processing on input data to obtain a convolution characteristic diagram; pooling the convolution characteristic map to obtain a pooled characteristic map; and performing nonlinear activation on the pooled feature map to obtain an activated feature map; wherein, the output of the last layer of the first convolutional neural network is the plurality of local view feature maps, and the input of the first layer of the first convolutional neural network is each view image in the plurality of view images.
In step S130, the plurality of local perspective feature maps are arranged into a three-dimensional input tensor, and then a second convolution neural network using a three-dimensional convolution kernel is used to obtain an inter-perspective correlation feature map. It should be understood that, since it is considered that, in the multiple perspective images of the assembled stainless steel cabinet, the perspective images have characteristic information associated with each other, in order to better extract the characteristic information about the assembly quality of the stainless steel cabinet in the multiple perspective images, in the technical solution of the present application, the multiple local perspective characteristic maps are further arranged as a three-dimensional input tensor, and then a second convolutional neural network using a three-dimensional convolution kernel is used to obtain an inter-perspective associated characteristic map. Here, the second convolutional neural network can deeply mine spatial feature information of the assembled stainless steel cabinet using a three-dimensional convolutional kernel.
Specifically, in this embodiment of the present application, after arranging the plurality of local perspective feature maps into a three-dimensional input tensor, a process of obtaining an inter-perspective associated feature map by using a second convolutional neural network of a three-dimensional convolutional kernel includes: the second convolutional neural network using the three-dimensional convolutional kernel performs, in forward transfer of layers, respectively: performing three-dimensional convolution processing on the input data based on the three-dimensional convolution kernel to obtain a convolution characteristic diagram; pooling the convolution characteristic map to obtain a pooled characteristic map; and performing nonlinear activation on the pooled feature map to obtain an activated feature map; wherein the output of the last layer of the second convolutional neural network is the inter-view correlation feature map, and the input of the first layer of the second convolutional neural network is the three-dimensional input tensor.
In step S140 and step S150, the plurality of local view angle feature maps are subjected to panorama stitching to obtain a panorama feature map, and the panorama feature map is passed through a non-local neural network to obtain a global association feature map. It should be appreciated that, considering that convolution is a typical local operation, for each pixel point in the multiple perspective images of the assembled stainless steel cabinet, the pixel points are not isolated, and the correlation between the pixels generates a foreground object. Therefore, in the technical solution of the present application, in order to extract the relevance between a certain pixel point and all the remaining pixel points of the multiple perspective images of the assembled stainless steel cabinet, a non-local neural network is used to further extract the features of the images. Namely, the plurality of local view angle characteristic maps are subjected to panoramic stitching to obtain a panoramic characteristic map. Accordingly, in one particular example, the plurality of local perspective feature maps may be stitched along a height dimension or a width dimension to obtain the panoramic feature map. And then, passing the panoramic feature map through a non-local neural network to obtain a global association feature map. Particularly, the non-local neural network captures remote dependence information by calculating the similarity of all pixel points of the image, further models context features, enables the network to pay attention to the whole content of the image, and further improves the extraction capability of the main network features in classification and detection tasks.
Specifically, in this embodiment of the present application, the process of obtaining the global associated feature map by passing the panoramic feature map through a non-local neural network includes: firstly, inputting the panoramic characteristic diagram into a first point convolution layer, a second point convolution layer and a third point convolution layer of the non-local neural network respectively to obtain a first characteristic diagram, a second characteristic diagram and a third characteristic diagram; then, calculating the weighted sum of the first feature map and the second feature map according to the position to obtain an intermediate fusion feature map; then, inputting the intermediate fusion feature map into a Softmax function to normalize the feature values of all positions in the intermediate fusion feature map to obtain a normalized intermediate fusion feature map; then, calculating the weighted sum of the normalized intermediate fusion feature map and the third feature map according to the position to obtain a re-fusion feature map; then, calculating the similarity among the characteristic values of all positions in the re-fused characteristic diagram by embedding a Gaussian similarity function into the re-fused characteristic diagram to obtain a global perception characteristic diagram; then, the global perception feature graph passes through a fourth convolution layer of the non-local neural network to obtain a channel adjustment global perception feature graph; and finally, calculating the weighted sum of the channel adjustment global perception feature map and the high-dimensional correlation local feature map according to the position to obtain the global correlation feature map.
In step S160 and step S170, the characteristic map is associated between the view anglesAnd respectively expanding the global association feature map into vectors to obtain inter-view association feature vectors and global association feature vectors, and fusing the inter-view association feature vectors and the global association feature vectors to obtain classification feature vectors. It should be understood that, further, when fusing the inter-view correlation feature map and the global correlation feature map, considering that the inter-view correlation feature map itself performs feature extraction through a first convolutional neural network and a second convolutional neural network which are cascaded, and the global correlation feature map further increases a layer depth through a convolution operation of a non-local neural network on the basis of the inter-view correlation feature map, in the technical solution of the present application, when fusing the inter-view correlation feature map and the global correlation feature map, hierarchical depth homography alignment fusion is performed. Specifically, the inter-view association feature map and the global association feature map are firstly expanded into an inter-view association feature vector V 1 And a global associated feature vector V 2 And then carrying out hierarchical depth homography alignment fusion on the feature vectors.
It should be understood that the hierarchical depth homography alignment fusion performs homography alignment of hierarchical scene depth streams based on vector differential expression according to the hierarchical depth characteristics based on the feature fusion of vector characterization, and uses the depth perception of the full-field Jing Shanying incidence matrix among vectors as the bias of superposition, so that the hierarchical depth homography alignment of features is effectively performed on the basis of distribution dislocation caused by different hierarchical depth features between the feature distributions of the incidence feature map among visual angles and the global incidence feature map, and thus the classification feature vector V after fusion is improved 3 To improve the accuracy of the assembly quality detection of the assembled stainless steel cabinet.
Specifically, in this embodiment of the present application, the process of expanding the inter-view association feature map and the global association feature map into vectors to obtain an inter-view association feature vector and a global association feature vector includes: dividing each feature matrix in the inter-view associated feature map and each feature matrix in the global associated feature map by taking a row vector as a dividing unit to obtain a plurality of first row vectors corresponding to the inter-view associated feature map and a plurality of second row vectors corresponding to the global associated feature map; splicing the plurality of first line vectors to obtain the associated characteristic vectors among the visual angles; and splicing the second line vectors to obtain the global correlation characteristic vector.
Specifically, in this embodiment of the present application, the process of fusing the inter-view associated feature vector and the global associated feature vector to obtain a classified feature vector includes: firstly, calculating the position-based difference between the inter-view associated feature vector and the global associated feature vector to obtain a difference feature vector. Then, the sum of the inter-view associated feature vector and the global associated feature vector according to the position is calculated to obtain a sum feature vector. Then, a full field Jing Shanying correlation matrix between the inter-view correlation feature vector and the global correlation feature vector is calculated. Accordingly, in one particular example, the product between the transposed vector of inter-view associated feature vectors and the global associated feature vector is calculated to yield the full-field Jing Shanying correlation matrix. And then, carrying out logarithm operation on the added characteristic vector to obtain a logarithm added characteristic vector, wherein the logarithm operation on the added characteristic vector represents that a logarithm function value of a characteristic value of each position in the added characteristic vector is calculated. Then, a norm of the differential feature vector is calculated as a hierarchical depth characteristic value of the differential feature vector. Then, the Frobenius norm of the full scene homography correlation matrix is calculated as the depth perception value of the full field Jing Shanying correlation matrix. And then, taking the hierarchical depth characteristic value as a weighting coefficient to weight the logarithm addition characteristic vector according to positions so as to obtain a weighted characteristic vector. And finally, taking the depth perception value as a bias, and calculating the sum of the depth perception value and the feature value of each position in the weighted feature vector to obtain the classified feature vector. That is, in one specific example, the formula that fuses the inter-view associated feature vector and the global associated feature vector is expressed as:
Figure BDA0003752653930000141
wherein V 1 Representing the inter-view associative feature vector, V 2 Represents the global associated feature vector, V 3 Representing the classification feature vector, | · caly |, calving the phosphor 1 Represents a norm of the vector, and | · |. Non-woven counting F Represents the Frobenius norm of the matrix,
Figure BDA0003752653930000142
and
Figure BDA0003752653930000143
respectively represent subtraction and addition by position, and |, indicates dot-by-position.
Fig. 4 illustrates a flowchart of fusing the inter-view correlation feature vector and the global correlation feature vector to obtain a classification feature vector in an intelligent manufacturing method of a stainless steel cabinet according to an embodiment of the present application. As shown in fig. 4, in the embodiment of the present application, fusing the inter-view associated feature vector and the global associated feature vector to obtain a classified feature vector includes: s210, calculating the position difference between the inter-view associated feature vector and the global associated feature vector to obtain a difference feature vector; s220, calculating the sum of the inter-view associated feature vectors and the global associated feature vectors according to positions to obtain a sum feature vector; s230, calculating a full-field Jing Shanying incidence matrix between the inter-view incidence feature vector and the global incidence feature vector; s240, carrying out logarithm operation on the added characteristic vector to obtain a logarithm added characteristic vector, wherein the logarithm operation on the added characteristic vector represents that a logarithm function value of a characteristic value of each position in the added characteristic vector is calculated; s250, calculating a norm of the differential feature vector as a hierarchical depth characteristic value of the differential feature vector; s260, calculating the Frobenius norm of the full-scene homographic incidence matrix as the depth perception value of the full-field Jing Shanying incidence matrix; s270, taking the hierarchical depth characteristic value as a weighting coefficient to weight the logarithm addition characteristic vector according to positions so as to obtain a weighted characteristic vector; and S280, taking the depth perception value as a bias, and calculating the sum value of the depth perception value and the feature value of each position in the weighted feature vector to obtain the classified feature vector.
In step S180, the classified feature vector is passed through a classifier to obtain a classification result, which is used to indicate whether the assembly quality of the assembled stainless steel cabinet meets a predetermined requirement. That is, in one specific example, the classification feature vector is processed using the classifier to obtain the classification result with the following formula: softmax { (W) n ,B n ):…:(W 1 ,B 1 ) I X }, wherein W 1 To W n As a weight matrix, B 1 To B n Is a bias vector and X is the classification feature vector.
In summary, the intelligent manufacturing method of the stainless steel cabinet according to the embodiment of the present application is elucidated, which employs a convolutional neural network model based on a deep learning technique to perform implicit association feature extraction on a plurality of perspective images of the assembled stainless steel cabinet collected from a plurality of perspectives through local and global aspects, and performs homography alignment of a layered scene depth stream based on vector differential expression by using a layered depth homography alignment fusion method during feature fusion, so as to improve a classification effect of the fused features, and further, can accurately detect the assembly quality of the assembled stainless steel cabinet.
Exemplary System
Fig. 5 illustrates a block diagram of an intelligent manufacturing system for stainless steel cabinets in accordance with an embodiment of the present application. As shown in fig. 5, the intelligent manufacturing system 500 for stainless steel cabinets according to the embodiment of the present application includes: an image data acquisition unit 510 for acquiring a plurality of perspective images of the assembled stainless steel cabinet from a plurality of perspectives by means of a camera; a local feature extraction unit 520, configured to pass each of the multiple perspective images through a first convolutional neural network as a feature extractor to obtain multiple local perspective feature maps; a three-dimensional convolution unit 530, configured to arrange the plurality of local perspective feature maps into a three-dimensional input tensor, and then obtain an inter-perspective association feature map by using a second convolution neural network of a three-dimensional convolution kernel; a panorama stitching unit 540, configured to perform panorama stitching on the multiple local view feature maps to obtain a panorama feature map; a global feature extraction unit 550, configured to pass the panoramic feature map through a non-local neural network to obtain a global associated feature map; a dimension reduction unit 560, configured to expand the inter-view association feature map and the global association feature map into vectors respectively to obtain an inter-view association feature vector and a global association feature vector; a feature fusion unit 570, configured to fuse the inter-view association feature vector and the global association feature vector to obtain a classification feature vector; and a classification unit 580 for passing the classified feature vectors through a classifier to obtain a classification result, wherein the classification result is used for indicating whether the assembly quality of the assembled stainless steel cabinet meets a predetermined requirement.
In one example, in the above intelligent manufacturing system 500 for stainless steel cabinets, the local feature extraction unit 520 is further configured to: each layer of the first convolutional neural network respectively performs the following operations on input data in the forward transmission of the layer: performing convolution processing on input data to obtain a convolution characteristic diagram; pooling the convolution characteristic map to obtain a pooled characteristic map; and performing nonlinear activation on the pooled feature map to obtain an activated feature map; wherein, the output of the last layer of the first convolutional neural network is the plurality of local view feature maps, and the input of the first layer of the first convolutional neural network is each view image in the plurality of view images.
In one example, in the above intelligent manufacturing system 500 for stainless steel cabinets, the three-dimensional convolution unit 530 is further configured to: the second convolutional neural network using the three-dimensional convolutional kernel performs, in forward transfer of layers, respectively: performing three-dimensional convolution processing on the input data based on the three-dimensional convolution kernel to obtain a convolution characteristic diagram; pooling the convolution characteristic map to obtain a pooled characteristic map; and performing nonlinear activation on the pooled feature map to obtain an activated feature map; wherein the output of the last layer of the second convolutional neural network is the inter-view correlation feature map, and the input of the first layer of the second convolutional neural network is the three-dimensional input tensor.
In one example, in the above intelligent manufacturing system 500 for stainless steel cabinets, the panoramic stitching unit 540 is further configured to: and splicing the plurality of local view angle characteristic graphs along the height dimension or the width dimension to obtain the panoramic characteristic graph.
In one example, in the above intelligent manufacturing system 500 for stainless steel cabinets, the global feature extraction unit 550 is further configured to: inputting the panoramic characteristic map into a first point convolutional layer, a second point convolutional layer and a third point convolutional layer of the non-local neural network respectively to obtain a first characteristic map, a second characteristic map and a third characteristic map; calculating a position-weighted sum of the first feature map and the second feature map to obtain an intermediate fused feature map; inputting the intermediate fusion feature map into a Softmax function to normalize the feature values of the positions in the intermediate fusion feature map to obtain a normalized intermediate fusion feature map; calculating a position-weighted sum of the normalized intermediate fused feature map and the third feature map to obtain a re-fused feature map; embedding a Gaussian similarity function into the re-fused feature map to calculate the similarity between feature values of all positions in the re-fused feature map so as to obtain a global perception feature map; passing the global perception feature map through a fourth convolution layer of the non-local neural network to obtain a channel adjustment global perception feature map; and calculating a position-weighted sum of the channel adjustment global perception feature map and the high-dimensional association local feature map to obtain the global association feature map.
In one example, in the above intelligent manufacturing system 500 for stainless steel cabinets, the dimension reduction unit 560 is further configured to: dividing each feature matrix in the inter-view associated feature map and each feature matrix in the global associated feature map by taking a row vector as a dividing unit to obtain a plurality of first row vectors corresponding to the inter-view associated feature map and a plurality of second row vectors corresponding to the global associated feature map; splicing the plurality of first line vectors to obtain the associated characteristic vectors among the visual angles; and splicing the second line vectors to obtain the global correlation characteristic vector.
In one example, in the above intelligent manufacturing system 500 of stainless steel cabinets, as shown in fig. 6, the feature fusion unit 570 includes: a difference subunit 571, configured to calculate a difference between the inter-view correlation feature vector and the global correlation feature vector according to a position to obtain a difference feature vector; a summing subunit 572, configured to calculate a sum by location of the inter-view associated feature vector and the global associated feature vector to obtain a summed feature vector; an association matrix sub-unit 573 configured to calculate a full-field Jing Shanying association matrix between the inter-view associated eigenvector and the global associated eigenvector; a logarithm calculation subunit 574, configured to perform a logarithm operation on the summed feature vector to obtain a logarithm summed feature vector, where the logarithm operation on the summed feature vector indicates to calculate a logarithm function value of a feature value of each position in the summed feature vector; a eigenvalue operator unit 575 for calculating a norm of the differential eigenvector as a layered depth characteristic value of the differential eigenvector; a depth perception value operator unit 576, configured to calculate a Frobenius norm of the full-scene homography incidence matrix as a depth perception value of the full-field Jing Shanying incidence matrix; a weighting subunit 577, configured to weight the logarithmic addition feature vector by position using the hierarchical depth characteristic value as a weighting coefficient to obtain a weighted feature vector; and a fusion subunit 578, configured to calculate, with the depth perception value as an offset, a sum of the depth perception value and the feature value at each position in the weighted feature vector to obtain the classification feature vector.
In one example, in the above intelligent manufacturing system 500 for stainless steel cabinets, the correlation matrix subunit is further configured to: calculating a product between the transposed vector of the inter-view relevance eigenvector and the global relevance eigenvector to obtain the full-field Jing Shanying relevance matrix.
In one example, in the above intelligent manufacturing system 500 for stainless steel cabinets, the sorting unit 580 is further configured to: processing the classification feature vector using the classifier to obtain the classification result with the following formula: softmax { (W) n ,B n ):…:(W 1 ,B 1 ) I X }, wherein W 1 To W n As a weight matrix, B 1 To B n Is a bias vector and X is the classification feature vector.
Here, it can be understood by those skilled in the art that the detailed functions and operations of the respective units and modules in the intelligent manufacturing system 500 of the stainless steel cabinet described above have been described in detail in the description of the intelligent manufacturing method of the stainless steel cabinet with reference to fig. 1 to 4, and thus, a repetitive description thereof will be omitted.
As described above, the intelligent manufacturing system 500 of a stainless steel cabinet according to an embodiment of the present application may be implemented in various terminal devices, such as a server of an intelligent manufacturing algorithm of a stainless steel cabinet, and the like. In one example, the intelligent manufacturing system 500 of stainless steel cabinets according to embodiments of the present application may be integrated into a terminal device as one software module and/or hardware module. For example, the intelligent manufacturing system 500 of the stainless steel cabinet may be a software module in the operating system of the terminal device, or may be an application developed for the terminal device; of course, the intelligent manufacturing system 500 of the stainless steel cabinet can also be one of many hardware modules of the terminal device.
Alternatively, in another example, the intelligent manufacturing system 500 of the stainless steel cabinet and the terminal device may also be separate devices, and the intelligent manufacturing system 500 of the stainless steel cabinet may be connected to the terminal device through a wired and/or wireless network and transmit the interactive information according to an agreed data format.

Claims (10)

1. An intelligent manufacturing method of a stainless steel cabinet is characterized by comprising the following steps:
acquiring a plurality of perspective images of the assembled stainless steel cabinet from a plurality of perspectives by a camera;
respectively passing each view image in the plurality of view images through a first convolutional neural network serving as a feature extractor to obtain a plurality of local view feature maps;
arranging the plurality of local view angle characteristic graphs into a three-dimensional input tensor, and then obtaining an inter-view angle association characteristic graph by using a second convolution neural network of a three-dimensional convolution kernel;
carrying out panoramic stitching on the plurality of local view angle characteristic graphs to obtain a panoramic characteristic graph;
passing the panoramic feature map through a non-local neural network to obtain a global association feature map;
respectively expanding the inter-view association feature map and the global association feature map into vectors to obtain an inter-view association feature vector and a global association feature vector;
fusing the inter-view association feature vector and the global association feature vector to obtain a classification feature vector; and
and passing the classified feature vector through a classifier to obtain a classification result, wherein the classification result is used for indicating whether the assembling quality of the assembled stainless steel cabinet meets a preset requirement or not.
2. The intelligent manufacturing method of stainless steel cabinet as claimed in claim 1, wherein passing each of the plurality of perspective images through a first convolutional neural network as a feature extractor to obtain a plurality of local perspective feature maps, respectively, comprises: each layer of the first convolutional neural network performs input data in forward transmission of the layer respectively:
carrying out convolution processing on input data to obtain a convolution characteristic diagram;
pooling the convolution characteristic map to obtain a pooled characteristic map; and
performing nonlinear activation on the pooled feature map to obtain an activated feature map;
wherein, the output of the last layer of the first convolutional neural network is the plurality of local view feature maps, and the input of the first layer of the first convolutional neural network is each view image in the plurality of view images.
3. The intelligent manufacturing method of stainless steel cabinet as claimed in claim 2, wherein the arranging the plurality of local perspective eigenmaps as three-dimensional input tensors and obtaining the inter-perspective correlation eigenmap by using a second convolution neural network of three-dimensional convolution kernels comprises: the second convolutional neural network using the three-dimensional convolutional kernel performs, in forward transfer of layers, respectively:
performing three-dimensional convolution processing on the input data based on the three-dimensional convolution kernel to obtain a convolution characteristic diagram;
pooling the convolution characteristic map to obtain a pooled characteristic map; and
performing nonlinear activation on the pooled feature map to obtain an activated feature map;
wherein the output of the last layer of the second convolutional neural network is the inter-view correlation feature map, and the input of the first layer of the second convolutional neural network is the three-dimensional input tensor.
4. The intelligent manufacturing method of stainless steel cabinets of claim 3, wherein the panoramic image feature map obtained by panoramic stitching the plurality of local view angle feature maps comprises:
and splicing the plurality of local view angle characteristic graphs along the height dimension or the width dimension to obtain the panoramic characteristic graph.
5. The intelligent manufacturing method of stainless steel cabinets of claim 4, wherein passing the panoramic signature over a non-local neural network to obtain a global correlation signature comprises:
inputting the panoramic characteristic map into a first point convolutional layer, a second point convolutional layer and a third point convolutional layer of the non-local neural network respectively to obtain a first characteristic map, a second characteristic map and a third characteristic map;
calculating a position-weighted sum of the first feature map and the second feature map to obtain an intermediate fused feature map;
inputting the intermediate fusion feature map into a Softmax function to normalize the feature values of the positions in the intermediate fusion feature map to obtain a normalized intermediate fusion feature map;
calculating a position-weighted sum of the normalized intermediate fused feature map and the third feature map to obtain a re-fused feature map;
calculating the similarity among the feature values of all positions in the re-fused feature map by embedding a Gaussian similarity function into the re-fused feature map to obtain a global perception feature map;
passing the global perception feature map through a fourth convolution layer of the non-local neural network to obtain a channel adjustment global perception feature map; and
and calculating a position-weighted sum of the channel adjustment global perception feature map and the high-dimensional association local feature map to obtain the global association feature map.
6. The intelligent manufacturing method of stainless steel cabinets of claim 5, wherein expanding the inter-view correlation feature map and the global correlation feature map into vectors to obtain inter-view correlation feature vectors and global correlation feature vectors, respectively, comprises:
dividing each feature matrix in the inter-view correlation feature map and each feature matrix in the global correlation feature map by taking a row vector as a dividing unit to obtain a plurality of first row vectors corresponding to the inter-view correlation feature map and a plurality of second row vectors corresponding to the global correlation feature map;
splicing the plurality of first line vectors to obtain the associated characteristic vectors among the visual angles; and
and splicing the second line vectors to obtain the global associated feature vector.
7. The intelligent manufacturing method of stainless steel cabinets of claim 6, wherein fusing the inter-view associated feature vectors and the global associated feature vectors to derive classified feature vectors comprises:
calculating the position difference between the inter-view correlation feature vector and the global correlation feature vector to obtain a difference feature vector;
calculating the sum of the inter-view associated feature vectors and the global associated feature vector according to positions to obtain a sum feature vector;
calculating a full-field Jing Shanying correlation matrix between the inter-view correlation feature vector and the global correlation feature vector;
performing a logarithm operation on the added feature vector to obtain a logarithm added feature vector, wherein the logarithm operation on the added feature vector represents calculating a logarithm function value of a feature value of each position in the added feature vector;
calculating a norm of the differential feature vector as a hierarchical depth characteristic value of the differential feature vector;
calculating the Frobenius norm of the full-scene homographic incidence matrix as the depth perception value of the full-field Jing Shanying incidence matrix;
weighting the logarithmic summation feature vector according to the position by taking the hierarchical depth characteristic value as a weighting coefficient to obtain a weighted feature vector; and
and calculating the sum of the depth perception value and the feature value of each position in the weighted feature vector by taking the depth perception value as a bias to obtain the classified feature vector.
8. The intelligent manufacturing method of stainless steel cabinets of claim 7, wherein calculating a full field Jing Shanying correlation matrix between the inter-view correlation feature vectors and the global correlation feature vectors comprises:
calculating a product between the transposed vector of the inter-view relevance eigenvector and the global relevance eigenvector to obtain the full-field Jing Shanying relevance matrix.
9. The intelligent manufacturing method of stainless steel cabinets of claim 8, wherein passing the classification feature vectors through a classifier to obtain classification results comprises:
processing the classification feature vector using the classifier to obtain the classification result with the following formula: softmax { (W) n ,B n ):…:(W 1 ,B 1 ) | X }, wherein W 1 To W n As a weight matrix, B 1 To B n Is a bias vector and X is the classification feature vector.
10. An intelligent manufacturing system for stainless steel cabinets, comprising:
an image data acquisition unit for acquiring a plurality of perspective images of the assembled stainless steel cabinet from a plurality of perspectives by a camera;
the local feature extraction unit is used for enabling each view angle image in the plurality of view angle images to pass through a first convolutional neural network serving as a feature extractor respectively so as to obtain a plurality of local view angle feature maps;
the three-dimensional convolution unit is used for arranging the plurality of local view angle characteristic graphs into a three-dimensional input tensor and then obtaining an inter-view angle correlation characteristic graph through a second convolution neural network using a three-dimensional convolution kernel;
the panorama splicing unit is used for carrying out panorama splicing on the plurality of local visual angle characteristic graphs to obtain a panorama characteristic graph;
the global feature extraction unit is used for enabling the panoramic feature map to pass through a non-local neural network to obtain a global associated feature map;
the dimension reduction unit is used for respectively expanding the inter-view association feature map and the global association feature map into vectors so as to obtain an inter-view association feature vector and a global association feature vector;
the feature fusion unit is used for fusing the inter-view associated feature vector and the global associated feature vector to obtain a classified feature vector; and
and the classification unit is used for enabling the classification characteristic vectors to pass through a classifier to obtain a classification result, and the classification result is used for indicating whether the assembly quality of the assembled stainless steel cabinet meets a preset requirement or not.
CN202210849271.9A 2022-07-19 2022-07-19 Intelligent manufacturing method and system of stainless steel cabinet Pending CN115205543A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210849271.9A CN115205543A (en) 2022-07-19 2022-07-19 Intelligent manufacturing method and system of stainless steel cabinet

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210849271.9A CN115205543A (en) 2022-07-19 2022-07-19 Intelligent manufacturing method and system of stainless steel cabinet

Publications (1)

Publication Number Publication Date
CN115205543A true CN115205543A (en) 2022-10-18

Family

ID=83582596

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210849271.9A Pending CN115205543A (en) 2022-07-19 2022-07-19 Intelligent manufacturing method and system of stainless steel cabinet

Country Status (1)

Country Link
CN (1) CN115205543A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116343134A (en) * 2023-05-30 2023-06-27 山西双驱电子科技有限公司 System and method for transmitting driving test vehicle signals
CN116468699A (en) * 2023-04-23 2023-07-21 浙江酷趣网络科技有限公司杭州分公司 Intelligent production system and method for fabric changing color along with light intensity
CN116597163A (en) * 2023-05-18 2023-08-15 广东省旭晟半导体股份有限公司 Infrared optical lens and method for manufacturing the same
CN117094895A (en) * 2023-09-05 2023-11-21 杭州一隅千象科技有限公司 Image panorama stitching method and system

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116468699A (en) * 2023-04-23 2023-07-21 浙江酷趣网络科技有限公司杭州分公司 Intelligent production system and method for fabric changing color along with light intensity
CN116468699B (en) * 2023-04-23 2024-06-07 浙江酷趣网络科技有限公司杭州分公司 Intelligent production system and method for fabric changing color along with light intensity
CN116597163A (en) * 2023-05-18 2023-08-15 广东省旭晟半导体股份有限公司 Infrared optical lens and method for manufacturing the same
CN116343134A (en) * 2023-05-30 2023-06-27 山西双驱电子科技有限公司 System and method for transmitting driving test vehicle signals
CN117094895A (en) * 2023-09-05 2023-11-21 杭州一隅千象科技有限公司 Image panorama stitching method and system
CN117094895B (en) * 2023-09-05 2024-03-26 杭州一隅千象科技有限公司 Image panorama stitching method and system

Similar Documents

Publication Publication Date Title
CN115205543A (en) Intelligent manufacturing method and system of stainless steel cabinet
Zhang et al. Hierarchical feature fusion with mixed convolution attention for single image dehazing
CN110276411B (en) Image classification method, device, equipment, storage medium and medical electronic equipment
CN111199233B (en) Improved deep learning pornographic image identification method
CN113065558A (en) Lightweight small target detection method combined with attention mechanism
CN112446476A (en) Neural network model compression method, device, storage medium and chip
CN111754396B (en) Face image processing method, device, computer equipment and storage medium
CN110929622A (en) Video classification method, model training method, device, equipment and storage medium
CN112446270A (en) Training method of pedestrian re-identification network, and pedestrian re-identification method and device
CN108805151B (en) Image classification method based on depth similarity network
CN112215201A (en) Method and device for evaluating face recognition model and classification model aiming at image
CN110879982A (en) Crowd counting system and method
CN112052877B (en) Picture fine granularity classification method based on cascade enhancement network
CN114419349B (en) Image matching method and device
CN112084952B (en) Video point location tracking method based on self-supervision training
CN106355195A (en) The system and method used to measure image resolution value
CN110222718A (en) The method and device of image procossing
CN112949453B (en) Training method of smoke and fire detection model, smoke and fire detection method and equipment
CN114648496A (en) Intelligent medical system
CN113378620A (en) Cross-camera pedestrian re-identification method in surveillance video noise environment
CN111582202A (en) Intelligent course system
CN113313133A (en) Training method for generating countermeasure network and animation image generation method
CN116485743A (en) No-reference image quality evaluation method, system, electronic equipment and storage medium
CN116189160A (en) Infrared dim target detection method based on local contrast mechanism
CN114842506A (en) Human body posture estimation method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination