CN112784919A - Intelligent manufacturing multi-mode data oriented classification method - Google Patents

Intelligent manufacturing multi-mode data oriented classification method Download PDF

Info

Publication number
CN112784919A
CN112784919A CN202110146422.XA CN202110146422A CN112784919A CN 112784919 A CN112784919 A CN 112784919A CN 202110146422 A CN202110146422 A CN 202110146422A CN 112784919 A CN112784919 A CN 112784919A
Authority
CN
China
Prior art keywords
data
matrix
representing
modal
preprocessed
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110146422.XA
Other languages
Chinese (zh)
Other versions
CN112784919B (en
Inventor
黎志豪
余志文
杨楷翔
孟献兵
陈俊龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South China University of Technology SCUT
Original Assignee
South China University of Technology SCUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South China University of Technology SCUT filed Critical South China University of Technology SCUT
Priority to CN202110146422.XA priority Critical patent/CN112784919B/en
Publication of CN112784919A publication Critical patent/CN112784919A/en
Application granted granted Critical
Publication of CN112784919B publication Critical patent/CN112784919B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10—File systems; File servers
    • G06F16/18—File system types
    • G06F16/182—Distributed file systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21—Design, administration or maintenance of databases
    • G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28—Databases characterised by their database models, e.g. relational or object models
    • G06F16/284—Relational databases
    • G06F16/285—Clustering or classification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F16/35—Clustering; Classification
    • G06F16/353—Clustering; Classification into predefined classes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/10—Pre-processing; Data cleansing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/20—Natural language analysis
    • G06F40/279—Recognition of textual entities
    • G06F40/289—Phrasal analysis, e.g. finite state techniques or chunking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Biomedical Technology (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a classification method for intelligent manufacturing multi-modal data, which comprises the following steps: 1) collecting and cleaning production data logs to obtain multi-mode data; 2) dividing the multi-modal data according to the data composition form of the multi-modal data, and performing corresponding preprocessing; 3) and performing feature extraction and feature fusion on the preprocessed multi-modal data, and classifying the fused features. According to the method, multi-modal data are preprocessed, the surface features of the data are extracted and the deep features of the data are mined in combination with a self-encoder and an embedding processing mode, the data are classified in real time and the result is displayed by utilizing a fully-connected feedforward deep neural network based on an online learning mode, the accuracy of multi-modal data classification is effectively improved, and the AUC index is improved.

Description

Intelligent manufacturing multi-mode data oriented classification method
Technical Field
The invention relates to the field of computer artificial intelligence, in particular to a classification method for intelligent manufacturing multi-modal data.
Background
With the arrival of the 4.0 era of industry and the rapid development of artificial intelligence, the production and manufacturing of many traditional industrial industries and emerging pharmaceutical and pharmaceutical industries are also going to be intelligent. In the era of intelligent manufacturing big data, a large amount of manufacturing data which are complex in structure and difficult to analyze can be generated in the industrial production and pharmaceutical production processes. How to mine the hidden value behind the massive multi-modal production data and effectively classify the hidden value is a key development direction in the field of intelligent manufacturing research at the present stage. Aiming at the characteristics of poor compatibility, low expansibility, high modal imbalance, high dimension attribute and the like of the current intelligent manufacturing multi-modal data, how to ensure the consistency, accuracy, integrity and reliability of the data and improve the real-time property, compatibility and expansibility of multi-modal data processing is a key point for efficiently classifying the intelligent manufacturing multi-modal data.
Disclosure of Invention
The invention aims to provide a method for classifying multi-modal data for intelligent manufacturing, which can effectively overcome the defect of complicated feature processing of the multi-modal data, and can achieve the purposes of automatically extracting features to improve the accuracy of data classification and improve AUC (automatic characteristic collection) indexes.
In order to achieve the purpose, the technical scheme provided by the invention is as follows: a classification method for intelligent manufacturing multi-modal data comprises the following steps:
1) collecting and cleaning production data logs to obtain multi-mode data;
2) dividing the multi-modal data according to the data composition form of the multi-modal data, and performing corresponding preprocessing;
3) and performing feature extraction and feature fusion on the preprocessed multi-modal data, and classifying the fused features.
In the step 1), collecting and cleaning production data logs, and acquiring multi-mode data refers to collecting data logs of an intelligent manufacturing production platform, and screening and filtering abnormal data and noise data in the data logs; the abnormal data refers to that for all records of the production log, in a given time slice, values of the data under certain variable dimensions exceed a reasonable range, or the data do not conform to normal distribution under the 3 sigma principle, and the data are considered unreasonable and abnormal; the noise data refers to that the difference between certain sample data and normal sample data is larger than a threshold value due to abnormal factors such as network faults, data sample loss, timestamp deviation and data basic feature loss when the intelligent manufacturing production platform collects logs, and then the data can be regarded as noise data; the abnormal data and the noise data are screened and filtered in the data cleaning stage, the screened and filtered data samples are stored in a storage module based on a distributed file system (HDFS), and a corresponding Hive database table is created to obtain original multi-modal data.
In step 2), the multi-modal data is divided according to the data composition form, and corresponding preprocessing is performed, namely the server preprocesses the data in different forms by using different methods through a preprocessing layer to obtain the multi-modal data suitable for subsequent processing, and the method comprises the following steps:
2.1) dividing the multi-modal data into image data, text data and numerical data according to a data composition form;
2.2) acquiring a pixel value matrix of the image data obtained in the step 2.1) and carrying out standardization preprocessing:
Figure BDA0002930610270000021
in the formula, the set of pixel matrices of all the original image data is represented as
Figure BDA0002930610270000023
n1Representing the number of original image data, auA pixel matrix representing the u-th original image data, wherein u ranges from 1 to n1；μAA mean value of a pixel matrix representing all of the original image data; sigmaAA standard deviation of a pixel matrix representing all of the original image data;
Figure BDA0002930610270000022
the method comprises the steps of representing a pixel matrix obtained after the u-th original image data are subjected to standardization processing;
when each original image data is standardized, replacing the obtained standardized pixel matrix with the corresponding pixel matrix of the original image data to obtain a preprocessed image data set
Figure BDA0002930610270000031
Carrying out word vector pre-training treatment on the text data obtained in the step 2.1):
performing preliminary Word segmentation on the text data, performing Word vector training on the text data by using a Word2Vec method according to the Word segmentation Word bank result, converting the text data into numerical data, and collecting the preprocessed text data into a preprocessed text data set
Figure BDA0002930610270000032
n2Which represents the number of text data to be displayed,
Figure BDA0002930610270000033
denotes the n-th2Pre-processing the text data;
carrying out data regularization treatment on the numerical data obtained in the step 2.1):
Figure BDA0002930610270000034
Figure BDA0002930610270000035
in the formula, the set of all numerical data is represented as
Figure BDA0002930610270000036
n3Number of data representing numerical type, crRepresenting the r-th data, wherein r has a value ranging from 1 to n3(ii) a n represents the dimension of the data, RnRepresenting an n-dimensional real number space;
Figure BDA0002930610270000037
denotes crThe d-th dimension of (1), wherein d ranges from 1 to n; l isq(cr) Denotes crQ norm of (a), wherein the value of q is set by a user; c'rDenotes crThe result after regularization treatment;
after each numerical data is regularized, replacing the original numerical data with the obtained regularized data to obtain a preprocessed numerical data set
Figure BDA0002930610270000038
After all data are preprocessed, integrating the data together to obtain a final multi-modal data set X { A, B, C }1,x2,...,xmWhere m is n1+n2+n3Representing the number, x, of multimodal data setskRepresenting the kth data, k ranges from 1 to m.
In step 3), the method for extracting and fusing the features of the preprocessed multi-modal data and classifying the fused features comprises the following steps:
3.1) preprocessing the multi-modal data set X ═{x1,x2,...,xmInputting the data into a self-encoder comprising an encoder and a decoder, reconstructing the encoder and generating the decoder, and taking the output of the decoder as a characteristic F1Wherein the reconstruction loss function is:
Figure BDA0002930610270000041
wherein h represents an encoder; g represents a decoder; λ represents a hyper-parameter, the value of which is set by the user; m represents the number of data; x is the number ofkRepresenting the kth data, wherein the k value range is 1 to m;
Figure BDA0002930610270000042
representing data xkF norm of the jacobian matrix of (d); l isAERepresenting a loss function; g (h (x)k) ) represents data xkSequentially reconstructing by an encoder h and generating a result by a decoder g; | | g (h (x)k))-xk||1Denotes g (h (x)k) And x)k1 norm of the difference of (a);
at the loss function LAEWhen convergence is reached, the feature F is obtained1I.e. F1The matrix is a matrix with m rows and L columns, wherein m represents the number of data, and L represents the dimension of each data;
3.2) characterization of F obtained in step 3.1)1Duplicate copies were made to yield three signatures, F respectively1、F2、F3To F2And F3Feature embedding (embedding) processing, noted as F'2、F'3：
F'2＝W2·F2
F'3＝W3·F3
In the formula, W2、W3A parameter matrix representing m rows and L columns; w2·F2Represents W2And F2Performing dot product;W3·F3represents W3And F3Performing dot product; f'2、F'3Representing the resulting embedded features, both being a matrix of m rows and L columns;
3.3) embedding feature F'2And F'3After the softmax processing, the feature F is obtained1Performing weighted fusion, wherein the specific process comprises the following steps:
Figure BDA0002930610270000051
in the formula (I), the compound is shown in the specification,
Figure BDA00029306102700000516
is represented by F'3Is a matrix of L rows and m columns;
Figure BDA00029306102700000517
is represented by F'2And
Figure BDA00029306102700000518
obtaining m rows and m columns of matrix after matrix multiplication, and dividing each column of the matrix by a constant
Figure BDA0002930610270000055
Figure BDA0002930610270000056
Presentation pair
Figure BDA0002930610270000057
Performing softmax processing on each line; f represents
Figure BDA0002930610270000058
And F1Performing matrix multiplication to obtain a matrix of m rows and L columns, namely the final fusion characteristic;
adding the obtained fusion characteristic F into a full-connection feedforward deep neural network of the T layer for training, wherein the formula is expressed as follows:
Figure BDA0002930610270000059
in the formula, ht、ht+1Respectively representing output results of a T-th layer and a T + 1-th layer of fully-connected feedforward deep neural network, wherein the value range of T is from 1 to T-1; wt 1、Wt 2Representing a weight parameter of the t-th layer fully-connected feedforward deep neural network;
Figure BDA00029306102700000510
respectively represent and Wt 1、Wt 2Corresponding bias parameters; f (-) represents the Leaky-ReLU function, which is formulated as:
Figure BDA00029306102700000511
wherein a ranges from 0 to 1;
taking the output result h of the last layer of the fully-connected feedforward deep neural networkTWeight parameter
Figure BDA00029306102700000512
And bias parameter
Figure BDA00029306102700000513
The total number of categories of multimodal data is C; to hT、
Figure BDA00029306102700000514
Performing softmax processing to obtain a classification result of each datum:
Figure BDA00029306102700000515
Figure BDA0002930610270000061
Figure BDA0002930610270000062
in the formula (I), the compound is shown in the specification,
Figure BDA0002930610270000063
to represent
Figure BDA0002930610270000064
And
Figure BDA0002930610270000065
performing matrix multiplication; z denotes a matrix of m rows and C columns, ZiThe ith row of Z is represented as a C-dimensional vector, wherein the value range of i is 1 to m; exp (z)i) Represents a pair ziEach element of (1) is subjected to exponential operation with e as a base, and the result is still a C-dimensional vector;
Figure BDA0002930610270000066
denotes ziThe v-th dimension element of (1);
Figure BDA0002930610270000067
represents a pair ziEach element of (1) is subjected to exponential operation with e as a base and summed, and the result is a constant value;
Figure BDA0002930610270000068
denotes exp (z)i) Each element in (1) is divided by
Figure BDA0002930610270000069
The result is a C-dimensional vector; p is a radical ofiRepresents from
Figure BDA00029306102700000610
Selecting the maximum value as the classification probability value of the ith data;
3.4) adopting a cross entropy loss function and an ADAM algorithm to carry out iterative optimization on the fully-connected feedforward deep neural network in the step 3.3), pre-training the network under the condition of off-line learning, and adopting an on-line learning mechanism to carry out parameter real-time updating on the pre-trained network so as to classify the data in real time, wherein the process is as follows:
reading a preset amount of data samples from the HDFS, and obtaining preprocessed data through the step 2); carrying out iterative training on the preprocessed data for preset times through steps 3.1), 3.2) and 3.3), and solving the fully-connected feedforward deep neural network by using a cross entropy loss function J added with a regular term, wherein the formula is expressed as:
Figure BDA0002930610270000071
in the formula, pjIs the calculated probability value of the jth data, yjThe method comprises the following steps that (1) the method is a real class label, N is the total number of samples of network pre-training, and beta is a regularization parameter; w is ajThe weight parameter is the jth data; the formula uses ADAM algorithm to carry out iterative optimization;
when the pre-training process reaches a convergence state, the network is subjected to fine-tuning updating in an online learning mode, namely, a preset amount of data is read from a real-time sample according to a batch (batch), after corresponding preprocessing, the data is input into a fully-connected feedforward deep neural network for training, a server receives a parameter updating result of the network in real time to update the network, and classification and identification are carried out on the data sample of the batch (batch) of the latest network, so that a classification result of each data is obtained and is visually displayed.
Compared with the prior art, the invention has the following advantages:
1. the invention can acquire the data log in real time, and accurately clean the data log according to the actual production requirement to acquire the needed multi-mode data.
2. The invention can automatically divide and preprocess the multi-mode data in the actual production environment, and can adopt different preprocessing methods aiming at different source data.
3. The method carries out feature extraction and feature fusion on the multi-modal data through two modes of self-encoder and low-dimensional embedding, can automatically extract the surface features of the data and mine the deep features of the data without manually designing feature engineering, improves the accuracy of data classification and promotes AUC indexes.
4. The invention classifies the data in real time in an online learning mode, realizes the visual display of data classification and realizes the visual management of intelligent manufacturing data.
5. The method has the advantages of wide use space in an intelligent manufacturing system, simple operation, strong adaptability and wide prospect in intelligent manufacturing data analysis and decision.
Drawings
FIG. 1 is a logic flow diagram of the method of the present invention.
Detailed Description
The present invention will be described in further detail with reference to examples, but the embodiments of the present invention are not limited thereto.
As shown in fig. 1, the classification method for intelligent manufacturing multi-modal data provided by this embodiment includes the following steps:
1) and collecting and cleaning a data log of the flexible production pharmaceutical platform to obtain multi-mode pharmaceutical data. The data log collection and cleaning of the flexible production pharmaceutical platform means that the pharmaceutical production data log is collected, and abnormal data and noise data in the production log are screened and filtered according to all records of the production log. The abnormal data refers to that the value of the pharmaceutical data in certain variable dimensions exceeds a reasonable range or does not comply with normal distribution under the 3 sigma principle in a set time granularity, and the data are considered unreasonable and abnormal; the noisy data refers to abnormal factors which cause that some pharmaceutical data are different from normal pharmaceutical data by more than a threshold value due to network faults, data sample missing, timestamp deviation and data base feature missing which may happen when the flexible production pharmaceutical platform collects logs, and the pharmaceutical data are considered as the noisy data. The above-mentioned abnormal data and noise data are removed in the data cleansing stage.
And storing the cleaned pharmaceutical data into a storage module based on a distributed file system (HDFS), and creating a corresponding Hive database table.
2) Dividing pharmaceutical data according to data composition form, and carrying out corresponding pretreatment, wherein the method comprises the following steps:
and 2.1) reading a certain amount of processed production logs from the Hive database table, and dividing the production logs into image pharmaceutical data, text pharmaceutical data and numerical pharmaceutical data according to the composition form of the pharmaceutical data.
2.2) carrying out standardization preprocessing on the pixel matrix of the image pharmaceutical data, wherein the standardization formula is as follows:
Figure BDA0002930610270000091
in the formula, the pixel matrix set of all the original image pharmaceutical data is expressed as
Figure BDA0002930610270000098
n1Number of pharmaceutical data representing original image class, auA pixel matrix for representing the pharmaceutical data of the u-th original image class, wherein u has a value ranging from 1 to n1；μAMean of pixel matrix representing all original image class pharmaceutical data; sigmaAA standard deviation of a pixel matrix representing all of the original image-like pharmaceutical data;
Figure BDA0002930610270000092
and the pixel matrix is obtained by standardizing the u-th original image pharmaceutical data.
When each original image pharmaceutical data is standardized, replacing the pixel matrix of the corresponding original image pharmaceutical data with the obtained standardized pixel matrix to obtain a preprocessed image pharmaceutical data set
Figure BDA0002930610270000093
Aiming at the text pharmaceutical data, the preprocessing process comprises the following steps: performing preliminary Word segmentation processing on the text pharmaceutical data by using an NLTK toolkit, and using a Word2Vec tool in a Gensim library according to the result of a Word segmentation lexiconPerforming word vector training on the text pharmaceutical data, converting the text pharmaceutical data into numerical data, and collecting the preprocessed text pharmaceutical data into a set
Figure BDA0002930610270000094
Denotes the n-th2Pre-processed textual pharmaceutical data.
Aiming at numerical pharmaceutical data, regularization pretreatment is carried out on the numerical pharmaceutical data, and the specific formula is as follows:
Figure BDA0002930610270000095
Figure BDA0002930610270000096
wherein the set of all numerical pharmaceutical data is represented as
Figure BDA0002930610270000099
n3Number of numerical pharmaceutical data, crRepresenting the r-th numerical pharmaceutical data, wherein r ranges from 1 to n3(ii) a n represents the dimension of the numerical pharmaceutical data, RnRepresenting an n-dimensional real number space;
Figure BDA0002930610270000097
denotes crThe d-th dimension of (1), wherein d ranges from 1 to n; l isq(cr) Denotes crQ norm of (a), wherein the value of q is set by the user himself; c'rDenotes crAnd (5) carrying out regularization processing on the obtained product.
After each numerical pharmaceutical data is regularized, replacing each original numerical pharmaceutical data with the obtained regularized pharmaceutical data to obtain a preprocessed numerical pharmaceutical data set
Figure BDA0002930610270000103
When all ofAfter the pharmaceutical data are preprocessed, the pharmaceutical data are integrated together to obtain a final multi-modal pharmaceutical data set X ═ { A, B, C } ═ X1,x2,...,xmWhere m is n1+n2+n3Number, x, representing a multimodal pharmaceutical data setkRepresenting the kth data, k ranges from 1 to m.
3) The method comprises the following steps of performing feature extraction and feature fusion on the preprocessed pharmaceutical data, and classifying the fused features:
3.1) preprocessing the multi-modal pharmaceutical data set X ═ { X1,x2,...,xmInputting the data into a self-encoder comprising an encoder and a decoder, reconstructing the encoder and generating the decoder, and taking the output of the decoder as a characteristic F1Wherein the reconstruction loss function is:
Figure BDA0002930610270000101
wherein h represents an encoder; g represents a decoder; λ represents a hyper-parameter, the value of which is set by the user; m represents the number of pharmaceutical data; x is the number ofkRepresenting the kth pharmaceutical data, wherein the k value range is 1 to m;
Figure BDA0002930610270000102
representing pharmaceutical data xkF norm of the jacobian matrix of (d); l isAERepresenting a loss function; g (h (x)k) Express pharmaceutical data xkSequentially reconstructing by an encoder h and generating a result by a decoder g; | | g (h (x)k))-xk||1Denotes g (h (x)k) And x)k1 norm of the difference of (a).
At the loss function LAEWhen convergence is reached, the feature F is obtained1I.e. F1The matrix is a matrix with m rows and L columns, m represents the number of pharmaceutical data, and L represents the dimension of each data.
3.2) comparison of feature F obtained in step 3.1)1Duplicate copies were made to yield three signatures, F respectively1，F2，F3To F2And F3Feature embedding (embedding) processing, noted as F'2，F'3：
F'2＝W2 T·F2
F'3＝W3 T·F3
In the formula, W2、W3A parameter matrix representing m rows and L columns; w2·F2Represents W2And F2Performing dot product; w3·F3Represents W3And F3Performing dot product; f'2、F'3The resulting embedded features are represented, both being a matrix of m rows and L columns.
3.3) embedding feature F'2And F'3After the softmax processing, the feature F is obtained1And performing feature fusion, wherein the specific formula is as follows:
Figure BDA0002930610270000111
in the formula (I), the compound is shown in the specification,
Figure BDA00029306102700001111
is represented by F'3Is a matrix of L rows and m columns;
Figure BDA00029306102700001112
is represented by F'2And
Figure BDA00029306102700001113
obtaining m rows and m columns of matrix after matrix multiplication, and dividing each column of the matrix by a constant
Figure BDA0002930610270000115
Figure BDA0002930610270000116
Presentation pair
Figure BDA0002930610270000117
Performing softmax processing on each line; f represents
Figure BDA0002930610270000118
And F1And performing matrix multiplication to obtain a matrix with m rows and L columns, namely the final fusion characteristic.
Adding the obtained fusion characteristic F into a full-connection feedforward deep neural network of the T layer for training, wherein the formula is expressed as follows:
Figure BDA0002930610270000119
in the formula, ht、ht+1Respectively representing output results of a T-th layer and a T + 1-th layer of fully-connected feedforward deep neural network, wherein the value range of T is from 1 to T-1; wt 1、Wt 2Representing a weight parameter of the t-th layer fully-connected feedforward deep neural network;
Figure BDA00029306102700001110
respectively represent and Wt 1、Wt 2Corresponding bias parameters; f (-) represents the Leaky-ReLU function, which is formulated as:
Figure BDA0002930610270000121
wherein a ranges from 0 to 1.
Taking the output result h of the last layer of the fully-connected feedforward deep neural networkTWeight parameter
Figure BDA0002930610270000122
Offset parameter
Figure BDA0002930610270000123
The total number of categories of multimodal pharmaceutical data is C. To hT、
Figure BDA0002930610270000124
Performing softmax processing to obtain a classification result of each pharmaceutical data:
Figure BDA0002930610270000125
Figure BDA0002930610270000126
Figure BDA0002930610270000127
in the formula (I), the compound is shown in the specification,
Figure BDA0002930610270000128
to represent
Figure BDA0002930610270000129
And
Figure BDA00029306102700001210
performing matrix multiplication; z denotes a matrix of m rows and C columns, ZiThe ith row of Z is represented as a C-dimensional vector, wherein the value range of i is 1 to m; exp (z)i) Represents a pair ziEach element of (1) is subjected to exponential operation with e as a base, and the result is still a C-dimensional vector;
Figure BDA00029306102700001211
denotes ziThe v-th dimension element of (1);
Figure BDA00029306102700001212
represents a pair ziEach element of (1) is subjected to exponential operation with e as a base and summed, and the result is a constant value;
Figure BDA00029306102700001213
denotes exp (z)i) Each element in (1) is divided by
Figure BDA00029306102700001214
The result is a C-dimensional vector; p is a radical ofiRepresents from
Figure BDA00029306102700001215
The largest value is selected as the classification probability value of the ith pharmaceutical data.
3.4) adopting a cross entropy loss function and an ADAM algorithm to carry out iterative optimization on the fully-connected feedforward deep neural network in the step 3.3), pre-training the network under the condition of off-line learning, and adopting an on-line learning mechanism to carry out parameter real-time updating on the pre-trained network so as to classify the data in real time, wherein the process is as follows:
reading a certain amount of pharmaceutical data from the HDFS, and obtaining original characteristics through the step 2); carrying out iterative training on the obtained original features for a certain number of times through steps 3.1), 3.2) and 3.3), and solving the fully-connected feedforward neural network by using a cross entropy loss function J added with a regular term, wherein a specific formula is expressed as follows:
Figure BDA0002930610270000131
in the formula, pjIs the calculated probability value of the jth pharmaceutical data, yjThe classification label is a real class label, N is the total number of the pharmaceutical data pre-trained by the network, and beta is a regularization parameter; w is ajIs the weight parameter of the jth pharmaceutical data. The above formula is iteratively optimized using an ADAM algorithm.
And when the pre-training process reaches a convergence state, performing fine tuning updating on the network in an online learning mode. Reading a certain amount of data from the pharmaceutical data produced in real time according to batches (batch), inputting the data into a fully-connected feedforward deep neural network for training after corresponding preprocessing, receiving the parameter updating result of the network by a server in real time to update the network, classifying and identifying the pharmaceutical data of the batch (batch) by adopting the latest network, obtaining the classification result of each pharmaceutical data and carrying out visual display.
The above embodiments are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments, and any other changes, modifications, substitutions, combinations, and simplifications which do not depart from the spirit and principle of the present invention should be construed as equivalents thereof, and all such changes, modifications, substitutions, combinations, and simplifications are intended to be included in the scope of the present invention.

Claims (4)

1. The classification method for the intelligent manufacturing multi-modal data is characterized by comprising the following steps of:
1) collecting and cleaning production data logs to obtain multi-mode data;
2) dividing the multi-modal data according to the data composition form of the multi-modal data, and performing corresponding preprocessing;
3) and performing feature extraction and feature fusion on the preprocessed multi-modal data, and classifying the fused features.
2. The method for classifying multimodal data on intelligent manufacturing according to claim 1, wherein: in the step 1), collecting and cleaning production data logs, and acquiring multi-mode data refers to collecting data logs of an intelligent manufacturing production platform, and screening and filtering abnormal data and noise data in the data logs; the abnormal data refers to that for all records of the production log, in a given time slice, values of the data under certain variable dimensions exceed a reasonable range, or the data do not conform to normal distribution under the 3 sigma principle, and the data are considered unreasonable and abnormal; the noise data refers to that the difference between certain sample data and normal sample data is larger than a threshold value due to abnormal factors such as network faults, data sample loss, timestamp deviation and data basic feature loss when the intelligent manufacturing production platform collects logs, and then the data can be regarded as noise data; the abnormal data and the noise data are screened and filtered in the data cleaning stage, the screened and filtered data samples are stored in a storage module based on a distributed file system (HDFS), and a corresponding Hive database table is created to obtain original multi-modal data.
3. The method for classifying multimodal data on intelligent manufacturing according to claim 1, wherein: in step 2), the multi-modal data is divided according to the data composition form, and corresponding preprocessing is performed, namely the server preprocesses the data in different forms by using different methods through a preprocessing layer to obtain the multi-modal data suitable for subsequent processing, and the method comprises the following steps:
2.1) dividing the multi-modal data into image data, text data and numerical data according to a data composition form;
2.2) acquiring a pixel value matrix of the image data obtained in the step 2.1) and carrying out standardization preprocessing:
Figure FDA0002930610260000021
in the formula, the set of pixel matrices of all the original image data is represented as
Figure FDA0002930610260000022
n1Representing the number of original image data, auA pixel matrix representing the u-th original image data, wherein u ranges from 1 to n1；μAA mean value of a pixel matrix representing all of the original image data; sigmaAA standard deviation of a pixel matrix representing all of the original image data;
Figure FDA0002930610260000023
the method comprises the steps of representing a pixel matrix obtained after the u-th original image data are subjected to standardization processing;
when each original image data is standardized, replacing the obtained standardized pixel matrix with the corresponding pixel matrix of the original image data to obtain a preprocessed image data set
Figure FDA0002930610260000024
Carrying out word vector pre-training treatment on the text data obtained in the step 2.1):
performing preliminary Word segmentation on the text data, performing Word vector training on the text data by using a Word2Vec method according to the Word segmentation Word bank result, converting the text data into numerical data, and collecting the preprocessed text data into a preprocessed text data set
Figure FDA0002930610260000025
n2Which represents the number of text data to be displayed,
Figure FDA0002930610260000026
denotes the n-th2Pre-processing the text data;
carrying out data regularization treatment on the numerical data obtained in the step 2.1):
Figure FDA0002930610260000027
Figure FDA0002930610260000028
in the formula, the set of all numerical data is represented as
Figure FDA0002930610260000029
n3Number of data representing numerical type, crRepresenting the r-th data, wherein r has a value ranging from 1 to n3(ii) a n represents the dimension of the data, RnRepresenting an n-dimensional real number space;
Figure FDA0002930610260000031
denotes crThe d-th dimension of (1), wherein d ranges from 1 to n; l isq(cr) Denotes crQ norm of (a), wherein the value of q is set by a user; c'rDenotes crThe result after regularization treatment;
after each numerical data is regularized, replacing the original numerical data with the obtained regularized data to obtain a preprocessed numerical data set
Figure FDA0002930610260000032
After all data are preprocessed, integrating the data together to obtain a final multi-modal data set X { A, B, C }1,x2,...,xmWhere m is n1+n2+n3Representing the number, x, of multimodal data setskRepresenting the kth data, k ranges from 1 to m.
4. The method for classifying multimodal data on intelligent manufacturing according to claim 1, wherein: in step 3), the method for extracting and fusing the features of the preprocessed multi-modal data and classifying the fused features comprises the following steps:
3.1) on the preprocessed multimodal data set X ═ { X1,x2,...,xmInputting the data into a self-encoder comprising an encoder and a decoder, reconstructing the encoder and generating the decoder, and taking the output of the decoder as a characteristic F1Wherein the reconstruction loss function is:
Figure FDA0002930610260000033
wherein h represents an encoder; g represents a decoder; λ represents a hyper-parameter, the value of which is set by the user; m represents the number of data; x is the number ofkRepresenting the kth data, wherein the k value range is 1 to m;
Figure FDA0002930610260000034
representing data xkF norm of the jacobian matrix of (d); l isAERepresents the lossA function; g (h (x)k) ) represents data xkSequentially reconstructing by an encoder h and generating a result by a decoder g; | | g (h (x)k))-xk||1Denotes g (h (x)k) And x)k1 norm of the difference of (a);
at the loss function LAEWhen convergence is reached, the feature F is obtained1I.e. F1The matrix is a matrix with m rows and L columns, wherein m represents the number of data, and L represents the dimension of each data;
3.2) characterization of F obtained in step 3.1)1Duplicate copies were made to yield three signatures, F respectively1、F2、F3To F2And F3Is subjected to characteristic embedding treatment and is recorded as F'2、F′3：
F′2＝W2·F2
F′3＝W3·F3
In the formula, W2、W3A parameter matrix representing m rows and L columns; w2·F2Represents W2And F2Performing dot product; w3·F3Represents W3And F3Performing dot product; f'2、F′3Representing the resulting embedded features, both being a matrix of m rows and L columns;
3.3) embedding feature F'2And F'3After the softmax processing, the feature F is obtained1Performing weighted fusion, wherein the specific process comprises the following steps:
Figure FDA0002930610260000041
in the formula, F3'TIs represented by F'3Is a matrix of L rows and m columns;
Figure FDA0002930610260000042
is represented by F'2And F3'TObtaining m rows and m columns of matrix after matrix multiplication, and dividing each column of the matrix by a constant
Figure FDA0002930610260000043
Figure FDA0002930610260000044
Presentation pair
Figure FDA0002930610260000045
Performing softmax processing on each line; f represents
Figure FDA0002930610260000046
And F1Performing matrix multiplication to obtain a matrix of m rows and L columns, namely the final fusion characteristic;
adding the obtained fusion characteristic F into a full-connection feedforward deep neural network of the T layer for training, wherein the formula is expressed as follows:
Figure FDA0002930610260000048
in the formula, ht、ht+1Respectively representing output results of a T-th layer and a T + 1-th layer of fully-connected feedforward deep neural network, wherein the value range of T is from 1 to T-1; wt 1、Wt 2Representing a weight parameter of the t-th layer fully-connected feedforward deep neural network;
Figure FDA0002930610260000047
respectively represent and Wt 1、Wt 2Corresponding bias parameters; f (-) represents the Leaky-ReLU function, which is formulated as:
Figure FDA0002930610260000051
wherein a ranges from 0 to 1;
get allOutput result h of last layer of connected feedforward deep neural networkTWeight parameter
Figure FDA0002930610260000052
And bias parameter
Figure FDA0002930610260000053
The total number of categories of multimodal data is C; to hT、
Figure FDA0002930610260000054
Performing softmax processing to obtain a classification result of each datum:
Figure FDA0002930610260000055
Figure FDA0002930610260000056
Figure FDA0002930610260000057
in the formula (I), the compound is shown in the specification,
Figure FDA0002930610260000058
to represent
Figure FDA0002930610260000059
And
Figure FDA00029306102600000510
performing matrix multiplication; z denotes a matrix of m rows and C columns, ZiThe ith row of Z is represented as a C-dimensional vector, wherein the value range of i is 1 to m; exp (z)i) Represents a pair ziEach element of (1) is subjected to exponential operation with e as a base, and the result is still a C-dimensional vector;
Figure FDA00029306102600000511
denotes ziThe v-th dimension element of (1);
Figure FDA00029306102600000512
represents a pair ziEach element of (1) is subjected to exponential operation with e as a base and summed, and the result is a constant value;
Figure FDA00029306102600000513
denotes exp (z)i) Each element in (1) is divided by
Figure FDA00029306102600000514
The result is a C-dimensional vector; p is a radical ofiRepresents from
Figure FDA00029306102600000515
Selecting the maximum value as the classification probability value of the ith data;
3.4) adopting a cross entropy loss function and an ADAM algorithm to carry out iterative optimization on the fully-connected feedforward deep neural network in the step 3.3), pre-training the network under the condition of off-line learning, and adopting an on-line learning mechanism to carry out parameter real-time updating on the pre-trained network so as to classify the data in real time, wherein the process is as follows:
reading a preset amount of data samples from the HDFS, and obtaining preprocessed data through the step 2); carrying out iterative training on the preprocessed data for preset times through steps 3.1), 3.2) and 3.3), and solving the fully-connected feedforward deep neural network by using a cross entropy loss function J added with a regular term, wherein the formula is expressed as:
Figure FDA0002930610260000061
in the formula, pjIs the calculated probability value of the jth data, yjThe method comprises the following steps that (1) the method is a real class label, N is the total number of samples of network pre-training, and beta is a regularization parameter;wjthe weight parameter is the jth data; the formula uses ADAM algorithm to carry out iterative optimization;
when the pre-training process reaches a convergence state, the network is subjected to fine tuning updating in an online learning mode, namely, a preset amount of data is read from a real-time sample according to batches, after corresponding pre-processing is carried out, the data is input into a fully-connected feedforward deep neural network for training, a server receives a parameter updating result of the network in real time to update the network, a latest data sample of the network batch is adopted for classification and identification, and a classification result of each data is obtained and is displayed visually.
CN202110146422.XA 2021-02-03 2021-02-03 Classification method for intelligent manufacturing multi-mode data Active CN112784919B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110146422.XA CN112784919B (en) 2021-02-03 2021-02-03 Classification method for intelligent manufacturing multi-mode data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110146422.XA CN112784919B (en) 2021-02-03 2021-02-03 Classification method for intelligent manufacturing multi-mode data

Publications (2)

Publication Number Publication Date
CN112784919A true CN112784919A (en) 2021-05-11
CN112784919B CN112784919B (en) 2023-09-05

Family

ID=75760650

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110146422.XA Active CN112784919B (en) 2021-02-03 2021-02-03 Classification method for intelligent manufacturing multi-mode data

Country Status (1)

Country Link
CN (1) CN112784919B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113742428A (en) * 2021-09-20 2021-12-03 郑州颂和礼致信息科技有限公司 Neural network data set storage method based on block chain
CN114372181A (en) * 2021-12-27 2022-04-19 华南理工大学 Intelligent planning method for equipment production based on multi-mode data

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108629630A (en) * 2018-05-08 2018-10-09 广州太平洋电脑信息咨询有限公司 A kind of feature based intersects the advertisement recommendation method of joint deep neural network
WO2019238976A1 (en) * 2018-06-15 2019-12-19 Université de Liège Image classification using neural networks
CN111444960A (en) * 2020-03-26 2020-07-24 上海交通大学 Skin disease image classification system based on multi-mode data input
CN111612066A (en) * 2020-05-21 2020-09-01 成都理工大学 Remote sensing image classification method based on depth fusion convolutional neural network
WO2021007812A1 (en) * 2019-07-17 2021-01-21 深圳大学 Deep neural network hyperparameter optimization method, electronic device and storage medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108629630A (en) * 2018-05-08 2018-10-09 广州太平洋电脑信息咨询有限公司 A kind of feature based intersects the advertisement recommendation method of joint deep neural network
WO2019238976A1 (en) * 2018-06-15 2019-12-19 Université de Liège Image classification using neural networks
WO2021007812A1 (en) * 2019-07-17 2021-01-21 深圳大学 Deep neural network hyperparameter optimization method, electronic device and storage medium
CN111444960A (en) * 2020-03-26 2020-07-24 上海交通大学 Skin disease image classification system based on multi-mode data input
CN111612066A (en) * 2020-05-21 2020-09-01 成都理工大学 Remote sensing image classification method based on depth fusion convolutional neural network

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
敬明?;: "基于深度神经网络的多模态特征自适应聚类方法", 计算机应用与软件, no. 10, pages 268 - 275 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113742428A (en) * 2021-09-20 2021-12-03 郑州颂和礼致信息科技有限公司 Neural network data set storage method based on block chain
CN113742428B (en) * 2021-09-20 2023-11-03 易点生活电子商务有限公司 Neural network data set storage method based on blockchain
CN114372181A (en) * 2021-12-27 2022-04-19 华南理工大学 Intelligent planning method for equipment production based on multi-mode data
CN114372181B (en) * 2021-12-27 2024-06-07 华南理工大学 Equipment production intelligent planning method based on multi-mode data

Also Published As

Publication number Publication date
CN112784919B (en) 2023-09-05

Similar Documents

Publication Publication Date Title
CN111126386B (en) Sequence domain adaptation method based on countermeasure learning in scene text recognition
CN110597735A (en) Software defect prediction method for open-source software defect feature deep learning
EP4027300A1 (en) Apparatus, program, and method for anomaly detection and classification
CN108647226B (en) Hybrid recommendation method based on variational automatic encoder
CN104933444B (en) A kind of design method of the multi-level clustering syncretizing mechanism towards multidimensional property data
CN111882446A (en) Abnormal account detection method based on graph convolution network
CN112015863A (en) Multi-feature fusion Chinese text classification method based on graph neural network
CN112784919B (en) Classification method for intelligent manufacturing multi-mode data
CN115578137A (en) Agricultural product future price prediction method and system based on text mining and deep learning model
CN115205521A (en) Kitchen waste detection method based on neural network
CN114881173A (en) Resume classification method and device based on self-attention mechanism
CN110516064A (en) A kind of Aeronautical R&D paper classification method based on deep learning
Dharwadkar et al. Floriculture classification using simple neural network and deep learning
CN112434145A (en) Picture-viewing poetry method based on image recognition and natural language processing
CN114925196B (en) Auxiliary eliminating method for abnormal blood test value of diabetes under multi-layer sensing network
CN114119562B (en) Brake disc outer surface defect detection method and system based on deep learning
CN113065005B (en) Legal provision recommendation method based on knowledge graph and text classification model
CN115391523A (en) Wind power plant multi-source heterogeneous data processing method and device
CN114169433A (en) Industrial fault prediction method based on federal learning + image learning + CNN
Kong et al. A one-shot learning approach for similarity retrieval of wafer bin maps with unknown failure pattern
CN113094567A (en) Malicious complaint identification method and system based on text clustering
CN112179846A (en) Prefabricated convex window defect detection system based on improved Faster R-CNN
Angayarkanni et al. Recognition of Disease in Leaves Using Genetic Algorithm and Neural Network Based Feature Selection
Rao et al. Markov random field classification technique for plant leaf disease detection
CN116341990B (en) Knowledge management evaluation method and system for infrastructure engineering

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant