CN112270337A - Heterogeneous information acquisition method and device based on feature mapping - Google Patents
Heterogeneous information acquisition method and device based on feature mapping Download PDFInfo
- Publication number
- CN112270337A CN112270337A CN202010974882.7A CN202010974882A CN112270337A CN 112270337 A CN112270337 A CN 112270337A CN 202010974882 A CN202010974882 A CN 202010974882A CN 112270337 A CN112270337 A CN 112270337A
- Authority
- CN
- China
- Prior art keywords
- data
- information
- original data
- data information
- fusion
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 65
- 238000013507 mapping Methods 0.000 title claims abstract description 40
- 230000004927 fusion Effects 0.000 claims abstract description 88
- 238000012545 processing Methods 0.000 claims abstract description 71
- 238000003860 storage Methods 0.000 claims abstract description 43
- 238000000605 extraction Methods 0.000 claims abstract description 22
- 238000007499 fusion processing Methods 0.000 claims abstract description 22
- 238000004364 calculation method Methods 0.000 claims description 37
- 238000013178 mathematical model Methods 0.000 claims description 30
- 238000013499 data model Methods 0.000 claims description 24
- 230000006835 compression Effects 0.000 claims description 16
- 238000007906 compression Methods 0.000 claims description 16
- 230000006870 function Effects 0.000 claims description 12
- 238000006243 chemical reaction Methods 0.000 claims description 6
- 238000003062 neural network model Methods 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 6
- 238000011176 pooling Methods 0.000 claims description 6
- 238000013500 data storage Methods 0.000 claims description 3
- 238000011084 recovery Methods 0.000 claims 1
- 238000007726 management method Methods 0.000 description 9
- 238000012423 maintenance Methods 0.000 description 5
- 238000004519 manufacturing process Methods 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 2
- 230000006399 behavior Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000012938 design process Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 238000011144 upstream manufacturing Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 1
- 230000003090 exacerbative effect Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 230000002265 prevention Effects 0.000 description 1
- 238000004886 process control Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses a heterogeneous information acquisition method and device based on feature mapping, wherein the method comprises the following steps: acquiring original data in various industrial control systems to obtain original data information; performing data dimension reduction processing on the original data information to obtain original data information after dimension reduction; performing data fusion processing on the original data information subjected to dimensionality reduction by using a characteristic extraction mode to obtain fused data information; and inputting the fused data information into a fusion storage module for storage based on the time sequence. In the embodiment of the invention, the storage management of the original data is greatly simplified; and realize that can be based on the characteristic data of the dimensionless to unitedly amalgamate between different physical dimension data structures; and when external, avoids direct exposure of the original sensitive data.
Description
Technical Field
The invention relates to the technical field of industrial data acquisition, in particular to a heterogeneous information acquisition method and device based on feature mapping.
Background
Industrial automation and informatization enable more and more industrial departments to realize global control of production processes through industrial control systems (such as ERP/MES/DCS/SCADA systems and the like); on the other hand, the industrial control system is often obviously different in the field implementation process under the influence of technical factors such as product process flow, upstream and downstream matching capacity and the like. Taking chemical industrial control as an example, the types of common industrial field control buses are more than 200, and the collected data relate to more than 40 physical dimensions. From the upstream and downstream of the chemical supply chain, the number of matched equipment manufacturers can reach more than 60 at most, and the technical scheme is emphasized. Therefore, the field process control of a typical industrial control system is an application scenario with obvious heterogeneous characteristics. In addition, from a non-technical perspective, in consideration of prevention of technical monopoly, technical secrecy, control safety and the like, a medium-large industrial department generally splits an industrial control system into different supply conditions, and different suppliers respectively perform equipment installation and debugging, thereby further exacerbating the problem of effective management of data types in the industrial control system.
Therefore, how to perform efficient information acquisition on massive heterogeneous states of different physical dimensions in an industrial control system and meanwhile achieve privacy protection on the basis of the confidentiality and safety requirements of core industrial process data through a data feature learning method will become an important basic support method for information effective interconnection in the future industrial 4.0 direction.
Disclosure of Invention
The invention aims to overcome the defects of the prior art, and provides a method and a device for acquiring heterogeneous information based on feature mapping, which greatly simplify the storage management of original data; and realize that can be based on the characteristic data of the dimensionless to unitedly amalgamate between different physical dimension data structures; and when external, avoids direct exposure of the original sensitive data.
In order to solve the above technical problem, an embodiment of the present invention provides a method for obtaining heterogeneous information based on feature mapping, where the method includes:
acquiring original data in various industrial control systems to obtain original data information;
performing data dimension reduction processing on the original data information to obtain original data information after dimension reduction;
performing data fusion processing on the original data information subjected to dimensionality reduction by using a characteristic extraction mode to obtain fused data information;
and inputting the fused data information into a fusion storage module for storage based on the time sequence.
Optionally, the method further includes:
calling a prior data model in a data model base based on the corresponding mathematical model to perform operation processing on the fused data information stored in the fusion storage module according to different types of characteristic values, and outputting a calculation result;
judging whether the calculation result meets an expected result or not;
and if not, performing data fusion processing on the original data information subjected to the dimensionality reduction again by using a feature extraction mode based on the calculation result to obtain fused data information.
Optionally, the corresponding mathematical model is a corresponding mathematical model which is constructed according to the application scenario requirements and has an operation structure and calculation parameters required by the application scenario, and can run normally and recover errors.
Optionally, the prior data model includes a template of a corresponding mathematical model, a callable sub-function, a function library, and other model types with prior knowledge.
Optionally, the acquiring the raw data in various industrial control systems to obtain raw data information includes:
judging the acquisition mode of the acquired original data, wherein the acquisition mode comprises contact acquisition and non-contact acquisition;
when the acquisition mode of the original data to be acquired is judged to be contact acquisition, acquiring the original data of various industrial control systems based on contact to obtain original data information, wherein the original data information is recordable digital information;
when the acquisition mode of the original data to be acquired is judged to be non-contact acquisition, parameter optimization processing is carried out on non-contact acquisition equipment according to actual industrial field conditions to obtain the processed non-contact acquisition equipment, wherein the non-contact acquisition equipment comprises industrial vision equipment, industrial remote sensing equipment and industrial physical characteristic conversion device equipment;
and acquiring original data of various industrial control systems based on the processed non-contact acquisition equipment to obtain original data information, wherein the original data information is image information.
Optionally, the performing data dimension reduction processing on the original data information to obtain dimension-reduced original data information includes:
when the original data information is acquired in a contact manner, lossless compression and dimension reduction processing are carried out on the recordable digital information to acquire original data information after dimension reduction;
when the original data information is acquired in a non-contact manner, compressing and dimension reduction processing is carried out on the image information to acquire original data information after dimension reduction; and compressing and dimension reducing the image information, wherein the compressing and dimension reducing processing comprises image denoising, compressing and pooling processing.
Optionally, the performing data fusion processing on the original data information after the dimensionality reduction by using a feature extraction manner to obtain fused data information includes:
removing physical dimensions of the original data subjected to dimensionality reduction of different related physical elements, taking the original data as input data, and inputting the input data into a data fusion model;
taking a fusion classification target function as an output value of the data fusion model to obtain a fusion classification result;
and finishing a fusion mapping process between the input data and the fusion classification result in the data fusion model based on a neural network model, and reserving all parameters generated in the fusion mapping process as characteristic values to obtain fused data information.
Optionally, the fused data information is a characteristic value without physical dimension.
In addition, an embodiment of the present invention further provides a device for acquiring heterogeneous information based on feature mapping, where the device includes:
a data acquisition module: the system is used for collecting original data in various industrial control systems to obtain original data information;
a data processing module: the system is used for performing data dimension reduction processing on the original data information to obtain original data information after dimension reduction;
a data fusion module: the system is used for performing data fusion processing on the original data information subjected to the dimensionality reduction by utilizing a characteristic extraction mode to obtain fused data information;
a data storage module: and the data information fusion module is used for inputting the fused data information into the fusion storage module for storage based on the time sequence.
Optionally, the apparatus further comprises:
an operation processing module: the device is used for calling a prior data model in a data model base based on a corresponding mathematical model to perform operation processing on the fused data information stored in the fusion storage module according to different types of characteristic values and outputting a calculation result;
a judging module: the device is used for judging whether the calculation result meets an expected result or not; and when the expected result is judged not to be met, performing data fusion processing on the original data information subjected to dimensionality reduction again by using a feature extraction mode based on the calculation result to obtain fused data information.
In the embodiment of the invention, the storage management of the original data is greatly simplified; the unified fusion of different physical dimension data structures based on dimensionless characteristic data is realized, so that the collected data not only retains the characteristic significance of original data, but also can avoid the problems of excessively large data structure and difficult maintenance and storage caused by carrying physical dimensions in the parameters of an industrial control system; and when external, avoids direct exposure of the original sensitive data.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.
Fig. 1 is a schematic flowchart of a feature mapping-based heterogeneous information acquisition method in an embodiment of the present invention;
fig. 2 is a schematic structural composition diagram of a heterogeneous information acquisition apparatus based on feature mapping in an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Examples
Referring to fig. 1, fig. 1 is a schematic flowchart illustrating a method for obtaining heterogeneous information based on feature mapping according to an embodiment of the present invention.
As shown in fig. 1, a method for obtaining heterogeneous information based on feature mapping includes:
s11: acquiring original data in various industrial control systems to obtain original data information;
in the specific implementation process of the present invention, the acquiring of the raw data in various industrial control systems to obtain the raw data information includes: judging the acquisition mode of the acquired original data, wherein the acquisition mode comprises contact acquisition and non-contact acquisition; when the acquisition mode of the original data to be acquired is judged to be contact acquisition, acquiring the original data of various industrial control systems based on contact to obtain original data information, wherein the original data information is recordable digital information; when the acquisition mode of the original data to be acquired is judged to be non-contact acquisition, parameter optimization processing is carried out on non-contact acquisition equipment according to actual industrial field conditions to obtain the processed non-contact acquisition equipment, wherein the non-contact acquisition equipment comprises industrial vision equipment, industrial remote sensing equipment and industrial physical characteristic conversion device equipment; and acquiring original data of various industrial control systems based on the processed non-contact acquisition equipment to obtain original data information, wherein the original data information is image information.
Specifically, in an industrial control system, the types of data are many, some data can be directly acquired in a contact manner, and some data need to be acquired in a non-contact manner; therefore, before data acquisition, firstly, the acquisition mode of the acquired original data needs to be judged, wherein the acquisition mode comprises a contact acquisition mode and a non-contact acquisition mode; when the acquisition mode of the raw data to be acquired is judged to be contact acquisition, then the raw data of various industrial control systems are acquired according to the contact acquisition mode to obtain raw data information, and the raw data information is recordable digital information.
When the acquisition mode of the original data to be acquired is judged to be non-contact acquisition, parameter optimization processing needs to be carried out on non-contact acquisition equipment according to actual industrial field conditions, so that the processed non-contact acquisition equipment is obtained, wherein the non-contact acquisition equipment comprises industrial vision equipment, industrial remote sensing equipment and industrial physical characteristic conversion device equipment; then, acquiring the original data of various industrial control systems by using the processed non-contact acquisition equipment to obtain original data information; these raw data information are image information.
S12: performing data dimension reduction processing on the original data information to obtain original data information after dimension reduction;
in a specific implementation process of the present invention, the performing data dimension reduction processing on the original data information to obtain dimension-reduced original data information includes: when the original data information is acquired in a contact manner, lossless compression and dimension reduction processing are carried out on the recordable digital information to acquire original data information after dimension reduction; when the original data information is acquired in a non-contact manner, compressing and dimension reduction processing is carried out on the image information to acquire original data information after dimension reduction; and compressing and dimension reducing the image information, wherein the compressing and dimension reducing processing comprises image denoising, compressing and pooling processing.
Specifically, the original data information is subjected to data dimension reduction processing, wherein the original data information is in different forms, the collected data dimension reduction modes are different, when the original data information is acquired in a contact manner, namely, when the original data information is recordable digitalized information, lossless compression and dimension reduction processing are required to be performed on the original data information, the lossless compression processing is performed to be a digital compression algorithm, after the original data information is compressed by the digital compression algorithm, space mapping processing is performed by the data space mapping algorithm, and finally lossless compression and dimension reduction processing are realized, so that the original data information after dimension reduction is obtained.
When the original data information is acquired in a non-contact manner, compressing and dimension reduction processing is carried out on the original data information to obtain original data information after dimension reduction; since the original data information is image information, the compression and dimension reduction processing includes: and denoising the image information, compressing, pooling, and performing space mapping by using a data space mapping algorithm to obtain original data information after dimension reduction.
S13: performing data fusion processing on the original data information subjected to dimensionality reduction by using a characteristic extraction mode to obtain fused data information;
in a specific implementation process of the present invention, the performing data fusion processing on the original data information after the dimensionality reduction by using a feature extraction manner to obtain fused data information includes: removing physical dimensions of the original data subjected to dimensionality reduction of different related physical elements, taking the original data as input data, and inputting the input data into a data fusion model; taking a fusion classification target function as an output value of the data fusion model to obtain a fusion classification result; and finishing a fusion mapping process between the input data and the fusion classification result in the data fusion model based on a neural network model, and reserving all parameters generated in the fusion mapping process as characteristic values to obtain fused data information.
Further, the fused data information is a characteristic value without physical dimension.
Specifically, data fusion processing is performed on the original data information after dimensionality reduction, mainly data fusion processing is performed on the original data information after dimensionality reduction acquired in a contact mode and the original data information after dimensionality reduction acquired in a non-contact mode, and fusion processing is performed on the data through a feature extraction method.
Removing physical dimensions of the original data (the original data information after dimension reduction acquired by contact and the original data information after dimension reduction acquired by non-contact) after dimension reduction of different physical elements, taking the original data as input data, and inputting all the input data into a data fusion model; then, taking a fusion classification target function as an output value of the data fusion model to obtain a fusion classification result; and then completing a fusion mapping process between the input data and the fusion classification result in the data fusion model according to the neural network model or other classification network models, and reserving all parameters generated in the fusion mapping process as characteristic values to obtain fused data information.
And the fused data information is a characteristic value without physical dimension.
S14: inputting the fused data information into a fusion storage module for storage based on the time sequence;
in the specific implementation process of the invention, after the fused data information is obtained after the data fusion is finished, the fused data information with the characteristic values of the physical dimension or not is input into the fusion storage module for storage according to the time sequence.
S15: calling a prior data model in a data model base based on the corresponding mathematical model to perform operation processing on the fused data information stored in the fusion storage module according to different types of characteristic values, and outputting a calculation result;
in the specific implementation process of the invention, the corresponding mathematical model is a corresponding mathematical model which is constructed according to the application scene requirements and has the operation structure and the calculation parameters required by the application scene, and can normally run and recover errors.
Further, the prior data model includes a template of a corresponding mathematical model, a callable sub-function, a function library and other model types with prior knowledge.
Specifically, according to specific requirements, a corresponding mathematical model is used for calling a prior data model in a data model base to perform operation processing on fused data information stored in a fusion storage module according to different types of characteristic values, and a calculation result is output; when different types of characteristic values are operated, the operation process includes but is not limited to execution of various mathematical models such as algebraic operation and geometric mapping and result acquisition; the output process of the mathematical model includes, but is not limited to, design and interpretation. The design process is that mathematical models which have various operation structures and calculation parameters, can normally run and can recover errors are constructed according to the requirements of application scenes. The interpretation process is to interpret all the operation results, and when the operation results meet the requirements, a correct conclusion is output by the mathematical model; the corresponding prior data model is stored in the data model base and comprises a template of the corresponding mathematical model, a callable sub-function, a function base and other model types with prior knowledge.
S16: judging whether the calculation result meets an expected result or not;
in the specific implementation process of the invention, whether the calculation result meets the expected result is judged, when the calculation result does not meet the expected result, the calculation result is fed back to the step S13, and in the step S13, the original data information after dimension reduction is subjected to data fusion again by using a feature extraction mode according to the calculation result, so that the fused data information is obtained.
S17: and if so, accessing the fused data information to a public network.
In the specific implementation process of the invention, if the operation result is in accordance with the expectation, the fused data information is accessed into the public network; and according to the mathematical principle of zero knowledge proof, the industrial control system can prove that the industrial control system corresponding to the heterogeneous data acquisition device has the capacity required by the application party without showing the original data to the application party.
Through the embodiment, the heterogeneous acquisition problem of the common industrial control system during information interconnection is solved: the information bus and the storage processing mode of the industrial department need to be pertinently optimized according to industrial characteristics, so that the common industrial information bus can be dozens of types, and the information system can be hundreds of types, thereby generating tens of thousands of industrial information data combinations. Therefore, a unified information acquisition and processing device is needed, and a data acquisition process without physical dimension can be performed for various heterogeneous industrial environments, so that the acquired data not only retains the characteristic significance of original data, but also can avoid the problems that the data structure is too large and the maintenance and storage are difficult if the parameters of an industrial control system carry physical dimensions; problem of feature learning and privacy protection: the industrial department has extremely high privacy protection requirements on key data of industrial cores, and the core data cannot be accessed through a public network; on the other hand, the industrial internet also requires that the acquired heterogeneous data have interconnection capacity; therefore, it is necessary to extract key features of key raw data that is not allowed to access the public network based on the main features of the data, and draw a conclusion that the main features of the information can be completely expressed without exposing the raw data.
Therefore, the implementation of the embodiment can perform unified dimensionless processing on various kinds of heterogeneous information collected in the industrial field according to the industrial field operation rules. The heterogeneous information refers to a data set formed by various information including but not limited to industrial optical data, remote control remote sensing data, various industrial bus data, enterprise financial systems, sales systems and the like. The industrial field operation rule refers to various kinds of knowledge expected to be acquired from the data set, and includes but is not limited to behaviors such as abnormal alarming, yield prediction and the like; in the parameter convergence calculation process of the artificial intelligence network, no direct physical dimension mapping relation exists between original data and classification results, and the intermediate calculation parameters do not have any physical significance. Therefore, the project performs two processing steps of dimension reduction and fusion on the industrial heterogeneous information, extracts dimensionless characteristics of the industrial control original data based on a neural network mode, performs data fusion based on the dimensionless characteristics, and greatly simplifies the storage and management of the original data. For example, financial systems, ERP management systems and MES/DCS/SCADA production systems of enterprises often generate different structural data, and data fusion communication of the systems is usually a main informatization contradiction of enterprise operation. The invention realizes unified information acquisition and processing of various heterogeneous data by two steps of dimension reduction and data fusion of the heterogeneous data, and data structures with different physical dimensions can be unified and fused based on dimensionless characteristic data; aiming at the industrial privacy protection appeal that typical key data cannot be subjected to information interaction through a network, the method and the device avoid direct exposure of original sensitive data by performing feature extraction on the key data. On the basis of the extracted data features, model analysis is performed in a user planning model through a basic principle of zero knowledge proof, so that a complete proof of the capability of the extracted data features is obtained.
In the embodiment of the invention, the storage management of the original data is greatly simplified; the unified fusion of different physical dimension data structures based on dimensionless characteristic data is realized, so that the collected data not only retains the characteristic significance of original data, but also can avoid the problems of excessively large data structure and difficult maintenance and storage caused by carrying physical dimensions in the parameters of an industrial control system; and when external, avoids direct exposure of the original sensitive data.
Examples
Referring to fig. 2, fig. 2 is a schematic structural composition diagram of a heterogeneous information acquisition device based on feature mapping according to an embodiment of the present invention.
As shown in fig. 2, an apparatus for obtaining heterogeneous information based on feature mapping, the apparatus comprising:
the data acquisition module 21: the system is used for collecting original data in various industrial control systems to obtain original data information;
in the specific implementation process of the present invention, the acquiring of the raw data in various industrial control systems to obtain the raw data information includes: judging the acquisition mode of the acquired original data, wherein the acquisition mode comprises contact acquisition and non-contact acquisition; when the acquisition mode of the original data to be acquired is judged to be contact acquisition, acquiring the original data of various industrial control systems based on contact to obtain original data information, wherein the original data information is recordable digital information; when the acquisition mode of the original data to be acquired is judged to be non-contact acquisition, parameter optimization processing is carried out on non-contact acquisition equipment according to actual industrial field conditions to obtain the processed non-contact acquisition equipment, wherein the non-contact acquisition equipment comprises industrial vision equipment, industrial remote sensing equipment and industrial physical characteristic conversion device equipment; and acquiring original data of various industrial control systems based on the processed non-contact acquisition equipment to obtain original data information, wherein the original data information is image information.
Specifically, in an industrial control system, the types of data are many, some data can be directly acquired in a contact manner, and some data need to be acquired in a non-contact manner; therefore, before data acquisition, firstly, the acquisition mode of the acquired original data needs to be judged, wherein the acquisition mode comprises a contact acquisition mode and a non-contact acquisition mode; when the acquisition mode of the raw data to be acquired is judged to be contact acquisition, then the raw data of various industrial control systems are acquired according to the contact acquisition mode to obtain raw data information, and the raw data information is recordable digital information.
When the acquisition mode of the original data to be acquired is judged to be non-contact acquisition, parameter optimization processing needs to be carried out on non-contact acquisition equipment according to actual industrial field conditions, so that the processed non-contact acquisition equipment is obtained, wherein the non-contact acquisition equipment comprises industrial vision equipment, industrial remote sensing equipment and industrial physical characteristic conversion device equipment; then, acquiring the original data of various industrial control systems by using the processed non-contact acquisition equipment to obtain original data information; these raw data information are image information.
The data processing module 22: the system is used for performing data dimension reduction processing on the original data information to obtain original data information after dimension reduction;
in a specific implementation process of the present invention, the performing data dimension reduction processing on the original data information to obtain dimension-reduced original data information includes: when the original data information is acquired in a contact manner, lossless compression and dimension reduction processing are carried out on the recordable digital information to acquire original data information after dimension reduction; when the original data information is acquired in a non-contact manner, compressing and dimension reduction processing is carried out on the image information to acquire original data information after dimension reduction; and compressing and dimension reducing the image information, wherein the compressing and dimension reducing processing comprises image denoising, compressing and pooling processing.
Specifically, the original data information is subjected to data dimension reduction processing, wherein the original data information is in different forms, the collected data dimension reduction modes are different, when the original data information is acquired in a contact manner, namely, when the original data information is recordable digitalized information, lossless compression and dimension reduction processing are required to be performed on the original data information, the lossless compression processing is performed to be a digital compression algorithm, after the original data information is compressed by the digital compression algorithm, space mapping processing is performed by the data space mapping algorithm, and finally lossless compression and dimension reduction processing are realized, so that the original data information after dimension reduction is obtained.
When the original data information is acquired in a non-contact manner, compressing and dimension reduction processing is carried out on the original data information to obtain original data information after dimension reduction; since the original data information is image information, the compression and dimension reduction processing includes: and denoising the image information, compressing, pooling, and performing space mapping by using a data space mapping algorithm to obtain original data information after dimension reduction.
The data fusion module 23: the system is used for performing data fusion processing on the original data information subjected to the dimensionality reduction by utilizing a characteristic extraction mode to obtain fused data information;
in a specific implementation process of the present invention, the performing data fusion processing on the original data information after the dimensionality reduction by using a feature extraction manner to obtain fused data information includes: removing physical dimensions of the original data subjected to dimensionality reduction of different related physical elements, taking the original data as input data, and inputting the input data into a data fusion model; taking a fusion classification target function as an output value of the data fusion model to obtain a fusion classification result; and finishing a fusion mapping process between the input data and the fusion classification result in the data fusion model based on a neural network model, and reserving all parameters generated in the fusion mapping process as characteristic values to obtain fused data information.
Further, the fused data information is a characteristic value without physical dimension.
Specifically, data fusion processing is performed on the original data information after dimensionality reduction, mainly data fusion processing is performed on the original data information after dimensionality reduction acquired in a contact mode and the original data information after dimensionality reduction acquired in a non-contact mode, and fusion processing is performed on the data through a feature extraction method.
Removing physical dimensions of the original data (the original data information after dimension reduction acquired by contact and the original data information after dimension reduction acquired by non-contact) after dimension reduction of different physical elements, taking the original data as input data, and inputting all the input data into a data fusion model; then, taking a fusion classification target function as an output value of the data fusion model to obtain a fusion classification result; and then completing a fusion mapping process between the input data and the fusion classification result in the data fusion model according to the neural network model or other classification network models, and reserving all parameters generated in the fusion mapping process as characteristic values to obtain fused data information.
And the fused data information is a characteristic value without physical dimension.
The data storage module 24: the fusion storage module is used for inputting the fused data information into the fusion storage module for storage based on the time sequence;
in the specific implementation process of the invention, after the fused data information is obtained after the data fusion is finished, the fused data information with the characteristic values of the physical dimension or not is input into the fusion storage module for storage according to the time sequence.
The operation processing module 25: the device is used for calling a prior data model in a data model base based on a corresponding mathematical model to perform operation processing on the fused data information stored in the fusion storage module according to different types of characteristic values and outputting a calculation result;
in the specific implementation process of the invention, the corresponding mathematical model is a corresponding mathematical model which is constructed according to the application scene requirements and has the operation structure and the calculation parameters required by the application scene, and can normally run and recover errors.
Further, the prior data model includes a template of a corresponding mathematical model, a callable sub-function, a function library and other model types with prior knowledge.
Specifically, according to specific requirements, a corresponding mathematical model is used for calling a prior data model in a data model base to perform operation processing on fused data information stored in a fusion storage module according to different types of characteristic values, and a calculation result is output; when different types of characteristic values are operated, the operation process includes but is not limited to execution of various mathematical models such as algebraic operation and geometric mapping and result acquisition; the output process of the mathematical model includes, but is not limited to, design and interpretation. The design process is that mathematical models which have various operation structures and calculation parameters, can normally run and can recover errors are constructed according to the requirements of application scenes. The interpretation process is to interpret all the operation results, and when the operation results meet the requirements, a correct conclusion is output by the mathematical model; the corresponding prior data model is stored in the data model base and comprises a template of the corresponding mathematical model, a callable sub-function, a function base and other model types with prior knowledge.
The judging module 26: the device is used for judging whether the calculation result meets an expected result or not; and when the expected result is judged not to be met, performing data fusion processing on the original data information subjected to dimensionality reduction again by using a feature extraction mode based on the calculation result to obtain fused data information.
In the specific implementation process of the present invention, it is determined whether the calculation result meets the expected result, when it is determined that the calculation result does not meet the expected result, the calculation result is fed back to the data fusion module 23, and the original data information after dimension reduction is subjected to data fusion again in the data fusion module 23 by using a feature extraction method according to the calculation result, so as to obtain the fused data information.
In the specific implementation process of the invention, if the operation result is in accordance with the expectation, the fused data information is accessed into the public network; and according to the mathematical principle of zero knowledge proof, the industrial control system can prove that the industrial control system corresponding to the heterogeneous data acquisition device has the capacity required by the application party without showing the original data to the application party.
Through the embodiment, the heterogeneous acquisition problem of the common industrial control system during information interconnection is solved: the information bus and the storage processing mode of the industrial department need to be pertinently optimized according to industrial characteristics, so that the common industrial information bus can be dozens of types, and the information system can be hundreds of types, thereby generating tens of thousands of industrial information data combinations. Therefore, a unified information acquisition and processing device is needed, and a data acquisition process without physical dimension can be performed for various heterogeneous industrial environments, so that the acquired data not only retains the characteristic significance of original data, but also can avoid the problems that the data structure is too large and the maintenance and storage are difficult if the parameters of an industrial control system carry physical dimensions; problem of feature learning and privacy protection: the industrial department has extremely high privacy protection requirements on key data of industrial cores, and the core data cannot be accessed through a public network; on the other hand, the industrial internet also requires that the acquired heterogeneous data have interconnection capacity; therefore, it is necessary to extract key features of key raw data that is not allowed to access the public network based on the main features of the data, and draw a conclusion that the main features of the information can be completely expressed without exposing the raw data.
Therefore, the implementation of the embodiment can perform unified dimensionless processing on various kinds of heterogeneous information collected in the industrial field according to the industrial field operation rules. The heterogeneous information refers to a data set formed by various information including but not limited to industrial optical data, remote control remote sensing data, various industrial bus data, enterprise financial systems, sales systems and the like. The industrial field operation rule refers to various kinds of knowledge expected to be acquired from the data set, and includes but is not limited to behaviors such as abnormal alarming, yield prediction and the like; in the parameter convergence calculation process of the artificial intelligence network, no direct physical dimension mapping relation exists between original data and classification results, and the intermediate calculation parameters do not have any physical significance. Therefore, the project performs two processing steps of dimension reduction and fusion on the industrial heterogeneous information, extracts dimensionless characteristics of the industrial control original data based on a neural network mode, performs data fusion based on the dimensionless characteristics, and greatly simplifies the storage and management of the original data. For example, financial systems, ERP management systems and MES/DCS/SCADA production systems of enterprises often generate different structural data, and data fusion communication of the systems is usually a main informatization contradiction of enterprise operation. The invention realizes unified information acquisition and processing of various heterogeneous data by two steps of dimension reduction and data fusion of the heterogeneous data, and data structures with different physical dimensions can be unified and fused based on dimensionless characteristic data; aiming at the industrial privacy protection appeal that typical key data cannot be subjected to information interaction through a network, the method and the device avoid direct exposure of original sensitive data by performing feature extraction on the key data. On the basis of the extracted data features, model analysis is performed in a user planning model through a basic principle of zero knowledge proof, so that a complete proof of the capability of the extracted data features is obtained.
In the embodiment of the invention, the storage management of the original data is greatly simplified; the unified fusion of different physical dimension data structures based on dimensionless characteristic data is realized, so that the collected data not only retains the characteristic significance of original data, but also can avoid the problems of excessively large data structure and difficult maintenance and storage caused by carrying physical dimensions in the parameters of an industrial control system; and when external, avoids direct exposure of the original sensitive data.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable storage medium, and the storage medium may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like.
In addition, the method and the apparatus for obtaining heterogeneous information based on feature mapping according to the embodiments of the present invention are described in detail above, and a specific example should be used herein to explain the principle and the implementation manner of the present invention, and the description of the above embodiment is only used to help understanding the method and the core idea of the present invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.
Claims (10)
1. A heterogeneous information acquisition method based on feature mapping is characterized by comprising the following steps:
acquiring original data in various industrial control systems to obtain original data information;
performing data dimension reduction processing on the original data information to obtain original data information after dimension reduction;
performing data fusion processing on the original data information subjected to dimensionality reduction by using a characteristic extraction mode to obtain fused data information;
and inputting the fused data information into a fusion storage module for storage based on the time sequence.
2. The heterogeneous information acquisition method according to claim 1, further comprising:
calling a prior data model in a data model base based on the corresponding mathematical model to perform operation processing on the fused data information stored in the fusion storage module according to different types of characteristic values, and outputting a calculation result;
judging whether the calculation result meets an expected result or not;
and if not, performing data fusion processing on the original data information subjected to the dimensionality reduction again by using a feature extraction mode based on the calculation result to obtain fused data information.
3. The method for acquiring heterogeneous information according to claim 2, wherein the corresponding mathematical model is a corresponding mathematical model which is constructed according to application scenario requirements and has an operation structure, calculation parameters, normal operation and error recovery required by the application scenario.
4. The method of claim 2, wherein the prior data model comprises a template of a corresponding mathematical model, a callable sub-function, a library of functions, and other model types with prior knowledge.
5. The method for acquiring heterogeneous information according to claim 1, wherein the acquiring raw data in various industrial control systems to obtain raw data information comprises:
judging the acquisition mode of the acquired original data, wherein the acquisition mode comprises contact acquisition and non-contact acquisition;
when the acquisition mode of the original data to be acquired is judged to be contact acquisition, acquiring the original data of various industrial control systems based on the contact acquisition mode to obtain original data information, wherein the original data information is recordable digital information;
when the acquisition mode of the original data to be acquired is judged to be non-contact acquisition, parameter optimization processing is carried out on non-contact acquisition equipment according to actual industrial field conditions to obtain the processed non-contact acquisition equipment, wherein the non-contact acquisition equipment comprises industrial vision equipment, industrial remote sensing equipment and industrial physical characteristic conversion device equipment;
and acquiring original data of various industrial control systems based on the processed non-contact acquisition equipment to obtain original data information, wherein the original data information is image information.
6. The method for acquiring heterogeneous information according to claim 5, wherein the performing data dimension reduction processing on the original data information to obtain dimension-reduced original data information includes:
when the original data information is acquired in a contact manner, lossless compression and dimension reduction processing are carried out on the recordable digital information to acquire original data information after dimension reduction;
when the original data information is acquired in a non-contact manner, compressing and dimension reduction processing is carried out on the image information to acquire original data information after dimension reduction; and compressing and dimension reducing the image information, wherein the compressing and dimension reducing processing comprises image denoising, compressing and pooling processing.
7. The method for acquiring heterogeneous information according to claim 1, wherein the performing data fusion processing on the original data information after the dimensionality reduction by using a feature extraction manner to obtain fused data information comprises:
removing physical dimensions of the original data subjected to dimensionality reduction of different related physical elements, taking the original data as input data, and inputting the input data into a data fusion model;
taking a fusion classification target function as an output value of the data fusion model to obtain a fusion classification result;
and finishing a fusion mapping process between the input data and the fusion classification result in the data fusion model based on a neural network model, and reserving all parameters generated in the fusion mapping process as characteristic values to obtain fused data information.
8. The method according to claim 7, wherein the fused data information is a feature value without physical dimension.
9. An apparatus for obtaining heterogeneous information based on feature mapping, the apparatus comprising:
a data acquisition module: the system is used for collecting original data in various industrial control systems to obtain original data information;
a data processing module: the system is used for performing data dimension reduction processing on the original data information to obtain original data information after dimension reduction;
a data fusion module: the system is used for performing data fusion processing on the original data information subjected to the dimensionality reduction by utilizing a characteristic extraction mode to obtain fused data information;
a data storage module: and the data information fusion module is used for inputting the fused data information into the fusion storage module for storage based on the time sequence.
10. The heterogeneous information acquisition apparatus according to claim 9, wherein the apparatus further comprises:
an operation processing module: the device is used for calling a prior data model in a data model base based on a corresponding mathematical model to perform operation processing on the fused data information stored in the fusion storage module according to different types of characteristic values and outputting a calculation result;
a judging module: the device is used for judging whether the calculation result meets an expected result or not; and when the expected result is judged not to be met, performing data fusion processing on the original data information subjected to dimensionality reduction again by using a feature extraction mode based on the calculation result to obtain fused data information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010974882.7A CN112270337A (en) | 2020-09-16 | 2020-09-16 | Heterogeneous information acquisition method and device based on feature mapping |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010974882.7A CN112270337A (en) | 2020-09-16 | 2020-09-16 | Heterogeneous information acquisition method and device based on feature mapping |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112270337A true CN112270337A (en) | 2021-01-26 |
Family
ID=74348793
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010974882.7A Pending CN112270337A (en) | 2020-09-16 | 2020-09-16 | Heterogeneous information acquisition method and device based on feature mapping |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112270337A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113110351A (en) * | 2021-04-28 | 2021-07-13 | 广东省科学院智能制造研究所 | Industrial production field heterogeneous state data acquisition system and method |
| CN113626511A (en) * | 2021-08-12 | 2021-11-09 | 山东勤成健康科技股份有限公司 | Heterogeneous database fusion access system |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109241169A (en) * | 2018-11-16 | 2019-01-18 | 广西电网有限责任公司电力科学研究院 | The multi-source heterogeneous data fusion geo-database integration method of power distribution network operation information |
| CN109460794A (en) * | 2018-11-19 | 2019-03-12 | 广东工业大学 | A kind of data characteristics extracting method, system and electronic equipment and storage medium |
| CN110502678A (en) * | 2019-08-09 | 2019-11-26 | 国网山西省电力公司 | Bulk power grid regulates and controls multidimensional data fusion method, terminal device and storage medium |
-
2020
- 2020-09-16 CN CN202010974882.7A patent/CN112270337A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109241169A (en) * | 2018-11-16 | 2019-01-18 | 广西电网有限责任公司电力科学研究院 | The multi-source heterogeneous data fusion geo-database integration method of power distribution network operation information |
| CN109460794A (en) * | 2018-11-19 | 2019-03-12 | 广东工业大学 | A kind of data characteristics extracting method, system and electronic equipment and storage medium |
| CN110502678A (en) * | 2019-08-09 | 2019-11-26 | 国网山西省电力公司 | Bulk power grid regulates and controls multidimensional data fusion method, terminal device and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113110351A (en) * | 2021-04-28 | 2021-07-13 | 广东省科学院智能制造研究所 | Industrial production field heterogeneous state data acquisition system and method |
| CN113626511A (en) * | 2021-08-12 | 2021-11-09 | 山东勤成健康科技股份有限公司 | Heterogeneous database fusion access system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CA3037326C (en) | Sparse neural network based anomaly detection in multi-dimensional time series | |
| CN112270337A (en) | Heterogeneous information acquisition method and device based on feature mapping | |
| CN111931172B (en) | Financial system business process abnormality early warning method and device | |
| CN110471945B (en) | Active data processing method, system, computer equipment and storage medium | |
| KR102359090B1 (en) | Method and System for Real-time Abnormal Insider Event Detection on Enterprise Resource Planning System | |
| CN111898247A (en) | Landslide displacement prediction method, equipment and storage medium | |
| CN111859451A (en) | Processing system of multi-source multi-modal data and method applying same | |
| CN116403253A (en) | Face recognition monitoring management system and method based on convolutional neural network | |
| CN112597238A (en) | Method, system, device and medium for establishing knowledge graph based on personnel information | |
| CN114782161A (en) | Method, device, storage medium and electronic device for identifying risky users | |
| CN117495422A (en) | Cost management system and method based on power communication network construction | |
| CN117291575A (en) | Equipment maintenance method, equipment maintenance device, computer equipment and storage medium | |
| CN115934484B (en) | Diffusion model data enhancement-based anomaly detection method, storage medium and apparatus | |
| CN117408959A (en) | Model training method, defect detection method, device, electronic equipment and medium | |
| JP2022007232A (en) | Information processing method and information processing device | |
| CN116707859A (en) | Feature rule extraction method and device, and network intrusion detection method and device | |
| CN113447572B (en) | Steel rail flaw detection method, electronic device, steel rail flaw detection vehicle and readable storage medium | |
| CN110087230B (en) | Data processing method, data processing device, storage medium and electronic equipment | |
| CN111859370B (en) | Method, apparatus, electronic device and computer readable storage medium for identifying service | |
| CN114841267A (en) | Real-time prediction method and device, electronic equipment and computer program product | |
| CN111597896B (en) | Abnormal face recognition method, recognition device, recognition apparatus, and storage medium | |
| CN114330369A (en) | Local production marketing management method, device and equipment based on intelligent voice analysis | |
| CN114565794A (en) | Bearing fault classification method, device, equipment and storage medium | |
| CN114648527B (en) | Urothelial cell slide image classification method, device, equipment and medium | |
| CN114020974B (en) | Sample data determination and dialogue intention identification method, storage medium, and program product |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |