CN116881017A - Collaborative virtual maintenance training system and method - Google Patents
Collaborative virtual maintenance training system and method Download PDFInfo
- Publication number
- CN116881017A CN116881017A CN202310935654.2A CN202310935654A CN116881017A CN 116881017 A CN116881017 A CN 116881017A CN 202310935654 A CN202310935654 A CN 202310935654A CN 116881017 A CN116881017 A CN 116881017A
- Authority
- CN
- China
- Prior art keywords
- system operation
- feature vector
- semantic
- interaction
- semantic coding
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012549 training Methods 0.000 title claims abstract description 61
- 238000000034 method Methods 0.000 title claims abstract description 31
- 230000002452 interceptive effect Effects 0.000 claims abstract description 31
- 238000001514 detection method Methods 0.000 claims abstract description 24
- 239000013598 vector Substances 0.000 claims description 166
- 230000003993 interaction Effects 0.000 claims description 77
- 238000005457 optimization Methods 0.000 claims description 22
- 230000007246 mechanism Effects 0.000 claims description 20
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000012423 maintenance Methods 0.000 abstract description 15
- 230000006870 function Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 9
- 238000005516 engineering process Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 6
- 238000012986 modification Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 5
- 239000011159 matrix material Substances 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 230000009466 transformation Effects 0.000 description 4
- 230000004913 activation Effects 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 3
- 230000006399 behavior Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000004927 fusion Effects 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 238000004088 simulation Methods 0.000 description 3
- 241000282414 Homo sapiens Species 0.000 description 2
- 241000699666 Mus <mouse, genus> Species 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000007477 logistic regression Methods 0.000 description 2
- 210000002569 neuron Anatomy 0.000 description 2
- 230000008447 perception Effects 0.000 description 2
- 230000008439 repair process Effects 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 206010000117 Abnormal behaviour Diseases 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 230000008451 emotion Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 235000003642 hunger Nutrition 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000036651 mood Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000004043 responsiveness Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/52—Program synchronisation; Mutual exclusion, e.g. by means of semaphores
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/048—Activation functions
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application discloses a collaborative virtual maintenance training system and a collaborative virtual maintenance training method, which are technically characterized in that: the method comprises the following steps: firstly, acquiring a first system operation and a second system operation, then, carrying out semantic feature interactive coding on the first system operation and the second system operation to obtain interactive features between the system operations, and then, determining whether the first system operation and the second system operation have conflict or not based on the interactive features between the system operations. Therefore, the accuracy and the efficiency of conflict detection can be improved, so that the stability and the safety of the system are improved, and better training and practice environments are provided for maintenance personnel.
Description
Technical Field
The present application relates to the field of virtual repair training, and more particularly, to a collaborative virtual repair training system and method.
Background
The collaborative virtual maintenance is to cooperatively perform maintenance work of equipment under the remote condition through a virtual technology and a collaborative work platform, combines the concepts of virtual reality, augmented reality and collaborative work, and can help technicians to perform real-time collaborative maintenance between different places.
With the continuous development of technology, virtual maintenance training systems are widely used in the industry. The system can simulate a real maintenance environment, provides practical opportunities for maintenance personnel, and improves the skills and capabilities of the maintenance personnel. However, in the virtual maintenance training system, there is a problem of operation conflict. An operation conflict refers to the situation where there is incompatibility or contradiction between two or more operations occurring simultaneously in the system. This may lead to abnormal behaviour of the system and even threaten the security of the system.
Accordingly, an optimized collaborative virtual maintenance training system is desired that automatically detects system operational conflicts.
Disclosure of Invention
In order to solve the technical problems, the application provides a collaborative virtual maintenance training system and a collaborative virtual maintenance training method, which can carry out semantic interactive association coding on two system operations after the two system operations are acquired so as to automatically carry out conflict detection on the two system operations.
According to one aspect of the present application, there is provided a collaborative virtual maintenance training system comprising:
the system operation acquisition module is used for acquiring a first system operation and a second system operation;
the system operation semantic understanding module is used for carrying out semantic feature interactive coding on the first system operation and the second system operation so as to obtain interactive features between the system operations; and
and the system operation conflict detection module is used for determining whether conflict exists between the first system operation and the second system operation based on the interaction characteristics between the system operations.
Optionally, the system operates a semantic understanding module, including:
the first system operation semantic understanding unit is used for carrying out semantic coding on the first system operation to obtain a first system operation semantic coding feature vector;
the second system operation semantic understanding unit is used for carrying out semantic coding on the second system operation to obtain a second system operation semantic coding feature vector; and
and the system operation semantic interaction unit is used for carrying out feature interaction on the first system operation semantic coding feature vector and the second system operation semantic coding feature vector to obtain a system operation interaction feature vector serving as the system operation interaction feature.
Optionally, the first system operation semantic understanding unit is configured to:
passing the first system operation through a semantic encoder comprising an embedded layer to obtain the first system operation semantic coding feature vector.
Optionally, the second system operates a semantic understanding unit for:
and enabling the second system operation to pass through the semantic encoder comprising the embedded layer to obtain the second system operation semantic encoding feature vector.
Optionally, the system operates a semantic interaction unit for:
and performing feature interaction based on an attention mechanism on the first system operation semantic coding feature vector and the second system operation semantic coding feature vector by using an inter-feature attention layer to obtain the inter-system operation interaction feature vector.
Optionally, the system operation conflict detection module includes:
the feature distribution optimizing unit is used for respectively carrying out feature distribution optimization on the first system operation semantic coding feature vector and the second system operation semantic coding feature vector based on the interaction feature vector between the system operations so as to obtain an optimized first system operation semantic coding feature vector and an optimized second system operation semantic coding feature vector;
the inter-system operation interaction feature optimization unit is used for calculating an inter-system operation interaction feature vector based on the optimized first system operation semantic coding feature vector and the optimized second system operation semantic coding feature vector; and
and the conflict detection unit is used for enabling the interaction characteristic vector between the optimized system operation to pass through a classifier to obtain a classification result, and the classification result is used for indicating whether the first system operation and the second system operation have conflict or not.
Optionally, the feature distribution optimizing unit includes:
a weighting factor calculation subunit, configured to calculate quantized transferable sensing factors of transferable features of the first system operation semantic coding feature vector and the second system operation semantic coding feature vector based on the inter-system operation interaction feature vector, respectively, to obtain a first weighting factor and a second weighting factor; and
and the weighted optimization subunit is used for weighted optimization of the first system operation semantic coding feature vector and the second system operation semantic coding feature vector by taking the first weighted factor and the second weighted factor as weighted coefficients so as to obtain the optimized first system operation semantic coding feature vector and the optimized second system operation semantic coding feature vector.
Optionally, the weighting factor calculation subunit is configured to:
calculating quantized transferable sensing factors of transferable features of the first system operation semantic coding feature vector and the second system operation semantic coding feature vector respectively according to the following optimization formula based on the inter-system operation interaction feature vector to obtain the first weighting factor and the second weighting factor;
wherein, the optimization formula is:
wherein V is 1 、V 2 And V c The first system operation semantic coding feature vector, the second system operation semantic coding feature vector and the inter-system operation interaction feature vector are respectively f i Is the i-th position feature value of one feature vector of the first system operation semantic coding feature vector, the second system operation semantic coding feature vector and the inter-system operation interaction feature vector, log is a logarithmic function based on 2, alpha is a weighted super parameter, w 1 And w 2 The first weighting factor and the second weighting factor, respectively.
According to another aspect of the present application, there is provided a collaborative virtual maintenance training method, comprising:
acquiring a first system operation and a second system operation;
performing semantic feature interactive coding on the first system operation and the second system operation to obtain interactive features among the system operations; and
based on the inter-system operation interaction characteristics, it is determined whether a conflict exists between the first system operation and the second system operation.
Optionally, performing semantic feature interactive coding on the first system operation and the second system operation to obtain interactive features between the system operations, including:
carrying out semantic coding on the first system operation to obtain a first system operation semantic coding feature vector;
carrying out semantic coding on the second system operation to obtain a second system operation semantic coding feature vector; and
and carrying out feature interaction on the first system operation semantic coding feature vector and the second system operation semantic coding feature vector to obtain a system inter-operation interaction feature vector serving as the system inter-operation interaction feature.
Compared with the prior art, the collaborative virtual maintenance training system and method provided by the application have the advantages that the first system operation and the second system operation are firstly obtained, then semantic feature interactive coding is carried out on the first system operation and the second system operation to obtain interactive features between the system operations, and then whether the first system operation and the second system operation have conflict is determined based on the interactive features between the system operations. Therefore, the accuracy and the efficiency of conflict detection can be improved, so that the stability and the safety of the system are improved, and better training and practice environments are provided for maintenance personnel.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly introduced below, the following drawings not being drawn to scale with respect to actual dimensions, emphasis instead being placed upon illustrating the gist of the present application.
FIG. 1 is a block diagram schematic of a collaborative virtual maintenance training system according to an embodiment of the application;
FIG. 2 is a block diagram schematic of the system operational semantic understanding module in a collaborative virtual maintenance training system according to an embodiment of the present application;
FIG. 3 is a block diagram schematic of the system operation conflict detection module in a collaborative virtual maintenance training system according to an embodiment of the present application;
FIG. 4 is a block diagram schematic of the feature distribution optimization unit in a collaborative virtual maintenance training system according to an embodiment of the present application;
FIG. 5 is a flow chart of a collaborative virtual maintenance training method according to an embodiment of the application;
FIG. 6 is a schematic diagram of a system architecture of a collaborative virtual maintenance training method according to an embodiment of the application;
fig. 7 is an application scenario diagram of a collaborative virtual maintenance training system according to an embodiment of the application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some, but not all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are also within the scope of the application.
Hereinafter, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only some embodiments of the present application and not all embodiments of the present application, and it should be understood that the present application is not limited by the example embodiments described herein.
FIG. 1 is a block diagram of a collaborative virtual maintenance training system according to an embodiment of the application. As shown in fig. 1, a collaborative virtual maintenance training system 100 according to an embodiment of the application includes: a system operation acquisition module 110 for acquiring a first system operation and a second system operation; the system operation semantic understanding module 120 is configured to perform semantic feature interactive encoding on the first system operation and the second system operation to obtain interactive features between system operations; and a system operation conflict detection module 130, configured to determine whether a conflict exists between the first system operation and the second system operation based on the inter-system operation interaction feature.
Specifically, in the technical scheme of the application, first, a first system operation and a second system operation are acquired. Then, in order to extract semantic feature information about the first system operation and the second system operation, the first system operation is further subjected to semantic coding to obtain a first system operation semantic coding feature vector, and the second system operation is subjected to semantic coding to obtain a second system operation semantic coding feature vector. It should be understood that, in the technical solution of the present application, by performing semantic encoding on the first system operation and the second system operation, semantic key information and features related to the first system operation and the second system operation, such as global semantic association feature information including a target, an influence range, and an execution step of the system operation, can be sufficiently captured. In this way, the conflict or similarity between the two system operation semantic features can be further detected through subsequent interactive association of the two system operation semantic features, so as to help judge whether the potential problem of the conflict exists in the system operation.
Further, the attention mechanism-based feature interaction is performed on the first system operation semantic coding feature vector and the second system operation semantic coding feature vector by using an inter-feature attention layer to obtain a system operation interaction feature vector, so that the association and the interaction between the first system operation semantic feature and the second system operation semantic feature are captured. It should be appreciated that since the goal of the traditional attention mechanism is to learn an attention weight matrix, a greater weight is given to important features and a lesser weight is given to secondary features, thereby selecting more critical information to the current task goal. This approach is more focused on weighting the importance of individual features, while ignoring the dependency between features. The attention layer between the features can capture the correlation and the mutual influence between the first system operation semantic features and the second system operation semantic features through the feature interaction based on an attention mechanism, learn the dependency relationship between different features, and interact and integrate the features according to the dependency relationship, so that an interaction feature vector between the system operations is obtained.
Accordingly, as shown in fig. 2, the system operation semantic understanding module 120 includes: a first system operation semantic understanding unit 121, configured to perform semantic encoding on the first system operation to obtain a first system operation semantic encoding feature vector; a second system operation semantic understanding unit 122, configured to perform semantic encoding on the second system operation to obtain a second system operation semantic encoding feature vector; and a system operation semantic interaction unit 123, configured to perform feature interaction on the first system operation semantic coding feature vector and the second system operation semantic coding feature vector to obtain a system inter-operation interaction feature vector as the system inter-operation interaction feature.
More specifically, the first system operation semantic understanding unit 121 is configured to: passing the first system operation through a semantic encoder comprising an embedded layer to obtain the first system operation semantic coding feature vector. The second system operation semantic understanding unit 122 is configured to: and enabling the second system operation to pass through the semantic encoder comprising the embedded layer to obtain the second system operation semantic encoding feature vector. It should be appreciated that a semantic encoder comprising an embedded layer is a neural network structure for converting system operations into semantically encoded feature vectors. The embedded layer is part of the encoder that converts the input system operation into a low-dimensional dense vector representation, capturing semantic information of the system operation. Specifically, the semantic encoder containing the embedded layer accepts as input the system operation and maps it into a continuous vector space by learning. The goal of the embedding layer is to map similar system operations into adjacent vector space for feature interaction and comparison in the vector space. By using an embedded layer, system operations can be represented as vectors of fixed length, such vector representations having some semantic relevance. In this way, semantic similarity between system operations can be measured by calculating vector distances between them, thereby enabling semantic understanding and interaction of system operations. That is, a semantic encoder including an embedded layer is a neural network structure for converting system operations into semantically encoded feature vectors, which maps the system operations into a continuous vector space through learning to achieve semantic understanding and interaction of the system operations.
More specifically, the system operates the semantic interaction unit 123 for: and performing feature interaction based on an attention mechanism on the first system operation semantic coding feature vector and the second system operation semantic coding feature vector by using an inter-feature attention layer to obtain the inter-system operation interaction feature vector. It should be appreciated that the attention mechanism (Attention Mechanism) is a technique commonly used in machine learning and natural language processing to simulate human attention behavior that can help a model focus important information on the part of interest when processing sequence data, thereby improving the performance and behavior of the model. The role of the attention mechanism is to assign a weight or attention weight to each input based on the importance of the different parts of the input, so that the model can selectively focus on the parts related to the current task and ignore the parts not related when processing the sequence data. By using the attention mechanism, the model can dynamically adjust their contribution in generating the output based on the importance of each word in the input sequence, thereby better capturing the semantic and contextual information of the input sequence. The advantages of the attention mechanism include: 1. providing an interpretability of the input sequence, it can be appreciated that the model focuses on important information during processing; 2. the performance and generalization capability of the model are improved, so that the model can better process long sequences and complex semantic relations; 3. the perceptibility of the model to different parts of the input sequence is enhanced, and the accuracy and the robustness of the model are improved. In other words, the attention mechanism is a technology for simulating the attention behavior of human beings, plays an important role in the sequence data processing, and can help the model focus on the concerned part, thereby improving the performance and performance of the model.
And then, the interaction feature vector between the system operations passes through a classifier to obtain a classification result, wherein the classification result is used for indicating whether the first system operation and the second system operation have conflict. That is, the interactive correlation characteristic information between the first system operation semantic characteristic and the second system operation semantic characteristic is used for classifying, so that whether the two system operations conflict or not is detected, the stability and the safety of the system are improved, and better training and practice environments are provided for maintenance personnel.
Accordingly, as shown in fig. 3, the system operation conflict detection module 130 includes: the feature distribution optimizing unit 131 is configured to perform feature distribution optimization on the first system operation semantic coding feature vector and the second system operation semantic coding feature vector based on the inter-system operation interaction feature vector, so as to obtain an optimized first system operation semantic coding feature vector and an optimized second system operation semantic coding feature vector; a system inter-operation interaction feature optimization unit 132, configured to calculate an inter-operation interaction feature vector of the optimized system based on the optimized first system operation semantic coding feature vector and the optimized second system operation semantic coding feature vector; and a conflict detection unit 133, configured to pass the interaction feature vector between the optimized system operations through a classifier to obtain a classification result, where the classification result is used to indicate whether there is a conflict between the first system operation and the second system operation.
More specifically, as shown in fig. 4, the feature distribution optimizing unit 131 includes: a weighting factor calculation subunit 1311, configured to calculate quantized transferable sensing factors of transferable features of the first system operation semantic coding feature vector and the second system operation semantic coding feature vector based on the inter-system operation interaction feature vector, respectively, to obtain a first weighting factor and a second weighting factor; and a weighted optimization subunit 1312, configured to perform weighted optimization on the first system operation semantic coding feature vector and the second system operation semantic coding feature vector by using the first weighting factor and the second weighting factor as weighting coefficients, so as to obtain the optimized first system operation semantic coding feature vector and the optimized second system operation semantic coding feature vector.
In particular, in the technical solution of the present application, the first system operation semantic coding feature vector and the second system operation semantic coding feature vector express the context semantic association feature representations of the first system operation and the second system operation, respectively, so that the first system operation semantic coding feature vector and the second system operation semantic coding feature vector have more significant feature distribution differences considering that the source data differences are amplified through semantic feature extraction of a semantic encoder, so that dependency relationships between feature information are extracted using an attention mechanism to obtain the inter-system operation interaction feature directionsWhen the method is used, the first system operation semantic coding feature vector and the wind speed time sequence feature vector also have feature distribution domain transfer difference of the cross-semantic distribution difference of the travel resistance time sequence feature vector, so that the feature interactive fusion effect of the attention mechanism is influenced, and the expression effect of the travel resistance time sequence feature vector is also influenced. Based on this, the applicant of the present application operates on the first system operation semantically encoded feature vector, e.g., denoted as V 1 And said second system operation semantic coding feature vector, e.g. denoted V 2 And the inter-system-operation interaction feature vector, e.g., denoted as V c A quantized transferable sensing factor of its transferable characteristics is calculated.
Accordingly, in one specific example, the weighting factor calculation subunit 1311 is configured to: calculating quantized transferable sensing factors of transferable features of the first system operation semantic coding feature vector and the second system operation semantic coding feature vector respectively according to the following optimization formula based on the inter-system operation interaction feature vector to obtain the first weighting factor and the second weighting factor; wherein, the optimization formula is:
wherein V is 1 、V 2 And V c The first system operation semantic coding feature vector, the second system operation semantic coding feature vector and the inter-system operation interaction feature vector are respectively f i Is the i-th position feature value of one feature vector of the first system operation semantic coding feature vector, the second system operation semantic coding feature vector and the inter-system operation interaction feature vector, log is a logarithmic function based on 2, alpha is a weighted super parameter, w 1 And w 2 The first weighting factor and the second weighting factor, respectively.
The quantized transferable sensing factors of the transferable features are used for respectively estimating the domain uncertainty from the feature space domain to the classification target domain through the uncertainty measurement under the domain transfer, and the domain uncertainty estimation can be used for identifying feature representations transferred between domains, so that the feature interactive fusion is carried out by weighting the first system operation semantic coding feature vector and the second system operation semantic coding feature vector by taking the factors as weights and then using an attention mechanism, whether feature mapping is effectively transferred between domains or not can be identified through the cross-domain alignment from the feature space domain to the classification target domain, and the transferable property of transferable features in the first system operation semantic coding feature vector and the second system operation semantic coding feature vector is quantitatively perceived, so that the feature interactive fusion of inter-domain self-adaption is realized, and the expression effect of the system operation interactive feature vector is improved. Therefore, the system operation conflict detection can be automatically carried out based on actual conditions, so that the accuracy and the efficiency of the conflict detection are improved, the stability and the safety of the system are improved, and better training and practice environments are provided for maintenance personnel.
More specifically, the collision detection unit 133 is further configured to: performing full-connection coding on the interaction feature vector between the operation of the optimizing system by using a full-connection layer of the classifier to obtain a coding classification feature vector; and inputting the coding classification feature vector into a Softmax classification function of the classifier to obtain the classification result.
That is, in the technical solution of the present disclosure, the labels of the classifier include that there is a conflict between the first system operation and the second system operation (first label), and that there is no conflict between the first system operation and the second system operation (second label), where the classifier determines, through a soft maximum function, to which classification label the interaction feature vector between the optimized system operations belongs. It should be noted that the first tag p1 and the second tag p2 do not include a manually set concept, and in fact, during the training process, the computer model does not have a concept of "whether there is a conflict between the first system operation and the second system operation", which is simply that there are two kinds of classification tags and the probability that the output feature is under the two kinds of classification tags, that is, the sum of p1 and p2 is one. Therefore, the classification result of whether the first system operation and the second system operation have conflict is actually converted into the classification probability distribution conforming to the two classifications of the natural law through classifying the labels, and the physical meaning of the natural probability distribution of the labels is essentially used instead of the language text meaning of whether the first system operation and the second system operation have conflict.
It should be appreciated that the role of the classifier is to learn the classification rules and classifier using a given class, known training data, and then classify (or predict) the unknown data. Logistic regression (logistics), SVM, etc. are commonly used to solve the classification problem, and for multi-classification problems (multi-class classification), logistic regression or SVM can be used as well, but multiple bi-classifications are required to compose multiple classifications, but this is error-prone and inefficient, and the commonly used multi-classification method is the Softmax classification function.
It should be appreciated that the fully connected layer (Fully Connected Layer), also known as dense or fully connected layer, is a common layer type in deep neural networks, and its main function is to multiply the input eigenvectors with a weight matrix and perform a nonlinear transformation by an activation function to generate an output eigenvalue. The structure of the full connection layer comprises: 1. input: a feature vector, typically represented as a one-dimensional array; 2. weight matrix: consisting of a learnable parameter for multiplying the input features with the connection weights of each neuron; 3. bias term: each neuron has a bias term for introducing an offset, which increases the flexibility of the model; 4. activation function: and carrying out nonlinear mapping on the result of the linear transformation, and introducing nonlinear capability. The fully connected layer is used for performing linear transformation and nonlinear mapping on the input feature vector so as to extract a higher-level feature representation. By learning appropriate weights and biasing terms, the fully connected layer can perform complex feature extraction and combination on the input, helping the model to better understand the structure and relationship of the input data. In the conflict detection unit, a full-join layer is used to full-join encode the inter-operation interaction feature vector of the optimization system. Specifically, the full-connection layer multiplies the interaction feature vector between the operation of the optimization system by the weight matrix, and performs nonlinear transformation through an activation function to obtain the coding classification feature vector. This encoded classification feature vector may be input into the Softmax classification function of the classifier to obtain the final classification result. Through the function of the full connection layer, the model can extract more abstract and expressive feature representation from interaction features between operation of an optimized system, so that the classification accuracy and performance are improved.
In summary, the collaborative virtual maintenance training system 100 according to the embodiment of the present application is illustrated, which can perform semantic interactive association coding on two system operations after the two system operations are collected, so as to automatically perform conflict detection on the two system operations.
As described above, the collaborative virtual maintenance training system 100 according to the embodiment of the present application may be implemented in various terminal devices, for example, a server or the like having a collaborative virtual maintenance training algorithm according to the embodiment of the present application. In one example, collaborative virtual maintenance training system 100 in accordance with embodiments of the present application may be integrated into a terminal device as a software module and/or hardware module. For example, the collaborative virtual maintenance training system 100 according to embodiments of the present application may be a software module in the operating system of the terminal device, or may be an application developed for the terminal device; of course, the collaborative virtual maintenance training system 100 according to an embodiment of the present application may also be one of a plurality of hardware modules of the terminal device.
Alternatively, in another example, the collaborative virtual maintenance training system 100 and the terminal device according to the embodiment of the present application may be separate devices, and the collaborative virtual maintenance training system 100 may be connected to the terminal device through a wired and/or wireless network and transmit interactive information according to a agreed data format.
Fig. 5 is a flow chart of a collaborative virtual maintenance training method according to an embodiment of the application. Fig. 6 is a schematic diagram of a system architecture of a collaborative virtual maintenance training method according to an embodiment of the application. As shown in fig. 5 and 6, a collaborative virtual maintenance training method according to an embodiment of the present application includes: s110, acquiring a first system operation and a second system operation; s120, carrying out semantic feature interactive coding on the first system operation and the second system operation to obtain interactive features among the system operations; and S130, determining whether a conflict exists between the first system operation and the second system operation based on the interaction characteristics between the system operations.
In a specific example, in the collaborative virtual maintenance training method, performing semantic feature interactive coding on the first system operation and the second system operation to obtain inter-system operation interactive features includes: carrying out semantic coding on the first system operation to obtain a first system operation semantic coding feature vector; carrying out semantic coding on the second system operation to obtain a second system operation semantic coding feature vector; and performing feature interaction on the first system operation semantic coding feature vector and the second system operation semantic coding feature vector to obtain a system inter-operation interaction feature vector serving as the system inter-operation interaction feature.
Here, it will be appreciated by those skilled in the art that the specific operations of the respective steps in the above-described collaborative virtual maintenance training method have been described in detail in the above description of the collaborative virtual maintenance training system 100 with reference to fig. 1 to 4, and thus, repetitive descriptions thereof will be omitted.
Fig. 7 is an application scenario diagram of a collaborative virtual maintenance training system according to an embodiment of the application. As shown in fig. 7, in this application scenario, first, a first system operation (e.g., D1 illustrated in fig. 7) and a second system operation (e.g., D2 illustrated in fig. 7) are acquired, and then, the first system operation and the second system operation are input into a server (e.g., S illustrated in fig. 7) deployed with a collaborative virtual maintenance training algorithm, wherein the server is capable of processing the first system operation and the second system operation using the collaborative virtual maintenance training algorithm to obtain a classification result for indicating whether there is a conflict between the first system operation and the second system operation.
It should be understood that the collaborative virtual maintenance training software takes a mature J2EE platform as a development framework, adopts an SOA technology and a 3D front-end display technology, and the front-end display adopts a virtual reality technology, so that visual experience is provided for trained personnel; and constructing a system-level maintenance training evaluation platform by utilizing technologies such as 3D modeling, virtual reality, large space positioning, multi-person cooperation, motion capture and the like. The functional modules related to the system comprise a system management module, a simulation operation module, a simulation detection module, a simulation maintenance module and an assessment module. The technical problems of concurrency conflict control, consistency realization and collaborative perception are mainly solved by collaborative virtual maintenance.
Wherein, concurrency conflict control can be resolved from three aspects:
1) Collision avoidance. The specific method for limiting the user operation by a certain rule so that the user operation cannot conflict is as follows: 1. concurrent control based on locks. The user needs to apply before operation, and if the opposite side does not release the lock, the current user is in a long-term hunger state. The mechanism has low responsiveness and poor real-time interactivity for users. 2. Optimistic concurrency control. The transaction is allowed to execute unimpeded until all operations are completed, then validated at commit time, committed if validated, and restarted otherwise. This mechanism tolerates temporary collisions, handling collisions centrally, and restarting is too costly. 3. And (5) performing time scale concurrency control. Each transaction is assigned 1 time scale, and the execution sequence of the transactions is determined by the size of the time scale. This mechanism has a good degree of concurrency. 4. Token mechanism. At the same time, only one user can acquire the access token, and other collaborators can only wait for the release of the current access token. It is a simple exclusive method that affects the user's collaboration. Collisions are an essential phenomenon in the collaborative process. The adoption of various collision avoidance techniques and means can only reduce and avoid a certain number of collisions of a certain type to a certain extent, but cannot completely eliminate the collisions.
2) And (5) conflict detection. And carrying out conflict judgment on concurrent operation in the system through a certain judgment rule. The use of collision avoidance means can only reduce, but cannot completely eliminate, collisions. In order to find the conflict early, resolve the conflict as early as possible in the primary stage, avoid invalid work and huge reworking amount caused by the propagation of the conflict, influence the cooperative work quality and efficiency, and detect the conflict in time by adopting effective technical means when the conflict is already potential and about to happen or happens but has not propagated further, and correspondingly make correct processing, the specific method is as follows: 1. collision detection based on Petri net. 2. Collision detection based on true values. 3. Constraint-based conflict detection. 4. Collision detection based on heuristic classification.
3) Conflict resolution. And coordinating the conflicting operations according to a certain rule. When the system detects the occurrence of a conflict, corresponding countermeasures and suggestions must be provided to the co-members to resolve the conflict, depending on the nature, form and content of the conflict generated. The method comprises the following steps: 1. and (5) backtracking. After the conflict occurs, the process rolls back forward until the conflict operation is canceled, but it is difficult to determine a reasonable backtracking span. 2. The constraint relaxes. This approach is a compromise to achieve conflict resolution at the cost of target modification. 3. Arbitration and negotiation resolution. All work must be suspended before conflict resolution, whether or not a user is engaged in a conflict operation with the current, severely affecting the continuity of collaboration. 4. The mechanism well maintains the operation intention of a cooperator, but with the gradual deep operation, the problem that the versions are more and are difficult to control can occur.
Regarding consistency implementation issues, in CVM systems, an operation object is typically replicated to the local end of each member. Thus, each member operates the local copy, thereby realizing operation localization, improving the response time of the system, and then sending the operation result to other remote users. Under ideal network environment, each node in the system can receive the operation sent by other nodes in time, and the operation is executed at each node according to the correct order, so that each node in the system can maintain the state consistency. However, in the practical system, there is a contradiction between huge traffic and limited network bandwidth, network congestion, delay and packet loss are unavoidable, so that the time points of receiving information by each node are different, the same operation is performed at different time points, and multiple nodes are in inconsistent states. Such inconsistent conditions can severely impact system usage and even cause the system to fail in a significant number of errors.
Regarding the problem of collaborative awareness, common methods for supporting collaborative awareness are: 1) Video tools. It is common for 2 or more participants to capture video information via a computer and transmit it to the other participants. 2) A speech tool. Any member in the group can recognize who is speaking through sound, sense emotion, mood, tone and the like of a speaker, enhance the cooperative perception among the collaborators and improve the working efficiency among the collaborators. 3) The whiteboard tool simply simulates a physical whiteboard or blackboard on a computer screen, each participant can draw a figure on the whiteboard, input characters, and support ideological communication and collaborative work among the group members. 4) The remote pointer captures the native mouse information, passes the information to the remote machine, and simulates the operation of the mouse in software on the remote machine. 5) Color identification. Commonly found in collaborative editing systems, each collaborator's operating section is assigned a different color. 6) A multi-user scroll bar. The right part is a common vertical scroll bar, each member can be used to manipulate its own view; the left part is 1 or more light bars in block to identify different active locations of all present users, different members being represented by different color bars.
Further, system management software is used to manage user identity information, usage information, and set maintenance faults. The information comprises user name, password, user type, maintenance training time, maintenance training content, operation training time, operation training content, examination result and the like. Statistics, queries, etc. may be performed on the information. The software consists of a registration login module, a recording module, a statistics inquiry module, a maintenance fault setting module and other submodules. The system is provided with a plurality of databases (tool library, knowledge library, old subject library, etc.).
Furthermore, those skilled in the art will appreciate that the various aspects of the application are illustrated and described in the context of a number of patentable categories or circumstances, including any novel and useful procedures, machines, products, or materials, or any novel and useful modifications thereof. Accordingly, aspects of the application may be performed entirely by hardware, entirely by software (including firmware, resident software, micro-code, etc.) or by a combination of hardware and software. The above hardware or software may be referred to as a "data block," module, "" engine, "" unit, "" component, "or" system. Furthermore, aspects of the application may take the form of a computer product, comprising computer-readable program code, embodied in one or more computer-readable media.
The foregoing is illustrative of the present application and is not to be construed as limiting thereof. Although a few exemplary embodiments of this application have been described, those skilled in the art will readily appreciate that many modifications are possible in the exemplary embodiments without materially departing from the novel teachings and advantages of this application. Accordingly, all such modifications are intended to be included within the scope of this application as defined in the following claims. It is to be understood that the foregoing is illustrative of the present application and is not to be construed as limited to the specific embodiments disclosed, and that modifications to the disclosed embodiments, as well as other embodiments, are intended to be included within the scope of the appended claims. The application is defined by the claims and their equivalents.
Claims (10)
1. A collaborative virtual maintenance training system, comprising:
the system operation acquisition module is used for acquiring a first system operation and a second system operation;
the system operation semantic understanding module is used for carrying out semantic feature interactive coding on the first system operation and the second system operation so as to obtain interactive features between the system operations; and
and the system operation conflict detection module is used for determining whether conflict exists between the first system operation and the second system operation based on the interaction characteristics between the system operations.
2. The collaborative virtual maintenance training system of claim 1, wherein the system operates a semantic understanding module comprising:
the first system operation semantic understanding unit is used for carrying out semantic coding on the first system operation to obtain a first system operation semantic coding feature vector;
the second system operation semantic understanding unit is used for carrying out semantic coding on the second system operation to obtain a second system operation semantic coding feature vector; and
and the system operation semantic interaction unit is used for carrying out feature interaction on the first system operation semantic coding feature vector and the second system operation semantic coding feature vector to obtain a system operation interaction feature vector serving as the system operation interaction feature.
3. The collaborative virtual maintenance training system of claim 2, wherein the first system operation semantic understanding unit is configured to:
passing the first system operation through a semantic encoder comprising an embedded layer to obtain the first system operation semantic coding feature vector.
4. A collaborative virtual maintenance training system according to claim 3, wherein the second system operates a semantic understanding unit to:
and enabling the second system operation to pass through the semantic encoder comprising the embedded layer to obtain the second system operation semantic encoding feature vector.
5. The collaborative virtual maintenance training system of claim 4, wherein the system operates a semantic interaction unit to:
and performing feature interaction based on an attention mechanism on the first system operation semantic coding feature vector and the second system operation semantic coding feature vector by using an inter-feature attention layer to obtain the inter-system operation interaction feature vector.
6. The collaborative virtual maintenance training system of claim 5, wherein the system operation conflict detection module comprises:
the feature distribution optimizing unit is used for respectively carrying out feature distribution optimization on the first system operation semantic coding feature vector and the second system operation semantic coding feature vector based on the interaction feature vector between the system operations so as to obtain an optimized first system operation semantic coding feature vector and an optimized second system operation semantic coding feature vector;
the inter-system operation interaction feature optimization unit is used for calculating an inter-system operation interaction feature vector based on the optimized first system operation semantic coding feature vector and the optimized second system operation semantic coding feature vector; and
and the conflict detection unit is used for enabling the interaction characteristic vector between the optimized system operation to pass through a classifier to obtain a classification result, and the classification result is used for indicating whether the first system operation and the second system operation have conflict or not.
7. The collaborative virtual maintenance training system of claim 6, wherein the feature distribution optimization unit comprises:
a weighting factor calculation subunit, configured to calculate quantized transferable sensing factors of transferable features of the first system operation semantic coding feature vector and the second system operation semantic coding feature vector based on the inter-system operation interaction feature vector, respectively, to obtain a first weighting factor and a second weighting factor; and
and the weighted optimization subunit is used for weighted optimization of the first system operation semantic coding feature vector and the second system operation semantic coding feature vector by taking the first weighted factor and the second weighted factor as weighted coefficients so as to obtain the optimized first system operation semantic coding feature vector and the optimized second system operation semantic coding feature vector.
8. The collaborative virtual maintenance training system of claim 7, wherein the weighting factor calculation subunit is configured to:
calculating quantized transferable sensing factors of transferable features of the first system operation semantic coding feature vector and the second system operation semantic coding feature vector respectively according to the following optimization formula based on the inter-system operation interaction feature vector to obtain the first weighting factor and the second weighting factor;
wherein, the optimization formula is:
wherein V is 1 、V 2 And V c The first system operation semantic coding feature vector, the second system operation semantic coding feature vector and the inter-system operation interaction feature vector are respectively f i Is the i-th position feature value of one feature vector of the first system operation semantic coding feature vector, the second system operation semantic coding feature vector and the inter-system operation interaction feature vector, log is a logarithmic function based on 2, alpha is a weighted super parameter, w 1 And w 2 The first weighting factor and the second weighting factor, respectively.
9. A collaborative virtual maintenance training method, comprising:
acquiring a first system operation and a second system operation;
performing semantic feature interactive coding on the first system operation and the second system operation to obtain interactive features among the system operations; and
based on the inter-system operation interaction characteristics, it is determined whether a conflict exists between the first system operation and the second system operation.
10. The collaborative virtual maintenance training method of claim 9, wherein semantically encoding the first system operation and the second system operation to obtain inter-system-operation interaction features comprises:
carrying out semantic coding on the first system operation to obtain a first system operation semantic coding feature vector;
carrying out semantic coding on the second system operation to obtain a second system operation semantic coding feature vector; and
and carrying out feature interaction on the first system operation semantic coding feature vector and the second system operation semantic coding feature vector to obtain a system inter-operation interaction feature vector serving as the system inter-operation interaction feature.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310935654.2A CN116881017B (en) | 2023-07-27 | 2023-07-27 | Collaborative virtual maintenance training system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310935654.2A CN116881017B (en) | 2023-07-27 | 2023-07-27 | Collaborative virtual maintenance training system and method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116881017A true CN116881017A (en) | 2023-10-13 |
| CN116881017B CN116881017B (en) | 2024-05-28 |
Family
ID=88260292
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310935654.2A Active CN116881017B (en) | 2023-07-27 | 2023-07-27 | Collaborative virtual maintenance training system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116881017B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117593934A (en) * | 2024-01-17 | 2024-02-23 | 长春职业技术学院 | Virtual simulation training system based on industrial robot |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| RU2239233C2 (en) * | 2002-10-01 | 2004-10-27 | Гинзбург Виталий Вениаминович | Method for forming and optimizing of model for system complex |
| CN101441677A (en) * | 2008-12-25 | 2009-05-27 | 上海交通大学 | Natural interactive virtual assembly system based on product full semantic model |
| CN106774949A (en) * | 2017-03-09 | 2017-05-31 | 北京神州四达科技有限公司 | Collaborative simulation exchange method, device and system |
| CN107748983A (en) * | 2017-10-23 | 2018-03-02 | 云南大学 | Method is detected and corrected in a kind of compatibility of cooperation service process |
| CN109343841A (en) * | 2018-10-15 | 2019-02-15 | 上海理工大学 | Semantic conflict digestion procedure under real-time collaborative programmed environment |
| CN116151270A (en) * | 2023-02-23 | 2023-05-23 | 重庆赛力斯新能源汽车设计院有限公司 | Parking test system and method |
| CN116204266A (en) * | 2023-05-04 | 2023-06-02 | 深圳市联合信息技术有限公司 | Remote assisted information creation operation and maintenance system and method thereof |
| CN116419293A (en) * | 2021-12-29 | 2023-07-11 | 华为技术有限公司 | Method and device for processing intent conflict |
-
2023
- 2023-07-27 CN CN202310935654.2A patent/CN116881017B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| RU2239233C2 (en) * | 2002-10-01 | 2004-10-27 | Гинзбург Виталий Вениаминович | Method for forming and optimizing of model for system complex |
| CN101441677A (en) * | 2008-12-25 | 2009-05-27 | 上海交通大学 | Natural interactive virtual assembly system based on product full semantic model |
| CN106774949A (en) * | 2017-03-09 | 2017-05-31 | 北京神州四达科技有限公司 | Collaborative simulation exchange method, device and system |
| CN107748983A (en) * | 2017-10-23 | 2018-03-02 | 云南大学 | Method is detected and corrected in a kind of compatibility of cooperation service process |
| CN109343841A (en) * | 2018-10-15 | 2019-02-15 | 上海理工大学 | Semantic conflict digestion procedure under real-time collaborative programmed environment |
| CN116419293A (en) * | 2021-12-29 | 2023-07-11 | 华为技术有限公司 | Method and device for processing intent conflict |
| CN116151270A (en) * | 2023-02-23 | 2023-05-23 | 重庆赛力斯新能源汽车设计院有限公司 | Parking test system and method |
| CN116204266A (en) * | 2023-05-04 | 2023-06-02 | 深圳市联合信息技术有限公司 | Remote assisted information creation operation and maintenance system and method thereof |
Non-Patent Citations (2)
| Title |
|---|
| 李忠霖: "基于特征依赖关系的协同设计并发控制研究", 《中国优秀硕士学位论文全文数据库信息科技辑（月刊）》, no. 02, 15 August 2007 (2007-08-15), pages 1 * |
| 欧立铭等: "协同虚拟维修及其关键技术", 《舰船科学技术》, vol. 32, no. 11, 30 November 2010 (2010-11-30), pages 122 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117593934A (en) * | 2024-01-17 | 2024-02-23 | 长春职业技术学院 | Virtual simulation training system based on industrial robot |
| CN117593934B (en) * | 2024-01-17 | 2024-04-05 | 长春职业技术学院 | Virtual simulation training system based on industrial robot |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116881017B (en) | 2024-05-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2023115761A1 (en) | Event detection method and apparatus based on temporal knowledge graph | |
| CN111506722A (en) | Knowledge graph question-answering method, device and equipment based on deep learning technology | |
| CN110598070B (en) | Application type identification method and device, server and storage medium | |
| CN112257441B (en) | Named entity recognition enhancement method based on counterfactual generation | |
| JPH07295989A (en) | Device that forms interpreter to analyze data | |
| CN112988963B (en) | User intention prediction method, device, equipment and medium based on multi-flow nodes | |
| CN116881017B (en) | Collaborative virtual maintenance training system and method | |
| CN113628059A (en) | Associated user identification method and device based on multilayer graph attention network | |
| CN111666766A (en) | Data processing method, device and equipment | |
| CN116415581A (en) | Teaching data analysis system based on intelligent education | |
| CN114999006A (en) | Multi-modal emotion analysis method, device and equipment based on uncertainty estimation | |
| CN113128196A (en) | Text information processing method and device, storage medium | |
| CN117333146A (en) | Manpower resource management system and method based on artificial intelligence | |
| CN111159241B (en) | Click conversion estimation method and device | |
| CN115510188A (en) | Text keyword association method, device, equipment and storage medium | |
| CN114676705B (en) | Dialogue relation processing method, computer and readable storage medium | |
| CN117520209B (en) | Code review method, device, computer equipment and storage medium | |
| CN115222443A (en) | Client group division method, device, equipment and storage medium | |
| Asha et al. | Artificial Neural Networks based DIGI Writing | |
| CN114048314A (en) | Natural language steganalysis method | |
| Zheng et al. | DJMF: A discriminative joint multi-task framework for multimodal sentiment analysis based on intra-and inter-task dynamics | |
| Song | [Retracted] An Evaluation Method of English Teaching Ability Based on Deep Learning | |
| KR20220066554A (en) | Method, apparatus and computer program for buildding knowledge graph using qa model | |
| Pan et al. | Quality-aware clip for blind image quality assessment | |
| CN118035800A (en) | Model training method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |