CN116698283A

CN116698283A - Intelligent monitoring system and method for load gravity center deviation

Info

Publication number: CN116698283A
Application number: CN202310684287.3A
Authority: CN
Inventors: 戴肖肖; 张汉章; 施彬华; 张建东; 陈春喜; 蒋连杰; 杨帆
Original assignee: Zhejiang Jialift Warehouse Equipment Co ltd
Current assignee: Zhejiang Jialift Warehouse Equipment Co ltd
Priority date: 2023-06-09
Filing date: 2023-06-09
Publication date: 2023-09-05

Abstract

The application relates to the field of intelligent monitoring, and particularly discloses an intelligent monitoring system and method for load gravity center deviation, which are used for judging directly from analog signals as input by utilizing deep learning and artificial intelligence technology without analog-digital conversion, so that errors caused by signal processing are avoided. By the method, the load of the motor vehicle is monitored and judged in real time, and potential threat of overload to road traffic safety is avoided.

Description

Intelligent monitoring system and method for load gravity center deviation

Technical Field

The application relates to the field of intelligent monitoring, in particular to an intelligent monitoring system and method for load gravity center deviation.

Background

At present, a static wagon balance detection method is mainly adopted for overload of the motor vehicle, but the wagon balance set by the method has higher cost and is influenced by the geographical position of arrangement, and the motor vehicle can only be detected after driving to the position where the wagon balance is set.

In this regard, CN 102566542B provides a vehicle load safety monitoring system and method to solve the above-mentioned problems. In the technical solution of the patent, an analog signal of a vehicle body load is acquired by installing a sensor, the perceived analog signal is amplified, filtered and digitally converted, and whether the offset of the load exceeds a safety standard is judged based on the comparison between the obtained vehicle-mounted load digital value and a predetermined threshold value. However, the acquired analog signal has a lot of noise, and the quality of signal processing can affect the accuracy of judgment. Thus, an optimized solution is desired.

Disclosure of Invention

The present application has been made to solve the above-mentioned technical problems. The embodiment of the application provides an intelligent monitoring system and method for load gravity center deviation, which directly judges from analog signals as input by utilizing deep learning and artificial intelligence technology without analog-to-digital conversion, so that errors caused by signal processing are avoided. By the method, the load of the motor vehicle is monitored and judged in real time, and potential threat of overload to road traffic safety is avoided.

According to one aspect of the present application, there is provided an intelligent monitoring system for load center of gravity shifting, comprising:

the analog signal acquisition unit is used for acquiring a vehicle body load analog signal acquired by the sensor;

the sliding window sampling unit is used for sampling the sliding window based on the sampling window on the vehicle body load analog signal to obtain a plurality of vehicle body load analog sampling windows;

the characteristic filtering unit is used for respectively passing the plurality of vehicle body load simulation sampling windows through a convolutional neural network model serving as a filter to obtain waveform characteristic vectors of the plurality of vehicle body load simulation sampling windows;

the context global feature extraction unit is used for enabling the waveform feature vectors of the plurality of vehicle body load simulation sampling windows to pass through a context encoder based on a converter so as to obtain a vehicle body load simulation waveform global feature vector;

The multi-scale local feature extraction unit is used for arranging the waveform feature vectors of the plurality of vehicle body load simulation sampling windows into one-dimensional feature vectors and obtaining vehicle body load simulation waveform local association feature vectors through a local association feature extractor comprising a first convolution layer and a second convolution layer;

the fusion unit is used for fusing the global characteristic vector of the vehicle body load simulation waveform and the local association characteristic vector of the vehicle body load simulation waveform to obtain a classification characteristic vector; and

and the monitoring result generating unit is used for enabling the classification feature vector to pass through a classifier to obtain a classification result, and the classification result is used for indicating whether the load deviation exceeds a preset safety range or not.

In the intelligent monitoring system for load barycenter shift, the convolutional neural network model used as the filter comprises an input layer, a first convolutional layer, a first activation function, a first pooling layer, a second convolutional layer, a second activation function, a second pooling layer, a full connection layer, a third activation function and an output layer.

In the intelligent monitoring system for load center of gravity offset, the first convolution layer uses 16 convolution kernels with the size of 3x3, the stride is 1, the second convolution layer uses 32 convolution kernels with the size of 3x3, the stride is 1, the first activation function, the second activation function and the third activation function use ReLU, the first pooling layer and the second pooling layer all use maximum pooling, the pooling size is 2x2, and the stride is 2.

In the above intelligent monitoring system for load center of gravity shift, the context global feature extraction unit includes: the query vector construction subunit is used for carrying out one-dimensional arrangement on the waveform characteristic vectors of the plurality of vehicle body load simulation sampling windows to obtain the waveform characteristic vectors of the global vehicle body load simulation sampling windows; a self-attention subunit, configured to calculate a product between the global bodywork load analog sampling window waveform feature vector and a transpose vector of each bodywork load analog sampling window waveform feature vector in the plurality of bodywork load analog sampling window waveform feature vectors to obtain a plurality of self-attention correlation matrices; the normalization subunit is used for respectively performing normalization processing on each self-attention correlation matrix in the plurality of self-attention correlation matrices to obtain a plurality of normalized self-attention correlation matrices; the attention calculating subunit is used for obtaining a plurality of probability values through a Softmax classification function by each normalized self-attention correlation matrix in the normalized self-attention correlation matrices; the attention applying subunit is configured to weight each vehicle body load simulation sampling window waveform feature vector in the plurality of vehicle body load simulation sampling window waveform feature vectors by using each probability value in the plurality of probability values as a weight, so as to obtain the plurality of upper and lower Wen Yuyi vehicle body load simulation sampling window waveform feature vectors; and the cascading subunit is used for cascading the waveform characteristic vectors of the plurality of upper and lower Wen Yuyi vehicle body load simulation sampling windows to obtain the vehicle body load simulation waveform global characteristic vector.

In the above-mentioned intelligent monitoring system for load center of gravity shifting, the multi-scale local feature extraction unit includes: a first scale local feature extraction subunit, configured to input the one-dimensional feature vector into a first convolution layer of the local correlation feature extractor to obtain a first scale vehicle body load analog waveform local correlation feature vector, where the first convolution layer has a first one-dimensional convolution kernel with a first length; a second scale local feature extraction subunit, configured to input the one-dimensional feature vector into a second convolution layer of the local correlation feature extractor to obtain a second scale vehicle body load analog waveform local correlation feature vector, where the second convolution layer has a second one-dimensional convolution kernel with a second length, and the first length is different from the second length; and the multi-scale cascading subunit is used for cascading the local association characteristic vector of the vehicle body load simulation waveform and the local association characteristic vector of the vehicle body load simulation waveform to obtain the local association characteristic vector of the vehicle body load simulation waveform. Wherein the first scale local feature extraction subunit is configured to: performing one-dimensional convolution coding on the one-dimensional feature vector by using a first convolution layer of the local correlation feature extractor according to the following one-dimensional convolution formula to obtain a first-scale vehicle body load simulation waveform local correlation feature vector; wherein, the formula is:

Wherein a is the width of a first convolution kernel in the X direction, F (a) is a first convolution kernel parameter vector, G (X-a) is a local vector matrix operated with a convolution kernel function, w is the size of a first one-dimensional convolution kernel, X represents the one-dimensional feature vector, and Cov (X) represents one-dimensional convolution encoding of the one-dimensional feature vector; and, the second scale local feature extraction subunit is configured to: performing one-dimensional convolution coding on the one-dimensional feature vector by using a second convolution layer of the local correlation feature extractor according to the following one-dimensional convolution formula to obtain a second-scale vehicle body load simulation waveform local correlation feature vector; wherein, the formula is:

wherein b is the width of the second convolution kernel in the X direction, F (b) is a second convolution kernel parameter vector, G (X-b) is a local vector matrix operated with a convolution kernel function, m is the size of the second one-dimensional convolution kernel, X represents the one-dimensional feature vector, and Cov (X) represents one-dimensional convolution encoding of the one-dimensional feature vector.

In the above-mentioned intelligent monitoring system for load center of gravity shift, the monitoring result generating unit includes: a full-connection coding subunit, configured to perform full-connection coding on the classification feature vector by using multiple full-connection layers of the classifier to obtain a coded classification feature vector; and a classification result generation subunit, configured to pass the encoded classification feature vector through a Softmax classification function of the classifier to obtain the classification result.

The intelligent monitoring system for load barycenter offset further comprises a training module for training the convolutional neural network model used as a filter, the context encoder based on a converter, the local correlation feature extractor comprising a first convolution layer and a second convolution layer and the classifier.

In the above-mentioned intelligent monitoring system of load focus skew, training module includes: a training data acquisition unit configured to acquire training data including a training vehicle body load analog signal acquired by a sensor, and a true value of whether a load offset exceeds a predetermined safety range; the training sliding window sampling unit is used for sampling the sliding window based on the training vehicle body load analog signal to obtain a plurality of training vehicle body load analog sampling windows; the training feature filtering unit is used for enabling the plurality of training vehicle body load simulation sampling windows to respectively pass through the convolutional neural network model serving as a filter so as to obtain waveform feature vectors of the plurality of training vehicle body load simulation sampling windows; the training context global feature extraction unit is used for enabling the plurality of training vehicle body load simulation sampling window waveform feature vectors to pass through the context encoder based on the converter so as to obtain training vehicle body load simulation waveform global feature vectors; the training multi-scale local feature extraction unit is used for arranging the waveform feature vectors of the plurality of training vehicle body load simulation sampling windows into one-dimensional feature vectors and then obtaining training vehicle body load simulation waveform local association feature vectors through the local association feature extractor comprising the first convolution layer and the second convolution layer; the training fusion unit is used for fusing the training vehicle body load simulation waveform global feature vector and the training vehicle body load simulation waveform local association feature vector to obtain a training classification feature vector; the classification loss unit is used for passing the training classification feature vector through a classifier to obtain a classification loss function value; the pseudo-cycle difference penalty factor operation unit is used for calculating pseudo-cycle difference penalty factors of the training vehicle body load simulation waveform global feature vector and the training vehicle body load simulation waveform local association feature vector; and a training unit for training the convolutional neural network model as a filter, the converter-based context encoder, the local correlation feature extractor comprising the first convolution layer and the second convolution layer, and the classifier with a weighted sum of the classification loss function value and the pseudo-cyclic difference penalty factor as a loss function value, and propagating through a gradient descent direction.

In the above intelligent monitoring system for load center of gravity shift, the pseudo cycle difference penalty factor operation unit is configured to: calculating a pseudo-cycle difference penalty factor of the training vehicle body load simulation waveform global feature vector and the training vehicle body load simulation waveform local association feature vector by using the following loss formula as the pseudo-cycle difference penalty loss function value; wherein, the loss formula is:

wherein V is ₁ Is the global feature vector of the training vehicle body load simulation waveform, V ₂ Is the local associated feature vector of the training vehicle body load simulation waveform, D (V ₁ ，V ₂ ) For a distance matrix between the training vehicle body load simulation waveform global feature vector and the training vehicle body load simulation waveform local association feature vector, I.I _F The Frobenius norm of the matrix, L is the length of the eigenvector, d (V ₁ ，V ₂ ) Is the distance between the global feature vector of the training vehicle body load simulation waveform and the local associated feature vector of the training vehicle body load simulation waveform, I.I ₂ Is the two norms of the vector, log represents the logarithmic function value based on 2, and alpha and beta are weighted hyper-parameters,is the pseudo-loop difference penalty loss function value.

According to another aspect of the present application, there is provided an intelligent monitoring method for load center of gravity shift, including:

Taking a vehicle body load analog signal acquired by a sensor;

sampling the vehicle body load analog signal by a sliding window based on a sampling window to obtain a plurality of vehicle body load analog sampling windows;

the plurality of vehicle body load simulation sampling windows respectively pass through a convolutional neural network model serving as a filter to obtain waveform characteristic vectors of the plurality of vehicle body load simulation sampling windows;

the wave form feature vectors of the plurality of vehicle body load simulation sampling windows pass through a context encoder based on a converter to obtain a vehicle body load simulation wave form global feature vector;

the waveform feature vectors of the plurality of vehicle body load simulation sampling windows are arranged into one-dimensional feature vectors, and then the one-dimensional feature vectors are passed through a local correlation feature extractor comprising a first convolution layer and a second convolution layer to obtain vehicle body load simulation waveform local correlation feature vectors;

fusing the global feature vector of the vehicle body load simulation waveform and the local association feature vector of the vehicle body load simulation waveform to obtain a classification feature vector; and

the classification feature vector is passed through a classifier to obtain a classification result, which is used to indicate whether the load offset exceeds a predetermined safety range.

According to still another aspect of the present application, there is provided an electronic apparatus including: a processor; and a memory having stored therein computer program instructions that, when executed by the processor, cause the processor to perform the intelligent monitoring method of load center of gravity shifting as described above.

According to a further aspect of the present application there is provided a computer readable medium having stored thereon computer program instructions which, when executed by a processor, cause the processor to perform an intelligent monitoring method of load centre of gravity shifting as described above.

Compared with the prior art, the intelligent monitoring system and the method for the load gravity center deviation, provided by the application, are used for judging directly from analog signals as input by utilizing deep learning and artificial intelligence technology without analog-to-digital conversion, so that errors caused by signal processing are avoided. By the method, the load of the motor vehicle is monitored and judged in real time, and potential threat of overload to road traffic safety is avoided.

Drawings

The above and other objects, features and advantages of the present application will become more apparent by describing embodiments of the present application in more detail with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of embodiments of the application and are incorporated in and constitute a part of this specification, illustrate the application and together with the embodiments of the application, and not constitute a limitation to the application. In the drawings, like reference numerals generally refer to like parts or steps.

FIG. 1 is a schematic diagram of a scenario of an intelligent monitoring system for load center of gravity shifting according to an embodiment of the present application;

FIG. 2 is a block diagram of an intelligent monitoring system for load center of gravity shifting according to an embodiment of the present application;

FIG. 3 is a block diagram of a training module in an intelligent monitoring system for load center of gravity shifting according to an embodiment of the present application;

FIG. 4 is a system architecture diagram of an intelligent monitoring system for load center of gravity shifting according to an embodiment of the present application;

FIG. 5 is a system architecture diagram of a training module in an intelligent monitoring system for load center of gravity shifting according to an embodiment of the present application;

FIG. 6 is a block diagram of a contextual global feature extraction unit in an intelligent monitoring system for load centroid shifting according to an embodiment of the present application;

FIG. 7 is a block diagram of a multi-scale local feature extraction unit in an intelligent monitoring system for load center of gravity shifting according to an embodiment of the application;

FIG. 8 is a flow chart of a method for intelligent monitoring of load center of gravity shifting according to an embodiment of the present application;

fig. 9 is a block diagram of an electronic device according to an embodiment of the application.

Detailed Description

Hereinafter, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only some embodiments of the present application and not all embodiments of the present application, and it should be understood that the present application is not limited by the example embodiments described herein.

Summary of the application

Aiming at the technical problems, the technical conception of the application is to utilize deep learning and artificial intelligence technology to directly judge from analog signals as input without analog-to-digital conversion, thus avoiding errors caused by signal processing. By the method, the load of the motor vehicle is monitored and judged in real time, and potential threat of overload to road traffic safety is avoided.

Specifically, in the technical scheme of the application, firstly, a vehicle body load analog signal acquired by a sensor is acquired. The vehicle body load analog signal can directly reflect the load condition of the vehicle, and is a method for directly and accurately acquiring load information.

And then, sliding window sampling based on the sampling window is carried out on the vehicle body load analog signal so as to obtain a plurality of vehicle body load analog sampling windows. Specifically, sampling the body load analog signal with a sliding window based on sampling windows may divide the signal into a plurality of sampling windows, each of which may contain a certain amount of sampling data. The purpose of this is to improve the robustness and stability of the model, avoiding errors and fluctuations due to uneven distribution of data, noise, etc.

And then, respectively passing the plurality of vehicle body load simulation sampling windows through a convolutional neural network model serving as a filter to obtain a plurality of vehicle body load simulation sampling window waveform characteristic vectors. Here, the convolutional neural network model may perform feature extraction and dimension reduction processing on the input data through a convolutional operation and a pooling operation, thereby obtaining a more representative feature vector. These feature vectors may contain information on frequency, amplitude, phase, etc. in the sampling window and may be used to describe the state and change in the load of the vehicle. By analyzing and processing the feature vectors of the sampling windows, the load condition of the vehicle can be judged more accurately, and the reliability and the precision of the system are improved.

In a specific example of the present application, the convolutional neural network model as a filter includes an input layer, a first convolutional layer, a first activation function, a first pooling layer, a second convolutional layer, a second activation function, a second pooling layer, a full connection layer, a third activation function, and an output layer. The first convolution layer uses 16 convolution kernels with the size of 3x3, the stride is 1, the second convolution layer uses 32 convolution kernels with the size of 3x3, the stride is 1, the first activation function, the second activation function and the third activation function use ReLU, the first pooling layer and the second pooling layer all use maximum pooling, the pooling size is 2x2, the stride is 2, and the fully connected layer expands the output of the second pooling layer into a one-dimensional vector and performs fully connected operation.

Further, the plurality of vehicle body load simulation sampling window waveform feature vectors are passed through a context encoder based on a converter to obtain a vehicle body load simulation waveform global feature vector. Here, the context encoder is a deep learning-based model that can combine local feature vectors into global feature vectors by learning the relationship between input data and semantic information. These global feature vectors may contain more rich and abstract information that may be used to describe the overall state and change of the vehicle load. That is, the local feature vectors extracted in each sampling window can be converted into global feature vectors by the context encoder based on the converter by simulating the waveform feature vectors of the sampling windows for the plurality of vehicle body loads.

Considering that local correlation features exist among the sampling windows, in the technical scheme of the application, after the waveform feature vectors of the plurality of vehicle body load simulation sampling windows are arranged into one-dimensional feature vectors, the vehicle body load simulation waveform local correlation feature vectors are obtained through a local correlation feature extractor comprising a first convolution layer and a second convolution layer. The local correlation feature vectors between sampling windows, such as local relations and interactions between different positions in the sampling windows, are extracted in this way, and can be used to describe the local states and changes of the vehicle load.

The global feature vector of the vehicle body load simulation waveform and the local association feature vector of the vehicle body load simulation waveform reflect the overall state and the local state of the vehicle load respectively, and describe and extract the features of the vehicle load on different layers. The global characteristic vector of the vehicle body load simulation waveform and the local association characteristic vector of the vehicle body load simulation waveform are fused, so that more comprehensive and accurate classification characteristic vectors can be obtained.

The classification feature vector is then passed through a classifier to obtain a classification result that is used to indicate whether the load offset exceeds a predetermined safety range. The classifier is a machine learning model, and can be classified and judged according to the input feature vector. Here, the classification feature vector is taken as an input, and is matched with a classifier model trained in advance, so that a classification result is obtained. The classification result may be used to indicate whether the vehicle load offset exceeds a predetermined safety range. In practical application, if the classification result shows that the current load deviation exceeds the preset safety range, an alarm mechanism needs to be triggered to remind a vehicle driver or related personnel to process, so that safety accidents are avoided.

In the technical scheme of the application, the global feature vector of the vehicle body load simulation waveform expresses contextual global semantic association features of image feature semantics of each vehicle body load simulation sampling window, and the local feature vector of the vehicle body load simulation waveform expresses local semantic association features of image feature semantics of each vehicle body load simulation sampling window, so that under the condition that the global association expression and the local association expression of the image feature semantics are respectively carried out on the global feature vector of the vehicle body load simulation waveform and the local association feature vector of the vehicle body load simulation waveform, the global feature vector of the vehicle body load simulation waveform and the local association feature vector of the vehicle body load simulation waveform have unbalance between overall feature distribution, and the feature expression effect of the classification feature vector obtained by fusing the global feature vector of the vehicle body load simulation waveform and the local association feature vector of the vehicle body load simulation waveform is influenced.

Based on this, the applicant of the present application further introduced, in addition to the classification loss function for the classification feature vectorFor the body load simulation waveform global feature vector, e.g. denoted V ₁ And the local correlation feature vector of the vehicle body load simulation waveform, for example, marked as V ₂ As a loss function, the pseudo-cyclic difference penalty factor of (a) is expressed in detail as:

D(V ₁ ，V ₂ ) Is the characteristic vector V ₁ And V ₂ Distance matrix between, i.e. the eigenvalue of the (i, j) th position of said distance matrix is the eigenvector V ₁ Is the ith eigenvalue v of (2) _1i And feature vector V ₂ The j-th eigenvalue v _2j Distance between I I.I _F The Frobenius norm of the matrix, L is the length of the eigenvector, d (V ₁ ，V ₂ ) Is the feature vector V ₁ And V ₂ Distance between them, e.g. Euclidean distance, |·|| ₂ Is the two norms of the vector, log represents the base 2 logarithm, and α and β are weighted hyper-parameters.

Here, the vehicle body load simulation waveform global feature vector V is considered ₁ And the local correlation characteristic vector V of the vehicle body load simulation waveform ₂ The imbalance distribution therebetween causes gradient propagation anomalies during a model training process based on back propagation of gradient descent, thereby forming a pseudo-loop of model parameter updates, the pseudo-loop of model parameter updates being treated as a true loop during the model training process of minimizing the loss function by introducing penalty factors for expressing both spatial and numerical relationships of closely related numerical pairs of eigenvalues to achieve the vehicle body load simulation waveform global eigenvector V by simulated activation of gradient propagation ₁ And the local correlation characteristic vector V of the vehicle body load simulation waveform ₂ Progressive coupling of the respective feature distributions improves the fusion feature expression effect of the classification feature vectors.

Based on this, the application provides an intelligent monitoring system for load gravity center deviation, which comprises: the analog signal acquisition unit is used for acquiring a vehicle body load analog signal acquired by the sensor; the sliding window sampling unit is used for sampling the sliding window based on the sampling window on the vehicle body load analog signal to obtain a plurality of vehicle body load analog sampling windows; the characteristic filtering unit is used for respectively passing the plurality of vehicle body load simulation sampling windows through a convolutional neural network model serving as a filter to obtain waveform characteristic vectors of the plurality of vehicle body load simulation sampling windows; the context global feature extraction unit is used for enabling the waveform feature vectors of the plurality of vehicle body load simulation sampling windows to pass through a context encoder based on a converter so as to obtain a vehicle body load simulation waveform global feature vector; the multi-scale local feature extraction unit is used for arranging the waveform feature vectors of the plurality of vehicle body load simulation sampling windows into one-dimensional feature vectors and obtaining vehicle body load simulation waveform local association feature vectors through a local association feature extractor comprising a first convolution layer and a second convolution layer; the fusion unit is used for fusing the global characteristic vector of the vehicle body load simulation waveform and the local association characteristic vector of the vehicle body load simulation waveform to obtain a classification characteristic vector; and a monitoring result generating unit, configured to pass the classification feature vector through a classifier to obtain a classification result, where the classification result is used to indicate whether the load offset exceeds a predetermined safety range.

Fig. 1 is a schematic view of a scenario of an intelligent monitoring system for load center of gravity shifting according to an embodiment of the present application. As shown in fig. 1, in this application scenario, a vehicle body load analog signal is acquired by a sensor (e.g., V as illustrated in fig. 1). The signal is then input to a server (e.g., S in fig. 1) that deploys an intelligent monitoring algorithm for load center of gravity shifting, wherein the server is capable of processing the input signal with the intelligent monitoring algorithm for load center of gravity shifting to generate a classification result that indicates whether the load shifting exceeds a predetermined safety range.

Having described the basic principles of the present application, various non-limiting embodiments of the present application will now be described in detail with reference to the accompanying drawings.

Exemplary System

FIG. 2 is a block diagram of an intelligent monitoring system for load center of gravity shifting in accordance with an embodiment of the present application. As shown in fig. 2, the intelligent monitoring system 300 for load center of gravity shift according to an embodiment of the present application includes: an inference module comprising: an analog signal acquisition unit 310; a sliding window sampling unit 320; a feature filtering unit 330; a context global feature extraction unit 340; a multi-scale local feature extraction unit 350; a fusion unit 360; and a monitoring result generation unit 370.

The analog signal acquisition unit 310 is configured to acquire a vehicle body load analog signal acquired by a sensor; the sliding window sampling unit 320 is configured to perform sliding window sampling based on a sampling window on the vehicle body load analog signal to obtain a plurality of vehicle body load analog sampling windows; the feature filtering unit 330 is configured to pass the plurality of vehicle body load analog sampling windows through a convolutional neural network model serving as a filter to obtain waveform feature vectors of the plurality of vehicle body load analog sampling windows; the context global feature extraction unit 340 is configured to pass the plurality of vehicle body load analog sampling window waveform feature vectors through a context encoder based on a converter to obtain a vehicle body load analog waveform global feature vector; the multi-scale local feature extraction unit 350 is configured to arrange the waveform feature vectors of the plurality of vehicle body load analog sampling windows into one-dimensional feature vectors, and then obtain vehicle body load analog waveform local associated feature vectors through a local associated feature extractor including a first convolution layer and a second convolution layer; the fusion unit 360 is configured to fuse the global feature vector of the vehicle body load simulation waveform and the local association feature vector of the vehicle body load simulation waveform to obtain a classification feature vector; and the monitoring result generating unit 370 is configured to pass the classification feature vector through a classifier to obtain a classification result, where the classification result is used to indicate whether the load offset exceeds a predetermined safety range.

Fig. 4 is a system architecture diagram of an intelligent monitoring system for load center of gravity shifting according to an embodiment of the present application. As shown in fig. 4, in the inference module, firstly, the analog signal acquisition unit 310 acquires the vehicle body load analog signal acquired by the sensor; then, the sliding window sampling unit 320 performs sliding window sampling based on the sampling window on the vehicle body load analog signal obtained by the analog signal collecting unit 310 to obtain a plurality of vehicle body load analog sampling windows; the feature filtering unit 330 respectively passes the plurality of vehicle body load simulation sampling windows obtained by the sliding window sampling unit 320 through a convolutional neural network model serving as a filter to obtain waveform feature vectors of the plurality of vehicle body load simulation sampling windows; the context global feature extraction unit 340 obtains the vehicle body load simulation waveform global feature vector by passing the plurality of vehicle body load simulation sampling window waveform feature vectors obtained by the feature filtering unit 330 through a context encoder based on a converter; then, the multi-scale local feature extraction unit 350 arranges the waveform feature vectors of the plurality of vehicle body load simulation sampling windows obtained by the feature filtering unit 330 into one-dimensional feature vectors, and then obtains vehicle body load simulation waveform local association feature vectors through a local association feature extractor comprising a first convolution layer and a second convolution layer; the fusion unit 360 fuses the global feature vector of the vehicle body load simulation waveform obtained by the context global feature extraction unit 340 and the local association feature vector of the vehicle body load simulation waveform obtained by the multi-scale local feature extraction unit 350 to obtain a classification feature vector; further, the monitoring result generating unit 370 passes the classification feature vector obtained by the fusion unit 360 through a classifier to obtain a classification result indicating whether the load offset exceeds a predetermined safety range.

Specifically, during operation of the intelligent monitoring system 300 for load center of gravity shifting, the analog signal acquisition unit 310 is configured to acquire a vehicle body load analog signal acquired by a sensor. It should be appreciated that in the intelligent monitoring of the load center of gravity shift, the load center of gravity shift can be determined directly from the analog signal as input without performing analog-to-digital conversion, thus avoiding errors caused by signal processing. Thus, in one specific example of the present application, a vehicle body load analog signal acquired by a sensor is acquired. The vehicle body load analog signal can directly reflect the load condition of the vehicle, and is a method for directly and accurately acquiring load information.

Specifically, during the operation of the intelligent monitoring system 300 for load center of gravity shift, the sliding window sampling unit 320 is configured to sample the vehicle body load analog signal through a sliding window based on a sampling window to obtain a plurality of vehicle body load analog sampling windows. That is, after the vehicle body load analog signal is obtained, sliding window sampling based on a sampling window is further performed on the vehicle body load analog signal to obtain a plurality of vehicle body load analog sampling windows. Specifically, sliding window sampling of the vehicle body load analog signal based on sampling windows may divide the signal into a plurality of sampling windows, each of which may contain a certain amount of sampling data. The purpose of this is to improve the robustness and stability of the model, avoiding errors and fluctuations due to uneven distribution of data, noise, etc.

Specifically, during the operation of the intelligent monitoring system 300 for load center of gravity shift, the feature filtering unit 330 is configured to pass the plurality of vehicle body load analog sampling windows through a convolutional neural network model as a filter to obtain waveform feature vectors of the plurality of vehicle body load analog sampling windows. In the technical scheme of the application, the convolutional neural network model serving as the filter is used for carrying out feature mining on the plurality of vehicle body load simulation sampling windows to obtain the waveform feature vectors of the plurality of vehicle body load simulation sampling windows, and the convolutional neural network model can carry out feature extraction and dimension reduction processing on input data through convolution operation and pooling operation, so that more representative feature vectors are obtained. These feature vectors may contain information on frequency, amplitude, phase, etc. in the sampling window and may be used to describe the state and change in the load of the vehicle. By analyzing and processing the feature vectors of the sampling windows, the load condition of the vehicle can be judged more accurately, and the reliability and the precision of the system are improved. In a specific example of the present application, the convolutional neural network model as a filter includes an input layer, a first convolutional layer, a first activation function, a first pooling layer, a second convolutional layer, a second activation function, a second pooling layer, a full connection layer, a third activation function, and an output layer. The first convolution layer uses 16 convolution kernels with the size of 3x3, the stride is 1, the second convolution layer uses 32 convolution kernels with the size of 3x3, the stride is 1, the first activation function, the second activation function and the third activation function use ReLU, the first pooling layer and the second pooling layer all use maximum pooling, the pooling size is 2x2, the stride is 2, and the fully connected layer expands the output of the second pooling layer into a one-dimensional vector and performs fully connected operation.

Specifically, during operation of the intelligent monitoring system 300 for load center of gravity shift, the contextual global feature extraction unit 340 is configured to pass the plurality of vehicle body load analog sampling window waveform feature vectors through a context encoder based on a converter to obtain a vehicle body load analog waveform global feature vector. And passing the plurality of vehicle body load simulation sampling window waveform feature vectors through a context encoder based on a converter to obtain a vehicle body load simulation waveform global feature vector. Here, the context encoder is a deep learning-based model that can combine local feature vectors into global feature vectors by learning the relationship between input data and semantic information. These global feature vectors may contain more rich and abstract information that may be used to describe the overall state and change of the vehicle load. That is, the local feature vectors extracted in each sampling window can be converted into global feature vectors by the context encoder based on the converter by simulating the waveform feature vectors of the sampling windows for the plurality of vehicle body loads.

Fig. 6 is a block diagram of a context global feature extraction unit in an intelligent monitoring system for load centroid shifting according to an embodiment of the application. As shown in fig. 6, the contextual global feature extraction unit 340 includes: a query vector construction subunit 341, configured to one-dimensionally arrange the plurality of vehicle body load analog sampling window waveform feature vectors to obtain a global vehicle body load analog sampling window waveform feature vector; a self-attention subunit 342, configured to calculate a product between the global bodywork load analog sampling window waveform feature vector and a transpose vector of each bodywork load analog sampling window waveform feature vector in the plurality of bodywork load analog sampling window waveform feature vectors to obtain a plurality of self-attention correlation matrices; a normalization subunit 343, configured to perform normalization processing on each of the plurality of self-attention correlation matrices to obtain a plurality of normalized self-attention correlation matrices; a attention calculating subunit 344, configured to obtain a plurality of probability values by passing each normalized self-attention correlation matrix of the plurality of normalized self-attention correlation matrices through a Softmax classification function; an attention applying subunit 345, configured to weight each of the plurality of vehicle body load analog sampling window waveform feature vectors with each of the plurality of probability values as a weight, so as to obtain the plurality of up-down Wen Yuyi vehicle body load analog sampling window waveform feature vectors; and a cascade subunit 346, configured to cascade the plurality of upper and lower Wen Yuyi bodywork load simulation sampling window waveform feature vectors to obtain the bodywork load simulation waveform global feature vector.

Specifically, during the operation of the intelligent monitoring system 300 for load center of gravity shift, the multi-scale local feature extraction unit 350 is configured to arrange the waveform feature vectors of the plurality of vehicle body load analog sampling windows into one-dimensional feature vectors, and then obtain the vehicle body load analog waveform local correlation feature vectors through a local correlation feature extractor including a first convolution layer and a second convolution layer. Considering that local correlation features exist among the sampling windows, in the technical scheme of the application, after the waveform feature vectors of the plurality of vehicle body load simulation sampling windows are arranged into one-dimensional feature vectors, the vehicle body load simulation waveform local correlation feature vectors are obtained through a local correlation feature extractor comprising a first convolution layer and a second convolution layer. The local correlation feature vectors between sampling windows, such as local relations and interactions between different positions in the sampling windows, are extracted in this way, and can be used to describe the local states and changes of the vehicle load.

FIG. 7 is a block diagram of a multi-scale local feature extraction unit in an intelligent monitoring system for load center of gravity shifting according to an embodiment of the application. As shown in fig. 7, the multi-scale local feature extraction unit 350 includes: a first scale local feature extraction subunit 351, configured to input the one-dimensional feature vector into a first convolution layer of the local associated feature extractor to obtain a first scale vehicle body load analog waveform local associated feature vector, where the first convolution layer has a first one-dimensional convolution kernel with a first length; a second scale local feature extraction subunit 352 configured to input the one-dimensional feature vector into a second convolution layer of the local associated feature extractor to obtain a second scale vehicle body load simulation waveform local associated feature vector, where the second convolution layer has a second one-dimensional convolution kernel of a second length, and the first length is different from the second length; and a multi-scale cascade subunit 353, configured to cascade the locally-associated feature vector of the vehicle body load simulation waveform and the locally-associated feature vector of the vehicle body load simulation waveform to obtain the locally-associated feature vector of the vehicle body load simulation waveform. Wherein the first scale local feature extraction subunit 351 includes: performing one-dimensional convolution coding on the one-dimensional feature vector by using a first convolution layer of the local correlation feature extractor according to the following one-dimensional convolution formula to obtain a first-scale vehicle body load simulation waveform local correlation feature vector; wherein, the formula is:

Wherein a is the width of a first convolution kernel in the X direction, F (a) is a first convolution kernel parameter vector, G (X-a) is a local vector matrix operated with a convolution kernel function, w is the size of a first one-dimensional convolution kernel, X represents the one-dimensional feature vector, and Cov (X) represents one-dimensional convolution encoding of the one-dimensional feature vector; and, the second scale local feature extraction subunit 352 includes: performing one-dimensional convolution coding on the one-dimensional feature vector by using a second convolution layer of the local correlation feature extractor according to the following one-dimensional convolution formula to obtain a second-scale vehicle body load simulation waveform local correlation feature vector; wherein, the formula is:

Specifically, during the operation of the intelligent monitoring system 300 for load center of gravity shift, the fusion unit 360 is configured to fuse the global feature vector of the vehicle body load simulation waveform and the locally associated feature vector of the vehicle body load simulation waveform to obtain a classification feature vector. That is, after the global feature vector of the vehicle body load simulation waveform and the local association feature vector of the vehicle body load simulation waveform are obtained, the global state and the local state of the vehicle load are fused by further carrying out feature fusion on the global feature vector and the local association feature vector of the vehicle body load simulation waveform, so that more comprehensive and accurate classification feature vectors can be obtained.

Specifically, during operation of the intelligent monitoring system 300 for load center of gravity shifting, the monitoring result generating unit 370 is configured to pass the classification feature vector through a classifier to obtain a classification result, where the classification result is used to indicate whether the load shifting exceeds a predetermined safety range. That is, after the classification feature vector is obtained, it is further subjected to classification processing by a classifier to obtain a classification result indicating whether the load offset exceeds a predetermined safety range. Specifically, the classifier includes a plurality of fully connected layers and a Softmax layer cascaded with a last fully connected layer of the plurality of fully connected layers. In the classification processing of the classifier, multiple full-connection encoding is carried out on the classification feature vectors by using multiple full-connection layers of the classifier to obtain encoded classification feature vectors; further, the encoded classification feature vector is input to a Softmax layer of the classifier, i.e., the encoded classification feature vector is classified using the Softmax classification function to obtain a classification label. The classifier is a machine learning model, and can be classified and judged according to the input feature vector. Here, the classification feature vector is taken as an input, and is matched with a classifier model trained in advance, so that a classification result is obtained. The classification result may be used to indicate whether the vehicle load offset exceeds a predetermined safety range. In practical application, if the classification result shows that the current load deviation exceeds the preset safety range, an alarm mechanism needs to be triggered to remind a vehicle driver or related personnel to process, so that safety accidents are avoided.

It should be appreciated that training of the convolutional neural network model as a filter, the converter-based context encoder, the local correlation feature extractor comprising the first convolutional layer and the second convolutional layer, and the classifier is required before the inference is made using the neural network model described above. That is, in the intelligent monitoring system for load center of gravity shifting of the present application, the system further comprises a training module for training the convolutional neural network model as a filter, the context encoder based on a converter, the local correlation feature extractor comprising the first convolution layer and the second convolution layer, and the classifier.

FIG. 3 is a block diagram of a training module in an intelligent monitoring system for load center of gravity shifting according to an embodiment of the present application. As shown in fig. 3, the intelligent monitoring system 300 for load center of gravity shifting according to an embodiment of the present application further includes a training module 400, including: a training data acquisition unit 410; training a sliding window sampling unit 420; training a feature filtering unit 430; a training context global feature extraction unit 440; training the multi-scale local feature extraction unit 450; training the fusion unit 460; a classification loss unit 470; a pseudo-loop difference penalty factor operation unit 480; and a training unit 490.

Wherein the training data obtaining unit 410 is configured to obtain training data, where the training data includes a training vehicle body load analog signal collected by a sensor, and a true value of whether a load offset exceeds a predetermined safety range; the training sliding window sampling unit 420 is configured to perform sliding window sampling based on a sampling window on the training vehicle body load analog signal to obtain a plurality of training vehicle body load analog sampling windows; the training feature filtering unit 430 is configured to pass the plurality of training vehicle body load analog sampling windows through the convolutional neural network model as a filter to obtain waveform feature vectors of the plurality of training vehicle body load analog sampling windows; the training context global feature extraction unit 440 is configured to pass the plurality of training vehicle body load simulation sampling window waveform feature vectors through the context encoder based on the converter to obtain training vehicle body load simulation waveform global feature vectors; the training multi-scale local feature extraction unit 450 is configured to arrange the waveform feature vectors of the plurality of training vehicle body load simulation sampling windows into one-dimensional feature vectors, and then obtain training vehicle body load simulation waveform local association feature vectors through the local association feature extractor comprising the first convolution layer and the second convolution layer; the training fusion unit 460 is configured to fuse the training vehicle body load simulation waveform global feature vector and the training vehicle body load simulation waveform local association feature vector to obtain a training classification feature vector; the classification loss unit 470 is configured to pass the training classification feature vector through a classifier to obtain a classification loss function value; the pseudo-cyclic difference penalty factor operation unit 480 is configured to calculate a pseudo-cyclic difference penalty factor of the training vehicle body load analog waveform global feature vector and the training vehicle body load analog waveform local association feature vector; and the training unit 490 is configured to train the convolutional neural network model as a filter, the converter-based context encoder, the local correlation feature extractor including the first convolution layer and the second convolution layer, and the classifier with a weighted sum of the classification loss function value and the pseudo-cyclic difference penalty factor as a loss function value, and propagating through a direction of gradient descent.

Fig. 5 is a system architecture diagram of a training module in an intelligent monitoring system for load center of gravity shifting according to an embodiment of the present application. As shown in fig. 5, in the training module, training data including a training vehicle body load analog signal acquired by a sensor and a true value of whether a load offset exceeds a predetermined safety range is first acquired by the training data acquisition unit 410; next, the training sliding window sampling unit 420 performs sliding window sampling based on the training vehicle body load analog signal acquired by the training data acquisition unit 410 to obtain a plurality of training vehicle body load analog sampling windows; the training feature filtering unit 430 respectively passes the plurality of training vehicle body load simulation sampling windows obtained by the training sliding window sampling unit 420 through the convolutional neural network model as a filter to obtain a plurality of training vehicle body load simulation sampling window waveform feature vectors; the training context global feature extraction unit 440 passes the plurality of training vehicle body load analog sampling window waveform feature vectors obtained by the training feature filtering unit 430 through the context encoder based on the converter to obtain a training vehicle body load analog waveform global feature vector; the training multi-scale local feature extraction unit 450 arranges the waveform feature vectors of the plurality of training vehicle body load simulation sampling windows obtained by the training feature filtering unit 430 into one-dimensional feature vectors, and then obtains the training vehicle body load simulation waveform local association feature vectors through the local association feature extractor comprising the first convolution layer and the second convolution layer; then, the training fusion unit 460 fuses the training vehicle body load analog waveform global feature vector obtained by the training context global feature extraction unit 440 and the training vehicle body load analog waveform local association feature vector obtained by the training multi-scale local feature extraction unit 450 to obtain a training classification feature vector; the classification loss unit 470 passes the training classification feature vector obtained by the fusion of the training fusion unit 460 through a classifier to obtain a classification loss function value; then, the pseudo-cyclic difference penalty factor calculation unit 480 calculates a pseudo-cyclic difference penalty factor of the training vehicle body load simulation waveform global feature vector and the training vehicle body load simulation waveform local association feature vector; further, the training unit 490 trains the convolutional neural network model as a filter, the converter-based context encoder, the local correlation feature extractor including the first convolution layer and the second convolution layer, and the classifier with a weighted sum of the classification loss function value and the pseudo-cyclic difference penalty factor as a loss function value, and traveling in a gradient descent direction.

Based on this, the applicant of the present application further introduces global feature vectors, e.g. denoted V, for the vehicle body load simulation waveforms in addition to the classification loss function for the classification feature vectors ₁ And the local correlation feature vector of the vehicle body load simulation waveform, for example, marked as V ₂ As a loss function, the pseudo-cyclic difference penalty factor of (a) is expressed in detail as:

wherein V is ₁ Is the global feature vector of the training vehicle body load simulation waveform, V ₂ Is the local associated feature vector of the training vehicle body load simulation waveform, D (V ₁ ，V ₂ ) For a distance matrix between the training vehicle body load simulation waveform global feature vector and the training vehicle body load simulation waveform local association feature vector, I.I _F The Frobenius norm of the matrix, L is the length of the eigenvector, d (V ₁ ，V ₂ ) Is the distance between the global feature vector of the training vehicle body load simulation waveform and the local associated feature vector of the training vehicle body load simulation waveform, I.I ₂ Is the two norms of the vector, log represents the logarithmic function value based on 2, and alpha and beta are weighted hyper-parameters,is the pseudo-loop difference penalty loss function value. Here, the vehicle body load simulation waveform global feature vector V is considered ₁ And the local correlation characteristic vector V of the vehicle body load simulation waveform ₂ The imbalance distribution therebetween causes gradient propagation anomalies during a model training process based on back propagation of gradient descent, thereby forming a pseudo-loop of model parameter updates, the pseudo-loop of model parameter updates being treated as a true loop during the model training process of minimizing the loss function by introducing penalty factors for expressing both spatial and numerical relationships of closely related numerical pairs of eigenvalues to achieve the vehicle body load simulation waveform global eigenvector V by simulated activation of gradient propagation ₁ And the local correlation characteristic vector V of the vehicle body load simulation waveform ₂ Progressive coupling of the respective feature distributions improves the fusion feature expression effect of the classification feature vectors.

In summary, an intelligent monitoring system 300 for load center of gravity shifting according to an embodiment of the present application is illustrated that uses deep learning and artificial intelligence techniques to determine directly from analog signals as input without analog to digital conversion, thus avoiding errors due to signal processing. By the method, the load of the motor vehicle is monitored and judged in real time, and potential threat of overload to road traffic safety is avoided.

As described above, the intelligent monitoring system for load center of gravity shift according to the embodiment of the present application can be implemented in various terminal devices. In one example, the intelligent monitoring system 300 for load center of gravity shifting according to an embodiment of the present application may be integrated into the terminal device as a software module and/or hardware module. For example, the intelligent monitoring system 300 of load center of gravity shifting may be a software module in the operating system of the terminal device or may be an application developed for the terminal device; of course, the intelligent monitoring system 300 of the load center of gravity shift can also be one of a plurality of hardware modules of the terminal device.

Alternatively, in another example, the load center of gravity shifting intelligent monitoring system 300 and the terminal device may be separate devices, and the load center of gravity shifting intelligent monitoring system 300 may be connected to the terminal device through a wired and/or wireless network and transmit interactive information in a agreed data format.

Exemplary method

Fig. 8 is a flow chart of an intelligent monitoring method of load center of gravity shifting according to an embodiment of the present application. As shown in fig. 8, the intelligent monitoring method for load center of gravity shift according to the embodiment of the application includes the steps of: s110, acquiring a vehicle body load analog signal acquired by a sensor; s120, sliding window sampling based on sampling windows is carried out on the vehicle body load analog signals so as to obtain a plurality of vehicle body load analog sampling windows; s130, respectively passing the plurality of vehicle body load simulation sampling windows through a convolutional neural network model serving as a filter to obtain waveform characteristic vectors of the plurality of vehicle body load simulation sampling windows; s140, passing the waveform feature vectors of the plurality of vehicle body load simulation sampling windows through a context encoder based on a converter to obtain a vehicle body load simulation waveform global feature vector; s150, arranging the waveform feature vectors of the plurality of vehicle body load simulation sampling windows into one-dimensional feature vectors, and then obtaining vehicle body load simulation waveform local association feature vectors through a local association feature extractor comprising a first convolution layer and a second convolution layer; s160, fusing the global feature vector of the vehicle body load simulation waveform and the local association feature vector of the vehicle body load simulation waveform to obtain a classification feature vector; and S170, passing the classification feature vector through a classifier to obtain a classification result, wherein the classification result is used for indicating whether the load offset exceeds a preset safety range.

In one example, in the intelligent monitoring method of load center of gravity shift, the step S130 includes: the convolutional neural network model as a filter comprises an input layer, a first convolutional layer, a first activation function, a first pooling layer, a second convolutional layer, a second activation function, a second pooling layer, a full connection layer, a third activation function and an output layer. The first convolution layer uses 16 convolution kernels with the size of 3x3, the stride is 1, the second convolution layer uses 32 convolution kernels with the size of 3x3, the stride is 1, the first activation function, the second activation function and the third activation function use ReLU, the first pooling layer and the second pooling layer all use maximum pooling, the pooling size is 2x2, and the stride is 2.

In one example, in the intelligent monitoring method of load center of gravity shift, the step S140 includes: one-dimensional arrangement is carried out on the waveform characteristic vectors of the plurality of vehicle body load simulation sampling windows so as to obtain waveform characteristic vectors of global vehicle body load simulation sampling windows; calculating the product between the global vehicle body load simulation sampling window waveform characteristic vector and the transpose vector of each vehicle body load simulation sampling window waveform characteristic vector in the plurality of vehicle body load simulation sampling window waveform characteristic vectors to obtain a plurality of self-attention correlation matrices; respectively carrying out standardization processing on each self-attention correlation matrix in the plurality of self-attention correlation matrices to obtain a plurality of standardized self-attention correlation matrices; obtaining a plurality of probability values by using a Softmax classification function through each normalized self-attention correlation matrix in the normalized self-attention correlation matrices; weighting each vehicle body load simulation sampling window waveform characteristic vector in the vehicle body load simulation sampling window waveform characteristic vectors by taking each probability value in the probability values as a weight so as to obtain the upper and lower Wen Yuyi vehicle body load simulation sampling window waveform characteristic vectors; and cascading the waveform feature vectors of the upper Wen Yuyi and lower Wen Yuyi vehicle body load simulation sampling windows to obtain the vehicle body load simulation waveform global feature vector.

In one example, in the intelligent monitoring method of load center of gravity shift, the step S50 includes: inputting the one-dimensional feature vector into a first convolution layer of the local correlation feature extractor to obtain a first-scale vehicle body load simulation waveform local correlation feature vector, wherein the first convolution layer is provided with a first one-dimensional convolution kernel with a first length; inputting the one-dimensional feature vector into a second convolution layer of the local correlation feature extractor to obtain a second-scale vehicle body load simulation waveform local correlation feature vector, wherein the second convolution layer has a second one-dimensional convolution kernel with a second length, and the first length is different from the second length; and cascading the vehicle body load simulation waveform local association feature vector and the vehicle body load simulation waveform local association feature vector to obtain the vehicle body load simulation waveform local association feature vector. Inputting the one-dimensional feature vector into a first convolution layer of the local correlation feature extractor to obtain a first-scale vehicle body load simulation waveform local correlation feature vector, wherein the method comprises the following steps of: performing one-dimensional convolution coding on the one-dimensional feature vector by using a first convolution layer of the local correlation feature extractor according to the following one-dimensional convolution formula to obtain a first-scale vehicle body load simulation waveform local correlation feature vector; wherein, the formula is:

Wherein a is the width of a first convolution kernel in the X direction, F (a) is a first convolution kernel parameter vector, G (X-a) is a local vector matrix operated with a convolution kernel function, w is the size of a first one-dimensional convolution kernel, X represents the one-dimensional feature vector, and Cov (X) represents one-dimensional convolution encoding of the one-dimensional feature vector; and inputting the one-dimensional feature vector into a second convolution layer of the local correlation feature extractor to obtain a second scale vehicle body load simulation waveform local correlation feature vector, comprising: performing one-dimensional convolution coding on the one-dimensional feature vector by using a second convolution layer of the local correlation feature extractor according to the following one-dimensional convolution formula to obtain a second-scale vehicle body load simulation waveform local correlation feature vector; wherein, the formula is:

In one example, in the intelligent monitoring method of load center of gravity shift, the step S170 includes: performing full-connection coding on the classification feature vectors by using a plurality of full-connection layers of the classifier to obtain coded classification feature vectors; and passing the coding classification feature vector through a Softmax classification function of the classifier to obtain the classification result.

In summary, the intelligent monitoring method of the load center of gravity shift according to the embodiment of the application is explained, which directly judges from an analog signal as an input by utilizing deep learning and artificial intelligence technology without analog-to-digital conversion, thus avoiding errors caused by signal processing. By the method, the load of the motor vehicle is monitored and judged in real time, and potential threat of overload to road traffic safety is avoided.

Exemplary electronic device

Next, an electronic device according to an embodiment of the present application is described with reference to fig. 9.

Fig. 9 illustrates a block diagram of an electronic device according to an embodiment of the application.

As shown in fig. 9, the electronic device 10 includes one or more processors 11 and a memory 12.

The processor 11 may be a Central Processing Unit (CPU) or other form of processing unit having data processing and/or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

Memory 12 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, random Access Memory (RAM) and/or cache memory (cache), and the like. The non-volatile memory may include, for example, read Only Memory (ROM), hard disk, flash memory, and the like. One or more computer program instructions may be stored on the computer readable storage medium that may be executed by the processor 11 to perform the functions in the intelligent monitoring system for load center of gravity shifting and/or other desired functions of the various embodiments of the present application described above. Various contents such as a vehicle body load simulation waveform local correlation feature vector may also be stored in the computer-readable storage medium.

In one example, the electronic device 10 may further include: an input device 13 and an output device 14, which are interconnected by a bus system and/or other forms of connection mechanisms (not shown).

The input means 13 may comprise, for example, a keyboard, a mouse, etc.

The output device 14 may output various information including the classification result and the like to the outside. The output means 14 may include, for example, a display, speakers, a printer, and a communication network and remote output devices connected thereto, etc.

Of course, only some of the components of the electronic device 10 that are relevant to the present application are shown in fig. 9 for simplicity, components such as buses, input/output interfaces, etc. are omitted. In addition, the electronic device 10 may include any other suitable components depending on the particular application.

Exemplary computer program product and computer readable storage Medium

In addition to the methods and apparatus described above, embodiments of the application may also be a computer program product comprising computer program instructions which, when executed by a processor, cause the processor to perform steps in the functions of the intelligent monitoring method of load centre of gravity shifting according to various embodiments of the application described in the "exemplary systems" section of this specification.

The computer program product may write program code for performing operations of embodiments of the present application in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server.

Furthermore, embodiments of the present application may also be a computer-readable storage medium, having stored thereon computer program instructions, which when executed by a processor, cause the processor to perform steps in the functions of the intelligent monitoring method of load center of gravity shifting according to various embodiments of the present application described in the above section of the "exemplary system" of the present specification.

The computer readable storage medium may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium would include the following: an electrical connection having one or more wires, a portable disk, a hard disk, random Access Memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The basic principles of the present application have been described above in connection with specific embodiments, however, it should be noted that the advantages, benefits, effects, etc. mentioned in the present application are merely examples and not intended to be limiting, and these advantages, benefits, effects, etc. are not to be considered as essential to the various embodiments of the present application. Furthermore, the specific details disclosed herein are for purposes of illustration and understanding only, and are not intended to be limiting, as the application is not necessarily limited to practice with the above described specific details.

The block diagrams of the devices, apparatuses, devices, systems referred to in the present application are only illustrative examples and are not intended to require or imply that the connections, arrangements, configurations must be made in the manner shown in the block diagrams. As will be appreciated by one of skill in the art, the devices, apparatuses, devices, systems may be connected, arranged, configured in any manner. Words such as "including," "comprising," "having," and the like are words of openness and mean "including but not limited to," and are used interchangeably therewith. The terms "or" and "as used herein refer to and are used interchangeably with the term" and/or "unless the context clearly indicates otherwise. The term "such as" as used herein refers to, and is used interchangeably with, the phrase "such as, but not limited to.

It is also noted that in the apparatus, devices and methods of the present application, the components or steps may be disassembled and/or assembled. Such decomposition and/or recombination should be considered as equivalent aspects of the present application.

The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the application. Thus, the present application is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit embodiments of the application to the form disclosed herein. Although a number of example aspects and embodiments have been discussed above, a person of ordinary skill in the art will recognize certain variations, modifications, alterations, additions, and subcombinations thereof.

Claims

1. An intelligent monitoring system for load center of gravity shifting, comprising:

2. The intelligent monitoring system of load center of gravity shifting according to claim 1, wherein the convolutional neural network model as a filter comprises an input layer, a first convolutional layer, a first activation function, a first pooling layer, a second convolutional layer, a second activation function, a second pooling layer, a full connection layer, a third activation function, and an output layer.

3. The intelligent monitoring system of load center of gravity shifting according to claim 2, wherein the first convolution layer uses 16 convolution kernels of 3x3 size with a stride of 1, the second convolution layer uses 32 convolution kernels of 3x3 size with a stride of 1, the first, second and third activation functions use ReLU, the first and second pooling layers each use maximum pooling with a pooling size of 2x2 with a stride of 2.

4. The intelligent monitoring system of load center of gravity shifting according to claim 3, wherein the contextual global feature extraction unit comprises:

the query vector construction subunit is used for carrying out one-dimensional arrangement on the waveform characteristic vectors of the plurality of vehicle body load simulation sampling windows to obtain the waveform characteristic vectors of the global vehicle body load simulation sampling windows;

A self-attention subunit, configured to calculate a product between the global bodywork load analog sampling window waveform feature vector and a transpose vector of each bodywork load analog sampling window waveform feature vector in the plurality of bodywork load analog sampling window waveform feature vectors to obtain a plurality of self-attention correlation matrices;

the normalization subunit is used for respectively performing normalization processing on each self-attention correlation matrix in the plurality of self-attention correlation matrices to obtain a plurality of normalized self-attention correlation matrices;

the attention calculating subunit is used for obtaining a plurality of probability values through a Softmax classification function by each normalized self-attention correlation matrix in the normalized self-attention correlation matrices;

the attention applying subunit is configured to weight each vehicle body load simulation sampling window waveform feature vector in the plurality of vehicle body load simulation sampling window waveform feature vectors by using each probability value in the plurality of probability values as a weight, so as to obtain the plurality of upper and lower Wen Yuyi vehicle body load simulation sampling window waveform feature vectors; and

and the cascading subunit is used for cascading the waveform characteristic vectors of the upper Wen Yuyi and lower Wen Yuyi vehicle body load simulation sampling windows to obtain the vehicle body load simulation waveform global characteristic vector.

5. The intelligent monitoring system of load center of gravity shifting according to claim 4, wherein the multi-scale local feature extraction unit comprises:

a first scale local feature extraction subunit, configured to input the one-dimensional feature vector into a first convolution layer of the local correlation feature extractor to obtain a first scale vehicle body load analog waveform local correlation feature vector, where the first convolution layer has a first one-dimensional convolution kernel with a first length;

a second scale local feature extraction subunit, configured to input the one-dimensional feature vector into a second convolution layer of the local correlation feature extractor to obtain a second scale vehicle body load analog waveform local correlation feature vector, where the second convolution layer has a second one-dimensional convolution kernel with a second length, and the first length is different from the second length; and

and the multi-scale cascading subunit is used for cascading the local association characteristic vector of the vehicle body load simulation waveform and the local association characteristic vector of the vehicle body load simulation waveform to obtain the local association characteristic vector of the vehicle body load simulation waveform.

Wherein the first scale local feature extraction subunit is configured to: performing one-dimensional convolution coding on the one-dimensional feature vector by using a first convolution layer of the local correlation feature extractor according to the following one-dimensional convolution formula to obtain a first-scale vehicle body load simulation waveform local correlation feature vector;

Wherein, the formula is:

wherein a is the width of a first convolution kernel in the X direction, F (a) is a first convolution kernel parameter vector, G (X-a) is a local vector matrix operated with a convolution kernel function, w is the size of a first one-dimensional convolution kernel, X represents the one-dimensional feature vector, and Cov (X) represents one-dimensional convolution encoding of the one-dimensional feature vector; and

the second scale local feature extraction subunit is configured to: performing one-dimensional convolution coding on the one-dimensional feature vector by using a second convolution layer of the local correlation feature extractor according to the following one-dimensional convolution formula to obtain a second-scale vehicle body load simulation waveform local correlation feature vector;

wherein, the formula is:

6. The intelligent monitoring system of load center of gravity shifting according to claim 5, wherein the monitoring result generating unit includes:

A full-connection coding subunit, configured to perform full-connection coding on the classification feature vector by using multiple full-connection layers of the classifier to obtain a coded classification feature vector; and

and the classification result generation subunit is used for passing the coding classification feature vector through a Softmax classification function of the classifier to obtain the classification result.

7. The intelligent load center of gravity shifting monitoring system according to claim 6, further comprising a training module for training the filter convolutional neural network model, the converter-based context encoder, the local correlation feature extractor comprising a first convolutional layer and a second convolutional layer, and the classifier.

8. The intelligent monitoring system of load center of gravity shifting according to claim 7, wherein the training module comprises:

a training data acquisition unit configured to acquire training data including a training vehicle body load analog signal acquired by a sensor, and a true value of whether a load offset exceeds a predetermined safety range;

the training sliding window sampling unit is used for sampling the sliding window based on the training vehicle body load analog signal to obtain a plurality of training vehicle body load analog sampling windows;

The training feature filtering unit is used for enabling the plurality of training vehicle body load simulation sampling windows to respectively pass through the convolutional neural network model serving as a filter so as to obtain waveform feature vectors of the plurality of training vehicle body load simulation sampling windows;

the training context global feature extraction unit is used for enabling the plurality of training vehicle body load simulation sampling window waveform feature vectors to pass through the context encoder based on the converter so as to obtain training vehicle body load simulation waveform global feature vectors;

the training multi-scale local feature extraction unit is used for arranging the waveform feature vectors of the plurality of training vehicle body load simulation sampling windows into one-dimensional feature vectors and then obtaining training vehicle body load simulation waveform local association feature vectors through the local association feature extractor comprising the first convolution layer and the second convolution layer;

the training fusion unit is used for fusing the training vehicle body load simulation waveform global feature vector and the training vehicle body load simulation waveform local association feature vector to obtain a training classification feature vector;

the classification loss unit is used for passing the training classification feature vector through a classifier to obtain a classification loss function value;

the pseudo-cycle difference penalty factor operation unit is used for calculating pseudo-cycle difference penalty factors of the training vehicle body load simulation waveform global feature vector and the training vehicle body load simulation waveform local association feature vector; and

A training unit for training the convolutional neural network model as a filter, the converter-based context encoder, the local correlation feature extractor comprising a first convolutional layer and a second convolutional layer, and the classifier with a weighted sum of the classification loss function value and the pseudo-cyclic difference penalty factor as a loss function value, and propagating in a gradient descent direction.

9. The intelligent monitoring system of load center of gravity shifting according to claim 8, wherein the pseudo cyclic difference penalty factor operation unit is configured to: calculating a pseudo-cycle difference penalty factor of the training vehicle body load simulation waveform global feature vector and the training vehicle body load simulation waveform local association feature vector by using the following loss formula as the pseudo-cycle difference penalty loss function value;

wherein, the loss formula is:

wherein V is ₁ Is the global feature vector of the training vehicle body load simulation waveform, V ₂ Is the local associated feature vector of the training vehicle body load simulation waveform, D (V ₁ ,V ₂ ) For the distance matrix between the training vehicle body load simulation waveform global feature vector and the training vehicle body load simulation waveform local association feature vector, the sum of the distance matrix and the distance matrix is II _F The Frobenius norm of the matrix, L is the length of the eigenvector, d (V ₁ ,V ₂ ) Is the distance between the global characteristic vector of the training vehicle body load simulation waveform and the local associated characteristic vector of the training vehicle body load simulation waveform, and is II ₂ Is the two norms of the vector, log represents the logarithmic function value based on 2, and alpha and beta are weighted hyper-parameters,is the pseudo-loop difference penalty loss function value.

10. An intelligent monitoring method for load gravity center deviation is characterized by comprising the following steps:

taking a vehicle body load analog signal acquired by a sensor;