CN116907510A - Intelligent motion recognition method based on Internet of things technology - Google Patents
Intelligent motion recognition method based on Internet of things technology Download PDFInfo
- Publication number
- CN116907510A CN116907510A CN202310886760.6A CN202310886760A CN116907510A CN 116907510 A CN116907510 A CN 116907510A CN 202310886760 A CN202310886760 A CN 202310886760A CN 116907510 A CN116907510 A CN 116907510A
- Authority
- CN
- China
- Prior art keywords
- feature set
- training
- internet
- target
- unit time
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 68
- 238000005516 engineering process Methods 0.000 title claims abstract description 18
- 238000012549 training Methods 0.000 claims abstract description 82
- 230000007246 mechanism Effects 0.000 claims abstract description 17
- 238000013507 mapping Methods 0.000 claims abstract description 15
- 230000015654 memory Effects 0.000 claims abstract description 12
- 230000007787 long-term memory Effects 0.000 claims abstract description 10
- 238000012545 processing Methods 0.000 claims abstract description 8
- 230000006403 short-term memory Effects 0.000 claims abstract description 7
- 239000013598 vector Substances 0.000 claims description 53
- 230000008569 process Effects 0.000 claims description 34
- 239000011159 matrix material Substances 0.000 claims description 16
- 230000004927 fusion Effects 0.000 claims description 14
- 230000006870 function Effects 0.000 claims description 11
- 238000005457 optimization Methods 0.000 claims description 9
- 238000004364 calculation method Methods 0.000 claims description 6
- 238000010606 normalization Methods 0.000 claims description 5
- 238000002759 z-score normalization Methods 0.000 claims description 5
- 230000002265 prevention Effects 0.000 claims description 4
- 238000012795 verification Methods 0.000 claims description 4
- 238000005728 strengthening Methods 0.000 abstract 1
- 238000000605 extraction Methods 0.000 description 6
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000007781 pre-processing Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 239000000306 component Substances 0.000 description 3
- 238000013480 data collection Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000003542 behavioural effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 239000008358 core component Substances 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000002787 reinforcement Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/20—Instruments for performing navigational calculations
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/005—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 with correlation of navigation data from several sources, e.g. map or contour matching
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S19/00—Satellite radio beacon positioning systems; Determining position, velocity or attitude using signals transmitted by such systems
- G01S19/38—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system
- G01S19/39—Determining a navigation solution using signals transmitted by a satellite radio beacon positioning system the satellite radio beacon positioning system transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
- G01S19/42—Determining position
- G01S19/48—Determining position by combining or switching between position solutions derived from the satellite radio beacon positioning system and position solutions derived from a further system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/10—Pre-processing; Data cleansing
- G06F18/15—Statistical pre-processing, e.g. techniques for normalisation or restoring missing data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F18/2131—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on a transform domain processing, e.g. wavelet transform
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
- G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/12—Protocols specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Remote Sensing (AREA)
- Radar, Positioning & Navigation (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computing Systems (AREA)
- General Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Automation & Control Theory (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Evolutionary Biology (AREA)
- Signal Processing (AREA)
- Probability & Statistics with Applications (AREA)
- Medical Informatics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention relates to the field of data processing, in particular to an intelligent motion recognition method based on the technology of the Internet of things, which comprises the following steps: acquiring various original information of a target in each unit time through the Internet of things, extracting an initial feature set through the various original information, splicing the corresponding initial feature set to obtain a first feature set, obtaining a strengthening mapping feature set through a self-attention mechanism, splicing to obtain a second feature set, configuring a long-term and short-term memory network, inputting the second feature set for training to obtain a pre-training model, and further training to obtain a final model. According to the method, more comprehensive information is collected through different types of Internet of things equipment, the self-attention mechanism is adopted to obtain the enhanced mapping feature set, the long-period memory network training model is used, and after the final model is obtained, the prediction precision of the target motion trail is improved.
Description
Technical Field
The invention relates to the technical field of motion recognition, in particular to an intelligent motion recognition method based on the internet of things technology.
Background
In an automated warehouse or factory environment, autonomous navigational robots, drones, or vehicles are changing the manner in which logistics and production lines operate. These autonomous navigational units need to perform tasks in complex and dynamic environments, including moving to a designated destination on a predetermined path and avoiding other moving obstacle targets en route. In this case, the ability to predict the target motion profile in real time becomes critical.
Current solutions rely mainly on sensors of the autonomous navigation unit itself to scan the surrounding environment, such as using cameras, lidar, infrared sensors or ultrasonic sensors. The sensors can collect information about the surrounding environment, help autonomous navigational units identify the trajectory of the target and make predictions, and then plan new paths. The autonomous navigation unit adjusts the motion trail of the autonomous navigation unit according to the information, avoids potential obstacles, and completes tasks. Patent document publication No. CN106054889B discloses a robot autonomous obstacle avoidance method and device, patent document publication No. CN110597269B discloses a vehicle autonomous obstacle avoidance method and a vehicle autonomous obstacle avoidance system, and both methods rely on own sensors for obstacle recognition and avoidance.
However, such a single position sensor-dependent solution may suffer from a problem of low prediction accuracy.
Disclosure of Invention
Therefore, the invention provides an intelligent motion recognition method based on the internet of things technology, which solves the problem of low accuracy in recognition of target motion in the prior art.
In order to achieve the above object, an aspect of the present invention provides an intelligent motion recognition method based on the internet of things technology, the method comprising:
acquiring first original information of a target in each unit time through radio frequency identification equipment in the Internet of things, acquiring second original information of the target in each unit time through a global positioning system in the Internet of things, acquiring third original information of the target in each unit time through an infrared motion sensor in the Internet of things, and acquiring fourth original information of the target in each unit time through a laser scanner in the Internet of things;
extracting a first initial feature set based on the first original information, extracting a second initial feature set based on the second original information, extracting a third initial feature set based on the third original information, and extracting a fourth initial feature set based on the fourth original information;
the first initial feature set, the second initial feature set and the third initial feature set are spliced to obtain a first feature set, an enhanced mapping feature set of a fourth initial feature set for the first feature set is obtained through a self-attention mechanism, and the enhanced mapping feature set is spliced with the first feature set to obtain a second feature set;
configuring a long-term and short-term memory network;
inputting the second feature set into the long-short-period memory network for training to obtain a pre-training model capable of predicting a target motion track, inputting the second feature set obtained in a first unit time into the pre-training model to obtain a predicted track of the target in a second unit time, comparing the predicted track with an actual track to calculate a prediction error after the second unit time is over, taking the prediction error as a third feature set, and inputting the third feature set and the second feature set into the pre-training model for training to obtain a final model.
Further, after the final model is obtained, inputting the second feature set obtained in the first unit time into the final model, so that a predicted track of the target in the second unit time can be obtained;
in the second unit time, if the prediction error does not appear, inputting a second feature set obtained in the second unit time into a final model to obtain a predicted track of the target in the third unit time;
and if the prediction error occurs, inputting the second feature set and the third feature set obtained in the second unit time into a final model to obtain a predicted track of the target in the third unit time.
Further, extracting the first initial feature set based on the first original information includes unicoding the first original information to extract the first initial feature set;
extracting a second initial feature set based on the second original information includes using a min-max normalization process on the second original information to extract a second initial feature set;
extracting a third initial feature set based on the third initial information includes converting the third initial information from a time domain to a frequency domain using fourier transform, and then extracting a main frequency component in the frequency domain as the third initial feature set;
extracting the fourth initial feature set based on the fourth original information includes performing a Z-score normalization process on the fourth original information to extract the fourth initial feature set.
Further, obtaining the first feature set includes: and parallelly splicing the values of the first initial feature set, the second initial feature set and the third initial feature set at the same time to form a matrix, so as to obtain a first feature set.
Further, the enhanced mapping feature set is obtained by calculating feature vector scores of a query vector, a key vector and a value vector through a self-attention mechanism of a transducer model.
Further, the obtaining of the query vector, the key vector, and the value vector includes: obtaining a query vector by multiplying a weight matrix by the first feature set, obtaining a key vector and a value vector by multiplying a weight matrix by the fourth initial feature set, wherein the weight matrix is randomly initialized based on a calculation flow of a transducer model.
Further, configuring the long-term memory network includes: adopting a multi-layer long-short-term memory network structure, and performing preliminary processing on the second feature set by the first layer network to generate intermediate state features;
the second layer network processes the intermediate state characteristics generated by the first layer to obtain high-level characteristics;
the third layer network converts the output of the second layer into the motion state prediction of the target;
during the training process, each layer of network updates network parameters through a time back propagation algorithm to minimize a loss function that measures the difference between the predicted target motion state and the true motion state.
Further, obtaining a pre-training model capable of predicting a target motion trajectory includes:
and when the second feature set is input into the long-short-term memory network for training, applying an early stop and regularization over-fitting prevention technology, wherein the output of the pre-training model is a set of coordinates, and obtaining a predicted track of the target after fitting.
Further, comparing the predicted trajectory with the actual trajectory to calculate a prediction error includes: and calculating the difference between the predicted track and the actual track at each moment in unit time, summing the squares of each difference, and finally dividing the sum by the total moment number to obtain a mean square error, wherein the mean square error is the predicted error.
Further, inputting the third feature set and the second feature set into the pre-training model for training, and obtaining a final model includes: and splicing the third feature set and the second feature set according to a time sequence to obtain a fusion feature set, inputting the fusion feature set into the pre-training model for training, dividing the fusion feature set into a training set and a verification set in the training process, performing parameter optimization by using the loss function and a random gradient descent strategy, dividing data into a plurality of batches by adopting a batch training mode, and training by using only one batch of data each time.
Compared with the prior art, the invention has the beneficial effects that a plurality of different types of internet of things equipment are comprehensively used for collecting more comprehensive information, including radio frequency identification equipment, a global positioning system, an infrared motion sensor and a laser scanner. These devices can provide richer, more accurate data than a single position sensor, thereby improving the accuracy of target motion trajectory prediction.
In particular, the invention adopts a self-attention mechanism to strengthen the mapping feature set, thereby further improving the utilization efficiency of the data. The self-attention mechanism enables the system to better understand and analyze correlations and complex patterns in the data, thereby enhancing the performance of the predictive model.
In particular, the invention can significantly improve the motion prediction capability of autonomous navigation units in complex and dynamic environments. Through the use of long and short term memory networks, the invention can learn and memorize past data, thereby more accurately predicting future tracks.
In particular, the invention can update the prediction result in real time, and after each unit time is finished, the prediction error of the last unit time is introduced as a new feature set to train, so that the prediction result can be timely adjusted under the condition that the parameters of the model are unchanged, and the model adapts to the dynamic change of the environment.
In summary, the method and the device can adapt to complex and dynamic environments while improving the accuracy of predicting the target motion trail, and provide more accurate and reliable prediction for the motion of the autonomous navigation unit in logistics and production lines, thereby effectively improving the task execution capability.
Drawings
Fig. 1 is a flowchart of an intelligent motion recognition method based on the internet of things technology according to an embodiment of the present invention.
Detailed Description
In order that the objects and advantages of the invention will become more apparent, the invention will be further described with reference to the following examples; it should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are merely for explaining the technical principles of the present invention, and are not intended to limit the scope of the present invention.
It should be noted that, in the description of the present invention, terms such as "upper," "lower," "left," "right," "inner," "outer," and the like indicate directions or positional relationships based on the directions or positional relationships shown in the drawings, which are merely for convenience of description, and do not indicate or imply that the apparatus or elements must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as limiting the present invention.
Furthermore, it should be noted that, in the description of the present invention, unless explicitly specified and limited otherwise, the terms "mounted," "connected," and "connected" are to be construed broadly, and may be either fixedly connected, detachably connected, or integrally connected, for example; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. The specific meaning of the above terms in the present invention can be understood by those skilled in the art according to the specific circumstances.
Referring to fig. 1, an intelligent motion recognition method based on the internet of things technology provided by an embodiment of the present invention includes:
step S100: acquiring first original information of a target in each unit time through radio frequency identification equipment in the Internet of things, acquiring second original information of the target in each unit time through a global positioning system in the Internet of things, acquiring third original information of the target in each unit time through an infrared motion sensor in the Internet of things, and acquiring fourth original information of the target in each unit time through a laser scanner in the Internet of things;
step S200: extracting a first initial feature set based on the first original information, extracting a second initial feature set based on the second original information, extracting a third initial feature set based on the third original information, and extracting a fourth initial feature set based on the fourth original information;
step S300: the first initial feature set, the second initial feature set and the third initial feature set are spliced to obtain a first feature set, an enhanced mapping feature set of a fourth initial feature set for the first feature set is obtained through a self-attention mechanism, and the enhanced mapping feature set is spliced with the first feature set to obtain a second feature set;
step S400: configuring a long-term and short-term memory network;
step S500: inputting the second feature set into the long-short-period memory network for training to obtain a pre-training model capable of predicting a target motion track, inputting the second feature set obtained in a first unit time into the pre-training model to obtain a predicted track of the target in a second unit time, comparing the predicted track with an actual track to calculate a prediction error after the second unit time is over, taking the prediction error as a third feature set, and inputting the third feature set and the second feature set into the pre-training model for training to obtain a final model.
Specifically, in step S100, four kinds of internet of things devices, namely, a Radio Frequency Identification Device (RFID), a Global Positioning System (GPS), an infrared motion sensor, and a laser scanner are relied upon. These devices are all key components to achieve target motion recognition. The RFID can identify and track the target in real time in the environment of the Internet of things, and provides the identity and state information of the target. GPS can provide accurate geographic location and speed of movement of objects. The infrared motion sensor is capable of capturing motion and behavioral characteristics of a target, such as walking, running, jumping, and the like. The laser scanner is then able to provide high precision three-dimensional information of the environment and the shape of the target. The four devices are connected through a wireless network and work cooperatively to acquire the comprehensive information of the target.
After step S100, we will obtain a series of raw data related to the motion of the object, including the identification information, position information, motion characteristics and three-dimensional shape information of the object. This information is critical to accurately predicting the motion trajectory of the target, which will be used to generate feature sets that will be used as inputs to the deep learning model for training and prediction. Step S100 is the first step of the whole intelligent motion recognition method, and is also the most basic data collection step, and the quality of the data collection step directly affects the effect of the subsequent steps. For the whole invention, the step provides the basic data required by the whole intelligent motion recognition method, and is the basis for realizing effective motion recognition.
Specifically, in step S200, based on four types of original information, we need to perform feature extraction for each type of information. Feature extraction generally depends on the nature of the data and the target task.
Specifically, extracting the first initial feature set based on the first original information includes performing one-time encoding on the first original information to extract the first initial feature set;
extracting a second initial feature set based on the second original information includes using a min-max normalization process on the second original information to extract a second initial feature set;
extracting a third initial feature set based on the third initial information includes converting the third initial information from a time domain to a frequency domain using fourier transform, and then extracting a main frequency component in the frequency domain as the third initial feature set;
extracting the fourth initial feature set based on the fourth original information includes performing a Z-score normalization process on the fourth original information to extract the fourth initial feature set.
Specifically, step S200 involves some feature extraction and preprocessing techniques. The first original information is acquired through radio frequency identification equipment, and specifically is a unique identifier of a target; the second original information is obtained through a global positioning system, and specifically is geographic coordinates of a target; the third initial information is obtained through an infrared motion sensor, and specifically is the position coordinates and the motion state of the target; the fourth raw information is acquired by a laser scanner, in particular the distance or depth information of the object. For the first original information, features are extracted by means of One-hot Encoding (One-hot Encoding), which is used to process category type data, which is converted into numerical type data capable of being input into a model by Encoding. For the second raw information, the data is processed by means of a Min-max normalization (Min-Max Normalization), which can map the data to a specific range and reduce the influence of outliers on the model. For the third raw information, the frequency domain information is extracted using fourier transform (Fourier Transform), and the dynamic characteristics of the data can be analyzed and understood from the frequency domain perspective. For the fourth original information, Z score normalization (Z-Score Normalization) is carried out, and the method can normalize data to a standard normal distribution with a mean value of 0 and a variance of 1, so that optimization of an algorithm and stability of a model are facilitated.
Specifically, after step S200 is completed, we obtain four initial feature sets obtained by performing various processes and feature extraction on four kinds of original information, where the feature sets are data after preprocessing and feature engineering, and are more suitable for being input into a model for training and prediction. In the whole process, step S200 is in a data preprocessing and feature extraction stage, which is to convert the original data into feature data that can be input into the model, and is the basis and premise of the whole model training. For the whole invention, the step ensures the quality of model input data, provides possibility for training and optimizing the model afterwards, and is beneficial to improving the accuracy of model prediction.
Specifically, in step S300, feature merging and self-attention mechanisms are involved. First, the first, second and third initial feature sets are spliced to obtain a first feature set, which is mainly feature fusion, and features from various sources are combined together to obtain a more comprehensive feature representation. Next, a fourth initial feature set is processed by a Self-attention mechanism (Self-Attention Mechanism) that can weight each feature entered, extracting relatively important feature information. This mechanism has the capability of capturing relationships between features and is particularly effective for sequential data. The enhanced mapping feature set obtained through the self-attention mechanism is spliced with the first feature set to form a second feature set.
Specifically, obtaining the first feature set includes: and parallelly splicing the values of the first initial feature set, the second initial feature set and the third initial feature set at the same time to form a matrix, so as to obtain a first feature set.
In particular, the above mainly relates to stitching of feature vectors. This is a common feature processing technique used to combine features from different sources together to form a more comprehensive characterization. For example, at a certain time, the first initial feature set is [0.2,0.3], the second initial feature set is [0.1,0.6], and the third initial feature set is [0.4,0.7], then at that time, the result of stitching the three feature sets is [ [0.2,0.1,0.4], [0.3,0.6,0.7] ]. Thus, we obtain a feature vector containing all the initial feature information.
Specifically, a two-dimensional matrix is obtained after stitching, wherein each row contains all the features at a certain point in time. The splicing in the mode ensures that the influence of time continuity is fully considered in the characteristic processing stage, and more information is provided for subsequent model training. This step is in the feature integration stage throughout the process, i.e., the combination of the initial feature sets from the previous step.
Specifically, the enhancement map feature set is obtained by performing feature vector score calculation of a query vector, a key vector and a value vector through a self-attention mechanism of a transducer model.
In particular, the self-attention mechanism is a core component of the transducer model. It obtains the enhanced mapping feature set by computing scores for the query vector, the key vector, and the value vector.
Specifically, the obtaining of the query vector, the key vector, and the value vector includes: obtaining a query vector by multiplying a weight matrix by the first feature set, obtaining a key vector and a value vector by multiplying a weight matrix by the fourth initial feature set, wherein the weight matrix is randomly initialized based on a calculation flow of a transducer model.
In particular, the process of acquisition of query vectors, key vectors, and value vectors is accomplished by multiplying a weight matrix by a feature set. As a specific example, we have a first feature set that is the vector [ [0.2,0.1,0.4], [0.3,0.6,0.7] ], while we have randomly initialized weight matrices w_q, w_k, and w_v. These weight matrices are used to generate query vectors, key vectors, and value vectors. By multiplying the first feature set with the weight matrix W q, a query vector can be obtained. The key vector and the value vector are also multiplied by a fourth initial feature set (e.g., a 1 x 4 vector 0.6,0.3,0.2,0.9) using the corresponding weight matrix.
Specifically, by computing the dot product of the query vector and the key vector, and then performing a softmax function conversion, a set of weights are obtained, which are used to weight sum the value vectors, resulting in an enhanced mapping feature set.
Specifically, the enhanced mapping feature set has stronger representativeness, effectively extracts important information and weakens unimportant information, and can better represent the characteristics of the target object. The step is in the feature processing stage of the whole operation process, and is to further process the input features, extract more effective information and prepare for subsequent model training.
Specifically, a more descriptive and differentiated second feature set is obtained after completion of step S300. The method is a feature set enhanced through feature fusion and a self-attention mechanism, contains comprehensive information of original data, and enhances the identification and grabbing of key features. In the whole flow, the step still belongs to a feature preprocessing stage, which is a further expansion of feature extraction, and the expression capacity of the features is improved through different technical means. The method has the significance that the learning effect of the subsequent model can be improved through the reinforcement and fusion of the characteristics, and the prediction accuracy is improved, so that the motion trail of the target is predicted better.
Specifically, in step S400, a long-short-term memory network is configured to initialize the pre-training model.
Specifically, configuring the long-term memory network includes: adopting a multi-layer long-short-term memory network structure, and performing preliminary processing on the second feature set by the first layer network to generate intermediate state features;
the second layer network processes the intermediate state characteristics generated by the first layer to obtain high-level characteristics;
the third layer network converts the output of the second layer into the motion state prediction of the target;
during the training process, each layer of network updates network parameters through a time back propagation algorithm to minimize a loss function that measures the difference between the predicted target motion state and the true motion state.
Specifically, in step S400, the process involves a multi-layered long and short term memory network (LSTM) structure. In this configuration, the first layer network performs a preliminary process on the second feature set generated in step S300 to generate intermediate state features. The second level network then processes these intermediate state features again, generating higher level features. The third level network then converts these high level features into a motion state prediction of the target. In the whole training process, the parameters of each layer of network are updated through a time back propagation algorithm so as to minimize a loss function, and the loss function measures the difference between the motion state of the predicted target and the actual motion state.
Specifically, in step S500, the second feature set is input into the long-short term memory network for training, to obtain a pre-training model capable of predicting the target motion trajectory, and in a simple example, it is assumed that there is a second feature set, which is a 1×4 vector [0.2,0.3,0.7,0.1], and after the first layer network processing, we obtain a new intermediate state feature vector, for example [0.4,0.5,0.2,0.1]. The second tier network then processes it into a higher level feature vector, e.g. [0.6,0.3,0.1,0.1]. Finally, the third layer network converts it to predicted target motion states, e.g., [0.7,0.8], [0.6,0.9], [0.5,0.8].
Specifically, the obtaining of the pre-training model capable of predicting the target motion trajectory includes: and when the second feature set is input into the long-short-term memory network for training, applying an early stop and regularization over-fitting prevention technology, wherein the output of the pre-training model is a set of coordinates, and obtaining a predicted track of the target after fitting.
Specifically, the technical features involved in step S500 are mainly the training process of the pre-training model and the generation of the predicted trajectory. In the training process, the second feature set is input into a long-term and short-term memory network for training, and meanwhile, an early stop and regularized overfitting prevention technology is applied to avoid overlearning of the model on the training set, so that the generalization capability of the model on unseen samples is damaged. The output of the pre-trained model is a set of coordinates that can be used to fit the predicted trajectory of the target. For a specific example, assuming the second feature set is a 1 x 3 vector [0.2,0.3,0.5], after training through the LSTM network we may get a set of output coordinates such as [0.7,0.8], [0.6,0.9], [0.5,0.8]. These coordinate points may be linked together to form a trajectory, which is the predicted trajectory of the object.
Specifically, after the pre-training model is obtained, the model can output a set of coordinates according to the input feature set, and the set of coordinates can form a predicted track of the target after fitting. The pre-training model can preliminarily predict the motion trail of the target in the future. Preparation is made for further obtaining the final model.
Specifically, comparing the predicted trajectory with the actual trajectory to calculate a prediction error includes: and calculating the difference between the predicted track and the actual track at each moment in unit time, summing the squares of each difference, and finally dividing the sum by the total moment number to obtain a mean square error, wherein the mean square error is the predicted error.
In particular, the process of calculating the prediction error mainly involves a comparison between the predicted trajectory and the actual trajectory, and a calculation of the mean square error. The predicted trajectory is a set of coordinates output by the pre-training model, and the actual trajectory is the motion trajectory of the target object in the actual environment. By comparing the two trajectories, the difference between the predicted trajectory and the actual trajectory at each time can be calculated. And summing each difference value after squaring, dividing the sum by the total time quantity to obtain a result which is a mean square error, wherein the value can be used for measuring the prediction accuracy of the prediction model. Assuming that three times are set, the difference between the predicted and actual trajectories is 0.1,0.2,0.1, respectively, the mean square error is ((0.1≡2+0.2≡2+0.1≡2)/3) =0.02.
Specifically, after the prediction error is calculated, a specific value of the prediction error is obtained, and this value reflects the prediction accuracy of the prediction model. This result will provide important feedback about the performance of the model, which is beneficial to tuning and optimizing the model, improving the prediction accuracy of the model. This is an important step in model training, and a third feature set is obtained, based on which the pre-trained model is further trained, enabling optimization of the model.
Specifically, inputting the third feature set and the second feature set into the pre-training model for training, and obtaining a final model includes: and splicing the third feature set and the second feature set according to a time sequence to obtain a fusion feature set, inputting the fusion feature set into the pre-training model for training, dividing the fusion feature set into a training set and a verification set in the training process, performing parameter optimization by using the loss function and a random gradient descent strategy, dividing data into a plurality of batches by adopting a batch training mode, and training by using only one batch of data each time.
Specifically, the method mainly relates to the technical characteristics of feature set splicing, training of a pre-trained model, parameter optimization by using a loss function and a random gradient descent strategy, and the like. First, the third feature set and the second feature set are spliced according to the time sequence, and a fusion feature set is obtained. This fused feature set is then input into a pre-training model for training. In the training process, the fusion feature set is divided into a training set and a verification set, and then parameter optimization is performed by using a loss function and a random gradient descent strategy.
Specifically, the third feature set, that is, the prediction error, is not used as a label, but is a part of the feature, and the final model does not adjust its parameters according to the third feature set in practical application, but remains stable.
Specifically, the final model will be obtained after this step is completed. This model is trained and optimized based on the fused feature set, so that it can better utilize the information in the third feature set and the second feature set for prediction. This step is in the model training and optimization stage during the whole operation process, and its objective is to improve the prediction performance of the model through training and optimization. Implementation of this step may improve the accuracy of the predictive model for the entire invention, thereby improving the user experience. The method has the advantages that the method can utilize various characteristic information to predict, and the accuracy of prediction is improved.
Specifically, the final output stage of the whole operation process marks the establishment of the whole prediction model, and key technical support is provided for realizing safer and more accurate prediction.
Specifically, after the final model is obtained, inputting the second feature set obtained in the first unit time into the final model, so that a predicted track of a target in the second unit time can be obtained;
in the second unit time, if the prediction error does not appear, inputting a second feature set obtained in the second unit time into a final model to obtain a predicted track of the target in the third unit time;
and if the prediction error occurs, inputting the second feature set and the third feature set obtained in the second unit time into a final model to obtain a predicted track of the target in the third unit time.
Specifically, after the final model is obtained, the predicted trajectory of the target in different unit time may be acquired using different feature sets as inputs according to the presence or absence of the prediction error. This step mainly involves the application of a predictive model and the determination of prediction errors. And under the condition that no prediction error occurs, only the second characteristic set input model of the corresponding unit time is needed to predict. And under the condition that the prediction error occurs, the second feature set and the third feature set of the corresponding unit time are input into the model together for prediction. For example, assuming that the second feature set of the first unit time is [0.1,0.2,0.3], the predicted trajectory of the second unit time may be [0.7,0.8], [0.6,0.9], and [0.5,0.8] after being input to the model without a prediction error. If the prediction of the second unit time is in error, the second feature set [0.15,0.25,0.35] of the second unit time and the third feature set [0.02] need to be input into the model together, and the predicted track of the third unit time can be obtained as [0.4,0.3], [0.5,0.4], [0.6,0.5].
In particular, we will get predicted trajectories of the target at different units of time. The accuracy of this predicted trajectory is affected by the prediction error, which, when it occurs, can be further improved by using the second feature set and the third feature set as inputs at the same time. This step is in the model application stage during the whole operation process, and aims to adjust the input data according to the actual situation so as to optimize the prediction result. For the whole invention, this step helps to realize more accurate track prediction, thereby improving the practicability and practical value of the prediction model.
Thus far, the technical solution of the present invention has been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of protection of the present invention is not limited to these specific embodiments. Equivalent modifications and substitutions for related technical features may be made by those skilled in the art without departing from the principles of the present invention, and such modifications and substitutions will be within the scope of the present invention.
The foregoing description is only of the preferred embodiments of the invention and is not intended to limit the invention; various modifications and variations of the present invention will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. An intelligent motion recognition method based on the internet of things technology is characterized by comprising the following steps:
acquiring first original information of a target in each unit time through radio frequency identification equipment in the Internet of things, acquiring second original information of the target in each unit time through a global positioning system in the Internet of things, acquiring third original information of the target in each unit time through an infrared motion sensor in the Internet of things, and acquiring fourth original information of the target in each unit time through a laser scanner in the Internet of things;
extracting a first initial feature set based on the first original information, extracting a second initial feature set based on the second original information, extracting a third initial feature set based on the third original information, and extracting a fourth initial feature set based on the fourth original information;
the first initial feature set, the second initial feature set and the third initial feature set are spliced to obtain a first feature set, an enhanced mapping feature set of a fourth initial feature set for the first feature set is obtained through a self-attention mechanism, and the enhanced mapping feature set is spliced with the first feature set to obtain a second feature set;
configuring a long-term and short-term memory network;
inputting the second feature set into the long-short-period memory network for training to obtain a pre-training model capable of predicting a target motion track, inputting the second feature set obtained in a first unit time into the pre-training model to obtain a predicted track of the target in a second unit time, comparing the predicted track with an actual track to calculate a prediction error after the second unit time is over, taking the prediction error as a third feature set, and inputting the third feature set and the second feature set into the pre-training model for training to obtain a final model.
2. The intelligent motion recognition method based on the internet of things technology according to claim 1, further comprising: after the final model is obtained, inputting the second feature set obtained in the first unit time into the final model, so that a predicted track of a target in the second unit time can be obtained;
if the prediction error does not appear in the second unit time, inputting a second feature set obtained in the second unit time into a final model to obtain a predicted track of the target in the third unit time;
if the prediction error occurs, the second feature set and the third feature set obtained in the second unit time are input into a final model to obtain a predicted track of the target in the third unit time.
3. The intelligent motion recognition method based on the internet of things technology according to claim 2, wherein extracting a first initial feature set based on the first original information comprises:
performing one-time thermal encoding on the first original information to extract a first initial feature set;
extracting a second initial feature set based on the second original information includes using a min-max normalization process on the second original information to extract a second initial feature set;
extracting a third initial feature set based on the third initial information includes converting the third initial information from a time domain to a frequency domain using fourier transform, and then extracting a main frequency component in the frequency domain as the third initial feature set;
extracting the fourth initial feature set based on the fourth original information includes performing a Z-score normalization process on the fourth original information to extract the fourth initial feature set.
4. The method for intelligent motion recognition based on internet of things according to claim 3, wherein obtaining the first feature set comprises: and parallelly splicing the values of the first initial feature set, the second initial feature set and the third initial feature set at the same time to form a matrix, so as to obtain a first feature set.
5. The method for intelligent motion recognition based on the internet of things according to claim 4, wherein the enhanced mapping feature set is obtained by performing feature vector score calculation of query vectors, key vectors and value vectors through a self-attention mechanism of a transducer model.
6. The internet of things-based intelligent motion recognition method of claim 5, wherein the obtaining of the query vector, the key vector, and the value vector comprises: obtaining a query vector by multiplying a weight matrix by the first feature set, obtaining a key vector and a value vector by multiplying a weight matrix by the fourth initial feature set, wherein the weight matrix is randomly initialized based on the calculation flow of the transducer model.
7. The intelligent motion recognition method based on the internet of things technology according to claim 6, wherein configuring the long-term memory network comprises: adopting a multi-layer long-short-term memory network structure, and performing preliminary processing on the second feature set by the first layer network to generate intermediate state features;
the second layer network processes the intermediate state characteristics generated by the first layer to obtain high-level characteristics;
the third layer network converts the output of the second layer into the motion state prediction of the target;
during the training process, each layer of network updates network parameters through a time back propagation algorithm to minimize a loss function that measures the difference between the predicted target motion state and the true motion state.
8. The method for intelligent motion recognition based on the internet of things of claim 7, wherein obtaining a pre-training model capable of predicting a target motion trajectory comprises:
and when the second feature set is input into the long-short-term memory network for training, applying an early stop and regularization over-fitting prevention technology, wherein the output of the pre-training model is a set of coordinates, and obtaining a predicted track of the target after fitting.
9. The intelligent motion recognition method based on the internet of things technology according to claim 8, wherein comparing the predicted trajectory with an actual trajectory to calculate a prediction error comprises: and calculating the difference between the predicted track and the actual track at each moment in unit time, summing the squares of each difference, and finally dividing the sum by the total moment number to obtain a mean square error, wherein the mean square error is the predicted error.
10. The intelligent motion recognition method based on the internet of things technology according to claim 9, wherein inputting the third feature set and the second feature set into the pre-training model for training, and obtaining a final model comprises: and splicing the third feature set and the second feature set according to a time sequence to obtain a fusion feature set, inputting the fusion feature set into the pre-training model for training, dividing the fusion feature set into a training set and a verification set in the training process, performing parameter optimization by using the loss function and a random gradient descent strategy, dividing data into a plurality of batches by adopting a batch training mode, and training by using only one batch of data each time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310886760.6A CN116907510B (en) | 2023-07-19 | 2023-07-19 | Intelligent motion recognition method based on Internet of things technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310886760.6A CN116907510B (en) | 2023-07-19 | 2023-07-19 | Intelligent motion recognition method based on Internet of things technology |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116907510A true CN116907510A (en) | 2023-10-20 |
| CN116907510B CN116907510B (en) | 2024-08-09 |
Family
ID=88364266
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310886760.6A Active CN116907510B (en) | 2023-07-19 | 2023-07-19 | Intelligent motion recognition method based on Internet of things technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116907510B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117841751A (en) * | 2024-03-08 | 2024-04-09 | 沧州智慧城科技有限公司 | Charging gun system and charging method based on intention recognition |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200309541A1 (en) * | 2019-03-28 | 2020-10-01 | Nexar Ltd. | Localization and mapping methods using vast imagery and sensory data collected from land and air vehicles |
| CN113204718A (en) * | 2021-04-22 | 2021-08-03 | 武汉大学 | Vehicle track destination prediction method considering space-time semantics and driving state |
| US20220011122A1 (en) * | 2020-07-09 | 2022-01-13 | Beijing Tusen Weilai Technology Co., Ltd. | Trajectory prediction method and device |
| CN114021660A (en) * | 2021-11-10 | 2022-02-08 | 云从科技集团股份有限公司 | Training method, device, medium and equipment for feature mining model |
| CN114065884A (en) * | 2021-10-20 | 2022-02-18 | 国网浙江省电力有限公司杭州供电公司 | Infrared electronic tag position prediction method combined with local attention mechanism |
| CN114529799A (en) * | 2022-01-06 | 2022-05-24 | 浙江工业大学 | Aircraft multi-target tracking method based on improved YOLOV5 algorithm |
| CN115048870A (en) * | 2022-07-02 | 2022-09-13 | 南京航空航天大学 | Target track identification method based on residual error network and attention mechanism |
| CN115376099A (en) * | 2022-07-21 | 2022-11-22 | 长安大学 | Track prediction method based on target local terminal and multi-head attention mechanism |
| CN115907154A (en) * | 2022-11-22 | 2023-04-04 | 中冶南方都市环保工程技术股份有限公司 | Industrial internet prediction method and system based on frequency domain and long-term and short-term feature fusion |
-
2023
- 2023-07-19 CN CN202310886760.6A patent/CN116907510B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200309541A1 (en) * | 2019-03-28 | 2020-10-01 | Nexar Ltd. | Localization and mapping methods using vast imagery and sensory data collected from land and air vehicles |
| US20220011122A1 (en) * | 2020-07-09 | 2022-01-13 | Beijing Tusen Weilai Technology Co., Ltd. | Trajectory prediction method and device |
| CN113204718A (en) * | 2021-04-22 | 2021-08-03 | 武汉大学 | Vehicle track destination prediction method considering space-time semantics and driving state |
| CN114065884A (en) * | 2021-10-20 | 2022-02-18 | 国网浙江省电力有限公司杭州供电公司 | Infrared electronic tag position prediction method combined with local attention mechanism |
| CN114021660A (en) * | 2021-11-10 | 2022-02-08 | 云从科技集团股份有限公司 | Training method, device, medium and equipment for feature mining model |
| CN114529799A (en) * | 2022-01-06 | 2022-05-24 | 浙江工业大学 | Aircraft multi-target tracking method based on improved YOLOV5 algorithm |
| CN115048870A (en) * | 2022-07-02 | 2022-09-13 | 南京航空航天大学 | Target track identification method based on residual error network and attention mechanism |
| CN115376099A (en) * | 2022-07-21 | 2022-11-22 | 长安大学 | Track prediction method based on target local terminal and multi-head attention mechanism |
| CN115907154A (en) * | 2022-11-22 | 2023-04-04 | 中冶南方都市环保工程技术股份有限公司 | Industrial internet prediction method and system based on frequency domain and long-term and short-term feature fusion |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117841751A (en) * | 2024-03-08 | 2024-04-09 | 沧州智慧城科技有限公司 | Charging gun system and charging method based on intention recognition |
| CN117841751B (en) * | 2024-03-08 | 2024-05-17 | 沧州智慧城科技有限公司 | Charging gun system and charging method based on intention recognition |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116907510B (en) | 2024-08-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN114384920B (en) | Dynamic obstacle avoidance method based on real-time construction of local grid map | |
| Yuan et al. | A novel GRU-RNN network model for dynamic path planning of mobile robot | |
| Zhao et al. | A spatial-temporal attention model for human trajectory prediction. | |
| Grigorescu et al. | Neurotrajectory: A neuroevolutionary approach to local state trajectory learning for autonomous vehicles | |
| Chen et al. | Robot navigation with map-based deep reinforcement learning | |
| CN113205466A (en) | Incomplete point cloud completion method based on hidden space topological structure constraint | |
| CN106714110A (en) | Auto building method and system of Wi-Fi position fingerprint map | |
| CN112148008B (en) | Real-time unmanned aerial vehicle path prediction method based on deep reinforcement learning | |
| KR102288933B1 (en) | METHOD, APPARATUS AND SYSTEM FOR BUILDING OF precision road map | |
| CN116907510B (en) | Intelligent motion recognition method based on Internet of things technology | |
| CN114708435B (en) | Obstacle size prediction and uncertainty analysis method based on semantic segmentation | |
| CN112857370A (en) | Robot map-free navigation method based on time sequence information modeling | |
| Xue et al. | Fuzzy controller for autonomous vehicle based on rough sets | |
| CN113219506A (en) | Positioning method for multimode fusion seamless switching | |
| Prokop et al. | Neuro-heuristic pallet detection for automated guided vehicle navigation | |
| CN114386599B (en) | Method and device for training trajectory prediction model and trajectory planning | |
| Lei et al. | Digital twin‐based multi‐objective autonomous vehicle navigation approach as applied in infrastructure construction | |
| CN110926470B (en) | AGV navigation control method and system | |
| Daass et al. | Design of multi-sensor fusion architectures based on the covariance intersection algorithm—estimating calculation burdens | |
| Gao et al. | A prediction method of localizability based on deep learning | |
| CN116429112A (en) | Multi-robot co-location method and device, equipment and storage medium | |
| Khan et al. | Learning sample-efficient target reaching for mobile robots | |
| CN115900725A (en) | Path planning device, electronic device, storage medium and related method | |
| CN115373383A (en) | Autonomous obstacle avoidance method and device for garbage recovery unmanned boat and related equipment | |
| Wang et al. | Research on SLAM road sign observation based on particle filter |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |