CN115757372A - Data missing value filling method and system based on GRU model - Google Patents
Data missing value filling method and system based on GRU model Download PDFInfo
- Publication number
- CN115757372A CN115757372A CN202211355037.7A CN202211355037A CN115757372A CN 115757372 A CN115757372 A CN 115757372A CN 202211355037 A CN202211355037 A CN 202211355037A CN 115757372 A CN115757372 A CN 115757372A
- Authority
- CN
- China
- Prior art keywords
- data
- livestock
- gru model
- sensor data
- model
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/70—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in livestock or poultry
Landscapes
- Testing Or Calibration Of Command Recording Devices (AREA)
Abstract
The invention discloses a data missing value filling method and a system based on a GRU model, wherein the method comprises the following steps: acquiring sensor data samples of the livestock based on ear tag sensors of the livestock; training a pre-constructed GRU model based on a sensor data sample to generate a target GRU model; and acquiring sensor data to be filled, and inputting the sensor data to be filled into the target GRU model to obtain a data missing value filling result. The embodiment of the invention can well relieve the problem of insufficient data acquisition of the sensor caused by signals, is beneficial to the following data analysis and mining work, and provides convenience for automatically monitoring the real-time condition of livestock.
Description
Technical Field
The invention relates to the technical field of data processing, in particular to a data missing value filling method and system based on a GRU model.
Background
The application of the sensor in the present society is more and more extensive, and the sensor is already used in the aspects of social production, for example, the new foundation strategy of the country, the monitoring of an equipment system, the monitoring of a meteorological environment and the like all need to use a sensor technology; the overall improvement of living standard brings about a rapid increase of consumption of livestock products such as cow, sheep, egg and milk, and the requirement of meat quality is also high in water vessels, so that the production of high-quality meat depends on good livestock industry, the daily monitoring of livestock such as pig, cow and sheep becomes more and more important, and the sensor plays an increasingly important role therein.
In the prior art, ear tag sensors nailed on ears of livestock are generally adopted to monitor domestic animals such as pigs, cattle and sheep, the sensors have the functions of monitoring the motion acceleration of the cultured animals and the body temperature of the animals, and collected data are beneficial to monitoring and further data mining by rear-end personnel; the more complete the data acquisition, the more helpful the subsequent work of the personnel involved. Due to various unstable environmental factors possibly occurring on the site, the situation of unstable signals is very easy to occur in the sensor function, and thus the data acquisition is incomplete.
The existing data missing value filling method has low calculation precision and high calculation complexity; overfitting situations tend to occur at the time of prediction.
The prior art is therefore still subject to further development.
Disclosure of Invention
In order to solve the above technical problems, embodiments of the present invention provide a data missing value filling method and system based on a GRU model, which can solve the problems of low calculation accuracy and high calculation complexity of the data missing value filling method in the prior art; easily generating the overfitting situation at the time of prediction.
A first aspect of an embodiment of the present invention provides a data missing value filling method based on a GRU model, including:
acquiring sensor data samples of the livestock based on ear tag sensors of the livestock;
training a pre-constructed GRU model based on a sensor data sample to generate a target GRU model;
and acquiring sensor data to be filled, and inputting the sensor data to be filled into the target GRU model to obtain a data missing value filling result.
Optionally, the
Obtaining sensor data samples of livestock based on ear tag sensors of the livestock, comprising:
acquiring real-time sensor data of livestock based on ear tag sensors of the livestock, wherein the real-time data of the livestock comprises temperature data, motion data and environmental temperature data of the livestock;
mapping the real-time data of the livestock and the identification numbers of the livestock and then storing the mapping data into a time sequence database;
when a sensor data sample generation instruction is detected, acquiring an identification number of the livestock based on the ear tag number of the livestock;
reading sensor data corresponding to the livestock in the time sequence database according to the identification number of the livestock;
extracting sensor data according to the time steps, converting the extracted data format into a sample format of a GRU model, and generating an initial sensor data sample;
and carrying out normalization processing on the initial sensor data sample to generate a sensor data sample.
Optionally, training a pre-constructed GRU model based on the sensor data samples, before generating the target GRU model, includes:
and constructing an initial GRU model, and adding a full connection layer on an output layer of the initial GRU model to generate a pre-constructed GRU model.
Optionally, training a pre-constructed GRU model based on the sensor data samples to generate a target GRU model includes:
segmenting a sensor data sample to generate a training set, a testing set and a verification set;
inputting a training set into a pre-constructed GRU model for training;
optimizing model parameters through a predefined label and a loss function in the training process;
and after the optimization is completed, generating a target GRU model and storing the target GRU model.
Optionally, the obtaining sensor data to be filled, and inputting the sensor data to be filled into the target GRU model to obtain a data missing value filling result includes:
acquiring sensor data to be filled, and extracting data missing values and non-missing values in the sensor data;
extracting time sequence data of non-missing values of a preset length before the missing value as input data, and carrying out normalization processing on the input data and inputting the input data into a GRU model;
and acquiring an output result of the GRU model, and performing inverse normalization processing on the output result to obtain a prediction result of the data deficiency value.
A second aspect of the embodiments of the present invention provides a data missing value padding system based on a GRU model, where the system includes: a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program when executed by the processor implementing the steps of:
acquiring sensor data samples of the livestock based on ear tag sensors of the livestock;
training a pre-constructed GRU model based on a sensor data sample to generate a target GRU model;
and acquiring sensor data to be filled, and inputting the sensor data to be filled into the target GRU model to obtain a data missing value filling result.
Optionally, the computer program when executed by the processor further implements the steps of:
acquiring real-time sensor data of the livestock based on ear tag sensors of the livestock, wherein the real-time data of the livestock comprises temperature data, motion data and environment temperature data of the livestock;
mapping the real-time data of the livestock and the identification numbers of the livestock and then storing the mapping data to a time sequence database;
when a sensor data sample generation instruction is detected, acquiring an identification number of the livestock based on the ear tag number of the livestock;
reading sensor data corresponding to the livestock in the time sequence database according to the identification number of the livestock;
extracting sensor data according to the time step number, converting the extracted data format into a sample format of a GRU model, and generating an initial sensor data sample;
and carrying out normalization processing on the initial sensor data sample to generate a sensor data sample.
Optionally, the computer program when executed by the processor further implements the steps of:
and constructing an initial GRU model, and adding a full connection layer on an output layer of the initial GRU model to generate a pre-constructed GRU model.
Optionally, the computer program when executed by the processor further implements the steps of:
segmenting a sensor data sample to generate a training set, a testing set and a verification set;
inputting a training set into a pre-constructed GRU model for training;
optimizing model parameters through a predefined label and a loss function in the training process;
and after the optimization is completed, generating a target GRU model and storing the target GRU model.
A third aspect of embodiments of the present invention provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer-executable instructions, and when executed by one or more processors, the computer-executable instructions may cause the one or more processors to perform the above-mentioned method for filling missing data values based on a GRU model.
According to the technical scheme provided by the embodiment of the invention, a sensor data sample of the livestock is obtained based on the ear tag sensor of the livestock; training a pre-constructed GRU model based on a sensor data sample to generate a target GRU model; and acquiring sensor data to be filled, and inputting the sensor data to be filled into the target GRU model to obtain a data missing value filling result. The embodiment of the invention can well relieve the problem of insufficient data acquisition of the sensor caused by signals, is beneficial to the following data analysis and mining work, and provides convenience for automatically monitoring the real-time condition of livestock.
Drawings
Fig. 1 is a schematic flowchart of an embodiment of a data missing value filling method based on a GRU model according to the present invention;
fig. 2 is a schematic hardware structure diagram of another embodiment of a data missing value padding system based on a GRU model according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The following detailed description of embodiments of the invention refers to the accompanying drawings.
Referring to fig. 1, fig. 1 is a flowchart illustrating an embodiment of a data missing value padding method based on a GRU model according to the present invention. As shown in fig. 1, includes:
s100, acquiring sensor data samples of livestock based on ear tag sensors of the livestock;
s200, training a pre-constructed GRU model based on a sensor data sample to generate a target GRU model;
and S300, acquiring sensor data to be filled, and inputting the sensor data to be filled into the target GRU model to obtain a data missing value filling result.
In specific implementation, the sensor corresponding to the embodiment of the invention is an ear tag sensor which is nailed on the livestock ear, the sensor has the function of monitoring the movement acceleration of the cultured animal and the animal body temperature, and the collected data is beneficial to monitoring and further data mining by back-end personnel; the more complete the data acquisition the more helpful the subsequent work of the personnel involved. Due to various unstable environmental factors possibly occurring on the site, the function of the sensor is very easy to cause the situation of unstable signals, thereby resulting in incomplete data acquisition, and therefore, the missing value completion is helpful to the integrity of the data. The method adopts a GRU model and a full link layer algorithm, trains the model through the existing data, and predicts the specific value of the next piece of data, so that the data is received more completely.
Acquiring an identification number of the livestock based on the ear tag number of the livestock; acquiring sensor data of the ear tag sensor based on the identification number of the livestock; extracting the sensor data according to the time step number to generate an input/output format required by a GRU model; normalizing the extracted data; inputting the normalized data into a GRU model for training, optimizing model parameters through a label and a loss function, and further verifying; generating a trained model for storage, and deploying the trained model to a platform corresponding to a relevant interface; when the sensor data to be filled are detected, missing values to be marked are sorted into a format required by the model according to corresponding rules; and predicting through a model deployed on the platform, and outputting a result through reverse normalization to generate a data missing value filling result.
And predicting missing values of the sensor in the data transmission process by a cyclic neural network mode in deep learning. The sensors are mainly ear tag sensors on ears of pigs, cattle and sheep, and the returned data mainly comprise motion data and temperature data. In reality, the sensors often have insufficient data reception due to on-site objective conditions; high-accuracy missing value prediction is necessary, and the GRU algorithm of the time can be well qualified for the prediction task.
The sensor missing value is predicted based on the artificial intelligence deep learning GRU algorithm, and the method has the advantages of strong generalization capability, convenience in deployment, good accuracy and stability.
Further, acquiring sensor data samples of the livestock based on the ear tag sensors of the livestock, comprising:
acquiring real-time sensor data of livestock based on ear tag sensors of the livestock, wherein the real-time data of the livestock comprises temperature data, motion data and environmental temperature data of the livestock;
mapping the real-time data of the livestock and the identification numbers of the livestock and then storing the mapping data into a time sequence database;
when a sensor data sample generation instruction is detected, acquiring an identification number of the livestock based on the ear tag number of the livestock;
reading sensor data corresponding to the livestock in the time sequence database according to the identification number of the livestock;
extracting sensor data according to the time steps, converting the extracted data format into a sample format of a GRU model, and generating an initial sensor data sample;
and carrying out normalization processing on the initial sensor data sample to generate a sensor data sample.
In specific implementation, the background data mainly comprises livestock information data and real-time data, so that the background data mainly comprises data such as registration id numbers, registration dates, working conditions and age and weight of livestock, and mainly reflects state information of the livestock, a farm, field equipment and the like, which are mainly transmitted back through the ear tag sensor and the base station (the information data of the livestock mainly comprises data such as registration id numbers, registration dates, working conditions and age and weight of the livestock, which are not stored in the ear tag sensor, the data are transmitted back through the ear tag sensor and are real-time data, which need to be subjected to analysis emphatically), the Mysql and TDengine databases are used for storage, the real-time data are stored in the TDengine database, and aiming at the data in the TDengine, the data acquisition needs to pass through a taos interface, the corresponding ear tag numbers are input in codes, the id numbers corresponding to the ear tags are acquired, and relevant real-time data are acquired from the TDengine through the id numbers, the set time periods and the taos interface; and after the model is obtained, the body temperature and the movement acceleration data of the livestock in a specific time period need to be extracted to serve as the training data of the model. The data characteristics required to be obtained comprise three-axis motion data, temperature and environment temperature of all livestock (an ear tag sensor comprises an acceleration sensor and a temperature sensor, and the acceleration sensor records x _ axis, y _ axis and z _ axis of x, y and z in three directions.
Analyzing according to the extracted real-time data returned by the livestock, and as the data is time series data and changes along with time change, time step number time _ step needs to be set, and the data after the time step time period is taken as a prediction tag (the same column of data) of the current time data; (the time step number refers to that the time step number is t2-t1 if the real-time data is from time t1 to t2, the data after t2 is taken, and the data corresponding to the time t2+1 is taken as the data to be predicted; for example, the data in a time period of 0-5 seconds is taken, and if the data time interval is 1 second, the time step is 5-0=5 seconds, and then the data at the time 6 second is taken as the tag data) so as to form the current data and the tag into a data set,
further, training a pre-constructed GRU model based on sensor data samples, before generating a target GRU model, comprises:
and constructing an initial GRU model, and adding a full connection layer on an output layer of the initial GRU model to generate a pre-constructed GRU model.
In specific implementation, the GRU model is a deep learning model for processing time sequence data, is changed based on RNN, and is different from the RNN model in that the GRU network well solves the gradient problem caused in the long-term memory and back propagation processes; the method has the characteristics of less parameter quantity, higher training speed, reduction of the risk of overfitting and the like. Unlike other deep learning models, GRU is mainly directed to sequence data and therefore can be well used to process real-time data returned by sensors. The model structure of the GRU is based on gate functions to control state values and inputs.
The output result of the GRU model is also multi-dimensional vector data, which cannot be used as the final result, and needs to be output through a full connection layer, and the output dimension of the full connection is 1.
Further, training a pre-constructed GRU model based on the sensor data samples to generate a target GRU model, comprising:
segmenting a sensor data sample to generate a training set, a testing set and a verification set;
inputting a training set into a pre-constructed GRU model for training;
optimizing model parameters through a predefined label and a loss function in the training process;
and after the optimization is completed, generating a target GRU model and storing the target GRU model.
In specific implementation, the data set is divided into three parts, namely a training set, a test set and a verification set (the general training set accounts for more than 60%, and the test set and the verification set respectively account for 20%).
When the data set is cut, the training set is input into a GRU model (GRU parameters comprise the number of hidden layers, the number of hidden units, whether the parameters are bidirectional and the like, and are actually preset) to be calculated, the output of the GRU is input into a full connection layer, the full connection layer is used for final output, and the output result and the label value are subjected to parameter optimization and fitting through a set loss function and back propagation.
Because the process of predicting the missing value by the GRU model belongs to regression prediction and a single number is output, the adopted loss function is RMSE, and the adopted optimizer is Adam optimizer which has the function of automatically adjusting the learning rate according to the condition of parameter optimization. The loss of the output result of the GRU and the training label is calculated through an RMSE function, then the parameters are derived through the loss function, and the parameters of the optimized model are continuously modified in a gradient descending mode, so that the whole algorithm model gradually achieves the effect of accurately predicting the missing value.
Further, acquiring sensor data to be filled, inputting the sensor data to be filled into the target GRU model, and obtaining a data missing value filling result, including:
acquiring sensor data to be filled, and extracting data missing values and non-missing values in the sensor data;
extracting time sequence data of non-missing values with preset length before the missing value as input data, and performing normalization processing on the input data to input the input data into a GRU model;
and acquiring an output result of the GRU model, and performing inverse normalization processing on the output result to obtain a prediction result of the data missing value.
In specific implementation, the data source is mainly real-time data, so that the number of features is small, the data is mainly processed through normalization to accelerate model convergence, and the model training time is reduced. And recovering the output data through a set reverse normalization formula.
Normalization is to unify data with inconsistent dimensions to a certain range ([ 0,1 ]) to unify dimensions and improve the learning speed of the model.
Normalization formula:
Processed_x＝(X-X_min)/(X_max-X_min)
the formula of inverse normalization:
X＝Processed_x*/(X_max-X_min)+X_min
x is an input array;
x _ min: a minimum value in the array;
x _ max: maximum value in the array.
Deploying the model on line, setting a relevant input/output interface, and inputting a corresponding ear tag number to display relevant real-time data in the ear tag; extracting data (which can be extracted by a pandas tool) of a missing value through a set rule algorithm, extracting time sequence data of a non-missing value with a certain length before the missing value as input data, performing normalization processing on the input data, inputting the normalized input data into a model, and finally outputting the normalized input data as a final result through reverse normalization.
The parameters of the embodiments of the present invention are explained as follows:
the gate function: the operation combination based on the sigmoid function and the point multiplication in the GRU can selectively determine which parameters pass through;
label number is the ear tag number of the livestock;
x _ axis is the acceleration value in the x direction;
x _ axis is the acceleration value in the y direction;
z _ axis is the acceleration value in the z direction;
total _ length: the sum (characteristic) of absolute values of the moving acceleration of the livestock in the xyz three directions;
the adam optimizer: an optimization algorithm needed when the model is propagated backwards;
the pandas: a data processing tool operates based on the python language.
In some other embodiments, other models may be used, such as BiGRU, a bidirectional GRU model that is improved based on GRU; theoretically, longer time series input information can be better taken into account. Other algorithms such as RNN, ARIMA, bilSTM, etc. also have the function of predicting time sequence data, and when the data does not have long correlation, the above methods can be used as alternatives.
The invention also provides a concrete application embodiment of the data missing value filling method based on the GRU model, and the method comprises the following steps:
inputting the ear mark number (label _ number) corresponding to the bred animal so as to obtain the mark _ id of the bred animal;
detecting whether the feedback conditions of the ear tags and the base station of the cultured animals are normal (whether the cultured animals are in the columns and whether the base station has no fault) based on the mark _ id of the cultured animals;
if the ear tag of the cultured animal and the feedback condition of the base station are abnormal, the program operation is finished, and the id of the cultured animal at the next end is continuously traversed;
if the ear tag and the base station feedback condition of the cultured animal are normal, checking whether the real-time data of the cultured animal can be read by TDengine;
if the real-time data of the bred animals cannot be read by TDengine, ending the program operation, and continuously traversing the bred animal id of the next head;
if the real-time data of the cultured animals can be read by the TDengine, extracting the real-time data of temperature and motion from the TDengine based on the mark _ id of the cultured animals;
extracting the data according to the time step number (time _ step), and generating an input/output format required by the GRU;
carrying out normalization processing on the extracted data;
designing a GRU model, and adding a full connection layer at the output part of the model; inputting the normalized data into a model for training, optimizing model parameters through a label and a loss function, and further verifying;
when a new data set comes in, marking missing values through related rule codes and arranging the missing values into a format required by a model;
storing the trained model, and corresponding to the corresponding interface, deploying the model to a platform;
and predicting through a model deployed on the platform, and outputting the result through reverse normalization.
The embodiment of the invention mainly adopts a GRU model with a simpler model structure on the basis of the existing algorithm, and is univariate prediction, and the number of missing values is not too much, so that the model structure has enough performance to predict data with lower characteristic dimensionality. And the existing equipment is not so complex in data acquisition and has good excavatability, so that the deployment on a platform and hardware is facilitated, and the cost is also reduced.
It should be noted that, a certain order does not necessarily exist between the above steps, and those skilled in the art can understand, according to the description of the embodiments of the present invention, that in different embodiments, the above steps may have different execution orders, that is, may be executed in parallel, may also be executed interchangeably, and the like.
With reference to fig. 2, fig. 2 is a schematic hardware structure diagram of another embodiment of a data missing value filling system based on a GRU model in an embodiment of the present invention, and as shown in fig. 2, the system 10 includes: a memory 101, a processor 102 and a computer program stored on the memory and executable on the processor, the computer program realizing the following steps when executed by the processor 101:
acquiring sensor data samples of the livestock based on ear tag sensors of the livestock;
training a pre-constructed GRU model based on a sensor data sample to generate a target GRU model;
and acquiring sensor data to be filled, and inputting the sensor data to be filled into the target GRU model to obtain a data missing value filling result.
The specific implementation steps are the same as those of the method embodiments, and are not described herein again.
Optionally, the computer program when executed by the processor 101 further realizes the steps of:
acquiring real-time sensor data of the livestock based on ear tag sensors of the livestock, wherein the real-time data of the livestock comprises temperature data, motion data and environment temperature data of the livestock;
mapping the real-time data of the livestock and the identification numbers of the livestock and then storing the mapping data to a time sequence database;
when a sensor data sample generation instruction is detected, acquiring an identification number of the livestock based on the ear tag number of the livestock;
reading sensor data corresponding to the livestock in the time sequence database according to the identification number of the livestock;
extracting sensor data according to the time steps, converting the extracted data format into a sample format of a GRU model, and generating an initial sensor data sample;
and carrying out normalization processing on the initial sensor data sample to generate a sensor data sample.
The specific implementation steps are the same as those of the method embodiments, and are not described herein again.
Optionally, the computer program when executed by the processor 101 further implements the steps of:
and constructing an initial GRU model, and adding a full connection layer on an output layer of the initial GRU model to generate a pre-constructed GRU model.
The specific implementation steps are the same as those of the method embodiments, and are not described herein again.
Optionally, the computer program when executed by the processor 101 further implements the steps of:
segmenting a sensor data sample to generate a training set, a testing set and a verification set;
inputting a training set into a pre-constructed GRU model for training;
optimizing model parameters through a predefined label and a loss function in the training process;
and after the optimization is completed, generating a target GRU model and storing the target GRU model.
The specific implementation steps are the same as those of the method embodiments, and are not described herein again.
Optionally, the computer program when executed by the processor 101 further implements the steps of:
acquiring sensor data to be filled, and extracting data missing values and non-missing values in the sensor data;
extracting time sequence data of non-missing values of a preset length before the missing value as input data, and carrying out normalization processing on the input data and inputting the input data into a GRU model;
and acquiring an output result of the GRU model, and performing inverse normalization processing on the output result to obtain a prediction result of the data deficiency value.
The specific implementation steps are the same as those of the method embodiments, and are not described herein again.
Embodiments of the present invention provide a non-transitory computer-readable storage medium storing computer-executable instructions for execution by one or more processors, for example, to perform steps S100-S300 of the method of fig. 1 described above.
By way of example, nonvolatile storage media can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically erasable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM), which acts as external cache memory. By way of illustration and not limitation, RAM is available in many forms such as Synchronous RAM (SRAM), dynamic RAM, (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The disclosed memory components or memory of the operating environment described in embodiments of the invention are intended to comprise one or more of these and/or any other suitable types of memory.
The above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. A data missing value filling method based on a GRU model is characterized by comprising the following steps:
acquiring sensor data samples of the livestock based on ear tag sensors of the livestock;
training a pre-constructed GRU model based on a sensor data sample to generate a target GRU model;
and acquiring sensor data to be filled, and inputting the sensor data to be filled into the target GRU model to obtain a data missing value filling result.
2. The GRU model-based data deficiency value filling method according to claim 1, wherein the livestock-based earmark sensor obtaining sensor data samples of livestock comprises:
acquiring real-time sensor data of livestock based on ear tag sensors of the livestock, wherein the real-time data of the livestock comprises temperature data, motion data and environmental temperature data of the livestock;
mapping the real-time data of the livestock and the identification numbers of the livestock and then storing the mapping data to a time sequence database;
when a sensor data sample generation instruction is detected, acquiring an identification number of the livestock based on the ear tag number of the livestock;
reading sensor data corresponding to the livestock in the time sequence database according to the identification number of the livestock;
extracting sensor data according to the time step number, converting the extracted data format into a sample format of a GRU model, and generating an initial sensor data sample;
and carrying out normalization processing on the initial sensor data sample to generate a sensor data sample.
3. The method of claim 2, wherein training a pre-constructed GRU model based on sensor data samples prior to generating a target GRU model comprises:
and constructing an initial GRU model, and adding a full connection layer on an output layer of the initial GRU model to generate a pre-constructed GRU model.
4. The method of claim 3, wherein training a pre-constructed GRU model based on sensor data samples to generate a target GRU model comprises:
segmenting a sensor data sample to generate a training set, a testing set and a verification set;
inputting a training set into a pre-constructed GRU model for training;
optimizing model parameters through a predefined label and a loss function in the training process;
and after the optimization is completed, generating a target GRU model, and storing the target GRU model.
5. The method of claim 4, wherein the obtaining sensor data to be filled and inputting the sensor data to be filled into the target GRU model to obtain a data missing value filling result comprises:
acquiring sensor data to be filled, and extracting data missing values and non-missing values in the sensor data;
extracting time sequence data of non-missing values of a preset length before the missing value as input data, and carrying out normalization processing on the input data and inputting the input data into a GRU model;
and acquiring an output result of the GRU model, and performing inverse normalization processing on the output result to obtain a prediction result of the data deficiency value.
6. A data deficiency value population system based on a GRU model, the system comprising: a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program when executed by the processor implementing the steps of:
acquiring sensor data samples of the livestock based on ear tag sensors of the livestock;
training a pre-constructed GRU model based on a sensor data sample to generate a target GRU model;
and acquiring sensor data to be filled, and inputting the sensor data to be filled into the target GRU model to obtain a data missing value filling result.
7. The GRU model-based data deficiency value population system of claim 6, wherein the computer program, when executed by the processor, further performs the steps of:
acquiring real-time sensor data of livestock based on ear tag sensors of the livestock, wherein the real-time data of the livestock comprises temperature data, motion data and environmental temperature data of the livestock;
mapping the real-time data of the livestock and the identification numbers of the livestock and then storing the mapping data to a time sequence database;
when a sensor data sample generation instruction is detected, acquiring an identification number of the livestock based on the ear tag number of the livestock;
reading sensor data corresponding to the livestock in the time sequence database according to the identification number of the livestock;
extracting sensor data according to the time step number, converting the extracted data format into a sample format of a GRU model, and generating an initial sensor data sample;
and carrying out normalization processing on the initial sensor data sample to generate a sensor data sample.
8. The GRU model-based data deficiency value population system of claim 7, wherein the computer program when executed by the processor further implements the steps of:
and constructing an initial GRU model, and adding a full connection layer on an output layer of the initial GRU model to generate a pre-constructed GRU model.
9. The GRU model-based data deficiency value population system of claim 8, wherein the computer program when executed by the processor further performs the steps of:
segmenting a sensor data sample to generate a training set, a testing set and a verification set;
inputting a training set into a pre-constructed GRU model for training;
optimizing model parameters through a predefined label and a loss function in the training process;
and after the optimization is completed, generating a target GRU model, and storing the target GRU model.
10. A non-transitory computer-readable storage medium storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform the GRU model-based data deficiency value population method of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211355037.7A CN115757372A (en) | 2022-11-01 | 2022-11-01 | Data missing value filling method and system based on GRU model |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211355037.7A CN115757372A (en) | 2022-11-01 | 2022-11-01 | Data missing value filling method and system based on GRU model |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115757372A true CN115757372A (en) | 2023-03-07 |
Family
ID=85354985
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211355037.7A Pending CN115757372A (en) | 2022-11-01 | 2022-11-01 | Data missing value filling method and system based on GRU model |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115757372A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117435870A (en) * | 2023-12-21 | 2024-01-23 | 国网天津市电力公司营销服务中心 | Load data real-time filling method, system, equipment and medium |
-
2022
- 2022-11-01 CN CN202211355037.7A patent/CN115757372A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117435870A (en) * | 2023-12-21 | 2024-01-23 | 国网天津市电力公司营销服务中心 | Load data real-time filling method, system, equipment and medium |
| CN117435870B (en) * | 2023-12-21 | 2024-03-29 | 国网天津市电力公司营销服务中心 | Load data real-time filling method, system, equipment and medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Bao et al. | Artificial intelligence in animal farming: A systematic literature review | |
| CN109816221B (en) | Project risk decision method, apparatus, computer device and storage medium | |
| CN102183621B (en) | Aquaculture dissolved oxygen concentration online forecasting method and system | |
| US11650968B2 (en) | Systems and methods for predictive early stopping in neural network training | |
| AU2017283867B2 (en) | Information processing device, method, and program | |
| US10948312B2 (en) | Environmental monitoring systems, methods and media | |
| WO2023108987A1 (en) | Risk prediction method and apparatus based on reinforcement learning, and device and storage medium | |
| CN109918687B (en) | Train dynamics simulation method and simulation platform based on machine learning | |
| CN112365033B (en) | Wind power interval prediction method, system and storage medium | |
| CN115757372A (en) | Data missing value filling method and system based on GRU model | |
| CN115545334B (en) | Land utilization type prediction method and device, electronic equipment and storage medium | |
| Zhang et al. | Prediction on the highest price of the stock based on PSO-LSTM neural network | |
| Wu | MOOC learning behavior analysis and teaching intelligent decision support method based on improved decision tree C4. 5 algorithm | |
| Kumar et al. | Stock Price Prediction Using Machine Learning | |
| CN115796001A (en) | Data missing value prediction method and system based on seq2seq | |
| CN111290953A (en) | Method and device for analyzing test logs | |
| CN111524023A (en) | Greenhouse adjusting method and system | |
| CN116757313A (en) | Training method of water quality prediction model, water quality prediction method and device | |
| CN114418182B (en) | Beef cattle breeding optimization method and device based on machine learning | |
| CN115293249A (en) | Power system typical scene probability prediction method based on dynamic time sequence prediction | |
| CN113361768A (en) | Grain depot health condition prediction method, storage device and server | |
| KR102330859B1 (en) | Method, apparatus and system for providing pet fecal image analysis information | |
| Chen et al. | A prediction model for piggery ammonia concentration based on least squares support vector regression using fruit fly optimisation algorithm | |
| Tao et al. | Research on Time-Series-Data-Prediction Based on Neural Network Technology | |
| CN117521502A (en) | Wind power plant life prediction method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |