KR20200036219A - Method for predicting price of agricultural product and sales volume using Long Short-Term Memory - Google Patents

Abstract

A method for predicting farm prices and sales of farm products of the present invention in a device for predicting farm prices and sales of farm products comprises the steps of: processing inputted data when the data related to farm prices and sales of farm products is inputted; generating a long short-term memory model (LSTM) network; learning the data processed in the LSTM network; and predicting the farm prices and the sales of the farm products using the learned data. According to the present invention, by predicting the farm prices and the sales of the farm products using a LSTM, the farm prices and the sales of the farm products may be more easily and accurately predicted.

Description

Method for predicting price of agricultural product and sales volume using Long Short-Term Memory}

The present invention relates to predicting the price and sales of agricultural products, and more particularly, to predicting the price and sales of agricultural products using LSTM.

The agricultural product price forecast is provided by the Agricultural Research Center of the Korea Rural Economic Research Institute, which provides information on the transaction price and import trend of the agricultural products imported into a specific agricultural product wholesale center in the last 3 months, or provides wholesale price and import information for the last 10 days. However, the actual wholesale price and import information for the last 10 days can be checked only 2-3 information from a week ago when searching for the information by today's date.

As described above, the existing agricultural product price prediction system only provides information on the previous agricultural product price trend or the import quantity trend, and fails to predict the agricultural product price tomorrow or a week later or to provide the predicted price information in real time online. .

In particular, not only is it difficult for all processes from sowing to harvesting to proceed seamlessly according to the original plan, and agricultural products are often subject to irresistible situations that cannot be controlled artificially, unlike the production of agricultural products. However, depending on the supply and demand, the role of farmers or agricultural products distribution centers can be large in determining that farmers can sell agricultural products at an advantageous time. However, since the prices of agricultural products fluctuate in real time, it is not easy for farmers or local officials of agricultural products distribution centers to predict the prices of agricultural products tomorrow or a week later to determine when to ship agricultural products.

Conventional agricultural product information was collected monthly through telephone surveys by monitor agents commissioned by item and region (local agricultural cooperatives, agricultural technology center employees, etc.), and was used to identify production and transaction trends in mountain regions. Because it cannot be collected quickly and accurately due to the movement of departments, frequent absences due to work in rural fields, and performance of tasks other than monitoring activities, the collected agricultural products production area information is used as the basic information for grasping production and trading trends There is.

In addition, the importance of the agricultural observation information support service has been increasing since 1999, but the conventional agricultural observation information support system that does this simply stores or provides the survey results (agricultural observation information) in a database, and the stored survey There is a limitation in that it is not possible to provide an analysis result service that performs time series analysis using the result.

As such, the area of consumption of agricultural products is not only targeted to consumers of various agricultural products with unique characteristics, but is also influenced by various media such as unpredictable consumption trends, social atmosphere, events, and accidents. Therefore, it can be seen that observation of consumption for agricultural products is difficult to perform and accuracy is not guaranteed.

Republic of Korea Registered Patent 10-1764147

The present invention has been devised to solve the above problems, and a method for predicting agricultural products prices and sales volumes using Long Short-Term Memory (LSTM) to more accurately and quickly predict agricultural products prices and sales volumes. The purpose is to provide. That is, in the present invention, the purpose of the present invention is to predict the future consumption price by analyzing the consumption pattern of agri-food through deep learning using the collected structured and unstructured data (SNS data, broadcast data). .

The objects of the present invention are not limited to the above-mentioned objects, and other objects not mentioned will be clearly understood by those skilled in the art from the following description.

In the method for predicting the price and sales of agricultural products in the apparatus for predicting the price and sales of agricultural products of the present invention for achieving the above object, when data related to the prices and sales of agricultural products are input, processing the input data, LSTM (Long Short-Term Memory model) creating a network, learning the processed data in the LSTM network, and predicting the agricultural product price and sales using the learned data.

In the step of processing the input data, normalizing the input data for each column, generating a data sequence for the input and a data sequence for the output to train the normalized data, and a learning dataset to be learned. It may include (dataset), and generating a test dataset for checking the learned content.

In the step of learning the processed data in the LSTM network, the learning dataset may be trained using the LSTM network.

The data related to the agricultural product price and sales amount is divided into structured data and unstructured data, and the structured data includes a predetermined wholesale price, retail price, sales amount, and sales amount of the agricultural product, and the unstructured data is an SNS in which the predefined agricultural product is mentioned, News, broadcast data may be included.

According to the present invention, by predicting the price and sales of agricultural products by using LSTM (Long Short-Term Memory), it is possible to more easily and accurately predict the price and sales of agricultural products.

In addition, the present invention has the effect that by predicting the unpredictable prices and sales of agricultural products, it helps farmers and consumers to make reasonable decisions, and socially efficient resource allocation.

1 is a flowchart showing a method of predicting agricultural product prices and sales volume using LSTM according to an embodiment of the present invention.
2 is a diagram illustrating a set of data used to predict the price and sales of agricultural products according to an embodiment of the present invention.
FIG. 3 is a diagram illustrating a data set used to derive prediction data of a purchase amount based on panel data according to an embodiment of the present invention.
4 is a diagram illustrating a data set used to derive prediction data of a wholesale market price amount according to an embodiment of the present invention.
5 is a diagram illustrating a data set used to derive prediction data of wholesale market imports according to an embodiment of the present invention.
6 illustrates a data normalization method for predicting wholesale market consumption price according to an embodiment of the present invention.
7 is a hyperparameter for predicting wholesale market consumption price according to an embodiment of the present invention.
8 and 9 are diagrams illustrating an LSTM network configuration according to an embodiment of the present invention.
10 is a diagram illustrating an LSTM according to an embodiment of the present invention.
11 illustrates an LSTM learning method according to an embodiment of the present invention.
12 and 13 are graphs showing results of wholesale market consumption price prediction using LSTM according to an embodiment of the present invention.

The present invention can be applied to various changes and can have various embodiments, and specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention.

The terms used in this application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, terms such as “include” or “have” are intended to indicate that a feature, number, step, operation, component, part, or combination thereof described on the specification exists, and that one or more other features are present. It should be understood that the existence or addition possibilities of fields or numbers, steps, operations, components, parts or combinations thereof are not excluded in advance.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person skilled in the art to which the present invention pertains. Terms, such as those defined in a commonly used dictionary, should be interpreted as having meanings consistent with meanings in the context of related technologies, and should not be interpreted as ideal or excessively formal meanings unless explicitly defined in the present application. Does not.

In addition, in the description with reference to the accompanying drawings, the same components are given the same reference numerals regardless of the reference numerals, and redundant descriptions thereof will be omitted. In the description of the present invention, when it is determined that detailed descriptions of related known technologies may unnecessarily obscure the subject matter of the present invention, detailed descriptions thereof will be omitted.

The present invention relates to a method for predicting agricultural product prices and sales volume.

The subject that performs the method for predicting the price and sales of agricultural products of the present invention may be referred to as a computer device for performing the method for predicting the price and sales of agricultural products, or a control unit or processor for overall controlling the system or device for performing the price and sales of agricultural products (processor). That is, the method for predicting agricultural product price and sales amount of the present invention may be composed of an algorithm which is a kind of software, and the software may be executed in a controller or processor of a system, apparatus for performing a method for predicting agricultural product price and sales amount.

1 is a flowchart showing a method of predicting agricultural product prices and sales volume using LSTM according to an embodiment of the present invention.

Referring to FIG. 1, in the method for predicting agricultural product price and sales amount of the present invention, when data on a specific agricultural product price and sales amount is input, the input data is processed (S110, S120, S130). In the present invention, data on prices and sales of agricultural products can be divided into structured data and unstructured data. The structured data is data such as wholesale price, retail price, sales amount, and sales amount of a specific agricultural product, and the unstructured data is data such as SNS, news, broadcast, etc. in which a specific agricultural product is mentioned.

Next, a long short-term memory model (LSTM) network is generated (S140).

Then, learning is performed on the processed data in the LSTM network (S150).

And, using the learned data to predict the price of agricultural products and sales data (S160).

The step of processing the input data (S110, S120, S130), normalizing the input data for each column (S110), a data sequence for input and a data sequence for output to train the normalized data (sequence) ) (S120) and a training dataset to be learned (training dataset) and a test dataset (testing dataset) to check the learned content (S130).

And, the training dataset can be trained using the LSTM network.

In the present invention, normalizing the data (min-max scaling) is to prevent data from being too small or too large, preventing learning, and using a minimum value and a maximum value, X is a value between 0 and 1 Convert to, and apply to each column.

In the present invention, the price and sales of agricultural products can be predicted using LSTM Recurrent Neural Network (RNN).

10 is a diagram illustrating an LSTM according to an embodiment of the present invention.

Referring to FIG. 10, LSTM (Long Short-Term Memory models) is a representative improvement model that alleviates the slope disappearance problem of RNN by a modification of the existing RNN model. This model includes a memory cell in a hidden node, which is a gate that stores values in the storage, outputs them, and adjusts the forgetting value.

The LSTM is composed of an input gate for an input value, an output gate for an output value, and a forget gate for an oblivion value in addition to the memory cell. The input / output parameters for each gate are as follows.

 The learning algorithm of LSTM uses Backpropagation in the same way as RNN, the input data is sequence data, and the learning method is as follows.

11 illustrates an LSTM learning method according to an embodiment of the present invention.

Referring to FIG. 11, in LSTM RNN, input data is sequence data, weight U _i , W _i for an input gate, weight U _o , W _o , V _o for an output gate, weight U _f , W _f for an oblivion gate , Weight U _c , W _c for the memory cell, and the deviation term b _i , b _o , b _f , b _c of each gate are used, and the output data is represented by the output value of the LSTM.

And learning continues until the last point of input data and the contents are as follows. First, the input value of the current point and the value of the previous hidden layer are multiplied by the weight to add a deviation term to create an i _t value, and the value obtained from the previous hidden layer is obtained by applying the method of the input gate using the hyperbolic tangent to obtain a _t . After that, the value of the forgetting gate f _t is obtained through the sigmoid function, and c _t is determined which value to use with the result of the newly input value and the previous output value in the memory cell. Then, the sum of the input value of the current point, the value of the previous hidden layer and the weight of the value of the previous state storage and the deviation term are added to obtain the output gate value of o _t using the sigmoid function, and then the value of o _t and c _The final predicted value is output using the hyperbolic tangent result of _t .

To determine the gradient in the learning of LSTM RNN, the partial differentials for each gate of the error function E and the kth element in the internal state are calculated as follows.

[Equation 1]

And, the gradient delivered to the t-1 step is as follows.

[Equation 2]

And, the gradient delivered to the p step is as follows.

[Equation 3]

이 곱해지는 것보다 기울기 소멸 문제에서 유리하다.Here, the gradient delivered to the p step is in the form of multiplying the value of the oblivion gate by p as follows. In addition, since the forgetting gate value is a result of applying sigmoid, it is a value between the intervals (0,1).

This multiplication is more advantageous in the disappearance of slope problem.

Since the LSTM model includes three gates, the number of weights and deviation terms is about 4 times that of a typical RNN training, which takes longer to execute and learn than the RNN model, but it reduces the problem of gradient disappearance and provides more accurate results. Can be obtained.

Referring to FIG. 11, input data is data related to agricultural product price and sales amount as sequence data, weight U _i , W _i for an input gate, weight U _o , W _o , V _o for an output gate, and an oblivion gate The weight U _f , W _f , the weight U _c , W _c for the memory cell, and the deviation term b _i , b _o , b _f , b _c of each gate are used, and the output data is the output value of the LSTM. Is the next predicted value.

The prediction method is as follows.

First, the data to be predicted is normalized, and data sequences for input and output are generated to train the normalized data. Next, we create a training dataset for learning and a test dataset for checking the learned content. And, in creating an LSTM network network, in the present invention, LSTM cells are defined using various parameters such as the output size of each cell and forget bias, and then stacked RNNs wrapped with the number of layers of stacked layers are generated, and these LSTM cells are generated. Connect and configure. Since the connection configuration method predicts one data of the next day using the data of the past date, it is implemented in many-to-one form. Using this implemented LSTM network, we train the training dataset created earlier. Then, by using the learned contents, we predict the price and sales of agricultural products to be predicted the next day.

In one embodiment of the present invention, the actual experiment was conducted in the following manner.

In the present invention, an experiment was conducted to predict the wholesale market consumption price of onions among agricultural products, and the LSTM RNN method using weekly data related to the wholesale market consumption price period from January 1, 2014 to August 31, 2014 It was learned using. And, the deep learning library used tensorflow and proceeded in GPU mode. For data learning, a total of 10 epochs were learned for the input data, and it was found that it took about 20 minutes. The hyperparameters for the experiment of predicting the consumption price of the wholesale market are as follows.

7 is a hyperparameter for predicting wholesale market consumption price according to an embodiment of the present invention.

In FIG. 7, input_data_column_cnt = 31 # number of columns of input data (variable number), output_data_column_cnt = 1 # number of columns of result data, seq_length = 2 # length of 1 sequence (number of time series data input), rnn_cell_hidden_dim = 20 # (Hidden) output size of each cell, forget_bias = 0.5 # forgetting bias (default 1.0), num_stacked_layers = 1 # number of stacked LSTM layers, keep_prob = 1.0 # ratio to keep when dropout, epoch_num = 1000 # number of epochs (Enter the number of times to learn the entire data for learning repeatedly), learning_rate = 0.01 # Hyperparameters of learning rate are illustrated.

2 is a diagram illustrating a set of data used to predict the price and sales of agricultural products according to an embodiment of the present invention.

In the example of Fig. 2, column names are defined as follows.

FIG. 3 is a diagram illustrating a data set used to derive prediction data of a purchase amount based on panel data according to an embodiment of the present invention.

In the embodiment of Fig. 3, usage data is as follows.

Using these data, the following prediction data were calculated.

4 is a diagram illustrating a data set used to derive prediction data of a wholesale market price amount according to an embodiment of the present invention.

Usage data in the embodiment of Figure 4 is as follows.

Using these data, the following prediction data were calculated.

That is, in FIG. 4, the wholesale market price is predicted by using the onion purchase amount based on the panel data predicted in FIG. 3 as usage data.

5 is a diagram illustrating a data set used to derive prediction data of wholesale market imports according to an embodiment of the present invention.

The usage data in the embodiment of FIG. 5 is as follows.

Using these data, the following prediction data were calculated.

That is, in FIG. 5, the wholesale market import amount is predicted by using the wholesale market price predicted in FIG. 4 as usage data.

In the present invention, the reason for normalizing the data (min-max scaling) is to prevent the data from being too small or too large, preventing learning, and using the minimum and maximum values between X and 0 to 1. Convert to value and apply to each column

6 illustrates a data normalization method for predicting wholesale market consumption price according to an embodiment of the present invention.

In FIG. 6, data normalization for the panel_purchase_amount column is illustrated.

8 and 9 are diagrams illustrating an LSTM network configuration according to an embodiment of the present invention.

Referring to FIG. 8, in the LSTM network according to an embodiment of the present invention, LSTM cells are generated, but the output size of each cell is 20, forget_bias is 0.5, and state_is_tuple is True (the received and returned state is c_state and 2-tuples of m_state). Then, stacked RNNs wrapped with Num_stacked_layers layers are created and LSTM cells are connected.

Since the LSTM network of the present invention predicts one data of the next day using the data of the past date, it is preferable that it is a Many-to-one form.

Referring to FIG. 9, it can be seen that in the LSTM network, two sequences are input from the input layer to the hidden layer, and sequentially transmitted from the hidden layer to output the output value.

12 and 13 are graphs showing results of wholesale market consumption price prediction using LSTM according to an embodiment of the present invention.

The graph of FIG. 12 is a result of learning and predicting weekly data related to wholesale market consumption prices for a period from January 1, 2014 to August 31, 2014, from September 1, 2014 to 2014 The results are compared with weekly test data until December 31, 2016.

13 is a graph in which the normalized value is returned to the original value by using the Reverse_min_max_scaling function in the graph of FIG. 12. For reference, in the Reverse_min_max_scaling function, if you enter the x value of the original before normalization and the x value you want to return, the normalized value is returned.

In Figs. 12 and 13, red indicates current data, and blue indicates predicted data.

Referring to FIGS. 12 and 13, it can be seen that as a result of comparing the predicted data using the LSTM of the present invention with the method for predicting the price and sales of agricultural products with actual data, there is little difference.

Therefore, if the method for predicting the price and sales of agricultural products proposed by the present invention is used, it can be expected that a more efficient plan can be established for agricultural producers and consumers based on the predicted results, and that socially efficient resource allocation is possible.

Although the present invention has been described using several preferred embodiments, these embodiments are illustrative and not limiting. Those skilled in the art to which the present invention pertains will understand that various changes and modifications can be made without departing from the spirit of the present invention and the scope of the rights set forth in the appended claims.

Claims (4)

In the method for predicting agricultural product prices and sales in a device for predicting agricultural product prices and sales,
If data related to agricultural product prices and sales volumes are input, processing the input data;
Creating a long short-term memory model (LSTM) network;
Progressing learning on the processed data in the LSTM network; And
And predicting the agricultural product price and sales amount using the learned data.
The method according to claim 1,
In the step of processing the input data,
Normalizing the input data for each column;
Generating a data sequence for the input and a data sequence for the output to train the normalized data; And
A method for predicting agricultural product prices and sales, comprising the step of generating a dataset for learning to learn and a test dataset for checking the learned content.
The method according to claim 2,
In the process of learning the data processed in the LSTM network,
A method for predicting agricultural product prices and sales, characterized by learning the dataset for learning using the LSTM network.
The method according to claim 1,
Data related to the price and sales of agricultural products are divided into structured data and unstructured data,
The structured data includes the wholesale price, retail price, sales volume, and sales amount of the predetermined agricultural products,
The unstructured data is a method for predicting agricultural product prices and sales, characterized in that the predetermined agricultural products include mentioned SNS, news, and broadcast data.

Images