CN116977035A - Agricultural product recommendation method based on LightGBM and deep learning - Google Patents
Agricultural product recommendation method based on LightGBM and deep learning Download PDFInfo
- Publication number
- CN116977035A CN116977035A CN202311235697.6A CN202311235697A CN116977035A CN 116977035 A CN116977035 A CN 116977035A CN 202311235697 A CN202311235697 A CN 202311235697A CN 116977035 A CN116977035 A CN 116977035A
- Authority
- CN
- China
- Prior art keywords
- agricultural product
- data set
- product data
- lightgbm
- neural network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 238000013135 deep learning Methods 0.000 title claims abstract description 17
- 238000013528 artificial neural network Methods 0.000 claims abstract description 42
- 238000010606 normalization Methods 0.000 claims abstract description 15
- 238000007781 pre-processing Methods 0.000 claims abstract description 6
- 239000013598 vector Substances 0.000 claims description 17
- 230000006870 function Effects 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 15
- 238000005065 mining Methods 0.000 claims description 8
- 230000004927 fusion Effects 0.000 claims description 6
- 238000005457 optimization Methods 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 4
- 230000004913 activation Effects 0.000 claims description 3
- 230000008569 process Effects 0.000 claims description 3
- 230000011218 segmentation Effects 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 238000009825 accumulation Methods 0.000 claims description 2
- 238000006243 chemical reaction Methods 0.000 claims description 2
- 238000000586 desensitisation Methods 0.000 claims description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 12
- 230000008859 change Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 230000015556 catabolic process Effects 0.000 description 2
- 238000006731 degradation reaction Methods 0.000 description 2
- 210000002569 neuron Anatomy 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000003066 decision tree Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005429 filling process Methods 0.000 description 1
- 235000013305 food Nutrition 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/0601—Electronic shopping [e-shopping]
- G06Q30/0631—Item recommendations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/211—Selection of the most significant subset of features
- G06F18/2113—Selection of the most significant subset of features by ranking or filtering the set of features, e.g. using a measure of variance or of feature cross-correlation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Business, Economics & Management (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Accounting & Taxation (AREA)
- Computing Systems (AREA)
- Finance (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Marketing (AREA)
- General Business, Economics & Management (AREA)
- Strategic Management (AREA)
- Economics (AREA)
- Development Economics (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The invention discloses an agricultural product recommendation method based on LightGBM and deep learning, wherein the method comprises the following steps: preprocessing agricultural product data information through a linear normalization method, tag coding and the like; the characteristics in the agricultural product data information are fused and extracted by utilizing the LightGBM, and the characteristics with highest classification accuracy are selected as the input of the neural network layer; then, respectively acquiring a linear cross combination characteristic relation and a nonlinear association relation between the agricultural product data characteristics by using a cross network and a deep neural network; and finally, carrying out linear weighted combination on the results obtained by the preamble network to obtain the final agricultural product recommendation result. The invention adopts the lightGBM, the cross network and the deep neural network to mine the characteristic relation among the agricultural product data, so as to solve the sparsity problem of the agricultural product data; and valuable information is mined from the agricultural product information according to user preference to conduct intelligent recommendation, so that consumers can select agricultural products meeting own requirements more conveniently.
Description
Technical Field
The invention belongs to the field of data analysis and intelligent recommendation, and particularly relates to an agricultural product recommendation method based on LightGBM and deep learning.
Background
With the improvement of the living standard of people and the change of consumption concepts, more and more people begin to pay attention to the health, quality and safety of foods, and the requirements on agricultural products are also higher and higher. However, the conventional shopping manner often requires a great deal of time and effort for consumers to find suitable agricultural products, and often cannot guarantee the quality and safety of the agricultural products. Therefore, the agricultural product recommendation method can help consumers to select the agricultural products meeting the demands more conveniently, and can help agricultural product producers and sellers to promote the products more accurately, so that sales and customer satisfaction are improved.
The LightGBM is a high-efficiency and distributed gradient lifting frame and has the characteristics of high efficiency, high accuracy, low memory occupation and the like; in the agricultural product recommendation method, the LightGBM can be used for extracting the characteristics of users and commodities, constructing characteristic engineering and carrying out quick and efficient recommendation calculation. The deep learning technology has strong learning ability and is widely applied to the fields of computer vision, voice recognition, natural language processing and the like, the deep learning technology is applied to a recommendation method, so that the deep learning technology has more efficient learning ability, hidden relations among features are mined by using a neural network, and the recommendation accuracy can be improved.
Disclosure of Invention
The invention aims to provide an agricultural product recommendation method based on LightGBM and deep learning, which better recommends interesting agricultural products for users, thereby improving the purchase willingness of the users and further improving the sales of the agricultural products.
The invention provides an agricultural product recommendation method based on LightGBM and deep learning, which comprises the following steps:
step S1, preprocessing data in an agricultural product data set to enable a recommendation model to obtain basic data for feature processing to obtain an agricultural product data set B;
s2, taking the agricultural product data set B as input data of a LightGBM module, performing fusion optimization on features of the agricultural product data B through the LightGBM module, selecting features with highest classification accuracy as feature results, and obtaining effective integer result leaf vectors;
step S3, splicing all leaf node index values generated in the LightGBM module to form a new discrete agricultural product data set T, and splicing the new discrete agricultural product data set T with the agricultural product data set B to obtain a latest agricultural product data set V;
s4, taking the agricultural product data set V as input data of a neural network layer, and mining hidden relations of higher-order features in the data set through a cross network and a deep neural network;
and S5, carrying out linear weighted combination on the results obtained by the neural network layer, calculating by the full-connection layer, and obtaining the agricultural product recommendation result after the function is activated.
Further, in the step S1, in the input layer of the agricultural product recommendation model, a reasonable numerical filling process is performed on the blank value part in the agricultural product data set as follows:
considering that the data set is subjected to desensitization treatment, filling sparse data and blank values of the discrete data in the data set to be numerical values or characters which do not exist in the current data column;
carrying out normalization processing on sparse data in the agricultural product data set without the null value at a model input layer, and carrying out tag coding on the discrete data;
the sparse data normalization processing in the data set adopts a linear normalization method to perform linear transformation on the sparse data, and the result is mapped into the [0,1] interval after linear normalization.
Wherein the linear normalization method is expressed as follows:
（1）
wherein, the liquid crystal display device comprises a liquid crystal display device,as a result of the data maximum value,as a result of the minimum value of the data,in the event of a data value being a data value,data values obtained for linear normalization;
and performing discretization processing on the discrete data, and performing label conversion through a LabelEncoder to finally form a full-value agricultural product data set B without a blank value.
Further, the specific implementation manner of step S2 is as follows:
inputting the full-digital agricultural product data set B obtained after preprocessing of an input layer into a LightGBM module of a model, and performing fusion optimization on the characteristic relevance of the agricultural product data set B by utilizing a histogram algorithm in the LightGBM module to convert each characteristic column in the agricultural product data set B into a histogram; the characteristic data in the agricultural product data set B are equally divided into K bins, an integer is allocated to each bin, then the data value in the bin is replaced by the integer allocated to the bin, meanwhile, a histogram with the width of K is formed according to the integer of the bin, statistics and accumulation are carried out on the histogram when the data are traversed, after the data are traversed once, corresponding statistics are accumulated on the histogram, and then the optimal segmentation points are traversed and found according to the discrete value of the histogram.
The method comprises the steps of reducing the number of samples in a data set B by using a single-side gradient sampling algorithm GOSS in a LightGBM module, removing most small gradient samples in a farm data set B, selecting a feature with highest classification accuracy as a feature result, and obtaining an effective integer result leaf vector; all values of features to be split in the agricultural product data set B are ordered in a descending order according to the absolute value, m ∗ data with the largest absolute value are selected, and n ∗ data are randomly selected from the rest smaller gradient data; then randomly selecting n ∗% of data multiplied by a constant (1-m)/n, so that the algorithm can pay more attention to the undertraining samples and cannot generate excessive change on the distribution of the data set B; finally, using (m+n) ∗% data to calculate information gain;
wherein the LightGBM model objective function formula is as follows:
（2）
（3）
the formula of the second-order Taylor expansion result of the function is as follows:
（4）
wherein, the liquid crystal display device comprises a liquid crystal display device,as the true value of the tag,as a result of the k-1 th learning,for the regularized term sums of the first k-1 trees,is a regular term of the kth tree;as a function of the object to be processed,training errors for samples, meaning finding a suitable treeThe value of the function is minimized.
Further, the specific representation of step S3 is as follows:
after the processing of the LightGBM module is completed, the agricultural product data set B generates leaf node index values, and then all the generated leaf node index values are spliced to form a new discrete data set T, so that the sparsity problem of the agricultural product data set B is relieved by the data set T;
combining and splicing the agricultural product data set B and the agricultural product data set T to obtain an agricultural product data set V;
the agricultural product data set V enhances the effectiveness and the interpretability of the original data set, improves the utilization rate of the data characteristics, and enhances the accuracy of the recommendation model.
Further, the neural network layer processing procedure in the agricultural product recommendation model in step S4 is as follows:
the neural network layer in the agricultural product recommendation model is formed by combining a cross network and a deep neural network in parallel, an input part adopts an input sharing principle, and each neural network part processes the same data set so that an output part has relevance and connectivity;
the cross network of the agricultural product recommendation model is responsible for mining the linear cross combination relation between the higher-order features in the agricultural product data set V; the cross network consists of cross layers, wherein each layer of input data can cross with the current layer of data, and linear cross combination characteristics in the data are extracted; to be used forLayer for example, the agricultural product dataset V is characterized by crossing at the crossing layer, after one is completedThe layer will add its input back again and solve the problem of network performance degradation by using the residual idea; the crossover network can automatically learn the crossover characteristics of limited high orders and the weight parameters corresponding to the crossover characteristics, the degree of the crossover characteristics increases with the deepening of the layer depth, the time complexity and the space complexity of the crossover network linearly increase with the increase of the input dimension, and the crossover network has better generalization capability.
First, theLayer outputThe result formula is shown below:
（5）
the output result formula of the Cross network is as follows:
（6）
where x represents input data, b represents Bias,andrepresent the firstLayer and the firstThe output of the layer Cross layer,andrepresenting the connection parameters between the two layers, d representing the dimension of the feature,representing the vector formed by the superposition of the embedded vector and the continuous feature vector.
The deep neural network of the agricultural product recommendation model is responsible for mining nonlinear association relations between higher-order features in the agricultural product data set V; the deep neural network consists of an input layer, an output layer and a hidden layer, is a fully connected neural network, each data characteristic can implicitly interact with other characteristics, and can extract nonlinear high-order characteristics in a data set.
In the deep neural network, the input layers take discrete feature vectors in the agricultural product data set V as input, the neuron of each input layer receives the features in one discrete feature vector, and in the fully-connected hidden layer, the feature information is continuously transmitted; and each characteristic is interactively combined, and finally, nonlinear high-order characteristics are extracted by an output layer.
The output formula is as follows:
（7）
wherein, the liquid crystal display device comprises a liquid crystal display device,representing a non-linear transfer function and,and b is a parameter, and x is an output value of the previous layer.
Further, the specific representation of step S5 is as follows:
the agricultural product recommendation model adopts equal weight weighting, the output results of the cross network and the deep neural network are subjected to linear weighting combination, the spliced result is calculated through a full-connection layer, and the recommendation result of the agricultural product is obtained after the Sigmoid activation function acts.
The output formula is shown in the following 8.
（8）
Compared with the prior art, the invention has the advantages that: the agricultural product recommendation method based on the LightGBM and the deep learning can further excavate the characteristic relation between the agricultural product data information and solve the data sparsity problem, transform, fuse and extract the characteristics of the agricultural products by utilizing the gradient lifting decision tree, and select the characteristics to obtain effective integer result leaf vectors; and then, acquiring a linear cross combination relation of the high-order features and a nonlinear high-order feature association relation by using a cross network and a deep neural network, and fully mining a hidden relation among the high-order features, so that the recommendation accuracy is improved. Meanwhile, consumers are helped to select the agricultural products meeting the demands more conveniently, and meanwhile, producers and sellers of the agricultural products can be helped to promote the products more accurately, so that sales and customer satisfaction are improved.
Drawings
The following describes the embodiments of the present invention in further detail with reference to the accompanying drawings.
FIG. 1 is a flow chart of a method of recommending agricultural products according to the present invention;
fig. 2 is a model diagram of an agricultural product recommendation method of the present invention.
Detailed Description
It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
The example provides a farm product recommendation method based on LightGBM and deep learning, comprising the following steps:
step S1, preprocessing data in an agricultural product data set to enable a recommendation model to obtain basic data for feature processing to obtain an agricultural product data set B;
s2, taking the agricultural product data set B as input data of a LightGBM module, performing fusion optimization on features of the agricultural product data B through the LightGBM module, selecting features with highest classification accuracy as feature results, and obtaining effective integer result leaf vectors;
step S3, splicing all leaf node index values generated in the LightGBM module to form a new discrete agricultural product data set T, and splicing the new discrete agricultural product data set T with the agricultural product data set B to obtain a latest agricultural product data set V;
s4, taking the agricultural product data set V as input data of a neural network layer, and mining hidden relations of higher-order features in the data set through a cross network and a deep neural network;
and S5, carrying out linear weighted combination on the results obtained by the neural network layer, calculating by the full-connection layer, and obtaining the agricultural product recommendation result after the function is activated.
In this example, the data information such as the agricultural product information, the user purchase information, the agricultural product sales information and the like obtained in the step S1 are integrated into an agricultural product data set, and the unoccupied value part existing in the agricultural product data set is subjected to reasonable numerical filling in the input layer of the agricultural product recommendation model.
The filling data is sparse data and the null value of discrete data in the filling agricultural product data set is a numerical value or character which does not exist in the current data column, so that the sparse data and the null value of the discrete data in the filling agricultural product data set become an agricultural product data set without the null value;
and then carrying out normalization processing on sparse data in the agricultural product data set without the vacancy value, carrying out linear transformation on the sparse data by adopting a linear normalization method, and mapping a result into a [0,1] interval after linear normalization.
Wherein the linear normalization method is expressed as follows:
（1）
wherein, the liquid crystal display device comprises a liquid crystal display device,as a result of the data maximum value,as a result of the minimum value of the data,in the event of a data value being a data value,data values obtained for linear normalization;
and performing Label encoding processing on the discrete data in the agricultural product data set without the vacancy value, and performing discretization processing on the data to finally form a full-value data set B without the vacancy value.
In this example, the specific implementation manner of step S2 is as follows:
and inputting the full-value data set B without the vacancy value into a LightGBM module of the model, performing fusion optimization on the feature relevance of the agricultural product data set B by utilizing a histogram algorithm in the LightGBM module, taking the feature set with the highest classification accuracy as a feature selection result, and obtaining an effective integer result leaf vector.
Firstly, converting each characteristic column in an agricultural product data set B into a histogram, dividing the characteristic data in the agricultural product data set B into K bins uniformly, allocating an integer to each bin, replacing the data value in the bin with the integer allocated to the bin, and forming a histogram with the width of K according to the integer of the bin. And when traversing the data, counting and accumulating the histograms, after the data is traversed once, accumulating corresponding statistics by the histograms, and then traversing according to the discrete values of the histograms to find the optimal segmentation points.
Then, a single-side gradient sampling algorithm GOSS is adopted, the number of samples in an agricultural product data set B is reduced, most small gradient samples in the agricultural product data set B are eliminated, all values of features to be split in the agricultural product data set B are ordered in descending order according to the absolute value, m ∗% of data with the largest absolute value are selected, and n ∗% of data are randomly selected from the rest small gradient data; then randomly selecting n ∗% of data multiplied by a constant (1-m)/n, so that the algorithm can pay more attention to the undertraining samples and cannot generate excessive change on the distribution of the data set B; finally the information gain was calculated using (m+n) ∗ data by 100%.
Wherein the LightGBM model objective function formula is as follows:
（2）
（3）
the formula of the second-order Taylor expansion result of the function is as follows:
（4）
wherein, the liquid crystal display device comprises a liquid crystal display device,as the true value of the tag,as a result of the k-1 th learning,for the regularized term sums of the first k-1 trees,is a regular term of the kth tree;as a function of the object to be processed,training errors for samples, meaning finding a suitable treeThe value of the function is minimized.
In this example, the step S3 is specifically shown as follows:
after the processing of the LightGBM module is completed, the agricultural product data set B generates leaf node index values, and then all the generated leaf node index values are spliced to form a new discrete data set T, so that the sparsity problem of the agricultural product data set B is relieved by the data set T;
and combining and splicing the agricultural product data set B and the data set T to obtain an agricultural product data set V, wherein the agricultural product data set V enhances the effectiveness and the interpretability of the original data set, improves the utilization rate of the data characteristics and enhances the accuracy of the recommendation model.
In this example, the specific implementation manner of step S4 is as follows:
the agricultural product data set V is input into a neural network layer of a model, and training is carried out through a cross network and a deep neural network in the neural network layer, wherein the neural network layer in the model is formed by combining the cross network and the deep neural network in parallel, the input part adopts an input sharing principle, and each neural network part processes the same data set, so that the output part has relevance and connectivity.
The cross network of the agricultural product recommendation model is responsible for mining the linear cross combination relation between the higher-order features in the agricultural product data set V; the cross network consists of cross layers, wherein each layer of input data can cross with the current layer of data, and linear cross combination characteristics in the data are extracted; to be used forLayer for example, the agricultural product dataset V is characterized by crossing at the crossing layer, after one is completedThe layer will add its input back again and solve the problem of network performance degradation by using the residual idea; the crossover network can automatically learn limited high-order crossover features and corresponding weight parameters in the agricultural product data set, the crossover feature degree increases with the deepening of the layer depth, the time complexity and the space complexity of the crossover network linearly increase with the increase of the input dimension, and the crossover network has better generalization capability.
First, theThe layer output result formula is as follows:
（5）
the output result formula of the Cross network is as follows:
（6）
where x represents input data, b represents Bias,andrepresent the firstLayer and the firstThe output of the layer Cross layer,andrepresenting the connection parameters between the two layers, d representing the dimension of the feature,representing the vector formed by the superposition of the embedded vector and the continuous feature vector.
The deep neural network of the agricultural product recommendation model is responsible for mining nonlinear association relations between higher-order features in the agricultural product data set V; the deep neural network is a fully-connected neural network, each data feature can implicitly interact with other features, and nonlinear high-order features in a data set can be extracted. The agricultural product data set V is arranged in an input layer of the deep neural network, wherein discrete feature vectors in the data set V are used as input, the neuron of each input layer receives features in one discrete feature vector, and feature information is continuously transmitted in the fully-connected hidden layer; the characteristics are interactively combined, and finally, nonlinear high-order characteristics are extracted by an output layer; implicit interaction can be carried out between the data features of the agricultural product data set V, and nonlinear association relations between the features are mined.
The output formula is as follows:
（7）
wherein, the liquid crystal display device comprises a liquid crystal display device,representing a non-linear transfer function and,and b is ginsengThe number x is the output value of the previous layer.
In this example, the specific representation of step S5 is as follows:
the agricultural product recommendation model adopts equal weight weighting, the output results of the cross network and the deep neural network are subjected to linear weighting combination, the spliced result is calculated through a full-connection layer, and the recommendation result of the agricultural product is obtained after the Sigmoid activation function acts.
The output formula is shown as the following 8:
（8）
the foregoing is merely illustrative of specific embodiments of the present invention, and the scope of the present invention is not limited thereto, but any changes or substitutions within the technical scope of the present invention should be covered by the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (6)
1. The agricultural product recommending method based on the LightGBM and the deep learning is characterized by comprising the following steps of:
step S1, preprocessing data in an agricultural product data set, reasonably filling numerical values in a blank value part in the data set at an input layer, normalizing sparse data, and performing tag coding on discrete data to enable a recommendation model to obtain basic data for feature processing to obtain an agricultural product data set B;
step S2, taking the agricultural product data set B as input data of a LightGBM module, performing feature fusion optimization on the agricultural product data set B through a histogram algorithm, a unilateral gradient sampling algorithm GOSS and a reciprocal feature binding algorithm in the LightGBM module, selecting features with highest classification accuracy as feature results, and obtaining effective integer result leaf vectors;
step S3, splicing all leaf node index values generated in the LightGBM module to form a new discrete agricultural product data set T, and splicing the new discrete agricultural product data set T with the agricultural product data set B to obtain a latest agricultural product data set V;
s4, taking the agricultural product data set V as input data of a neural network layer, and mining hidden relations of higher-order features in the data set through a cross network and a deep neural network; the method comprises the steps of utilizing a cross network to excavate a linear cross combination relation between high-order features in an agricultural product data set V, and utilizing a deep neural network to excavate a nonlinear association relation between the high-order features in the agricultural product data set V;
and S5, carrying out linear weighted combination on the results obtained by the neural network layer, calculating by the full-connection layer, and obtaining the agricultural product recommendation result after the function is activated.
2. The agricultural product recommendation method based on LightGBM and deep learning as claimed in claim 1, wherein the step S1 of preprocessing the agricultural product data set at the agricultural product recommendation model input layer comprises the following steps:
filling the blank value of the agricultural product data set at the input layer, wherein the filling value is a numerical value or character which does not exist in the current data column when the blank value is filled because the data set is subjected to desensitization treatment;
normalizing sparse data in the agricultural product data set without the vacancy value, and performing tag coding on the discrete data;
the sparse data normalization processing adopts a linear normalization method to perform linear transformation on sparse data, and the result is mapped into a [0,1] interval after linear normalization;
and performing discretization processing on the discrete data, and performing label conversion by using a LabelEncoder to finally form a full-value agricultural product data set B without a blank value.
3. The agricultural product recommendation method based on LightGBM and deep learning as claimed in claim 1, wherein step S2 comprises the following steps:
carrying out fusion optimization on the feature relevance of the agricultural product data set B by utilizing a histogram algorithm in the LightGBM module, converting each feature column in the agricultural product data set B into a histogram, carrying out statistics and accumulation on the histogram when traversing data, accumulating corresponding statistics on the histogram after traversing the data once, and then traversing according to discrete values of the histogram to find optimal segmentation points;
and (3) reducing the number of samples in the data set B by utilizing a single-side gradient sampling algorithm GOSS in the LightGBM module, removing most small gradient samples in the agricultural product data set B, selecting the feature with the highest classification accuracy as a feature result, and obtaining an effective integer result leaf vector.
4. The agricultural product recommendation method based on LightGBM and deep learning as claimed in claim 1, wherein step S3 comprises the following steps:
after the processing of the LightGBM module is completed, the agricultural product data set B generates leaf node index values, and then all the generated leaf node index values are spliced to form a new discrete data set T, so that the sparsity problem of the agricultural product data set B is relieved by the data set T; and simultaneously combining and splicing the agricultural product data set B and the agricultural product data set T to obtain an agricultural product data set V.
5. The agricultural product recommendation method based on LightGBM and deep learning as claimed in claim 1, wherein the neural network layer in step S4 comprises:
the neural network layer is formed by parallel combination of a cross network and a deep neural network, an input part of the neural network layer adopts an input sharing principle, and each neural network part processes the same data set so that an output part has relevance and connectivity;
the cross network digs a linear cross combination relation between higher-order features in the agricultural product data set V;
the deep neural network digs nonlinear association relations between higher-order features in the agricultural product data set V.
6. The agricultural product recommendation method based on LightGBM and deep learning as claimed in claim 1, wherein step S5 specifically comprises the steps of:
the agricultural product recommendation model adopts equal weight weighting, the output results of the cross network and the deep neural network are subjected to linear weighting combination, the spliced result is calculated through a full-connection layer, and the recommendation result of the agricultural product is obtained after the Sigmoid activation function acts.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311235697.6A CN116977035A (en) | 2023-09-25 | 2023-09-25 | Agricultural product recommendation method based on LightGBM and deep learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311235697.6A CN116977035A (en) | 2023-09-25 | 2023-09-25 | Agricultural product recommendation method based on LightGBM and deep learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116977035A true CN116977035A (en) | 2023-10-31 |
Family
ID=88479974
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311235697.6A Pending CN116977035A (en) | 2023-09-25 | 2023-09-25 | Agricultural product recommendation method based on LightGBM and deep learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116977035A (en) |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112487291A (en) * | 2020-11-28 | 2021-03-12 | 重庆邮电大学 | Big data-based personalized news recommendation method and device |
| WO2021147568A1 (en) * | 2020-08-27 | 2021-07-29 | 平安科技（深圳）有限公司 | Gbdt high-order feature combination-based recommendation method, apparatus, and storage medium |
| CN113344615A (en) * | 2021-05-27 | 2021-09-03 | 上海数鸣人工智能科技有限公司 | Marketing activity prediction method based on GBDT and DL fusion model |
| JP6980958B1 (en) * | 2021-06-23 | 2021-12-15 | 中国科学院西北生態環境資源研究院 | Rural area classification garbage identification method based on deep learning |
| CN114677200A (en) * | 2022-04-01 | 2022-06-28 | 重庆邮电大学 | Business information recommendation method and device based on multi-party high-dimensional data longitudinal federal learning |
| CN114782114A (en) * | 2022-05-26 | 2022-07-22 | 郑州航空港经济综合实验区空港人工智能研究院 | Air ticket price prediction method based on CNN-GRU and SSA-GWO-LightGBM |
| WO2022175970A1 (en) * | 2021-02-17 | 2022-08-25 | Waycool Foods And Products Private Limited | System and method for determining one or more agri-measures |
| CN115203264A (en) * | 2022-05-13 | 2022-10-18 | 青岛文达通科技股份有限公司 | Urban tourism route recommendation method and system based on LightGBM algorithm |
| CN115578137A (en) * | 2022-10-31 | 2023-01-06 | 华南农业大学 | Agricultural product future price prediction method and system based on text mining and deep learning model |
| CN116186378A (en) * | 2022-07-29 | 2023-05-30 | 号百信息服务有限公司 | Package recommendation method, package recommendation device, electronic equipment and storage medium |
| CN116304299A (en) * | 2023-01-03 | 2023-06-23 | 电子科技大学 | Personalized recommendation method integrating user interest evolution and gradient promotion algorithm |
| CN116362836A (en) * | 2023-03-30 | 2023-06-30 | 东北林业大学 | Agricultural product recommendation algorithm based on user behavior sequence |
| CN116452293A (en) * | 2023-04-18 | 2023-07-18 | 重庆邮电大学 | Deep learning recommendation method and system integrating audience characteristics of articles |
-
2023
- 2023-09-25 CN CN202311235697.6A patent/CN116977035A/en active Pending
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021147568A1 (en) * | 2020-08-27 | 2021-07-29 | 平安科技（深圳）有限公司 | Gbdt high-order feature combination-based recommendation method, apparatus, and storage medium |
| CN112487291A (en) * | 2020-11-28 | 2021-03-12 | 重庆邮电大学 | Big data-based personalized news recommendation method and device |
| WO2022175970A1 (en) * | 2021-02-17 | 2022-08-25 | Waycool Foods And Products Private Limited | System and method for determining one or more agri-measures |
| CN113344615A (en) * | 2021-05-27 | 2021-09-03 | 上海数鸣人工智能科技有限公司 | Marketing activity prediction method based on GBDT and DL fusion model |
| JP6980958B1 (en) * | 2021-06-23 | 2021-12-15 | 中国科学院西北生態環境資源研究院 | Rural area classification garbage identification method based on deep learning |
| CN114677200A (en) * | 2022-04-01 | 2022-06-28 | 重庆邮电大学 | Business information recommendation method and device based on multi-party high-dimensional data longitudinal federal learning |
| CN115203264A (en) * | 2022-05-13 | 2022-10-18 | 青岛文达通科技股份有限公司 | Urban tourism route recommendation method and system based on LightGBM algorithm |
| CN114782114A (en) * | 2022-05-26 | 2022-07-22 | 郑州航空港经济综合实验区空港人工智能研究院 | Air ticket price prediction method based on CNN-GRU and SSA-GWO-LightGBM |
| CN116186378A (en) * | 2022-07-29 | 2023-05-30 | 号百信息服务有限公司 | Package recommendation method, package recommendation device, electronic equipment and storage medium |
| CN115578137A (en) * | 2022-10-31 | 2023-01-06 | 华南农业大学 | Agricultural product future price prediction method and system based on text mining and deep learning model |
| CN116304299A (en) * | 2023-01-03 | 2023-06-23 | 电子科技大学 | Personalized recommendation method integrating user interest evolution and gradient promotion algorithm |
| CN116362836A (en) * | 2023-03-30 | 2023-06-30 | 东北林业大学 | Agricultural product recommendation algorithm based on user behavior sequence |
| CN116452293A (en) * | 2023-04-18 | 2023-07-18 | 重庆邮电大学 | Deep learning recommendation method and system integrating audience characteristics of articles |
Non-Patent Citations (3)
| Title |
|---|
| HOUCHOU HAN: "LFDNN: A Novel Hybrid Recommendation Model Based on DeepFM and LightGBM", 《ENTROPY》, vol. 25, no. 4, pages 1 - 15 * |
| 方孝荣;: "基于深度学习BP人工神经网络的农产品质量检测", 金华职业技术学院学报, no. 03, pages 73 - 75 * |
| 杨正成: "基于 LightGBM 的广告商品平台推荐系统设计与应用", 《科技创新与应用》, vol. 12, no. 30, pages 1 - 6 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Jiajun et al. | Ultra-short term wind prediction with wavelet transform, deep belief network and ensemble learning | |
| Windarto | Implementation of data mining on rice imports by major country of origin using algorithm using k-means clustering method | |
| CN109903117B (en) | Knowledge graph processing method and device for commodity recommendation | |
| CN110648163B (en) | Recommendation algorithm based on user comments | |
| Zhang et al. | An improved deep computation model based on canonical polyadic decomposition | |
| CN111667022A (en) | User data processing method and device, computer equipment and storage medium | |
| Li et al. | Hierarchical graph attention networks for semi-supervised node classification | |
| CN112819171B (en) | Data searching method, system and computer storage medium based on table function | |
| CN107247753A (en) | A kind of similar users choosing method and device | |
| CN111539444A (en) | Gaussian mixture model method for modified mode recognition and statistical modeling | |
| CN114693397A (en) | Multi-view multi-modal commodity recommendation method based on attention neural network | |
| CN110781401A (en) | Top-n project recommendation method based on collaborative autoregressive flow | |
| CN115131698A (en) | Video attribute determination method, device, equipment and storage medium | |
| Du et al. | Multi-feature based network for multivariate time series classification | |
| CN113159892B (en) | Commodity recommendation method based on multi-mode commodity feature fusion | |
| Bhadoria et al. | Bunch graph based dimensionality reduction using auto-encoder for character recognition | |
| Raikar et al. | Efficiency comparison of supervised and unsupervised classifier on content based classification using shape, color, texture | |
| CN112905906A (en) | Recommendation method and system fusing local collaboration and feature intersection | |
| CN116977035A (en) | Agricultural product recommendation method based on LightGBM and deep learning | |
| CN113033090B (en) | Push model training method, data push device and storage medium | |
| CN114187546A (en) | Combined action recognition method and system | |
| CN108805628B (en) | Electronic commerce recommendation method | |
| CN112561599A (en) | Click rate prediction method based on attention network learning and fusing domain feature interaction | |
| Wang et al. | Stock Price Forecasting Based on Wavelet Filtering and Ensembled Machine Learning Model | |
| Patel | An Innovative Approach for Association Rule Mining In Grocery Dataset Based On Non-Negative Matrix Factorization And Autoencoder |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |