本說明書實施例目的是提供一種訓練方法、特徵提取方法及相關裝置，能夠以較高的效率，訓練出能夠關聯短期特性和長期特性的模型。
為了實現上述目的，本說明書實施例是這樣實現的：
第一方面，提供一種訓練方法，包括：
將樣本對象對應目標分類下的第一短期特徵集輸入至循環神經網路，得到第二短期特徵集，其中，該第一短期特徵集中的各短期特徵對應有相同的第一時間粒度；
將該第二短期特徵集按照時間順序組合成長期特徵集，其中，該長期特徵集中的各長期特徵對應有相同的第二時間粒度，該第二時間粒度大於該第一時間粒度；
將該長期特徵集輸入至卷積神經網路，得到該目標對象對應該目標分類下的目標特徵集；
將該目標特徵集輸入至用於識別該目標分類的分類模型，以基於該分類模型針對該樣本對象的識別結果，對該循環神經網路和該卷積神經網路進行訓練。
第二方面，提供一種特徵提取方法，包括：
將目標對象屬於目標分類下的第一短期特徵集輸入至循環神經網路，得到第二短期特徵集，其中，該第一短期特徵集中的各短期特徵對應有相同的第一時間粒度；
將該第二短期特徵集按照時間順序組合成長期特徵集，其中，該長期特徵集中的各長期特徵對應有相同的第二時間粒度，該第二時間粒度大於該第一時間粒度；
將該長期特徵集輸入至卷積神經網路，得到該目標對象對應該目標分類下的目標特徵集；
其中，該循環神經網路和該卷積神經網路是將樣本對象的目標特徵集輸入至具有識別該目標分類的分類模型後，基於該分類模型得到針對該樣本對象的識別結果，對該循環神經網路和該卷積神經網路進行訓練所得到的，該樣本對象的目標特徵集是基於該循環神經網路和該卷積神經網路確定得到的。
第三方面，提供一種神經網路的訓練裝置，包括：
第一處理模組，將樣本對象對應目標分類下的第一短期特徵集輸入至循環神經網路，得到第二短期特徵集；其中，該第一短期特徵集中的各短期特徵對應有相同的第一時間粒度；
第一組合模組，將該第二短期特徵集按照時間順序組合成長期特徵集，其中，該長期特徵集中的各長期特徵對應有相同的第二時間粒度，該第二時間粒度大於該第一時間粒度；
第二處理模組，將該長期特徵集輸入至卷積神經網路，得到該目標對象對應該目標分類下的目標特徵集；
訓練模組，將該目標特徵集輸入至用於識別該目標分類的分類模型，以基於該分類模型針對該樣本對象的識別結果，對該循環神經網路和該卷積神經網路進行訓練。
第四方面，提供一種電子設備包括：記憶體、處理器及儲存在該記憶體上並可在該處理器上運行的電腦程式，該電腦程式被該處理器執行：
將樣本對象對應目標分類下的第一短期特徵集輸入至循環神經網路，得到第二短期特徵集；其中，該第一短期特徵集中的各短期特徵對應有相同的第一時間粒度；
將該第二短期特徵集按照時間順序組合成長期特徵集，其中，該長期特徵集中的各長期特徵對應有相同的第二時間粒度，該第二時間粒度大於該第一時間粒度；
將該長期特徵集輸入至卷積神經網路，得到該目標對象對應該目標分類下的目標特徵集；
將該目標特徵集輸入至用於識別該目標分類的分類模型，以基於該分類模型針對該樣本對象的識別結果，對該循環神經網路和該卷積神經網路進行訓練。
第五方面，提供一種電腦可讀取儲存媒體，該電腦可讀取儲存媒體上儲存有電腦程式，該電腦程式被處理器執行時實現如下步驟：
將樣本對象對應目標分類下的第一短期特徵集輸入至循環神經網路，得到第二短期特徵集；其中，該第一短期特徵集中的各短期特徵對應有相同的第一時間粒度；
將該第二短期特徵集按照時間順序組合成長期特徵集，其中，該長期特徵集中的各長期特徵對應有相同的第二時間粒度，該第二時間粒度大於該第一時間粒度；
將該長期特徵集輸入至卷積神經網路，得到該目標對象對應該目標分類下的目標特徵集；
將該目標特徵集輸入至用於識別該目標分類的分類模型，以基於該分類模型針對該樣本對象的識別結果，對該循環神經網路和該卷積神經網路進行訓練。
第六方面，提供一種特徵提取裝置，包括：
第三處理模組，將目標對象屬於目標分類下的第一短期特徵集輸入至循環神經網路，得到第二短期特徵集；其中，該第一短期特徵集中的各短期特徵對應有相同的第一時間粒度；
第二組合模組，將該第二短期特徵集按照時間順序組合成長期特徵集，其中，該長期特徵集中的各長期特徵對應有相同的第二時間粒度，該第二時間粒度大於該第一時間粒度；
第四處理模組，將該長期特徵集輸入至卷積神經網路，得到該目標對象對應該目標分類下的目標特徵集；
其中，該循環神經網路和該卷積神經網路是將樣本對象的目標特徵集輸入至具有識別該目標分類的分類模型後，基於該分類模型得到的識別結果，對該循環神經網路和該卷積神經網路進行訓練所得到的，該樣本對象的目標特徵集是基於該循環神經網路和該卷積神經網路確定得到的。
第七方面，提供一種電子設備，包括：
將目標對象屬於目標分類下的第一短期特徵集輸入至循環神經網路，得到第二短期特徵集；其中，該第一短期特徵集中的各短期特徵對應有相同的第一時間粒度；
將該第二短期特徵集按照時間順序組合成長期特徵集，其中，該長期特徵集中的各長期特徵對應有相同的第二時間粒度，該第二時間粒度大於該第一時間粒度；
將該長期特徵集輸入至卷積神經網路，得到該目標對象對應該目標分類下的目標特徵集；
其中，該循環神經網路和該卷積神經網路是將樣本對象的目標特徵集輸入至具有識別該目標分類的分類模型後，基於該分類模型得到的識別結果，對該循環神經網路和該卷積神經網路進行訓練所得到的，該樣本對象的目標特徵集是基於該循環神經網路和該卷積神經網路確定得到的。
第八方面，提供一種電腦可讀取儲存媒體，該電腦可讀取儲存媒體上儲存有電腦程式，該電腦程式被處理器執行時實現如下步驟：
將目標對象屬於目標分類下的第一短期特徵集輸入至循環神經網路，得到第二短期特徵集；其中，該第一短期特徵集中的各短期特徵對應有相同的第一時間粒度；
將該第二短期特徵集按照時間順序組合成長期特徵集，其中，該長期特徵集中的各長期特徵對應有相同的第二時間粒度，該第二時間粒度大於該第一時間粒度；
將該長期特徵集輸入至卷積神經網路，得到該目標對象對應該目標分類下的目標特徵集；
其中，該循環神經網路和該卷積神經網路是將樣本對象的目標特徵集輸入至具有識別該目標分類的分類模型後，基於該分類模型得到的識別結果，對該循環神經網路和該卷積神經網路進行訓練所得到的，該樣本對象的目標特徵集是基於該循環神經網路和該卷積神經網路確定得到的。
本說明書實施例的方案採用RNN+CNN的模型結構，在訓練過程中，將短期特徵組成長期特徵，並進一步將長期特徵轉換為單維度的目標特徵後輸入至分類器，從而根據分類器的輸出結果調整RNN和CNN的參數，以達到訓練目的。顯然，整個訓練過程同時使用了短期特徵和長期特徵，不僅大幅提高了訓練效率，還能夠使模型學習到短期特徵和長期特徵之間的隱形聯繫，從而獲得更好的模型性能。The purpose of the embodiments of this specification is to provide a training method, a feature extraction method, and related devices, which can train a model that can correlate short-term characteristics and long-term characteristics with higher efficiency.
In order to achieve the above objectives, the embodiments of this specification are implemented as follows:
In the first aspect, a training method is provided, including:
Input the first short-term feature set under the target classification corresponding to the sample object to the recurrent neural network to obtain the second short-term feature set, wherein each short-term feature in the first short-term feature set corresponds to the same first time granularity;
Combining the second short-term feature set into a long-term feature set in a chronological order, wherein each long-term feature in the long-term feature set corresponds to the same second time granularity, and the second time granularity is greater than the first time granularity;
Input the long-term feature set to the convolutional neural network to obtain the target feature set under the target classification corresponding to the target object;
The target feature set is input to a classification model for recognizing the target classification, so as to train the recurrent neural network and the convolutional neural network based on the recognition result of the classification model for the sample object.
In a second aspect, a feature extraction method is provided, including:
Inputting the first short-term feature set of the target object belonging to the target classification to the recurrent neural network to obtain a second short-term feature set, wherein each short-term feature in the first short-term feature set corresponds to the same first time granularity;
Combining the second short-term feature set into a long-term feature set in a chronological order, wherein each long-term feature in the long-term feature set corresponds to the same second time granularity, and the second time granularity is greater than the first time granularity;
Input the long-term feature set to the convolutional neural network to obtain the target feature set under the target classification corresponding to the target object;
Wherein, the cyclic neural network and the convolutional neural network input the target feature set of the sample object into a classification model that recognizes the target classification, and obtain the recognition result for the sample object based on the classification model. The neural network and the convolutional neural network are trained, and the target feature set of the sample object is determined based on the recurrent neural network and the convolutional neural network.
In a third aspect, a neural network training device is provided, including:
The first processing module inputs the first short-term feature set under the target classification corresponding to the sample object to the recurrent neural network to obtain the second short-term feature set; wherein, each short-term feature in the first short-term feature set corresponds to the same first short-term feature set One time granularity;
The first combination module combines the second short-term feature set into a long-term feature set in chronological order, wherein each long-term feature in the long-term feature set corresponds to the same second time granularity, and the second time granularity is larger than the first time granularity. Time granularity
The second processing module inputs the long-term feature set to the convolutional neural network to obtain the target feature set under the target classification corresponding to the target object;
The training module inputs the target feature set to a classification model for identifying the target classification, so as to train the recurrent neural network and the convolutional neural network based on the recognition result of the classification model for the sample object.
In a fourth aspect, there is provided an electronic device including: a memory, a processor, and a computer program stored on the memory and running on the processor, the computer program being executed by the processor:
Input the first short-term feature set under the target classification corresponding to the sample object to the recurrent neural network to obtain the second short-term feature set; wherein, each short-term feature in the first short-term feature set corresponds to the same first time granularity;
Combining the second short-term feature set into a long-term feature set in a chronological order, wherein each long-term feature in the long-term feature set corresponds to the same second time granularity, and the second time granularity is greater than the first time granularity;
Input the long-term feature set to the convolutional neural network to obtain the target feature set under the target classification corresponding to the target object;
The target feature set is input to a classification model for recognizing the target classification, so as to train the recurrent neural network and the convolutional neural network based on the recognition result of the classification model for the sample object.
In a fifth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the following steps are implemented:
Input the first short-term feature set under the target classification corresponding to the sample object to the recurrent neural network to obtain the second short-term feature set; wherein, each short-term feature in the first short-term feature set corresponds to the same first time granularity;
Combining the second short-term feature set into a long-term feature set in a chronological order, wherein each long-term feature in the long-term feature set corresponds to the same second time granularity, and the second time granularity is greater than the first time granularity;
Input the long-term feature set to the convolutional neural network to obtain the target feature set under the target classification corresponding to the target object;
The target feature set is input to a classification model for recognizing the target classification, so as to train the recurrent neural network and the convolutional neural network based on the recognition result of the classification model for the sample object.
In a sixth aspect, a feature extraction device is provided, including:
The third processing module inputs the first short-term feature set under the target classification to the recurrent neural network to obtain the second short-term feature set; wherein, each short-term feature in the first short-term feature set corresponds to the same first short-term feature set One time granularity;
The second combination module combines the second short-term feature set into a long-term feature set in chronological order, wherein each long-term feature in the long-term feature set corresponds to the same second time granularity, and the second time granularity is larger than the first time granularity. Time granularity
The fourth processing module inputs the long-term feature set to the convolutional neural network to obtain the target feature set under the target classification corresponding to the target object;
Among them, the recurrent neural network and the convolutional neural network input the target feature set of the sample object into the classification model that recognizes the target classification, and based on the recognition result obtained by the classification model, the recurrent neural network and The convolutional neural network is obtained by training, and the target feature set of the sample object is determined based on the recurrent neural network and the convolutional neural network.
In a seventh aspect, an electronic device is provided, including:
Input the first short-term feature set of the target object belonging to the target classification to the recurrent neural network to obtain the second short-term feature set; wherein, each short-term feature in the first short-term feature set corresponds to the same first time granularity;
Combining the second short-term feature set into a long-term feature set in a chronological order, wherein each long-term feature in the long-term feature set corresponds to the same second time granularity, and the second time granularity is greater than the first time granularity;
Input the long-term feature set to the convolutional neural network to obtain the target feature set under the target classification corresponding to the target object;
Among them, the recurrent neural network and the convolutional neural network input the target feature set of the sample object into the classification model that recognizes the target classification, and based on the recognition result obtained by the classification model, the recurrent neural network and The convolutional neural network is obtained by training, and the target feature set of the sample object is determined based on the recurrent neural network and the convolutional neural network.
In an eighth aspect, a computer-readable storage medium is provided, the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the following steps:
Input the first short-term feature set of the target object belonging to the target classification to the recurrent neural network to obtain the second short-term feature set; wherein, each short-term feature in the first short-term feature set corresponds to the same first time granularity;
Combining the second short-term feature set into a long-term feature set in a chronological order, wherein each long-term feature in the long-term feature set corresponds to the same second time granularity, and the second time granularity is greater than the first time granularity;
Input the long-term feature set to the convolutional neural network to obtain the target feature set under the target classification corresponding to the target object;
Among them, the recurrent neural network and the convolutional neural network input the target feature set of the sample object into the classification model that recognizes the target classification, and based on the recognition result obtained by the classification model, the recurrent neural network and The convolutional neural network is obtained by training, and the target feature set of the sample object is determined based on the recurrent neural network and the convolutional neural network.
The solution in the embodiment of this specification adopts the RNN+CNN model structure. In the training process, short-term features are formed into long-term features, and the long-term features are further converted into single-dimensional target features and then input to the classifier, so as to be based on the output of the classifier As a result, the parameters of RNN and CNN are adjusted to achieve the purpose of training. Obviously, the entire training process uses both short-term features and long-term features, which not only greatly improves training efficiency, but also enables the model to learn the invisible connection between short-term features and long-term features, thereby obtaining better model performance.
為了使本技術領域的人員更好地理解本說明書中的技術方案，下面將結合本說明書實施例中的圖式，對本說明書實施例中的技術方案進行清楚、完整地描述，顯然，所描述的實施例僅僅是本說明書一部分實施例，而不是全部的實施例。基於本說明書中的實施例，本領域普通技術人員在沒有作出創造性勞動前提下所獲得的所有其他實施例，都應當屬於本說明書保護的範圍。
如前所述，現有技術的模型訓練方法是針對不同時間粒度的特徵，單獨對模型(模型由神經網路組成)進行訓練。比如，先將短期特徵輸入至模型，並根據輸出結果對模型參數進行調整。之後，再進一步將長期特徵輸入至模型，並根據輸出結果對模型參數進行調整。這種方式下，首先訓練效率不高；其次，整個模型雖然是基於短期特徵和長期特徵進行了學習，但是訓練過程是完全獨立的，無法形成短期特徵和長期特徵之間的隱性關聯，導致模型訓練後達不到較佳的性能。
針對上述問題，本文件旨在提供一種可以將短期特徵和長期特徵同時對模型進行訓練的技術方案。進一步地，還提供基於訓練後的模型實現相關應用的的技術方案。
圖1是本說明書實施例訓練方法的流程圖。圖1所示的方法可以由下文相對應的裝置執行，包括：
步驟S102，將樣本對象對應目標分類下的第一短期特徵集輸入至循環神經網路(RNN，Recurrent Neural Network)，得到第二短期特徵集，第一短期特徵集中的各短期特徵對應有相同的第一時間粒度。
其中，循環神經網路作為待訓練模型中的一部分。第一短期特徵可以是比較直觀的樣本對象的短期特徵，這些短期特徵可以通過較為常規的特徵提取方式獲取得到，本說明書實施例不對獲取方法作具體限定。
本步驟中，將第一短期特徵集輸入至RNN的目的是由RNN對第一短期特徵集進行提煉，得到隱性的第二短期特徵集。第二短期特徵集中的短期特徵可以與第一短期特徵集中的短期特徵對應有相同的時間粒度，即第一時間粒度。
步驟S104，將第二短期特徵集按照時間順序組合成長期特徵集，長期特徵集中的各長期特徵對應有相同的第二時間粒度，第二時間粒度大於第一時間粒度。
顯然，長期特徵是通過短期特徵組合而成的，因此不僅可以提現出樣本對象的長期特性，也能夠提現出樣本對象的短期特性。
此外，應理解的是，第一時間粒度和第二時間粒度可以根據實際需要進行靈活設置，本說明書實施例不作具體限定。作為示例性介紹，假設第一時間粒度為一天、第二時間粒度為一周，則第二短期特徵集中包含有樣本對象每天的短期特徵。本步驟具體將樣本對象相鄰7天的短期特徵進行組合，得到樣本對象一周的長期特徵。
步驟S106，將長期特徵集輸入至卷積神經網路(CNN，Convolutional Neural Networks)，得到目標對象對應目標分類下的目標特徵集。
其中，CNN作為待訓練模型中的一部分，與上述RNN的用途大致相同，可對長期特徵集作進一步提煉，獲得更高階的目標特徵集。
步驟S108，將目標特徵集輸入至用於識別目標分類的分類模型，以基於分類模型針對樣本對象的識別結果，對循環神經網路和卷積神經網路進行訓練。
其中，分類模型是訓練時所需要引用的部分，並不限定作為待訓練模型的一部分。
此外，訓練方式並不唯一，取決於分類模型的具體結構。
如果分類模型採用的是分類器結構，則本步驟可以基於有監督的訓練方式對分類模型進行訓練。即，將目標特徵集作為用於識別分類模型的輸入，將樣本對象的標籤(標籤用於指示樣本用對象是否符合目標分類)作為分類模型的輸出，以基於分類模型針對樣本對象的識別結果，對RNN和CNN進行訓練。
如果分類模型採用的是解碼器結構，則本步驟可以基於無監督的訓練方式對分類模型進行訓練。無監督的訓練方式不需要使用標籤，因此本步驟可以直接將目標特徵集作為用於識別分類模型的輸入，以基於分類模型針對樣本對象的識別結果，對RNN和CNN進行訓練。此外，在訓練過程中，還可以基於識別結果，對分類模型進行訓練，從而提高分類模型的識別準確率，保證RNN和CNN的訓練效果。
基於圖1所示的訓練方法方法可以知道，本說明書實施例的方案採用RNN+CNN的模型結構，在訓練過程中，將短期特徵組成長期特徵，並進一步將長期特徵轉換為單維度的目標特徵後輸入至分類器，從而根據分類器的輸出結果調整RNN和CNN的參數，以達到訓練目的。顯然，整個訓練過程同時使用了短期特徵和長期特徵，不僅大幅提高了訓練效率，還能夠使模型學習到短期特徵和長期特徵之間的隱形聯繫，從而獲得更好的模型性能。
下面對說明書實施例的訓練方法進行詳細介紹。
本說明書實施例的訓練方法同時使用短期特徵和長期特徵對目標模型進行訓練。如圖2所示，訓練結構包括：RNN→CNN→分類模型。其中，RNN+CNN屬於待訓練的目標模型，分類模型是訓練過程中添加的臨時部分，並不作為目標模型的一部分。
本說明書實施例的訓練方法首先將樣本對象對應目標分類下的第一短期特徵集輸入至RNN，得到由RNN輸出的第二短期特徵集。
這裡所述的RNN可以是長短期記憶網路、閘控循環單元網路以及自注意力機制網路中的任一者，或者，可以包括：長短期記憶網路、閘控循環單元網路以及自注意力機制網路中的至少一者。由於RNN屬於現有技術，本文不再具體贅述。
應理解，RNN並不會改變短期特徵的時間粒度，因此輸入獲得的第二短期特徵集中的短期特徵可以與第一短期特徵集中的短期特徵對應有相同的時間粒度。
在獲得RNN輸出的第二短期特徵集後，即可按照時間順序對第二短期特徵集中的短期特徵進行組合，得到對應有更大時間粒度的長期特徵。
這裡需要說明的是，特徵的組合方法並不唯一，本說明書實施例不作具體限定。作為其中一種可行的方案，可以採用向量組合方式將短期特徵組合成長期特徵。比如：將短期特徵A(q,w,e)和短期特徵B(a,s,d)進行組合，可以得到的長期特徵AB(q,w,e,a,s,d)。應理解，長期特徵是由短期特徵拼接而成的，因此含有樣本對象短期的特性。
之後，將組合而成的長期特徵輸入至CNN，由CNN進一步提煉出的目標特徵集。
應理解，CNN與RNN一樣，具有不同的實現方式，本說明書實施例不作具體限定。
作為示例性介紹，CNN可以包括：卷積層、池化層和全連接層。卷積層用於對長期特徵集進行卷積處理，得到卷積層輸出特徵集。池化層用於基於最大值池化演算法和/或均值池化演算法，對卷積層輸出特徵集進行池化處理，得到池化層輸出特徵集。全連接層用於將池化層輸出特徵集轉換為單一維度的適用於分類模型的目標特徵集。
在獲得目標特徵集後，即可將目標特徵集的目標特徵輸入至分類模型，由分類模型對樣本對象進行分類，以識別樣本對象是否符合目標分類。
這裡，樣本對象是否符合目標分類屬於已知資訊，分類模型輸出的識別結果屬於訓練結果，訓練結果並不一定是真實結果。之後，根據損失函數來計算訓練結果與真實結果之間的損失，並以降低損失為目的，對RNN、CNN以及分類模型的參數進行調整(也可以不對分類器的參數進行調整，取決於分類模型是否有調整需求)，以達到訓練目的。
下面結合一個實際的應用場景，對本說明書實施例的訓練方法進行實例介紹。
本應用場景用於訓練刻劃金融風險特徵的學習模型。其中，學習模型採用長短期記憶網路(LSTM，Long Short-Term Memory)+文本捲進循環網路(Text-CNN)的結構，對應的流程包括：
步驟一，獲取支付應用程式中樣本對象的金融業務資料，並基於語義分析演算法，按照每半小的時間粒度，對金融業務資料進行基礎特徵的提取，得到一個月的第一短期特徵集。
在本應用場景中，第一短期特徵集可以但不限於是樣本對象每半小時所對應的交易總金額、交易總筆數以及交易對手總數。這些刻劃的是樣本對象在短時間內的交易行為，一些不正常的交易模式(如快進快出)可以被這些短期特徵捕捉到。
步驟二，將第一短期特徵集輸入至LSTM，得到LSTM輸出的第二短期特徵集。
其中， LSTM數量並不限於一個。作為示例性介紹，LSTM可以與第一短期特徵集的天數一一對應，這樣每個Lstm的輸出代表了一天的短期隱藏特徵。
步驟三，將第二短期特徵集按照時間順序進行組合，得到長期特徵集。
如前所述，之前獲取了每半小時的短期隱藏特徵，但是只能代表半小時的交易動態，為了得到樣本對象長期的交易動態，按時間順序將半小時的短期隱藏特徵拼接成每天的長期特徵。應理解，長期特徵的資料格式應適用於後續的TextCnn。
步驟四，將長期特徵集輸入至TextCnn，由TextCnn提煉出目標特徵集。
其中，TextCnn的卷積層長度可以自由設置，比如長度為2則可以捕獲樣本對象相鄰2天的局部行為變化，如果長度為6，可以捕捉相鄰6天的局部行為變化。也就是說，通過卷積核不同尺寸的組合實現對樣本對象不同時間粒度的特徵學習。
TextCnn的池化層對卷積提的輸出特徵再進行Pooling操作。本應用場景中，池化層可以同時採用最大值池化(Max-Pooling)演算法與(Avg-Pooling)演算法。其中，Max-Pooling主要用來保留特徵發生變化的主要資訊，Avg-Pooling用來保留特徵平均狀態。
TextCnn的全連接層將Pooling操作得到的特徵集進行整合降維，得到適合輸入分類模型的單一維度的目標特徵集。
步驟五，將目標特徵集輸入至分類模型，以對LSTM和TextCnn進行訓練。
其中，分類模型可以採用二分類交叉熵機制。在二分類問題中，分類模型的損失函數具體為交叉熵損失函數，樣本對象的標籤取值只能是1或0,1表示樣本對象符合目標分類，0表示樣本對象不符合目標分類。
假設某個樣本對象的真實標籤為yt，該樣本對象yt=1的概率為yp,則損失函數可以為：-log(yt|yp)=-[yt*log(yp)+(1-yt)log(1-yp)]。對於整個學習模型而言，其損失函數就是所有樣本對象的損失函數非負的平均值。
目標特徵集輸入在輸入分類模型後，會得到分類模型識別樣本對象是否屬於風險對象的識別結果。之後，基於損失函數計算識別結果會與標籤取值的損失，並以降低損失為目的，來調整LSTM和TextCnn的參數。
以上是對本說明書實施例的方法的介紹。應理解，在不脫離本文上述原理基礎之上，還可以進行適當的變化，這些變化也應視為本說明書實施例的保護範圍。
此外，如圖3所示，本說明書實施例還提供一種特徵提取方法，包括：
步驟302，將目標對象屬於目標分類下的第一短期特徵集輸入至循環神經網路，得到第二短期特徵集，第一短期特徵集中的各短期特徵對應有相同的第一時間粒度。
步驟304，將第二短期特徵集按照時間順序組合成長期特徵集，長期特徵集中的各長期特徵對應有相同的第二時間粒度，該第二時間粒度大於該第一時間粒度。
步驟306，將該長期特徵集輸入至卷積神經網路，得到該目標對象對應該目標分類下的目標特徵集。
其中，目標特徵集中的目標特徵即最終提煉得到的目標對象的隱性特徵。
應理解，上述循環神經網路和上述卷積神經網路是由圖1所示的訓練方法所訓練得到的。即，該循環神經網路和該卷積神經網路是將樣本對象的目標特徵集輸入至具有識別該目標分類的分類模型後，基於該分類模型得到針對該樣本對象的識別結果，對該循環神經網路和該卷積神經網路進行訓練所得到的，該樣本對象的目標特徵集是基於該循環神經網路和該卷積神經網路確定得到的。
基於圖3所示的特徵提取方法可以知道，本說明書實施例的方案僅需要將目標對象的短期特徵輸入至RNN+CNN的模型，即由模型機械方式提煉出即呈現短期特性，又呈現長期特性的目標特徵，可用於對目標對象進行更全面的刻劃，挖掘出人工難以找到的隱性特徵。
此外，如圖4所示，本說明書實施例還提供一種神經網路的訓練裝置400，包括：
第一處理模組410，將樣本對象對應目標分類下的第一短期特徵集輸入至循環神經網路，得到第二短期特徵集；其中，該第一短期特徵集中的各短期特徵對應有相同的第一時間粒度；
第一組合模組420，將該第二短期特徵集按照時間順序組合成長期特徵集，其中，該長期特徵集中的各長期特徵對應有相同的第二時間粒度，該第二時間粒度大於該第一時間粒度；
第二處理模組430，將該長期特徵集輸入至卷積神經網路，得到該目標對象對應該目標分類下的目標特徵集；
訓練模組440，將該目標特徵集輸入至用於識別該目標分類的分類模型，以基於該分類模型針對該樣本對象的識別結果，對該循環神經網路和該卷積神經網路進行訓練。
基於圖4所示的訓練裝置可以知道，本說明書實施例的方案採用RNN+CNN的模型結構，在訓練過程中，將短期特徵組成長期特徵，並進一步將長期特徵轉換為單維度的目標特徵後輸入至分類器，從而根據分類器的輸出結果調整RNN和CNN的參數，以達到訓練目的。顯然，整個訓練過程同時使用了短期特徵和長期特徵，不僅大幅提高了訓練效率，還能夠使模型學習到短期特徵和長期特徵之間的隱形聯繫，從而獲得更好的模型性能。
可選地，訓練模組440在執行時，具體將該目標特徵集作為用於識別該目標分類的分類模型的輸入，將該樣本對象的標籤作為該分類模型的輸出，以基於該分類模型針對該樣本對象的識別結果，對該循環神經網路和該卷積神經網路進行訓練，其中，該樣本對象的標籤用於指示該樣本用對象是否符合該目標分類。
可選地，該循環神經網路包括以下至少一者：
長短期記憶網路、閘控循環單元網路以及自注意力機制網路。
可選地，該卷積神經網路包括：文本捲進循環網路。
可選地，該卷積神經網路包括：
卷積層，對長期特徵集進行卷積處理，得到卷積層輸出特徵集；
池化層，基於最大值池化演算法和/或均值池化演算法，對該卷積層輸出特徵集進行池化處理，得到池化層輸出特徵集；
全連接層，將池化層輸出特徵集轉換為單一維度的目標特徵集。
可選地，該樣本對象為支付應用程式用戶，該目標分類為金融風險，該第一短期特徵集包括以下至少一種特徵維度的短期特徵：
該支付應用程式用戶在各第一時間粒度所對應的交易總金額、交易總筆數以及交易對手總數。
顯然，本說明書實施例的訓練裝置可以作為上述圖1所示的訓練方法的執行主體，因此能夠實現該訓練方法在圖1和圖2所實現的功能。由於原理相同，本文不再贅述。
此外，如圖5所示，本說明書實施例還提供一種特徵提取裝置500，包括：
第三處理模組510，將目標對象屬於目標分類下的第一短期特徵集輸入至循環神經網路，得到第二短期特徵集；其中，該第一短期特徵集中的各短期特徵對應有相同的第一時間粒度；
第二組合模組520，將該第二短期特徵集按照時間順序組合成長期特徵集，其中，該長期特徵集中的各長期特徵對應有相同的第二時間粒度，該第二時間粒度大於該第一時間粒度；
第四處理模組530，將該長期特徵集輸入至卷積神經網路，得到該目標對象對應該目標分類下的目標特徵集；
其中，該循環神經網路和該卷積神經網路是將樣本對象的目標特徵集輸入至具有識別該目標分類的分類模型後，基於該分類模型得到的識別結果，對該循環神經網路和該卷積神經網路進行訓練所得到的，該樣本對象的目標特徵集是基於該循環神經網路和該卷積神經網路確定得到的。
基於圖5所示的特徵提取裝置可以知道，本說明書實施例的方案僅需要將目標對象的短期特徵輸入至RNN+CNN的模型，即由模型機械方式提煉出即呈現短期特性，又呈現長期特性的目標特徵，可用於對目標對象進行更全面的刻劃，挖掘出人工難以找到的隱性特徵。
顯然，本說明書實施例的特徵提取裝置可以作為上述圖3所示的特徵提取方法的執行主體，因此能夠實現該特徵提取方法在圖3所實現的功能。由於原理相同，本文不再贅述。
圖6是本說明書的一個實施例電子設備的結構示意圖。請參考圖6，在硬體層面，該電子設備包括處理器，可選地還包括內部匯流排、網路介面、記憶體。其中，記憶體可能包含內部記憶體，例如高速隨機存取記憶體(Random-Access Memory，RAM)，也可能還包括非揮發性記憶體(non-volatile memory)，例如至少1個磁碟記憶體等。當然，該電子設備還可能包括其他業務所需要的硬體。
處理器、網路介面和記憶體可以通過內部匯流排相互連接，該內部匯流排可以是ISA(Industry Standard Architecture，工業標準體系結構)匯流排、PCI(Peripheral Component Interconnect，外設部件互連標準)匯流排或EISA(Extended Industry Standard Architecture，擴展工業標準結構)匯流排等。該匯流排可以分為地址匯流排、資料匯流排、控制匯流排等。為便於表示，圖6中僅用一個雙向箭頭表示，但並不表示僅有一根匯流排或一種類型的匯流排。
記憶體，用於存放程式。具體地，程式可以包括程式碼，該程式碼包括電腦操作指令。記憶體可以包括內部記憶體和非揮發性記憶體，並向處理器提供指令和資料。
其中，處理器從非揮發性記憶體中讀取對應的電腦程式到內部記憶體中然後運行，在邏輯層面上形成神經網路的訓練裝置。處理器，執行記憶體所存放的程式，並具體用於執行以下操作：
將樣本對象對應目標分類下的第一短期特徵集輸入至循環神經網路，得到第二短期特徵集，其中，該第一短期特徵集中的各短期特徵對應有相同的第一時間粒度；
將該第二短期特徵集按照時間順序組合成長期特徵集，其中，該長期特徵集中的各長期特徵對應有相同的第二時間粒度，該第二時間粒度大於該第一時間粒度；
將該長期特徵集輸入至卷積神經網路，得到該目標對象對應該目標分類下的目標特徵集；
將該目標特徵集輸入至用於識別該目標分類的分類模型，以基於該分類模型針對該樣本對象的識別結果，對該循環神經網路和該卷積神經網路進行訓練。
其中，處理器從非揮發性記憶體中讀取對應的電腦程式到內部記憶體中然後運行，在邏輯層面上還可以形成特徵提取裝置。處理器，執行記憶體所存放的程式，並具體用於執行以下操作：
將目標對象屬於目標分類下的第一短期特徵集輸入至循環神經網路，得到第二短期特徵集，其中，該第一短期特徵集中的各短期特徵對應有相同的第一時間粒度；
將該第二短期特徵集按照時間順序組合成長期特徵集，其中，該長期特徵集中的各長期特徵對應有相同的第二時間粒度，該第二時間粒度大於該第一時間粒度；
將該長期特徵集輸入至卷積神經網路，得到該目標對象對應該目標分類下的目標特徵集；
其中，該循環神經網路和該卷積神經網路是將樣本對象的目標特徵集輸入至具有識別該目標分類的分類模型後，基於該分類模型得到針對該樣本對象的識別結果，對該循環神經網路和該卷積神經網路進行訓練所得到的，該樣本對象的目標特徵集是基於該循環神經網路和該卷積神經網路確定得到的。
上述如本說明書圖1所示實施例揭示的訓練方法或者圖3所示實施例揭示的特徵提取方法由處理器實現。處理器可能是一種積體電路晶片，具有信號的處理能力。在實現過程中，上述方法的各步驟可以通過處理器中的硬體的積體邏輯電路或者軟體形式的指令完成。上述的處理器可以是通用處理器，包括中央處理器(Central Processing Unit，CPU)、網路處理器(Network Processor，NP)等；還可以是數位信號處理器(Digital Signal Processor，DSP)、專用積體電路(Application Specific Integrated Circuit，ASIC)、現場可程式化閘陣列(Field－Programmable Gate Array，FPGA)或者其他可程式化邏輯器件、離散閘或者電晶體邏輯器件、離散硬體組件。可以實現或者執行本說明書實施例中的公開的各方法、步驟及邏輯方塊圖。通用處理器可以是微處理器或者該處理器也可以是任何常規的處理器等。結合本說明書實施例所公開的方法的步驟可以直接體現為硬體譯碼處理器執行完成，或者用譯碼處理器中的硬體及軟體模組組合執行完成。軟體模組可以位於隨機記憶體，快閃記憶體、唯讀記憶體，可程式化唯讀記憶體或者電可抹除及寫入可程式化記憶體、暫存器等本領域成熟的儲存媒體中。該儲存媒體位於記憶體，處理器讀取記憶體中的資訊，結合其硬體完成上述方法的步驟。
應理解，本說明書實施例的電子設備可以實現上述訓練裝置在圖1和圖2所示的實施例的功能，或者上述特徵提取裝置在圖所示的實施例的功能。由於原理相同，本文不再贅述。
當然，除了軟體實現方式之外，本說明書的電子設備並不排除其他實現方式，比如邏輯器件抑或軟硬體結合的方式等等，也就是說以下處理流程的執行主體並不限定於各個邏輯單元，也可以是硬體或邏輯器件。
此外，本說明書實施例還提出了一種電腦可讀取儲存媒體，該電腦可讀取儲存媒體儲存一個或多個程式，該一個或多個程式包括指令。
其中，該指令當被包括多個應用程式的可攜式電子設備執行時，能夠使該可攜式電子設備執行圖1所示實施例的訓練方法，並具體用於執行以下方法：
將樣本對象對應目標分類下的第一短期特徵集輸入至循環神經網路，得到第二短期特徵集，其中，該第一短期特徵集中的各短期特徵對應有相同的第一時間粒度；
將該第二短期特徵集按照時間順序組合成長期特徵集，其中，該長期特徵集中的各長期特徵對應有相同的第二時間粒度，該第二時間粒度大於該第一時間粒度；
將該長期特徵集輸入至卷積神經網路，得到該目標對象對應該目標分類下的目標特徵集；
將該目標特徵集輸入至用於識別該目標分類的分類模型，以基於該分類模型針對該樣本對象的識別結果，對該循環神經網路和該卷積神經網路進行訓練。
或者，指令當被包括多個應用程式的可攜式電子設備執行時，能夠使該可攜式電子設備執行圖3所示實施例的特徵提取方法，並具體用於執行以下方法：
將目標對象屬於目標分類下的第一短期特徵集輸入至循環神經網路，得到第二短期特徵集，其中，該第一短期特徵集中的各短期特徵對應有相同的第一時間粒度；
將該第二短期特徵集按照時間順序組合成長期特徵集，其中，該長期特徵集中的各長期特徵對應有相同的第二時間粒度，該第二時間粒度大於該第一時間粒度；
將該長期特徵集輸入至卷積神經網路，得到該目標對象對應該目標分類下的目標特徵集；
其中，該循環神經網路和該卷積神經網路是將樣本對象的目標特徵集輸入至具有識別該目標分類的分類模型後，基於該分類模型得到針對該樣本對象的識別結果，對該循環神經網路和該卷積神經網路進行訓練所得到的，該樣本對象的目標特徵集是基於該循環神經網路和該卷積神經網路確定得到的。
應理解，上述指令當被包括多個應用程式的可攜式電子設備執行時，能夠使上文所述的訓練裝置實現圖1和圖2所示實施例的功能，或者，能夠使上文所述的特徵提取裝置實現圖3所示實施例的功能，本文不再贅述。
本領域技術人員應明白，本說明書的實施例可提供為方法、系統或電腦程式產品。因此，本說明書可採用完全硬體實施例、完全軟體實施例或結合軟體和硬體方面的實施例的形式。而且，本說明書可採用在一個或多個其中包含有電腦可用程式碼的電腦可用儲存媒體(包括但不限於磁碟記憶體、CD-ROM、光學記憶體等)上實施的電腦程式產品的形式。
上述對本說明書特定實施例進行了描述。其它實施例在所附申請專利範圍的範圍內。在一些情況下，在申請專利範圍中記載的動作或步驟可以按照不同於實施例中的順序來執行並且仍然可以實現期望的結果。另外，在圖式中描繪的過程不一定要求示出的特定順序或者連續順序才能實現期望的結果。在某些實施方式中，多任務處理和並行處理也是可以的或者可能是有利的。
以上僅為本說明書的實施例而已，並不用於限制本說明書。對於本領域技術人員來說，本說明書可以有各種更改和變化。凡在本說明書的精神和原理之內所作的任何修改、等同替換、改進等，均應包含在本說明書的請求項範圍之內。此外，本領域普通技術人員在沒有作出創造性勞動前提下所獲得的所有其他實施例，都應當屬於本文件的保護範圍。In order to enable those skilled in the art to better understand the technical solutions in this specification, the following will clearly and completely describe the technical solutions in the embodiments of this specification in conjunction with the drawings in the embodiments of this specification. Obviously, the described The embodiments are only a part of the embodiments in this specification, rather than all the embodiments. Based on the embodiments in this specification, all other embodiments obtained by a person of ordinary skill in the art without creative work shall fall within the protection scope of this specification.
As mentioned above, the prior art model training method is to separately train the model (the model is composed of a neural network) according to the characteristics of different time granularities. For example, first input short-term features into the model, and adjust the model parameters according to the output results. After that, the long-term features are further input to the model, and the model parameters are adjusted according to the output results. In this way, firstly, the training efficiency is not high; secondly, although the entire model is based on short-term and long-term features for learning, the training process is completely independent, and the implicit association between short-term and long-term features cannot be formed, resulting in The model cannot achieve better performance after training.
In response to the above problems, this document aims to provide a technical solution that can simultaneously train the model with short-term features and long-term features. Further, it also provides a technical solution for realizing related applications based on the trained model.
Fig. 1 is a flowchart of a training method according to an embodiment of this specification. The method shown in Figure 1 can be executed by the following corresponding devices, including:
Step S102: Input the first short-term feature set under the target classification corresponding to the sample object to the Recurrent Neural Network (RNN) to obtain a second short-term feature set. Each short-term feature in the first short-term feature set corresponds to the same The first time granularity.
Among them, the recurrent neural network is part of the model to be trained. The first short-term feature may be a relatively intuitive short-term feature of the sample object, and these short-term features can be obtained through a relatively conventional feature extraction method, and the embodiment of this specification does not specifically limit the obtaining method.
In this step, the purpose of inputting the first short-term feature set to the RNN is to refine the first short-term feature set by the RNN to obtain the implicit second short-term feature set. The short-term features in the second short-term feature set may have the same time granularity corresponding to the short-term features in the first short-term feature set, that is, the first time granularity.
Step S104: Combine the second short-term feature set into a long-term feature set in a chronological order. Each long-term feature in the long-term feature set corresponds to the same second time granularity, and the second time granularity is greater than the first time granularity.
Obviously, the long-term characteristics are composed of short-term characteristics, so not only the long-term characteristics of the sample object can be extracted, but also the short-term characteristics of the sample object can be extracted.
In addition, it should be understood that the first time granularity and the second time granularity can be flexibly set according to actual needs, which are not specifically limited in the embodiment of this specification. As an exemplary introduction, assuming that the first time granularity is one day and the second time granularity is one week, the second short-term feature set includes daily short-term features of the sample object. This step specifically combines the short-term characteristics of the sample object for 7 days adjacent to each other to obtain the long-term characteristics of the sample object for one week.
In step S106, the long-term feature set is input to a convolutional neural network (CNN, Convolutional Neural Networks) to obtain a target feature set under the target classification corresponding to the target object.
Among them, CNN, as a part of the model to be trained, has roughly the same purpose as the above-mentioned RNN, and can further refine the long-term feature set to obtain a higher-order target feature set.
In step S108, the target feature set is input to the classification model for identifying the target classification, so as to train the recurrent neural network and the convolutional neural network based on the recognition result of the classification model for the sample object.
Among them, the classification model is a part that needs to be cited during training, and is not limited to be a part of the model to be trained.
In addition, the training method is not unique and depends on the specific structure of the classification model.
If the classification model adopts a classifier structure, this step can train the classification model based on a supervised training method. That is, the target feature set is used as the input for identifying the classification model, and the label of the sample object (the label is used to indicate whether the sample object meets the target classification) is used as the output of the classification model to identify the result of the sample object based on the classification model. Train RNN and CNN.
If the classification model adopts the decoder structure, this step can train the classification model based on an unsupervised training method. The unsupervised training method does not need to use labels, so this step can directly use the target feature set as the input for the recognition of the classification model, so as to train the RNN and CNN based on the recognition result of the classification model for the sample object. In addition, in the training process, the classification model can also be trained based on the recognition result, thereby improving the recognition accuracy of the classification model and ensuring the training effect of RNN and CNN.
Based on the training method shown in Figure 1, it can be known that the scheme of the embodiment of this specification adopts the RNN+CNN model structure. In the training process, short-term features are formed into long-term features, and the long-term features are further converted into single-dimensional target features. Then input to the classifier, so as to adjust the parameters of RNN and CNN according to the output result of the classifier to achieve the purpose of training. Obviously, the entire training process uses both short-term features and long-term features, which not only greatly improves training efficiency, but also enables the model to learn the invisible connection between short-term features and long-term features, thereby obtaining better model performance.
The training method of the embodiment of the specification will be introduced in detail below.
The training method of the embodiment of this specification uses both short-term features and long-term features to train the target model. As shown in Figure 2, the training structure includes: RNN→CNN→classification model. Among them, RNN+CNN belongs to the target model to be trained, and the classification model is a temporary part added during the training process and is not part of the target model.
The training method of the embodiment of the present specification first inputs the first short-term feature set under the target classification corresponding to the sample object to the RNN, and obtains the second short-term feature set output by the RNN.
The RNN mentioned here can be any one of a long and short-term memory network, a gated recurrent unit network, and a self-attention mechanism network, or it can include: a long and short-term memory network, a gated recurrent unit network, and At least one of the self-attention mechanism network. Since RNN belongs to the prior art, this article will not go into details.
It should be understood that the RNN does not change the time granularity of the short-term features, so the short-term features in the second short-term feature set obtained by input may have the same time granularity corresponding to the short-term features in the first short-term feature set.
After obtaining the second short-term feature set output by the RNN, the short-term features in the second short-term feature set can be combined in chronological order to obtain corresponding long-term features with greater time granularity.
It should be noted here that the combination method of the features is not unique, and the embodiment of the specification does not specifically limit it. As one of the feasible solutions, a vector combination method can be used to combine short-term features into long-term features. For example: Combine short-term features A (q, w, e) and short-term features B (a, s, d) to obtain long-term features AB (q, w, e, a, s, d). It should be understood that long-term features are spliced from short-term features, and therefore contain short-term features of the sample object.
After that, the combined long-term features are input to CNN, and the target feature set is further refined by CNN.
It should be understood that, like RNN, CNN has different implementation modes, which are not specifically limited in the embodiment of this specification.
As an exemplary introduction, a CNN may include: a convolutional layer, a pooling layer, and a fully connected layer. The convolution layer is used to perform convolution processing on the long-term feature set to obtain the output feature set of the convolution layer. The pooling layer is used to pool the output feature set of the convolutional layer based on the maximum pooling algorithm and/or the average pooling algorithm to obtain the output feature set of the pooling layer. The fully connected layer is used to convert the output feature set of the pooling layer into a single-dimensional target feature set suitable for the classification model.
After the target feature set is obtained, the target features of the target feature set can be input to the classification model, and the sample object is classified by the classification model to identify whether the sample object meets the target classification.
Here, whether the sample object conforms to the target classification belongs to known information, the recognition result output by the classification model belongs to the training result, and the training result is not necessarily the real result. After that, calculate the loss between the training result and the real result according to the loss function, and adjust the parameters of RNN, CNN and the classification model for the purpose of reducing the loss (it is not necessary to adjust the parameters of the classifier, depending on the classification model Is there any need for adjustment) in order to achieve the purpose of training.
In the following, in combination with an actual application scenario, the training method of the embodiment of this specification will be introduced as an example.
This application scenario is used to train a learning model that characterizes financial risks. Among them, the learning model adopts the structure of Long Short-Term Memory (LSTM) + Text-CNN (Text-CNN), and the corresponding processes include:
Step 1: Obtain the financial business data of the sample objects in the payment application, and based on the semantic analysis algorithm, extract the basic features of the financial business data at every half-small time granularity to obtain the first short-term feature set of a month.
In this application scenario, the first short-term feature set can be, but is not limited to, the total transaction amount, the total number of transactions, and the total number of counterparties corresponding to the sample object every half hour. These characterizations are the trading behaviors of the sample objects in a short period of time, and some abnormal trading patterns (such as fast forward and fast out) can be captured by these short-term characteristics.
Step 2: Input the first short-term feature set to the LSTM to obtain the second short-term feature set output by the LSTM.
Among them, the number of LSTMs is not limited to one. As an exemplary introduction, the LSTM may have a one-to-one correspondence with the days of the first short-term feature set, so that the output of each Lstm represents a day's short-term hidden features.
Step 3: Combine the second short-term feature set in chronological order to obtain a long-term feature set.
As mentioned earlier, the short-term hidden features of every half hour were previously obtained, but they can only represent the trading dynamics of half an hour. In order to obtain the long-term trading dynamics of the sample objects, the short-term hidden features of half an hour are spliced into the long-term daily in chronological order. feature. It should be understood that the long-term feature data format should be applied to subsequent TextCnn.
Step 4: Input the long-term feature set into TextCnn, and then extract the target feature set from TextCnn.
Among them, the length of the convolutional layer of TextCnn can be set freely. For example, if the length is 2, it can capture the local behavior changes of the sample object in the adjacent 2 days, and if the length is 6, it can capture the local behavior changes of the adjacent 6 days. That is to say, through the combination of different sizes of the convolution kernel, the feature learning of the sample objects at different time granularities is realized.
The pooling layer of TextCnn performs a Pooling operation on the output features extracted by the convolution. In this application scenario, the pooling layer can use the Max-Pooling algorithm and the Avg-Pooling algorithm at the same time. Among them, Max-Pooling is mainly used to retain the main information of the feature change, and Avg-Pooling is used to retain the average state of the feature.
The fully connected layer of TextCnn integrates the feature sets obtained from the Pooling operation to reduce the dimensionality, and obtains a single-dimensional target feature set suitable for the input classification model.
Step 5: Input the target feature set into the classification model to train LSTM and TextCnn.
Among them, the classification model can adopt a two-class cross-entropy mechanism. In the binary classification problem, the loss function of the classification model is specifically the cross-entropy loss function, and the label value of the sample object can only be 1 or 0. 1 indicates that the sample object meets the target classification, and 0 indicates that the sample object does not meet the target classification.
Assuming that the true label of a sample object is yt, and the probability of the sample object yt=1 is yp, the loss function can be: -log(yt|yp)=-[yt*log(yp)+(1-yt) log(1-yp)]. For the entire learning model, its loss function is the non-negative average of the loss functions of all sample objects.
After the target feature set is input into the classification model, the classification model will obtain the recognition result of whether the sample object belongs to the risk object. After that, the loss of the recognition result and the label value is calculated based on the loss function, and the parameters of LSTM and TextCnn are adjusted for the purpose of reducing the loss.
The above is an introduction to the method of the embodiment of this specification. It should be understood that appropriate changes can be made without departing from the foregoing principles herein, and these changes should also be regarded as the protection scope of the embodiments of this specification.
In addition, as shown in FIG. 3, an embodiment of this specification also provides a feature extraction method, including:
Step 302: Input the first short-term feature set of the target object under the target classification to the recurrent neural network to obtain a second short-term feature set. Each short-term feature in the first short-term feature set corresponds to the same first time granularity.
Step 304: Combine the second short-term feature set into a long-term feature set in chronological order. Each long-term feature in the long-term feature set corresponds to the same second time granularity, and the second time granularity is greater than the first time granularity.
Step 306: Input the long-term feature set into the convolutional neural network to obtain the target feature set under the target classification corresponding to the target object.
Among them, the target features in the target feature set are the hidden features of the target object finally refined.
It should be understood that the above-mentioned cyclic neural network and the above-mentioned convolutional neural network are trained by the training method shown in FIG. 1. That is, the cyclic neural network and the convolutional neural network input the target feature set of the sample object into a classification model that recognizes the target classification, and then obtain the recognition result for the sample object based on the classification model, and the cyclic The neural network and the convolutional neural network are trained, and the target feature set of the sample object is determined based on the recurrent neural network and the convolutional neural network.
Based on the feature extraction method shown in Figure 3, it can be known that the solution of the embodiment of this specification only needs to input the short-term features of the target object into the RNN+CNN model, that is, it is extracted from the model mechanically and presents both short-term characteristics and long-term characteristics. The target features of, can be used to more comprehensively describe the target object, and dig out hidden features that are difficult to find manually.
In addition, as shown in FIG. 4, an embodiment of this specification also provides a neural network training device 400, which includes:
The first processing module 410 inputs the first short-term feature set under the target classification corresponding to the sample object into the recurrent neural network to obtain the second short-term feature set; wherein, each short-term feature in the first short-term feature set corresponds to the same First time granularity;
The first combination module 420 combines the second short-term feature set into a long-term feature set in chronological order, wherein each long-term feature in the long-term feature set corresponds to the same second time granularity, and the second time granularity is larger than the first time granularity. One time granularity;
The second processing module 430 inputs the long-term feature set to the convolutional neural network to obtain the target feature set under the target classification corresponding to the target object;
The training module 440 inputs the target feature set to a classification model for identifying the target classification, so as to train the recurrent neural network and the convolutional neural network based on the recognition result of the classification model for the sample object .
Based on the training device shown in Figure 4, it can be known that the solution of the embodiment of this specification adopts the RNN+CNN model structure. During the training process, short-term features are formed into long-term features, and the long-term features are further converted into single-dimensional target features. Input to the classifier, and adjust the parameters of RNN and CNN according to the output result of the classifier to achieve the training purpose. Obviously, the whole training process uses both short-term features and long-term features, which not only greatly improves training efficiency, but also enables the model to learn the invisible connection between short-term features and long-term features, thereby obtaining better model performance.
Optionally, during execution of the training module 440, the target feature set is specifically used as the input of the classification model used to identify the target classification, and the label of the sample object is used as the output of the classification model to target the target feature set based on the classification model. The recognition result of the sample object is trained on the cyclic neural network and the convolutional neural network, wherein the label of the sample object is used to indicate whether the sample object conforms to the target classification.
Optionally, the recurrent neural network includes at least one of the following:
Long and short-term memory network, gated loop unit network, and self-attention mechanism network.
Optionally, the convolutional neural network includes: a text roll-in loop network.
Optionally, the convolutional neural network includes:
Convolutional layer, convolution processing the long-term feature set to obtain the convolutional layer output feature set;
The pooling layer, based on the maximum pooling algorithm and/or the mean pooling algorithm, performs pooling processing on the output feature set of the convolutional layer to obtain the output feature set of the pooling layer;
The fully connected layer converts the output feature set of the pooling layer into a single-dimensional target feature set.
Optionally, the sample object is a payment application user, the target is classified as a financial risk, and the first short-term feature set includes short-term features of at least one of the following feature dimensions:
The total transaction amount, total number of transactions, and total number of counterparties corresponding to the payment application user at each first time granularity.
Obviously, the training device of the embodiment of the present specification can be used as the execution body of the training method shown in FIG. 1, and therefore can realize the functions of the training method in FIGS. 1 and 2. Since the principle is the same, this article will not go into details.
In addition, as shown in FIG. 5, an embodiment of this specification also provides a feature extraction device 500, including:
The third processing module 510 inputs the first short-term feature set under the target classification into the recurrent neural network to obtain the second short-term feature set; wherein, each short-term feature in the first short-term feature set corresponds to the same First time granularity;
The second combination module 520 combines the second short-term feature set into a long-term feature set in chronological order, wherein each long-term feature in the long-term feature set corresponds to the same second time granularity, and the second time granularity is larger than the first time granularity. One time granularity;
The fourth processing module 530 inputs the long-term feature set to the convolutional neural network to obtain the target feature set under the target classification corresponding to the target object;
Among them, the recurrent neural network and the convolutional neural network input the target feature set of the sample object into the classification model that recognizes the target classification, and based on the recognition result obtained by the classification model, the recurrent neural network and The convolutional neural network is obtained by training, and the target feature set of the sample object is determined based on the recurrent neural network and the convolutional neural network.
Based on the feature extraction device shown in Figure 5, it can be known that the solution of the embodiment of this specification only needs to input the short-term features of the target object into the RNN+CNN model, that is, it is extracted by the model mechanically and presents both short-term characteristics and long-term characteristics. The target features of, can be used to more comprehensively describe the target object, and dig out hidden features that are difficult to find manually.
Obviously, the feature extraction device of the embodiment of the present specification can be used as the execution subject of the feature extraction method shown in FIG. 3, and therefore can realize the function of the feature extraction method in FIG. 3. Since the principle is the same, this article will not go into details.
Fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present specification. Please refer to FIG. 6, at the hardware level, the electronic device includes a processor, and optionally an internal bus, a network interface, and a memory. Among them, the memory may include internal memory, such as high-speed random-access memory (Random-Access Memory, RAM), and may also include non-volatile memory (non-volatile memory), such as at least one magnetic disk memory. Wait. Of course, the electronic equipment may also include hardware required by other businesses.
The processor, network interface, and memory can be connected to each other through an internal bus, which can be an ISA (Industry Standard Architecture) bus or PCI (Peripheral Component Interconnect) Bus or EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of presentation, only one double-headed arrow is used in FIG. 6, but it does not mean that there is only one busbar or one type of busbar.
Memory, used to store programs. Specifically, the program may include program code, and the program code includes computer operation instructions. The memory may include internal memory and non-volatile memory, and provide instructions and data to the processor.
Among them, the processor reads the corresponding computer program from the non-volatile memory to the internal memory and then runs it to form a training device for the neural network on the logical level. The processor executes the programs stored in the memory, and is specifically used to perform the following operations:
Input the first short-term feature set under the target classification corresponding to the sample object to the recurrent neural network to obtain the second short-term feature set, wherein each short-term feature in the first short-term feature set corresponds to the same first time granularity;
Combining the second short-term feature set into a long-term feature set in a chronological order, wherein each long-term feature in the long-term feature set corresponds to the same second time granularity, and the second time granularity is greater than the first time granularity;
Input the long-term feature set to the convolutional neural network to obtain the target feature set under the target classification corresponding to the target object;
The target feature set is input to a classification model for recognizing the target classification, so as to train the recurrent neural network and the convolutional neural network based on the recognition result of the classification model for the sample object.
Among them, the processor reads the corresponding computer program from the non-volatile memory to the internal memory and then runs it. At the logical level, it can also form a feature extraction device. The processor executes the programs stored in the memory, and is specifically used to perform the following operations:
Inputting the first short-term feature set of the target object belonging to the target classification to the recurrent neural network to obtain a second short-term feature set, wherein each short-term feature in the first short-term feature set corresponds to the same first time granularity;
Combining the second short-term feature set into a long-term feature set in a chronological order, wherein each long-term feature in the long-term feature set corresponds to the same second time granularity, and the second time granularity is greater than the first time granularity;
Input the long-term feature set to the convolutional neural network to obtain the target feature set under the target classification corresponding to the target object;
Wherein, the cyclic neural network and the convolutional neural network input the target feature set of the sample object into a classification model that recognizes the target classification, and obtain the recognition result for the sample object based on the classification model. The neural network and the convolutional neural network are trained, and the target feature set of the sample object is determined based on the recurrent neural network and the convolutional neural network.
The above-mentioned training method disclosed in the embodiment shown in FIG. 1 of this specification or the feature extraction method disclosed in the embodiment shown in FIG. 3 is implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, the steps of the above method can be completed by hardware integrated logic circuits in the processor or instructions in the form of software. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (Network Processor, NP), etc.; it may also be a digital signal processor (DSP), a dedicated Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The methods, steps, and logic block diagrams disclosed in the embodiments of this specification can be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like. The steps of the method disclosed in the embodiments of this specification can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can be located in random memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable and writeable programmable memory, register and other mature storage media in the field in. The storage medium is located in the memory, and the processor reads the information in the memory, and completes the steps of the above method in combination with its hardware.
It should be understood that the electronic device of the embodiment of this specification can realize the function of the above-mentioned training apparatus in the embodiment shown in FIG. 1 and FIG. 2 or the function of the above-mentioned feature extraction apparatus in the embodiment shown in the figure. Since the principle is the same, this article will not go into details.
Of course, in addition to the software implementation, the electronic equipment in this specification does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., which means that the execution body of the following processing flow is not limited to each logic unit , It can also be a hardware or logic device.
In addition, the embodiment of this specification also proposes a computer-readable storage medium. The computer-readable storage medium stores one or more programs, and the one or more programs include instructions.
Wherein, when the instruction is executed by a portable electronic device including multiple application programs, the portable electronic device can execute the training method of the embodiment shown in FIG. 1, and is specifically used to execute the following methods:
Input the first short-term feature set under the target classification corresponding to the sample object to the recurrent neural network to obtain the second short-term feature set, wherein each short-term feature in the first short-term feature set corresponds to the same first time granularity;
Combining the second short-term feature set into a long-term feature set in a chronological order, wherein each long-term feature in the long-term feature set corresponds to the same second time granularity, and the second time granularity is greater than the first time granularity;
Input the long-term feature set to the convolutional neural network to obtain the target feature set under the target classification corresponding to the target object;
The target feature set is input to a classification model for recognizing the target classification, so as to train the recurrent neural network and the convolutional neural network based on the recognition result of the classification model for the sample object.
Or, when the instruction is executed by a portable electronic device that includes multiple application programs, the portable electronic device can execute the feature extraction method of the embodiment shown in FIG. 3, and is specifically used to execute the following method:
Inputting the first short-term feature set of the target object belonging to the target classification to the recurrent neural network to obtain a second short-term feature set, wherein each short-term feature in the first short-term feature set corresponds to the same first time granularity;
Combining the second short-term feature set into a long-term feature set in a chronological order, wherein each long-term feature in the long-term feature set corresponds to the same second time granularity, and the second time granularity is greater than the first time granularity;
Input the long-term feature set to the convolutional neural network to obtain the target feature set under the target classification corresponding to the target object;
Wherein, the cyclic neural network and the convolutional neural network input the target feature set of the sample object into a classification model that recognizes the target classification, and obtain the recognition result for the sample object based on the classification model. The neural network and the convolutional neural network are trained, and the target feature set of the sample object is determined based on the recurrent neural network and the convolutional neural network.
It should be understood that the above instructions, when executed by a portable electronic device including multiple applications, can enable the training device described above to implement the functions of the embodiments shown in FIG. 1 and FIG. The feature extraction device described above implements the functions of the embodiment shown in FIG. 3, and will not be repeated here.
Those skilled in the art should understand that the embodiments of this specification can be provided as methods, systems or computer program products. Therefore, this specification may adopt the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware. Moreover, this manual can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program codes. .
The foregoing describes specific embodiments of this specification. Other embodiments are within the scope of the attached patent application. In some cases, the actions or steps described in the scope of the patent application may be performed in a different order from the embodiment and still achieve desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown in order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
The above are only examples of this specification, and are not intended to limit this specification. For those skilled in the art, this specification can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification shall be included in the scope of the claims of this specification. In addition, all other embodiments obtained by a person of ordinary skill in the art without creative work shall fall within the protection scope of this document.
