AU2018101531A4 - Stock forecast model based on text news by random forest - Google Patents

Abstract

The main purpose of this project is to use random forest (RF) algorithm to analyze the correlation between the stock news on the historical days and the ups and downs of the stocks the next day. We then find out the information hidden behind these data by sorting and screening stocks, that is, the key words or key events related to the rise and fall of the stocks to predict the rise and fall of the stocks in the near future. By using this method, after entering the news data of the stock market on that day, we can predict the rise and fall of one stock the next day. This relatively accurate method can help shareholders get rid of the risk of stock investment and even can guarantee stable investment returns. This model uses random forest algorithm to carry out excavating and classifying the information of text news. Random forest is a combinatorial classifier, which can be used for the classification and screening of the stocks. The essence of it is a set of tree classifiers. Among them, the base classifier H (x, beta k) is a classification decision tree constructed by CART algorithm without pruning. x is the input vector and beta k is an independent and identically distributed random vector which determine the growth process of the single tree (base classifier). The output is determined by a simple majority voting method. READ ALL THE CONTENTS INPUT OF THE NEWS DATA TRAIN THE CLEANING W2V MODEL READ THE CONTENTS ACCORDING TO THE DATES TAKE T HE AVERAGE OF THE WORD VECTORS VECTORISATION INPUT(x) INPUT(y) RANDOM Fig.1 General Flow-Chart

Description

1 TITLE

Stock forecast model based on text news by random forest

FIELD OF THE INVENTION

This model involves using natural language algorithm to excavate hidden information behind historical text news and analyse the relationship between text news data and the ups and downs if the stock in order to forecast stock prices in the short-term.

BACKGROUND OF THE INVENTION

The rapid development of cloud computing, Internet of Things, social networking and other emerging information technologies has led to a dramatic increase in global data volume, and has pushed mankind into the era of big data. Big data presents many distinctive characteristics, and the amount of data increases exponentially. There are different types of data and the proportion of semi-structured and unstructured data is increasing rapidly. When the amount of data increases to a certain scale, the hidden value of the data increases. However, as time goes by, the validity of data value decreases rapidly, and the requirement of data computing ability and data authenticity increases continuously. Internet, finance, health care, business, government management, and security are the six most eye-catching applications of big data. Financial data is the second largest and most remarkable application which is the key of stock market forecast and financial analysis. Nowadays, internet plays a more and more important role in our daily life. A growing number of investors obtain related information through the Internet and deliver this information to other investors, which provides investors with favourable conditions to obtain investment information and make investment decisions.

Without excavating and analysis, massive information is just a bunch of meaningless data. The method of analysing this massive information and extracting the valuable information hidden behind them is the data mining technology. Data mining is a new field of large data applications. Its goal is to get the results that users are interested in through the analysis of historical data. There is a huge amount of text news data imported into the data warehouse every day in the huge market. These data undoubtedly have a great impact on the investors’ investment decisions, and then affect the short-term stock prices. Economists analyse the relationship between news and stock prices to dig for information hidden behind these data, which has pretty important reference value for investors to make correct investment decisions, for the healthy development of the stock market, and for the government departments to introduce new plans. Because of the randomness and 2complexity of stock news data, it is more difficult to discover the underlying laws, which is a more challenging research field.

As a very important part of the market economy, stock has attracted wide attention and research all the time. Since the emergence of stocks, many analytical methods have been gradually constructed, such as Dow's analysis, K-line diagram analysis, point diagram analysis, moving average method, morphological analysis, trend analysis, angle analysis and so on. With the popularization and application of computer technology in the field of securities analysis, new index analysis methods has been constantly introduced. However, strictly speaking, these methods are only analytical means which can not directly predict the dynamics of the stock market. In addition, people tried to build models to predict stock market by means of regression analysis and other statistical methods. However, the traditional forecasting technology or methods have a very big difficulty, that is, the amount of data need to be processed is very large. Because the stock market is influenced by many factors, such as political and economic factors, its internal principles are very complex and the cycle of some change rules may be a year or even a few years, so it is necessary to analyse a large number of data to get what you want, while the traditional forecasting technologies can not draw very satisfactory results. In recent years, great progress has been made in the research and development of data mining technology. The application of various data mining technologies promote people's ability to analyse and process massive data, and also bring better economic benefits to people. Therefore, data mining technology will have great potential in stock market forecasting. A tragic rear-end collision occurred in Wenzhou section of Zhejiang Province. On the first trading day after the accident, the Shanghai Stock Exchange Index fell by 2.96%.Similarly, on August 25, 2011, former Apple CEO Steve Jobs announced his resignation, and on the first trading day after that, the company's stock fell by 0.65%. From those two events above, we know that once a major hot news is reported, it will immediately cause fluctuations in the relevant stock prices, and the stock prices will almost always be absorbed by the market when things happened. But in fact, news or events with different characteristics will have different impacts on the stock market, which requires in-depth mining and machine teaming of text news to find out which market hot spots will impact the stock prices, and then help investors capture hot news.

Through the analysis of literature research, it is found that scholars at home and abroad have carried out researches on stock forecasting relatively early, but at present the domestic researches on using text news data to predict the rise and fall of the stocks is still in its infancy. The infancy mainly presents in the following aspects: first, the insufficient collection and research on text big 3data .Large amount of data does not mean high value of data. A large number of unrelated data have brought great negative impact on data analysis and processing. Therefore, it is of great theoretical and practical significance to reduce the impact of unrelated data on stock prices in the stage of data acquisition. Secondly, there is insufficient research on text big data mining. In the process of reading the literature, we find that the current use of text mining technology to analyse news and other text content and study the impact of the analysis results on stock prices mainly concentrated on several machine learning algorithms, such as support vector machine, genetic algorithm and reverse neural network which are complex, with large amount of data and computation and long processing time. Finally, there is insufficient research on the influencing factors of stock prices fluctuation. We lack multi-angle consideration of the factors and lack the theoretical standard of model validation.

By searching all kinds of literature journals, we realize that text mining is the key to excavate the factors which affect the rise and fall of the stock prices. At present, there is a shortage of researches on stock prices forecast using text mining technology, which combines historical text news, daily historical ups and downs of the stocks with machine learning method to realize short-term stock prices forecast. Therefore, we make a innovation based on the lack of research in this field. We use random forest algorithm to excavate the implicated information behind those text news, and to classify those text news. Then we can build a relationship between the news and the ups and downs of the stocks to forecast the rise and fall of the stocks in the short term.

SUMMER OF THE INVENTION

The method described in this paper builds a stock prediction system supported by text data mining. This model uses a great deal of text news data from 2013 to 2016.These data are collected from the daily news on Chinese finance and economics webpage like Sina Economic and East Money net, including daily raise or fall, the comments of some Stock Investors and some official communique. We make good use of the python language and use random forest to classify the text news. After calculating, training and testing, we obtain the stock forecast model. The testing results indicate that this model can forecast the rise and fall of the stock price with an accuracy of 81.2%, which means that this model has great reference for investment.

After obtaining the raw data, we first organise these data by arranging the daily stock news and industry news of each stock together respectively. Then the organised data of all nine stocks are arranged into a worksheet to form a whole data set. After arrangement, we need to use python to read this whole data set and carry out data mining and analysis. 4

Let me introduce our forecast model in general. First, we input preliminarily reorganised data and train the word2vector model (a deep learning model developed by Google for word vectorisation). Our raw data are divided into industry news and stock news, and data for each stock includes dates, stock text news, and stock price tags. We arrange news data of all nine stocks together, remove all the unrelated information, and then input all the text data into word2vector. After training, we get the word2vector model which can turn the text data into vectors. Meanwhile, since the stock data collected from Internet are massive, consistent and mixed with plentiful meaningless words, the data cannot be read to the model directly. Thus, we clean the text news data to eliminate the negative influence of gibberish. We clean the data using some special text processing techniques word segmentation and stop words removing to extract data to be used for training the random forest classifier. Next, we match the cleaned text data for each stock at each day with word vectors in the word2vector model to vectorize the whole text. Finally, we input vector A (text data) and Y (tag) into random forest model.

Above is a general introduction of the procedures of our forecast model. During the whole process, there are still several steps of our systems that are worth more detailed explanations.

Firstly, the cleaning of raw data is of great importance since the stock data collected from Internet are massive, consistent and mixed with plentiful meaningless words, the data cannot be read to the model directly. Therefore, we use a package named Jieba segmentation which is specially designed to cut long Chinese sentences to separated words. We then delete the stop words like ‘is’, ‘and’ and ‘so’ in our text, according to the Harbin Institute of Technology Stop words List, in order to reduce redundancy. In this way, we obtain the cleaned data separated by blank and get ready for the next step: text vectorisation.

Secondly, training the Random Forest is also one of the most important steps of the whole process. The first thing is initialisation, about which we will go in to details later. After that, we input a general training set which includes all training samples. Next, we generate new training sets based on the given general training set. Finally, we train each decision tree inside the forest with its corresponding training set.

Thirdly, testing of the classifier is necessary as well. We input the pre-allocated data into the model to test the validity of the model. Before the test, we divided the data into training set and testing set. Every piece of data in the training set will be processed by each decision tree and the trees will output the results respectively, and the whole model will give a final output with voting mechanism. 5

As above, we summarise the overall process and key steps of this stock prediction model. The model uses random forest to classify and extract information of a great deal of text news data to forecast the rise and fall of the stocks. Random forest is not a novel arithmetic, which has been used to solve a lot of problems, but it is novel to use this arithmetic to mine text news data and then to predict the stock prices. Thus we have innovatively applied random forest algorithm on stock prediction.

DESCRIPTION OF THE DRAWINGS

Fig.l General Flow-Chart

Stepl Input data and the data are preliminarily reorganized as the data form that can construct word2vector.

Raw stock data are divided into industry new and stock new. Each stock data includes dates, stock text news data, and stock price tags. Our raw data are divided into industry news and stock news, and data for each stock includes dates, news text, and stock price tags. We arrange news data of all nine stocks together, remove all the unrelated information, and then input all the text data into word2vector.

Step2 Input all the text data into word2vector. After training, we get a word2vector model which can match each word with its corresponding vectorised representation. Meanwhile, we clean the data with some special text processing techniques.

Word2vector: See Fig.2

Data Cleaning: See Fig.2

Step3 Input the text data obtained from step 2 into the word2vector model to vectorize the text data.

Step4 Input vector X (text data) and Y (tables) to train the random forest classifier.

Fig.2 Data Cleaning

The data are collected form the daily news on Chinese finance and economics webpage like Sina Economic and East Money net, including daily raise or fall, the comments of some Stock Investors and some official communique.

Since the stock data collected from Internet are massive, consistent and mixed with plentiful 6meaningless words, the data cannot be read to the model directly. Thus, we clean the Chinese text data to eliminate the negative influence of gibberish.

Step 1 We use a technology named Jieba segmentation which is specially designed to cut the long Chinese sentences to some single words.

Step 2 We delete stop words in our data like ‘is’, ’and’, ’so’, according to the Harbin Institute of Technology Stop words List to redundancy.

Step 3 Since the significant words are usually Chinese characters, we filtrate the data and reserve only Chinese characters to increase average information included in each word.

In this way, we obtain the cleaned data ready for text vectorisation.

Fig.3 Training the Random Forest Classifier:

Stepl Initialisation: See Fig.4

Step2 Input general training set S: General training set includes all training samples and are in the form of a matrix. The first 100 columns of the matrix are vectorised representations of text contents of each sample (1 stock at 1 day). The last column of the matrix are the labels (ups and downs of the stock: -1 or 1).

Step3 Bagging (Boostrap Aggregating): Generating t new training sets based on the given general training set S. Each new set extracts the same amount of samples randomly from set S with replacement and each submodel (a single decision tree) is trained with its corresponding subset.

Step4 Training each decision tree inside the forest: See Fig.5

Fig.4 Initialisation

Param 1 nestimators (default = 10): Using nestimators = 100

Number of trees in the forest.

Param 2 criterion (default = ”gini”): Using criterion = default.

The function to measure the quality of a split.

Available options are: 1. “gini”: Defining the best split as the split that reduces Gini Impurity in the largest scale. 72. “entropy”: Defining the best split as the split that gives the most information gain.

Param 3 max_features (default = ’’auto”): Using max_features = default.

The number of features to consider when looking for the best split.

Available options are: 1. input int variable: consider maxfeatures features at each split. 2. input float variable: max features is the proportion of features considered at each split, which means that int(max_features* n features) features are considered. 3. “auto” (or “sqrt”): max_features(=sqrt(n_features)) are considered at each split. 4. “Iog2”: max_features (=log2(n_features)) are considered at each split. 5. None: every single feature is considered at each split.

Param 4 maxdepth (default = none) Using maxdepth = 5

The maximum depth of the tree.

Available options: 1. input int variable: The depth of the trees will be no more than max depth. 2. None: Trees will expand until all leaves are pure or until all leaves contain less than minsamplessplit samples.

Param 5 min samples split (default= 2): Using min samples split =default.

The minimum number of samples required to split an internal node.

Available options: 1. input int variable: each split should at least contain min samples split samples. 2. input float variable: min samples split is the proportion of features that each split should contain, which means that ceil(min_samples_leaf * n samples) is the minimum number of samples in each split.

Param 6 min_samples_leaf(default= 1): Using minsamplesleaf = default.

The minimum number of samples required to be at a leaf node.

Available options: 1. input int variable: each leaf node should at least contain min samples leaf samples. 2. input float variable: min samples leaf is the proportion of features that each leaf node should contain, which means that ceil(min_samples_leaf * n samples) is the minimum number of samples in each leaf node. 8Fig.5 Training with submodel (training ith decision tree)

Stepl Input training set Si:

Step2 Set the complete training set as root node.

Step3 Find the best split of the node:

The samples are divided into 2 parts with a certain feature and a threshold chosen. Our model is using Gini Impurity in the estimation of each split. In the procedure, sqrt(total amount of features) are taken into consideration and we choose a division in which the Gini Impurity of the samples drops mostly.

Step4 Split the node into to 2 leaf nodes.

Step5 Repeat Step 3 and 4 until terminal conditions are satisfied:

Condition 1: Consider the number of samples in a leaf node. If the number is less than minsamplesleaf, the expansion of the decision tree should be stoped and the node should not be considered as a leaf node.

Condition 2: Consider the number of samples in a split. If the total number of samples are less than minsamplessplit, the expansion of the decision tree should be stoped and the node should not be split into 2 parts.

Step6 Output classified result: The classified result is the class which takes the largest part of all the samples in the node.

Fig.6 Testing the classifier

We input the pre-allocated data into the model to test the validity of the model.

Stepl Before the test, the data are divided into training set and testing set with a ratio of 9:1. Then the testing set data is imported into the model one by one.

Step2 Every piece of data will be processed by each decision tree and the trees will output the results respectively.

Here is a description of data processing within each decision tree: The data first enter the root node, and decide to enter its left child node or right child node according to the chosen feature Xk at its current node and a threshold th until reach a leaf node, and output the predicted value and the 9 probability.

Step3 The above steps are carried out repeatedly to get all predicted value (probability that the input data is classified as each class) of all the decision trees. Then the average probability of each class is calculated and the class with highest probability will be outputted as the result. EXAMPLE 1

We input the text news of one day into the stock forecast model, and then input the code of a stock you want to know. By calculating and analysing the text news data using this model, we can forecast the rise and fall of the stock price in the next trading day. Based on the predicted ups and downs, stockholders can decide whether to buy or sell the stock on the next trading day. This will help investors capture hot news information, and then help them to make more accurate investment judgments and decisions. EXAMPLE 2

This stock price forecast model can be used not only to forecast the stock price, but also to classify, screen and excavate other text data to find other hidden information you want. For example, we can use this model to excavate investors' comments on various financial websites, and then dig out the emotional keywords of some investors. Using these emotional keywords, we can predict investors' investment decisions in the short term, such as the purchase and sale of a certain stock on the following day. Then we can also predict the rise and fall of a stock in the short run.

Claims (2)

1. CLAIM:
2. 1. A method is mainly based on random forest algorithm for stock price prediction, in which the use of text news data for stock price forecast mainly focuses on the other machine learning algorithms, such as support vector machine, genetic algorithm and back-propagation neural network, which are comparatively more complex due to its large amount of data and computation as well as long processing time.; compared with these machine learning algorithms, random forest has its own advantages; random forest algorithm has good fault tolerance and robustness because of its randomness of training sets and attribute which is compatible with the lots of abnormal conditions and distractors among current stock market; compared with the support vector machine, the performance of random forest algorithm is significantly superior in classification, in addition, the random forest algorithm has high prediction accuracy and is not prone to overfitting; based on the above, we choose random forest algorithm as an algorithm for data excavating and analysis; the application of random forest algorithm to this field is relatively rare, so it is also an innovation in this field.