KR102504880B1 - System and method for producing customized investment portfolio - Google Patents

Abstract

The present invention relates to a method for generating a customized investment portfolio, which provides an investment portfolio appropriate for a user's investment tendencies or investment goals, and a system thereof. According to one embodiment of the present invention, the method for generating a customized investment portfolio comprises the steps of: providing an investment application to a user terminal; storing a user's investment application activity data through the investment application; storing the user's investment goal data through the investment application; storing market data and user asset data; using the market data, the asset data, the investment application activity data, and the investment target data as learning data to train a deep reinforcement learning model to determine items to propose to purchase; and determining purchase proposal items through the deep reinforcement learning model. At least a first deep reinforcement learning algorithm of the deep reinforcement learning model is an algorithm for determining a purchase proposal item according to the user's investment goal, and uses a reward function determining the value of the reward acquired when buying, selling, or holding the item, wherein the reward function is defined differently depending on the investment goal, and is evaluated on the basis of the achievement of the user's investment goal.

Description

Method and system for creating a customized investment portfolio {SYSTEM AND METHOD FOR PRODUCING CUSTOMIZED INVESTMENT PORTFOLIO}

The present invention relates to a method and system for creating a customized investment portfolio. More specifically, it relates to a method and system for generating a user-customized investment portfolio based on deep reinforcement learning.

In asset investment, investment stocks or portfolios are recommended based on the customer's investment propensity, including the customer's target rate of return.

Such investment items or portfolio recommendations are generally made through professional consultation through branch staff, etc., but recently, in many cases, they are made through automated portfolio creation tools such as robot advisors.

In the case of current automated investment items or portfolios such as robot advisors, the customer's investment propensity is identified through survey questions, and an appropriate portfolio model is automatically suggested.

However, there is a problem in that it is difficult to accurately grasp the individual's investment propensity in various aspects because these questionnaire items are not provided differently for each individual but are identical in bulk. In addition, due to these problems, it is difficult to set investment goals customized for each individual.

In addition, due to the limitations of identifying such investment propensity, the investment propensity identified by the survey questions is only to the extent that it is grouped into a plurality of groups, and the decision of the investment item or portfolio is based on the market situation.

KR 10-2021-0004812 A (2021. 01. 13.)

The present invention is to solve the above problems, using the user's investment app activity data to identify the user's investment propensity or investment goal and to provide a customized investment portfolio based on deep reinforcement learning. It is intended to provide a generation method and a system therefor.

The problem to be solved by the present invention is not limited to the contents mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

According to an embodiment of the present invention, a method for creating a customized investment portfolio includes providing an investment app to a user terminal, storing user's investment app activity data through the investment app, and investing the user through the investment app. The step of storing target data, the step of storing market data and asset data of the user, and determining the purchase proposal item by using the market data, the asset data, the investment app activity data, and the investment target data as learning data. Learning a deep reinforcement learning model to perform a deep reinforcement learning model; As an algorithm for determining a proposed item to buy according to, the first deep reinforcement learning algorithm uses a reward function that determines a value of a reward obtained when buying, selling, or holding an item, and the reward function is the investment target. It is defined differently depending on the user's investment goal, and is evaluated based on the achievement of the user's investment goal.
The first deep reinforcement learning algorithm of the deep reinforcement learning model is learned using asset data and market data for each user as learning data.
The deep reinforcement learning model uses at least a second deep reinforcement learning algorithm, and the second deep reinforcement learning algorithm is an algorithm for calculating user preference, and uses app activity data for each user and asset data for each user as learning data. is learned by

The deep reinforcement learning model uses at least a third deep reinforcement learning algorithm, and the third deep reinforcement learning algorithm is an algorithm for calculating an investment target of the user, and includes a first deep reinforcement learning algorithm and a second deep reinforcement learning algorithm. It is learned using the result values of the algorithm and investment target data for each user as learning data.

delete

delete

The investment target for each user includes at least a target rate of return, a maximum loss rate, and an investment period, and the reward function is

is defined as

The step of determining a purchase offer item using the deep reinforcement learning model includes selecting a buy offer item pool consisting of a plurality of buy offer items through a first deep reinforcement learning algorithm, and using a second deep reinforcement learning algorithm. The method may include scoring the purchase proposal items of the purchase offer item pool based on the calculated preference of the user, and determining a plurality of purchase offer items in a high order as the purchase offer items.

As a customized investment portfolio creation system according to an embodiment of the present invention,

An investment app service module that provides investment services by providing investment apps to user terminals, and a database unit including investment app activity data for each user, asset data for each user, market data, and target data for each user, and determining purchase proposal items In the investment portfolio creation module, the deep reinforcement learned to determine a purchase offer item using the market data, the asset data, the investment app activity data, and the investment target data as learning data. A learning model is applied, and at least a first deep reinforcement learning algorithm of the deep reinforcement learning model is an algorithm for determining a purchase proposal item according to a user's investment goal, and the first deep reinforcement learning algorithm is to buy or sell an item. Alternatively, a compensation function for determining a value of compensation obtained when holding is used, and the compensation function may be defined differently according to the investment target and evaluated based on the achievement of the user's investment target.

According to the user-customized investment portfolio generation method and system according to an embodiment of the present invention, the user's investment propensity or investment goal can be identified using the user's investment app activity data, and an investment portfolio suitable for it can be provided.

1 is a conceptual diagram schematically illustrating an entire system, that is, a customized investment portfolio creation system 10 according to an embodiment of the present invention.
2 shows an example of an investment information (public offering stock information) screen provided by an investment app. 3 shows an example of a screen of a list of items of interest designated by a user through an investment app. 4 shows an example of a screen through which a user shares investment-related content or opinions on items of interest with other users through an investment app. FIG. 5 shows an example of a screen that provides a purchase offer item to a user through an investment app.
6 is a configuration diagram of a database according to an embodiment of the present invention.
7 is an exemplary view illustrating deep reinforcement learning algorithms of the investment portfolio generation module 200.
8 is an operational flow diagram illustrating a method for creating a customized investment portfolio according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail so that those skilled in the art can easily practice with reference to the accompanying drawings. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. In addition, in order to clearly describe the embodiment of the present invention in the drawings, parts irrelevant to the description are omitted.

Terms used in this specification are only used to describe specific embodiments and are not intended to limit the present invention. Singular expressions may include plural expressions unless the context clearly dictates otherwise.

In this specification, terms such as "include", "have" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one Or it may be understood that it does not exclude in advance the possibility of the existence or addition of other features, numbers, steps, operations, components, parts, or combinations thereof.

In addition, the components appearing in the embodiments of the present invention are shown independently to represent different characteristic functions, and it does not mean that each component is made of separate hardware or a single software component unit. That is, each component is listed and described as each component for convenience of description, and at least two components of each component may be combined to form one component, or one component may be divided into a plurality of components to perform a function. An integrated embodiment and a separate embodiment of each of these components are also included in the scope of the present invention unless departing from the essence of the present invention.

In addition, the following embodiments are provided to more clearly explain to those with average knowledge in the art, and the shapes and sizes of elements in the drawings may be exaggerated for clearer explanation.

Hereinafter, a preferred embodiment according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a conceptual diagram schematically illustrating an entire system, that is, a customized investment portfolio creation system 10 according to an embodiment of the present invention. Referring to FIG. 1, a customized investment portfolio system 10 according to an embodiment of the present invention is composed of a server including an investment app service module 100, an investment portfolio creation module 200, and a database unit 300. can Here, the customized investment portfolio system 10 may be composed of one server or a plurality of servers. That is, a function performed in one server may be divided into a plurality of servers and performed. Hereinafter, for convenience, it will be described on the premise that all functions are performed in one server.

Here, the investment app service is a service in which an investment-related app is installed and provided on a user terminal, and is typically provided by an investment-related platform service provider (investment-related services from provision of investment information to transaction connection and asset management). service provider). The investment app service module 100 communicates with the investment app installed on the user terminal, provides investment-related services, and receives user's investment app activities from the investment app and stores them in the database unit 300 .

For example, FIG. 2 shows an example of an investment information (public offering stock information) screen provided by an investment app. 3 shows an example of a screen of a list of items of interest designated by a user through an investment app. 4 shows an example of a screen through which a user shares investment-related content or opinions on items of interest with other users through an investment app. FIG. 5 shows an example of a screen that provides a purchase offer item to a user through an investment app.

As such, the investment app provides investment-related content to the user, and the user can designate an item of interest or share an opinion on the investment-related content or item of interest with other users. In addition, when a user purchases or sells a predetermined item through an investment app, the user may write a trading diary recording the record and provide it to the user, and the investment app may provide the user with a purchase proposal item. As will be described later, the determination of purchase proposal items (ie, investment portfolio creation) according to an embodiment of the present invention can be performed through the investment portfolio creation model 200, and the determined purchase proposal items are provided to the user through the investment app. can be provided.

6 is a configuration diagram of a database according to an embodiment of the present invention. As shown in FIG. 6, the database unit 300 includes investment app activity data 310 for each user, asset data 330 for each user, market data 350, target data 370 for each user, and learning data 390. ) may be included.

The investment app activity data for each user 310 , asset data for each user 330 , and target data for each user 370 are data stored for each user.

The user-specific investment app activity data 310 refers to data stored in which activities carried out through the investment app to which the user has signed up are transmitted to the investment service module 100 . For example, as a user uses an investment app, as investment app activity data, each user (1) trading journal writing record, (2) trading journal writing cycle, (3) interest item, (4) interest item registration cycle, ( 5) The nature of the content you are interested in, (6) the nature of the created content, and (7) the app activity cycle (app log information) can be stored. Here, the nature of the content of interest is, for example, the nature of the content designated by the user as the content of interest, and the nature may be a category of the content (eg, public offering stock, US stock). In addition, the nature of the created content may be a category of the investment-related content when the user shares an opinion on the investment-related content with other users.

The asset data 330 for each user is asset data for each individual and may be provided from a financial company. For example, asset data such as holding asset information and sale and purchase detail information may be provided from a plurality of financial companies through the My Data API service using personal information. The information provided from the financial company is refined and is asset data for each user to be used in the present invention, (1) current status information by item, (2) current status information by product, (3) cash holding status information, (4) average holding period information, ( 5) Risk preference information can be stored. Here, the risk preference information may be average stop loss information and realized profit rate information.

The market data 350 is investment market data and may be provided through Koscom, Nasdaq, Refinitiv, and the like. For example, the market data may include sector and product information of items by country.

User-specific investment target data 370 may be collected from the user through the investment app. For example, the user can enter a target rate of return, a maximum loss rate, and an investment period through an investment app, and this investment target information is transmitted to the investment app service module 100 through the investment app, and the user-specific investment target data 370 can be stored in

The learning data 390 is learning data for training the deep reinforcement learning algorithm of the investment portfolio generation module 200 to be described later, and includes the investment app activity data 310 for each user, asset data 330 for each user, and market data ( 350), it can be built by refining/integrating from the target data 370 for each user. Here, refinement/integration includes integrating the separated data and refining the data format. In particular, since the initial creators of the investment app activity data 310, market data 350, and asset data 330 for each user are different (eg, the asset data 330 for each user is provided by a financial institution), these In order to use the data as training data for deep reinforcement learning algorithms, it may be necessary to refine/integrate these data.

7 is an exemplary view illustrating deep reinforcement learning algorithms of the investment portfolio generation module 200. As shown in FIG. 7 , the investment portfolio generation module 200 may include a first deep reinforcement learning algorithm 220, a second deep reinforcement learning algorithm 240, and a third deep reinforcement learning algorithm 260. there is.

According to an embodiment of the present invention, the investment portfolio generation module 200 uses the learning data 390 to perform the first deep reinforcement learning algorithm 220, the second deep reinforcement learning algorithm 240, and the third deep reinforcement. The learning algorithm 260 is trained, and the first deep reinforcement learning algorithm 220, the second deep reinforcement learning algorithm 240, and the third deep reinforcement learning algorithm 260 are trained, and each user's investment app activity data and asset By inputting data and market data, an investment portfolio for each user can be determined. As will be described later, a value input by the user to the investment app or a value calculated by the third deep reinforcement learning algorithm 260 may be input as the investment target data of the individual user.

First, the first deep reinforcement learning algorithm 220 will be described as follows. The first deep reinforcement learning algorithm 240 is an algorithm for determining purchase proposals that can achieve the user's investment goal based on the investment goal input by an individual user or calculated by the third deep reinforcement learning algorithm 260. For example, when an individual user's investment goal is input, a value obtained by indexing the probability of achieving the individual user's investment goal through purchase of the corresponding stock for each stock can be output as a result value.

The first deep reinforcement learning algorithm 220 uses a reward function that determines a reward value obtained when buying, selling, or holding an item, but the reward function may be defined differently according to each user's investment goal. In other words, the first deep reinforcement learning algorithm 220 defines a reward function to reflect investment goals for each user. Here, the reward function means a function that determines the reward value obtained when an agent takes an action in reinforcement learning, and through this, the agent learns how to behave.

The agent goes through trial and error to obtain the maximum reward and learns to explore the most efficient path by itself through several iterations. Here, the action of the agent is selling, buying, or holding the item. In other words, by using the user's asset data and the user's investment activity data to better reflect the user's investment propensity, the user's customized investment goal is set, and a reward function that reflects the customized investment goal is applied. It becomes possible to provide an investment portfolio.

The first deep reinforcement learning algorithm 220 is an algorithm for selecting a pool of stocks to buy, and selects pools of stocks to buy that may be of interest by reflecting the user's past transaction history, buying behavior, and investment propensity. Therefore, asset data and market data for each user are used as learning data.

For example, the compensation function may be as follows.

Item (1) is a penalty item when the portfolio assets are less than the target amount.

α: Constant for weight adjustment

Pt+1: portfolio return from time t to time t+1

GOAL: Goal Amount

min: As an abbreviation of minimum, it is an operation that selects the smaller number between (Pt+1 - GOAL) and 0. If the portfolio return (Pt+1) is still less than the target amount (GOAL) (i.e. (Pt+1 - GOAL < 0), a penalty equal to the difference is imposed, and if the portfolio return (Pt+1) is less than the target amount If (GOAL) is exceeded (i.e., Pt+1 - GOAL > 0), the goal has been achieved, so 0 is selected and no penalty is given.

(1) The use of the Exponential function in item No. 1 is to impose a larger penalty when the current asset amount falls far short of the target amount.

Item (2) is an item for giving compensation when a profit is made, and it does not simply take the return of the current portfolio as a value, but takes as a value how much it exceeds the benchmark return. The benchmark portfolio is used as an equal distribution of total assets.

β: constant for weight control

Bt+1: Benchmark return from time t to t+1

Item (3) is an item related to Sharp ratio. The Sharpe ratio is an indicator of excess return per unit of risk and serves as a risk adjustment.

Item (4) is similar to item (3), that is, the Sharpe ratio item, which adjusts the risk so that the model does not make too aggressive choices.

θ: Constant for weight control

MDD (Max Drawdown): The maximum observed loss from the peak to the trough of a portfolio before reaching a new peak.

Item (5) means transaction cost. In other words, it is an item that reflects various additional costs such as transaction fees, transaction taxes, and transfer taxes when investing.

μ: Constant for weight adjustment

It: additional investment at time t

As described above, the reward function may be defined differently according to each user's investment goal. Each user's investment goal (target rate of return, maximum loss rate, investment period) is input from the user, but when it is actually input to the reward function, it is converted into a target amount called GOAL and reflected.

Xt is the current portfolio state, Ut is the action to be performed at time t (which stock to sell/buy/hold), and Ct is the next portfolio state determined by Xt and Ut.

Next, the second deep reinforcement learning algorithm 240 is a deep reinforcement learning algorithm for determining a purchase offer item according to the user's preference. For example, when individual user's investment app activity data 310 and asset data 330 are input, the deep reinforcement learning algorithm sets the individual user's preference for each item (exponentializes the probability that the individual user prefers the item) as the result value. can be output as

As described above, the investment app activity data for each user stores user activity information related to which stocks the user is interested in and how long the investment period is, and the asset data for each user stores the user's actual stocks. User information related to what stocks he currently holds and which stocks he has sold or bought in the past is stored. In addition, the purchase proposal item determined by the investment portfolio creation module 200 is recommended to the user through the investment app, and the result (ie, whether or not the user buys) is displayed through the investment app for each user of the database unit 300. It can be stored in activity data.

Accordingly, the second deep reinforcement learning algorithm 240 may be trained to determine the user's preferred purchase offer item using such data as learning data.

The second deep reinforcement learning algorithm 240 is an algorithm for calculating the user's preference, and the user's app activity data is the most important data. In other words, it learns the user's preference results by combining the user's app activity data and asset data.

The third deep reinforcement learning algorithm uses the investment target data as learning/input data in addition to the results of the first deep reinforcement learning algorithm (pool of stock candidates to buy) and the second deep reinforcement learning algorithm (user preference). Through this, the third deep reinforcement learning algorithm 260 may be trained to predict investment goals (target rate of return, maximum loss rate, and investment period) for each user. In this case, the calculated investment target is automatically provided to the user through the investment app, and the user can modify the automatically presented investment target as needed. The modified investment target is stored in the database unit 300 and can be used as learning data.

As will be described later, the investment portfolio generation module 200 uses the first deep reinforcement learning algorithm 220, the second deep reinforcement learning algorithm 240, and the third deep reinforcement learning algorithm 260 described above, and individual users You can decide which items to offer to buy.

For example, an investment goal calculated through the third deep reinforcement learning algorithm 260 may be used as an investment goal of an individual user, or a value input by an individual user through an investment app may be used.

For example, by using the first deep reinforcement learning algorithm 220, the user's investment goal can be achieved by culling proposed items to buy, and selecting them as a pool. Thereafter, the user's preference may be calculated for the purchase proposal items in the pool using the second deep reinforcement learning algorithm 240 . It is possible to select and propose a plurality of items in high order by scoring the items to be purchased based on the user's calculated preference. 5 is an exemplary diagram of an app window for recommending the selected purchase offering items to the user by scoring the purchase proposal items in the pool according to the user's preference and selecting three items with high rank.

According to the above-described embodiment of the present invention, even if a plurality of users have the same portfolio, (1) the investment portfolio creation module creates a pool of different purchase proposals according to customized investment goals; (2) Since the purchase proposal items in the pool are also selected differently according to the user's preference, the user is recommended different purchase proposal items (ie, customized investment portfolio for each user).

8 is an operational flow diagram illustrating a method for creating a customized investment portfolio according to an embodiment of the present invention.

Referring to Figure 8, according to an embodiment of the present invention, the server provides an investment app to the user terminal (S100). The user's activities in the investment app are stored in the investment app activity data of the server (S200). In addition, the user's investment goal entered through the investment app by the user is transmitted to the server, and stored in the investment target data of the server (S300).

The server stores asset data and market data for each user (S400). Asset data for each user may be provided from a financial institution.

Next, the investment portfolio creation engine of the server uses market data, asset data, investment app activity data, and investment goals as learning data to train a deep reinforcement learning model for determining purchase proposals (S500).

Here, the deep reinforcement learning model may include at least a first deep reinforcement learning algorithm, a second deep reinforcement learning algorithm, and a third deep reinforcement learning algorithm.

The first deep reinforcement learning algorithm is an algorithm for determining purchase proposal stocks according to the user's investment goal, the second deep reinforcement learning algorithm is an algorithm for determining purchase proposal stocks according to the user's preference, and the third deep reinforcement learning algorithm is an algorithm for determining purchase proposal stocks according to the user's preference. The reinforcement learning algorithm is an algorithm for determining the user's investment goal.

Next, the server determines a purchase offer item through a deep reinforcement learning model (S600). For example, by using the first deep reinforcement learning algorithm, the user's investment goal can be achieved by selecting the proposed purchase items, selecting them as a pool, and then calculating the user's preference for the proposed purchase items in the pool using a second algorithm. can do. By scoring the purchase offer stocks based on the user's calculated preference, a plurality of buy offer stocks with high rankings may be determined as the final purchase offer stocks.

100: investment app service module
200: investment portfolio creation module
300: database unit
310: investment app activity data per user
330: asset data per user
350: market data
370: investment target data by user
390: learning data

Claims (11)

As a method of generating a customized investment portfolio performed in the investment app service server,
providing an investment app to a user terminal;
storing investment app activity data for each user received from the investment app;
Storing the user-specific investment target data received from the investment app;
Storing market data and asset data for each user;
Learning a deep reinforcement learning model for determining purchase proposal items using the market data, the asset data for each user, the investment app activity data for each user, and the investment target data for each user as learning data;
Determining a purchase offer item through the deep reinforcement learning model,
The deep reinforcement learning model includes at least a first deep reinforcement learning algorithm, a second deep reinforcement learning algorithm, and a third deep reinforcement learning algorithm,
The first deep reinforcement learning algorithm is an algorithm for determining a purchase proposal item according to the user's investment goal, and the first deep reinforcement learning algorithm determines the value of the reward obtained when buying, selling, or holding the item. A compensation function is used, and the compensation function is defined differently according to the investment goal and evaluated based on the achievement of the user's investment goal,
The first deep reinforcement learning algorithm is learned using the user-specific asset data and market data as learning data,
The second deep reinforcement learning algorithm is an algorithm for calculating user preferences, and is learned using investment app activity data for each user and asset data for each user as learning data,
The third deep reinforcement learning algorithm is an algorithm for calculating the investment goal of the user, and the result values of the first deep reinforcement learning algorithm and the second deep reinforcement learning algorithm and the user-specific investment target data are used as learning data. become,
The investment target data for each user includes at least a target return rate, a maximum loss rate, and an investment period,
The investment app activity data for each user includes an item of interest, a period of registering an item of interest, and an active period of the app.
How to create a custom investment portfolio. delete delete delete According to claim 1,
The reward function is

defined as,
How to create a custom investment portfolio. According to claim 1,
In the step of determining stocks to offer to buy using the deep reinforcement learning model,
Selecting a purchase offer stock pool consisting of a plurality of buy offer stocks through a first deep reinforcement learning algorithm;
Scoring the purchase proposal items in the purchase proposal pool with the user's preferences calculated through a second deep reinforcement learning algorithm and determining a plurality of purchase proposal items as purchase proposal items with a high rank,
How to create a custom investment portfolio. As a customized investment portfolio creation system,
An investment app service module that provides an investment service by providing an investment app to a user terminal;
A database unit including investment app activity data for each user, asset data for each user, market data, and investment target data for each user;
Includes an investment portfolio creation module that determines stocks to buy,
In the investment portfolio creation module, a deep reinforcement learning model learned to determine purchase proposals using the market data, the asset data for each user, the investment app activity data for each user, and the investment target data for each user as learning data applied,
At least the first deep reinforcement learning algorithm of the deep reinforcement learning model is an algorithm for determining a purchase proposal item according to the user's investment goal, and the first deep reinforcement learning algorithm is obtained when buying, selling, or holding an item. A reward function for determining the value of the reward is used, and the reward function is defined differently according to the investment goal and evaluated based on the achievement of the user's investment goal,
The first deep reinforcement learning algorithm of the deep reinforcement learning model is learned using the user-specific asset data and the market data as learning data,
The deep reinforcement learning model uses at least a second deep reinforcement learning algorithm, and the second deep reinforcement learning algorithm is an algorithm for calculating user preference, and the investment app activity data for each user and the asset data for each user are used. It is learned using the learning data,
The deep reinforcement learning model uses at least a third deep reinforcement learning algorithm, and the third deep reinforcement learning algorithm is an algorithm for calculating the investment target of the user, and the first deep reinforcement learning algorithm and the second deep reinforcement learning algorithm are used. It is learned using the result values of the algorithm and the investment target data for each user as learning data,
The investment target data for each user includes at least a target return rate, a maximum loss rate, and an investment period,
The investment app activity data for each user includes an item of interest, a period of registering an item of interest, and an active period of the app.
A custom investment portfolio creation system. delete delete delete According to claim 7,
The reward function is

defined as,
A custom investment portfolio creation system.

Images