KR100979516B1 - Service recommendation method for network-based robot, and service recommendation apparatus - Google Patents

Abstract

Even if you do not have detailed information about the user and the service provided, you can learn the service usage pattern of the user and actively provide various services by learning the user's service usage pattern through interaction with the user. Service recommendation for network-based robots.

When there is a service request from the user of the network-based robot, the table is trained based on the context information and the action that is the service requested by the user. When there is a service recommendation request, the service is recommended to the user based on the contents of the table. According to the user's response to the recommended service, the table is updated based on the context information and the action of the recommended service.

Here, the table learns the service usage pattern of the user in the table learning step and the table updating step, and recommends the service to the user based on the contents of the table learned in the table learning step and the table updating step in the service recommendation step.

Network-based robotics, learning, service recommendation

Description

Service recommendation method for network-based robot, and service recommendation apparatus

The present invention relates to a service recommendation method and a service recommendation apparatus for a network-based robot. More particularly, the network-based robot learns a user's service usage pattern through reinforcement learning to actively provide various services appropriately for each user. It relates to a service recommendation method and a service recommendation device.

The present invention is derived from the research conducted as part of the IT new growth engine core technology development project of the Ministry of Information and Communication [Task management number: 2006-S-026-02, Title: Development of URC server framework for active services].

Recently, as the industrial robot market is saturated, research on network-based robots is actively conducted to create a new market. Network-based robots are a new concept of intelligent service robots that combine network and information technology with existing robots. They can be with users anytime and anywhere and provide appropriate services for users.

Robots generally have three functional elements that sense the external environment, judge the situation based on the sensed environment, and act accordingly. The ultimate goal of network-based robots is to distribute and process these three functions that were handled by the robot itself through the network, and furthermore, to fully utilize external sensing and processing functions through the network.

In other words, the network-based robot can utilize the sensing function of an externally equipped sensor, rather than adding hardware to the robot to increase the sensing function. In addition, network-based robots allow users to take advantage of the processing power of network-connected remote servers rather than to increase their own processing power, and to provide various content services through remote content servers. It can provide various functions and services.

As the ubiquitous computing environment arrives, network-based robots need personalized recommendation related technologies that can actively provide services based on various contextual information as well as services provided according to the explicit needs of users. It became. In the network-based robot that combines the network function with the existing robot, it is possible to provide various services to the user, and to recognize the user's situation and recommend the service according to the user's interest rather than the service that simply responds to the user's request. The technology that can be needed.

Popular recommendation techniques currently include Content-based Recommendation and Collaborative Recommendation. Content-based recommendation is based on the field of information retrieval, and is a technique for recommending high similarity by comparing the user's profile with the components of the item to be recommended, for example, 2006-0069143. In addition, the collaboration recommendation is a technique of finding another user having a profile similar to the target user who is the target of the recommendation, and recommending the item to which the other user has given high evaluation to the target user. 0112723. In addition, there are demographic recommendation techniques that make recommendations using demographic information such as gender, age, and occupation of the user, and hybrid recommendation techniques that combine the above content-based recommendation and collaborative recommendation.

However, most of the existing recommendation techniques require detailed information about the recommendation items or prior information about the user. However, since the definition of specific components for the service provided by the network-based robot is not sufficient, and it is difficult to collect information about users for security or privacy protection, the existing information that must hold a lot of information in advance. It is often difficult to apply the recommendation technique, and it is not easy to systematically collect and apply this information.

The present invention is to solve the above problems, even if the user does not have the detailed information about the dictionary information and the provided service, the network-based robot by the reinforcement learning that can be learned through interaction with the user It is an object of the present invention to provide a service recommendation method and a service recommendation device for a network-based robot capable of actively and appropriately providing various services by learning a service usage pattern of the service.

Here, reinforcement learning is a machine learning method in which learning occurs through interaction of reward and action without a preset model for a given environment. Compared to other machine learning methods, reinforcement learning does not predetermine any behaviors by learners in the learning process, and has a trial and error search process through learning and error process through interaction with the external environment. .

Accordingly, the service recommendation method and the service recommendation apparatus according to the present invention can provide a personalized recommendation service suitable for a user by learning a service usage pattern of the user while going through an interaction process with the user without prior information on the user. Make sure In addition, by recording the user's use of the service according to the time and place to reflect in the recommendation it is possible to recommend a more suitable service for the situation information.

In order to achieve the above object, the service recommendation method for a network-based robot according to the present invention corresponds to a situation information about a surrounding situation and a service provided by the network-based robot when a network-based robot requests a service from a user. A table learning step of performing a learning on each table by updating a table value corresponding to the requested service based on an action that is a service requested by a user among each action, and a service recommendation request from the user or the context aware server. The service recommendation step of recommending a service to the user based on the learning contents of each table, and when the service recommended in the service recommendation step is provided through the network-based robot, according to the user's response to the corresponding service. Tables for Recommended Services Table update step to update the table.
At this time, the service usage pattern and service preference of the user are learned based on the table learning contents of the table learning step and the table updating step, and the result is reflected in the service recommendation step.

The table includes, as a factor, a Q-learning information table including status information of the user and / or network-based robot and actions corresponding to the services provided by the network-based robot, wherein the status information includes status information, Preferably, the table further includes an hourly service usage frequency table including days of the week, time zone information, and actions as factors, and the situation information further includes day of week and time zone information. The table further includes a place-specific service usage frequency table including place information and actions as factors, and the situation information further includes place information. The table may further include a user information table for associating a table with a user for a plurality of users of the network-based robot.

In the table learning step, the table value corresponding to the service requested by the compensation value R _S is updated. In the table updating step, when the user's response to the recommended service is positive, the table value is set by the compensation value R _P. It is preferable to update the value of the table and update the value of the table by the compensation value R _N when the user's reaction to the recommended service is negative.

In the service recommendation step, it is preferable to select a service to be recommended by normalizing and summing values of the tables learned in the learning step with respect to each action.

According to the present invention, the following effects can be obtained.

Network-based robots can provide a variety of services through the network, by learning the service usage pattern of the user to recommend the appropriate service to the user from among various services, it is possible to reduce the overhead of service selection.

In particular, even though detailed information on the services provided and prior information on the user are not collected, various services can be actively and appropriately provided by learning the service usage pattern of the user by learning through interaction with the user. have.

In addition, since service usage patterns of various users can be individually learned for each user, it is possible to provide a personalized and customized service even when several people use one network-based robot as in the case of a family.

In addition, by using the contextual information provided from the context awareness server, the user can learn the service usage pattern based on a lot of information, and the possibility of recommending an appropriate service is improved.

If the network aware robot is able to recognize a place or the context aware server can provide information about the place, it is possible to recommend a service suitable for a specific place.

In addition, it is possible to efficiently recommend a service by learning a service usage pattern of a user for each day and time and applying the same to a service recommendation.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments will be described with reference to the accompanying drawings to help understand the present invention. The following examples are provided to more easily understand the present invention, and the present invention is not limited by these examples.

1 is a block diagram of an entire system according to the present invention. The entire system according to the present invention is composed of a network-based robot 10, a context aware server 20 and a content server 40, and requests a service from the network-based robot 10 and services from the network-based robot 10 There is a user 30 who is provided with.

The network-based robot 10 is connected to the situation recognition server 20 and the content server 40 through a network, and although not specifically illustrated, provides a service such as a movie, news, weather guide, etc. to the user 30. To display, to provide services such as music, voice, speaker, microphone for voice recognition, driving motor for moving, communication device for communication with the context recognition server 20 and content server 40, from the user It is equipped with various devices such as an input interface for receiving input.

The situation recognition server 20 is connected to the network-based robot 10 through a network, and although not specifically illustrated, information from various sensors including an illuminance sensor, a voice sensor, a temperature sensor, and other input information may be used. It recognizes the current status information and stores it.

The content server 40 stores contents such as a movie, news, weather guide, music, and voice for the network-based robot 10 to provide to the user 30. As shown, the content server 40 may be installed at a remote location and connected to the network-based robot 10 via a network, or may be integrated with the situation recognition server 20 and provided. In addition, some of the content is stored in the network-based robot 10 may be provided directly to the user (30).

Referring to Figure 1 describes the operational flow of the entire system according to the present invention.

First, when the user 30 makes a service request 30-1 by using the input interface or voice recognition of the network-based robot 10, the network-based robot 10 provides the user with the content server 40. The content corresponding to the requested service is requested 10-1. Types of services requested by the user 30 are, for example, movies, news, weather information, music, voice, education, cooking, and games. The content server 40 provides the requested content 40-1 to the network-based robot 10, and the network-based robot 10 provides a service (content) to the user 30 (10-2). This is a service provision process that occurs when the user 30 explicitly requests a specific service.

On the other hand, the situation recognition server 20 that recognizes the current situation information, if there is a change in the situation information to be recognized, such as when the user wakes up in the morning, or the network-based robot (10) for more than a certain time If it is determined that it is necessary, such as when not providing, the service recommendation request 20-2 may be requested to the user 30 to recommend the service to the network-based robot 10. Alternatively, the user 30 may directly request the network-based robot 10 to recommend a service suitable for the current situation.

Here, since the situation recognition server 20 recognizes and stores the current situation information through information from various sensors and other input information, the situation recognition server 20 simultaneously with the service recommendation request 20-2 or the service recommendation request 20. Regardless of 2), context information is provided to the network-based robot 10 (20-1). The situation information may include, for example, various information such as current day and time information, identifier and location of the user 30, current temperature, illuminance, and the like.

When there is a service recommendation request, the network-based robot 10 recommends a service suitable for a user based on the situation information 20-1 provided from the situation recognition server 20 and the state information stored in itself. (10-3) When the user 30 gives a feedback 30-2 of a positive response to the recommended service, the user 30 requests the content server 40 to receive content corresponding to the recommended service, and then provides the service to the user 30. Provide (10-2).

One feature of the present invention is that when the user requests the service (30-1) as described above, the table performs the learning, and when the user feedback (30-2) for the service recommendation (10-3) table By updating, the service suitable for the current state is recommended to the user according to the collected learning information. Table learning, table updating, and service recommendation will be described in detail below with reference to FIGS.

2 illustrates a data structure for service recommendation according to the present invention. The left column is the content 50 of the data structure, and the right column is the specific data structure 60 corresponding to each content. The data of this data structure is preferably stored in the network-based robot 10, but is stored only in the context aware server 20 or shared by the context aware server 20 and the network-based robot 10 through synchronization. Or distributed storage.

The data structure according to the service recommendation method of the present invention includes a Q-learning information table (Q-TBL [s] [a]), an hourly service use frequency table (F-TBL [d] [t] [a]), and a location. Service usage frequency table P-TBL [p] [a], user information table U-TBL [u], and compensation values R _S , R _P , R _N. The network-based robot 10 recommends the service to the user 30 by referring to the data of the Q-learning information table, the hourly service usage frequency table, and the service usage frequency table for each location. Each data structure is explained in detail as follows.

First, the Q-learning information table Q-TBL [s] [a] is for recording a service used by a user using learning, and has at least two-dimensional factors of state (s) and behavior (a).

Here, the state s refers to the state information of the user 30 and / or the network-based robot 10. For example, the state s becomes a 'weather state' when the user wakes up in the morning and the user returns home. 'Return status', if the network-based robot 10 is providing a news service is a 'news status'. The state information is determined based on the situation information received from the situation recognition server 20 and the state information stored in itself. In addition, the state s includes a service providing state (eg, a news state) and a user state (eg, a weather state) provided by the network-based robot, and these are configured alone or in combination.

In addition, the action (a) refers to a service provided by the network-based robot 10. This can be a service recommendation, for example, movies, news, weather reports, music, voice, education, cooking, games, etc.

The value stored in Q-TBL [s] [a], the Q-learning information table, indicates the preference of behavior (a) in the state (s), and Q-TBL, the preference of behavior (a) in the state (s). A larger value of [s] [a] means that the user 30 frequently used the service (service) or responded positively to the recommendation.

Next, the hourly service usage frequency table F-TBL [d] [t] [a] is a frequency table that stores the number of times the service is used for each day / time zone, and includes at least a day (d) and a time zone (t). And a three-dimensional factor of action (a). Here, the day (d) may be divided into months, Tuesdays, Wednesdays, Thursdays, Fridays, Saturdays, and days, or may be divided into weekdays and weekends. In addition, the time zone t may be divided by every hour or may be divided into morning, afternoon, evening, and night. On the other hand, action (a) is the same as the above Q-learning information table.

Each user 30 of the network-based robot 10 may have a different preference for a service used for each day of the week and a time zone. In particular, a service used for a weekend tends to be significantly different. Therefore, the hourly service usage frequency table is to record the service usage preferences of a specific user by day and time zone by learning by reflecting this. By learning and recording the behavior (service) used by the user and the behavior that responded positively to the recommendation by day and time zone, the user can recommend the appropriate service with the user's preference according to the day / time of the service recommendation. do.

Next, the service usage frequency table P-TBL [p] [a] for each place is a frequency table that stores the number of times the service is used for each place, and at least two-dimensional factors of place (p) and behavior (a). Has Here, the place p may be divided into, for example, a bedroom, a kitchen, a nursery, and a living room. The place p may be classified when the network-based robot 10 can recognize a place, or the situation recognition server 20 recognizes and distinguishes a place of the user 30 and / or the network-based robot 10. May be On the other hand, action (a) is the same as the above Q-learning information table.

Since the preferences of the service used by the user 30 may differ from place to place, the service use frequency table for each place is to record the service use preference of a specific user by learning by reflecting this. By learning and recording the behavior (service) used by the user and the behavior that showed a positive response in the recommendation, it is possible to recommend the appropriate service having a high preference of the user to the user according to the place of the service recommendation.

Next, the user information table U-TBL [u] has information for matching a Q-learning information table corresponding to each user, a service frequency table for each hour, and a service frequency table for each place. When a network-based robot is used by a large number of users, it is possible to recommend a service reflecting the characteristics of each user by managing the table separately for each user.

Next, the compensation value is a value used to update the values of the above-described tables, and the reward or fine value received from the user 30 when the network-based robot 10 takes any action in the current state. Means. The compensation value in this embodiment defines three values of R _S , R _P , and R _N , and each compensation value has an appropriate value according to design.

First, R _S is the reward value when the user explicitly selects the action (service). Secondly, R _P is the reward value when the user responds positively to the service (behavior) recommended by the network-based robot. Third, R _N is the service (behavior recommended by the network-based robot. ), Which is a reward for a user's negative reaction to a. Here, R _S and R _P have positive values because they have to increase their preferences, and R _N have negative values because they have to decrease their preferences. On the other hand, the size of R _S and R _P can be determined according to the design. If the user explicitly selects the case, it is important to define R _S to be greater than R _{P. If} the user is positive about the recommended service, it is important to define R _S to be less than R _P. You can also set both values to the same value.

Detailed procedures of the service recommendation method implemented using the configuration diagram of the entire system of FIG. 1 and the data structure of FIG. 2 according to the present invention will be described with reference to FIGS. 3 and 4.

3 is a schematic flowchart of a service recommendation method for a network-based robot according to the present invention. The service recommendation method is composed of a learning information initialization step (S20), a table learning step (S40), a service recommendation step (S60), and a table update step (S70). In brief, the network-based robot initializes the learning information in step S20, learns the table in step S40 when there is a service request from the user in step S30, and the service in step S50. If there is a recommendation request, the service is recommended in step S60 and the table is updated in step S70 according to the user's response. By repeating this process, it is possible to recommend the appropriate service to the user by the learned table.

First, when the network-based robot is initially driven and the system starts (S10), the learning information of each table is initialized in the learning information initialization step (S20). The learning information initialization step S20 may be performed when initializing the learning information of the table corresponding to the user when a new user of the network-based robot is added.

The network-based robot determines whether there is a service request from the user (S30). If there is an explicit service request from the user (Y in S30), the process proceeds to the table learning step S40, whereby the table value corresponding to the service requested by the user is updated, thereby learning about each table. When the table learning step S40 is completed, the process returns to step S30.

On the other hand, if it is determined in step S30 that there is no service request from the user (N in S30), the process proceeds to step S50 to determine whether there is a service recommendation request from the situation recognition server. Of course, there may be a service recommendation request directly from the user, in which case it is determined whether there is a service recommendation request from the user.

If it is determined in step S50 that there is a service recommendation request (Y in S50), the process proceeds to the service recommendation step S60 and recommends the service to the user based on the learning contents of the table so far. If the user shows a positive or negative response to the recommended service, the process proceeds to a table update step S70 where the table value corresponding to the recommended service is updated to be used at the next service recommendation. When the table update step S70 is completed, the process returns to step S30. On the other hand, if it is determined in step S50 that there is no service recommendation request (N in S50), the process returns to step S30.

In the service recommendation method for a network-based robot according to the present invention, a service suitable for a user can be recommended by learning the service preference of the user and reflecting the learning result in the service recommendation according to the flow as shown in FIG. 3. Detailed description of the service recommendation method schematically illustrated in FIG. 3 will be described with reference to FIG. 4.

4 is a detailed flowchart of a service recommendation method for a network-based robot according to the present invention. The same reference numerals are given to the same steps as the service recommendation method of FIG.

First, in the learning information initialization step (S20), the learning information of the Q-learning information table (Q-TBL), the hourly service use frequency table (F-TBL), and the service use frequency table for each place (P-TBL) are '0'. Initialize with The compensation values R _S , R _P and R _N are also initialized to specific values, and the current state s is initialized based on the situation information provided from the situation recognition server 20. For example, initialize the current state s to the weather state.

The network-based robot determines whether there is a service request from the user (S30). If there is an explicit service request from the user (Y in S30), the process proceeds to table learning step S40. Table learning step (S40) is a step of learning the Q-TBL, F-TBL, P-TBL according to the service request content of the user.

In the table learning step S40, first, the action (a) which is the service selected by the user is identified (step S41). Since the user explicitly requests the service, R _S is selected as a compensation value for updating the table (step S42). Then, in step S43, table learning is performed using the action (a) and the reward value (R _S ), which are the services requested by the user.

In step S43, a table learning will be described in detail using an example where a user requests a 'weather guide' service while listening to news in a living room on a weekend morning. First, Q-TBL learning is performed using the current state (s) and the action (a) requested by the user. Here, the current state (s) is 'news', the action (a) requested by the user is 'weather guide'. Therefore, in Q-TBL [s] [a], the value corresponding to Q-TBL [news] [weather guidance] is updated by the compensation value R _S. The equation used for updating is, for example, Q-TBL [s] [a] = Q-TBL [s] [a] + λ _q R _S , where λ _q is an adjustment for adjusting the reflectance ratio of the compensation value R _S. As a value, it is determined according to the design and may have a value of 1. Q-TBL [News] [Weather Guide] is the result of learning by compensation value R _S , and the result of the user's listening to the news, the result of increasing the preference for requesting weather information service.

Then, F-TBL learning is performed using the current day of the week (d), time zone (t), and action (a). Here, the current day of the week (d) is 'weekend', the current time zone is 'am', and the action (a) requested by the user is 'weather guide'. Therefore, in F-TBL [d] [t] [a], the value corresponding to F-TBL [Weekend] [AM] [Weather Guide] is updated by the compensation value R _S. The equation used for updating is, for example, F-TBL [d] [t] [a] = F-TBL [d] [t] [a] + λ _f R _S , where λ _f is a reflection of the compensation value R _S As an adjustment value for adjusting the ratio, it is determined according to the design and may be equal to the λ _q value in Q-TBL. The value of F-TBL [Weekend] [Morning] [Weather Guide] increases according to the learning by the reward value R _S , and the user's preference for requesting weather information service on the weekend morning increases.

Then, P-TBL learning is performed using the current place (p) and behavior (a). Here, the current place (p) is 'living room', the action (a) requested by the user is 'weather guide'. Therefore, the value corresponding to P-TBL [living room] [weather guide] in P-TBL [p] [a] is updated by the compensation value R _S. The equation used for updating is, for example, P-TBL [p] [a] = P-TBL [p] [a] + λ _p R _S , where λ _p is an adjustment for adjusting the reflectance ratio of the compensation value R _S. As a value, it is determined according to design and may be equal to the λ _q value in Q-TBL. P-TBL [living room] [weather guide] is the result of learning by the compensation value R _S , the value is increased, the result of the user's preference for requesting weather information service in the living room is increased.

As described above, by updating a table value corresponding to the service requested by the user, learning is performed for each table, and when the table learning step S40 is completed, the process returns to step S30.

On the other hand, if it is determined in step S30 that there is no service request from the user (N in S30), the process proceeds to step S50 to determine whether there is a service recommendation request from the situation recognition server.

If it is determined in step S50 that there is a service recommendation request (Y in S50), the process proceeds to the service recommendation step S60 and the user is based on the learning contents of Q-TBL, F-TBL, and P-TBL so far. Recommend service to users by grasping preferences. The service recommendation step S60 has two stages: a preference calculation step S61 and an action recommendation step S62.

In the preference calculation step (S61), the user's preference is grasped based on the learning contents. To illustrate an embodiment of preference calculation, assume that a user is listening to music in the bedroom on a weekday evening.

In this embodiment, the case where Q-TBL, F-TBL, and P-TBL is used is taken as an example. However, depending on the implementation, there may be only Q-TBL, or only Q-TBL and F-TBL. In some cases. At this time, the deformation can be sufficiently applied with reference to the following embodiments.

First, normalization is performed to control specific gravity of Q-TBL, F-TBL, and P-TBL. Normalization described below is normalization such that Q-TBL, F-TBL, and P-TBL have a value between 0 and 1. W _q in formula (1) is a normalized Q-TBL, W _f in formula (2) is a normalized F-TBL, and W _p in formula (3) is a normalized P-TBL. In this example, state s is music, day d is weekday, time zone t is evening, and place p is bedroom.

식 (1)

Formula (1)

식 (2)

Formula (2)

식 (3)

Equation (3)

For example, W _q is a result obtained by normalizing the preference of each action (a), which is a service to be recommended for the current state (s), using Q-TBL. W _q , W _f , and W _p will be obtained by the number of types of behavior (a) that have been learned about the current state (s) for that user. The normalized W _q , W _f , and W _p values obtained by equations (1) to (3) are used to calculate a preference for each action (a), which is the service to be recommended.

를 계산하기 위한 것이다. 여기서

,

,

는 해당 사용자(u)에 관한 테이블의 정규화값이다. 그리고, α, β, γ는 각 테이블의 정규화값의 반영 비율을 규정하기 위한 상수로서, 1/3의 동일한 값을 가질 수도 있고 각 테이블의 중요도에 따라 다른 값을 가질 수도 있다. Equation (4) below is the user's (u) 's preference for each action (a)

To calculate. here

,

,

Is the normalized value of the table for the user u. In addition, α, β, and γ are constants for defining a reflection ratio of normalized values of respective tables, and may have the same value of 1/3 or may have different values depending on the importance of each table.

식 (4)

Equation (4)

Referring to the preferences for each behavior (a) calculated by Equation (4), the current situation, ie the user listening to music in the bedroom on a weekday evening, is the user's most preferred behavior (a). By selecting, the target of the service recommendation is determined. In step S62, the network-based robot recommends the action u, which is the determined service, to the user u. In this case, it is assumed that the network-based robot recommends the 'movie' service to the user in step S62.

Next, in the table updating step, a compensation value is determined and the table is updated according to the response shown by the user to the action (a) which is the service recommended in step S62. Specifically, if the user response in step S71 is affirmative, proceed to step S72 to select R _P as the compensation value R, and in step S71 if the user response is negative or the user within a certain time. If there is no response, the process proceeds to step S73 and R _N is selected as the compensation value R. Where R _P is a positive value and R _N is a negative value. The Q-TBL, F-TBL, and P-TBL are updated according to the user's response to the recommended service in step S74.

First, the Q-TBL update is performed using the current state (s) and the recommended service (a). Here, the current state (s) is 'music', and the recommended service (a) is 'movie'. Therefore, the value corresponding to Q-TBL [music] [movie] in Q-TBL [s] [a] is updated by the compensation value R (R _P or R _N ). The equation used for updating is, for example, Q-TBL [s] [a] = Q-TBL [s] [a] + λ _q R, where λ _q is an adjustment value for adjusting the reflectance ratio of the compensation value R. It may be determined according to the design and may have a value of 1. In addition, λ _q in the step (S74) will be equal to λ _q in the step (S43), and may be different. Q-TBL [music] [movie] is a result of increasing or decreasing the value according to the learning by the compensation value R, the result of the user increases or decreases the preference to watch a movie while listening to music.

Similarly, table updates are performed for F-TBL [weekday] [evening] [movie] and P-TBL [bedroom] [movie] similarly to Q-TBL. Details thereof will be omitted since they can be easily understood by referring to the description of step S43 and table update of Q-TBL.

As such, in step S70, the table value is updated according to the user's response to the recommended service so that it can be utilized in the next service recommendation. When the table update step S70 is completed, the process returns to step S30. On the other hand, if it is determined in step S50 that there is no service recommendation request (N in S50), the process returns to step S30.

The service recommendation method and service recommendation apparatus for a network-based robot according to the present invention can recommend a personalized service for each user through reinforcement learning, and thus, there will be great applicability in developing a user-friendly and active robot.

1 is a block diagram of an entire system according to the present invention,

2 is a data structure for recommending a service according to the present invention;

3 is a schematic flowchart of a service recommendation method for a network-based robot according to the present invention;

4 is a detailed flowchart of a service recommendation method for a network-based robot according to the present invention.

Claims (14)

As a service recommendation method for network-based robots, When a network-based robot requests a service from a user, the network-based robot corresponds to the service requested based on situation information on a surrounding situation and an action which is a service requested by the user among the actions corresponding to the service provided by the network-based robot. A table learning step of performing table learning by updating table values; A service recommendation step of recommending a service to the user based on the learning contents of each table when a service recommendation request is made from the user or the context aware server; And And a table updating step of updating a table corresponding to the recommended service according to the user's reaction to the service when providing the service recommended in the service recommendation step through the network-based robot. And learning service usage patterns and service preferences of the user based on table learning contents of the table learning step and the table updating step, and reflecting the learning result to the service recommendation step. delete delete delete delete delete delete As a service recommending device connected to the situation awareness server and the network, When there is a service request from the user, the service requested by the user based on the context information on the surrounding situation provided by the context aware server and the action corresponding to the service provided by the network-based robot is a service requested by the user. Table learning means for performing learning on each table by updating a corresponding table value; Service recommendation means for recommending a service to the user based on the learning contents of each table when a service recommendation request is input from the user or the context aware server; And Table updating means for updating a table corresponding to the recommended service according to a user's response to a service recommended by the service recommending means, The table learning means and the table updating means learn a service usage pattern and a service preference of the user based on the contents of the table, And the service recommendation means recommends a service to the user using the learned service usage pattern and service preference of the user. The method according to claim 8, The table includes, as a factor, a Q-learning information table including state information of the user and the network-based robot and actions corresponding to the services provided by the network-based robot. And the contextual information includes the state information. The method according to claim 9, The table further includes an hourly service usage frequency table including the day of the week, time zone information, and the action as a factor, The contextual information further includes the day of the week and time zone information. The method according to claim 9, The table further includes a place-specific service usage frequency table including place information and the action as a factor, And the contextual information further includes the place information. The method according to claim 8, The table further includes a user information table for associating a table with a user for a plurality of users. The method according to claim 8, The table learning means updates a table value corresponding to the service requested by the compensation value R _S , The table updating means updates the table value by the compensation value R _P when the user's response to the recommended service is positive, and by the compensation value R _N when the user's response to the recommended service is negative. And a service recommendation device for updating a table value. The method according to claim 8, And the service recommendation means selects a service to be recommended by normalizing and summing values of the tables learned by the table learning means with respect to the respective actions.

Images