WO2020153614A1

WO2020153614A1 - Method and platform for providing ai entities evolving via reinforced learning

Info

Publication number: WO2020153614A1
Application number: PCT/KR2019/018271
Authority: WO
Inventors: 강훈석; 김다일
Original assignee: ㈜티비스톰
Priority date: 2019-01-22
Filing date: 2019-12-23
Publication date: 2020-07-30
Also published as: KR20200094833A; KR102309682B1

Abstract

The present invention relates to a method and a platform for providing AI entities that evolve via reinforced learning. A user creates an AI entity and continuously trains same according to intentions of the user, and allows the AI entity to evolve into an independent entity desired by the user by means of reinforced learning, thereby allowing the AI entity to fill the role of an assistant desired by the user or to carry out SNS activities, and, additionally, in the home, school or office, allowing the AI entity to carry out the role a type of an agent that maintains a pleasant environment by, on behalf of the user, activating various electronic appliances at desired times using desired methods and configuring various IoT devices.

Description

Method and platform to provide evolving AI entities through reinforcement learning

The present invention relates to a method and platform for providing an AI entity that evolves through reinforcement learning, and more specifically, a user creates an AI entity to continuously learn as he/she wants, and the user himself/herself through reinforcement learning. By evolving into an independent entity, the user can act as a secretary, or enable independent SNS activities, and also use various household appliances instead of the user at home, school, or office at the time and method desired by the user. It relates to a method and a platform for providing evolving AI entities through reinforcement learning to operate and set up various IoT devices to act as a kind of agent to maintain a pleasant environment.

Recently, with the development of industrial technology and information communication technology, artificial intelligence technology including deep learning and voice recognition technology for recognizing users' voices are rapidly developing, and at the same time, multiple sensors and communication functions are embedded. As a result, IoT devices for providing various information and convenience to users are spreading.

These artificial intelligence technologies are becoming more and more advanced, and when combined with voice recognition technology, it recognizes the user's voice and supports the ability to remotely control the IoT devices desired by the user, while at the same time providing various information such as news and weather on the web. It is developing to a level that can be searched and provided to users.

Accordingly, an artificial intelligence system capable of recognizing a user's voice and searching and providing information desired by the user on the web according to the recognized voice or remotely controlling the plurality of IoT devices in real time. The public interest in AI systems) is increasing.

The artificial intelligence system continuously evolves into an independent entity by continuously learning the user's characteristics, so that the user can automatically control a plurality of IoT devices according to tastes or preferences, or provide information that the user actually needs. As described above, if a suitable service can be provided according to a user's characteristics, high satisfaction and convenience may be provided to the user.

However, the conventional artificial intelligence system is implemented not to provide a suitable service to the user according to the characteristics of the user, but simply to control the retrieved information or device according to predefined logic.

That is, in the conventional artificial intelligence system, it is not learned to provide a specific service according to the characteristics of the user, but it is concentrated to recognize the user's voice through artificial intelligence technology including voice recognition technology, and the user's voice In the case of recognizing, it performs a preset logic according to the recognized user's voice to provide a service that the user wants to receive.

For example, in a conventional artificial intelligence system, when the user wants to be recommended for music, when the artificial intelligence system inputs a command for the music recommendation by voice, the artificial intelligence system recognizes the voice for the music recommendation. Later, it is only to search for a recently released sound source or a sound source used by a large number of users and output the sound source.

That is, the conventional AI system is no longer evolved according to the user's tendency or characteristics in the initial state when it is provided in the home, and only provides a function for providing a service to the user using only simple logic or statistical values set in advance. I just apply.

This does not provide a service that is optimal for the needs of the user according to the individual tendencies or characteristics of the user, and also implies the problem of providing unnecessary services to the user.

Accordingly, in the present invention, an AI entity such as a character or an avatar corresponding to a specific user is generated, and the created AI entity is continuously reinforced learning according to the characteristics of the user, so that the individualized individual is independent of the specific user. By evolving into an entity, it can act as a secretary for the user, enable independent SNS activities, or operate various IoT devices at the time and method desired by the user, and at the same time music, news, schedule, weather, products In order to provide a variety of recommended services, such as people, I would like to suggest a way to provide.

Next, the prior art existing in the technical field of the present invention will be briefly described, and then the technical matters to be achieved differently from the prior art will be described.

First, Korean Patent Publication No. 2019-0001059 (2019.01.04.) relates to an artificial intelligence platform providing device and a content service method using the same, receiving input data in a predetermined format from a user terminal, and included in the input data After extracting the request information related to the user's request, based on the request information, select one of a plurality of content servers that provide content, and provide an AI platform providing device corresponding to the request information to the user and the same It relates to a content service method used.

That is, the prior art is to receive voice data from a user, recognize request information from the received voice data through speech recognition and natural language processing, and provide content corresponding to the recognized request information.

In other words, the prior art simply recognizes a user's voice through an AI platform and provides various contents according to the recognized user's voice.

On the other hand, according to the present invention, the user creates an AI object including a desired character or avatar, and continuously learns the generated AI object according to a user's characteristics, so that the user can evolve into an independent object. According to the characteristics of providing a variety of services suitable and efficient for the user, the prior art does not describe or suggest such technical features of the present invention.

In addition, Korean Registered Patent No. 1172002 (2012.08.01.) relates to an artificial intelligence digital device control system using a smartphone and a sensor, and a preset value set in advance by a comparison unit provided with the sensing value measured by the sensor. The artificial intelligence digital device control system using a smartphone and a sensor to control the operation of the air-conditioning device fixedly installed at a specific location after determining whether or not the air-conditioning device is driven by comparing the sensing values with will be.

In the prior art, environmental conditions including temperature and humidity are set in advance through a smartphone, and sensing values for environmental conditions including temperature and humidity measured through a sensor are compared with the preset environmental conditions, and the air conditioning is performed. The operation of the device is determined and the driving signal for the air conditioning device is transmitted through the IR transmitter according to the determination result, so that the air conditioning device can be automatically controlled.

That is, the prior art simply compares a preset value and a sensing value, and determines whether to drive the air conditioning system according to the comparison result, and continuously reinforces and learns the characteristics of the user input from the user. It is not intended to control a specific IoT device according to the characteristics of the user, or to provide various recommended services such as music.

On the other hand, according to the present invention, an AI object including a character, an avatar, etc. is generated, and user characteristic information is continuously input from a user, and a specific IoT device is controlled based on the received characteristic information or a service for each field is provided. Continuously reinforce learning by learning at least one or more learning models to provide the recommendation service suitable for the characteristics of the user who uses the AI object by evolving the generated AI object into an independent object, or by providing a specific IoT device. To be able to control it. Therefore, the prior art is clearly different from the technical features to be proposed in the present invention.

The present invention was created to solve the above problems, and the user creates an AI object including his avatar and character, and continuously strengthens the AI object through continuous interaction between the created AI and the user. By providing learning and evolving the AI entity as an independent entity, providing a method and platform for providing an AI entity that evolves through reinforcement learning that enables the AI entity to provide a service that meets the characteristics of the user. For that purpose.

In addition, when creating the AI entity, music, products, TV channels, news, etc. are recommended, or learning models specialized for at least one field including IoT device control, schedule management, and the like are respectively generated, and the user generates AI. Another object is to provide a method and platform for providing an evolving AI entity through reinforcement learning to advance the AI entity by enabling reinforcement learning for each learning model through interaction with the entity. Is done.

In addition, the present invention, by performing the interaction between the user and the AI entity, when receiving a recommendation service in a specific field that is the result of the interaction, by applying the interaction result to the learning model for the specific field, to the AI entity Another object is to provide a method and a platform for providing an evolving AI entity through reinforcement learning that enables the AI entity to gradually evolve by automatically performing reinforcement learning for Korea.

In addition, the present invention, according to the user's selection or periodically from the user, the preferred TV channel, air conditioning temperature by location, hourly lighting control information, preferred news field, people, music, artists, products, etc. By continuously performing reinforcement learning on the AI entity by receiving the characteristic information of the, it provides the evolving AI entity through reinforcement learning that provides a recommendation service including control, news, music, etc. of a specific IoT device suitable for the user's preference. Another object is to provide a method and platform.

In addition, the present invention, by learning the user's schedule including the user's schedule, alarm time, anniversary, etc., to automatically inform the user of the schedule through the AI entity, so that it can act as a secretary for the user Another object is to provide a method and platform for providing AI entities that evolve through reinforcement learning.

In addition, according to the present invention, when a user inputs a request for a recommendation service through voice or text into an AI entity through the interaction, the AI entity recognizes the request and schedules according to the learned result according to the recognized request. Another object is to provide a method and platform for providing an evolving AI entity through reinforcement learning to provide a propulsion service for IoT device 400 control, music, news, and the like.

In addition, the present invention is implemented so that the created AI entity can independently act on the SNS, and provides an AI entity that evolves through reinforcement learning to evolve into an entity independent of the user through interaction with other AI entities. Another object is to provide a method and platform.

In addition, the AI object is gradually advanced and evolved into an independent object through a learning method including self-learning, participatory learning, supervised learning, and self-learning, so that a user can search for and provide information or a specific IoT device. Provides a method and platform for providing evolving AI entities through reinforcement learning that enables them to automatically control, report schedules, and provide various services including recommending music, products, videos, etc. To do it for another purpose.

A method of providing an AI entity that evolves through reinforcement learning according to an embodiment of the present invention includes: an AI entity generation step in which a user creates an AI entity, and a learning model for the created AI entity through machine learning When the user interacts with the AI entity through the learning model generation step and the generated learning model, by applying the result of the interaction to the learning model, and performing reinforcement learning for the learning model, the interaction behavior of the user Accordingly, it characterized in that it comprises an AI entity evolution step to cause the generated AI entity to evolve into an independent entity.

In addition, the method further comprises, through the AI entity evolution step, while the AI entity interacts with another AI entity, further evolving into an individual entity with the user.

In addition, the AI entity is generated by creating an account on a web server or a cloud platform on a specific device or internet that is configured with independent hardware that performs the method, and functions as a secretary desired by the user, or enables SNS activities, In the home, school or office, the user operates at least one IoT device on behalf of the user at a desired time and method, or sets at least one IoT device to maintain the desired home, school, or office environment. Or a combination of these.

In addition, in the process of learning the learning model and generating a learning model, the method provides a user with an interface through which learning data can be input, and presents a method for learning to the user, thereby allowing the user to format learning data in a predetermined format. It characterized in that it further comprises a step of inputting the learning data to be input.

In addition, the learning model includes a learning model specialized for at least one field including CNN, RNN, or ANN, and the AI entity evolves by combining the at least one specialized learning model with each other, and the mutual coupling is related. It is characterized in that it is performed by quantifying and scaling the learning data according to the relevant degree in the field.

In addition, a platform for providing an AI entity that evolves through reinforcement learning according to an embodiment of the present invention, an AI entity generation unit that supports a user to create an AI entity, and learning about the created AI entity through machine learning When the user interacts with the generated AI entity through the learning model generation unit generating the model and the generated learning model, by applying the result of the interaction to the learning model, reinforcement learning for the learning model is performed. And, it characterized in that it comprises a learning model evolution unit that allows the generated AI entity to evolve as an independent entity according to the user's interaction behavior.

In addition, the learning model evolution unit, the AI entity through the process of evolving into the independent entity, while further interacting with other AI entities, characterized in that it further comprises to evolve into an independent entity with the user.

Further, when the learning model is generated by learning the learning model, the platform provides the user with an interface through which learning data can be input, and provides a user with a method to learn, thereby allowing the user to learn data in a predetermined format. It characterized in that it further comprises a learning data input unit for input.

As described above, the AI object reinforcement learning platform of the present invention allows the user to create an AI object including his alter ego or character or avatar, and the user can continuously reinforce the learned AI object according to his preference. By evolving the created AI entity into an independent entity, the optimal service suitable for the user's tendency or needs is provided, and at the same time, it can serve as the personal assistant of the user, thereby providing convenience to the user. It has the effect.

1 is a conceptual diagram schematically illustrating a method and platform for providing an AI entity that evolves through reinforcement learning according to an embodiment of the present invention.

2 is a view illustrating a method for performing reinforcement learning on an AI entity according to an embodiment of the present invention.

3 is a diagram illustrating a method of providing a recommendation service according to a user's request command through an AI entity according to an embodiment of the present invention.

4 is a block diagram showing the configuration of an AI entity providing platform according to an embodiment of the present invention.

5 is a flowchart illustrating a procedure for providing an AI entity that evolves through reinforcement learning according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of a method and platform for providing an AI entity that evolves through reinforcement learning of the present invention will be described in detail with reference to the accompanying drawings. The same reference numerals in each drawing denote the same members. Also, specific structural or functional descriptions of the embodiments of the present invention are exemplified for the purpose of describing the embodiments according to the present invention, and unless defined otherwise, all terms used herein, including technical or scientific terms. These have the same meaning as those generally understood by those of ordinary skill in the art. Terms such as those defined in a commonly used dictionary should be interpreted as having meanings consistent with meanings in the context of related technologies, and should not be interpreted as ideal or excessively formal meanings unless explicitly defined herein. It is desirable not to. In the present invention, an AI object is created by a user in a cyber space such as the Internet or a device in a real space, for example, a character or avatar (divided) as if creating an SNS account (connection window), and the corresponding character or avatar It refers to the sophistication and evolving of (divided) through the reinforcement learning according to artificial intelligence algorithms, by users or by activities on SNS.

As illustrated in FIG. 1, an AI entity reinforcement learning platform (hereinafter referred to as an AI entity providing platform) 100 according to an embodiment of the present invention includes an AI entity (such as a character or avatar for each user). 200) and continuously performing reinforcement learning on the created AI object 200, thereby fostering the AI object 200 to evolve as a single independent object, so that the AI object 200 is applicable. It performs a function to search for various information according to the user's preference, to automatically control a specific IoT device, or to provide various services, including reporting on the user's schedule and recommendations such as music, news, and videos. .

In addition, when the AI entity 200 is generated by the user, the AI entity providing platform 100 receives the user's characteristic information from the user and uses the entered characteristic information to provide a specific service to the user. Create a plurality of learning models for each field to provide.

That is, the characteristic information of the user becomes learning data for generating the learning model for each field.

In this case, the user installs the AI application provided from the AI object providing platform 100 to the user terminal 300 of the corresponding user, and executes the installed AI application to provide the AI object providing platform. After accessing (100), the AI entity 200 may be generated.

The AI entity providing platform 100 is for generating the AI entity 200 and for evolving the AI entity 200 through continuous reinforcement learning, thereby generating the AI entity 200 and allowing the user to It refers to a specific device that is provided in a public place such as a home, office, or school, and is composed of independent hardware. Meanwhile, the AI entity providing platform 100 may be implemented in the form of a cloud server or a web server.

The AI entity providing platform 100 interacts with the user through the created AI entity 200 to continuously evolve the AI entity 200, and the AI entity 200 To provide various services to users.

In addition, the created AI entity 200 is applied to the user terminal 300 of the corresponding user and can be implemented to perform the interaction anytime, anywhere, and a specific cloud platform or web server for providing an SNS service It can be applied to, and can be implemented to independently perform SNS activities according to user preferences.

Meanwhile, the AI entity 200 applied to the cloud platform or web server for providing the SNS service is generated by creating an SNS account for the cloud platform or web server.

In addition, the learning model for each field may include a TV channel recommendation learning model, a music recommendation learning model, an IoT device 400 control learning model, a news recommendation learning model, and a schedule recommendation learning model, and the AI entity. The 200 may provide the service to the user through the learning model for each field.

The learning model is not limited thereto, and may be subdivided into a plurality of learning models by the designer of the AI entity providing platform 100.

In addition, when the user wants to create the AI entity 200 through the AI entity providing platform 100, according to the AI entity creation procedure of the AI entity providing platform 100, the authentication information of the user (eg fingerprint, By providing voice, gesture) and personal information (eg, wallet information including user ID, name and account information, etc.), requesting the creation of the AI entity 200, and providing the AI entity providing platform 100 ) Generates an AI entity 200 for the user by creating an account for the user when there is an application for generation including the authentication information and personal information from the user.

The AI entity 200 generated as described above is applied to the AI entity providing platform 100 and can also be applied to the user terminal 300 of the corresponding user according to the user's selection, and the cloud providing the SNS service. It can be applied to platforms or web servers.

Also, it is natural that the AI entity providing platform 100 may generate the AI entity 200 for each user by receiving the authentication information and personal information from at least one or more users.

For example, when the AI object providing platform 100 is provided in the user's home, the authentication information and personal information are input for each member of the user's family, and the AI object 200 for each member is generated. Can be implemented.

In addition, the authentication information, to activate the AI entity 200, to use the AI entity 200, fingerprint recognition means provided in the user terminal 300 connected to the AI entity providing platform 100 , It may be input through a microphone or a camera, and the personal information may be input through an input means such as a keypad, a touch fan, etc. of the user terminal 300.

However, the fingerprint recognition means, microphone or camera and the input means may be provided in the AI entity providing platform 100, where the user is a fingerprint recognition means, microphone or The user's authentication information and personal information can be directly input through the camera and the input means.

In addition, in the process of generating the AI entity 200, the AI entity providing platform 100 provides customizing information for the AI entity 200, thereby allowing the AI entity 200 to be accessed by the user. It can be created by modeling according to the propensity.

The customizing information includes social elements including language and nationality to be applied to the AI entity 200, biological elements including gender and age, and appearance elements including face, hair and clothes, and emotions (eg, joy) , Facial expressions and gestures for sadness, etc.) and the name of the corresponding AI entity 200.

That is, the user sets the social element, biological element, appearance element, emotion element, and name for the AI object 200 based on the customization information provided by the AI object reinforcement learning platform 100, so that the user By modeling the AI entity 200 according to taste, an AI entity 200 that is more user friendly can be generated.

Meanwhile, the AI entity 200 is generated by including a unique identification code and a creation date for the corresponding AI entity 200, and access to the information of other members and the IoT device 400 is granted according to the user's selection. Is created.

In addition, the created AI entity 200 may be linked to a plurality of IoT devices 400 provided in the home, such as a user's home, office, or school, according to the user's setting. At this time, the user may set the access authority of the AI entity 200 to the IoT device 400.

In addition, the plurality of IoT devices 400 refers to various devices including lighting, TVs, and air conditioning units located in the home. The plurality of IoT device 400 devices may be configured as a home network system.

In addition, the user continuously performs reinforcement learning on the AI entity 200 using his characteristic information in order to receive at least one recommendation service suitable for his/her needs, thereby allowing the AI entity 200 to perform himself/herself. It can be upgraded to suit your propensity.

In addition, when the AI entity providing platform 100 generates the AI entity 200, it learns user characteristic information (that is, learning data) provided from a user to generate a learning model for each field for providing the service. .

At this time, the AI object providing platform 100 provides the user characteristic information input data format for inputting the user characteristic information to the user terminal 300 of the corresponding user through the user interface, and the user is provided with the provided user characteristic By inputting user-specific information of the corresponding user based on the information input data format, it is possible to generate a field-specific learning model for the AI entity 200.

That is, the AI entity providing platform 100 allows the user characteristic information to be input in a predetermined format, thereby providing a learning method for the learning model for each field according to a user's preference, thereby learning the learning model for each field. Is to create

On the other hand, the user characteristic information is the TV channel (eg, sports, culture, entertainment, drama, documentary, etc.) preferred by the user for each time zone, time, place (eg, living room, master room, calligraphy, kitchen, bathroom, etc.) and weather Lighting information including user's preferred lighting brightness, lighting color, lighting on/off time, user's preferred cooling/heating temperature by time, place and weather, and news areas preferred by the user (eg economy, sports, entertainment, etc.) ) And the music genre (e.g. popular music, classical music, etc.) and artists preferred by the user depending on the person, time, weather, or mood. ), schedule information including weather, bedtime, planned work (e.g. exercise, rest, study, etc.), anniversary (e.g. wedding anniversary, ritual, birthday, etc.), user's preferred product, and personality characteristics do.

In addition, the AI entity providing platform 100, when the user characteristic information is input based on the user characteristic information input data format, the user characteristic information for each field so that the user characteristic information can be applied to the learning model for each field Classify.

Thereafter, the AI entity providing platform 100 generates the learning model for each field by learning the classified user characteristic information for each field through the reinforcement learning network for each field.

That is, the user characteristic information is used as learning data of a field-specific learning model for the corresponding AI entity 200 in order to provide various services according to a user's preference.

Through this, the AI entity 200 automatically searches for and provides specific information (for example, news, people, weather) according to a preset time and a user's preference using the learning model for each field, or The IoT device 400 may be automatically controlled, report on a specific schedule, expenditure history, or recommend music.

The AI entity 200 evolves into an independent entity through continuous reinforcement learning on the generated learning model, and the reinforcement learning is performed through interaction with a user.

The interaction refers to a process of recognizing a user's request command for the AI entity 200 and providing at least one service for each field for the recognized request command. When a service is provided, the reinforcement learning is performed based on the result.

That is, when the AI entity providing platform 100 performs an interaction between the user and the AI entity 200, based on the result of the interaction, the weight of the learning model for each field is adjusted to adjust the weight for the corresponding learning model. By continuously performing reinforcement learning, the corresponding AI entity 200 can be gradually evolved.

Through this, the AI entity providing platform 100 advances the AI entity 200 to suit the user's propensity and provides a service suitable for the user's needs.

Meanwhile, in addition to using the interaction result, the reinforcement learning is also performed through self-learning, participatory learning, supervised learning, and autonomous learning of the AI entity 200, through which the AI entity 200 is more highly advanced and independent. It evolves into an individual.

In addition, the self-learning is automatically performed based on the interaction with the user, where the user's location and the frequency of selection of the provided service (for example, when at least one music is recommended, select and use specific music) Frequency of use), keywords frequently used by users (e.g. keywords for specific music genres (e.g. popular music), etc.) to provide users with services that users prefer to taste, music, etc. for their current location. It means learning to help.

In addition, the self-learning automatically collects other information related to the user's characteristic information and the user's characteristic information from the web, and applies the collected other information to the user's characteristic information to provide various information for the user. It can also be performed to provide services.

For example, when a specific weather, the AI entity 200 controls a specific IoT device 400 based on the user's preferred air-conditioning temperature or lighting information and music-specific information for each weather, or when recommending music, The AI entity 200 collects weather information (ie, temperature, humidity, etc.) at the current time from the Korea Meteorological Administration or the web, and matches the collected weather information with the user's characteristic information to apply to the learning model. , To control a plurality of IoT devices 400 according to the characteristics of the user, or to recommend a specific music.

In addition, the participatory learning means that the AI entity 200 analyzes and learns by analyzing the user's dietary preferences, music, and friend preferences, such as a predefined psychological test or taste game, and the AI The object 200 may be evolved to recommend food, products, music, friends, etc. suitable for the user through the participation learning.

In addition, the supervised learning is performed to learn user characteristic information received from the user and provide the service to the corresponding user according to the learned result.

For example, the AI entity 200 that has learned the user characteristic information for a TV channel (eg, sports, culture, entertainment, drama, documentary, etc.) preferred by the user for each time zone, at least one preferred by the user at the current time The above TV channels can be recommended.

In addition, the self-learning is performed by the AI entity 200 to autonomously learn and provide information to the user, based on the social networking information about the friend on the SNS provided by the user. When an article or anniversary is updated, it may be evolved to provide information to the user or to recommend news information, which is a recent issue.

In addition, the AI entity 200 may be implemented to learn the social network information on the user's SNS, analyze the family, work, school, location, and tastes of the social network that has a high frequency of interaction with the user and provide it to the user. In addition, education for the corresponding network can be implemented to analyze the capabilities including intimacy and expertise and provide them to the user.

That is, the AI entity 200 is evolved into a more advanced and independent entity through interaction with the user, self-learning, participatory learning, supervised learning, and autonomous learning.

In addition, the AI entity 200 may act as an independent entity on the SNS based on the characteristics of the user according to the result of the reinforcement learning, perform a recommendation activity for the SNS activity of another user, or another AI entity 200 ) To evolve into an entity independent of the user.

That is, the AI entity 200 forms a network of only the AI entity 200 through an interaction with another user or another AI entity 200 based on the user's personal preferences or characteristics based on the user's social network information, or forms a community It can be evolved to do it.

In addition, the AI entity 200 automatically provides the service according to a preset time through the reinforcement learning, or when the user inputs a request command through interaction with a user, recognizes the request command and recognizes the service It is implemented to provide private service according to one request order.

That is, the AI entity 200 inputs a preset condition (eg, current time information) into the learning model for each field using the learning model for each field according to a preset time, and the learning model for each field The service may be automatically provided based on the output data output from.

For example, when it is set to recommend news at a wake-up time (eg, 6 AM), the AI entity 200 inputs current time information into a learning model for recommending news. At this time, the learning model for recommending news outputs at least one news field and person preferred by the user according to the result of the reinforcement learning. Thereafter, the AI entity 200 searches for news on the news field and person on the web and provides the search result to the user, thereby providing a recommendation service for the news to the user.

In addition, when the AI entity 200 wants to provide a recommendation service through the interaction, the keyword is extracted from the user request command, and compared with the extracted keyword and at least one representative keyword for each preset recommended field. Depending on the result, it recognizes which recommendation service the user request command wants to receive.

For example, if a representative keyword for recommending music is set to "music", "song", and "sound source", and the user inputs a request command "recommended music", "music" from the request command When the keyword "is extracted", the comparison process recognizes that the user's request command is to receive a music recommendation.

At this time, the learning model for music recommendation outputs at least one music genre and artist preferred by the user according to the result of the reinforcement learning. Subsequently, the AI entity 200 provides a recommendation service for the music to the user by retrieving and recommending the music genre and the music source for the artist for the music genre on the web, or recommending an existing stored sound source. will be.

At this time, the user request command may be input by voice or text, and the AI entity 200 recognizes the input voice and text to extract the keyword.

Meanwhile, extracting a keyword by recognizing the voice or text may be performed through a pre-built language model and a morpheme analysis, such as a hidden markov model (HMM) model. However, in the present invention, the method for extracting keywords by recognizing the voice or text is not limited.

In addition, when the service is provided through the interaction, a predetermined gesture is performed through a camera, fingerprint sensor, microphone, or keypad provided in the AI device to which the AI entity 200 is applied, fingerprint authentication is performed, or the The name of the set AI entity 200 is called, and after activating the AI entity 200, the service may be provided by inputting the voice or text.

In addition, the AI entity 200, based on the result of performing the reinforcement learning, to automatically control the plurality of IoT devices 400 provided in the user's home, office, or school, displays current time information or weather information. By inputting the learning model for controlling the IoT device to control and select the control information for controlling the IoT device 400 to the user, to control the IoT device 400, or according to the control information, the IoT device ( 400) can be controlled automatically.

At this time, the AI entity 200 accesses a home network gateway and transmits control information for controlling a specific IoT device 400 to the corresponding IoT device 400 according to the user's selection, so that the IoT device remotely It is possible to automatically control the 400.

On the other hand, when the request command recognized through the interaction designates a target for a specific service or includes direct control information for the IoT device 400, the specified target is automatically searched for and provided, or the control information is provided. Based on this, it may be implemented to automatically control the IoT device 400.

For example, when the user inputs a request command such as "Set the temperature of the living room to 27 degrees" by voice or text, or when requesting music by specifying a specific music title, the AI entity 200 may: By extracting the keywords "living room", "temperature", and "27 degrees" from the request command, control information for the heating device is generated to maintain the temperature of the living room at 27 degrees, and the generated control information is generated by the corresponding heating device. By transmitting, the temperature of the living room can be maintained at 27 degrees, or the sound source for the specific music title is searched to provide the searched sound source to the user.

Through this, the AI entity 200 recommends music desired by the user or operates at least one IoT device 400 on the user's home, office, or school on behalf of the user at a time and method desired by the user. Alternatively, an operation for at least one IoT device 400 is set so that the user can maintain a desired home, office, or school environment.

In addition, if the user's schedule is set according to a preset time (for example, 6 am) to set a recommendation for a task to be performed by the user, the AI entity 200 is preset in the learning model for schedule recommendation. Enter the condition information (eg current time information). At this time, the schedule recommendation learning model outputs schedule information of the corresponding user according to the time and date according to the result of the reinforcement learning. Subsequently, the AI entity 200 selects at least one of the output schedule information according to a preset period (eg, one day, one week, one month) and provides it to the user, so that the corresponding user is connected to the schedule information. You will be recommended to follow.

In addition, when the user performs a request command for schedule information through the interaction, the AI entity 200 extracts a keyword from the request command, and extracts the keyword (eg, "schedule", "schedule") and the plurality By comparing the representative keywords of, it is recognized that the request command is a request for schedule recommendation, and by inputting time information into the learning model for schedule recommendation, schedule information for the request command can be extracted and provided to the user. .

For example, when the representative keyword for the schedule recommendation is set to "schedule", "schedule", and "anniversary", and the user inputs a request command "check today's schedule" by text or voice, the AI entity (200) extracts the keywords "today" and "schedule" from the request command, and compares the representative keyword with the extracted keyword to recognize that the request command is for schedule recommendation.

Subsequently, the AI entity 200 recommends what to do today by inputting current time information into a schedule recommendation learning model to extract schedule information for today and providing schedule information for the extracted today to the user. Is done.

Meanwhile, the learning model is generated through a machine learning algorithm including an artificial neural network (ANN), a convolutional neural network (CNN), or a recurrent neural network (RNN), and as described above, at least one or more specialized for a specific service Includes sectoral learning models.

As described above, the AI object providing platform 100 of the present invention is gradually advanced to suit the characteristics or inclinations of the user by continuously performing reinforcement learning on the created AI object 200. It will provide at least one service suitable for your needs.

As shown in FIG. 2, the process of performing reinforcement learning on the AI entity 200 using the AI entity providing platform 100 according to an embodiment of the present invention is first, the user provides the AI entity ( After installing the artificial intelligence application provided from 100) to the user terminal 300, and running the installed artificial intelligence application to access the AI object providing platform 100, the authentication information and personal information of the user Provided to the AI entity providing platform 100, the creation request for creating the AI entity 200 is transmitted through the AI entity providing platform 100 (①).

At this time, the user may create an AI entity 200 implemented as a character or an avatar by performing a modeling process for the AI entity 200 using customizing information provided from the AI entity providing platform 100. To make.

The AI entity 200 is generated for a plurality of users according to a place (for example, a user's home, office, or school) provided in the AI entity providing platform 100, and the created AI entity 200 is Applied to the AI entity providing platform 100, the AI entity providing platform 100 may be implemented to perform the function of the AI entity 200.

In addition, the created AI entity 200 is applied to the user terminal 300 or is applied to a cloud platform or web server providing SNS service, and the AI is provided through interworking with the AI entity providing platform 100. It may be implemented to perform the function of the object 200.

In addition, the AI entity providing platform 100 provides (②) a data format for inputting user characteristic information to the user terminal 300 through a user interface, and the user generates user characteristic information through the data format. To the AI entity providing platform 100 (③).

Meanwhile, the AI entity 200 includes a learning model for each field of the AI entity 200, and the learning model for each field is generated to provide various services to a user, and the AI entity providing platform 100 , Learning user characteristic information input from a user for each field to generate a learning model for each field.

That is, the reinforcement learning model is generated based on the user's characteristic information, and is generated by the user selecting his characteristic information or inputting it as text based on the data format provided from the AI entity providing platform 100 It is transmitted to the AI entity providing platform 100.

Meanwhile, the learning model for each field is generated by learning user characteristic information that is initially input from the user terminal 300. Thereafter, the continuously input user characteristic information may be used as reinforcement learning data for performing reinforcement learning for the learning model for each field.

For example, if the user characteristic information is for a TV channel (eg, sports, culture, entertainment, drama, documentary, etc.) that includes a user's preference (score) for each time period, the user characteristic information is recommended for TV channel By learning through a learning network, a learning model for TV channel recommendation is generated to recommend TV channels to corresponding users by time zone.

At this time, the input of the learning channel for TV channel recommendation becomes preset condition information (eg, current time information), and the output becomes at least one TV channel having high preference.

As another example, the lighting control information or time, wherein the user characteristic information includes preferences for preferred lighting brightness, lighting color, and on/off time by time and place (eg, living room, master room, study, kitchen, bathroom), and In the case of temperature control information including preferences for heating and cooling temperature values for each place, by controlling characteristic information of the user through a learning network for controlling the IoT device 400, control for controlling a specific IoT device 400 to the corresponding user Information will be recommended.

At this time, the input of the reinforcement learning network for controlling the IoT device 400 may be time information and weather information, and the output is control information for at least one IoT device 400 having high preference.

As another example, if the characteristic information is for a preferred music genre (eg, popular music, classical music, etc.) according to time, weather, or user's sensitivity, the learning network for recommending music to the user's characteristic information By learning through, a learning model for recommending music is generated to recommend at least one piece of music to a corresponding user.

At this time, the input of the learning model for music recommendation may be time information, weather information, emotion information, or a combination thereof, and the output may be at least one music genre and artist information having high preference. Thereafter, the AI entity 200 accesses a sound source site linked to the AI entity 200 based on the output music genre and artist information to at least one or more music sources of the music genre and the artist for the music genre. Search by and recommend to the user.

In addition, the created AI entity 200 provides at least one of the services using the learning model for each field according to a preset condition (for example, time to receive the service, etc.), or the user The service is provided in at least one according to a request command input from the user through interaction with (④).

Further, when the AI entity providing platform 100 provides a service for a specific field through interaction with the user through the AI entity 200, the interaction result is applied to a learning model for the specific field, By performing reinforcement learning on the learning model, the AI entity 200 can be advanced and evolved.

For example, the AI entity 200 basically provides a service for the specific field through the supervised learning, but when interacting with a user, the user may directly designate an object to be serviced. That is, when the corresponding user requests "music containing a specific title" as a request command, the AI entity 200 retrieves music for the title from a server providing music, such as a sound source site, and provides it to the user In addition, when the user's request frequency for the corresponding music is high, the reinforcement learning can be performed by adjusting the weight of the learning model for recommending the music to recommend the corresponding music.

That is, when the AI entity 200 and the user interact with the AI entity providing platform 100, the result of the interaction is input (ie, applied) to the learning model for each field, so that the learning model for each field is It is to be reinforced learning, and accordingly, the AI entity 200 can be gradually evolved.

On the other hand, it is as described above that the reinforcement learning can be performed through self-learning, participatory learning, supervised learning, and autonomous learning on the AI entity 200 in addition to the interaction result.

In addition, when the AI entity providing platform 100 automatically provides the service to the user according to a preset condition through supervised learning on the AI entity 200, the user selects a specific target for the service or , If the selection is rejected, it is as described above that reinforcement learning for the learning model can be performed by adjusting the weight for the specific object.

In addition, the user terminal 300 and the AI entity 200 applied on the SNS are implemented to provide the service using a learning model for each field in which the reinforcement learning is performed.

3 is a view illustrating a process of evolving an AI entity through an AI entity platform according to an embodiment of the present invention.

As illustrated in FIG. 3, in the process of evolving the created AI entity 200 through the AI entity platform 100 according to an embodiment of the present invention, the user first uses the AI entity 200. In order to activate the corresponding AI entity 200 first.

The activation is performed by performing a preset gesture or by fingerprint recognition or calling the name of the AI entity 200 set in the corresponding AI entity.

Next, when the AI entity 200 is activated, the corresponding user inputs a recommendation command for a service field that he or she wants to receive service.

The recommendation command may be input by voice or text, and when input by voice, may be performed through a microphone to the user terminal 300 to which the AI entity 200 is applied or the AI entity providing platform 100. .

Meanwhile, the user may input the recommendation command as text through a chat function of the SNS account or a chat function provided by the AI entity 200 automatically through the chat with the AI entity 200. At this time, the AI entity 200 participates in the chat as an independent entity.

Next, the AI entity 200 recognizes a voice or text for the input recommendation command, extracts a keyword from the recognized recommendation command, and recognizes a service field required by the user based on the extracted keyword.

That is, the AI entity 200 recognizes the recommended field according to the comparison result by comparing at least one keyword extracted from the recommendation command with a plurality of representative keywords for each predetermined field.

For example, when the representative keyword for recommending music is set to "song" and "music", and the keyword extracted from the recommendation command is "music", the AI entity 200 may recommend the recommendation command to the music recommendation service. Recognize as.

As another example, if the representative keyword for schedule recommendation is set to "schedule", "schedule", "anniversary", etc., and the keyword extracted from the recommendation command is "schedule" or "schedule", the AI entity 200 ) Means that the recommendation command is recognized as a schedule recommendation service.

As another example, when the representative keyword for TV channel recommendation is set to "channel", "TV", "drama", "documentary", "entertainment", etc., and the keyword extracted from the recommendation command is "drama", The AI entity 200 recognizes the recommendation command as a TV channel recommendation service.

Next, the AI entity 200 provides a service for the recognized service field to the user.

For example, when the user inputs a request command "Play sports channel" in voice or text, the AI entity 200 automatically turns on a specific sports channel or recommends at least one sports channel to select the user In accordance with this, it is possible to watch a specific sports channel.

Thereafter, the AI entity providing platform 100, when the provided service is equal to or more than a preset frequency, is applied to a learning model in a corresponding field, thereby performing reinforcement learning on the learning model, thereby providing the AI entity 200 ) To evolve.

That is, the AI entity providing platform 100 performs reinforcement learning on the learning model by allowing the interaction result to be applied to the corresponding learning model. Through this, the AI entity providing platform 100 enables the AI entity 200 to evolve through the reinforcement learning.

As shown in FIG. 4, the AI object providing platform 100 according to an embodiment of the present invention generates the AI object 200 by modeling the avatar or character according to the user interface unit 110 and the user's selection. The AI object generating unit 120, a learning data input unit 130 that provides a user with user characteristic information input data format to input learning data for user characteristic information, based on the inputted learning data, provides a learning model for each service field. It comprises a learning model generating unit 140 to generate and learning model evolution unit 150 to perform the reinforcement learning on the generated learning model, so that the generated AI entity 200 evolves.

The user interface unit 110 generates the AI entity 200 between the AI entity providing platform 100 and the user terminal 300, and related data for generating a learning model for the AI entity 200. It performs a function of providing a user interface to send and receive.

In addition, the AI object generating unit 120 automatically provides a specific service to the user, or an AI object 200 including an avatar, a character, etc. for providing at least one specific service through interaction with the user ).

On the other hand, the AI entity 200 receives authentication information and personal information of the corresponding user together with the request for creating the AI entity 200 from the user terminal 300, and issues a user account for the AI entity 200. Is created.

In addition, the AI entity generating unit 120 provides the customized information stored in advance to the user through the user interface unit 110, thereby allowing the user to select the customizing information, so that the characteristics of the avatar or character By setting the, it is possible to model the AI object 200 according to the user's preference.

The AI entity 200 is generated by having a specific set by the user according to the modeling result when applied on the user terminal 300 or the SNS or the AI entity providing platform 100.

Meanwhile, the customizing information includes social elements including language and nationality to be applied to the AI entity 200, biological elements including gender and age, and appearance elements including emotions, faces, headers, and costumes, and emotional elements and AI entities. Contains the name for 200.

That is, the AI entity providing platform 100 provides the customizing information through the user interface 110 to set a name for activating the corresponding AI entity 200, and the AI entity 200 Social elements, biological elements, appearance elements, and emotional elements to be applied to are sequentially provided, and selected to make it possible to model the AI entity 200 according to a user's preference.

In addition, the learning data input unit 130 provides a means for the user to input learning data (that is, user characteristic information) for generating a learning model for each service field for the created AI entity 200.

The learning data input unit 130 provides a user with a user characteristic information input data format previously defined through the user interface unit 110.

Thereafter, when the user inputs his/her user characteristic information based on the received user characteristic information input data format, the learning data input unit 130 classifies the input user characteristic information into service fields, and performs memory (not shown). City).

In addition, the learning model generating unit 140 performs a function of learning user characteristic information classified for each recommended field for each service field and generating a learning model for each service field.

Meanwhile, the learning model for each service field is generated by learning user characteristic information classified for each service field, a learning model for TV channel recommendation, a learning model for music recommendation, a learning model for controlling IoT device 400, and a news recommendation It may include a learning model, a learning model for schedule recommendation.

However, it is natural that the learning model for each field may be extended to various fields by the designer of the AI entity providing platform 100.

In addition, the AI entity providing platform 100 allows the created AI entity 200 to be applied to the user terminal 300 of a corresponding user, or by creating an account for an SNS server that provides an SNS service, thereby being independent on the SNS. It can be implemented to perform SNS activities as individuals.

In addition, the created AI entity 200 may automatically provide at least one service to the user by using the learning model for each service field learned through supervised learning, and perform the interaction by interacting with the user. According to one interaction, various information may be searched on the web and provided to a user, a specific IoT device 400 may be controlled, a schedule of the user may be reported, music, videos, news, and the like may be recommended.

In addition, the learning model evolution unit 150, through the interaction with the user by performing reinforcement learning on the generated learning model for each service field according to the result of the interaction, so that the generated AI entity 200 to evolve To perform the function.

In addition, the learning model evolution unit 150 may further include a function that enables the AI entity 200 to evolve by combining the at least one learning model with each other.

That is, the learning model evolution unit 150 combines at least one learning model specialized for a specific service according to a related degree of learning data for reinforcement learning so that the AI entity 200 can evolve.

For example, when recommending music for each time zone by weather, or when controlling a plurality of IoT devices 400 for adjusting the temperature or humidity in the house for each time zone by weather, a service and IoT recommending music by time zone for each weather It can be seen that the relevance of the service controlling the device 400 is high. That is, the learning data used for the learning model for recommending music or the learning model for controlling the IoT device includes weather and time, and it can be seen that the related degree of the learning data is high.

Therefore, the learning model evolution unit 150, by combining the learning model for music recommendation and the learning model for controlling the IoT device by mutually providing services for controlling music and IoT devices for each time of weather, The AI entity 200 can be evolved. In this case, the combining is performed by quantifying the learning data into a characteristic value according to the degree of relevance for the related field and scaling it to have a value between 0 and 1.

In addition, the learning model evolution unit 150 continuously performs reinforcement learning on the learning model for each service field through self-learning, participatory learning, supervised learning, and self-learning in addition to the interaction, so that the AI entity 200 is an independent entity. Can evolve into

On the other hand, since the self-learning, participatory learning, supervised learning, and autonomous learning were described with reference to FIG. 1, further detailed description will be omitted.

In addition, the learning model evolution unit 150 is not the first time the user characteristic information collected from the user characteristic information collection unit 130 is input, but by the service field for the AI entity 200 and the corresponding AI entity 200. As described above, when the learning model is generated and then input, it may further include performing reinforcement learning on the learning model for each service field using the corresponding user characteristic information.

As illustrated in FIG. 5, the procedure for providing an AI entity that evolves through reinforcement learning according to an embodiment of the present invention is first, the user accesses the AI entity providing platform 100 through the user terminal 300 , By requesting the AI entity providing platform 100 to create the AI entity 200, the AI entity 200 for the user is generated (S110 ).

Generating the AI entity 200, the user inputs the user's authentication information and personal information to the AI entity providing platform 100 to perform a creation request for the AI entity 200, and the AI entity The providing platform 100 is performed by issuing an account for the AI entity 200.

At this time, as described above, the AI entity providing platform 100 can model the AI entity 200 according to a user's preference by providing predefined customization information to the user.

Next, the AI entity providing platform 100 receives the user characteristic information of the corresponding user from the user (S120), by service field for the created AI entity 200 using the received user characteristic information Create a learning model (S120).

At this time, the AI object providing platform 100 provides a user characteristic information input data format through a user interface so that user characteristic information can be input in a preset format.

In addition, the AI entity providing platform 100 classifies the received user characteristic information for each service field, and learns each user characteristic information for each classified service field to generate a learning model for each service field.

In addition, as described above, the generated learning model for each service field is gradually advanced through reinforcement learning, and through this, the AI entity 200 is evolved into an independent entity.

Next, when the created AI entity and the user perform an interaction (S140), the AI entity 200 provides the interaction result of the interaction to the user, and the AI entity providing platform 100 comprises: The generated AI entity 200 is evolved by performing reinforcement learning on the generated learning model using the interaction result (S150).

In the above, the preferred embodiment according to the present invention has been mainly described, but the technical spirit of the present invention is not limited thereto, and each component of the present invention is changed or modified within the technical scope of the present invention in order to achieve the same purpose and effect. It could be.

In addition, although the preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention pertains without departing from the gist of the present invention claimed in the claims. In addition, various modifications may be implemented by a person having ordinary knowledge in the art, and these modifications should not be individually understood from the technical idea or prospect of the present invention.

According to the present invention, a user creates an AI object and continuously learns it as he wants, and through reinforcement learning, the user evolves into an independent object that the user himself wants. You can control the IoT device, report the schedule of the user, or recommend music, products, news, etc.

Claims

An AI entity creation step in which the user creates an AI entity;

A learning model generation step of generating a learning model for the created AI entity through machine learning; And

When a user interacts with an AI entity through the generated learning model, the result of the interaction is applied to the learning model to perform reinforcement learning on the learning model, thereby generating the AI according to the user's interaction behavior. A method of providing an AI entity that evolves through reinforcement learning, comprising: an AI entity evolution step that causes the entity to evolve into an independent entity.
The method according to claim 1,

The above method,

A method of providing an AI entity evolving through reinforcement learning, further comprising evolving into an individual entity with the user while the AI entity interacts with other AI entities through the AI entity evolution step.
The method according to claim 1,

The AI entity,

It is created by creating an account on a specific device composed of independent hardware that performs the above method or on a web server or cloud platform on the Internet,

To function as a secretary desired by the user, to enable SNS activities, to operate at least one IoT device on behalf of the user at home, school, or office in a time and manner desired by the user, or to operate at least one IoT device A method of providing an AI entity that evolves through reinforcement learning, characterized in that the user maintains a desired home, school, or office environment, or performs at least one or a combination of these.
The method according to claim 1,

The above method,

In the process of learning the learning model and generating a learning model, learning to allow a user to input learning data in a predetermined format by providing an interface for inputting learning data to a user and presenting a method for learning to the user Method for providing an AI entity evolving through reinforcement learning, characterized in that it further comprises a data input step.
The method according to claim 1,

The learning model includes learning models specialized for at least one field including CNN, RNN, or ANN,

The AI entity evolves by combining the at least one specialized learning model with each other,

The method for providing an AI entity evolving through reinforcement learning, wherein the mutual coupling is performed by quantifying and scaling learning data according to a related degree in a related field.
An AI object creation unit supporting the user to create an AI object;

A learning model generator for generating a learning model for the created AI object through machine learning; And

When a user interacts with the created AI entity through the generated learning model, the result of the interaction is applied to the learning model to perform reinforcement learning on the learning model, and according to the user's interaction behavior, A platform for providing AI entities that evolve through reinforcement learning, comprising: a learning model evolution unit that allows the generated AI entities to evolve into independent entities.
The method according to claim 6,

The learning model evolution unit,

Providing an AI entity evolving through reinforcement learning, further comprising allowing the AI entity to evolve as an independent entity while interacting with other AI entities through the process of evolving into the independent entity. platform.
The method according to claim 6,

The AI entity,

It is created by creating an account on a web server or cloud platform on a specific device or on the Internet that is composed of independent hardware that provides AI objects that evolve through the reinforcement learning.

To function as a secretary desired by the user, to enable SNS activities, to operate at least one IoT device on behalf of the user at home, school, or office in a time and manner desired by the user, or to operate at least one IoT device A platform that provides evolving AI entities through reinforcement learning, characterized by setting up to maintain a user's desired home, school, or office environment, or performing at least one or a combination of these.
The method according to claim 6,

The platform,

When learning model is generated by learning the learning model, by providing an interface for inputting learning data to the user and presenting a method for learning to the user, learning data allowing the user to input learning data in a predetermined format. A platform for providing an AI entity that evolves through reinforcement learning, characterized by further comprising an input unit.
The method according to claim 6,

The learning model includes a learning model specialized for at least one field including CNN, RNN, or ANN,

The AI entity evolves by combining the at least one specialized learning model with each other,

The mutual coupling is a platform for providing an AI entity that evolves through reinforcement learning, characterized in that it is performed by quantifying and scaling learning data according to a related degree in a related field.