KR20220053795A - System and method for providing artificial intelligence secretary service - Google Patents

Abstract

The present invention relates to a system and a method for providing an artificial intelligence secretary service, which are implemented to allow a service function and an artificial intelligence secretary function to interwork with each other, as well as a content (particularly, multimedia content) provision function. According to the present invention, a content server provides content, a service server provides a service, and an artificial intelligence server manages the content provided by the content server and the service provided by the service server to interwork with an artificial intelligence secretary function, and recognizes user voice to search for the content and to use the service. Also, a user device receives the user voice to transmit the same to the artificial intelligence server, and then, searches for the content for which a user desires to search, and uses the service, and a network connects the content server, the service server, the artificial intelligence server, and the user device to transceive data with each other.

Description

System and method for providing artificial intelligence secretary service {System and method for providing artificial intelligence secretary service}

The technical field of the present invention relates to a system and method for providing an artificial intelligence assistant service. More specifically, the provision of the assistant service with a screen visible is possible with a simple structure, and the reception of the screen in the screen output unit is data through scan Since it can be easily received from , and it is used in the cloud method, it is possible to use a lot of content at low cost. It relates to a system and method for providing an intelligent assistant service.

Artificial intelligence assistant service using voice recognition technology is being researched and developed in various ways. At this time, an artificial intelligence speaker is used for voice recognition. The corresponding artificial intelligence speaker requires the user to pronounce a predetermined call word in order to switch from the standby state to the wake-up mode, and the artificial intelligence speaker is activated through voice recognition of the call word Thereafter, when there is a user's request for providing a service, voice recognition for the requested voice and provision of the service accordingly are performed.

The call word required for activation of such an artificial intelligence speaker is not required only the first time, and even if the same user requests additional artificial intelligence service at a time interval, the call word is uttered prior to every request. This requires a cumbersome procedure to switch the artificial intelligence speaker to an active state.

In addition, when the call word is recognized, the artificial intelligence speaker provides a service according to the service request without going through separate authentication for the user with respect to the user's service request voice after that. For this reason, when there are multiple users (eg, A, B, C, D, etc.) in the space where the artificial intelligence speaker is installed, user A inputs a call word, and then user B in the same space requests service. Even for sounds uttered without intention, the artificial intelligence speaker recognizes it as user A's service request voice and causes a malfunctioning error.

In addition, since the artificial intelligence speaker cannot distinguish and recognize the voices of a plurality of users, when user A's service request, user B's service request, and user A's service request are sequentially made, these requests are classified by user. Therefore, connection processing cannot be performed, and each request has no choice but to be processed in parallel.

Korea Patent Registration No. 10-2087202 (registered on March 4, 2020) discloses a method for providing an artificial intelligence assistant service, and a voice recognition device used therein, the step of receiving a call word voice from a user, the voice recognition device ; determining, by the voice recognition device, whether a call word input by a user matches a preset call word; When the voice recognition device determines that the call word matches the preset call word, the parameter value for voiceprint analysis in the first service request voice signal input after the call word voice, and the first service request voice from the user authenticating the speaker by comparing parameter values for voiceprint analysis in the input caller voice signal; In the case where the voice recognition equipment utters the second service request voice without the call word being uttered after the first service request by the user, the parameter value for component analysis of the secondary service request voice signal and the first service request voice signal authenticating the speaker by comparing the parameter values for component analysis of ; and the voice recognition equipment, when the user utters a third service request voice without uttering a call word after the second service request, parameter values for component analysis in the tertiary service request voice signal and the first service request voice signal or authenticating the speaker by comparing parameter values for component analysis in the secondary service request voice signal, wherein the voice recognition equipment includes a parameter value for component analysis in the calling word voice for each ID of a plurality of users, the primary Parameter values and service request content for component analysis in the service request voice signal, parameter values for component analysis in the secondary service request voice signal, service request content, and parameter values and service request for component analysis in the tertiary service request voice signal It is characterized in that the contents are accumulated and stored. According to the disclosed technology, it is possible to continuously use the artificial intelligence assistant service without the need for the user to repeatedly input a predetermined call word, and by separately executing the speaker authentication procedure for the user's service request, the voice of an unauthorized third party can be heard. In addition to being able to prevent a malfunction by the user, when service request voices are accumulated from a plurality of users with legitimate authority, service requests can be classified for each user and linked processing can be performed.

Korean Patent No. 10-1976355 (registered on May 1, 2019) operates by connecting an AI speaker through a digital interface (eg, USB) to the outside of a set-top box lacking an artificial intelligence (AI) function and interworking. By doing so, it is disclosed with respect to an AI speaker device of an external connection type of a set-top box such as an AI function integrated in the set-top box as a whole, and an AI speaker system using the same. According to the disclosed technology, an AI speaker device for providing an AI speaker system through interworking by being externally connected through a digital interface to a set-top box, comprising: a microphone voice input unit for collecting and inputting surrounding voice signals in relation to the AI speaker; a digital external connection unit for externally connecting to the set-top box through a digital interface; a set-top box interlocking unit for operating in conjunction with the set-top box through a digital interface; As a playback audio buffer unit that receives and temporarily stores playback audio of multimedia content provided from a set-top box as an echo reference signal, it receives and temporarily stores playback audio of multimedia content provided from a set-top box as an echo reference signal in order a playback audio buffer unit configured to include a buffer memory and a clock controller controlling an operation clock of the buffer memory to be proportional to the occupancy rate of the buffer memory; an echo cancellation unit that removes an audio echo component of the content due to the set-top box from the surrounding audio signal collected by the microphone audio input unit with reference to the echo reference signal; a user voice processing unit that pre-processes the user's voice using the surrounding voice signal from which the reproduced audio echo component has been removed and transmits it to the set-top box through a digital external connection unit; It is characterized in that it is configured to include a speaker voice output unit for outputting the artificial intelligence response data obtained through the set-top box to the voice band.

In the prior art as described above, a high-performance system that interoperates with a service function and an artificial intelligence assistant function as well as a content (especially multimedia content) provision function has not been built. It has a disadvantage in that it is not possible to easily search for the desired content or use the service conveniently.

Korean Patent Registration No. 10-2087202 Korean Patent Registration No. 10-1976355

The problem to be solved by the present invention is to solve the disadvantages described above, and it is possible to provide a screen-visible assistant service with a simple structure, and the reception of the screen from the screen output unit is simply received from the data through the scan This makes it possible and uses the cloud method, so it is possible to use a lot of content at low cost, and provides not only the content (especially multimedia content) function, but also the artificial intelligence assistant service implemented so that the service function and the artificial intelligence assistant function can be interconnected. To provide a system and method.

As means for solving the above problems, according to one aspect of the present invention, a content server for providing content; a service server for providing services; an artificial intelligence server for managing the content provided by the content server and the service provided by the service server by interworking with an artificial intelligence assistant function, and for recognizing a user's voice to search for content and use the service; a user device for receiving a user's voice and transmitting it to the artificial intelligence server, then searching for content that the user wants to search for and using a service; and a network for connecting the content server, the service server, the artificial intelligence server, and the user device to transmit and receive data between each other.

In an embodiment, the content server transmits content information corresponding to the content name received from the artificial intelligence server to the artificial intelligence server.

In an embodiment, the content server searches for content corresponding to the content selection notified from the user device and transmits the searched content to the user device.

In an embodiment, the content server transmits a content file including the searched content to the user device in real time so that the user can view it through a screen.

In one embodiment, the content server is characterized in that the data on the content is matched with the detailed item information for each content and transmitted to the user device.

In an embodiment, the content server classifies the usage history of content by user ID for a plurality of user IDs preset for each user device in order to provide content corresponding to the individual service platform, and provides content information is transmitted to the user device.

In an embodiment, the service server is characterized in that the data on the service is matched with detailed item information for each service and transmitted to the user device.

In one embodiment, the service server, in order to provide a service corresponding to the individual service platform, classifies the service usage history for each user ID for a plurality of user IDs preset and registered for each user device, and provides the service. It is characterized in that it is transmitted to the user device.

In an embodiment, the artificial intelligence server distributes the content provided from the content server and transmits the distributed content to the user device.

In an embodiment, the artificial intelligence server processes a user response signal input from the user device.

In one embodiment, the artificial intelligence server, after inquiring from a database in which the device information corresponding to the user ID and the individual service platform corresponding to the user ID received from the user device is preset and registered, the individual service platform corresponding to the user ID is retrieved from the user. It is characterized in that it is activated by a user device corresponding to the ID.

In an embodiment, the artificial intelligence server analyzes the user voice received from the user device to extract the content name that the user wants to search, and then transmits the extracted content name to the content server.

In one embodiment, the artificial intelligence server is characterized in that it is provided with a platform that provides an interface capable of interworking with a user device to provide an artificial intelligence assistant service.

In one embodiment, the artificial intelligence server is an open standard for delegating access rights, and according to the agreement for granting an Internet user the right to access a user account to another web service or application, in the AI assistant platform It is characterized in that the client acquires an AI assistant access token or uses it when a user connects his or her account when using a specific extension.

In one embodiment, the artificial intelligence server registers a client through an artificial intelligence assistant developer console and uses client authentication information, which is authentication information obtained, to acquire an artificial intelligence assistant access token. do.

In an embodiment, the user device receives the distributed content from the artificial intelligence server and provides it to the user through a screen, and receives the service from the artificial intelligence server and provides it to the user through the screen.

In one embodiment, the user device, after transmitting the user ID to the artificial intelligence server, characterized in that the activation of the individual service platform provided by the artificial intelligence server.

In an embodiment, the user ID is an identifier for distinguishing individual messages, and it is characterized in that both the event message and the instruction message have individual user IDs.

In an embodiment, the event message is a message transmitted from the client to the artificial intelligence server, and is transmitted when a user request is transmitted or a state value of the client is changed.

In one embodiment, the instruction message is a message that specifies that the artificial intelligence server controls the behavior of the client, and is used when responding to an event message requested by the client or transmitting information to the client according to a specific condition. do.

In an embodiment, the user device provides the content information received from the content server to the user through a screen, and notifies the content selection to the content server when the user selects content, and from the content server It is characterized in that the received content is provided to the user through a screen.

In one embodiment, the user device is characterized in that it is provided with a terrestrial TV that outputs in-house airwave broadcasting content and transmits a user response signal received through an input means to the artificial intelligence server.

In an embodiment, the user device includes: an application layer that supports multi-codecs and includes a voice agent processing unit for selecting a service; Java virtual machine, middleware layer with streaming protocol; a system software layer including a device driver and system software of an operating system; It is characterized in that it includes a hardware layer consisting of hardware of CPU, media processor, flash RAM, and Ethernet module.

In an embodiment, the user equipment is characterized in that an IPv4 address or an IPv6 address is allocated.

In an embodiment, the user device receives image data and digital information related to a program by using a digital broadcast receiving device for receiving content, and provides the received image data and digital information to the user.

In an embodiment, the user device is characterized in that the information received from the artificial intelligence server is output and displayed on a screen so that the user can search for desired digital information.

In one embodiment, the user device is characterized in that it includes a desk view equipped with a screen touch monitor equipped with an artificial intelligence assistant platform.

In an embodiment, the user device generates a conversation ID whenever the user starts a new utterance, and includes the conversation ID when the client transmits a cognitive event message to the artificial intelligence server.

In an embodiment, the conversation ID is characterized in that it is used when connecting the response to which event message when sending the AI server-side response.

In one embodiment, the conversation ID allows the client to determine which event message is a response by looking at the conversation ID included in the instruction message, and if the conversation ID currently possessed by the client is different from the conversation ID of the instruction message, the instruction received In order to ignore the message, it is characterized in that it is included in the indication message.

In an embodiment, the network includes: a protocol for transmitting the content provided by the content server to the user device and providing a user response signal transmitted from the user device to the content server; It is characterized in that it is an IP network using a protocol for transmitting the service provided by the service server to the user device and providing a user response signal transmitted from the user device to the service server.

In an embodiment, the user device may include: an external server interworking unit for transmitting and receiving data in conjunction with the artificial intelligence server or the content server or the service server; a control unit connected to the external server interworking unit and controlling operation of a user device; a storage unit connected to the control unit to store programs or data necessary for the control of the control unit; an image signal processing unit connected to the control unit and performing signal processing on the image data received from the artificial intelligence server; a screen output unit for outputting and displaying the image data signal-processed by the image signal processing unit through a screen; a voice agent processing unit connected to the control unit, supporting multiple codecs, and selecting a service; an audio processing unit connected to the control unit, receiving a user's voice through a microphone, processing the audio, and transmitting the audio to the artificial intelligence server; and a speaker output unit connected to the control unit to output audio data received from the artificial intelligence server.

In an embodiment, the image signal processing unit performs signal processing on the content received from the artificial intelligence server and provides it to the screen output unit.

In an embodiment, the screen output unit outputs and displays content processed by the image signal processing unit on a screen, receives a user response signal and transmits it to the artificial intelligence server through a return channel of the user device do it with

In one embodiment, the voice agent processing unit, a content reproduction processing module for processing the reproduction of the content; a service reproduction processing module for reproducing a service; and an external IoT processing module for processing external IoT data by interworking with external IoT.

In one embodiment, the audio processing unit recognizes the audio generated by the user device itself, and when the phase is reversed, an echo cancellation module for removing the audio of the internal sound source on the principle that the sound is canceled; introduced into the microphone a noise canceling module for removing ambient noise; and a microphone voice input module for receiving a microphone audio input.

As a means for solving the above problems, according to another aspect of the present invention, the content server provides the content; providing a service by the service server; managing, by an artificial intelligence server, the content provided by the content server and the service provided by the service server by interworking with an artificial intelligence assistant function; receiving, by the user device, the user's voice and transmitting it to the artificial intelligence server; allowing the artificial intelligence server to search for content by recognizing a user's voice and use a service; and providing, by the user device, a step in which the user searches for the content the user wants to search for and uses the service.

As an effect of the present invention, it is possible to provide a screen-visible assistant service with a simple structure, and the reception of the screen from the screen output unit can be easily received from the data through the scan, and since it is used in the cloud method, many contents at a low cost By providing a system and method for providing an artificial intelligence assistant service that enables the use of content (especially multimedia content) as well as a service function and an artificial intelligence assistant function to interoperate with each other, by recognizing the user's voice This means that users can easily search for the content they want to search for and use the service conveniently.

1 is a view for explaining an artificial intelligence assistant service providing system according to an embodiment of the present invention.
FIG. 2 is a diagram for explaining the user device shown in FIG. 1 .
3 is a view for explaining a method of providing an artificial intelligence assistant service according to an embodiment of the present invention.
4 is a schematic explanatory diagram showing an actual use state of the artificial intelligence assistant service providing system according to the first embodiment of the present invention;
5 is an initial start screen of the desk view of the artificial intelligence assistant service providing system according to the first embodiment of the present invention;
6A, 6B, and 6C are diagrams for explaining the operation of the desk view of the artificial intelligence assistant service providing system according to the first embodiment of the present invention;

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those of ordinary skill in the art to which the present invention pertains can easily implement them. However, since the description of the present invention is merely an embodiment for structural or functional description, the scope of the present invention should not be construed as being limited by the embodiment described in the text. That is, since the embodiment is capable of various changes and may have various forms, it should be understood that the scope of the present invention includes equivalents capable of realizing the technical idea. In addition, since the object or effect presented in the present invention does not mean that a specific embodiment should include all of them or only such effects, it should not be understood that the scope of the present invention is limited thereby.

The meaning of the terms described in the present invention should be understood as follows.

Terms such as “first” and “second” are for distinguishing one component from another, and the scope of rights should not be limited by these terms. For example, a first component may be termed a second component, and similarly, a second component may also be termed a first component. When a component is referred to as being “connected to” another component, it may be directly connected to the other component, but it should be understood that other components may exist in between. On the other hand, when it is mentioned that a certain element is "directly connected" to another element, it should be understood that the other element does not exist in the middle. Meanwhile, other expressions describing the relationship between elements, that is, "between" and "between" or "neighboring to" and "directly adjacent to", etc., should be interpreted similarly.

The singular expression is to be understood to include the plural expression unless the context clearly dictates otherwise, and terms such as "comprise" or "have" are not intended to refer to the specified feature, number, step, action, component, part or any of them. It is intended to indicate that a combination exists, and it should be understood that it does not preclude the possibility of the existence or addition of one or more other features or numbers, steps, operations, components, parts, or combinations thereof.

All terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs, unless otherwise defined. Terms defined in the dictionary should be interpreted as being consistent with the meaning of the context of the related art, and cannot be interpreted as having an ideal or excessively formal meaning unless explicitly defined in the present invention.

Now, a system and method for providing an artificial intelligence assistant service according to an embodiment of the present invention will be described in detail with reference to the drawings.

1 is a view for explaining an artificial intelligence assistant service providing system according to an embodiment of the present invention.

Referring to FIG. 1 , the artificial intelligence assistant service providing system 100 includes a content server 110 , a service server 120 , an artificial intelligence server 130 , a user device 140 , and a network 150 .

The content server 110 provides various kinds of content (especially, multimedia content) to the artificial intelligence server 130 through the network 150 .

In an embodiment, the content server 110 receives the content name transmitted from the artificial intelligence server 130, and provides content information corresponding to the received content name (or via the artificial intelligence server 130). After transmitting to the user device 140 so that the user can check it through the screen, the user device 140 (or via the artificial intelligence server 130) is notified of the content selection and searches for the corresponding content Thus (or via the artificial intelligence server 130 ), it can be transmitted to the user device 140 .

In an embodiment, the content server 110 searches for content content corresponding to the content selection notified from the user device 140 and in real time so that the user can view the content file including the searched content content through the screen. It can be transmitted to the user device 140 .

In an embodiment, the content server 110 may match data on various content with detailed item information for each content and transmit it to the user device 140. For example, in the case of content such as music, the content It may be matched with detailed item information such as name, singer, and playback time and transmitted to the user device 140 . Other contents include various contents such as cuisine, restaurants, TV programming, stories, sports games, stock quotes, real estate quotes, and radio reception.

In an embodiment, the content server 110 categorizes content usage history for each user ID for a plurality of user IDs preset and registered for each user device 140 in order to provide content corresponding to an individual service platform. to transmit the content information to the user device 140 .

The service server 120 provides various services to the artificial intelligence server 130 through the network 150 .

Various services of the service server 120, for example, can control synchronized IoT devices such as digital door locks and lighting as a group or individually in a smart home service, and music & content services include customized music and audio It provides rich contents such as books and audio clips, children's & learning service provides children's songs, fairy tales, language and childcare contents in Korean, English, Chinese and other foreign languages, and life information service provides briefing, weather, news, finance , knowledge, people, sports, etc., the schedule management service provides personal assistant functions such as schedule, alarm, timer, briefing, and memo, and the cooking encyclopedia service provides voice search support services for more than 220 dishes and main ingredients. to provide.

In an embodiment, the service server 120 may match data on various services with detailed item information for each service (or via the artificial intelligence server 130 ) and transmit it to the user device 140 . , for example, in the case of weather content, it may be matched with detailed item information such as weather information for each period according to region and transmitted to the user device 140 .

In one embodiment, in order to provide a service (eg, calendar, etc.) corresponding to an individual service platform, the service server 120 provides each user with respect to a plurality of user IDs preset and registered for each user device 140 . By classifying service usage details by ID, the service may be transmitted to the user device 140 .

The artificial intelligence server 130 interworks various contents provided from the content server 110 through the network 150 and various services provided from the service server 120 through the network 150 together with the artificial intelligence assistant function. and manages it, and recognizes the user's voice transmitted from the user device 140 so that the user can search for the content he or she wants to search for and use the service.

In an embodiment, the artificial intelligence server 130 may distribute various types of content provided from the content server 110 and transmit the distributed content to the user device 140 .

In an embodiment, the artificial intelligence server 130 may process a user response signal input from a voice user interface (VUI).

In one embodiment, the artificial intelligence server 130 receives the user ID transmitted from the user device 140, and inquires in a database in which device information and individual service platform corresponding to the received user ID are preset and registered. After that, the individual service platform corresponding to the received user ID may be activated by the user device 140 corresponding to the received user ID.

In one embodiment, the artificial intelligence server 130 analyzes the user voice received from the user device 140 to extract the content name that the user wants to search, and then transmits the extracted content name to the content server 110 can give

In an embodiment, the artificial intelligence server 130 provides an interface capable of interworking with the artificial intelligence server 130 to the user device 140 that intends to provide an artificial intelligence assistant service as an artificial intelligence assistant interface connect (AIC). A platform can be provided.

In one embodiment, the artificial intelligence server 130 is an open standard for delegating access rights, and a protocol (eg, OAuth 2.0), in the AI assistant platform, the client can obtain an AI assistant access token or use it when the user connects his or her account when using a specific extension. In this case, client authentication information, which is authentication information obtained by registering a client through an artificial intelligence assistant developer console, may be used to acquire an artificial intelligence assistant access token.

The user device 140 receives the user's voice and transmits the received user's voice to the artificial intelligence server 130 through the network 150 , and then retrieves the content the user wants to search through the artificial intelligence server 130 . Search and use the service.

In an embodiment, the user device 140 is a device for using content and services, and may receive distributed content transmitted from the artificial intelligence server 130 and provide it to the user through a screen, and also the artificial intelligence server The service transmitted from the 130 may be received and provided to the user through the screen.

In an embodiment, the user device 140 may activate the individual service platform provided by the artificial intelligence server 130 after transmitting the user's ID to the artificial intelligence server 130 . Here, the user ID is an identifier for distinguishing individual messages, and both the event message and the instruction message have individual user IDs. In addition, the event message is a message transmitted from the client to the artificial intelligence server 130, and the corresponding message is transmitted when a user request (voice input) is transmitted or when the status value of the client is changed. And, the directive message is a message for an instruction commanded by the artificial intelligence server 130 to control the behavior of the client, and is used when responding to an event message requested by the client or transmitting information to the client according to a specific condition.

In an embodiment, the user device 140 may receive content information transmitted from the content server 110 and provide it to the user through a screen, and when the user selects content, the content server selects the content 110 , and then the content transmitted from the content server 110 may be received and provided to the user through the screen.

In an embodiment, the user device 140 outputs in-house airwave broadcasting content, and transmits a user response signal received through an input means such as a remote control or a screen touch to the artificial intelligence server 130 through the VA. A TV can be provided.

In an embodiment, the user device 140 may include 4 layers, and an IPv4 address or an IPv6 address may be assigned. Here, the 4th layer includes a voice agent processing unit 146 (see FIG. 2) for supporting various multi CODECs such as MPEG2, MPEG4, MPEG7, H.264, and WMV-9 and for selecting various services. application layer; Middleware layer loaded with Java Virtual Machine (JVM) and streaming protocols (RTP, RTSP); a system software layer including system software such as a device driver and an operating system; There is a hardware layer consisting of hardware such as CPU, media processor, flash RAM, and Ethernet module.

The user equipment 140 includes all four layers, and the four layers include a hardware layer consisting of hardware such as a CPU, a media processor, a flash RAM, and an Ethernet module as a first layer, and a device as a second layer. A system software layer including system software such as drivers and operating systems, a middleware layer equipped with a Java Virtual Machine (JVM) and streaming protocols (RTP, RTSP) as a third layer, and MPEG2 as a fourth layer , MPEG4, MPEG7, H.264, WMV-9, etc. support various multi codecs (Multi CODEC) and includes an application layer including a voice agent processing unit for selecting various services.

In an embodiment, the user device 140 may include a digital broadcast receiving device for receiving various types of content transmitted through the network 150 , and using the corresponding digital broadcast receiving device, not only image data but also all It is possible to receive digital information related to the program and provide it to the user.

In one embodiment, after receiving the information transmitted from the artificial intelligence server 130, the user device 140 outputs and displays the received information through the screen of the screen output unit 145 (refer to FIG. 2). , it is possible to allow the user to easily search for desired digital information.

In an embodiment, the user device 140 may include a device (desk view) equipped with a screen touch monitor equipped with an artificial intelligence assistant platform.

In an embodiment, the user device 140 generates a conversation ID whenever the user starts a new utterance, and the generated conversation ID when the client transmits a recognize event message to the artificial intelligence server 130 . can be included. Here, the conversation ID is used to connect the response to which event message when the artificial intelligence server 130 gives a response, and may be included in the instruction message. The client can determine which event message the response is by looking at the conversation ID included in the instruction message. If the conversation ID the client currently has and the conversation ID of the instruction message are different, the received instruction message can be ignored.

The network 150 includes a wired or wireless communication network, and connects the content server 110 , the service server 120 , the artificial intelligence server 130 , and the user device 140 to each other to provide wired or wireless data between each other. send and receive

In an embodiment, the network 150 may transmit various types of content provided from the content server 110 (or via the artificial intelligence server 130 ) to the user device 140 , and the user device 140 . ) (or via the artificial intelligence server 130) a protocol for providing a user response signal transmitted to the content server 110; Various services provided from the service server 120 (or via the artificial intelligence server 130 ) may be transmitted to the user device 140 , and from the user device 140 (or the artificial intelligence server 130 ) It may be an IP (internet protocol) network using a protocol for providing a transmitted user response signal to the service server 120 ).

The artificial intelligence assistant service providing system 100 having the configuration as described above recognizes the user's voice by implementing not only the content (especially multimedia content) provision function but also the service function and the artificial intelligence assistant function to interoperate. Thus, users can easily search for the content they want to search for and use the service conveniently.

FIG. 2 is a diagram for explaining the user device shown in FIG. 1 .

Referring to FIG. 2 , the user device 140 includes an external server interworking unit 141 , a control unit 142 , a storage unit 143 , an image signal processing unit 144 , a screen output unit 145 , and a voice agent processing unit ( 146 ), an audio processing unit 147 , and a speaker output unit 148 .

To explain this in more detail, the user device 140 interlocks with the artificial intelligence server 130 or the content server 110 or the service server 120 to transmit and receive data to and from an external server interworking unit 141 . ; a control unit 142 connected to the external server interworking unit 141 and controlling the operation of the user device 140; a storage unit 143 connected to the control unit 142 to store programs or data necessary for the control of the control unit; an image signal processing unit 144 connected to the control unit 142 and performing signal processing on the image data received from the artificial intelligence server 130; a screen output unit 145 for outputting and displaying the image data signal processed by the image signal processing unit 144 through a screen; a voice agent processing unit 146 connected to the control unit 142, supporting multiple codecs, and selecting a service; an audio processing unit (147) connected to the control unit (142), receiving a user's voice through a microphone, processing the audio, and transmitting the audio to the artificial intelligence server (130); and a speaker output unit 148 connected to the control unit 142 to output audio data received from the artificial intelligence server 130 .

The external server interworking unit 141 transmits and receives data by interworking with the artificial intelligence server 130 (or the content server 110 or the service server 120 ) under the control of the control unit 142 .

The control unit 142 includes the user device 140 (components (external server interworking unit 141 , image signal processing unit 144 ), screen output unit 145 , voice agent processing unit 146 , and audio processing unit 147 . ), the speaker output unit 148)) controls the driving.

The storage unit 143 stores a program or data necessary for the control of the control unit 142 .

The image signal processing unit 144 receives the image data received from the artificial intelligence server 130 through the control unit 142, performs signal processing on the received image data, and provides it to the screen output unit 145. give.

In one embodiment, the image signal processing unit 144 receives the content received from the artificial intelligence server 130 through the control unit 142, and performs signal processing on the received content to the screen output unit 145 can be provided as

The screen output unit 145 receives the signal-processed image data from the image signal processing unit 144 and outputs and displays the image data through the screen.

The screen output unit 145 may be a kitchen TV used in the kitchen, a desk view used in a living room or a room, or other viewing means for viewing a screen.

In an embodiment, the screen output unit 145 may receive the signal-processed content from the image signal processing unit 144 and output and display it on the screen, and transmit the user response signal input through the VA of the user device 140 . It can be transmitted to the artificial intelligence server 130 through the return channel.

The voice agent processing unit 146 supports various multi CODECs such as MPEG2, MPEG4, MPEG7, H.264, and WMV-9 and selects various services.

In one embodiment, the voice agent processing unit 146 may perform a role in the voice agent area of the application layer and apply it by utilizing 20 kinds of software templates.

In an embodiment, the voice agent processing unit 146 includes a content reproduction processing module 461 for reproducing and processing content, a service reproduction processing module 462 for reproducing and processing services, and external IoT data in conjunction with an external IoT. It may include an external IoT processing module 463 for processing.

The audio processing unit 147 receives a user's voice through a microphone, processes the inputted user's voice as audio, and transmits the received user's voice to the artificial intelligence server 130 through the control unit 142 .

In one embodiment, the audio processing unit 147 recognizes the audio generated by the user device 140 itself, and when the phase is reversed, the echo cancellation module ( 471), a noise cancellation module 472 for maintaining a clearer and cleaner voice by removing ambient noise such as air conditioner sound introduced into the microphone, and a microphone voice input module 473 for receiving high-performance microphone audio ) may be included.

The echo cancellation module 471 recognizes the audio generated by the device itself and removes the audio of the internal sound source by using the principle of canceling the sound when the phase is reversed.

The noise cancellation module 472 performs a function of maintaining a clearer and clearer voice by removing ambient noise, such as an air conditioner sound, introduced into the microphone.

The microphone voice input module 473 receives high-performance micro audio.

The speaker output unit 148 receives the audio data received from the artificial intelligence server 130 through the control unit 142, and outputs the received audio data.

3 is a view for explaining a method of providing an artificial intelligence assistant service according to an embodiment of the present invention.

Referring to FIG. 3 , the content server 110 provides various types of content (particularly, multimedia content) to the artificial intelligence server 130 through the network 150 ( S301 ).

In providing the content in step S301 described above, the content server 110 receives the content name transmitted from the artificial intelligence server 130, and provides content information corresponding to the received content name (or the artificial intelligence server). After transmitting it to the user device 140 (via 130) so that the user can check it through the screen, the user device 140 (or via the artificial intelligence server 130) is notified of the content selection. The corresponding content may be searched for (or via the artificial intelligence server 130 ) and transmitted to the user device 140 .

In providing the content in step S301 described above, the content server 110 searches for content corresponding to the content selection notified from the user device 140, and displays a content file including the searched content on the screen. It can be transmitted to the user device 140 in real time so that it can be viewed through the

In providing the content in step S301 described above, the content server 110 may match data on various content with detailed item information for each content and transmit it to the user device 140, for example, music. In the case of content, it can be transmitted to the user device 140 by matching it with detailed item information such as the content name, singer, and playback time.

In providing the content in step S301 described above, in order to provide the content corresponding to the individual service platform, in the content server 110 , for a plurality of user IDs preset and registered for each user device 140 , each user ID By classifying the usage history of the content, the content information may be transmitted to the user device 140 .

While the content is provided in the above-described step S301, the service server 120 provides various services to the artificial intelligence server 130 through the network 150 (S302).

In providing the service in step S302 described above, the service server 120 matches the data for each service with detailed item information for each service (or via the artificial intelligence server 130 ) the user device 140 . For example, in the case of weather content, it may be matched with detailed item information such as weather information for each period according to region and transmitted to the user device 140 .

In providing the service in the above-described step S302 , in order to provide a service (eg, calendar, etc.) corresponding to the individual service platform, the service server 120 includes a plurality of preset registered for each user device 140 . With respect to the user ID, the service usage history may be classified for each user ID, and the service may be transmitted to the user device 140 .

When the service is provided in the above-described step S302, in the artificial intelligence server 130, various contents provided from the content server 110 through the network 150 and the service server 120 provided through the network 150 It manages various services by interworking with the artificial intelligence assistant function, and recognizes the user's voice transmitted from the user device 140 to allow the user to search for the content to be searched and to use the service (S303).

In providing content search and service use in the above-described step S303, the artificial intelligence server 130 distributes various types of content provided from the content server 110 and transmits the distributed content to the user device 140 . there is.

In providing content search and service use in step S303 described above, the artificial intelligence server 130 may process a user response signal input from the VUI.

In providing content search and service use in the above-described step S303, the artificial intelligence server 130 receives the user ID transmitted from the user device 140, and device information and individual service corresponding to the received user ID. After the platform is inquired from the previously registered database, the individual service platform corresponding to the received user ID may be activated by the user device 140 corresponding to the received user ID.

In providing the content search and service use in the above-described step S303, the artificial intelligence server 130 analyzes the user voice received from the user device 140 to extract the name of the content that the user wants to search, and then the extracted content name may be transmitted to the content server 110 .

As the content search and service use are provided in the above-described step S303, the user device 140 receives the user's voice and transmits the received user's voice to the artificial intelligence server 130 through the network 150, The content that the user wants to search is searched for through the artificial intelligence server 130 and the service is used (S304).

In performing the content search and service use in the above-described step S304, the user device 140, which is a device for using the content and service, receives the distributed content transmitted from the artificial intelligence server 130 and provides it to the user through the screen. In addition, the service transmitted from the artificial intelligence server 130 may be received and provided to the user through the screen.

In performing the content search and service use in the above-described step S304 , the user device 140 transmits the user ID to the artificial intelligence server 130 , and then the individual service platform provided by the artificial intelligence server 130 . can be activated.

In performing the content search and service use in the above-described step S304, the user device 140 may receive the content information transmitted from the content server 110 and provide it to the user through the screen, and provide the content to the user by the user. In the case of selection, the content selection may be notified to the content server 110 , and then, content content transmitted from the content server 110 may be received and provided to the user through a screen.

In performing the content search and service use in the above-described step S304, the user device 140 having a terrestrial TV uses the terrestrial TV to output the in-house airwave broadcasting content, and input means such as a remote control or screen touch It is possible to transmit the user response signal received through the VA to the artificial intelligence server 130 .

In performing content search and service use in step S304 described above, the user device 140 supports various multi CODECs such as MPEG2, MPEG4, MPEG7, H.264, and WMV-9 and provides various services. an application layer comprising a voice agent processing unit 146 (see FIG. 2) for selecting; Middleware layer loaded with Java Virtual Machine (JVM) and streaming protocols (RTP, RTSP); a system software layer including system software such as a device driver and an operating system; It may include four layers such as a hardware layer composed of hardware such as CPU, media processor, flash RAM, Ethernet module, and the like, and an IPv4 address or an IPv6 address may be assigned.

In performing the content search and service use in step S304 described above, the user device 140 including the digital broadcast receiving device for receiving various contents transmitted through the network 150 uses the corresponding digital broadcast receiving device. Thus, it is possible to receive not only image data but also digital information related to all programs and provide it to the user.

In performing the content search and service use in the above-described step S304, the user device 140 receives the information transmitted from the artificial intelligence server 130, and then displays the received information to the screen output unit 145 (Fig. 2), the output is displayed on the screen so that the user can easily search for desired digital information.

The method for providing an artificial intelligence assistant service according to an embodiment of the present invention having the above-described configuration may largely include a device authentication step, a content use step, and a service use step.

The first device authentication step is a step of authenticating the user device 140 . First, when the user inputs a user ID using the user device 140 , the user device 140 uses the user ID to generate an artificial intelligence server. (130) is connected. At this time, when the artificial intelligence server 130 receives the user ID, inquires about device information corresponding to the received user ID, and then, the received user ID is an ID that can be controlled by the artificial intelligence server 130 . It is determined whether it is a registered ID.

As a result of the above determination, if the ID is not a registered ID, the artificial intelligence server 130 transmits an unavailable message to the user device 140 .

On the other hand, as a result of the above determination, if the received ID is registered, the following process is performed, and the artificial intelligence server 130 inquires the individual service platform corresponding to the received ID, and then the received ID It activates the searched individual service platform corresponding to the

The second content use step is a step in which the user uses the content through the user device 140 . First, when the user inputs a voice into the user device 140 (ie, utters a specific vocabulary), the user device 140 ) activates the VA and converts the user's voice into a standby state that can accept it. Then, when the user utters the name of the content to be searched as a voice, the user device 140 transmits the name of the content uttered by voice to the artificial intelligence server 130 .

The artificial intelligence server 130, after recognizing the voice uttered by the user through the user device 140, analyzes the recognized user's voice to extract the content name, and sets the extracted content name to the content server (110) is forwarded.

The content server 110 transmits content information (eg, content name, singer, playback time, etc. information) corresponding to the content name received from the artificial intelligence server 130 to the user device 140 . .

As described above, when there are several analyzed content names, the artificial intelligence server 130 may transmit the corresponding content list to the user device 140 and the corresponding content list and related information.

The user device 140 displays the content information received from the content server 110 on the screen of the screen output unit 145 and then starts playing the content.

The third service use step is a step in which the user uses the service through the user device 140 . When the user inputs a voice (ie, utters a specific vocabulary) to the user device 140 , the user device 140 uses the VA. It is activated to convert the user's voice into an acceptable standby state. Then, when the user utters a service name to be searched as a voice, the user device 140 transmits the uttered service name to the artificial intelligence server 130 .

The artificial intelligence server 130, after recognizing the voice uttered by the user through the user device 140, analyzes the recognized user's voice to extract the service name, and uses the extracted service name as the service server. (120).

The service server 120 transmits service information corresponding to the service name received from the artificial intelligence server 130 (for example, information such as a local name for today's weather, hourly status, etc.) to the user device 140, will give

As described above, when there are several analyzed service names, the artificial intelligence server 130 may transmit the corresponding service list to the user device 140 and the corresponding service list and related information.

The user device 140 displays the service information received from the service server 120 on the screen of the screen output unit 145 and then starts playing the service.

The specific actual use state of the artificial intelligence assistant service providing system according to the present invention having such a configuration will be described with reference to FIG. 4 , the desk view 145a used in the living room and the kitchen using the wired communication company IoT platform. A kitchen TV 145b was used as the screen output unit 145 .

As shown in FIG. 5 , on the initial start screen of the desk view 145a, a number of icons including live TV are displayed, and an icon of a microphone display 451 is displayed at the lower right corner.

To explain the operation of the desk view 145a, as shown in FIG. 6a, when the user accesses the Naver ID and says 'Hey Clover', the screen of the desk view 145a displays 'Yes, please tell me. When a voice signal is uttered together with the subtitle ' and the user says 'Naver NOW, play it', the screen of 'Naver NOW' is played on the screen of the desk view 145a as shown in FIG. 6a. .

In addition, when explaining the operation of the kitchen TV 145b, when the user accesses the Naver ID and says 'Hey Clova', a voice signal is displayed along with the subtitle 'Yes, please tell me' on the screen of the kitchen TV 145b. When a fire is ignited and the user says 'find a salad recipe', as shown in FIG. 6b , a screen of 'multiple salad recipes' is played on the screen of the kitchen TV 145b, and a desired salad recipe is selected here. The salad is ready to be made.

Similarly, by using the screen output unit 145 such as the desk view 145a or the kitchen TV 145b, as shown in FIG. 6c , various contents such as recommended restaurants and weather information can be conveniently used. will become

And, as shown in Fig. 6d, after connecting with Naver ID, the user says 'Hey Clova', and then changing the TV channel such as 'Play MBC' or changing the volume such as 'Volume up/down' , 'Run the radio', and specific radio AM or FM selections can be performed with voice, and 'Doorwalker' and 'Security room call' can also be performed with voice.

Above, the embodiment of the present invention is not implemented only through the above-described apparatus and/or operation method, but through a program for realizing a function corresponding to the configuration of the embodiment of the present invention, a recording medium in which the program is recorded, etc. It may be implemented, and such an implementation can be easily implemented by an expert in the technical field to which the present invention pertains from the description of the above-described embodiments. Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improved forms of the present invention are also provided by those skilled in the art using the basic concept of the present invention as defined in the following claims. is within the scope of the right.

100: artificial intelligence assistant service provision system
110: content server
120: service server
130: artificial intelligence server
140: user device
141: external server linkage
142: control unit
143: storage
144: video signal processing unit
145: screen output unit
146: voice agent processing unit
461: content reproduction processing module
462: service regeneration processing module
463: external IoT processing module
147: audio processing unit
471: echo cancellation module
472: noise cancellation module
473: microphone voice input module
148: speaker output unit
150: network

Claims (6)

In the artificial intelligence assistant service providing system 100,
The artificial intelligence assistant service providing system 100,
a content server 110 for providing content;
a service server 120 for providing a service;
Artificial intelligence server 130 for managing the content provided by the content server and the service provided by the service server by interworking with the artificial intelligence assistant function, recognizing the user's voice to search for content and use the service ;
a user device 140 for receiving a user's voice and transmitting it to the artificial intelligence server, searching for a content that the user wants to search for and using a service; and
and a network (150) for connecting the content server, the service server, the artificial intelligence server, and the user devices to each other to transmit and receive data.
According to claim 1,
The content server 110,
Transmitting content information corresponding to the content name received from the artificial intelligence server 130 to the artificial intelligence server 130,
According to the content selection notified from the user device 140, the corresponding content is retrieved and transmitted to the user device 140,
The artificial intelligence assistant service providing system, characterized in that the content file including the searched content is transmitted to the user device (140) in real time so that the user can view it through the screen.
According to claim 1,
The user device 140 may receive the distributed content transmitted from the artificial intelligence server 130 and provide it to the user through the screen, and receive the service transmitted from the artificial intelligence server 130 through the screen can be provided to users,
The user device 140 includes an external server interworking unit 141 for transmitting and receiving data in conjunction with the artificial intelligence server 130 or the content server 110 or the service server 120;
a control unit 142 connected to the external server interworking unit 141 and controlling the operation of the user device 140; a storage unit 143 connected to the control unit 142 to store programs or data necessary for the control of the control unit;
an image signal processing unit 144 connected to the control unit 142 and performing signal processing on the image data received from the artificial intelligence server 130;
a screen output unit 145 for outputting and displaying the image data signal processed by the image signal processing unit 144 through a screen;
a voice agent processing unit 146 connected to the control unit 142, supporting multiple codecs, and selecting a service;
an audio processing unit 147 connected to the control unit 142, receiving a user's voice through a microphone, processing the audio, and transmitting the audio to the artificial intelligence server 130;
and a speaker output unit (148) connected to the control unit (142) to output audio data received from the artificial intelligence server (130).
4. The method of claim 3,
The voice agent processing unit 146 includes a content reproduction processing module 461 for reproducing and processing content, a service reproduction processing module 462 for reproducing and processing services, and interworking with external IoT to process external IoT data. Including an external IoT processing module 463,
The audio processing unit 147 recognizes the audio generated by the user device 140 itself, and when the phase is reversed, the sound is canceled by an echo cancellation module 471 for removing the audio of the internal sound source, a microphone Including a noise cancellation module 472 for maintaining a clearer and cleaner voice by removing ambient noise such as the sound of the air conditioner introduced into the air conditioner, and a microphone voice input module 473 for receiving high-performance micro audio Artificial intelligence assistant service providing system, characterized in that.
According to claim 1,
The artificial intelligence server 130 may process a user response signal input from a voice user interface (VUI),
The artificial intelligence server 130 is provided with an artificial intelligence assistant platform,
In the AI assistant platform, the client can obtain an AI assistant access token or use it when a user connects his or her account when using a specific extension, and the AI assistant developer console (developer console) An artificial intelligence assistant service providing system, characterized in that the client authentication information, which is authentication information obtained by registering a client through the console), can be used to acquire an artificial intelligence assistant access token.
In the method of providing an artificial intelligence assistant service,
The method of providing the artificial intelligence assistant service,
providing content by a content server;
providing a service by the service server;
managing, by an artificial intelligence server, the content provided by the content server and the service provided by the service server by interworking with an artificial intelligence assistant function;
receiving, by the user device, the user's voice and transmitting it to the artificial intelligence server;
allowing the artificial intelligence server to search for content by recognizing a user's voice and use a service; and
and providing, by the user device, searching for the content the user wants to search for and using the service.

Images