KR20200024427A - Ai assitant voice matching system and operation method thereof - Google Patents

Abstract

The present invention relates to an AI assistant voice matching method through a content purchase (play) history. The AI assistant voice matching method which automatically generates an AI assistant voice comprises the steps of: collecting content information consumed by a user for a certain period of time; detecting a predetermined number of pieces of candidate content based on the collected content information; selecting target content based on user preference scores of the candidate content; and analyzing sound sources included in the selected target content to extract a representative voice and determining the representative voice as a user preference voice.

Description

AI assistant voice matching system and its operation method {AI ASSITANT VOICE MATCHING SYSTEM AND OPERATION METHOD THEREOF}

The present invention relates to an AI assistant voice matching system, and more particularly, to an AI assistant voice matching system capable of generating and providing a user-customized AI assistant voice based on information on a content purchase (playback) history. It relates to the operation method.

Recently, with the rapid development of information and communication technology, interest and demand for the Internet of Things (IoT) technology are rapidly increasing. Such IoT can be defined in various ways, depending on the point of view of it. In essence, however, IoT is an intelligent information communication technology or service that enables communication between people and things, and things and things by connecting various things to a communication network based on the Internet.

The IoT technology is applied in various technical fields such as smart home, smart health, smart car, and the like. In particular, research on smart home services incorporating IoT technology into home network systems has been actively conducted.

Smart home service refers to a system that seeks to improve living standards through IoT devices with IoT functions in a residential environment with a communication network. Connects home appliances such as TVs, refrigerators, and air-conditioners, energy consumers such as electricity and water, and security services to a communication network, allowing the user to remotely monitor the situation information in the home via a smartphone or voice controller (or AI speaker). Real-time check and control can be enabled. In particular, smart home services that can remotely control IoT devices in the home through AI speakers are increasing.

The AI speaker can provide an interactive AI assistant service using voice recognition technology and artificial intelligence technology. Here, the interactive AI assistant service refers to a service in which artificial intelligence serves as a speaker's secretary. These AI speakers have various services such as personal schedule management service, SNS management service, app execution service, internet shopping service, email management service, messenger management service, multimedia playback service, weather / traffic / travel information service, IoT device control service, etc. Can be provided.

However, since the voice (voice) output from the conventional AI speaker is a mechanical voice provided unilaterally by the service providing server, there is a problem that the intimacy between the user and the AI assistant is difficult to increase. In addition, in the service providing server, it is troublesome to define, produce, and modify all voices that can be output from the AI speaker. In addition, the service providing server should provide various AI secretary voices according to user preferences, but since there are too many kinds of voices that are subject to user preferences, there is a practical limitation in that these voices cannot be processed and provided. . Therefore, there is a need for a method for easily outputting a user's preferred voice through the AI speaker.

The present invention aims to solve the above and other problems. Another purpose is to extract the user's preferred voice included in the specific content based on the information on the content purchase (playback) history, and AI assistant voice that can automatically generate an AI assistant voice based on the characteristics of the user's preferred voice To provide a matching system and its operation method.

Still another object is an AI assistant voice matching system capable of automatically generating a customized AI assistant voice based on content information consumed by a user for a certain period of time, and providing the user customized AI assistant voice to the corresponding user, and a method of operating the same. In providing.

According to an aspect of the present invention to achieve the above or another object, collecting the content information consumed by the user for a certain period of time; Detecting a predetermined number of candidate contents based on the collected content information; Selecting target content based on a user preference score of the candidate contents; And extracting a representative voice by analyzing sound sources included in the selected target content, and determining the representative voice as a user's preferred voice.

More preferably, the detecting step may include calculating a cumulative playing time of contents consumed by a user for a predetermined period of time and detecting candidate contents based on the cumulative playing time. The detecting may include calculating a cumulative number of reproductions of contents consumed by a user for a predetermined period of time and detecting candidate contents based on the cumulative number of reproductions.

More preferably, the selecting step may further include classifying each of the candidate contents into any one of predetermined content types. The selecting may further include calculating a user preference score of candidate contents using weight information corresponding to the classified content type. In the selecting step, the candidate content having the highest user preference score is selected as the target content.

More preferably, the determining may include distinguishing one or more voices included in target content and extracting the representative voice based on a cumulative playing time for each voice.

More preferably, the AI assistant voice matching method may further include extracting feature values of a user preferred voice and processing the AI assistant voice using the extracted feature values. The AI assistant voice matching method may further include transmitting the processed AI assistant voice to an AI speaker.

According to another aspect of the invention, the process of collecting the content information consumed by the user for a certain period of time; Detecting a predetermined number of candidate contents based on the collected content information; Selecting target content based on a user preference score of the candidate contents; And extracting a representative voice by analyzing sound sources included in the selected target content, and determining the representative voice as a user's preferred voice on a computer.

According to another aspect of the invention, the content purchase information collection unit for collecting the content information purchased by the user for a certain period; Detects a predetermined number of candidate contents based on the content information acquired through the content purchase information collecting unit, selects target content based on a user preference score of the candidate contents, and analyzes a sound source included in the selected target content A preferred speech extractor for extracting a representative speech; And an AI assistant voice processing unit extracting feature values of the representative voice detected through the preferred voice extractor and processing the AI assistant voice using the feature values.

The effects of the AI assistant voice matching system and its operation method according to embodiments of the present invention will be described below.

According to at least one of the embodiments of the present invention, by providing the user with a customized AI assistant voice (voice) generated based on the content purchase (playback) history, the user side can listen to the AI assistant voice that matches his or her taste. In addition, in terms of service provider, there is an advantage in that no extra cost and time are incurred in producing the AI assistant voice.

According to at least one of the embodiments of the present invention, by providing the user with a customized AI assistant voice (voice) generated based on the content information consumed by the user for a certain period of time, the user's preference for the AI assistant can be increased. It also has the advantage of improving the intimacy between the user and AI assistant.

However, the effects that can be achieved by the AI assistant voice matching system and its operation method according to the embodiments of the present invention are not limited to those mentioned above, and other effects not mentioned are included in the following description. It will be clearly understood by those skilled in the art.

1 is an overall configuration diagram of an AI assistant voice matching system according to an embodiment of the present invention;
2 is a block diagram of an AI speaker according to an embodiment of the present invention;
3 is a block diagram of an AI assistant voice matching server according to an embodiment of the present invention;
4 is a flowchart illustrating an AI assistant voice matching method according to an embodiment of the present invention;
5 is a view for explaining an example of the AI assistant voice automatic matching method through the content purchase history.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings, and the same or similar components will be given the same reference numerals regardless of the reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or mixed in consideration of ease of specification, and do not have distinct meanings or roles. In other words, the term 'part' used in the present invention refers to a hardware component such as software, FPGA or ASIC, and 'part' plays a role. But wealth is not limited to software or hardware. The 'unit' may be configured to be in an addressable storage medium and may be configured to play one or more processors. Thus, as an example, a 'part' may include components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, Subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided within the components and 'parts' may be combined into a smaller number of components and 'parts' or further separated into additional components and 'parts'.

In addition, in the following description of the embodiments disclosed herein, when it is determined that the detailed description of the related known technology may obscure the gist of the embodiments disclosed herein, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easily understanding the embodiments disclosed in the present specification, the technical idea disclosed in the specification by the accompanying drawings are not limited, and all changes included in the spirit and scope of the present invention. It should be understood to include equivalents and substitutes.

The present invention extracts a user's preferred voice included in a specific content based on information on a content purchase (playback) history and automatically generates an AI assistant voice based on a feature of the user's preferred voice. We propose a system and an operation method thereof. The present invention also provides an AI assistant voice matching system capable of automatically generating a customized AI assistant voice based on content information consumed by a user for a certain period of time, and providing the user customized AI assistant voice to the user. Suggest a method.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is an overall configuration diagram of an AI assistant voice matching system according to an embodiment of the present invention.

Referring to FIG. 1, the AI assistant voice matching system 100 according to an embodiment of the present invention includes a router 110, an AI speaker 120, and a plurality of IoT devices 130-1 to 130 located in a home. -N), an internet network 140 located outside the home, and an AI assistant voice providing server 150 and the like. The AI assistant voice providing server 150 may be built in an operation server of a service provider or a platform provider.

The router (Access Point, 110) is a device that connects communication to connect the AI speaker 120 and a plurality of IoT devices (130-1 ~ 130-N) located in the home to the Internet network 140. The router 110 may perform a function of sharing an IP address provided by an internet service provider so that a plurality of terminals existing in the home can share each other.

The AI speaker (or voice controller) 120 may be connected to the router 110 through a wired / wireless communication interface, and may communicate with the AI assistant voice providing server 150 located outside the home through the router 110. Can be.

The AI speaker 120 is directly connected to a plurality of IoT devices 130-1 to 130 -N through a wired / wireless communication interface to communicate with the IoT devices 130-1 to 130 -N. can do. Meanwhile, although not shown in the drawing, in another embodiment, the AI speaker 120 may be connected to a home hub (not shown) through a wired / wireless communication interface, and a plurality of IoT devices may be provided through the home hub. It may also perform communication with (130-1 ~ 130-N).

The AI speaker 120 may provide an AI assistant service corresponding to the voice command of the user. For example, the AI speaker 120 checks the state of the plurality of IoT devices 130-1 to 130 -N existing in the home in response to a voice command of the user, or corresponding devices 130-1 to 130-. The operation of N) can be controlled. In addition, the AI speaker 120 may provide an AI assistant voice service based on information received from the AI assistant voice providing server 150.

The plurality of IoT devices 130-1 to 130 -N are IoT devices equipped with an Internet of Things (IoT) function, and are a smart appliance, a security device, and a lighting device. devices) and energy devices. The plurality of IoT devices 130-1 to 130 -N may exchange data with an AI speaker 120, a home hub, a set-top box, and the like.

The AI assistant voice providing server 150 may be connected to the router 110 existing in the home through the Internet network 140, and may communicate with the AI speaker 120 through the router 110.

The AI assistant voice providing server 150 may automatically generate a customized AI assistant voice (voice) based on content information consumed by the user for a predetermined period of time. The customized AI assistant voice generated as described above may be provided to the user through the AI speaker 120.

2 is a block diagram illustrating an AI speaker according to an embodiment of the present invention.

Referring to FIG. 2, the AI speaker 200 according to an exemplary embodiment may include a communication unit 210, an input unit 220, an output unit 230, a memory 240, and a controller 250. . The components shown in FIG. 2 are not essential to implementing the AI speaker 200, so the AI speaker described herein may have more or fewer components than those listed above.

The communication unit 210 may include a wired communication module for supporting wired communication and a short-range communication module for supporting short-range wireless communication. The wired communication module is a wired communication network constructed according to technical standards or communication methods (for example, Ethernet, Power Line Communication, Home PNA, IEEE 1394, etc.) for wired communication. Send and receive wired signals with at least one of a communication modem, a router, and an IoT device. The short range communication module is for short range communication, and includes Bluetooth, Bluetooth (RF), Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra-Wideband (UWB), ZigBee, NFC ( Near field communication (WLAN), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, or Wireless USB (Wireless Universal Serial Bus) technology may be used to support short-range wireless communication.

The input unit 220 may include a microphone for inputting an audio signal. The microphone processes external sound signals into electrical voice data. The processed voice data may be variously used according to a function (or an application program being executed) performed by the AI speaker 200. Meanwhile, various noise reduction algorithms may be implemented in the microphone to remove noise generated in the process of receiving an external sound signal.

The output unit 230 is used to generate an output related to visual, auditory, or tactile, and may include at least one of a display unit, an audio output unit, a hap tip module, and an optical output unit. The sound output unit may output audio data stored in the memory 240. The sound output unit may output a sound signal related to a function performed by the AI speaker 200. The sound output unit may include a speaker and a buzzer.

The memory 240 stores data supporting various functions of the AI speaker 200. The memory 240 may store an application program driven by the AI speaker 200, data for operating the AI speaker 200, and instructions.

The controller 250 controls an operation related to an application program stored in the memory 240 and generally the overall operation of the AI speaker 200. Furthermore, in order to implement various embodiments described below on the AI speaker 200 according to the present invention, the controller 250 may control a combination of at least one of the above-described components.

The controller 250 may further include a voice recognition module 255. The voice recognition module 255 drives the voice recognition engine to which the voice recognition algorithm is applied to recognize the external voice input through the microphone. That is, the speech recognition module 255 converts the external voice input through the microphone into digital data, amplifies the pre-emphasis, and detects the start point and the end point of the digitally converted voice. do. Subsequently, the speech recognition module 255 extracts speech feature values for the speech between the detected starting point and the ending point to recognize the unique speech or timbre.

Meanwhile, in the present exemplary embodiment, the voice recognition module 255 is implemented in the controller 250, but the present invention is not limited thereto and may be configured independently of the controller 250. Furthermore, the voice recognition module may be installed on an external platform interworking with the AI speaker 200.

3 is a block diagram of an AI assistant voice providing server according to an embodiment of the present invention.

Referring to FIG. 3, the AI assistant voice providing server 300 according to an embodiment of the present invention may include a communication unit 310, a content purchase information collection unit 320, a preferred voice extraction unit 330, and an AI assistant voice processing unit. 340, a database 350, and a controller 360. The components shown in FIG. 3 are not essential to implementing the AI assistant voice providing server 300, so that the AI assistant voice providing server described herein is more or less than the components listed above. You can have them.

The communication unit 310 may include a wired communication module for supporting wired communication and a wireless communication module for supporting wireless communication. The wired communication module is a wired communication network constructed according to technical standards or communication methods (for example, Ethernet, Power Line Communication, Home PNA, IEEE 1394, etc.) for wired communication. Sends and receives wired signals with at least one of a communication modem, a router, and another server.

The wireless communication module may include technical standards or communication schemes for wireless communication (for example, Global System for Mobile Communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (CDMA2000), and EV-DO). Enhanced Voice-Data Optimized or Enhanced Voice-Data Only (WCDMA), Wideband CDMA (WCDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), Long Term Evolution-Advanced) and transmits and receives a radio signal with at least one of the base station and the external terminal on the wireless communication network established.

The content purchase information collecting unit 320 may collect information about contents purchased by the user for a predetermined period of time. In this case, the content may include video content including a TV program, a movie, a video on demand (VOD), YouTube, and the like, and audio content including a radio, a podcast, music, and the like.

The content purchase information collecting unit 320 may recognize the speaker by voice recognition of the voices of the users input through the AI speaker, and collect information on the purchased contents for each identified speaker.

The preferred voice extractor 330 may extract a preference voice of the corresponding users based on content information purchased by the users for a predetermined period of time.

The preferred speech extractor 330 calculates at least one of a cumulative playing time and a cumulative playing count of the purchased contents, and based on the calculated value, a predetermined number of cumulative playing upper contents (hereinafter, for convenience of description, 'higher'). Content "or" candidate content ".

The preferred voice extractor 330 may classify the upper contents by types and calculate a user preference score of the upper contents based on weight information for each type of content.

The preferred voice extractor 330 may detect an optimum content based on a user preference score, and determine the representative voice extracted from the optimum content as the user preferred voice.

The AI assistant voice processing unit 340 may extract feature values (or feature vectors) of the user's preferred voice and process the AI assistant voice (voice) based on the extracted feature values. The AI assistant voice thus processed may be provided to the user through the AI speaker.

The database (or memory) 350 stores data supporting various functions of the AI assistant voice providing server 300. The database 350 may store a plurality of application programs or applications that are run on the AI assistant voice providing server 300, data for operating the AI assistant voice providing server 300, and instructions. .

The database 350 may include a content database DB for storing contents purchased by the user and a voice database DB for storing voice data extracted from the purchase contents.

The controller 360 controls an operation related to an application program stored in the database 350 and generally the overall operation of the AI assistant voice providing server 300. Furthermore, in order to implement various embodiments described below on the AI assistant voice providing server 300 according to the present invention, the controller 360 may control a combination of at least one of the above-described components.

The controller 360 may automatically generate a customized AI assistant voice based on content information purchased by a user for a predetermined period of time, and control an overall operation of transmitting the customized AI assistant voice to the AI speaker 200. .

Meanwhile, in the present exemplary embodiment, the AI assistant voice providing server 300 includes the content purchase information collecting unit 320, but the present invention is not necessarily limited thereto. It may also include an information collector. The content reproduction information collecting unit may collect information about contents consumed (played) by a user for a predetermined period of time.

As described above, the AI assistant voice providing server according to the present invention provides the user with a customized AI assistant voice generated based on content information consumed (played) by the user for a predetermined period of time. You can listen to your favorite voice and improve the intimacy between you and the AI assistant.

4 is a flowchart illustrating an AI assistant voice matching method according to an embodiment of the present invention.

Referring to FIG. 4, the AI assistant voice providing server 300 may collect content purchase information of users for a predetermined period of time (S410). The AI assistant voice providing server 300 may collect content purchase information in conjunction with an AI speaker 200, a set-top box, a service provider, a platform provider, and the like. The content purchase information may include content identification information, purchase date information, purchase time information, purchase cost information, purchase place information, and the like.

The AI assistant voice providing server 300 may predetermine (or set) a type of purchased contents and a weight for each type of content and store the corresponding information in a database (S420).

For example, as shown in Table 1 below, the types of purchased contents may include a first content type (education / document / presentation / culture / animation), a second content type (movie / drama / theater), and a third content type (entertainment). / Entertainment / sports), a fourth content type (concert, performance, art, music), and a fifth content type (other). In addition, the weight for each content type may be set to '100' for the first content type, '70' for the second content type, and to '50' for the third content type. The fourth content type may be set to '20' and the fifth content type may be set to '10'. That is, high weights can be assigned to content types suitable for AI voice (voice).

contents Content type soundtrack weight A First content type
(Education, documentary, current affairs, culture, annie) Narration / Voice Voice 100 B Second content type
(Movie, drama, theater) Actor voice 70 C Third content type
(Entertainment, entertainment, sports) MC voice 50 D 4th content type
(Concerts, performances, art, music) Song voice 20 E Fifth content type
(Other) US Classifieds 10

The AI assistant voice providing server 300 may calculate the cumulative playing time of the contents purchased by the user for a predetermined period of time (S430). The AI assistant voice providing server 300 may sequentially arrange the purchased contents according to the longest length of the cumulative playing time and then detect a predetermined number of higher contents (S440).

The AI assistant voice providing server 300 may classify each of the detected upper content contents into any one of predetermined content types (S450). This is to apply different weights according to the type of content.

The AI assistant voice providing server 300 may calculate a user preference score of the higher contents using the preset weight information for each content type (S460). For example, the AI assistant voice providing server 300 may calculate a user preference score K using Equation 1 below.

Meanwhile, as another example, the AI assistant voice providing server 300 may calculate a user preference score K using Equation 2 below. In addition, it will be apparent to those skilled in the art that the user preference score K may be calculated using various other methods.

,

는 재생 횟수에 관한 가중치,

는 누적재생시간에 관한 가중치임.here,

,

Is the weight for playback count,

Is the weight for cumulative playing time.

The AI assistant voice providing server 300 may detect the optimal content based on the user preference score (S470). That is, the AI assistant voice providing server 300 may determine the content having the highest user preference score among the higher contents as the optimal content (or target content).

The AI assistant voice providing server 300 may detect the representative voice by analyzing sound source information included in the optimum content (S480). For example, the AI assistant voice providing server 300 may classify voices (voices) played in the corresponding content and extract the representative voice based on the cumulative playing time for each voice. The AI assistant voice providing server 300 may determine the representative voice extracted from the optimal content as the user's preferred voice.

The AI assistant voice providing server 300 may detect feature values (or feature vectors) of the user's preferred voice and process the AI assistant voice (voice) based on the detected feature values (S490). The AI assistant voice thus processed may be output to the user through the AI speaker.

As described above, the AI assistant voice matching method according to the present invention provides the user with a customized AI assistant voice generated based on content information consumed (played) by the user for a predetermined period of time. You can listen to your favorite voice and improve the intimacy between you and the AI assistant.

5 is a view for explaining an example of the AI assistant voice automatic matching method through the content purchase (playback) history. As shown in FIG. 5A, the AI assistant voice providing server 300 may calculate a cumulative playing time of contents purchased by a user for a predetermined period of time. The AI assistant voice providing server 300 may detect the first to fifth contents (radio star, sister who buys rice well, I live alone. One piece, BTS) based on the upper cumulative playing time.

Next, as illustrated in FIG. 5B, the AI assistant voice providing server 300 may classify the first to fifth contents into one of predetermined content types. For example, the AI assistant voice providing server 300 sets the first content to 'content type C', the second content to 'content type B', the third content to 'content type C', and the fourth content to 'content type'. A 'and the fifth content may be classified as' content type D'.

Next, as illustrated in FIG. 5C, the AI assistant voice providing server 300 may calculate a user preference score for each of the first to fifth contents. In this case, the AI assistant voice providing server 300 may calculate a user preference score based on the weight information for each content and the number of times of playing the content.

Subsequently, as shown in FIG. 5D, the AI assistant voice providing server 300 may detect the second content having the highest user preference score (eg, a sister who buys rice well). The AI assistant voice providing server 300 may extract a representative voice (eg, Hong Gil-dong voice) by analyzing sound sources included in the second content, and determine the extracted representative voice as the user's preferred voice.

The present invention described above can be embodied as computer readable codes on a medium in which a program is recorded. The computer readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include hard disk drives (HDDs), solid state disks (SSDs), silicon disk drives (SDDs), ROMs, RAM, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, and the like. There is this. In addition, the computer may include a control unit of the terminal. Accordingly, the above detailed description should not be construed as limiting in all aspects and should be considered as illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

100: AI assistant voice matching system 110: router
120/200: AI speaker 130: IoT device
140: Internet network 150/300: AI secretary voice providing server
310: communication unit 320: content purchase information collection unit
330: Preferred voice extraction unit 340: AI assistant voice processing unit
350: database 360: control unit

Claims (11)

Collecting content information consumed by a user for a certain period of time;
Detecting a predetermined number of candidate contents based on the collected content information;
Selecting target contents based on a user preference score of the candidate contents; And
Analyzing the sound sources included in the selected target content to extract a representative voice, and determining the representative voice as a user's preferred voice. The method of claim 1, wherein the detecting step,
And calculating the cumulative playing time of the contents consumed by the user for a predetermined period of time, and detecting the candidate contents based on the cumulative playing time. The method of claim 1, wherein the detecting step,
And calculating the cumulative reproduction count of the contents consumed by the user for a predetermined period of time and detecting the candidate contents based on the cumulative reproduction count. The method of claim 1, wherein the selecting step,
And classifying each of the candidate contents into any one of predetermined content types. The method of claim 4, wherein the selecting step,
And calculating a user preference score of the candidate contents by using weight information corresponding to the classified content type. The method of claim 5, wherein the selecting step,
And selecting candidate content having the highest user preference score as the target content. The method of claim 1, wherein the determining step,
And classifying one or more voices included in the target content and extracting the representative voice based on a cumulative playing time for each voice. The method of claim 1,
Extracting feature values of the user preferred voice and processing the AI assistant voice using the extracted feature values. The method of claim 8,
And transmitting the processed AI assistant voice to an AI speaker. A program stored on a computer readable recording medium for executing the method according to any one of claims 1 to 9 on a computer. A content purchase information collection unit for collecting content information purchased by the user for a predetermined period of time;
Detects a predetermined number of candidate contents based on the content information acquired through the content purchase information collecting unit, selects target contents based on a user preference score of the candidate contents, and includes the selected target contents in the selected target contents. A preferred voice extractor configured to extract the representative voice by analyzing the sound source; And
And an AI assistant voice processing unit for extracting feature values of the representative voice detected through the preferred voice extractor and processing the AI assistant voice using the feature values.

Images