KR20220045327A - Apparatus, method and computer program for providing interaction service based on voice recognition - Google Patents

Abstract

A speech recognition device that provides an interaction service based on a speech recognition comprises: an input part that receives the speech data; a speaker recognition part that recognizes a speaker based on the received speech data; an intention extraction part that recognizes an interaction target and an interaction command from the speech data, and extracts a speaker-customized continuous interaction intention based on the recognized interaction target and the interaction command; and a service providing part that provides a continuous interaction service based on the extracted continuous interaction intention.

Description

Speech recognition device, method and computer program for providing speech recognition-based interaction service

The present invention relates to a voice recognition apparatus, method, and computer program for providing a voice recognition-based interaction service.

Users are accustomed to continuous interaction services using an existing touch-based interface in the case of moving a playback section in media content, manipulating scrolling within a screen, adjusting volume, and the like.

In addition, the conventional voice recognition-based service has a problem in that it can only recognize the user's utterance and output a one-shot response according to it, and cannot provide an intuitive continuous interaction service to the user. Accordingly, fatigue caused by the user having to utter a command several times and inconvenience in being unable to perform precise manipulations are caused. In addition, there were cases in which functions that are possible in a touch-based interface such as drag and drop and swipe could not be performed.

Korean Patent No. 1197010 (Registered on October 26, 2012)

The present invention is to solve the problems of the prior art described above, receiving voice data, recognizing a speaker based on the received voice data, recognizing an interaction target and an interaction command from the voice data, and recognizing an interaction target and To extract a speaker-customized continuous interaction intention based on an interaction command, and to provide a continuous interaction service based on the extracted continuous interaction intention.

An object of the present invention is to provide a method for intuitively providing a continuous interaction service based on the speaker's utterance habit.

However, the technical problems to be achieved by the present embodiment are not limited to the technical problems described above, and other technical problems may exist.

As a means for achieving the above-described technical problem, an embodiment of the present invention provides a voice recognition apparatus for providing a voice recognition-based interaction service, an input unit for receiving voice data, and a speaker based on the received voice data A speaker recognition unit that recognizes, an intention extractor that recognizes an interaction target and an interaction command from the voice data, and extracts a speaker-customized continuous interaction intention based on the recognized interaction target and interaction command, and the extracted continuous interaction intent It may include a service provider that provides a continuous interaction service.

In an embodiment, the intention extractor may extract the continuous interaction intention further based on the recognized speaker's speech pattern model.

In an embodiment, the intention extractor may extract the continuous interaction intention based on whether a duration of the last syllable of the interaction command exceeds a trigger value based on the speech pattern model.

In an embodiment, a control information derivation unit for deriving continuous interaction control information for controlling the operation of the continuous interaction service from the voice data may be further included.

In an embodiment, the control information deriving unit may derive the continuous interaction control information based on at least one of a pitch change amount and a volume change amount of the voice data.

In an embodiment, the control information derivation unit may further derive whether there is an intention to control the operation speed of the continuous interaction service in the voice data based on the recognized speaker's speech pattern model.

In an embodiment, the control information derivation unit increases the operation speed of the continuous interaction service when the pitch change amount has a positive value, and increases the operation speed of the continuous interaction service when the pitch change amount has a negative value It is possible to derive the continuous interaction control information to reduce .

In an embodiment, the control information derivation unit increases the operation speed of the continuous interaction service when the volume change amount has a positive value, and increases the operation speed of the continuous interaction service when the volume change amount has a negative value. It is possible to derive the continuous interaction control information to reduce .

Another embodiment of the present invention provides a voice recognition method for providing a voice recognition-based interaction service, comprising: receiving voice data; recognizing a speaker based on the received voice data; an interaction target from the voice data; Recognizing an interaction command, extracting a speaker-customized continuous interaction intention based on the recognized interaction target and interaction command, and providing a continuous interaction service based on the extracted continuous interaction intention.

Another embodiment of the present invention provides a computer program stored in a computer readable recording medium including a sequence of instructions for providing a voice recognition-based interaction service, wherein the computer program inputs voice data when the computer program is executed by a computing device receiving, recognizing a speaker based on the received voice data, recognizing an interaction target and an interaction command from the voice data, extracting a speaker-customized continuous interaction intention based on the recognized interaction target and interaction command, and extracting the extracted and a sequence of instructions to provide a continuous interaction service based on the continuous interaction intent.

The above-described problem solving means are merely exemplary, and should not be construed as limiting the present invention. In addition to the exemplary embodiments described above, there may be additional embodiments described in the drawings and detailed description.

According to any one of the above-described problem solving means of the present invention, voice data is received, a speaker is recognized based on the received voice data, an interaction target and an interaction command are recognized from the voice data, and the recognized interaction target and interaction A speaker-customized continuous interaction intention may be extracted based on the command, and a continuous interaction service may be provided based on the extracted continuous interaction intention.

In addition, it is possible to intuitively provide a continuous interaction service based on the speaker's utterance habit.

1 is a block diagram of a voice recognition apparatus according to an embodiment of the present invention.
2 is an exemplary diagram for explaining a continuous interaction service according to an embodiment of the present invention.
3 is an exemplary diagram for explaining a method of extracting a continuous interaction intention according to an embodiment of the present invention.
4A and 4B are exemplary diagrams for explaining a method of deriving continuous interaction control information according to an embodiment of the present invention.
5 is an exemplary diagram for explaining a method of deriving continuous interaction control information based on a volume change amount of an interaction command.
6 is a flowchart of a voice recognition method according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement them. However, the present invention may be embodied in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is "connected" with another part, this includes not only the case of being "directly connected" but also the case of being "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated, and one or more other features However, it is to be understood that the existence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded in advance.

In this specification, a "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. In addition, one unit may be implemented using two or more hardware, and two or more units may be implemented by one hardware. Meanwhile, '~ unit' is not limited to software or hardware, and '~ unit' may be configured to be in an addressable storage medium or may be configured to reproduce one or more processors. Accordingly, as an example, '~' indicates components such as software components, object-oriented software components, class components, and task components, and processes, functions, properties, and procedures. , subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. The functions provided in the components and '~ units' may be combined into a smaller number of components and '~ units' or further separated into additional components and '~ units'. In addition, components and '~ units' may be implemented to play one or more CPUs in a device or secure multimedia card.

The "network" referred to below means a connection structure capable of exchanging information between each node, such as terminals and servers, and includes a local area network (LAN), a wide area network (WAN). , the Internet (WWW: World Wide Web), wired and wireless data networks, telephone networks, wired and wireless television networks, and the like. Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi, Bluetooth communication, infrared communication, ultrasound Communication, Visible Light Communication (VLC), LiFi, and the like are included, but are not limited thereto.

Some of the operations or functions described as being performed by the terminal or device in this specification may be instead performed by a server connected to the terminal or device. Similarly, some of the operations or functions described as being performed by the server may also be performed in a terminal or device connected to the server.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a voice recognition apparatus according to an embodiment of the present invention. Referring to FIG. 1 , the voice recognition apparatus 100 may include an input unit 110 , a speaker recognizer 120 , an intention extractor 130 , a control information derivation unit 140 , and a service provider 150 . there is.

The voice recognition apparatus 100 may provide a voice recognition-based interaction service. Here, the voice recognition-based interaction service refers to a service that provides a function that allows a user to control interfaces of various devices through voice utterance.

The voice recognition apparatus 100 according to the present invention may provide a continuous interaction service. Here, the continuous interaction service refers to a service that provides a function to continuously control the degree of operation of a specific function, rather than simply turning on/off a specific function according to a user's command or outputting a one-shot response.

2 exemplarily illustrates a case to which the continuous interaction service according to the present invention can be applied. Referring to FIG. 2A , in order to select one or more items included in the list, a continuous interaction service may be applied when moving in units of a plurality of items.

Referring to (b) of FIG. 2 , a continuous interaction service may be applied in order to control to what extent a playback section is moved when a playback section is moved during video playback. Also, referring to FIG. 2C , a continuous interaction service may be applied even when an icon is dragged/dropped.

In addition, the continuous interaction service may be applied to the case of manipulating scrolling within the screen, adjusting the volume, and the like.

Referring back to FIG. 1 , the input unit 110 may receive voice data. The input unit 110 may receive voice data from, for example, a user terminal, an external server, a cloud, or an external device. The received voice data may include, for example, a call word and a command. The call word is for initiating the service of the voice recognition apparatus 100 , and after recognizing a preset call word, the voice recognition apparatus 100 may perform an operation according to the command after the call word.

For example, when a call word such as "Giga Genie", "Genie", "Friend", or "Baby" is recognized, the voice recognition apparatus 100 may perform an operation according to the command after the call word. there is.

Here, the received voice data may be converted into a character string through voice recognition using natural language processing. Such voice recognition may be performed through the voice recognition apparatus 100 or a cloud server. For example, when voice recognition is performed through a cloud server, the cloud server performs natural language processing based on a voice information database that supports to derive context-appropriate spaces and meaningful strings from voice data to perform voice recognition. can be done

The speaker recognition unit 120 may recognize the speaker based on the received voice data. The speaker recognizer 120 may identify the speaker of the received voice data, for example, based on a speaker database in which voice data of a plurality of speakers is stored.

The voice recognition apparatus 100 may provide a speaker-customized continuous interaction service by using the identified speaker's speech pattern model. The voice recognition apparatus 100 may provide an intuitive continuous interaction service by reflecting each user's speech habit according to the speaker's speech pattern model. According to the present invention, a user experience can be improved by providing a speaker-customized continuous interaction service based on voice recognition.

The intention extractor 130 may recognize an interaction target and an interaction command from the voice data. For example, the intention extractor 130 may recognize an interaction target and an interaction command from a string derived through voice recognition.

The command for executing the continuous interaction service of the voice recognition apparatus 100 may include, for example, an interaction target and an interaction command. Here, the interaction target may refer to an area that the user wants to control through continuous interaction, and the interaction command may mean a control command to be applied to the interaction target.

For example, in the command "Scroll down", "scroll" may be an interaction target, and "get down" may be an interaction command. In addition, in the command "move to the right of the play bar", "play bar" may be an interaction target, and "move to the right" may be an interaction command.

The intention extractor 130 may determine whether there is an intention to apply the continuous interaction to the input voice data based on the continuous interaction intention.

The intention extractor 130 may extract a speaker-customized continuous interaction intention based on the recognized interaction target and interaction command. The intention extractor 130 may extract the continuous interaction intention further based on the speaker's speech pattern model.

The speaker's speech pattern model may be generated by estimating the occurrence probability value of each phoneme for each speaker, for example, based on labeling data in which repeated phonemes (connections) and silences of speech data are not removed. Based on the analysis of speech patterns for each phoneme unit that allows self-loop estimation, a speaker pronouncing a specific phoneme for a long time learns and infers by repeating the same phoneme, and a speaker pronouncing a specific phoneme short becomes the next phoneme. can be transferred to The speaker's utterance pattern model may be generated by estimating and learning a probability value that another voicing will occur immediately after the utterance of a specific voicing.

The intention extractor 130 may derive a trigger value based on the speaker's speech pattern model and the continuous interaction intention generation pattern. The continuous interaction intention generation pattern may include, for example, information about a command utterance context, a type of content to be provided, a screen configuration, and the like.

The intention extractor 130 may extract the continuous interaction intention based on whether the duration of the last syllable of the interaction command exceeds a trigger value based on the speech pattern model.

3 is an exemplary diagram for explaining a method of extracting a continuous interaction intention according to an embodiment of the present invention. Referring to FIG. 3 , the voice data may include “My friend” corresponding to a call word, “Scroll” corresponding to an interaction target, and “Download” corresponding to an interaction command.

In the example shown in FIG. 3 , the continuous interaction intention can be extracted based on whether the duration of the last syllable of “ryeo” corresponding to the command exceeds a trigger value based on the speaker’s utterance pattern model. .

The control information derivation unit 140 may derive continuous interaction control information for controlling the operation of the continuous interaction service from the voice data.

The control information derivation unit 140 may derive continuous interaction control information based on at least one of a pitch change amount and a volume change amount of the voice data.

The control information derivation unit 140 may further derive whether there is an intention to control the operation speed of the continuous interaction service in the voice data based on the recognized speaker's speech pattern model.

The control information derivation unit 140 derives, for example, an operation speed control intention trigger value based on the speaker's speech pattern model, and the absolute value of the difference between the pitch of the first syllable and the last syllable of the voice data is the trigger value. Whether there is an intention to control the operation speed of the continuous interaction service may be derived based on whether it is exceeded.

The control information derivation unit 140 may derive continuous interaction control information based on the pitch change amount of the voice data. The control information derivation unit 140 may derive continuous interaction control information based on, for example, a difference between the pitch of the first syllable of the voice data and the pitch of the last syllable of the voice data.

The control information derivation unit 140 may, for example, associate a preset value with each pitch and calculate the pitch change amount based on the values corresponding to the pitch of the first syllable and the pitch of the last syllable of the voice data.

For example, the control information derivation unit 140 may derive the pitch change amount v of the voice data based on Equation 1 below. Here, p _f may be a numerical value corresponding to the pitch of the last syllable of the voice data, and p ₁ may mean a numerical value corresponding to the pitch of the first syllable of the voice data.

The control information derivation unit 140 may derive whether there is an intention to control the operation speed of the continuous interaction service based on whether the absolute value of the pitch change amount v of the voice data exceeds the trigger value.

Also, the control information derivation unit 140 may derive continuous interaction control information based on the value of the pitch change amount v of the voice data.

For example, the continuous interaction control information increases the speed of the operation of the continuous interaction service when the pitch change of voice data has a positive value, and increases the operation speed of the continuous interaction service when the pitch change of the voice data has a negative value. It may contain information that causes the speed to be reduced. Also, as the absolute value of the pitch change amount of the voice data increases, the speed may be greatly increased or decreased.

4A and 4B are exemplary views for explaining a method of deriving continuous interaction control information based on a pitch change amount of an interaction command.

As shown in FIG. 4A , the voice recognition apparatus 100 may associate a preset numerical value with each pitch. Next, referring to FIG. 4B , the voice recognition apparatus 100 controls continuous interaction based on a difference between 1, which is a value corresponding to the pitch of the first syllable “Chin,” of the voice data, and 8, which is a value corresponding to the pitch of the last syllable, “ryeo” of the voice data. information can be derived.

For example, when the motion speed control intention trigger value is 5, in the example shown in FIG. 4B , 7, which is the absolute value of the difference between the pitch of the first syllable and the last syllable of the voice data, exceeds the motion speed control intent trigger value. Therefore, the control information derivation unit 140 may determine that there is an intention to control the operation speed of the continuous interaction service in the voice data.

In addition, in the example shown in FIG. 4B, since 7, which is the difference between the pitch of the first syllable and the pitch of the last syllable of the voice data, has a positive value, the continuous interaction control information derived by the control information derivation unit 140 is a continuous interaction service. It may include information to increase the speed of its operation.

The control information derivation unit 140 derives, for example, an operation speed control intention trigger value based on the speaker's speech pattern model, and continues based on whether the difference between the maximum volume and the minimum volume of the voice data exceeds the trigger value. It is possible to derive whether there is an intention to control the operation speed of the interaction service.

For example, the speech recognition apparatus 100 may derive the difference u ₁ between the maximum volume and the minimum volume of speech data based on Equation 2 below. Here, r _Max may represent the maximum volume of the voice data, and r _min may represent the minimum volume of the voice data.

The voice recognition apparatus 100 may derive whether there is an intention to control the operation speed of the continuous interaction service based on whether the difference u ₁ between the maximum volume and the minimum volume of the voice data exceeds a trigger value.

The control information derivation unit 140 may derive continuous interaction control information based on the volume change amount of the voice data. The control information derivation unit 140 may calculate a pitch change amount based on, for example, the volume of the first syllable and the volume of the last syllable of the voice data.

For example, the control information derivation unit 140 may derive the volume change amount u ₂ of the voice data based on Equation 3 below. Here, r _f may represent the volume of the last syllable of the voice data, and r ₁ may represent the volume of the first syllable of the voice data.

The control information derivation unit 140 may derive continuous interaction control information based on the value of the volume change amount u ₂ of the voice data.

For example, the continuous interaction control information increases the speed of the operation of the continuous interaction service when the volume change amount of the voice data has a positive value, and increases the operation speed of the continuous interaction service when the volume change amount of the voice data has a negative value. It may contain information that causes the speed to be reduced. Also, as the absolute value of the pitch change amount of the voice data increases, the speed may be greatly increased or decreased.

5 is an exemplary diagram for explaining a method of deriving continuous interaction control information based on a volume change amount of an interaction command. Referring to FIG. 5 , when the minimum volume of voice data is 32 db, the maximum volume is 59 db, and the operation speed control intention trigger value is 15, the difference between the maximum volume and the minimum volume of the voice data is 59-32=22 Exceeds the operating speed control intention trigger value. Accordingly, the control information derivation unit 140 may determine that there is an intention to control the operation speed of the continuous interaction service in the voice data.

In addition, when the volume of the last syllable of the voice data is 59 db and the volume of the first syllable is 32 db, 22, the volume change amount of the voice data, has a positive value. The continuous interaction control information may include information to increase the speed of the operation of the continuous interaction service.

The service providing unit 150 may provide a continuous interaction service based on the extracted continuous interaction intention. The service providing unit 150 may provide a speaker-customized continuous interaction service based on the characteristics of the speaker recognized by the speaker recognition unit 120 . Here, the speaker customized continuous interaction service is provided by a trigger value based on the aforementioned speaker's speech pattern model and continuous interaction intention generation pattern.

6 is a flowchart of a voice recognition method according to an embodiment of the present invention. The voice recognition method 600 performed by the voice recognition apparatus 100 illustrated in FIG. 6 includes steps that are time-series processed by the voice recognition apparatus 100 according to the embodiment illustrated in FIG. 1 . Accordingly, even if omitted below, it is also applied to the voice recognition method performed by the voice recognition apparatus 100 according to the embodiment shown in FIG. 1 .

In step S610, the voice recognition apparatus 100 may receive voice data.

In operation S620, the voice recognition apparatus 100 may recognize the speaker based on the received voice data.

In operation S630, the voice recognition apparatus 100 may recognize an interaction target and an interaction command from the voice data.

In operation S640 , the speech recognition apparatus 100 may extract a speaker-customized continuous interaction intention based on the recognized interaction target and interaction command.

In operation S650 , the voice recognition apparatus 100 may provide a continuous interaction service based on the extracted continuous interaction intention.

In the above description, steps S610 to S650 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. In addition, some steps may be omitted as needed, and the order between the steps may be switched.

The voice recognition method performed in the voice recognition apparatus described with reference to FIGS. 1 to 6 may be implemented in the form of a computer program stored in a medium executed by a computer or a recording medium including instructions executable by the computer.

Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer-readable media may include computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The above description of the present invention is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

100: speech recognition device
110: input unit
120: speaker recognition unit
130: intention extraction unit
140: control information derivation unit
150: service provider

Claims (17)

A voice recognition device for providing a voice recognition-based interaction service, comprising:
an input unit for receiving voice data;
a speaker recognition unit for recognizing a speaker based on the received voice data;
an intention extracting unit for recognizing an interaction target and an interaction command from the voice data, and extracting a speaker-customized continuous interaction intention based on the recognized interaction target and interaction command; and
A service providing unit that provides a continuous interaction service based on the extracted continuous interaction intention
A voice recognition device comprising a.
The method of claim 1,
and the intention extracting unit extracts the continuous interaction intention further based on the recognized speaker's speech pattern model.
3. The method of claim 2,
The intention extraction unit extracts the continuous interaction intention based on whether the duration of the last syllable of the interaction command exceeds a trigger value based on the speech pattern model.
The method of claim 1,
The voice recognition apparatus further comprising a control information derivation unit for deriving continuous interaction control information for controlling the operation of the continuous interaction service from the voice data.
5. The method of claim 4,
wherein the control information derivation unit derives the continuous interaction control information based on at least one of a pitch change amount and a volume change amount of the voice data.
6. The method of claim 5,
The control information derivation unit further derives whether there is an intention to control the operation speed of the continuous interaction service in the voice data based on the recognized speaker's utterance pattern model.
6. The method of claim 5,
The control information derivation unit increases the operation speed of the continuous interaction service when the pitch change amount has a positive value, and decreases the operation speed of the continuous interaction service when the pitch change amount has a negative value To derive the continuous interaction control information, speech recognition apparatus.
6. The method of claim 5,
The control information derivation unit increases the operation speed of the continuous interaction service when the volume change amount has a positive value, and decreases the operation speed of the continuous interaction service when the volume change amount has a negative value To derive the continuous interaction control information, speech recognition apparatus.
A voice recognition method for providing a voice recognition-based interaction service by a voice recognition device, the voice recognition method comprising:
receiving voice data;
recognizing a speaker based on the received voice data;
recognizing an interaction target and an interaction command from the voice data;
extracting a speaker-customized continuous interaction intention based on the recognized interaction target and interaction command; and
providing a continuous interaction service based on the extracted continuous interaction intention
That comprising a, speech recognition method.
10. The method of claim 9,
The step of extracting the continuous interaction intention is further based on the recognized speaker's speech pattern model.
11. The method of claim 10,
The step of extracting the continuous interaction intention is based on whether the duration of the last syllable of the interaction command exceeds a trigger value based on the speech pattern model.
10. The method of claim 9,
The method further comprising the step of deriving continuous interaction control information for controlling the operation of the continuous interaction service from the voice data.
13. The method of claim 12,
The step of deriving the continuous interaction control information is based on at least one of a pitch change amount and a volume change amount of the voice data.
14. The method of claim 13,
The method further comprising the step of deriving whether there is an intention to control the operation speed of the continuous interaction service in the voice data based on the recognized speaker's speech pattern model.
14. The method of claim 13,
The continuous interaction control information increases the operation speed of the continuous interaction service when the pitch change amount has a positive value, and decreases the operation speed of the continuous interaction service when the pitch change amount has a negative value. That is, a voice recognition method.
14. The method of claim 13,
The continuous interaction control information increases the operation speed of the continuous interaction service when the volume change amount has a positive value, and decreases the operation speed of the continuous interaction service when the volume change amount has a negative value. That is, a voice recognition method.
In a computer program stored in a computer-readable recording medium including a sequence of instructions for providing a voice recognition-based interaction service,
When the computer program is executed by a computing device,
receive voice data,
Recognizing a speaker based on the received voice data,
Recognizing an interaction target and an interaction command from the voice data,
Extracting a speaker-customized continuous interaction intention based on the recognized interaction target and interaction command,
A computer program stored in a computer-readable recording medium comprising a sequence of instructions to provide a continuous interaction service based on the extracted continuous interaction intention.

Images