KR20180047597A - Method for providing of voice recognition service using information of voice signal with wakeup and apparatus thereof - Google Patents

Abstract

The present invention relates to a method for providing a voice recognition service. More particularly, the present invention relates to a service server capable of providing a voice recognition service in cooperation with a sound output apparatus including a microphone and a speaker. The sound output apparatus generates voice signal information for performing preprocessing based on the initial interval of an input voice signal and performs voice recognition more accurately using the voice signal information.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a method and apparatus for providing a voice recognition service using voice signal information,

The present invention relates to a method of providing a voice recognition service, and more particularly, to a service server capable of providing a voice recognition service in cooperation with an audio output device including a microphone and a speaker, And a service server generates a speech recognition service using speech signal information capable of performing speech recognition more accurately by using the speech signal information, .

The contents described in this section merely provide background information on the present embodiment and do not constitute the prior art.

As technology develops, various services using voice recognition technology have been introduced in many fields. Speech recognition technology is a series of processes for understanding human speech and converting it into character information that can be handled by a computer. A speech recognition service using a speech recognition technology recognizes a user's speech, Quot; and " a "

However, there is a problem that the services using the speech recognition technology up to now recognize the user's voice and provide only a fragmentary service corresponding to the user's voice. For example, there is a voice recognition service in which a user speaks a voice for a specific word using his or her mobile phone, the mobile phone recognizes it, and provides a search result corresponding to the word through a web browser. However, There is a problem that convenience of various users can not be satisfied by interlocking and providing only the corresponding search results.

Accordingly, it is necessary to develop a speech recognition service capable of enhancing the performance of speech recognition, grasping the user's intention more accurately, and providing various contents in cooperation with an external content providing server.

Korean Patent Laid-Open No. 10-2008-0083553, September 18, 2008 (Name: Image synthesis call service method and apparatus using speech recognition)

SUMMARY OF THE INVENTION The present invention has been proposed in order to solve the above problems, and it is an object of the present invention to provide a sound output apparatus including at least one multi-channel microphone for generating a voice signal when a user's voice is input, The present invention provides a method for providing a voice recognition service using voice signal information capable of extracting voice signal information including direction, channel, and noise information by using information on the voice,

Further, the present invention performs voice recognition after compensating for noise elimination and variations of the entire voice signal streamed from the sound output apparatus using the voice signal information received from the sound output apparatus, The present invention provides a method and apparatus for providing a voice recognition service using voice signal information capable of performing voice recognition.

However, the object of the present invention is not limited to the above-mentioned objects, and other objects not mentioned can be clearly understood from the following description.

According to another aspect of the present invention, there is provided a method of providing a voice recognition service using speech signal information, the method including: receiving a voice of a user who is speaking through at least one multi- Generating a signal; Extracting speech signal information including direction, channel, and noise information on a speech signal of a predetermined interval for performing a wakeup process on the generated speech signal; And if the wakeup word is included in the voice signal of the predetermined interval, the voice signal is sequentially streamed and transmitted to the service server, and the extracted voice signal information is transmitted to the service server The method comprising the steps of:

The generating of the voice signal may include generating a voice signal using a voice input through the at least one multi-channel microphone, Dividing a generated voice signal into frames of a predetermined length, and storing a frame of a predetermined interval in order to perform a wakeup process.

The extracting of the voice signal information may include extracting direction information using at least one of a arrival time, a reaching distance, and a voice strength of a voice input through the at least one multi-channel microphone, Checking and extracting channel information of the multi-channel microphone; Extracting a speech feature vector for each of the stored frames, and extracting a signal-to-noise ratio (SNR) using the extracted speech feature vector.

At this time, the step of transmitting to the service server may stream the voice signal corresponding to the stored frame to the service server, or may stream the voice signal generated after the stored frame to the service server and transmit the voice signal.

In addition, the step of transmitting to the service server may include the extracted voice signal information in the header information of the streaming voice signal to be transmitted to the service server.

According to another aspect of the present invention, there is provided a method of providing a voice recognition service using speech signal information, the method comprising: receiving a voice signal streamed from a voice output device, Receiving voice signal information extracted with respect to a voice signal of the first mobile terminal; Performing a preprocessing process for noise removal and disparity compensation in the streamed audio signal using the audio signal information; And a step of recognizing the speech signal subjected to the preprocessing process and analyzing the meaning, and delivering the corresponding speech recognition service to the sound output device according to the analyzed result.

In this case, the performing of the preprocessing may include filtering a specific frequency band for removing noise from the voice signal using the direction information and the noise information, performing noise removal using the configured filter, Processing of compensating the channel variation of the voice signal using the information.

Further, the present invention can provide a computer-readable recording medium on which a program for executing the above-described method is recorded.

According to an aspect of the present invention, there is provided an audio output apparatus including: a voice signal input unit for receiving a user's voice uttered through at least one multi-channel microphone and generating a voice signal; And extracting speech signal information including direction, channel, and noise information on a speech signal of a predetermined interval for performing a wakeup process from the speech signal transmitted through the speech signal input unit, A device controller for sequentially streaming the generated voice signal and transmitting the voice signal to the service server and controlling the extracted voice signal information to be transmitted to the service server when the voice recognition includes the wakeup word; And the like.

According to an aspect of the present invention, there is provided a service server for receiving a voice signal streamed from an audio output device, the voice signal being extracted from a voice signal of a predetermined interval for performing a wakeup process, A server communication unit for receiving information from the sound output apparatus; And a preprocessing process for noise removal and disparity compensation in the streamed audio signal using the audio signal information received through the server communication unit, recognizing the audio signal subjected to the preprocessing process, analyzing the meaning, And a server control unit for controlling the voice output unit to transmit the corresponding voice recognition service to the sound output unit according to the result of the determination.

At this time, the server control unit configures a filter of a specific frequency band for removing noise from the voice signal using the direction information and the noise information included in the voice signal information, performs noise removal using the configured filter, A preprocessing process for compensating for the channel variation of the voice signal using the channel information included in the voice signal information can be performed.

According to the present invention, there is provided an audio output apparatus capable of generating a voice signal by receiving a user's voice uttered through at least one multi-channel microphone, The voice signal information including direction, channel, and noise information is extracted from the voice signal at the time of performing the up process, so that the voice signal information can be extracted more quickly and transmitted to the service server.

According to another aspect of the present invention, there is provided a service server capable of providing a voice recognition service in cooperation with the sound output device, the service server performing a preprocessing process using voice signal information for an initial period of the voice signal provided by the sound output device It is possible to provide a faster voice recognition service without requiring a process to perform a preprocessing process and to perform preprocessing using voice signal information generated by an audio output device, Noise, and various other factors more accurately, thereby enhancing the performance of speech recognition.

Accordingly, the present invention can increase the satisfaction of users in providing various services through voice recognition, and it is possible to provide a fast and accurate voice recognition service.

In addition, various effects other than the above-described effects can be directly or implicitly disclosed in the detailed description according to the embodiment of the present invention to be described later.

1 is a block diagram illustrating a system for providing a voice recognition service according to an embodiment of the present invention.
2 is a data flow chart for explaining a method of providing a voice recognition service using voice signal information according to an embodiment of the present invention.
3 is a block diagram showing a main configuration of an audio output apparatus according to an embodiment of the present invention.
4 is a flowchart illustrating a method of providing a voice recognition service using voice signal information in an audio output apparatus according to an embodiment of the present invention.
5 is a configuration diagram illustrating a main configuration of a service server according to an embodiment of the present invention.
6 is a flowchart illustrating a method of providing a voice recognition service using voice signal information in a service server according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS For a more complete understanding of the nature and advantages of the present invention, reference should be made to the following detailed description taken in conjunction with the accompanying drawings, in which:

In the following description and the accompanying drawings, detailed description of well-known functions or constructions that may obscure the subject matter of the present invention will be omitted. It should be noted that the same constituent elements are denoted by the same reference numerals as possible throughout the drawings.

The terms and words used in the following description and drawings are not to be construed in an ordinary sense or a dictionary, and the inventor can properly define his or her invention as a concept of a term to be described in the best way It should be construed as meaning and concept consistent with the technical idea of the present invention. Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention, and not all of the technical ideas of the present invention are described. Therefore, It is to be understood that equivalents and modifications are possible.

Also, terms including ordinal numbers such as first, second, etc. are used to describe various elements, and are used only for the purpose of distinguishing one element from another, Not used. For example, without departing from the scope of the present invention, the second component may be referred to as a first component, and similarly, the first component may also be referred to as a second component.

In addition, when referring to an element as being "connected" or "connected" to another element, it means that it can be connected or connected logically or physically. In other words, it is to be understood that although an element may be directly connected or connected to another element, there may be other elements in between, or indirectly connected or connected.

Also, the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. It is also to be understood that the terms such as " comprising "or" having ", as used herein, are intended to specify the presence of stated features, integers, It should be understood that the foregoing does not preclude the presence or addition of other features, numbers, steps, operations, elements, parts, or combinations thereof.

Hereinafter, a method and apparatus for providing a voice recognition service using voice signal information according to an embodiment of the present invention will be described.

First, a speech recognition service providing system according to an embodiment of the present invention will be described.

1 is a block diagram illustrating a system for providing a voice recognition service according to an embodiment of the present invention.

Referring to FIG. 1, the voice recognition service providing system of the present invention may include a terminal 100, an audio output apparatus 200, and a service server 300. The terminal 100 and the sound output apparatus 200 of the present invention can be connected to each other by a local area communication method to transmit and receive information. The terminal 100 and the service server 300, the sound output apparatus 200, (300) may be connected via a communication network (500).

The terminal 100 refers to a user apparatus capable of transmitting and receiving various data according to a user's request. In particular, the terminal 100 of the present invention may be connected to the sound output apparatus 200 by a short distance communication method, and may transmit various setting information for operating the sound output apparatus 200 to the sound output apparatus 200. For example, when the sound output apparatus 200 provides a notification function, the terminal 100 may receive various information for notification setting from the user and provide the information to the sound output apparatus 200.

The terminal 100 of the present invention may transmit identification information of the terminal 100 to the sound output apparatus 200 while the sound output apparatus 200 is connected to the sound output apparatus 200 by a short distance communication method, 100 to the service server 300 through the communication network 500 to support more personalized service provisioning.

In addition, the terminal 100 according to the embodiment of the present invention can set or change a wakeup word. In other words, the sound output apparatus 100 of the present invention recognizes a wakeup word in a voice signal of a user and transmits a voice signal to the service server 300 as a wakeup word is recognized or performs a specified operation The terminal 100 can perform a process of specifying or changing a wakeup word at this time.

In addition, the terminal 100 according to the embodiment of the present invention may transmit information on the content providing server 400 to the service server 300. For example, assuming that there are a plurality of content providing servers 400 serving the same content, the terminal 100 can subscribe to a service provided by a specific content providing server 400 among the plurality of content providing servers 400 And the subscription information may be transmitted to the service server 300 so that the content provided by the content providing server 400 can be used through the audio output apparatus 200.

In addition, the terminal 100 of the present invention shares or provides the setting information of various applications provided in the terminal 100 with the sound output apparatus 200 or the service server 300 to provide more intelligence in the sound output apparatus 200 Service can be performed. For example, if the user has uttered "What is the appointment today?", The sound output apparatus 200 generates a voice signal to the service server 300, the service server 300 recognizes the voice of the user, A schedule management application provided in the terminal 100 to provide a response message including the confirmed response information such as "There is a alumni meeting at 7:00 Kangnam station" to the sound output device 200).

For this purpose, the terminal 100 of the present invention must be provided with a service application for controlling the sound output apparatus 200 in advance. The service application may be downloaded and installed through the service server 300 or a separate application store, or may be used as a cloud service method by accessing the service server 300.

The sound output apparatus 200, which outputs speech recognition service of the present invention through a speaker, generates a sound signal by receiving a user's speech uttered by a user within a certain radius and transmits the generated sound signal to the service server 300 And outputs the contents provided by the various contents providing servers 400 and provides the contents. In particular, the sound output apparatus 200 of the present invention includes a microphone and receives a user's utterance voice through a microphone and generates it as a voice signal. The sound output apparatus 200 of the present invention classifies a voice signal inputted through a microphone into a frame of a predetermined length and performs speech recognition on an initial section of the voice signal to determine whether a predetermined wakeup word is included Check. Here, the wake-up word means a word registered in advance in the sound output apparatus 200. When the wake-up word is included in the voice uttered by the user, the sound output apparatus 200 of the present invention outputs the generated voice signal To the service server 300 via the communication network 500. When it is determined that a wake-up word exists in the input voice signal, the audio output apparatus 200 of the present invention applies power to the communication module to stream the voice signal including the wake-up word to the service server 300, .

Particularly, the sound output apparatus 200 of the present invention can perform a preprocessing process on an initial section of the voice signal when the voice signal is transmitted to the service server 300. At this time, the sound output apparatus 200 of the present invention can generate voice signal information including direction, channel, and noise information corresponding to the voice signal. Herein, the direction information is information that can be extracted using at least one of the arrival time, the reaching distance, and the sound intensity of a voice input through each of the at least one multi-channel microphone, Channel information, and the noise information means a signal-to-noise ratio of the voice signal. The sound output apparatus 200 of the present invention can generate the voice signal information and transmit the voice signal information to the service server 300.

The sound output apparatus 200 of the present invention recognizes the voice signal and, when the response information suitable for the user's intention is received from the service server 300, the sound output apparatus 200 performs a process of outputting it as a voice signal through the provided speaker do. At this time, the sound output apparatus 200 of the present invention can receive a response message in the form of a voice signal from the service server 300, convert the received response message into an audible frequency band that can be recognized by the user, .

On the other hand, if the link information is received together with the response message from the service server 300, the sound output apparatus 200 of the present invention accesses the content providing server 400 using the link information, And receiving the contents corresponding to the corresponding link information in a streaming form and outputting the contents through the speaker. For example, when the link information is link information for the sound source A provided by the content providing server B 400b of FIG. 1, the sound output apparatus 200 connects to the content providing server B 400b via the communication network 500 And requests stream A to receive streaming data from content providing server B 400b and reproduce the streaming data. At this time, the link information may include approval information that the content providing server 400 can be connected. The sound output apparatus 200 receiving the link information may have permission to access the content providing server 400 The content providing server 400 can provide the content without any authentication process.

Meanwhile, the service server 300 refers to a service provider entity providing a voice recognition service according to an embodiment of the present invention. The service server 300 of the present invention can transmit / receive various information for providing a service in cooperation with the terminal 100. In addition, the service server 300 of the present invention can receive a voice signal generated by a user from the sound output apparatus 200 and provide a voice recognition service corresponding thereto.

In particular, the service server 300 of the present invention can receive the voice signal information together with the voice signal from the voice output device 200. The voice signal information refers to information related to a characteristic of a voice signal generated corresponding to an initial partial period of the voice signal, and may include direction, channel, and noise information. At this time, the voice signal information of the present invention may be received through a separate path from the voice signal that is streamed and received from the voice output device 200, and may be received in the header information of the voice signal.

When the voice signal information is received, the service server 300 may perform a preprocessing process for noise removal and disparity compensation using the voice signal information. In this case, the noise removal may be performed by removing a specific frequency band corresponding to the noise based on the direction and noise information in the speech signal information, and then noise may be removed according to a configured filter. Since the channel characteristics of the voice signal information may be varied in the process of being transmitted from the sound output apparatus 200 to the service server 300, the process of checking the original channel using the channel information of the voice signal information and compensating for the channel variation Can be performed.

Through the above process, the service server 300 of the present invention can perform voice recognition more efficiently and accurately, and thereby can grasp the intention of the user more accurately.

In addition, the service server 300 of the present invention can interoperate with a plurality of content providing servers 400 at a subsequent stage, and can receive content from a content providing server 400, and then receive the content from the content providing server 400 and provide the content to the sound output apparatus 200. At this time, the service server 300 of the present invention not only delivers the content provided by the content providing server 400 to the sound output apparatus 200, but also constructs and generates a response message according to the voice recognition service, Text to Speech) engine, and then transmits the converted voice signal to the sound output apparatus 200 in a streaming form.

The operation of the audio output apparatus 200 and the service server 300 of the present invention will be described in more detail with reference to the accompanying drawings. (CP) that performs a function of transmitting a service or information to the service server 300. [ In particular, the content providing server 400 of the present invention can transmit different types of responses according to the type of the query of the service server 300 or the type of the content provided by the service server 300 itself. For example, if the content providing server A 400a is a content providing server 400 capable of providing a streaming service for a sound source, the service providing server A 400a can transmit link information capable of streaming the corresponding sound source according to the request. On the other hand, if the content providing server B 400b is a server capable of providing specific contents such as weather and temperature, it can provide only the specific information (current weather, current temperature, etc.) have.

In addition, the content according to the present invention is a concept that refers to all application programs that perform specific functions such as movies, music, and the like, such as specific information such as weather, time, game application, and schedule management application. Such contents can be transmitted to the audio output apparatus 200 or to the service server 300 in the form of streaming data.

The communication network 500 supporting the above-described processes may be used not only for an IP-based wired communication network such as the Internet, but also for various types of wireless networks such as a long term evolution (LTE) network, a mobile communication network such as a WCDMA network, The backbone network, and the Internet network. However, since a variety of known technologies can be applied to the specific configuration and operation of the communication network, Are omitted.

In addition, the communication network 500 of the present invention may include a cloud computing network capable of storing computing resources such as hardware and software, and providing the computing resources required by the client to the terminals. Here, cloud computing refers to a computer environment in which information is permanently stored on a server on the Internet and temporarily stored in a client terminal such as a desktop, a tablet computer, a notebook, a netbook, or a smart phone. Cloud computing, Refers to a computer environment access network that stores information on a server on the Internet and makes the information available anytime and anywhere through various IT devices. At this time, the terminal 100 of the present invention can access the service application provided by the service server 300 for accessing the service server 300 by using a cloud computing method.

The main constitution of each device for the embodiment of the present invention has been outlined above. A processor mounted on each device according to an embodiment of the present invention may process program instructions for executing the method according to the present invention. In one implementation, the processor may be a single-threaded processor, and in other embodiments, the processor may be a multithreaded processor. Further, the processor is capable of processing instructions stored on a memory or storage device.

Hereinafter, a method of providing a voice recognition service using voice signal information according to an embodiment of the present invention will be described in more detail.

2 is a data flow chart for explaining a method of providing a voice recognition service using voice signal information according to an embodiment of the present invention.

Referring to FIG. 2, the sound output apparatus 200 of the present invention is basically in a standby state. In a standby state, the audio output apparatus 200 applies power to only some modules, such as a microphone for receiving user's voice, and disconnects unnecessary modules such as a communication module for communication with the service server 300 .

The standby state of the present invention can be collectively referred to as a state before speech recognition is started. In other words, even if a plurality of users are ignited near the sound output apparatus 200 or there is noisy noise and the voice signal is continuously recognized through the microphone, the voice signal input until the wake-up word is input is continuously deleted, It may mean a state in which it is controlled not to be transmitted to the server 300.

In this state, the sound output apparatus 200 of the present invention receives a voice transmitted from a user within a certain radius through a microphone (S101), and generates a voice signal (S103). At this time, the sound output apparatus 200 of the present invention can generate a user's voice input through a microphone in real time as a voice signal in the form of an electrical signal.

In particular, the sound output apparatus 200 of the present invention may store a voice signal for an initial period set in a previous step for performing a wakeup process, and perform a preprocessing process using a voice signal for the initial period .

For example, when converting the audio output apparatus 200 of the present invention into a voice signal of an electrical form using a voice input through each of the multi-channel microphones, at least one of the arrival time, Direction information can be extracted using any one of them. The sound output apparatus 200 of the present invention can identify and extract channel information of the multi-channel microphone. At this time, the sound output apparatus 200 of the present invention can check the channel information of each of the multi-channel microphones or check the channel information in which the channels of the multi-channel microphones are mixed. In addition, the sound output apparatus 200 of the present invention can extract a signal-to-noise ratio for the speech signal. At this time, the sound output apparatus 200 of the present invention can separate the speech signal into a frame having a predetermined length, extract a speech feature vector for each separated frame, and extract a signal-to-noise ratio using speech feature vectors .

Then, the sound output apparatus 200 of the present invention performs speech recognition (S105). At this time, the sound output apparatus 200 of the present invention may recognize a voice signal after performing a noise removal process or the like using the information extracted in step S103, and the noise removal process may be performed again by the service server 300 It is possible to recognize the voice signal without any special process.

Also, the sound output apparatus 200 of the present invention can recognize a voice signal for a preset initial period for performing a wake up process as described above. For example, if it is set to recognize a speech signal for the initial 10 frames, the sound output apparatus 200 of the present invention stores the speech signal interval for the 10 frames and performs a speech recognition process using the stored information can do. For this purpose, the sound output apparatus 200 of the present invention can store an input voice signal in a device storage unit, and judges whether there is a word corresponding to a wake up result (S107) If there is no word corresponding to the wakeup, the voice signal stored in the device storage unit is deleted, and if it exists, the voice signal separated into frames in real time may be streamed and transmitted to the service server 300 (S109). At this time, the sound output apparatus 200 of the present invention transmits voice signal information extracted for the initial section of the voice signal to the service server 300 together with the streaming voice signal. In addition, the audio output apparatus 200 of the present invention can transmit a voice signal from which the voice signal corresponding to the wake-up word is removed to the service server 300, and transmits a voice signal including the wake- 300).

The service server 300 may configure a filter for removing noise from a voice signal received from the sound output apparatus 200 (S111). In the general method, a voice signal is received and a preprocessing process for removing noise is performed using the received voice signal. However, the service server 300 of the present invention does not perform a separate determination and design process for noise removal , By using the voice signal information transmitted through the sound output apparatus 200, a filter for noise removal can be configured more quickly, and the noise can be removed using the configured filter. In addition, the service server 300 of the present invention may perform a process of compensating a channel variation of the voice signal using channel information included in voice signal information.

As described above, the service server 300 of the present invention performs a preprocessing process on the entire voice signal received using the voice signal information transmitted to the sound output apparatus 200 without performing a separate determination process for noise removal and channel variation It is possible to perform speech recognition more quickly and to perform a compensation process for a channel variation that may be generated in a communication process using channel information transmitted from the sound output device 200. [

Thereafter, the service server 300 of the present invention performs speech recognition on the preprocessed speech signal (S115). At this time, the service server 300 of the present invention extracts a speech feature vector from the received speech signal and confirms a phoneme corresponding to the extracted speech feature vector. Here, a phoneme refers to the smallest unit (eg, a, d, ㅓ, ㅕ, ㅁ) that distinguishes meaning. In phonemic recognition, a process of converting into a set of such units can be performed.

Thereafter, the service server 300 of the present invention constructs a word string, which is a set of phonemes, and performs a semantic analysis process on the formed word string (S117). That is, the service server 300 analyzes the sentence composed of the word sequence and confirms whether each word string corresponds to an entity corresponding to the entity (S119). Here, the entity may mean a predefined proper noun such as the name of a specific service desired by the user. For example, it is possible to extract a proper noun that has been promised in advance, such as weather, latest song, pop song, time, appointment, etc., as an entity. On the other hand, an intent means a word string including a user's intention. For example, the intent may correspond to a word string for expressing a user's intention except an entity such as 'play', 'what time is' have.

For example, when the user has uttered "Tell me music ", the service server 300 can analyze the 'music' as an entity and the 'tell me' intent. In addition, if the user has uttered "Turn on air purifier ", the service server 300 can analyze the 'air purifier' as an entity and the 'turn on' intent. In addition, the service server 300 of the present invention can store information on the service providing entity corresponding to each entity in the form of a table, and can identify the service providing entity, that is, the content providing server using the analyzed entity (S121). At this time, if there is no separate content providing server, the service server 300 may be a service providing entity.

In addition, the service server 300 of the present invention can store and manage information on a user's intention in the form of a table corresponding to the intent, accurately confirm the intention of the user using the analyzed intent, The service providing server 400 may request the service by requesting it (S123). For example, it is possible to confirm that the tent is 'play' and transmit the service request for playback to the content providing server 400, confirm that the tent is 'play' ) Or it can execute the device corresponding to the entity directly.

In another example, if the entity is 'weather' and the intent of 'how' is 'inquiry', the service server 300 confirms the content providing server a 400a capable of providing the weather content , And may request the content providing server a 400a to inquire about the weather. At this time, the service server 300 of the present invention confirms the identification information of the sound output apparatus 200, confirms information on the current position and time of the sound output apparatus 200, 400). ≪ / RTI > For example, when the current location of the sound output apparatus 200 is 'Seoul City', the service server 300 can query the weather for the current time in Seoul, 300 in response to a query request and transmits the corresponding response to the service server 300 (S125).

Thereafter, the service server 300 of the present invention constructs a response message using the received response (S127). That is, since the service server 300 of the present invention receives only the response information about the content from the content providing server 400, the service server 300 can perform a process of transmitting the response information to the sound output device 200.

First, if response information 'rain' is received from the content providing server 400, the service server 300 may construct a series of sentences including the contents. At this time, the series of sentences may be configured in consideration of the current position and the current time of the sound output apparatus 200. In the above example, if the current location of the sound output apparatus 200 is referred to as a city of Seoul, and the current time is assumed to be 2:00, the service server 300 includes the information as " &Quot;, < / RTI > When constructing the response message, the service server 300 of the present invention can construct a response message in consideration of the information set by the user. For example, when the setting information of the user for constituting the response message is transmitted from the terminal 100, a response message can be configured based on the setting information. When the user selects the type of the language as English, It is possible to convert the content transferred from the providing server 400 and construct a response message. In addition, if the user sets the style to 'friendly', the response message can be constructed with a familiar style, and if the user is set to the honorific language, the response message can be configured with a vernacular style.

The service server 300 of the present invention performs a process of converting the response message into a speech signal in order to output a response message constructed through the sound output apparatus 200 (S129), and transmits a response message converted into an electrical speech signal And transmits it to the sound output apparatus 200 in a real-time streaming form (S131). That is, when the response message converted into the voice signal is composed of 10 frames, the voice message is transmitted to the sound output apparatus 200 in real time on a frame-by-frame basis, and the sound output apparatus 200 outputs the response message through the speaker (S133).

Meanwhile, the audio output apparatus 200 of the present invention accesses the content providing server 400 using the link information transmitted from the service server 300, receives the corresponding content from the content providing server 400 in a streaming manner .

The operation in the voice recognition service providing system according to the embodiment of the present invention has been described above.

Hereinafter, a method of providing a voice recognition service using voice signal information according to an embodiment of the present invention will be described in more detail with reference to the drawings.

Hereinafter, the main configuration and operation of the sound output apparatus 200 according to the embodiment of the present invention will be described.

FIG. 3 is a configuration diagram illustrating a main configuration of an audio output apparatus according to an embodiment of the present invention. FIG. 4 is a flowchart illustrating a method of providing a voice recognition service using voice signal information in an audio output apparatus according to an embodiment of the present invention. Fig.

3, an audio output apparatus 200 according to the present invention includes a voice signal input unit 210, a device control unit 220, a device communication unit 230, an output unit 240, and a device storage unit 250 And the like.

First, the audio signal input unit 210 of the present invention may include an ADC 212 for converting an analog audio signal input through the microphone 211 into an electrical signal. In particular, the voice signal input unit 210 of the present invention receives voice generated through user's utterance through a plurality of multi-channel microphones 211.

At this time, the input voice may include surrounding noise and echo generated through a multi-channel microphone. The ADC 212 of the present invention converts the analog voice including the echo and noise into an electrical voice signal and transmits the voice signal to the device controller 220.

The device control unit 220 of the present invention may include a voice recognition module 221, a wakeup determination module 222, a signal processing module 223, and a service processing module 224.

At this time, the voice recognition module 221 may perform various processes for recognizing a voice signal in the form of an electrical signal transmitted through the ADC 212 of the voice signal input unit 210. In particular, the speech recognition module 221 of the present invention can perform a preprocessing process and speech recognition process for an initial partial segment of a speech signal. To this end, the voice recognition module 221 of the present invention generates voice signal information including directions, channels, and noise information corresponding to an initial partial period of a voice signal required for wakeup determination in the wakeup determination module 222 Respectively.

More specifically, when a voice is input through the plurality of multi-channel microphones 211 of the voice signal input unit 210 of the present invention, the voice recognition module 221 detects a time, a distance, or a sound intensity Decibel), it is possible to determine the direction of the user who uttered the voice in real time, and the direction information can be extracted on the basis of the direction.

In addition, the speech recognition module 221 of the present invention extracts channel information on a voice signal input from a microphone for each channel from a voice signal inputted through a plurality of multi-channel microphones 211 transmitted through the ADC 212 can do. At this time, the speech recognition module 221 of the present invention can extract channel information for each of the voice signals. In accordance with the implementation method, the voice recognition module 221 mixes a plurality of channels and extracts information about the mixed channel have.

Also, the speech recognition module 221 of the present invention can confirm the signal-to-noise ratio finally by checking the degree of noise between the channels in the speech signal. At this time, the speech recognition module 221 of the present invention classifies the speech signal according to a frame, extracts a speech feature vector for each divided frame, and then uses a spectrum of the speech feature vector to obtain a signal to noise ratio (SNR) Extraction can be performed.

The speech recognition module 221 of the present invention may perform a speech recognition process for determining a wakeup of a speech signal on a frame-by-frame basis. At this time, the speech recognition module 221 of the present invention may perform speech recognition on speech signals whose noise has been removed based on the extracted noise information, direction information, and the like.

In addition, the speech recognition module 221 of the present invention confirms the phoneme corresponding to the speech feature vector calculated on a frame-by-frame basis through the speech recognition information 251 of the device storage unit 250. Then, the speech recognition module 221 of the present invention constructs a set of phonemes and constructs a word sequence according to the word sequence, thereby completing the speech recognition process.

This process can be performed in real time on a frame-by-frame basis. The voice recognition module 221 performs voice recognition on an initial section of a voice signal input through the voice recognition module 221 to determine whether to perform an operation for wake- (222).

If the voice signal is recognized in real time, the wake-up determination module 222 can determine whether or not there is a wake-up word for wake-up. If the wakeup word exists, the wakeup determination module 222 of the present invention can control power to be applied to the standby module such as the device communication unit 230, the output unit 240, and the like.

The service processing module 223 of the present invention can transmit a subsequent voice signal to the service server 300 through the service interworking module 231 of the device communication unit 230. In particular, when it is determined that the wakeup word is included in the wakeup judgment module 222, the service processing module 223 of the present invention transmits the voice signal transmitted in real time via the voice signal input unit 210, And may be controlled to be streamed to the service server 300 through the device communication unit 230 and transmitted.

At this time, the service processing module 223 of the present invention may transmit the voice signal temporarily stored in the device storage unit 250 to the service server 300 for the wakeup judgment.

In addition, the voice signal information of the present invention may be transmitted to the service server 300 separately from the streaming voice signal, and may be transmitted to the service server 300 in the header information of the voice signal.

When the response message is received from the service server 300, the service processing module 223 of the present invention transmits the response message to the output unit 240. In this case, when link information for content access is received together with the service interworking module 231 of the device communication unit 230, the service processing module 223 of the present invention transmits the link information to the content providing server 400 through the link information And can control the process of receiving the streaming data for the corresponding content from the content providing server 400 through the service interworking module 231 and outputting the received streaming data through the output unit 240. [

The device communication unit 230 includes a service interlock module 231 and a terminal interlock module 232. The service interlock module 231 communicates with the service server 300 or the content providing server 400 via the communication network 500, Lt; RTI ID = 0.0 > and / or < / RTI > Meanwhile, the terminal interworking module 232 can exchange information with the terminal 100 through short-range communication.

The output unit 240 includes a DAC (Digital to Analog Converter) 241 for converting an audio signal, which is an electrical signal, into an analog form, and a speaker 242 for outputting a voice signal output through the DAC 241 And the like.

The device storage unit 25 stores and manages various information related to the driving of the audio output apparatus 200 according to the embodiment of the present invention. In particular, the device storage unit 250 of the present invention may include voice recognition information 251 for recognizing an input voice signal in real time, and may store and manage information 252 related to wakeup. That is, it can be used to compare whether or not the recognized word sequence corresponds to the wakeup word stored in the wakeup information 252. In addition, various information 253 related to the service process can be stored and managed.

In addition, the device storage unit 250 of the present invention can store the voice signal inputted in real time through the voice signal input unit 210 in units of frames, and store the stored voice signal at the request of the device control unit 220. [ To the communication unit 230. In addition, the stored signal can be deleted at the request of the device control unit 220.

The operation of the sound output apparatus 200 according to the embodiment of the present invention will be described with reference to FIG.

Referring to FIG. 4, the sound output apparatus 200 of the present invention can maintain a standby state until a wake-up word is recognized (S201).

In this state, when the sound output apparatus 200 of the present invention is input through a microphone having a voice transmitted from a user within a certain radius (S207), the sound output apparatus 200 generates a voice signal and stores a voice signal of a certain range . At this time, the sound output apparatus 200 of the present invention receives a voice of a user transmitted through a microphone, converts the voice into a voice signal in the form of an electrical signal, separates the converted voice signal into frames of a predetermined length, It is possible to perform speech recognition for wake-up judgment. In particular, the sound output apparatus 200 of the present invention performs a preprocessing process corresponding to a section for performing voice recognition for wakeup determination (S205), and generates voice signal information including direction, channel, and noise information (S207).

Thereafter, the sound output apparatus 200 of the present invention can remove noise using the generated speech signal information, and perform speech recognition on the speech signal from which noise has been removed (S209).

Thereafter, it is determined whether there is a word corresponding to the wakeup (S211). If there is a wakeup word, the service server 300 streams the voice signal and transmits it. At this time, the audio output apparatus 200 of the present invention can transmit the audio output signal to the service server 300 sequentially from the frame stored in the voice recognition step for wakeup determination, To the service server 300.

In addition, the sound output apparatus 200 of the present invention transmits the voice signal information extracted in step S207 separately from the voice signal or included in the header information of the voice signal to the service server 300 (S213).

The service server 300 receiving the voice signal recognizes the voice signal and recognizes the intention of the user, and can generate a response message for the voice signal that the user has uttered.

Then, the service server 300 transmits the generated response message to the sound output apparatus 200, and the sound output apparatus 200 of the present invention receives the response message converted into the voice signal, which is an electrical signal, from the service server 300 If it is received (S215), it can be outputted through the built-in speaker (S217). At this time, the sound output apparatus 200 of the present invention can perform a process of converting a response message into an analog form, and output a response message converted to analog.

On the other hand, if it is determined in step S211 that the wakeup word is not included, the temporarily stored voice signal is deleted and the extracted voice signal information corresponding to the voice signal is deleted (S219).

The main configuration and operation of the sound output apparatus 200 according to the embodiment of the present invention have been described above.

Hereinafter, the main configuration and operation of the service server 300 according to the embodiment of the present invention will be described.

FIG. 5 is a configuration diagram illustrating a main configuration of a service server according to an embodiment of the present invention. FIG. 6 illustrates a method of providing a voice recognition service using voice signal information in a service server according to an embodiment of the present invention Fig.

5, a service server 300 according to an exemplary embodiment of the present invention includes a server communication unit 310, a server storage unit 320, a server control unit 330, a conversion unit 340, (350).

The server communication unit 310 of the present invention can perform the function of transmitting and receiving information to and from the terminal 100 and the sound output apparatus 200. [ In particular, the server communication unit 310 of the present invention can receive various user setting information related to the control of the sound output device 200 and the voice recognition service of the present invention from the terminal 100 via the communication network 500. In addition, it is possible to receive the identification information of the sound output apparatus 200 from the terminal 100 and receive the subscriber information of the terminal 100. In addition, the server communication unit 310 of the present invention can receive a voice signal uttered by the user from the sound output apparatus 200, and the server control unit 330 generates A response message and link information can be transmitted.

The server storage unit 320 may store and manage various information related to the operation of the service server 300 of the present invention. In particular, the server storage unit 320 of the present invention can store and manage various kinds of information necessary for speech recognition and semantic analysis. For example, the server storage unit 320 can store and manage an acoustic model for voice recognition, You can store and manage information about entities and intents.

Describing the entities and intents of the present invention in more detail, the entities of the present invention may refer to predefined proper nouns, such as the name of a particular service a user desires. For example, it can mean a proper noun to be controlled, such as weather, latest song, pop song, time, appointment, timer, purifier, mobile phone, and the like. In addition, the server storage unit 320 of the present invention can store and manage information on a subject that can process the entity.

Service provider Entity Content providing server A Weather, temperature, humidity Content providing server B Latest Songs, Pop Songs, K-Pop, Pop, Latest Songs Service server Schedule, appointment, timer

That is, as shown in Table 1, the server storage unit 320 of the present invention can store and manage information on a service providing entity that can process a specific entity.

The intent that can store and manage a word string corresponding to an intent is a word string including an intention of a user. For example, the server storage unit 320 may store ' The user may correspond to a word sequence for expressing a desired intention to an entity such as 'how many times', 'how about', 'tell me', 'turn on', 'tell me'

Intent Intent play Let me play, I hear it, I want to hear it, Play it, stop Stop, stop, stop, stop. vaginal Check it out.

That is, as shown in Table 2, the server storage unit 320 can store and manage a word string indicative of a user's intent as an intent.

In addition, the server storage unit 320 of the present invention can store and manage various information related to the provision of the voice recognition service according to the embodiment of the present invention.

The server control unit 330 manages overall operation of the service server 300 according to the embodiment of the present invention and includes a preprocessing module 331, a voice recognition module 332, a semantic analysis module 334, and a service processing module 334).

The preprocessing module 331 performs a preprocessing step before voice recognition in a voice signal transmitted from the sound output apparatus 200 through the server communication unit 310. [ For example, a preprocessing process such as removing noise from a speech signal can be performed. In particular, the preprocessing module 331 of the present invention performs a preprocessing process using voice signal information transmitted from the sound output device 200. That is, the preprocessing module 331 of the present invention does not perform a separate process or calculation process for performing the preprocessing, but uses the direction of the voice signal information transmitted from the sound output device 200 and the noise information, It is possible to perform a noise removal process using a configured filter and to perform a preprocessing process for compensating a channel variation of a voice signal using channel information of the voice signal information .

The voice recognition module 332 recognizes the voice signal transmitted through the preprocessing module 331. For example, the speech recognition module 332 of the present invention extracts a speech feature vector from the received speech signal and confirms a phoneme corresponding to the extracted speech feature vector. Here, a phoneme refers to a minimum unit (eg, a, d, ㅓ, ㅕ, ㅁ) that distinguishes meaning, and a phoneme recognition corresponding phonetic feature vector can be converted into a set of such units . Thereafter, the speech recognition module 332 of the present invention can construct a sentence composed of word strings having meaning using the set of recognized phonemes. In addition, the speech recognition module 332 of the present invention may determine a suitable speech recognition model using the speech signal information transmitted from the sound output apparatus 200. [

The semantic analysis module 333 can determine whether or not a wakeup word is included based on the speech recognition result based on the speech recognition module 332. The semantic analysis module 333 analyzes each word string in the speech recognition sentence, It analyzes whether the column corresponds to an intent corresponding to an entity. For example, it is assumed that the user has uttered the phrase "EYE, now the weather is good" near the sound output apparatus 200. At this time, the word sequence of 'now' and 'how' is an intent, and the word sequence corresponding to the weather can be analyzed as an entity. At this time, the semantic analysis module 334 of the present invention grasps the user's more accurate intention of the entity with reference to <Table 1> and <Table 2>, and transmits the determined result to the service processing module 334 .

The service processing module 334 first identifies the service providing entity corresponding to the entity. In the above example, the content providing server A 400a can be confirmed as a service providing entity corresponding to the 'weather' entity. The service processing module 334 can grasp the intention of the user through the intent corresponding to 'now' and 'how' to the contents providing server A 400a and inquire weather information of the current time. At this time, the service processing module 334 of the present invention confirms the identification information of the sound output apparatus 200 and confirms the service subscription information corresponding to the identification information through the server storage unit 320. According to the confirmed information, the current position of the sound output apparatus 200 can be confirmed, and the weather information at the identified position can be inquired.

When the corresponding content is delivered from the content providing server 400a through the content server interworking unit 350, the service processing module 334 composes a response message including the corresponding content. That is, if the 'content' is transmitted through the content server interworking unit 350, the service processing module 334 includes the current location and time information of the user, and the response information transmitted from the content providing server a 400a , And a response message in the form of a sentence such as 'Seoul Gul, 2 o'clock and the weather is raining'.

The service processing module 334 of the present invention converts the response message into a voice signal through the TTS engine 341 of the conversion unit 341. Thereafter, when the converted response message is transmitted from the converting unit 340, the service processing module 334 can transmit the converted response message to the sound output apparatus 200 through the server communication unit 310. [

The service processing module 334 of the present invention receives the link information for content connection from the content providing server 400 through the content server linking unit 350 and transmits the received link information to the server communication unit 310 to the audio output apparatus 200 via the audio output apparatus 200.

A method of providing a voice recognition service using voice signal information in the service server 300 according to the present invention will be described with reference to flowcharts.

6 is a flowchart illustrating a main operation of the service server 300 according to an embodiment of the present invention. The service server 300 receives voice signal information together with a voice signal from the voice output apparatus 200 (S301 ).

Thereafter, the service server 300 of the present invention configures a filter for removing noise using the received voice signal information (S303). At this time, the service server 300 of the present invention constructs a filter of a specific frequency band for removing noise using the direction of the voice signal information and the noise information, and performs a process for removing noise from the voice signal using the configured filter (S305).

In addition, the service server 300 of the present invention can perform a preprocessing process of compensating for the channel variation of the voice signal using the channel information of the voice signal information transmitted from the sound output device 200.

Thereafter, the service server 300 of the present invention performs a process of recognizing a voice signal (S307). At this time, the service server 300 of the present invention extracts a speech feature vector from the received speech signal and confirms a phoneme corresponding to the extracted speech feature vector. Here, a phoneme refers to the smallest unit (eg, a, d, ㅓ, ㅕ, ㅁ) that distinguishes meaning. In phonemic recognition, a process of converting into a set of such units can be performed. Thereafter, the service server 300 of the present invention constructs a sentence composed of word strings having meaning by using the set of recognized phonemes, analyzes the meaning with respect to the word string (S309) (Step S311).

Here, the entity may mean a predefined proper noun such as the name of a specific service desired by the user. For example, it is possible to extract a proper noun that has been promised in advance, such as weather, latest song, pop song, time, appointment, etc., as an entity. On the other hand, an intent means a word string including a user's intention. For example, the intent may correspond to a word string for expressing a user's intention except an entity such as 'play', 'what time is' have.

If the sentence is taken as an example, if the user has uttered the sentence "Give me the latest music," the phrase 'up-to-date music' in the above sentence means an entity and 'play' means an intent.

The service server 300 of the present invention can identify the service providing entity for the latest music entity and the content providing server 400 for providing the latest music at step S313. Then, the service corresponding to the intent is inquired to the content providing server (S315). In the above example, 'play' is an intent, and the service server 300 can check the intention table corresponding to the user 'intention' and know that the intent of the user is 'play'. Thereafter, the service server 300 queries the content providing server for a playback service for 'latest music'.

The service server 300 of the present invention can receive a corresponding response from the content providing server 400 (S317). Thereafter, the service server 300 constructs a response message for transmission to the sound output apparatus 200 (S319), and converts it into a voice signal (S321). The response message may be a response message to the voice signal input by the user, and may generate a response message "Let's hear the latest music &quot;.

Then, the service server 300 transmits the converted response message to the sound output apparatus 200 (S323). The service server 300 of the present invention may transmit the link information to the sound output apparatus 200 when the link information is received from the content providing server as response information. It is also possible to generate a reproduction list and sequentially transmit the link information according to the reproduction list.

The method for providing the voice recognition service using the voice signal information according to the embodiment of the present invention has been described above.

The method for providing a voice recognition service using the voice signal information of the present invention as described above may be provided in the form of a computer readable medium suitable for storing computer program instructions and data.

In particular, the computer program of the present invention includes the steps of generating a voice signal by inputting voice of a user who is uttered through at least one multi-channel microphone, generating a voice signal of a predetermined interval for performing a wake- Extracting speech signal information including directions, channels, and noise information as objects, and if the speech signal is included in the speech signal of the predetermined interval and includes a wakeup word, the generated speech signals are sequentially streamed Transmitting the extracted voice signal information to the service server and transmitting the extracted voice signal information to the service server.

Such computer-readable media suitable for storing computer program instructions and data include, for example, magnetic media such as hard disks, floppy disks and magnetic tape, compact disk read only memory (CD-ROM) Optical media such as a DVD (Digital Video Disk), a magneto-optical medium such as a floppy disk, and a ROM (Read Only Memory), a RAM , Random Access Memory), flash memory, EPROM (Erasable Programmable ROM), and EEPROM (Electrically Erasable Programmable ROM). The processor and memory may be supplemented by, or incorporated in, special purpose logic circuits.

The computer readable recording medium may also be distributed over a networked computer system so that computer readable code can be stored and executed in a distributed manner. The functional program for implementing the present invention and the related code and code segment may be implemented by programmers in the technical field of the present invention in consideration of the system environment of the computer that reads the recording medium and executes the program, Or may be easily modified or modified by the user.

In addition, a computer program recorded on a computer-readable recording medium as described above includes instructions for performing the functions as described above, distributed and distributed through a recording medium, read and installed in a specific device, a specific computer, Thereby performing the above-described functions.

While the specification contains a number of specific implementation details, it should be understood that they are not to be construed as limitations on the scope of any invention or claim, but rather on the description of features that may be specific to a particular embodiment of a particular invention Should be understood. Certain features described herein in the context of separate embodiments may be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments, either individually or in any suitable subcombination. Further, although the features may operate in a particular combination and may be initially described as so claimed, one or more features from the claimed combination may in some cases be excluded from the combination, Or a variant of a subcombination.

Likewise, although the operations are depicted in the drawings in a particular order, it should be understood that such operations must be performed in that particular order or sequential order shown to achieve the desired result, or that all illustrated operations should be performed. In certain cases, multitasking and parallel processing may be advantageous. Also, the separation of the various system components of the above-described embodiments should not be understood as requiring such separation in all embodiments, and the described program components and systems will generally be integrated together into a single software product or packaged into multiple software products It should be understood.

The present invention relates to a method of providing a voice recognition service, and more particularly, to a service server capable of providing a voice recognition service in cooperation with an audio output device including a microphone and a speaker, And the service server can perform speech recognition more accurately by using the speech signal information. Accordingly, the present invention can provide speech recognition, artificial intelligence, multimedia It is possible to use it in various industries such as contents industry.

100: terminal 200: sound output device
210: audio signal input unit 220:
230: device communication unit 240: output unit
300: service server 310: server communication unit
320: server storage unit 330: server control unit
340: conversion unit 350: content server interface unit
400: Content providing server 500:

Claims (12)

The sound output device comprises:
Generating a voice signal by receiving a user's voice uttered through at least one multi-channel microphone;
Extracting speech signal information including direction, channel, and noise information on a speech signal of a predetermined interval for performing a wakeup process on the generated speech signal; And
If the wakeup word is included in the voice signal of the predetermined period, the voice signal is sequentially streamed to the service server, and the extracted voice signal information is transmitted to the service server ;
The method of claim 1, further comprising: The method according to claim 1,
The step of generating the speech signal
Generating a voice signal using a voice input through the at least one multi-channel microphone; And
Dividing a generated voice signal into frames of a predetermined length, and storing a frame of a predetermined interval to perform a wakeup process;
The method of claim 1, further comprising: 3. The method of claim 2,
The step of extracting the voice signal information
Extracting direction information using at least one of a arrival time, a reaching distance, and a voice strength of a voice inputted through the at least one multi-channel microphone;
Checking and extracting channel information of the multi-channel microphone; And
Extracting a speech feature vector for each of the stored frames and extracting a signal-to-noise ratio (SNR) using the extracted speech feature vector;
The method of claim 1, further comprising: 3. The method of claim 2,
The step of transmitting to the service server
And transmitting the streaming signal from the audio signal corresponding to the stored frame to the service server or streaming the generated audio signal after the stored frame to the service server and transmitting the streaming signal. Way. The method according to claim 1,
The step of transmitting to the service server
And transmitting the extracted voice signal information to the service server by including the extracted voice signal information in the header information of the voice signal that is streamed and transmitted. The service server,
Receiving voice signal information extracted for a voice signal of a predetermined interval for performing a wakeup process with a voice signal streamed from an audio output device;
Performing a preprocessing process for noise removal and disparity compensation in the streamed audio signal using the audio signal information; And
Analyzing the meaning of the speech signal subjected to the preprocessing process, and delivering the corresponding speech recognition service to the sound output device according to the analyzed result;
The method of claim 1, further comprising: The method according to claim 6,
The voice signal information
A channel, and noise information extracted from the voice signal of the predetermined period. 8. The method of claim 7,
The step of performing the pre-
A filter for a specific frequency band for removing noise from the voice signal is constructed using the direction information and the noise information, noise is removed using the configured filter, and a channel variation of the voice signal is calculated using the channel information, And a preprocessing step of compensating for the speech recognition service. A computer-readable recording medium storing a program for executing a method of providing a speech recognition service utilizing the speech signal information according to any one of claims 1 to 8. A voice signal input unit for receiving a voice of a user who is ignited through at least one multi-channel microphone and generating a voice signal; And
Extracting speech signal information including direction, channel, and noise information on a speech signal of a predetermined interval for performing a wakeup process from a speech signal transmitted through the speech signal input unit, A device controller for sequentially streaming the generated voice signal to the service server and controlling the extracted voice signal information to be transmitted to the service server when the voice signal includes a wakeup word;
And a sound output unit for outputting sound. A server communication unit for receiving a voice signal streamed from an audio output apparatus and receiving voice signal information extracted from a voice signal of a predetermined section for performing a wakeup process from the sound output apparatus; And
A preprocessing process for noise removal and disparity compensation is performed on the streamed audio signal using the audio signal information received through the server communication unit, the speech signal on which the preprocessing process has been performed is recognized to analyze the meaning, A server control unit for controlling the voice output unit to transmit the corresponding voice recognition service to the sound output unit according to the result;
The service server comprising: 12. The method of claim 11,
The server control unit
A filter for a specific frequency band for removing noise in the voice signal is configured using direction information and noise information included in the voice signal information, noise is removed using the configured filter,
And performs a preprocessing process of compensating a channel variation of the voice signal using the channel information included in the voice signal information.

Images