KR102527346B1 - Voice recognition device for vehicle, method for providing response in consideration of driving status of vehicle using the same, and computer program - Google Patents

Abstract

A method of providing a response in consideration of the driving state of a vehicle by a voice recognition device includes the steps of receiving a voice according to the utterance of a speaker in the vehicle, converting the received voice into text, determining the current driving state of the vehicle, and selecting the speaker from the text. Analyzes the ignition intention and determines whether the speaker's ignition is a general request for general vehicle actions unrelated to the speaker's safety or an urgent request for vehicle action related to the speaker's safety. The method may include determining whether the request is an emergency request and providing guidance for handling a problem situation of the vehicle according to a general request or an emergency request.

Description

[0001] Voice recognition device for vehicle, method for providing response in consideration of driving status of vehicle using the same, and computer program

The present invention relates to a voice recognition device for a vehicle, a method, and a computer program for responding in consideration of the driving state of the vehicle based on the voice of a speaker in the vehicle.

As voice recognition technology has recently developed, the application target of the technology is also gradually expanding. A typical example is a vehicle.

Voice recognition technology applied to a vehicle recognizes the voice of a driver or passenger driving the vehicle and performs a function corresponding to the voice. In this case, even if the driver is not familiar with operating various equipment provided in the vehicle, the vehicle function can be easily selected through voice, and the driver does not need to use his or her hands and can concentrate more on driving.

However, such conventional vehicle voice recognition technology has been limited to operation of vehicle functions or provision of driving environment information, such as navigation operation, radio operation, air conditioner operation, connection to a phone call, weather information, and the like.

While the driver is using the vehicle, various problem situations (eg, a steering wheel does not operate or a wiper does not operate) may occur in which vehicle-equipped devices do not operate normally. Existing vehicle voice recognition technology It was not possible to provide an appropriate action plan for this.

In addition, even in the same problem situation, the action plan may vary according to the vehicle driving state (eg, driving state, stopping state, etc.), but the existing vehicle voice recognition technology has not provided an appropriate action plan for this.

Therefore, in order to further advance the in-vehicle voice recognition technology, a solution to this problem is required.

The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to request a general vehicle action unrelated to the speaker's safety based on the current driving state of the user's ignition in the vehicle or the speaker's An object of the present invention is to provide a voice recognition device for a vehicle, a method, and a computer program for providing a response by distinguishing whether an urgent vehicle action is requested in relation to safety.

It is also an object of the present invention to provide a voice recognition apparatus for a vehicle, a method and a computer program for providing a response corresponding to each utterance according to the degree of urgency among multiple utterances of a user in a vehicle.

In order to achieve the above object, a method for providing a response in consideration of a driving state of a vehicle according to an embodiment of the present invention includes the steps of receiving a voice according to an utterance of a speaker in a vehicle, converting the received voice into text, and the Determining the current driving state of the vehicle, analyzing the speaker's utterance intention from the text, and based on the determined vehicle driving state, the speaker's utterance requires a general vehicle action unrelated to the speaker's safety Determining whether it is a general request or an emergency request that urgently requires vehicle measures in relation to the safety of the speaker, and providing guidance for handling a problem situation of the vehicle according to the general request or emergency request. there is.

Also, the driving state may include a state in which the vehicle is driving, a state in which the vehicle is stopped, and a state in which the vehicle is parked.

In the determining step, the intention of the speaker is analyzed using a speaker-dependent natural language understanding model, and the speaker's utterance is divided into the general request or the emergency request according to the driving state of the vehicle. .

The method may further include determining a current location of the vehicle, and the determining may include calculating a degree of urgency according to a correlation between the speaker's utterance, the determined current location, and the driving state, and the calculated urgency. The method may further include classifying the utterance of the speaker into the general request or the urgent request by comparing the degree of urgency with a preset reference value.

Further, in the providing of the guidance, when the speaker utters a plurality of utterances, guidance corresponding to each utterance may be provided according to the degree of urgency among the plurality of utterances.

Also, in the providing of the guidance, when the speaker's utterance is determined to be an emergency request, at least one of an emergency action method, an ARS connection service, and a rescue request service may be provided according to the degree of urgency.

On the other hand, the voice recognition device includes a voice input unit for receiving voice according to the utterance of a speaker in the vehicle, a voice-to-text converter for converting the received voice into text, a driving state determination unit for determining the current driving state of the vehicle, The speaker's utterance intention is analyzed from the text, and based on the determined driving state of the vehicle, whether the speaker's utterance is a general request requiring a general vehicle action unrelated to the speaker's safety or an emergency related to the speaker's safety It may include a natural language understanding unit that determines whether it is an emergency request to take action of the vehicle, and a control unit that provides guidance for handling a problem situation of the vehicle according to the general request or emergency request.

Also, the driving state may include a state in which the vehicle is driving, a state in which the vehicle is stopped, and a state in which the vehicle is parked.

The controller may further include a location determination unit that determines the current location of the vehicle, wherein the control unit calculates a degree of urgency according to a correlation between the speaker's speech, the determined current location, and the driving state, and the natural language understanding unit calculates the The utterance of the speaker may be classified as the general request or the emergency request by comparing the calculated urgency with a preset reference value.

In addition, when the speaker utters a plurality of utterances, the control unit may provide guidance corresponding to each utterance according to the urgency among the plurality of utterances.

In addition, when it is determined that the speaker's utterance is an emergency request, the controller may provide at least one of an emergency action method, an ARS connection service, and a rescue request service according to the degree of urgency.

Meanwhile, a program stored in a recording medium according to an embodiment of the present invention for achieving the above object may include a program code for executing the above-described response method considering the driving state of the vehicle.

According to the present invention, it is possible to provide an appropriate processing method for a corresponding problem situation based on the speech of a speaker in a vehicle.

In addition, according to the present invention, considering the driving state of the vehicle, whether the speaker's utterance is an emergency request that urgently requires action in relation to the speaker's safety or a general request that does not urgently require action in relation to the speaker's safety is distinguished. and can provide a response that meets the speaker's utterance intention.

In addition, according to the present invention, when there are a plurality of utterances of a speaker, the speaker may preferentially solve an urgent problem situation by providing a response corresponding to the utterance according to the degree of urgency.

1 is a diagram showing an example of utilization of a vehicle voice recognition apparatus according to an embodiment of the present invention.
2 is a block diagram illustrating a vehicle voice recognition apparatus according to an embodiment of the present invention.
3 is a block diagram showing a voice processing unit according to an embodiment of the present invention in more detail.
4 and 5 are flowcharts illustrating a method for providing a response according to an embodiment of the present invention.

The following merely illustrates the principles of the present invention. Therefore, those skilled in the art can invent various devices that embody the principles of the present invention and fall within the concept and scope of the present invention, even though not explicitly described or shown herein. In addition, all conditional terms and embodiments listed in this specification are, in principle, expressly intended only for the purpose of understanding the concept of the present invention, and should be understood not to be limited to such specifically listed embodiments and conditions. do.

In addition, it should be understood that all detailed descriptions reciting specific embodiments, as well as principles, aspects and embodiments of the present invention, are intended to encompass structural and functional equivalents of these matters. In addition, it should be understood that such equivalents include not only currently known equivalents but also equivalents developed in the future, that is, all devices invented to perform the same function regardless of structure.

Thus, for example, the block diagrams herein are to be understood as representing conceptual views of exemplary circuits embodying the principles of the present invention. Similarly, all flowcharts, state transition diagrams, pseudo code, etc., are meant to be tangibly represented on computer readable media and represent various processes performed by a computer or processor, whether or not the computer or processor is explicitly depicted. It should be.

The functions of various elements shown in the drawings including functional blocks represented by processors or similar concepts may be provided using dedicated hardware as well as hardware capable of executing software in conjunction with appropriate software. When provided by a processor, the functionality may be provided by a single dedicated processor, a single shared processor, or a plurality of separate processors, some of which may be shared.

In addition, the explicit use of terms presented as processor, control, or similar concepts should not be construed as exclusively citing hardware capable of executing software, but without limitation, digital signal processor (DSP) hardware, ROM for storing software (ROM), random access memory (RAM) and non-volatile memory. Other hardware for the governor's use may also be included.

In the claims of this specification, components expressed as means for performing the functions described in the detailed description include, for example, a combination of circuit elements performing the functions or all types of software including firmware/microcode, etc. It is intended to include any method that performs the function of performing the function, combined with suitable circuitry for executing the software to perform the function. Since the invention defined by these claims combines the functions provided by the various enumerated means and is combined in the manner required by the claims, any means capable of providing such functions is equivalent to that discerned from this specification. should be understood as

The above objects, features and advantages will become more apparent through the following detailed description in conjunction with the accompanying drawings, and accordingly, those skilled in the art to which the present invention belongs can easily implement the technical idea of the present invention. There will be. In addition, in describing the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram showing an example of utilization of a vehicle voice recognition apparatus according to an embodiment of the present invention. Referring to FIG. 1 , a speaker 10 located inside a vehicle may input a voice to a voice recognition device (not shown) through speech in order to use a voice recognition function of a voice recognition device (not shown) provided in the vehicle. Here, the speaker 10 may mean a person located inside the vehicle, such as a vehicle driver or a vehicle passenger.

The speaker's voice may be input to a voice recognition device provided in the vehicle in various driving situations while the vehicle is parked, stopped, and driven.

At this time, the speech of the speaker 10 for using the voice recognition function of the vehicle can be largely classified into two types.

The first type of ignition is “Turn on the radio (11)”, “The room temperature is 22 degrees (12)”, “Guide me to my house (13)”, “Tell me the weather today (14)” according to the driving condition of the vehicle. As an unaffected utterance, it may be an utterance for simple manipulation.

If the speaker 10 utters “Play the radio 11”, the voice recognition device recognizes the speaker 10's voice and sends a request corresponding to the recognized voice command to the vehicle's ECU (Electronic Control Unit). ), and the ECU can control the power of the vehicle's radio module to turn on.

The second type of utterance is a response that can vary depending on the driving condition of the vehicle, such as "I can't start (21)," "The steering wheel doesn't move (22)," and "The lights don't turn on (23)." As the ignition, it may be an ignition for notifying that the vehicle is in a problem situation and being guided to a solution therefor.

If the speaker 10 utters something like “The steering wheel does not move (22)”, the voice recognition device recognizes the speaker 10's utterance, and the driving state of the vehicle (ie, whether the vehicle is driving or not, the vehicle Based on whether the vehicle is stopped or the vehicle is parked, a solution for solving the corresponding problem situation is detected from a pre-stored database, and the detected solution can be guided to the speaker 10 through voice or display.

For example, when the steering wheel does not move while driving the vehicle and when the steering wheel does not move while the vehicle is stopped or parked, response methods are different, and when the voice recognition device does not provide an accurate response corresponding to the situation, the speaker (10 ) may result in a safety-threatening situation.

Accordingly, the voice recognition apparatus according to the present invention determines whether the voice received from the speaker in the vehicle is a general request requiring a general vehicle action unrelated to the speaker's safety or related to the speaker's safety, based on the driving state. This problem can be solved by determining whether it is an emergency request that urgently requires the vehicle to take action.

The voice recognition device according to the present invention will be described in more detail with reference to the accompanying drawings.

2 is a block diagram illustrating a voice recognition device according to an embodiment of the present invention. Referring to FIG. 2 , the voice recognition apparatus 100 may include all or part of a voice processing unit 110, a driving state determination unit 120, a location determination unit 130, and a control unit 140.

The voice processing unit 110 receives a voice according to the speaker's speech in the vehicle, and based on the received voice and the driving state of the vehicle, whether the speaker's speech is a general request requiring a general vehicle action unrelated to the speaker's safety or It may be determined whether the request is an emergency request that urgently requires vehicle measures in relation to the safety of the speaker.

The operation of the voice processing unit 110 will be described with reference to FIG. 3 .

3 is a block diagram showing a voice processing unit according to an embodiment of the present invention in more detail. Referring to FIG. 3 , the voice processing unit 110 includes a voice input unit 111, a voice-to-text conversion unit 112, a word correction unit 113, a natural language understanding unit 114, a natural language generator 115, and a voice synthesizer. It may include all or part of section 116 .

The voice input unit 111 may receive an input voice spoken by a speaker in the vehicle. To this end, the voice input unit 111 may include one or more microphones (not shown). In addition, the voice input unit 111 may use various noise removal algorithms to remove noise generated in the process of receiving the user's spoken voice. Specifically, the voice input unit 111 may include a filter (not shown) that removes noise of spoken voice, an amplifier (not shown) that amplifies and outputs a signal output from the filter, and the like.

The voice-to-text conversion unit 112 may convert a series of spoken voice signals input through the voice input unit 111 into text. The voice recognition unit may include an acoustic model, a language model, and a pronunciation model.

The acoustic model may include a statistical model of speech created by training how phonemes are uttered based on a plurality of speaker utterance data. This acoustic model may generate phoneme text based on the patient's voice input through the voice input unit 111 . Here, the acoustic model may be implemented in various ways, such as Hidden Markov Model (HMM), Gaussian mixture emissions (GMM)-HMM, and Deep Neural Network (DNN)-HMM obtained by applying a deep neural network to HMM. Here, the acoustic model may use, for example, a melfrequency cepstral coefficient (MFCC), which is the most representative method as a frequency domain energy estimation method, as a feature.

In addition, the language model may include an algorithm for finding regularities of grammar, syntax, words, etc. in natural language and using the regularities to increase the accuracy of a target to be searched for. At this time, a commonly used method is a statistical modeling technique that calculates probability values, which may include a method of representing language rules as probabilities in a speech signal input through a large corpus) and limiting the search area through probability values. can For example, the language model may use N-Gram.

In addition, the pronunciation model is a pronunciation dictionary model that converts text into phonemes (G2P: Grapheme-to-Phoneme), and can build a pronunciation dictionary not only for standard pronunciation but also for dialects.

The speech-to-text conversion unit 112 may use a speaker-independent speech recognition model that is not dependent on a specific speaker.

Meanwhile, when voice is converted into text through the voice-to-text conversion unit 112, the converted text may be input to the word correction unit 113.

In this case, the word correction unit 113 may correct the vehicle-related words included in the text to a preset vehicle-related word.

Specifically, the word correction unit 113 may divide text into word units and detect vehicle-related words from among the separated words. In this case, the word compensator 113 may separate the words by setting the minimum independent form in which at least one word is combined in word units.

Also, the word compensator 113 may determine whether the detected vehicle-related word is a standard word or a non-standard word by applying the detected vehicle-related word to a vehicle-related word database.

If the detected vehicle-related words are standard words, the word correction unit 113 may not correct the detected vehicle-related words.

However, when the detected vehicle-related words are non-standard words, the word correction unit 113 may correct the detected vehicle-related words to standard words by using the vehicle-related word database.

Here, the standard words refer to words commonly used in the vehicle industry, and the non-standard words refer to words not generally used in the vehicle industry.

In addition, the vehicle-related word database is high by matching non-standard words with standard words, and the word correction unit 113 can learn the vehicle-related word database based on a plurality of vehicle-related terms or situation description words used in the vehicle. there is.

For example, in a situation where the instrument panel of the vehicle does not operate properly, a vehicle driver who does not know the word “instrument panel” may say something like “the screen in front of the steering wheel does not operate” while describing the situation. In this case, the input voice may be converted into text through the voice-to-text conversion unit 112 and input into the word correction unit 113 . In this case, the word correction unit 113 applies the non-standard word "screen in front of the steering wheel" to the database of vehicle-related words, detects the corresponding standard word "instrument panel", corrects "screen in front of the steering wheel" to "instrument panel", It can be modified to "The instrument panel does not work".

As another example, in a situation where the vehicle's wipers are not working properly, a vehicle driver who does not know the word "wiper" may describe the situation and say something like "the windshield wiper is slow". In this case, the input voice may be converted into text through the voice-to-text conversion unit 112 and input into the word correction unit 113 . In this case, the word correction unit 113 applies the non-standard word "vehicle windshield wiper" to the vehicle-related word database to detect the corresponding standard word "wiper", and converts "vehicle windshield wiper" to "wiper". It can be modified to “Wipers are slow”.

According to the present invention, by using the database learned in the word correction unit 113 to perform preprocessing of converting non-standard words into standard words, it is possible to help reduce the weight of the natural language understanding unit 114 to be described later. In addition, this can improve the performance of the natural language understanding unit 114 to more accurately grasp the speaker's utterance intention in the vehicle.

Meanwhile, the text corrected by the word correction unit 113 may be input to the natural language understanding unit 114 .

The natural language understanding unit 114 may perform processing to understand the meaning of text input from the word correction unit 113 . In detail, the natural language understanding unit 114 may perform syntactic analysis or semantic analysis on text generated as a result of speech recognition to analyze the user's speech intention.

Here, the grammatical analysis may divide the query text into grammatical units (eg, words, phrases, morphemes, etc.) and determine which grammatical elements the divided units have.

In addition, semantic analysis may be performed using semantic matching, rule matching, formula matching, and the like.

Accordingly, the natural language understanding unit 114 may analyze the intention of the text generated as a result of recognizing the voice of the speaker in the vehicle. In particular, the natural language understanding unit 114 analyzes the speaker's utterance intention from the text based on the driving state of the vehicle, and determines whether the speaker's utterance is a general request for a general vehicle action unrelated to the speaker's safety or a concern for the speaker's safety. In relation to this, it may be determined whether the request is an emergency request that urgently requires the vehicle to take action. Here, the driving state may include one of a current driving state, a parking state, a stopping state, and a parking state.

For example, when the text generated as a result of voice recognition of a speaker in a vehicle is “steering wheel is not moving” and the driving state of the vehicle is driving, the natural language understanding unit 114 determines that the speaker's speech is safe for the speaker. It can be determined as an urgent request that urgently requires vehicle measures in relation to However, when the text generated as a result of the voice recognition of the speaker in the vehicle is “steering wheel is not moving” and the driving state of the vehicle is parked, the natural language understanding unit 114 determines that the speaker's utterance is irrelevant to the speaker's safety. It can be judged as a general request requiring a general vehicle action.

As another example, when the text generated as a result of voice recognition of a speaker in the vehicle is “the accelerator pedal is not working” and the driving state of the vehicle is driving, the natural language understanding unit 114 determines that the speaker's speech is important to the speaker's safety. In this regard, it may be determined as an urgent request that urgently requires the vehicle to take action. However, when the text generated as a result of voice recognition of the speaker in the vehicle is “the accelerator pedal is not working” and the driving state of the vehicle is parked, the natural language understanding unit 114 determines that the speaker's speech is irrelevant to the speaker's safety. It can be judged as a general request requiring a general vehicle action.

In addition, the natural language understanding unit 114 additionally considers at least one of the current location of the vehicle and surrounding environment information to determine whether the speaker's speech is a general request that requires a general vehicle action unrelated to the speaker's safety or It may be determined whether the request is an emergency request that urgently requires vehicle measures in relation to the safety of the speaker. Here, the surrounding environment information may be received from an external server or pre-stored, and the surrounding environment information may include weather around the vehicle, current time, and the like.

For example, when the text generated as a result of voice recognition of a speaker in a vehicle is "lights are not turned on" and the current location of the vehicle is in a tunnel, the natural language understanding unit 114 determines that the speaker's speech is safe for the speaker. It can be determined as an urgent request that urgently requires vehicle measures in relation to However, when the text generated as a result of the voice recognition of the speaker in the car is "The light is not turned on", and the current location of the vehicle is outdoors in broad daylight, the natural language understanding unit 114 determines that the speaker's utterance is safe for the speaker. It can be judged as a general request that requires general vehicle measures unrelated to

As another example, when the text generated as a result of voice recognition of a speaker in the vehicle is “The heater is not working” and the temperature around the vehicle is minus 20 degrees, the natural language understanding unit 114 determines that the speaker's speech is related to the speaker's safety. Therefore, it can be determined as an emergency request that urgently requires the vehicle to take action. However, when the text generated as a result of the voice recognition of the speaker in the vehicle is "The heater is not working" and the temperature around the vehicle is 20 degrees, the natural language understanding unit 114 determines that the speaker's speech is irrelevant to the speaker's safety. It can be judged as a general request requiring general vehicle action.

In addition, the natural language understanding unit 114 further considers the type of vehicle function module selected by the speaker to determine whether the speaker's speech is a general request requiring a general vehicle action unrelated to the speaker's safety or the speaker's speech. In relation to safety, it may be determined whether the request is an emergency request that urgently requires vehicle measures. Here, the vehicle function module may be a concept including all unit modules constituting vehicle functions, such as air conditioner, transmission, steering wheel, infotainment, audio, cluster, direction indicator, window, and seat.

For example, when texts generated as a result of voice recognition of a speaker in a vehicle are “the steering wheel is strange” and “the air conditioner is strange”, the natural language understanding unit 114 determines that the “steering wheel”, which is a function module of the vehicle, is It is determined that it is related to driving and has a high relevance to safety, and “air conditioner” is for an indoor environment and is determined to have a low relevance to safety. That is, the natural language understanding unit 114 may determine that the speaker's utterance is an urgent request for urgently requiring vehicle measures in relation to the speaker's safety with respect to "the steering wheel is abnormal", and the air conditioner is abnormal" For this case, the speaker's utterance can be determined as a general request requesting a general vehicle action unrelated to the speaker's safety.

Specifically, the driving state determination unit 120 and the position determination unit 130 of the voice recognition apparatus 100 may respectively determine the driving state and the current position of the vehicle when the speaker speaks, and the voice recognition apparatus 100 The control unit 140 of the vehicle may calculate the degree of urgency according to the correlation between the driving state of the vehicle, the current location, surrounding environment information, and the type of vehicle function module uttered by the speaker. At this time, the control unit 140 may use a neural network learned to calculate the degree of urgency as learning data of the speaker's spoken text labeled with the driving state, current location, surrounding environment information, and the type of vehicle function module uttered by the speaker. there is. Here, the degree of urgency is information indicating the degree of urgency of the speaker's utterance, and may indicate more urgency as the urgency value increases.

In addition, the neural network may be learned through supervised learning, unsupervised learning, and reinforcement learning.

In addition, the natural language understanding unit 114 may compare the calculated urgency value with a preset reference value to classify the speaker's utterance as a general request or an urgent request. Here, the reference value is a reference value for distinguishing a general request from an emergency request, and may be a variable value determined according to vehicle type, current location, surrounding environment information, driving state, age and gender of a speaker.

For example, when the urgency calculated for the speaker's utterance is greater than or equal to the reference value, the natural language understanding unit 114 may determine the speaker's utterance as an urgent request.

That is, the natural language understanding unit 114 may determine the utterance of the same speaker as a general request or an emergency request according to the driving state of the vehicle, the current location, and surrounding environment information.

Meanwhile, the natural language understanding unit 114 may distinguish the speaker's utterance by using a database in which general requests or emergency requests are matched according to the driving state of the vehicle, the current location, and surrounding environment information for each speaker's utterance.

Meanwhile, the natural language generation unit 115 may generate a response text using a knowledge-base based on the utterance intention analyzed by the natural language understanding unit 114 .

The text to speech (TTS) 116 may generate a response speech voice for the response text in the form of natural language speech generated by the natural language generator 115 . The voice generated by the voice synthesis unit 116 may be provided as a voice to a speaker in the vehicle.

Meanwhile, the above-described natural language understanding unit 114 may determine whether the request is a general request or an emergency request by analyzing a speaker's utterance intention using a natural language understanding model.

Here, the natural language understanding model of the natural language understanding unit 114 according to the present invention may be a speaker-dependent natural language understanding model classified according to the identity of the speaker and whether or not the speaker is driving.

For example, when there are two people using the vehicle, “A” and “B,” the natural language understanding unit 114 according to the present invention performs “when A is a driver”, “when A is a passenger”, and “when B is a driver”. A speaker-dependent natural language understanding model according to each case can be built by dividing into "when B is a passenger" and "when B is a passenger."

That is, depending on whether the speaker is driving or not, voice characteristics such as speed, clarity, tone, pitch, loudness, and nuance of the speaker's voice may differ from each other, and the words used, vocabulary, and grammatical construction may also vary. These characteristics may vary depending on who the speaker is.

Accordingly, the voice recognition apparatus 100 according to the present invention can determine whether the speaker is driving based on the location where the utterance occurred, and can identify the speaker's identity according to the voice, pitch, and tone of the utterance. In addition, the natural language understanding unit 114 may analyze the speaker's utterance intention in the vehicle using a speaker-dependent natural language understanding model classified according to the identity of the speaker and whether or not the speaker is driving. Accordingly, the natural language understanding unit 114 according to the present invention can more accurately grasp the speaker's utterance intention in the vehicle.

In addition, since the number of people using each vehicle is limited, according to the present invention, it is possible to build a model optimized for the use environment in the vehicle while increasing accuracy.

Meanwhile, the control unit 140 of the voice recognition apparatus 100 may provide guidance corresponding to the utterance of the speaker identified by the natural language understanding unit 114 .

As a remedy, if the speaker's utterance is determined to be a general request, the controller 140 provides a guide for processing the general request to the speaker, and if the speaker's utterance is determined to be an emergency request, the controller 140 provides instructions for processing the emergency request. Guidance may be provided to the speaker.

For example, if the speaker's utterance "I can't start the car" is determined as a general request, the controller 140 may provide instructions on how to start the engine, and if the speaker's utterance is determined as an emergency request, the controller 140 ( 140) may guide at least one or more of an emergency action method, an ARS connection service, and a rescue request service according to the level of urgency.

For example, in the case of an urgent request. The control unit 140 classifies the urgency into a plurality of urgency levels, controls to provide an emergency action method in the case of the first urgency having the lowest urgency, and provides a second urgency higher than the first urgency. In case of , it is controlled to be automatically connected to an emergency agent and ARS, and in the case of a third level of urgency, which has the highest level of urgency, it is possible to control a call to be automatically connected to an emergency rescue request service such as 119.

Meanwhile, when a speaker utters a plurality of utterances, the voice recognition apparatus 100 according to the present invention may provide guidance corresponding to each utterance according to the degree of urgency.

Specifically, the control unit 140 of the voice recognition apparatus 100 may calculate urgency levels corresponding to multiple utterances of speakers, and provide guidance corresponding to each utterance in the order of the calculated urgency levels.

For example, when the text generated as a result of voice recognition of a speaker in a vehicle is "Ignition does not turn on, heater does not work", the urgency value of "Ignition does not turn on" is higher than the urgency value of "heater does not work" In this case, the controller 140 may guide the processing method corresponding to "the engine does not turn on" and then guide the processing method corresponding to "the heater does not work" to the speaker.

In this way, when several situations occur at the same time, an appropriate response can be provided to the speaker by guiding a processing method corresponding to the utterance in the order of urgency of the utterance.

Hereinafter, a response providing method according to an embodiment of the present invention will be described with reference to a flow chart with reference to the subsequent drawings. Here, the described order is only an example of the present invention, and is not limited thereto.

4 is a flowchart schematically illustrating a response providing method according to an embodiment of the present invention. Referring to FIG. 4 , first, a voice according to an utterance of a speaker in a vehicle may be received (S110).

Then, the received voice may be converted into text (S120). Specifically, in step S120, a series of input spoken voice signals may be converted into text using an acoustic model, a language model, and a pronunciation model.

In addition, the speaker's utterance intention is analyzed from the text, and based on the driving condition of the vehicle, whether the speaker's utterance is a general request requiring general vehicle measures unrelated to the speaker's safety or an urgent vehicle related to the speaker's safety. It may be determined whether it is an urgent request requiring action (S130). Here, in step S130 of determining, the intention of the speaker may be analyzed using a natural language understanding model, and the natural language understanding model may be a speaker-dependent natural language understanding model classified according to the identity of the speaker and whether or not the speaker is driving.

If the speaker's utterance is determined to be a general request (S140:Y), guidance for handling the general request may be provided to the speaker (S150). For example, a solution to a problem situation of a vehicle may be provided to a speaker through voice or a display.

However, when it is determined that the speaker's utterance is an emergency request (S140:N), guidance for processing the emergency request may be provided to the speaker (S160). For example, in the case of an urgent request. It can be classified into multiple emergency levels according to the level of urgency, provide emergency measures according to the level of urgency, automatically connect ARS with an emergency counselor, or automatically connect calls to emergency rescue services such as 119. .

Meanwhile, although not shown in FIG. 5 , the method according to an embodiment of the present invention detects vehicle-related words included in the text between steps S120 and S130 and corrects the detected vehicle-related words with preset standard words. Further steps may be included.

5 is a flowchart illustrating a method for providing a response according to an embodiment of the present invention. Referring to FIG. 5 , the degree of urgency may be calculated according to the correlation between the driving state of the vehicle, the current location, and surrounding environment information (S210). Specifically, the degree of urgency may be calculated according to the correlation between the driving state of the vehicle, the current location, surrounding environment information, and the type of vehicle function module uttered by the speaker. At this time, the calculation of the degree of urgency may use a neural network learned to calculate the degree of urgency using the driving state, the current location, surrounding environment information, and the speaker's utterance text labeled with the type of vehicle function module uttered by the speaker as learning data. there is.

If the calculated urgency value is greater than the preset reference value (S220: Y), the utterance of the corresponding speaker may be determined as an emergency request (S230).

However, if the calculated urgency value is smaller than the preset reference value (S220: N), the utterance of the corresponding speaker may be determined as a general request (S240). And, it can be performed from step S140 described above.

According to the present invention, it is possible to provide an appropriate processing method for a corresponding problem situation based on the ignition of a speaker in the vehicle.

In addition, according to the present invention, considering the driving state of the vehicle, whether the speaker's utterance is an emergency request that urgently requires action in relation to the speaker's safety or a general request that does not urgently require action in relation to the speaker's safety is distinguished. and can provide a response that meets the speaker's utterance intention.

In addition, according to the present invention, when there are a plurality of utterances of a speaker, the speaker may preferentially solve an urgent problem situation by providing a response corresponding to the utterance according to the degree of urgency.

On the other hand, terms such as "first", "second", "third" and "fourth" in the specification and claims, if any, are used to distinguish between similar components, but not necessarily Used to describe a specific sequence or order of occurrence. The terms so used will be understood in light of the embodiments of the invention described herein. Likewise, where a method is described herein as comprising a series of steps, the order of those steps presented herein is not necessarily the order in which those steps may be performed, and any recited steps may be omitted and/or here Any other step not described may be added to the method. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention.

In addition, terms such as "left", "right", "front", "rear", "top", "bottom", "above", "below" in the specification and claims are used for explanation, It is not necessarily intended to describe an invariant relative position. It will be understood that the terms so used are interchangeable under appropriate circumstances such that the embodiments of the invention described herein may, for example, operate in directions other than those shown or described herein. As used herein, the term "connected" is defined as being directly or indirectly connected in an electrical or non-electrical manner. Objects described herein as "adjacent" to each other may be in physical contact with each other, in close proximity to each other, or in the same general scope or area as is appropriate for the context in which the phrase is used. The presence of the phrase “in one embodiment” herein refers to the same embodiment, although not necessarily.

In addition, in the specification and claims, 'connected', 'connected', 'engaged', 'fastened', 'coupled', 'coupled', etc., refer to various variations of these expressions directly with other components. It is used in the meaning of being connected or indirectly connected through other components.

On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

In addition, the suffixes "module" and "unit" for components used in this specification are given or used interchangeably in consideration of ease of writing the specification, and do not have meanings or roles that are distinct from each other by themselves.

Also, the terms used in this specification are for describing the embodiments and are not intended to limit the present invention. Singular expressions used herein include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "consisting of" or "comprising" should not be construed as necessarily including all of the various components or steps described in the specification, and some of the components or some of the steps It should be construed that it may not be included, or may further include additional components or steps.

So far, the present invention has been looked at with respect to its preferred embodiments. All embodiments and conditional examples disclosed throughout this specification are described with the intention of helping readers to understand the principles and concepts of the present invention by those skilled in the art having ordinary knowledge in the technical field of the present invention, and those skilled in the art It will be understood that it can be implemented in a modified form within the range that does not deviate from the essential characteristics of the present invention.

Therefore, the disclosed embodiments should be considered from an illustrative rather than a limiting point of view. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the equivalent scope will be construed as being included in the present invention.

Meanwhile, the above-described response providing method considering the driving state of the vehicle according to various embodiments of the present invention may be implemented as a program and provided to servers or devices. Accordingly, each device may access a server or device in which the program is stored and download the program.

In addition, the method according to various embodiments of the present invention described above may be implemented as a program and stored in various non-transitory computer readable media to be provided. A non-transitory readable medium is not a medium that stores data for a short moment, such as a register, cache, or memory, but a medium that stores data semi-permanently and can be read by a device. Specifically, the various applications or programs described above may be stored and provided in non-transitory readable media such as CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In addition, although the preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention belongs without departing from the gist of the present invention claimed in the claims. Of course, various modifications are possible by those skilled in the art, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

100: voice recognition device 110: voice processing unit
120: driving state determination unit 130: position determination unit
140: control unit

Claims (11)

In the response providing method considering the driving state of the vehicle,
Receiving a voice according to an utterance of a speaker in the vehicle;
converting the received voice into text;
determining a current driving state of the vehicle;
determining the current location of the vehicle;
The speaker's utterance intention is analyzed from the text, and based on the determined driving state of the vehicle, whether the speaker's utterance is a general request requiring a general vehicle action unrelated to the speaker's safety or related to the speaker's safety Determining whether an emergency request requires an urgent action of the vehicle; and
Providing guidance for handling the problem situation of the vehicle according to the general request or emergency request; Including,
Wherein the step of determining whether the request is an emergency further considers the current location of the vehicle and surrounding environment information to determine whether the speaker's utterance is a general request or an emergency request. According to claim 1,
The driving state includes at least two of a state in which the vehicle is driving, a state in which the vehicle is stopped, and a state in which the vehicle is parked. According to claim 2,
The determining may include calculating a degree of urgency according to a correlation between the speaker's utterance, the determined current location, and the driving state; and
and classifying the speaker's utterance into the general request or the emergency request by comparing the calculated urgency with a preset reference value. According to claim 3,
The step of providing the guidance is
When the speaker makes a plurality of utterances, a guide corresponding to each utterance is provided according to the degree of urgency among the plurality of utterances. According to claim 2,
The step of providing the guidance is
When the speaker's utterance is determined to be an emergency request, at least one of an emergency action method, an ARS connection service, and a rescue request service is provided according to the degree of urgency. In the voice recognition device,
a voice input unit for receiving a voice according to an utterance of a speaker in the vehicle;
a voice-to-text conversion unit that converts the received voice into text;
a driving state determination unit for determining a current driving state of the vehicle;
a location determining unit that determines the current location of the vehicle;
The speaker's utterance intention is analyzed from the text, and based on the determined driving state of the vehicle, whether the speaker's utterance is a general request requiring a general vehicle action unrelated to the speaker's safety or related to the speaker's safety a natural language understanding unit that determines whether an emergency request requires an urgent action by the vehicle; and
Including; a control unit that provides guidance for handling a problem situation of the vehicle according to the general request or emergency request;
The voice recognition device of claim 1 , wherein the natural language understanding unit further considers the current location of the vehicle and surrounding environment information to determine whether the speaker's utterance is a general request or an emergency request. According to claim 6,
The driving state includes a state in which the vehicle is driving, a state in which the vehicle is stopped, and a state in which the vehicle is parked. According to claim 7,
The control unit calculates a degree of urgency according to a correlation between the speaker's utterance, the determined current location, and the driving state;
The voice recognition apparatus of claim 1 , wherein the natural language understanding unit compares the calculated level of urgency with a preset reference value to classify the speaker's utterance as the general request or the emergency request. According to claim 8,
The control unit,
When the speaker utters a plurality of utterances, the voice recognition apparatus characterized in that providing a guide corresponding to each utterance according to the degree of urgency among the plurality of utterances. According to claim 7,
The voice recognition device, characterized in that the control unit provides at least one of an emergency action method, an ARS connection service, and a rescue request service according to the degree of urgency when it is determined that the speaker's utterance is an emergency request. A program stored in a computer readable recording medium containing program codes for executing the response providing method in consideration of the driving state of a vehicle according to any one of claims 1 to 5.

Images