JP2006137366A - Instrument control device for vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To respond to the varied operation circumstance and to more accurately ambiguous sound/voice instruction. <P>SOLUTION: The instrument control device 10 for the vehicle has a sound/voice recognition means 50 for recognizing the sound/voice inputted from a microphone 14 and converting it to a text; an ambiguousness determination means 52 for determining that the converted/produced text is clear or ambiguous in order to select the instrument becoming an operation object; a relevancy determination means 56 for investigating relevancy of the text and the past operation with reference to an operation history memorizing part 54 in which operation performed at the past is recorded and presuming the instrument becoming the operation object from a plurality of the instruments; and a priority setting means 58 for setting priority in every instrument. The instrument control device 10 for the vehicle further has an operation instruction means 62 for performing the operation corresponding to the text by the selected instrument by selecting the instrument based on the comparison result of the priority setting means 60; and an operation circumstance presumption means 64 for detecting or presuming the operation circumstance relating to the vehicle 12. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a vehicular equipment control device that can recognize voice spoken by a passenger using voice input means such as a microphone, and can automatically select and operate equipment based on the voice.

  Recently, many electronic devices have been installed in vehicles, and their functions are becoming more sophisticated. Under such circumstances, a vehicular equipment control device that enables voice operation has been developed to facilitate the operation of electronic equipment.

  By the way, although it can be said that it is desirable in terms of voice analysis that an instruction in a voice operation is performed in an accurate language as a vehicle equipment control device, the passenger is aware that it is an instruction to the vehicle equipment control device. It is good to be able to give instructions using natural words such as daily conversation without any problems.

  In view of such a background, Patent Literature 1 and Patent Literature 2 have been proposed as a method of responding to an ambiguous voice instruction.

  According to Patent Literature 1, when an abbreviation is complemented using a dialogue history, the dialogue history is based on the passage of time, externally applied factors (pressure caused by a strong beating), and internal factors (battery remaining amount). A response sentence is created while changing the weighting of, enabling a variety of conversations.

  According to Patent Literature 2, keywords for device identification, device control, and termination are prepared, and a device once identified by the device identification keyword is operated until the termination keyword is input. It is held as a continuation operation to achieve accurate voice control.

JP 2001-188783 A JP 2002-169484 A

  According to the technique disclosed in the above-mentioned Patent Document 1, when estimating the omitted items with reference to the conversation history, only the weight of the conversation is changed according to the elapsed time, the pressure, and the remaining battery level. When mounted on a vehicle, it is not possible to appropriately cope with the driving situation that changes depending on the place of travel and time. In addition, the information processing apparatus according to this technique only creates a response sentence, and does not disclose what kind of control is performed on what kind of device in response to the voice instruction. In particular, when there are a plurality of devices to be operated, which device should be selected is not described.

  In addition, according to the technique described in Patent Document 2, a device other than the device currently being specified and controlled is recognized until an end keyword for the device or a device specifying keyword for specifying another device is recognized. Cannot be controlled. Therefore, even when a voice instruction common to each device is input and there is a voice input to a device other than the device currently being controlled, control is performed on the device currently being controlled. , Can not control according to the will of the passenger.

  For example, when it is a while after instructing “Raise the volume of the audio” and you want to raise the set temperature of the air conditioner, instruct it to “Raise more” as in daily conversation. Then, in the technique described in Patent Document 2, the volume of the audio is increased regardless of the driving situation such as the temperature, and there is a possibility that the operation does not conform to the will of the passenger. In addition, there is a burden for the passenger to learn a plurality of keywords, and further simplicity of the voice instruction method is desired.

  The present invention has been made in consideration of such problems, and provides a vehicle equipment control device that can be applied to changing driving conditions and can more accurately determine ambiguous voice instructions. With the goal.

  The vehicle equipment control device according to the present invention records a voice input means for inputting voice, a voice recognition means for recognizing the voice input by the voice input means and converting it into text, and operations performed in the past. A relationship determination unit that refers to the operation history storage unit that has been performed, examines a relationship between the text and a past operation, and estimates a device to be operated from a plurality of devices, and a device estimated by the relationship determination unit Priority setting means for setting a priority for each device or each operation type of the device, and a device or a device based on the priority. An operation instruction means for selecting an operation type of the device and instructing the device to perform an operation corresponding to the text; and a driving situation estimation means for detecting or estimating a driving situation relating to the mounted vehicle, in front Priority setting means, and sets the priority based on the driving situation at least the operation is the elapsed time, and estimated from the time it was made (the invention according to claim 1).

  The present invention also provides a voice input means for inputting voice, a voice recognition means for recognizing the voice inputted by the voice input means and converting it into text, and an operation history storage in which operations performed in the past are recorded. A relationship determination unit that refers to a section, examines a relationship between the text and a past operation, and estimates a device to be operated from a plurality of devices, and a plurality of devices estimated by the relationship determination unit Priority setting means for setting a priority as a determination criterion for selecting a device to be preferentially operated for each device or each operation type of the device, and the operation type of the device or the device based on the priority The priority setting unit includes: an operation instruction unit that selects and instructs the device to perform an operation corresponding to the text; and a driving state estimation unit that detects or estimates a driving state related to the mounted vehicle. means When a voice input is performed, the priority corresponding to the selected device or the operation type of the device is increased, and at least one of the other priorities is decreased, and based on the estimated driving situation The priority is corrected (the invention according to claim 5).

  In this way, the past operation is referred to from the operation history unit to determine the relevance with the current voice instruction, and when there are multiple options as a result, the operation is performed based on the priority in consideration of the driving situation. Estimate the target device. Therefore, even if the contents of the voice instruction are ambiguous and there are a plurality of options to be processed, an appropriate selection process is possible. In addition, since the priority is set based on the driving situation, it is possible to perform an appropriate process according to the driving situation that changes every moment. The driving situation includes information on the driving state of the vehicle, the state of the on-board equipment (engine, transmission, air conditioner, audio, etc.), the current position, the external environment parameters such as the temperature inside and outside the passenger compartment, and / or the passengers. It is done.

  In this case, it has an input means for inputting a direct operation other than the voice of the passenger, the direct operation is further stored in the operation history storage unit, and the priority setting means further includes the direct operation The priority may be set based on the elapsed time from the point in time when the typical operation is performed and the estimated driving situation (the invention according to claim 2). The priority setting means increases the priority corresponding to the selected device or the operation type of the device and further decreases at least one of the other priorities when the direct operation is further performed. (Invention of claim 6).

  In this way, the direct operation performed manually by the passenger along with the voice input is reflected in the record of the operation history storage unit and the priority setting, so that the passenger can respond to the subsequent voice input instruction. More appropriate processing according to the intention of

  The priority setting means may set a pattern for changing the priority according to the elapsed time for each device (the invention according to claim 3). In other words, the priority of the target device that is immediately reflected in the operation, such as the volume of the audio, is set to a large rate of change, and the setting itself changes even if the operation is performed like the temperature setting of the air conditioner. However, the change rate of the priority of the target device that requires time until the inside of the vehicle reaches the appropriate temperature is set small. Thereby, selection is appropriately performed according to the reaction time of each device.

  The priority setting means is configured to increase a priority value corresponding to the operation when the operation is performed, and to decrease a corresponding priority value according to an elapsed time from the operation. The priority may be corrected based on the driving situation (the invention according to claim 4). As a result, the priority of the corresponding device increases each time the operation is performed, and an appropriate selection is made when the same device is operated next time. Moreover, since it correct | amends with a driving | running condition, an appropriate apparatus is selected according to the vehicle and the passenger's state at the time.

  The text generated by conversion by the voice recognition means includes ambiguity judgment means for judging whether the text is clear or ambiguous for selecting a device to be operated, and the text is determined by the ambiguity judgment means. Only when it is determined that it is ambiguous, the device may be estimated by the device selection means while determining whether or not there is a relationship between the text and the storage of the operation history storage unit (the invention according to claim 7). ).

  As a result, if the generated text is ambiguous such as containing an ambiguous word (abbreviated word, instruction word, etc.) or omitting the target word, it is appropriate based on the operation history storage unit and priority. On the other hand, when the text is clear, the device selection process is omitted and the calculation processing load is reduced.

  Further, the operation instructing means may output a response sentence that prompts re-input when there are a plurality of priorities of the same level among a plurality of levels or there is no priority that reaches a specified level (claim 8). invention). By outputting a response sentence that prompts such re-input, natural interactive voice input is realized.

  If the driving situation is the degree of driving burden on the passenger, under circumstances where the driving burden is high, change the priority change pattern according to the elapsed time since the last device operation, Therefore, it is possible to set a priority level (the invention according to claim 9). Examples of the driving burden include parameters such as vehicle speed, transmission gear ratio, steering wheel operation status, yaw rate, acceleration / deceleration frequency, uphill / downhill status, and the like.

  The driving status may be set as a temperature outside or inside the vehicle according to these temperatures (the invention according to claim 10). As a result, the temperature outside or inside the vehicle is added to the determination, not simply based on the elapsed time from the previous operation. Therefore, when the temperature inside and outside the vehicle is high, for example, by increasing the priority of the temperature adjustment of the air conditioner, it can be selected with priority over other devices, and processing according to the temperature and the will of the passenger is possible. .

  The driving status may be set as priorities according to noises outside or inside the vehicle (invention according to claim 11). Thereby, the noise outside or inside the vehicle is added to the determination, not just the determination based on the elapsed time from the previous operation. Therefore, when the noise inside and outside the vehicle is high, for example, by increasing the priority of the audio volume adjustment, it is easier to select it preferentially than other devices, and processing according to the noise and the position of the driver is possible. .

  The driving situation may be information related to a current position, a destination, or a route obtained from a navigation system (the invention according to claim 12). In other words, since it is highly likely that the passenger will give a voice input instruction to the navigation system at a place where it is determined that the passenger is likely to be conscious, this is based on the vehicle location information, destination information, and route information. Appropriate action can be taken by determining that the user has approached a certain point and raising the priority of the navigation system in advance.

  A passenger identification means for identifying a passenger; and an operation behavior storage section for storing operation behavior for each passenger; and the priority setting means is an operation behavior for each passenger stored in the operation behavior storage section. The priority may be set based on the above (invention of claim 13). Thereby, processing suitable for each passenger becomes possible.

  The operation instruction means selects a device corresponding to a priority at a higher level from among a plurality of levels, and when there are a plurality of priorities at the higher level, which to select A response sentence prompting confirmation may be output. By dividing the level in this way, the device can be easily selected (the invention according to claim 14).

  The device control apparatus for a vehicle according to the present invention refers to the past operation from the operation history unit to determine the relevance to the current voice instruction and, as a result, considers the driving situation when there are a plurality of options. The operation target device is estimated based on the priority. Therefore, even when the content of the voice instruction is ambiguous and there are a plurality of options to be processed, an appropriate selection process can be performed.

  In addition, by setting the priority based on the driving situation, it is possible to perform an appropriate selection process adapted to the changing driving situation.

  Furthermore, by reflecting the direct operation performed manually by the passenger in the record of the operation history storage unit and the setting of the priority, it is more appropriate for the subsequent voice input instruction in accordance with the intention of the passenger. Processing becomes possible.

  Furthermore, when the generated text is clear, the device selection process based on the operation history storage unit and the priority can be omitted to reduce the calculation processing load.

  DESCRIPTION OF EMBODIMENTS Hereinafter, a vehicle device control apparatus according to the present invention will be described with reference to FIGS.

  As shown in FIGS. 1 and 2, a vehicle device control apparatus 10 (hereinafter also simply referred to as a system) according to the present embodiment is mounted on a vehicle 12, and a microphone (audio) for inputting a passenger's voice. Input means 14, in-vehicle communication network 16, passenger identification switch (passenger identification means) 32, and speech switch 34. The microphone 14 is provided in the vicinity of the boundary between the roof and the front window in the vehicle. The microphone 14 may be provided at other locations (for example, a roof lower surface portion, an instrument panel portion, a headrest, a passenger's shoulder, a headset, and the like).

  The in-vehicle communication network 16 includes an air conditioner controller (hereinafter referred to as an air conditioner) 18, an audio 20, a navigation system 22, an engine controller 24, a panel operation unit 26, a sunroof controller 28 and a power window controller 30, a passenger identification switch 32, The speech switch 34 is connected to each other, and each of these devices can perform data communication with each other via the in-vehicle communication network 16. The air conditioner 18 is provided with a solar radiation sensor 36 and an air volume adjusting mechanism (not shown). The audio 20 is provided with a speaker 38, a volume adjusting mechanism and a volume recognizing unit (not shown). The speaker 38 is also used as an output unit of the vehicle device control device 10 and the navigation system 22.

  The occupant identification switch 32 is provided for identifying the occupant to the vehicle device control apparatus 10, and a plurality of switches (for example, 6 for 6 persons) are allocated to each occupant. Consists of.

  The utterance switch 34 is a switch that is pressed when giving a voice instruction to the vehicular device control apparatus 10, and can be easily operated because it is provided on the handle portion.

  A GPS (Global Positioning System) 40 or a gyro sensor is connected to the navigation system 22, and the position and coordinate information of the vehicle 12 can be acquired.

  The engine controller 24 controls the engine 42 and can detect the vehicle speed V of the vehicle 12 by the vehicle speed sensor 44. Further, the engine controller 24 can acquire the yaw rate Y of the vehicle 12 based on the vehicle speed V and a signal from a steering angle sensor (not shown). The yaw rate Y may be calculated while referring to a signal from a gyro sensor or the like.

  The panel operation unit 26 is provided with a switch or the like that is operated by the passenger. For example, the wiper speed W is controlled and the wiper speed W is supplied to the in-vehicle communication network 16. The power window controller 30 can raise and lower the window according to the state of the operation switch, and supplies the window opening degree O at that time to the in-vehicle communication network 16 according to a request from another device.

  As shown in FIG. 3, the vehicle device control apparatus 10 includes an A / D converter 46 that inputs a voice signal spoken by a passenger from a microphone 14 and converts the signal into digital data, and a frequency analysis of the converted digital data. Connected to the frequency analysis unit 48. The frequency analysis unit 48 performs frequency analysis on the input data to obtain analysis data.

  The vehicle device control apparatus 10 selects a speech recognition unit 50 that recognizes speech input from the microphone 14 via the frequency analysis unit 48 and converts it into text, and a device on which the converted text is an operation target. The relationship between the text and the past operation is examined by referring to the ambiguity determination means 52 for determining whether the operation is clear or ambiguous, and the operation history storage unit 54 in which the operations performed in the past are sequentially recorded. Relevance determining means 56 for estimating a device to be operated from a plurality of devices, and priority setting means 58 for setting a priority for each device. Here, the priority is a parameter serving as a determination criterion for selecting a device to be preferentially operated when there are a plurality of devices estimated by the relevance determination unit 56.

  The vehicle device control apparatus 10 further selects a device based on the priority comparison means 60 for comparing the priorities of each device and the result of the comparison, and causes the selected device to perform an operation corresponding to the text. Operation instruction means 62 for giving an instruction and driving situation estimation means 64 for detecting or estimating a driving situation relating to the vehicle 12 are provided. The operation instruction unit 62 receives the processing result via the response sentence generation unit 66 and outputs a predetermined operation instruction to the operation target device via the in-vehicle communication network 16 or outputs a response sentence to the speaker 38.

  The priority setting means 58 corresponds to steps S10 and S11 described later, and sets the priority based on the operation timing, the elapsed time from the timing, and the driving situation. The elapsed time in this case is measured by the elapsed time measuring means 68.

  The relevance determining means 56 is executed only when it is determined that the generated text is ambiguous. When the text is clear, the text is supplied to the response sentence generating means 66 according to the arrow Ar1. When the relevance determining unit 56 determines that the text is not related to the past operation, information to that effect is supplied to the response sentence generating unit 66 according to the arrow Ar2, and the operation instructing unit 62 is used. Then, a response message “Insufficient input contents” is output from the speaker 38.

  The operation history storage unit 54 is connected to an operation habit storage unit 70 that is a kind of database that stores the operation habits of each passenger identified by the passenger identification switch 32. The operation behavior storage unit 70 may be a database that records the operation behavior of each passenger in an independent format, or may be a relational database associated with a storage unit such as the operation history storage unit 54.

  Further, the driving condition estimation means 64 detects signals such as the vehicle speed V, the air conditioner airflow T, the yaw rate Y, the audio volume A, the position and coordinate information, the wiper speed W, the window opening O, the sunroof opening S, and the average vibration F. Estimate and calculate. The processing by the driving state estimating means 64 is performed every minute time based on an input routine different from the main routine processing shown in FIG.

  The voice recognition unit 50, the ambiguity determination unit 52, the relevance determination unit 56, the priority setting unit 58, the priority comparison unit 60, and the response sentence generation unit 66 form a main processing unit 72. Further, the operation history storage unit 54, the driving situation estimation unit 64, the elapsed time measurement unit 68, and the operation behavior storage unit 70 form a priority setting auxiliary unit 74 that assists the priority setting in the priority setting unit 58. .

  Input means 76 such as a switch that is directly operated by a passenger by hand is connected to the operation instruction means 62, and the operation instruction means 62 operates on the device based on an operation signal input from the input means 76. An instruction can be given or a response sentence can be output from the speaker 38.

  In practice, the vehicle device control apparatus 10 includes a central processing unit (CPU) that is a main control unit, a random access memory (RAM) that is a storage unit, a read only memory (ROM), a hard disk, and the like, as shown in FIG. Each means is realized by the CPU reading and executing the program stored in the storage unit. Also, the block diagram shown in FIG. 3 is conceptual for explanation, and in the actual vehicle device control apparatus 10, the relationship between each means, the flow of information exchange, and the like are not limited to this. Of course.

  Next, the operation of the vehicle device control apparatus 10 configured as described above will be described with reference to FIG. The main routine process shown in FIG. 4 is a process that is called when the speech switch 34 is turned on, and is executed by the cooperative operation of each means shown in FIG. In the following description, in order to avoid complication, the correspondence between each step and each of the above-described units will be described without explicit or distinction.

  First, in step S <b> 1, the voice recognition unit 50 inputs voice spoken by the passenger via the microphone 14, and performs voice feature extraction from the data converted by the A / D conversion unit 46 and the frequency analysis unit 48.

  In step S2, text analysis is performed for selecting a recognition result text from the feature extracted data. As a means for this text analysis, for example, a pattern matching technique or a statistical technique can be used. As a statistical technique, a hidden Markov model having a probabilistic finite state can be cited. In the hidden Markov model, the speech can be recognized with high probability by learning the speech model.

  In step S3, it is determined whether the recognized text is ambiguous or clear. Specifically, it is determined that the object is ambiguous when there is no object for specifying the operation target or an abbreviation or instruction word is included. If the text is ambiguous, the process proceeds to step S8. If the text is clear, the process proceeds to step S4. When the text is clear, the device selection process in steps S8 to S15 is omitted, and the calculation processing load is reduced.

  In step S4, it is confirmed whether or not the recognized text is an accurate command with no excess or deficiency. If it is determined that the text is accurate, the process proceeds to step S5. If the recognized text is an incorrect command, the process proceeds to step S7. Move.

  In step S5, the device is operated based on the recognized text. Specifically, by transmitting a command to the operation target device indicated by the text through the in-vehicle communication network 16, the operation target device performs an operation corresponding to the command. In step S <b> 5, the performed operation is stored in the operation history storage unit 54. The corresponding operations are also stored in the operation history storage unit 54 in steps S102, S202, and S302, which are the same processes as those in step S5.

  Thereafter, the priority for the operation target device is increased (step S6), and the current process shown in FIG. 4 is terminated. The process for increasing the priority will be described in step S103 (see FIG. 6).

  If the recognized text is inaccurate, a response message “Insufficient input content” is output from the speaker 38 in order to prompt re-input in step S7.

  On the other hand, in step S8 (when the text is ambiguous), a relevance determination process between the current input text and the previous and previous operations is performed. Specifically, the following two examples will be described in the case where the recognized text is “Make it higher”.

<Example 1> Previous operation: Target is the navigation system 22, and the operation content is the destination setting
Previous operation: Target is audio 20, operation is volume up

  In this case, it is determined that the destination setting for the previous operation is irrelevant to the word “raise / lower” for the current operation, and refers to the previous operation. Since the previous operation is an operation to increase the volume, it is determined that the operation is related to the audio 20 that is the operation target at that time.

<Example 2> Previous operation: The target is the air conditioner 18 and the operation content is to lower the set temperature.
Previous operation: Target is audio 20, operation is volume up

  In this case, the operation of lowering the set temperature of the previous operation is determined to be irrelevant because it has the opposite meaning to the word “raise” of the current instruction, and the previous operation is referred to. Since the previous operation is an operation to increase the volume, it is determined that the operation is related to the audio 20 that is the operation target at that time. If the current instruction is “Turn up after all”, it is an instruction to deny the past operation, so it is recognized that there has been an instruction “down” in the past, and the air conditioner 18 of the previous operation is recognized. It is judged to be related to lowering the set temperature.

<Example 3> Previous operation: The target is the air conditioner 18, and the operation content increases the set temperature.
Previous operation: Target is audio 20, operation is volume up

  In this case, both the previous operation and the previous operation are related to the operation of “raising”, but the previous operation with a short elapsed time is not given priority, and the determination based on the priority is performed as shown below. Is called.

  In step S9, branch processing based on the result of the relevance determination in step S8 is performed. That is, if it is determined in step S8 that the current text is related to the previous operation or the previous operation, the process proceeds to step S10, and if it is determined that there is no relationship, the process proceeds to step S7. , Prompt for re-entry.

  In step S10, the priority for the operating device is calculated. This priority is a numerical value set for each device to be operated, and changes in the priority Pa for the air conditioner 18 and the priority Po for the audio 20 are shown in FIG. Each priority takes a value from 0 to 100, and is set so as to decrease in proportion to the elapsed time, and the reduction rate (pattern to be changed) is set individually for each device. The decreasing rate of the priority Pa is 1 / sec, and the decreasing rate of the priority Po is 2 / sec. The reason why the decrease rate of the priority Pa is smaller than the decrease rate of the priority Po is that the responsiveness of the temperature change in the passenger compartment due to the operation of the air conditioner 18 is lower than the responsiveness of the volume change due to the operation of the audio 20.

  Each priority is set to increase by 80 points when the corresponding device is operated. Further, as will be described later, each priority is compared with a confirmation threshold Th1 set to 50 points and an execution threshold Th2 set to 70 points, and the operating device is estimated based on the comparison result. When one priority exceeds the execution threshold Th2, the operation of the corresponding device is immediately executed. When one priority exceeds the confirmation threshold Th1, it is executed after confirming the response.

  Specifically, after obtaining the elapsed time t from the execution time of the previous operation, calculation of Pa ← Pa−1 × t and Po ← Po−2 × t is performed. Limit and obtain priorities Pa and Po at that time. For other priorities, the same calculation is performed according to the respective reduction rates. The priority calculation may be performed periodically using a predetermined timer interrupt.

  In step S11, the priority correction calculation is performed based on the driving situation and the operating behavior of the passenger. This correction calculation will be described later.

  In step S12, with reference to each priority at that time, it is confirmed whether or not there is a priority that exceeds the confirmation threshold Th1 (= 50). If there is a priority that exceeds the confirmation threshold Th1, the process proceeds to step S13, and if not, the process proceeds to step S7 to prompt re-input.

  That is, at time t6 in FIG. 5, since each of the priorities Pa and Po is equal to or less than the confirmation threshold value Th1, the process proceeds to step S7.

  In step S13, it is confirmed whether there is only one priority exceeding the confirmation threshold Th1 (= 50). If there is one priority that exceeds the confirmation threshold Th1, the process proceeds to step S14. If a plurality of priorities exceeds the confirmation threshold Th1, the process proceeds to step S15.

  In step S14, the subroutine processing shown in FIG. 6 is executed, and the device is operated based on the specified priority.

  That is, in step S101 in FIG. 6, it is confirmed whether or not the identified priority exceeds the execution threshold Th2, and if it exceeds, the process proceeds to step S102, and if not, the process proceeds to step S104.

  In step S102, as in step S5, the device is operated based on the recognized text. For example, at time t4 in FIG. 5, since the priority Pa of the air conditioner 18 exceeds the execution threshold Th2, an operation on the air conditioner 18 is performed.

  In step S103, the priority value corresponding to the operated device is added by 80 points, and the added priority is restricted so as not to exceed 100. For example, if the air conditioner 18 is operated at time t1 in FIG. 5, the priority Pa before the corresponding operation is Pa = 0, so Pa ← 0 + 80 is updated as Pa = 80. Further, when the air conditioner 18 is operated again at time t2 after 40 seconds, the priority Pa is Pa = 40, so Pa = 120 is obtained as Pa ← 40 + 80. Do Pa = 100.

  At time t3, the priority Po is updated from 0 to 80, indicating that the audio 20 has been operated.

  On the other hand, in step S104, a confirmation response sentence for operating the device corresponding to the specified priority is created and uttered. For example, at time t5 in FIG. 5, since the priority Po of the audio 20 is equal to or higher than the confirmation threshold Th1 and less than the execution threshold Th2, a confirmation is made by uttering the word “Are you sure about the volume?” From the speaker 38. Do.

  In step S105, the passenger's response is input and text-analyzed. If the answer is affirmative such as “Yes”, the operation proceeds to step S102. If the answer is negative, such as “No”, in step S106, re-input is prompted in the same manner as in step S7.

  According to such processing based on steps S1 to S14, as shown in FIG. 7, a natural conversation is established between the occupant and the vehicle device control apparatus 10, and the occupant has a special consciousness. It is possible to give an instruction without voice. In FIG. 7, the time advances from the top to the bottom.

  On the other hand, in step S15 (see FIG. 4), processing corresponding to the priority matrix 80 is performed. This process is executed as a subroutine process shown in FIG. The priority matrix 80 is recorded in a predetermined storage unit in the vehicle device control apparatus 10.

  In step S201, the priority matrix 80 shown in FIG. The priority matrix 80 is a matrix for selecting one of the four cases 1 to 4, and the current value of the priority corresponding to the device that has been operated last time is equal to or higher than the confirmation threshold Th1 and the execution threshold Th2. When it is less than the upper level, the upper level is selected, and when it is equal to or higher than the execution threshold Th2, the lower level is selected. Further, the left column is selected when the current value of the priority corresponding to the device that has been operated the last time is equal to or greater than the confirmation threshold Th1 and less than the execution threshold Th2, and the right column is selected when equal to or greater than the execution threshold Th2. The upper left column is case 1, and candidate presentation is performed in later step S204. The upper right column is case 2, and equipment corresponding to the last time is adopted. The lower left column is case 3, and the device corresponding to the previous time is adopted. The lower right column is case 4, and candidate presentation is performed as in case 1.

  More specifically, as shown in FIG. 10, when the volume of the audio 20 is lowered two times before and the set temperature of the air conditioner 18 is lowered last time, the current value of the priority Po of the audio 20 is Po = 55. When the current value of the priority Pa of the air conditioner 18 is Pa = 65, the priority matrix 80 is referred to in order to determine which one should be selected. In this case, since the previous operation is for the air conditioner 18 and the current value of the priority Pa is 65, Th1 ≦ Pa <Th2 and the upper level of the priority matrix 80 is selected. Further, since the previous operation is for the audio 20 and the current value of the priority Po is Po = 55, Th1 ≦ Po <Th2 and the left column of the priority matrix 80 is selected. Eventually, case 1 in the upper right column is selected.

  In step S201, when the selected case is case 2 or case 3, the process proceeds to step S202, and when the selected case is case 1 or case 4, the process proceeds to step S204.

  In step S202, the device is operated on the device selected in the priority matrix 80 based on the recognized text as in step S5.

  In step S203, the priority of the corresponding device is increased as in step S103.

  On the other hand, in step S204, a response sentence that presents a plurality of device candidates to be selected is created and uttered. For example, in the example shown in FIG. 10, the speaker 38 utters the words “Which should I lower the volume or set temperature?” To confirm. In this case, even if there is a device with a “lowering” operation other than the audio 20 and the air conditioner 18 (for example, the window opening degree O), if the priority corresponding to the device is equal to or lower than the confirmation threshold Th1, the response message for candidate selection Can be excluded.

  In step S205, the response of the passenger is input and text-analyzed, and if it is a response for selecting a candidate device (for example, a response of “air conditioner”), the device is specified according to the response, and the step The operation proceeds to S203. If it is not a response for selecting a candidate device, re-input is prompted in step S206 as in step S7.

  As described above, in the process illustrated in FIG. 8, when any one of a plurality of candidate priorities is equal to or higher than the execution threshold Th2, a device corresponding to the priority is selected, and two or more are equal to or higher than the execution threshold Th2. If it is, selection confirmation of the device corresponding to the priority is performed. In addition, when there is no priority exceeding the execution threshold Th2, selection confirmation of devices corresponding to all candidate devices is performed. Therefore, at time t5 in FIG. 5, since both the priorities Pa and Po are equal to or higher than the confirmation threshold Th1 and less than the execution threshold Th2, Case 1 is selected based on the priority matrix 80, and Steps S204 and S205 are performed. Confirmation of selection is performed.

  In other words, in the process shown in FIG. 8, a device corresponding to a priority at a higher level is selected from the levels divided by a plurality of threshold values (that is, Th1 and Th2). When there are a plurality of priorities at the level, a response sentence prompting confirmation of which one to select is output. By dividing the level in this way, the device can be easily selected.

  The range for referring to the history is not limited to the past two times, and a history that is three or more times before may be referred to.

  According to the processing shown in FIG. 8, since it is possible to perform appropriate processing for three or more devices, for example, in response to the instruction “raise”, the air-conditioning air volume, the opening of each part of the air-conditioning grill, the window opening It can be applied in combination to operations such as degree O and sunroof opening S.

  After the process of steps S6 and S7 or the subroutine process shown in FIG. 6 or FIG. 8 is finished, the main routine process shown in FIG. 4 is finished.

  Next, a first correction calculation example to a sixth correction calculation example will be described as specific examples of the priority correction calculation in step S11.

  The first correction calculation example is an example in which correction is performed based on information on the vehicle state and the surrounding environment, and in particular, the correction is performed on the basis of the temperature outside the vehicle or the temperature inside the vehicle, and the determination based on the priority is compensated. can do. For example, if the outside air temperature is 34 ° C., the set temperature is 32 ° C., the volume of the audio 20 is −5 based on the default value, and the text is “lower”, the volume has already been sufficiently lowered. However, both the outside air temperature and the set temperature are considerably high, and it is estimated that this is an additional operation for the set temperature. For this reason, a process of multiplying the priority Pa of the air conditioner 18 by 1.5 (an increase process such as addition may be performed) is set to Pa ← Pa × 1.5. Thereby, when both the priority Pa of the air conditioner 18 and the priority Po of the audio 20 are not less than the confirmation threshold Th1 and not more than the execution threshold Th2, only the priority Pa can be reliably updated to the execution threshold Th2 or more. The temperature setting can be added smoothly.

  Note that, as a result of multiplying the priority Pa by 1.5, when both the priority Pa and the priority Po of the audio 20 are equal to or greater than the execution threshold Th2, it is determined in step S15 which is selected. The

  The second correction calculation example is an example in which correction is performed based on information on the vehicle state and the surrounding environment in the same manner as the first correction calculation example. In particular, in the example in which correction is performed based on noise outside the vehicle or in the vehicle interior. is there. That is, it is determined that the wind noise is loud in the high-speed running state, and the noise caused by the entrainment of air when the rotational speed N of the engine 42 is high and the window opening degree O and the sunroof opening degree S are large even in the low-speed running state. Is judged to be large. In such a case, when the volume of the audio 20 is -5 based on the default value, it is estimated that the sound of the audio 20 is difficult to hear. For this reason, a process of multiplying the priority Po of the audio 20 by 1.5 is performed to set Po ← Po × 1.5. Thereby, when the priority Pa of the air conditioner 18 and the priority Po of the audio 20 are both equal to or higher than the confirmation threshold Th1 and equal to or lower than the execution threshold Th2, only the priority Po can be reliably updated to be equal to or higher than the execution threshold Th2. The volume up operation can be performed smoothly.

  Noise outside or inside the vehicle is estimated based on the vehicle speed V, the rotational speed N, the window opening degree O, the average vibration F, the wiper speed W, or the like, or detected by a predetermined noise sensor.

  The third correction calculation example is an example in which correction is performed using the operation behavior storage unit 70 based on the operation behavior of the passenger. The operation behavior storage unit 70 stores the operations performed by the passenger in the past linked to information on the vehicle state and the surrounding environment at that time, and refers to this information in the past similar to the current driving situation. In the case of the above situation, processing for multiplying the priority corresponding to the device having a high frequency of operations performed at that time by 1.5 is performed. For example, when the temperature is 34 ° C., the set temperature is 32 ° C., the volume is −5 based on the default value, and the vehicle is stopped, and the frequency of the additional operation with respect to the set temperature is high, the priority of the air conditioner 18 The process of multiplying Pa by 1.5 is performed, and the temperature setting addition operation can be smoothly performed as in the case of the first correction calculation example.

  The fourth correction calculation example is an example in which information indicating that a passenger has approached a location that is determined to be conscious is acquired from the navigation system 22 and the priority corresponding to the navigation system 22 is increased based on the information. is there. Examples of such locations include large intersections, railroad crossings, junction / branch points, interchanges, toll booths, high-speed service areas, gas stations, parking lots, and rotary in front of the station. It is better to apply it to highly specific parts. In the vicinity of such a place, it is conceivable that the passenger omits and designates the place name, but by increasing the priority of the navigation system 22 in advance, the missing place name is automatically specified. This makes it possible to take appropriate measures.

  The fifth correction calculation example is an example in which the priority corresponding to the navigation system 22 is increased in the vicinity of the place where the route guidance by voice or display from the navigation system 22 is performed. At such timing, it is conceivable that the passenger gives a response instruction by omitting words corresponding to the information issued by the navigation system 22. For example, when the navigation system 22 says “Please turn right at the next intersection.”, The passenger includes or is omitted from the instruction word such as “That right?”, “Right?” Or “Which side?” It is conceivable to listen back by the words. Even in such a case, by increasing the priority of the navigation system 22 in advance, it is possible to automatically compensate for the missing words and take appropriate measures.

  The sixth correction calculation example is an example in which correction is performed based on the driving burden on the passenger. In this case, the driving burden can include parameters such as vehicle speed V, transmission gear ratio, steering wheel operation status, yaw rate Y, acceleration / deceleration frequency, uphill / downhill status, and the like. The state where the degree of driving burden is low is, for example, a situation where the vehicle is traveling on a flat road with a small curve at a constant speed of a prescribed cruise speed at a high gear ratio. Further, it is determined that the driving burden is low while the known auto-cruise function is being executed. Furthermore, even when the speed is low, when the acceleration / deceleration frequency and the yaw rate Y are small, it can be determined that the driving burden is small. Furthermore, based on the situation obtained from the navigation system 22, it is determined that the driving burden is low when driving on a suburban highway, and the steering operation is continuously performed when driving in an urban area. It is judged that the driving burden is high on mountain roads. In situations where the steering is continuously operated, the driving burden is high, there is no physical margin for operating the speech switch 34, and the operation interval tends to be long regardless of the continuity of the operation. There is. The degree of driving burden on the passenger may be estimated based on a physiological phenomenon such as a heart rate or a degree of sweat measured using an appropriate sensor. In this case, it is determined that the driving load is lower as the heart rate is lower or the sweating is lower.

  When the driving burden is high, it is assumed that the passenger's operation on the air conditioner 18 and the audio 20 etc. is delayed or the frequency is lowered, so the corresponding priority Pa and Po etc. rapidly decrease more than necessary. Change the decrease rate so that it does not occur. For example, as described above, the reduction rates of the priorities Pa and Po are 1 / sec and 2 / sec, but these reduction rates are changed to half of 0.5 / sec and 1 / sec, respectively, or the degree of operation burden Inversely proportionally smaller.

  In the sixth correction calculation example, the priority value is not directly changed, but can be indirectly changed by changing the decrease rate. By setting the decrease rate to be small, the change in priority according to the elapsed time from the previous operation is reduced, and it is possible to prevent the operation content from being checked more than necessary (see step S104).

  In addition, the vehicle device control apparatus 10 may be configured to execute voice processing based on the processing shown in FIG. The process in FIG. 11 is obtained by adding steps S301 and S302 to the process based on FIG. 4, and the same steps as steps S1 to S15, which are the same as the processes in FIG. It is numbered.

  In this case, in the first step S301, it is confirmed whether or not the occupant has performed a direct input operation using a hand with respect to the input means 76 (see FIG. 3). In step S302, an operation corresponding to the input is executed (step S302). If no input operation is performed, the process proceeds to step S1.

  The input means 76 uses an input element based on electricity, radio waves, light, or the like, and specifically includes operations of a switch, a dial, a remote controller, and the like. After performing step S302 which is a process corresponding to this input, the process proceeds to step S6 to increase the priority of the operation target device.

  In this way, by performing an operation corresponding to the direct input operation prior to the voice input in step S1, it is possible to perform an integrated response to both the voice input and the direct input, and the direct input. Since the priority of the corresponding device is raised after the operation is performed, an appropriate response can be made even if voice input is performed in an ambiguous language thereafter for the same device.

  Next, an example of setting the priority for each operation type of the device will be described with reference to FIG. In this example, the priority of the air conditioner is set separately for the priority Pat for the temperature setting operation and the priority Paf for the air volume setting operation. It is assumed that both the priority Pat and the priority Paf have a reduction rate of 1 / sec.

  First, at time t11, when the passenger instructs “Reduce the temperature of the air conditioner”, the system responds “Yes, lower the set temperature”, and the priority Pat corresponding to the operation is changed from 0. While raising to 80, the temperature setting of the air conditioner 18 is lowered.

  At time t12 after 15 seconds, the passenger responds “Yes, lower the airflow” by instructing “lower the airflow”, and the priority Paf corresponding to the operation starts from 0. While raising to 80, the air volume setting of the air conditioner 18 is lowered. The priority Pat decreases with time from time t11, and Pat = 65 at time t12.

  Thereafter, the priority Pat continues to decrease from 60, and the priority Paf starts decreasing from 80. Further, if the passenger instructs “lower it further” after this, different actions will be taken depending on the time.

  That is, at time t13 when 5 seconds have elapsed from time t12, when the passenger instructs “lower it further”, the priority Paf is 75 and the priority Pat is 60, so the execution threshold Th2 The magnitude relationship with the confirmation threshold Th1 is Paf> Th2, and Th2 ≧ Paf> Th1. Accordingly, the air volume setting corresponding to the priority level Paf is selected, and an operation for lowering the air volume is performed in response to “Yes, the air volume is lowered”, and the priority Paf is increased to 100.

  Further, at time t14 when 12 seconds have elapsed from time t12, when the passenger instructs “lower”, the priority Paf is 68 and the priority Pat is 53, so Th2 ≧ Pat > Th1, Th2 ≧ Paf> Th1. Therefore, the process according to the step S15 (see FIG. 4 and FIG. 8) is performed, and after confirming the passenger, the corresponding operation is performed.

  Further, at time t14 when 20 seconds have elapsed from time t12, when the passenger instructs “lower it further”, since the priority Paf is 60 and the priority Pat is 45, Th2 ≧ Pat > Th1, Th1 ≧ Paf. Therefore, the process according to the step S14 (see FIG. 4 and FIG. 6) is performed, and the corresponding operation is performed after asking the passenger to confirm that “Is it OK to set the air volume?”.

  In this way, even for the same device, by setting the priority for each operation type individually, an appropriate operation can be selected according to the priority even when the same content is instructed. become.

  Next, an example in which priority is set every time voice input is performed and each priority is held constant regardless of the elapsed time from the previous setting will be described with reference to FIG. In this example, when voice input is performed, the priority corresponding to the selected device or the operation type of the device is increased by 80, and the other priorities are decreased by 20. Each priority is limited to a numerical value within the range of 0-100.

  First, at time t21, when the passenger instructs “Raise the air conditioner temperature.”, The system responds “Yes, raise the air conditioner set temperature by 2 ° C.” and the priority corresponding to the operation. While increasing Pa from 0 to 80, the temperature setting of the air conditioner 18 is increased by 2 ° C.

  At time t22 after 45 seconds, the passenger responds “Yes, raise the set temperature of the air conditioner further” by instructing “Raise more”. The priority Pa is increased from 80 to 100. The air volume setting of the air conditioner 18 is further increased by 2 ° C.

  By the way, when the priority Pa is decreased at a rate of 1 / sec corresponding to the elapsed time, Pa = 35 at time t22, and since Th1> Pa, the air conditioner 18 is not selected. You will be prompted to re-enter. On the other hand, in the example shown in FIG. 13, although a long time has elapsed since time t21, since the priority Pa is maintained at 80, the operation on the air conditioner 18 is appropriately selected. It is suitable when the degree of driving burden is high and the operation interval tends to be long. In addition, since the temperature setting of the air conditioner 18 takes time until the effect appears after the operation, the time until the re-instruction tends to be long, so it is preferable to maintain the priority Pa.

  After this, at time t23, the passenger responds “Yes, I will guide you to the Tokyo station on the shortest route” by instructing “Make the destination Tokyo Station”. The priority Pn is raised from 0 to 80 and the priority Pa is lowered to 80, and the guidance setting by the navigation system 22 is performed.

  Further, at time t24, when the passenger instructs “Turn on the radio.”, The system responds “Yes, turn on the radio. The channel is ABC-FM.” The priority Po of the audio 20 Is set from 0 to 80, the priority Pa and Pn are lowered to 60, and the radio is turned on.

  After this, at time t26, when the passenger instructs “Raise more”, the system responds “Yes, turn up the volume”, raises the priority Po to 100 and raises the priorities Pa and Pn. It is set to 40 and the volume of the audio 20 is increased. In this case, even if the instruction content is the same as that at time t22, since the priority Po is Po> Th2, the volume setting of the audio 20 is preferentially selected over the set temperature of the air conditioner 18.

  When a voice input is performed, it is not necessary to lower all other priorities that do not correspond to the selected device (or device operation type), and at least one priority is lowered according to the situation or operation. It may be.

  In the case of the example shown in FIGS. 12 and 13, the priority Pat or Paf may be corrected based on the estimated driving situation. For example, when the temperature outside the vehicle interior is high, the priority Pat or Pa is set. The correction to be increased may be performed as appropriate.

  Moreover, the vehicle apparatus control apparatus 10 does not need to be a separate type from the navigation system 22, the air conditioner 18, etc., and a part or all of the functions may be integrated.

  The vehicle device control apparatus according to the present invention is not limited to the above-described embodiment, and can of course have various configurations without departing from the gist of the present invention.

It is a block block diagram of the vehicle by which the apparatus control apparatus for vehicles is mounted. It is a perspective view which shows the instrument panel vicinity of the vehicle by which the apparatus control apparatus for vehicles is mounted. It is a block block diagram of the internal structure of the apparatus control apparatus for vehicles, and its peripheral device. It is a flowchart which shows the process sequence of the apparatus control apparatus for vehicles. It is a time chart which shows the change of a priority. It is a subroutine flowchart which shows the detailed procedure of operation according to a priority. It is a table | surface which shows the relationship between the dialogue performed by using the apparatus control apparatus for vehicles, and a priority. It is a subroutine flowchart which shows the detailed procedure of the process corresponding to a priority matrix. It is a table | surface which shows a priority matrix. It is a table | surface which shows the relationship between the past operation and priority recorded on the operation history memory | storage part. It is a flowchart which shows the modification of the process sequence of the apparatus control apparatus for vehicles. It is a time chart which shows the change of the priority for every operation kind of apparatus. It is a time chart which shows the change of a priority, when a priority is set whenever input is performed and each priority is kept constant irrespective of elapsed time.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Vehicle apparatus control apparatus 12 ... Vehicle 14 ... Microphone 18 ... Air conditioner 20 ... Audio 22 ... Navigation system 24 ... Engine controller 32 ... Passenger identification switch 34 ... Speech switch 38 ... Speaker 50 ... Voice recognition means 52 ... Ambiguous judgment means 54 ... Operation history storage unit 56 ... Relevance determination unit 58 ... Priority setting unit 60 ... Priority comparison unit 62 ... Operation instruction unit 64 ... Driving situation estimation unit 66 ... Response sentence generation unit 70 ... Operation behavior storage unit 80 ... Priority Degree Matrix Pa, Paf, Pat, Pn, Po ... Priority Th1 ... Confirmation Threshold Th2 ... Execution Threshold

Claims (14)

Voice input means for inputting voice;
Voice recognition means for recognizing the voice input by the voice input means and converting it into text;
Reference is made to an operation history storage unit in which operations performed in the past are recorded, the relationship between the text and past operations is examined, and a relevance determination unit that estimates an operation target device from a plurality of devices
Priority setting means for setting a priority as a determination criterion for selecting a device to be preferentially operated when there are a plurality of devices estimated by the relevance determining unit for each device or each operation type of the device. When,
An operation instruction means for selecting an apparatus or an operation type of the apparatus based on the priority and instructing the apparatus to perform an operation corresponding to the text;
A driving situation estimation means for detecting or estimating a driving situation relating to the mounted vehicle;
Have
The apparatus for controlling a vehicle device, wherein the priority setting means sets the priority based on at least an elapsed time from the time when the operation is performed and the estimated driving situation. The vehicle equipment control device according to claim 1,
Has input means to input direct operations other than the passenger's voice,
The operation history storage unit further stores the direct operation,
The priority setting means further sets the priority based on the elapsed time from the time when the direct operation is performed and the estimated driving situation. The vehicle equipment control device according to claim 1 or 2,
The apparatus for controlling a vehicle device, wherein the priority setting means sets a pattern for changing the priority according to the elapsed time for each device. In the vehicular equipment control device according to any one of claims 1 to 3,
The priority setting means increases the priority value corresponding to the operation when the operation is performed, decreases the corresponding priority value according to the elapsed time from the operation, and based on the driving situation And a vehicle device control device that corrects the priority value. Voice input means for inputting voice;
Voice recognition means for recognizing the voice input by the voice input means and converting it into text;
Reference is made to an operation history storage unit in which operations performed in the past are recorded, the relationship between the text and past operations is examined, and a relevance determination unit that estimates a device to be operated from a plurality of devices,
Priority setting means for setting a priority as a determination criterion for selecting a device to be preferentially operated when there are a plurality of devices estimated by the relevance determining unit for each device or each operation type of the device. When,
An operation instruction means for selecting an apparatus or an operation type of the apparatus based on the priority and instructing the apparatus to perform an operation corresponding to the text;
A driving situation estimation means for detecting or estimating a driving situation relating to the mounted vehicle;
Have
The priority setting means increases the priority corresponding to the selected device or the operation type of the device when voice input is performed, lowers at least one of the other priorities, and estimates the estimated A device control apparatus for a vehicle, wherein the priority is corrected based on a driving situation. In the vehicle equipment control device according to claim 5,
Has input means to input direct operations other than the passenger's voice,
The operation history storage unit further stores the direct operation,
The priority setting means further increases the priority corresponding to the selected device or the operation type of the device when the direct operation is performed, lowers at least one of the other priorities, and estimates The apparatus control apparatus for vehicles characterized by correct | amending the said priority based on the said driving | running condition. In the vehicle equipment control device according to any one of claims 1 to 6,
An ambiguity judging means for judging whether the text converted and generated by the voice recognition means is clear or ambiguous for selecting a device to be operated;
Only when the text is determined to be ambiguous by the ambiguous judgment means, the equipment is estimated by the equipment selection means while judging whether or not the text is stored in the operation history storage unit. A vehicle equipment control device. In the vehicle equipment control device according to any one of claims 1 to 7,
The operation instruction means outputs a response sentence that prompts re-input when there are a plurality of priorities at the same level among the plurality of levels or when there is no priority that reaches a specified level. apparatus. In the vehicle equipment control device according to any one of claims 1 to 8,
The vehicle equipment control device, wherein the driving situation is a driving burden of a passenger. In the vehicle equipment control device according to any one of claims 1 to 9,
The device control apparatus for a vehicle, wherein the driving situation is a temperature outside or inside the vehicle. In the vehicle equipment control device according to any one of claims 1 to 10,
The device control apparatus for a vehicle, wherein the driving situation is noise outside or inside the vehicle. In the vehicle equipment control device according to any one of claims 1 to 11,
The vehicle equipment control apparatus, wherein the driving situation is information relating to a current location, a destination, or a route obtained from a navigation system. In the vehicular equipment control device according to any one of claims 1 to 12,
A passenger identification means for identifying the passenger;
An operation habit storage unit for storing operation habits for each passenger,
The apparatus for controlling a vehicle device, wherein the priority setting means sets a priority based on an operation habit for each passenger stored in the operation habit storage unit. In the vehicle equipment control device according to any one of claims 1 to 13,
The operation instruction means selects a device corresponding to a priority at a higher level from among a plurality of levels, and when there are a plurality of priorities at the higher level, which to select A vehicle equipment control device that outputs a response sentence for prompting confirmation.

Images