JP2018028733A - Input device, input device control method, display device and input program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform input operation quickly and accurately.SOLUTION: An input device pertaining to the present invention comprises an operation acceptance unit, a selection unit, a recognition unit, a determination unit, and a confirmation unit. The operation acceptance unit accepts operation of a user. The selection unit selects a prediction dictionary from a plurality of prediction dictionaries that corresponds to the operation accepted by the operation acceptance unit. The recognition unit recognizes a speech of the user and calculates the reliability of the recognition on the basis of the prediction dictionary selected by the selection unit. The determination unit determines whether or not to demand a confirmation by the user of the correctness of the recognition result of the recognition unit on the basis of the reliability calculated by the recognition unit. The confirmation unit causes the recognition result to be confirmed by the user when it is determined by the determination unit that the correctness of the recognition result be confirmed.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an input device, an input device control method, a display device, and an input program.

  There is a speech recognition device that improves the accuracy of speech recognition by selecting and using a speech recognition dictionary corresponding to a user operation from a plurality of speech recognition dictionaries. As such a speech recognition device, a technique has been proposed in which a user performs an operation of narrowing a search range from a map displayed on a screen, and a speech recognition dictionary corresponding to the narrowed search range is selected and used (for example, a patent). Reference 1).

JP 2002-372990 A

  However, in the above-described conventional technology, the case where the recognition result of speech recognition is incorrect is not taken into consideration. For this reason, if the recognition result is incorrect, the user has to start again from the operation of narrowing down the search range, and there is a possibility that the overall time required for voice input is increased.

  The present invention has been made in view of the above, and an object of the present invention is to provide an input device, an input device control method, a display device, and an input program that can perform an input operation quickly and accurately.

  In order to solve the above-described problems and achieve the object, the input device according to the present embodiment includes an operation reception unit, a selection unit, a recognition unit, a determination unit, and a confirmation unit. The operation reception unit receives a user operation. The selection unit selects the prediction dictionary corresponding to the operation received by the operation reception unit from a plurality of prediction dictionaries. The recognition unit recognizes the user's voice and calculates a reliability of the recognition based on the prediction dictionary selected by the selection unit. The determination unit determines whether to confirm with the user whether the recognition result by the recognition unit is correct based on the reliability calculated by the recognition unit. The confirmation unit causes the user to confirm the recognition result when the determination unit determines to confirm the correctness of the recognition result.

  ADVANTAGE OF THE INVENTION According to this invention, the input device which can perform input operation rapidly and correctly, the control method of an input device, a display apparatus, and an input program can be provided.

FIG. 1 is a diagram showing an overview of a method for controlling an input device according to the present embodiment. FIG. 2 is a block diagram illustrating a configuration of the display device according to the present embodiment. FIG. 3A is a diagram illustrating a specific example of a prediction dictionary database. FIG. 3B is a diagram illustrating a specific example of a prediction dictionary. FIG. 4 is a diagram illustrating an input operation in occupant authentication. FIG. 5A is a diagram (part 1) illustrating an input operation in destination setting. FIG. 5B is a diagram (part 2) illustrating an input operation in destination setting. FIG. 5C is a diagram (part 3) illustrating an input operation in destination setting. FIG. 6A is a diagram (part 1) illustrating a scene of changing the scale of an image. FIG. 6B is a diagram (part 2) illustrating a scene of changing the scale of an image. FIG. 7 is a flowchart showing an input procedure executed by the input device according to the present embodiment.

  Hereinafter, embodiments of an input device, an input device control method, a display device, and an input program according to the present invention will be described in detail with reference to the accompanying drawings. In addition, this invention is not limited by embodiment shown below.

  First, an outline of a method for controlling an input device according to the present embodiment will be described with reference to FIG. FIG. 1 is a diagram showing an overview of a method for controlling an input device according to the present embodiment. In FIG. 1, a case where the user performs audio music search using the input device 1 will be described.

  FIG. 1 illustrates a case where the input device 1 according to the present embodiment is a touch panel display integrally including a display unit and an operation unit, for example. The input device 1 is connected to a microphone M that collects ambient sounds.

  Here, in the above-described conventional technology, the recognition accuracy of voice recognition is improved by selecting and using a prediction dictionary corresponding to a user operation from a plurality of prediction dictionaries.

  However, in the conventional technology, the case where there is an error in the recognition result of the speech recognition has not been considered. For this reason, when there is a misrecognition, there is a possibility that the entire time required for voice input may be increased, such as starting again from the operation of selecting the prediction dictionary.

  Therefore, in the input device 1 according to the present embodiment, the reliability of speech recognition is calculated in addition to the recognition result of speech recognition, and whether or not to confirm the speech recognition recognition result to the user based on the calculated reliability is determined. I decided to judge. Thereby, when the reliability is low, the user can be given an opportunity to correct the recognition result, and when the reliability is high, confirmation to the user itself can be omitted. Accordingly, it is possible to shorten the overall time until a correct recognition result is obtained. Hereinafter, the outline of the control method of the input device 1 will be described more specifically.

  When the user operates the music search button with a finger, the input device 1 accepts a user operation as a music search operation (step S1). Subsequently, the input device 1 selects the prediction dictionary D1 corresponding to the music search from the plurality of prediction dictionaries D (step S2).

  Here, the prediction dictionary D refers to a dictionary in which a character string recognized by speech recognition is associated with speech data corresponding to the character string. For example, the prediction dictionary D1 corresponding to music search is a dictionary in which music names registered in audio and voice data corresponding to the music names are stored in association with each other.

  The input device 1 includes a plurality of prediction dictionaries D1 to D3... And selects a different prediction dictionary D according to the operation received in step S1.

  Subsequently, the input device 1 recognizes speech based on the prediction dictionary D1 and calculates the reliability described above (step S3). Specifically, first, the input device 1 acquires the user's voice from the microphone M as a voice signal. Subsequently, the input device 1 collates the voice signal with the voice data stored in the selected prediction dictionary D1, and sets, for example, the closest song name among the song names registered in the prediction dictionary D1 as a recognition result.

  Here, when there are a plurality of song names similar to the prediction dictionary D1, an incorrect song name may be recognized. Therefore, the input device 1 calculates the reliability of the recognition result based on the number of song titles similar to the song name that is the recognition result. Then, based on the calculated reliability, it is determined whether or not to confirm the correctness of the recognition result with the user (step S4). Details of step S4 will be described later with reference to FIG.

  Here, when it is determined that the user confirms the recognition result, that is, when the above-described reliability is low, the input device 1 displays the song name that is the recognition result on the touch panel display, and allows the user to confirm whether the recognition result is correct or incorrect. (Step S5).

  Then, the input device 1 accepts an operation corresponding to whether the recognition result is correct or incorrect by the user, thereby confirming the recognition result.

  Thereby, the user can correct immediately when the recognition result is incorrect. In other words, the user can input accurately.

  On the other hand, when it is determined in step S4 that the user does not confirm the correctness of the recognition result, that is, when the reliability is high, the input device 1 confirms the recognition result without confirming the recognition result with the user. . Thereby, the input device 1 can shorten time by the amount which does not confirm a recognition result with a user.

  Thus, in the control method of the input device 1 according to the present embodiment, the reliability of the recognition result is calculated when performing speech recognition, and whether or not to confirm the recognition result with the user using the calculated reliability is determined. judge. Therefore, the user can perform an input operation quickly and accurately.

  Further, since the input device 1 recognizes speech based on the selected prediction dictionary D, the processing load of speech recognition can be reduced as compared with the case where speech is recognized based on all prediction dictionaries D. .

  In the above-described example, the case where the input device 1 is a touch panel display is illustrated. However, the present invention is not limited to this, and the input device 1 may be separated from the operation unit and the display unit. In addition to the touch panel described above, input devices such as buttons, a mouse, and a keyboard can be used as the operation unit. Further, a predetermined number of input devices may be used in combination.

  Hereinafter, the display device 100 including the input device 1 controlled by the above-described control method will be described. Moreover, although the case where the display apparatus 100 functions as a display apparatus of the vehicle-mounted apparatus 50 mounted in the vehicle below is demonstrated, it is also applicable also to a smart phone, a tablet terminal, and a personal computer.

  A configuration example of the display device 100 including the input device 1 according to the present embodiment will be described with reference to FIG. FIG. 2 is a block diagram illustrating a configuration of the display device 100 according to the present embodiment.

  As shown in FIG. 2, the display device 100 includes an input device 1 and a touch panel display 30. The display device 100 is connected to the in-vehicle device 50 and the microphone M. The touch panel display 30 may include the input device 1.

  In this way, the user can perform a touch operation in conjunction with the display image displayed on the display unit 32 of the touch panel display 30, so that the overall time required for input can be omitted. .

  The in-vehicle device 50 includes various devices such as an audio, a navigation system, and an air conditioner mounted on the vehicle, for example. The in-vehicle device 50 controls the various devices described above based on the input result from the input device 1. Further, the in-vehicle device 50 causes the display unit 32 of the touch panel display 30 to display a display image reflecting the control result.

  The touch panel display 30 includes an operation unit 31 and a display unit 32. Moreover, the touch panel display 30 is arrange | positioned in the position where a user can visually recognize and operate easily, such as a center console of a vehicle, for example.

  The operation unit 31 is, for example, a capacitance type input device having an operation surface. When the user performs a touch operation, the operation unit 31 outputs an operation signal indicating the operation content based on the touch operation to the operation reception unit 11 of the input device 1.

  Specifically, when detecting the touch position of the touch operation, the operation unit 31 acquires an operation signal corresponding to the touch position from the in-vehicle device 50 and outputs the operation signal to the operation reception unit 11.

  For example, the operation unit 31 outputs an operation signal indicating music search to the operation reception unit 11 when a music search button is pressed. In addition, when an image such as a map is pressed, the operation unit 31 outputs an operation signal indicating that the map is pressed to the in-vehicle device 50. In addition, the operation unit 31 always outputs an operation signal corresponding to the touch position to the operation reception unit 11 when a touch operation is detected.

  The display unit 32 is, for example, a liquid crystal display, and displays the operation status of various devices of the in-vehicle device 50, the speech recognition result by the recognition unit 13 described later, and the like, thereby allowing the user to confirm the recognition result.

  The input device 1 includes a control unit 10 and a storage unit 20. The control unit 10 is, for example, a CPU (Central Processing Unit), and performs overall control of the input device 1. In addition, the control unit 10 includes an operation reception unit 11, a selection unit 12, a recognition unit 13, a determination unit 14, and a confirmation unit 15.

  The operation reception unit 11 receives a user operation. Specifically, the operation reception unit 11 receives an operation signal based on an operation from the user input via the operation unit 31. Then, the operation reception unit 11 outputs an operation instruction corresponding to the operation signal to the selection unit 12.

  In addition, for example, when the operation reception is being continued, the operation reception unit 11 outputs a permission notification that permits voice recognition to the recognition unit 13. That is, in a state where the operation reception unit 11 does not receive an operation, the recognition unit 13 does not output a permission notice, and the recognition unit 13 does not recognize a voice.

  That is, only when the user utters while pressing the operation surface, the recognition unit 13 recognizes the utterance. Thereby, the voice input at the timing not intended by the user can be suppressed.

  Further, the operation accepting unit 11 may accept a touch operation on a predetermined area on the operation surface of the operation unit 31 and may not accept a touch operation on a part other than the predetermined area.

  In other words, the recognition unit 13 may recognize the utterance only when the user utters while pressing a predetermined area on the operation surface. Therefore, since the area of the operation surface where voice recognition is performed is limited, voice input unintended by the user can be further suppressed. The predetermined area is preferably an area linked to an operation instruction, that is, an area where an operation button is displayed on the display unit 32, for example.

  Further, the operation reception unit 11 may output a permission notification at a timing other than the time during which the operation reception is continuing. That is, the recognizing unit 13 may recognize a voice other than when the operation accepting unit 11 is continuing to accept the operation. In such a case, for example, when the operation reception unit 11 acquires an operation signal from the operation unit 31, the operation reception unit 11 may output the permission notification to the recognition unit 13 continuously for a predetermined period.

  Further, the operation accepting unit 11 can accept a user operation from other than the operation unit 31 of the touch panel display 30. For example, an operation of a physical operation button of the user can be received, or a behavior can be detected from a captured image of the user and a user operation can be received based on the behavior.

  Specifically, the operation reception unit 11 receives an operation of pressing a destination setting button or a music search button arranged in the vicinity of the touch panel display 30, and sets a destination or music search according to the pressed button. Can be output to the selection unit 12.

  Further, the operation accepting unit 11 can detect behaviors such as the user's line-of-sight direction and finger orientation from the captured image of the user, and can accept the behavior as an operation for a device positioned ahead of the line-of-sight direction and the finger orientation.

  When the user's line-of-sight direction or finger orientation is facing audio, the operation accepting unit 11 can accept, for example, an operation related to audio (for example, music search).

  The selection unit 12 selects a prediction dictionary corresponding to a user operation received by the operation reception unit 11 from a plurality of prediction dictionaries. Specifically, the selection unit 12 selects a prediction dictionary corresponding to the operation instruction acquired from the operation reception unit 11 from the prediction dictionary database 21 of the storage unit 20. Then, the selection unit 12 outputs the identifier of the selected prediction dictionary to the recognition unit 13. Hereinafter, this identifier is represented by “prediction dictionary No”.

  The recognition unit 13 recognizes the user's voice based on the prediction dictionary selected by the selection unit 12 and calculates the reliability of the recognition. First, speech recognition processing by the recognition unit 13 will be described.

  The recognition unit 13 acquires the prediction dictionary No. from the selection unit 12 and the voice signal input from the microphone M. Subsequently, the recognizing unit 13 refers to the prediction dictionary having the same prediction dictionary No from the prediction dictionary database 21 and compares the feature data between the speech data of the prediction dictionary and the speech signal input from the microphone M.

  As a result of the comparison, the recognition unit 13 extracts a character string having a high likelihood from the prediction dictionary. Then, the recognition unit 13 outputs the extracted character string to the determination unit 14 as text data. The recognition unit 13 passes the prediction dictionary No received from the selection unit 12 to the recognition unit 13.

  Here, each time the character is recognized, the recognition unit 13 converts the character one by one into text data and outputs the text data to the determination unit 14.

  Next, a method for calculating reliability by the recognition unit 13 will be described. Here, the prediction dictionary database 21 and the prediction dictionary will be described with reference to FIGS. 3A and 3B. FIG. 3A is a diagram illustrating a specific example of the prediction dictionary database 21.

  As illustrated in FIG. 3A, the prediction dictionary database 21 stores a plurality of prediction dictionaries, and for each prediction dictionary, a device, an operation instruction, a prediction dictionary No, the number of registrations, and the like are associated and stored.

  In the prediction dictionary 1 corresponding to occupant authentication, for example, a character string such as the name of an occupant who has boarded the vehicle in the past is registered. In the prediction dictionary 2 corresponding to the destination setting, character strings such as an address, a place name, and a facility name are registered. In the prediction dictionary 4 corresponding to music search, character strings such as song names, artist names, album names, and playlist names are registered.

  The same prediction dictionary 3 corresponds to the map scale change and the surrounding monitor image scale change. In the prediction dictionary 3, for example, a character string including a keyword that means image scale change such as enlargement or reduction is included. It is registered.

  Here, the reliability is an index indicating the difficulty of misrecognition with other character strings in speech recognition. In addition, the reliability is determined in five stages according to, for example, the number of character strings registered for each prediction dictionary (the number of registrations shown in the figure).

  For example, the smaller the number of character strings registered in the prediction dictionary, the higher the reliability. This is because as the number of character strings registered in the prediction dictionary increases, there is a higher possibility of being mistaken for other character strings by voice recognition.

  Specifically, the recognition unit 13 calculates the reliability by referring to the prediction dictionary database 21 for the number of registrations of the prediction dictionary No. input from the selection unit 12, for example.

  For example, the recognition unit 13 calculates the reliability of the prediction dictionary as “5” when the registration number of the prediction dictionary is less than 10, and when the registration number is between 10 and 49, The reliability is calculated as “4”.

  The recognition unit 13 calculates the reliability as “3” when the number of registrations is between 50 and 99, and sets the reliability as “2” when the number of registrations is between 100 and 199. ". When the number of registrations is 200 or more, the reliability is calculated as “1”. Then, the recognition unit 13 notifies the determination unit 14 of the text data that is the recognition result and the calculated reliability.

  The reliability calculation method described above is merely an example, and can be arbitrarily changed. Further, the recognition unit 13 may calculate the reliability in consideration of, for example, the number of similar character strings. In such a case, for example, the recognition unit 13 calculates so that the reliability is lower as the number of similar character strings is larger.

  Note that the recognition unit 13 may calculate the recognition reliability from the recognition result. Here, a specific example of the prediction dictionary will be described with reference to FIG. 3B. FIG. 3B is a diagram illustrating a specific example of a prediction dictionary. As shown in FIG. 3B, groups, frequencies, voice data, and the like are stored in association with character strings in the prediction dictionary.

  Here, for example, a group is a group of character strings having similar voice data as the same aggregate. In FIG. 3B, the character string “A” and the character string “A ′” are similar in audio data, and the character string “A” and the character string “A ′” are similar to the character string “B”. It is assumed that the audio data is not.

  Therefore, in the example illustrated in FIG. 3B, the character string “A” and the character string “A ′” are classified into the group “A”. Further, the character string “B” does not belong to the group because similar speech data does not exist in the same prediction dictionary.

  Here, the recognition unit 13 calculates the reliability according to the number of groups to which the character string that is the recognition result belongs, for example. The recognizing unit 13 adds the calculated reliability to the value of the reliability calculated with reference to FIG. 3A (hereinafter referred to as “reliability based on the prediction dictionary”), and the reliability of the added value. The degree can also be output to the determination unit 14.

  Specifically, the recognizing unit 13 calculates so that the reliability is high when the recognized character string does not belong to any group. In this case, the recognition unit 13 calculates the reliability value as “1”, adds the calculated value to the reliability based on the prediction dictionary described above, and notifies the determination unit 14 of the calculated value.

  For example, when the recognized character string belongs to the group, the recognition unit 13 calculates the reliability as “−1”, and adds the calculated “−1” to the reliability based on the prediction dictionary, that is, , “1” is subtracted and notified to the determination unit 14.

  As described above, the recognition unit 13 calculates the reliability so that the recognized character string does not belong to the group. This is because there is a low possibility that a character string that does not belong to a group is mistaken for another character string in the same prediction dictionary. Therefore, in such a case, the number of times that the user confirms the recognition result can be reduced by calculating the reliability high. Thereby, the processing load of a user's confirmation process can be reduced.

  The calculation method of the recognition unit 13 is not limited to the above example. For example, the recognition unit 13 calculates the reliability so that the longer the character string of the recognition result, the lower the reliability. Moreover, the confirmation part 15 mentioned later calculates a correctness rate for every character string, and the recognition part 13 may determine reliability based on this correctness rate.

  Moreover, the recognition part 13 calculates low reliability, when the text data used as a recognition result is not contained in the referred prediction dictionary. Thereby, in such a case, the determination unit 14 determines to confirm with the user whether the text data is correct or incorrect. The confirmation unit 15 confirms the correctness of the text data with the user.

  This is because when the text data is not included in the prediction dictionary, there is a high possibility that the text data is erroneously recognized. By doing in this way, it can control that incorrect text data is outputted to in-vehicle device 50.

  Further, the frequency shown in FIG. 3B indicates the frequency used for each character string. In the example illustrated in FIG. 3B, the frequency of the character string “A” and the character string “B” is “low”, and the frequency of the character string “A ′” is “high”. . However, the frequency notation is not limited to “high” or “low”, and may be any number as long as the order of frequencies for each character string can be identified.

  The frequency is updated by the determination unit 14, for example. For example, the determination unit 14 can update the frequency based on the text data input from the recognition unit 13 so as to increase the frequency each time the same text data is input. In addition, the process based on this frequency is later mentioned using FIG. 5A-FIG. 5C.

  Returning to FIG. 2, the determination unit 14 will be described. The determination unit 14 determines whether or not to confirm with the user whether the recognition result by the recognition unit 13 is correct based on the reliability calculated by the recognition unit 13.

  Specifically, the determination unit 14 determines to confirm the correctness of the recognition result with the user when the reliability notified from the recognition unit 13 is a predetermined threshold value or less. In this case, the determination unit 14 outputs the text data to the confirmation unit 15 and instructs the user to confirm.

  The determination unit 14 determines whether the recognition result is correct or not without confirming with the user when the reliability is higher than a predetermined threshold. In this case, the determination unit 14 outputs the text data to the in-vehicle device 50.

  Note that the predetermined threshold value is, for example, a reliability value of “4”, but can be changed dynamically. For example, the threshold value may be set high when calling by calling the telephone directory in a hands-free call or the like. In other words, the threshold value may be changed according to the importance of input.

  The confirmation unit 15 causes the user to confirm the recognition result when the determination unit 14 determines to confirm the recognition result by the recognition unit 13 with the user. Specifically, when the confirmation unit 15 is instructed to confirm whether the recognition result is correct or incorrect, the confirmation unit 15 outputs and displays text data that is the recognition result on the display unit 32 of the touch panel display 30.

  Thereby, the user can confirm the correctness of text data. In addition, the confirmation part 15 does not ask | require the method of presenting a recognition result to a user, such as outputting this text data from a speaker (not shown) as a sound using a text reading function.

  And the operation reception part 11 receives operation according to the result of the correctness of text data by a user via the operation part 31, for example, and outputs the operation signal according to this result to the confirmation part 15.

  When the confirmation unit 15 acquires an operation signal indicating that the result is correct as a correct / incorrect result, the confirmation unit 15 outputs text data of the recognition result to the in-vehicle device 50. Further, when acquiring the operation signal indicating that the operation signal is incorrect, the confirmation unit 15 displays a screen such as a prediction candidate list and a keyboard for inputting characters on the display unit 32.

  Here, the prediction candidate list is, for example, a list of character strings belonging to the same group as the recognition result. This is because if there is an error in speech recognition, the user's utterance is likely to be another character string belonging to the same group.

  That is, when there is an error in the recognition result, the confirmation unit 15 displays the prediction candidate list on the display unit 32 and accepts an input candidate selection operation from the user's prediction candidate list. By doing in this way, the user can correct quickly.

  Moreover, the confirmation part 15 can acquire the correct / incorrect determination result by the user also by voice. Specifically, the confirmation unit 15 first acquires text data input from the recognition unit 13 after the text data is displayed on the display unit 32.

  Subsequently, when the text data includes a keyword that affirms a recognition result such as “high” or “sodayo”, the confirmation unit 15 assumes that the recognition result is correct. On the other hand, if such text data includes a keyword that denies the recognition result, such as “Chigau” or “Machigatte”, it is assumed that the recognition result is incorrect.

  In this case, the confirmation unit 15 temporarily interrupts the process of calculating the reliability of the recognition unit 13 and resumes the process when the keyword described above is input. In this way, it is possible to prevent the user from confirming the keyword itself.

  The confirmation unit 15 can also display a prediction candidate list on the display unit 32 during speech recognition based on the prediction dictionary selected by the selection unit 12. This will be described later with reference to FIGS. 5A to 5C.

  The storage unit 20 includes a prediction dictionary database 21. The storage unit 20 is a semiconductor memory device such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. Note that the prediction dictionary database 21 has already been described with reference to FIGS. 3A and 3B, so description thereof is omitted here.

  Subsequently, a specific example of the input operation of the display device 100 will be described with reference to FIGS. 4 to 6B. 4 and 5A to 5C explain the case where the user confirms whether the recognition result of the recognition unit 13 is correct or not, and FIGS. 6A and 6B explain the case where the user does not confirm the correctness of the recognition result. To do.

  First, the input operation of the display device 100 will be described with reference to FIG. FIG. 4 is a diagram illustrating an input operation in occupant authentication.

  The occupant authentication is intended to provide in-vehicle services such as music tailored to the occupant by authenticating the occupant of the vehicle. The occupant authentication is performed, for example, at the timing when the ACC power source of the vehicle is activated.

  For example, the in-vehicle device 50 captures an occupant in the vehicle interior with a camera (not shown) mounted on the vehicle at such timing, and authenticates each occupant by using a face authentication function for each occupant from the captured image.

  FIG. 4 shows a scene in which the recognition result by the occupant authentication is displayed on the touch panel display 30 and the user confirms whether the recognition result is correct or incorrect.

  Specifically, as shown in FIG. 4, an occupant image including the name of each authenticated occupant and a face is displayed on the touch panel display 30. The occupant image is displayed at a position corresponding to the occupant's seat, for example. In addition, in order to clarify the position of each occupant's seat, a handle indicating the driver's seat is displayed above the driver's seat.

  For example, it is assumed that the user notices that Mr. B has an error as a recognition result, and presses the region R1 where the passenger image of Mr. B is displayed with his finger to correct Mr. B.

  Here, the operation unit 31 of the touch panel display 30 outputs an operation signal indicating occupant authentication to the operation reception unit 11 of the input device 1 based on an operation of pressing down the region R1.

  Subsequently, the operation reception unit 11 outputs an operation instruction based on the operation signal to the selection unit 12. In addition, the operation reception unit 11 outputs a permission notification to the recognition unit 13. Here, the selection unit 12 selects the prediction dictionary 1 based on the operation instruction, and passes the selected prediction dictionary 1 to the recognition unit 13 (see FIG. 3A).

  For example, the recognition unit 13 performs voice recognition based on the prediction dictionary 1 when the above-described permission notification is acquired from the operation reception unit 11, that is, when the user presses the region R1.

  Here, when the recognition unit 13 recognizes a predetermined keyword that specifies a touch position such as “here” or “this place” in a state where the permission notification is acquired, the speech recognition process based on the prediction dictionary is performed. May be started.

  Thereby, the user's voice recognition and the desired intention can be confirmed based on both the touch position and the keyword. Accordingly, it is possible to suppress erroneous input due to voice input such as when the user unintentionally presses the touch panel display 30.

  In addition, when the recognition unit 13 starts speech input based on the prediction dictionary, the recognition unit 13 outputs text data that is a recognition result, that is, the occupant name to the determination unit 14. In addition, the recognition unit 13 calculates the reliability of recognition, and outputs the calculated reliability to the determination unit 14.

  As described above, the determination unit 14 determines that the reliability is equal to or less than the threshold value, outputs the occupant name as a recognition result to the confirmation unit 15, and instructs the user to confirm the occupant name.

  Then, the confirmation unit 15 displays the occupant name on the display unit 32 based on the instruction, thereby confirming the correctness of the occupant name to the user. Here, the confirmation unit 15 may confirm the correctness of the recognition result with the user by using an image such as an occupant image together with the occupant name.

  For example, when an occupant image is stored in the prediction dictionary database 21, the confirmation unit 15 may display the occupant image on the display unit 32. Further, the confirmation unit 15 may output the occupant name text data to the in-vehicle device 50 and cause the in-vehicle device 50 to display the occupant name occupant image on the display unit 32.

  As described above, by displaying the recognition result together with an image such as an occupant image, the user can grasp the recognition result more intuitively than in the case of confirming the recognition result only from characters.

  And the confirmation part 15 will confirm the correctness of a recognition result based on the operation signal input from the operation reception part 11, and the text data which means the affirmation and denial input from the recognition part 13. FIG.

  Next, an input operation when performing destination setting will be described with reference to FIGS. 5A to 5C. 5A to 5C are diagrams illustrating an input operation in destination setting. FIG. 5A shows a scene where the destination setting screen is displayed on the touch panel display 30. The following input operation can also be applied to audio music search.

  Further, the description will be made assuming that the prediction dictionary 2 corresponding to the destination setting has already been selected by the selection unit 12 (see FIG. 3A).

  For example, the user utters “Tokyo” in a state where the cursor P on the destination setting screen displayed on the display unit 32 of the touch panel display 30 is pressed, that is, in a state where the operation receiving unit 11 is continuing to accept operations. And In such a case, the operation reception unit 11 outputs a permission notification to the recognition unit 13.

  When the recognition unit 13 acquires the permission notification, the recognition unit 13 recognizes the voice based on the prediction dictionary 2 and outputs “Tokyo” to the determination unit 14 as text data. Further, the recognition unit 13 calculates the reliability and outputs the calculated reliability to the determination unit 14.

  The determination unit 14 determines that the reliability is equal to or less than the threshold, outputs the text data “Tokyo” input from the recognition unit 13 to the confirmation unit 15, and instructs the user to confirm.

  Here, the confirmation unit 15 can output the prediction candidate list to the display unit 32 based on the prediction dictionary. Specifically, the confirmation unit 15 refers to, for example, the frequency for each character string starting from “Tokyo” from the prediction dictionary 2 of the prediction dictionary database 21. For example, the confirmation unit 15 displays the prediction candidate list on the display unit 32 in descending order of frequency.

  In FIG. 5B, the case where Tokyo Disneyland, Tokyo station, etc. are displayed on the display part 32 as a candidate site which is a prediction candidate list is illustrated.

  Here, for example, when the user presses the area Rp1 in which Tokyo Disneyland is displayed, the operation accepting unit 11 accepts a selection operation of the area Rp1 via the operation unit 31, and confirms an operation signal corresponding to the operation. Output to.

  When acquiring the operation signal, the confirmation unit 15 outputs the text data “Tokyo Disneyland” to the in-vehicle device 50. Thereby, in the vehicle-mounted device 50, the destination is set to “Tokyo Disneyland”.

  In this way, by displaying the prediction candidate list based on the history such as the frequency and allowing the user to select it, the input operation can be performed quickly.

  Further, for example, when the user continuously utters “Disneyland” without selecting the region Rp1 or the region Rp2, the recognition unit 13 sends the text data of “Tokyo Disneyland” via the determination unit 14 to the confirmation unit. 15 is output.

  The confirmation unit 15 displays the text data on the display unit 32. Here, for example, a confirmation image shown in FIG. 5C is displayed on the display unit 32.

  For example, when the user performs a touch operation on the region R2 in which “Yes” is displayed, the confirmation unit 15 outputs text data corresponding to “Tokyo Disneyland” to the in-vehicle device 50, and the destination is determined.

  Further, when the user performs a touch operation on the region R <b> 3 in which “No” is displayed, the confirmation unit 15 outputs, for example, another prediction candidate list to the display unit 32. In such a case, the confirmation unit 15 displays the prediction candidate list based on the frequency and the distance from the current location, for example. The prediction candidate list is preferably displayed excluding the candidate sites shown in FIG. 5B.

  Further, when the user performs a touch operation on the region R <b> 4 in which “Return” is displayed, the operation reception unit 11 acquires an operation signal based on the touch operation from the operation reception unit 11 and outputs the operation signal to the confirmation unit 15. When the confirmation unit 15 acquires the operation signal, for example, the confirmation unit 15 returns the display unit 32 to the destination setting screen shown in FIG. 5A. Thereby, the user can redo the destination setting from the beginning.

  When the user performs a touch operation on the region R <b> 5 in which “manual input” is displayed, for example, the confirmation unit 15 displays a keyboard for manual input on the touch panel display 30. Thereby, the user can perform a reliable input operation by manual input using the keyboard.

  5C, when the user inputs text data corresponding to the regions R2 to R5 from the recognition unit 13, the confirmation unit 15 executes processing corresponding to the regions R2 to R5. You can also. That is, the user can also perform each operation corresponding to area | region R2-R5 by speaking the content corresponding to area | region R2-area | region R5.

  Thus, since the destination can be input by voice input, the user can easily input a long character string. Note that, for example, the destination setting prediction dictionary 2 may be further divided for each facility, address, and place name so that the divided prediction dictionary can be selected.

  Next, an input operation when changing the scale of an image will be described with reference to FIGS. 6A and 6B. 6A and 6B are diagrams showing scenes of changing the scale of an image.

  6A and 6B, the case where the scale of the surrounding monitor image is changed will be described. However, the present invention can also be applied to the case where the scale of the map image is changed or the scale of the image is changed.

  In addition, it is assumed that the prediction dictionary 3 corresponding to the scale change of the surrounding monitor image has already been selected by the selection unit 12 (see FIG. 3A).

  Here, the surrounding monitor image is an image that shows a state of the surroundings of the vehicle so as to look down from above the vehicle, for example, by combining the captured images captured by cameras arranged around the vehicle.

  FIG. 6A illustrates a scene in which the host vehicle C1 and the other vehicle C2 exist on the right rear side of the host vehicle C1 in the surrounding monitor image. Here, it is assumed that the host vehicle C1 moves backward.

  For example, when it is desired to enlarge and display a surrounding monitor image, it is assumed that “enlarge” is spoken while performing a touch operation on a portion of the touch panel display 30 to be enlarged.

  The recognizing unit 13 acquires the above-described permission notification from the operation receiving unit 11, and converts the voice signal input from the microphone M into “enlarged” text data in a state where the permission notification is acquired. Further, the recognition unit 13 calculates the reliability and outputs the calculated reliability to the determination unit 14.

  As described above, in this case, the determination unit 14 determines not to confirm the recognition result with the user, and outputs the text data to the in-vehicle device 50 without confirming with the user.

  In the in-vehicle device 50, when such text data is input, the touch position of the user is acquired from the operation unit 31, the image is enlarged by a predetermined magnification around the touch position, and the enlarged image is displayed on the display unit 32. indicate.

  As a result, as shown in FIG. 6B, the image at the position touched by the user is enlarged and displayed. For example, when the user speaks “reduction” while touching the image, the reduced image is displayed with the touch position as the center.

  As described above, since the scale of the image is changed using both the touch operation and the voice input, the user can easily change the scale of the image. In this case, since the correctness of the recognition result is not confirmed with the user, the overall time required for changing the scale of the image can be shortened.

  Next, an input procedure executed by the input device 1 according to this embodiment will be described with reference to FIG. FIG. 7 is a flowchart illustrating a processing procedure executed by the input apparatus 1 according to the present embodiment. In addition, the process shown below is repeatedly performed by the control part 10 of the input device 1. FIG.

  As shown in FIG. 7, first, the operation reception unit 11 receives a user operation (step S101). Subsequently, the selection unit 12 selects a prediction dictionary based on the operation (step S102).

  Subsequently, the recognition unit 13 recognizes speech based on the prediction dictionary (step S103). The recognizing unit 13 calculates the reliability of recognition based on the prediction dictionary selected by the selecting unit 12 (step S104).

  Subsequently, the determination unit 14 determines whether or not the reliability calculated by the recognition unit 13 exceeds a threshold value (step S105). In this determination, when the reliability is larger than the threshold (step S105, Yes), the determination unit 14 does not confirm the recognition result to the user, outputs the recognition result to the in-vehicle device 50 (step S110), and ends the process. .

  If the reliability of the prediction dictionary is equal to or lower than the threshold value in the determination in step S105 (No in step S105), the confirmation unit 15 outputs the recognition result to the display unit 32 and causes the user to confirm (step S106).

  Subsequently, the confirmation unit 15 determines whether or not the user has determined that the recognition result is correct (step S107). If it is determined in step S107 that the user determines that the recognition result is correct (step S107, Yes), the confirmation unit 15 outputs the recognition result to the in-vehicle device 50 (step S110) and ends the process.

  When the user determines that the recognition result is incorrect in the determination in step S107 (No in step S107), the confirmation unit 15 displays the prediction candidate list on the display unit 32 (step S108). In step S <b> 108, the confirmation unit 15 may display an input keyboard on the display unit 32.

  Subsequently, the confirmation unit 15 receives a selection operation (step S109), outputs the selected recognition result to the in-vehicle device 50 (step S110), and ends the process.

  As described above, the input device 1 according to the present embodiment includes the operation reception unit 11, the selection unit 12, the recognition unit 13, the determination unit 14, and the confirmation unit 15. The operation reception unit 11 receives a user operation. The selection unit 12 selects a prediction dictionary corresponding to the operation received by the operation reception unit 11 from a plurality of prediction dictionaries. The recognition unit 13 recognizes the user's voice based on the prediction dictionary selected by the selection unit 12 and calculates the reliability of the recognition. Based on the reliability calculated by the recognition unit 13, the determination unit 14 determines whether to confirm with the user whether the recognition result by the recognition unit 13 is correct or incorrect. The confirmation unit 15 causes the user to confirm the recognition result when the determination unit 14 determines to confirm the correctness of the recognition result. Therefore, an input operation can be performed quickly and accurately.

  In addition, although embodiment mentioned above demonstrated the case where the determination part 14 determined the necessity of a confirmation according to the reliability calculated by the recognition part 13, it is not restricted to this. That is, the recognition unit 13 may output all the recognition results to the confirmation unit 15, and the confirmation unit 15 may confirm all the recognition results.

DESCRIPTION OF SYMBOLS 1 Input device 10 Control part 11 Operation reception part 12 Selection part 13 Recognition part 14 Judgment part 15 Confirmation part 21 Predictive dictionary database 50 In-vehicle apparatus 100 Display apparatus

Claims (9)

An operation reception unit for receiving user operations;
A selection unit that selects the prediction dictionary corresponding to the operation received by the operation reception unit from a plurality of prediction dictionaries;
A recognition unit for recognizing the user's voice and calculating the reliability of the recognition based on the prediction dictionary selected by the selection unit;
Based on the reliability calculated by the recognition unit, a determination unit that determines whether to confirm the correctness of the recognition result by the recognition unit with the user;
An input device comprising: a confirmation unit that allows the user to confirm the recognition result when the determination unit determines to confirm the correctness of the recognition result. The recognition unit
The input device according to claim 1, wherein the operation receiving unit recognizes the voice when the operation is being accepted and does not recognize the voice when the operation is not being continued. The operation reception unit
Accept a touch operation on a predetermined area on the operation surface, do not accept the touch operation other than the predetermined area,
The recognition unit
The input device according to claim 1, wherein the voice is recognized when the operation reception unit receives the touch operation. The determination unit
4. The input device according to claim 1, wherein when the recognition result of the recognition unit is not included in the selected prediction dictionary, it is determined that the user confirms correctness. The confirmation unit
When the determination unit determines to confirm the correctness of the recognition result with the user, the input candidate list is output to the display device based on the prediction dictionary,
The operation reception unit
The input device according to any one of claims 1 to 4, wherein an input candidate selection operation is received from the input candidate list. The determination unit
The input device according to any one of claims 1 to 5, wherein when the prediction dictionary related to image scale change is selected by the selection unit, it is determined that the user does not confirm correctness. An input device according to any one of claims 1 to 6;
And a display unit that displays an image based on an output result of the input device. An operation reception process for receiving a user operation;
A selection step of selecting the prediction dictionary corresponding to the operation received in the operation reception step from a plurality of prediction dictionaries;
A recognition step of recognizing the user's voice based on the prediction dictionary selected in the selection step and calculating a reliability of the recognition;
A determination step of determining whether to confirm to the user whether the recognition result is correct or incorrect based on the reliability calculated in the recognition step;
And a confirmation step for allowing the user to confirm the recognition result when it is determined in the determination step to confirm whether the recognition result is correct or incorrect. An operation acceptance procedure for accepting user operations;
A selection procedure for selecting the prediction dictionary corresponding to the operation received in the operation reception procedure from a plurality of prediction dictionaries;
A recognition procedure for recognizing the user's voice and calculating the reliability of the recognition based on the prediction dictionary selected in the selection procedure;
Based on the reliability calculated in the recognition procedure, a determination procedure for determining whether to confirm the correctness of the recognition result by the recognition procedure with the user;
An input program for causing a computer to execute a confirmation procedure for allowing the user to confirm the recognition result when it is determined in the determination procedure to confirm the correctness of the recognition result.

Images