JP2023029982A - Operation input method, operation input system, and operation terminal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an operation input method that improves convenience in inputting of a user operation even in a situation where it is difficult for the user to perform an operation with a hand.

SOLUTION: An operation input method comprises steps of: collecting a voice uttered by a user while a first screen is displayed on an operation terminal; recognizing the collected voice; determining whether or not a recognition result matches input content for a confirmation item shown on the first screen; instructing, when the recognition result matches the input content for the confirmation item shown on the first screen, the operation terminal to switch display from the first screen to a second screen; determining whether or not the recognition result issued by the user matches the input content for the confirmation item displayed on a m-th screen while the m-th screen (an integer that satisfies 2≤m≤N, N: an integer greater than or equal to 4) is displayed on the operation terminal; and instructing, when the recognition result matches the input content for the confirmation item shown on the m-th screen, the operation terminal to switch the display from the m-th screen to a (m+1)th screen.

SELECTED DRAWING: Figure 2

COPYRIGHT: (C)2023,JPO&INPIT

Description

The present disclosure relates to an operation input method, an operation input system, and an operation terminal that operate using voice.

In Patent Document 1, by performing speech recognition processing on input speech corresponding to an address, a first candidate corresponding to each of a plurality of words forming the address is determined, and each of the determined plurality of words is A speech recognition device is disclosed that presents a user with a plurality of first candidates corresponding to . After presenting the recognition result, the speech recognition device accepts an instruction to correct the recognition result from the user. A second candidate is determined that excludes the corresponding first candidate. As a result, the number of utterances required to input the address by voice can be reduced, and the user's operation can be simplified.

JP 2017-102320 A

However, in the configuration of Patent Literature 1, for example, when there is a misrecognized word, processing such as pressing the back switch is required for the recognition result correction instruction from the user. Is required. For this reason, for example, when the user is in a situation where the user cannot use his or her hands, there is a problem that the user's convenience is reduced in that the recognition result cannot be corrected and the processing cannot proceed smoothly. rice field.

The present disclosure has been devised in view of the above-described conventional situation, and an operation input method and an operation input system that improve convenience when inputting user operations even in situations where it is difficult for the user to perform operations using hands. and to provide an operation terminal.

The present disclosure includes a step of collecting a voice uttered by a user while a first screen is displayed on an operation terminal; a step of recognizing the collected voice; determining whether or not the input contents for the confirmation items shown on the screen match the input contents for the confirmation items shown on the first screen; a step of instructing the operation terminal to switch display from to a second screen; determining whether or not the recognition result of the voice uttered by the user matches the input contents for the confirmation items displayed on the m-th screen; and and instructing the operation terminal to switch display from the m-th screen to the (m+1)-th screen when the input contents match the displayed confirmation items. .

Further, the present disclosure is an operation input system in which an operation terminal having a voice input device and a display device and a voice processing device are communicably connected, wherein the operation terminal displays a first screen on the display device. In the displayed state, the speech input device picks up the voice uttered by the user, the operation terminal or the speech processing device recognizes the picked-up speech, and the speech processing device recognizes the recognition result of the speech determining whether or not the input contents for the confirmation items shown on the first screen match the input contents for the confirmation items shown on the first screen; The voice processing device instructs the operation terminal to switch the display from the screen to the second screen, and the voice processing device instructs the operation terminal to display the m-th screen (integer satisfying 2 ≤ m ≤ N, N: 4 or more (integer) is displayed, it is determined whether or not the recognition result of the voice uttered by the user matches the input content for the confirmation item shown on the m-th screen, and the recognition result of the voice is the m-th screen. To provide an operation input system for instructing an operation terminal to switch display from the m-th screen to the (m+1)-th screen when input contents for confirmation items shown on the th screen match.

Further, the present disclosure includes a voice input device that collects a voice uttered by a user while a first screen is displayed on a display device, a recognition unit that recognizes the collected voice, and a recognition result of the voice. a control unit that determines whether or not the input contents of the confirmation items displayed on the first screen match the confirmation items displayed on the first screen. , the control unit instructs the display device to switch the display from the first screen to the second screen, and the control unit instructs the display device to display the mth screen (2 ≤ m ≤ (integer satisfying N, where N is an integer equal to or greater than 4) is displayed, it is determined whether or not the recognition result of the voice uttered by the user matches the input contents for the confirmation items shown on the m-th screen. , if the speech recognition result matches the input content for the confirmation item shown on the m-th screen, switching the display from the m-th screen to the (m+1)-th screen on the display device Provide an operation terminal to instruct.

Advantageous Effects of Invention According to the present disclosure, it is possible to improve convenience when inputting a user operation even in a situation where it is difficult for the user to perform an operation using hands.

1 is a diagram showing a schematic configuration of an operation input system according to Embodiment 1; FIG. Block diagram showing the hardware configuration of the operation terminal and voice processing device A table showing the content registered in the keyword database The figure which shows the transition of the voice input screen of a display apparatus. FIG. 4 is a sequence diagram showing an example of the operation procedure of speech recognition according to Embodiment 1; A sequence diagram showing an example of a speech recognition operation procedure following FIG. The figure which shows an example of the inspection result screen displayed on the display apparatus Flowchart showing an example of a speech recognition operation procedure according to Modification 1 of Embodiment 1 Flowchart showing an example of a speech recognition operation procedure following FIG. FIG. 10 is a diagram showing transition of voice input screens of the display device according to Embodiment 2; Sequence diagram showing the speech recognition procedure in the operation input system FIG. 11 is a sequence diagram showing the speech recognition procedure in the operation input system following FIG. Flowchart showing a speech recognition procedure according to Modification 1 of Embodiment 2 Flowchart showing the speech recognition procedure following FIG.

Hereinafter, embodiments specifically disclosing the configurations and actions of an operation input method, an operation input system, and an operation terminal according to the present disclosure will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed descriptions of well-known matters and redundant descriptions of substantially the same configurations may be omitted. This is to avoid unnecessary verbosity in the following description and to facilitate understanding by those skilled in the art. It should be noted that the accompanying drawings and the following description are provided for a thorough understanding of the present disclosure by those skilled in the art and are not intended to limit the claimed subject matter.

(Embodiment 1)
FIG. 1 is a diagram showing a schematic configuration of an operation input system 5 according to Embodiment 1. As shown in FIG. The operation input system 5 recognizes a voice uttered by a user carrying the operation terminal 10 and displays information corresponding to the voice recognition result. The operation input system 5 includes an operation terminal 10 connected to a network NW via an access point 40, and a voice processing device 50 connected to the network NW.

The access point 40 is a device that wirelessly connects the operation terminal 10 to the network NW. The network NW is a wide area network such as a wired LAN (Local Area Network), a wireless LAN, or the Internet.

The operation terminal 10 is a terminal shared by a plurality of users, and is composed of a tablet terminal capable of voice input and display. The tablet terminal has an application for transmitting input voice data as it is to the voice processing device 50, an application for recognizing the input voice and converting it into text data, and transmitting this text data to the voice processing device 50. An application is installed for each. In addition to the tablet terminal, the operation terminal 10 may be a computer terminal such as a smartphone, a notebook PC (Personal Computer), or a PDA (Personal Digital Assistant), which are electronic devices having similar information processing capability and communication function.

The voice processing device 50 is configured by a general-purpose computer. The voice processing device 50 recognizes voice based on the voice data transmitted from the operation terminal 10, converts it into text data, acquires display information corresponding to this text data, or generates screen data. to the operation terminal 10. Note that the voice processing device 50 may acquire display information corresponding to voice data or generate screen data and transmit it to the operation terminal 10 without converting it into text data.

FIG. 2 is a block diagram showing the hardware configuration of the operation terminal 10 and the audio processing device 50. As shown in FIG. The operation terminal 10 includes a processor 11 , a memory 12 , a communication circuit 13 , an audio input device 14 and a display device 15 .

The processor 11 is configured using, for example, a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field Programmable Gate Array), and controls the operation of each section of the operation terminal 10 . The processor 11 functions as a control unit of the operation terminal 10, and performs control processing for overall control of the operation of each unit of the operation terminal 10, data input/output processing with each unit of the operation terminal 10, and data calculation. (computation) to process and store data; The processor 11 operates according to execution of programs stored in a ROM (Read Only Memory) in the memory 12 .

The processor 11 also has a speech recognition unit 25 as a functional configuration. The voice recognition unit 25 is a software component configured according to the execution of a program stored in the ROM in the memory 12, recognizes voice input by the voice input device 14, and converts it into text data. The processor 11 displays various information on the display device 15 based on the screen data. Processor 11 also displays speech candidates on display device 15 to facilitate speech input. Processor 11 generates screen data for display on display device 15 .

The memory 12 includes, for example, RAM (Random Access Memory) and ROM (Read Only Memory), and temporarily stores programs and data necessary for executing operations of the operation terminal 10, information or data generated during operations, and the like. do. The RAM is a work memory used when the processor 11 operates, for example. The ROM pre-stores programs and data for controlling the processor 11, for example. The memory 12 also stores an employee ID (Identification Number) and a keyword table 12z in which a plurality of inspection items corresponding to each employee ID are registered (see FIG. 3).

The communication circuit 13 is composed of a communication circuit capable of communicating with the speech processing device 50 via the network NW, or a NIC (Network Interface Card) on which the communication circuit is mounted.

The voice input device 14 is composed of a microphone for picking up voice. The microphone may be either a directional microphone or an omnidirectional microphone.

The display device 15 displays the screen data transmitted from the audio processing device 50 according to instructions from the processor 11 . Note that when the operation terminal 10 generates screen data, the display device 15 displays the screen data generated by the processor 11 . Note that the display device 15 may be configured by a touch panel capable of touch input operation.

The audio processing device 50 includes a processor 51 , a memory 52 , a communication circuit 53 and a storage 54 .

The memory 52 includes, for example, RAM (Random Access Memory) and ROM (Read Only Memory), and temporarily stores programs and data necessary for executing the operation of the speech processing device 50, information or data generated during operation, and the like. save. The RAM is a work memory used when the processor 51 operates, for example. The ROM pre-stores programs and data for controlling the processor 51, for example.

The communication circuit 53 is composed of a communication circuit capable of communicating with the operation terminal 10 via the network NW, or a NIC (Network Interface Card) on which the communication circuit is mounted.

The storage 54 is a large-capacity storage medium, and stores, for example, speech data, speech recognition results, keyword database (DB) 541, and the like.

All employee IDs and a plurality of inspection items corresponding to each employee ID are registered in the keyword DB 541 .

The processor 51 is configured using, for example, a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or an FPGA (Field Programmable Gate Array), and controls the operation of each section of the audio processing device 50 . The processor 51 functions as a control unit of the audio processing device 50, and performs control processing for overall control of the operation of each unit of the audio processing device 50, data input/output processing with each unit of the audio processing device 50, Performs data arithmetic (calculation) processing and data storage processing. The processor 51 operates according to the execution of programs stored in a ROM (Read Only Memory) in the memory 52 .

The processor 51 has a control section 61, a speech recognition section 62, and a keyword matching section 63 as functional components. These units are functions realized by the processor 51 executing a control program stored in the built-in memory or the memory 52 .

The voice recognition unit 62 is a software component configured according to the execution of a program stored in the ROM in the memory 52, and recognizes the voice of the voice data transmitted from the operation terminal 10 and converts it into text data. . Note that when text data, which is the result of voice recognition already performed, is transmitted from the operation terminal 10 instead of voice data, the voice recognition processing by the voice recognition unit 62 is omitted.

The keyword matching unit 63 is a software-like component configured according to the execution of the program stored in the ROM in the memory 52, and collates the text that is the speech recognition result with the keywords registered in the keyword DB 541. FIG. For example, if the speech-recognized text is a number or a series of numbers (sequence) thereof, the keyword matching unit 63 collates this number with the employee ID registered in the keyword DB 541, and if they match, the user is an employee with this employee ID. When the employee ID is successfully authenticated, the keyword matching unit 63 picks up (extracts) one or more inspection items corresponding to this employee ID. The keyword matching unit 63 matches the next voice uttered by the user to the order of inspection items (check number). For example, if the employee ID is "123456", the inspection items to be performed by the employee with that employee ID include checking the engine, checking the brake disc, checking the accelerator pedal, checking the battery, and the like. Defined and picked up. The keyword matching unit 63 determines the order of inspection items as the utterance order of the voice uttered by the user.

The control unit 61 is a software component configured according to the execution of a program stored in the ROM in the memory 52, generates screen data including inspection items such as a speech recognition result and a check number, and outputs the screen data to the operation terminal 10. Send. Further, the control unit 61 stores the inspection item of the voice recognition result and the check number in the storage 54 .

FIG. 3 is a table showing registered contents of the keyword database (DB) 541. As shown in FIG. In the keyword database (DB) 541, for example, check 1: engine check, check 2: brake disc check, check 3: accelerator pedal check, and check items corresponding to employee ID "123456" are stored. Check 4: Checking the battery and the like are defined and registered in advance.

Check 1: Check engine oil, Check 2: Check coolant, Check 3: Check brake oil, Check 4: Check fuel amount, etc. is defined and registered in advance.

FIG. 4 is a diagram showing transition of the voice input screen of the display device 15. As shown in FIG. After being activated, the operation terminal 10 displays an employee ID input screen GA1 on the display device 15 . On the employee ID input screen GA1, a message ms1 of "enter employee ID" and an input box bx1 below it are displayed. The operation terminal 10 picks up the voice uttered by the user through the voice input device 14 and accepts the number included in the voice as the employee ID. In the example of FIG. 4, the user uttered the voice "123456", and "123456", which is the recognition result of the voice, is entered in the input box bx1.

When receiving the voice recognition result of the employee ID, the operation terminal 10 authenticates whether or not the employee corresponding to the employee ID is a pre-registered employee. The corresponding inspection item input screen GA2 is displayed. Above the input screen GA2, the employee ID input by recognizing the user's voice picked up when the previous input screen GA1 was displayed is displayed. The input screen GA2 displays check numbers 1, 2, 3, and N (N=4 in the example of FIG. 4), inspection details ct1, ct2, ct3, and ct4, and input boxes by1, by2, and by3 for each inspection item. , by4 are associated with each other in order and displayed. Note that N is a positive integer of 4 or more.

The check numbers 1 to N correspond to, for example, the order of utterances made by the user every time he/she finishes checking. As an example, the inspection content of check 1 is confirmation of the engine. In the input box by1, the recognition result (text data) of the voice "OK" uttered by the user as the inspection confirmation result of the engine mounted in the vehicle (not shown) is input.

Similarly, the inspection content of check 2 is confirmation of the brake disc. In the input box by2, the recognition result (text data) of the voice "OK" uttered by the user as the inspection confirmation result of the brake disc mounted in the vehicle is input.

The inspection content of check 3 is confirmation of the accelerator pedal. The recognition result (text data) of the voice "OK" uttered by the user as the check confirmation result of the accelerator pedal mounted in the vehicle is input to the input box by3.

The inspection content of check 4 is confirmation of the battery. In the input box by4, for example, the recognition result (text data) of the voice "NG" uttered by the user when the user determines that the battery voltage is low as a result of checking the battery installed in the vehicle is input. be done.

Next, operation procedures of the operation input system 5 according to the first embodiment will be described.

In order to make the following explanation easier to understand, as an example, the use case is an example in which an operator who inspects an object to be inspected (e.g., a vehicle) records the inspection result of the vehicle while the operator's hands are occupied. explain. As a state in which the hands are occupied, for example, a situation is assumed in which the worker puts away the used tools or washes his hands after the inspection is finished. In addition, it is assumed that the operator wears gloves on an operation terminal having a touch panel that allows input by touch operation with bare hands.

5 and 6 are sequence diagrams showing an example of the operation procedure of speech recognition.

In FIG. 5, when the operation terminal 10 is activated by the user's power-on operation, the processor 11 of the operation terminal 10 starts speech recognition operation. The processor 11 displays an employee ID input screen GA1 on the display device 15 (T1). With the employee ID input screen GA1 displayed, the user speaks (for example, the number "123456"). The processor 11 picks up the user's speech with the voice input device 14 (T2). Processor 11 transmits this audio data to audio processing device 50 via communication circuit 13, access point 40 and network NW (T3).

The control unit 61 of the processor 51 of the voice processing device 50 receives voice data transmitted from the operation terminal 10 via the communication circuit 53 and stores the voice data in the memory 52 . The speech recognition unit 62 of the processor 51 performs speech recognition on the speech data stored in the memory 52 (T4).

The keyword matching unit 63 of the processor 51 determines whether or not the voice-recognized text data is input for the corresponding item (employee ID input item) (T5). In this determination, the keyword matching unit 63 refers to the keyword DB 541 stored in the storage 54, determines whether or not there is an employee ID corresponding to the voice-recognized text data, and authenticates the user's employee ID. Furthermore, when the authentication result of the employee ID is OK, the keyword matching unit 63 picks up a plurality of inspection items corresponding to the employee ID registered in the keyword DB 541 .

The control unit 61 generates screen data for the inspection item input screen GA2 based on the authentication OK of the employee ID and a plurality of inspection items corresponding to the employee ID (T6). The control unit 61 transmits the screen data of the inspection item input screen GA2 to the operation terminal 10 via the communication circuit 53 and the network NW (T7).

Note that if the authentication result of the employee ID is NG, the control unit 61 of the voice processing device 50 waits for reception of voice data from the operation terminal 10 again. At this time, the control unit 61 may reply to the operation terminal 10 to the effect that the employee ID of the user cannot be authenticated, or may reply nothing. Further, when the inspection item corresponding to the employee ID is not registered in the keyword DB 541, the control unit 61 returns to the operation terminal 10 that the inspection item corresponding to the employee ID of the user is not registered. OK, you don't have to reply anything.

The processor 11 of the operation terminal 10 receives the screen data of the inspection item input screen GA2 from the voice processing device 50 via the communication circuit 13, the access point 40 and the network NW, and displays the inspection item input screen GA2 on the display device 15. (T8). Note that if the processor 11 does not receive the screen data of the inspection item input screen GA2 from the voice processing device 50 even after a while has passed since the voice input device 14 picked up the user's utterance, the processor 11 performs step T2. and repeat the same process. As a result, the user can speak the employee ID again, and can deal with the case where the sound cannot be picked up properly due to sudden noise or the like.

While the inspection item input screen GA2 is displayed on the display device 15, the processor 11 picks up the user's speech with the voice input device 14 (T9), as in step T2. At this time, the user pronounces, for example, "OK" or "NG" for the inspection item of check 1 (here, check of the engine). Processor 11 transmits this audio data to audio processing device 50 via communication circuit 13, access point 40 and network NW (T10). The control unit 61 of the voice processing device 50 receives voice data transmitted from the operation terminal 10 via the communication circuit 53 and stores the voice data in the memory 52 . The speech recognition unit 62 performs speech recognition on the speech data stored in the memory 52 (T11).

The control unit 61 determines whether or not the voice-recognized text data is input for the previous input item (employee ID input item) (T12). If the voice-recognized text data is an input for the employee ID entry item, that is, if it is a six-digit number, the control unit 61 operates the employee ID text data via the communication circuit 53 and the network NW. It is transmitted to the terminal 10 (T13). When the processor 11 of the operation terminal 10 receives the text data of the employee ID from the speech processing device 50 via the communication circuit 13, the access point 40 and the network NW, it determines that this reception is the correction of the employee ID, and corrects it. The inspection item input screen GA2 is updated so as to reflect the text data of the received employee ID (T14).

On the other hand, if the voice-recognized text data in step T12 is not an input for the employee ID input item, the control unit 61 performs an input (for example, "OK ”) is determined (T15). If the voice-recognized text is not the input for the inspection item of check 1, the control unit 61 returns to step T11 and waits until voice data is received again.

On the other hand, if the text data voice-recognized in step T15 is an input for the check item of check 1, the control unit 61 transmits the text data for the check item of check 1 to the operation terminal 10 via the communication circuit 53 and the network NW. (T16). The processor 11 of the operation terminal 10 receives the text data for the check item of check 1 from the voice processing device 50 via the communication circuit 13, the access point 40 and the network NW, and converts the check item so as to reflect this text data. is updated (T17). In the updated inspection item input screen GA2, the input box by1 for the confirmation of the engine in Check 1 displays the characters "OK" (see FIG. 4). Similar processing from procedure T9 to procedure T17 is repeated N times corresponding to the number of inspection items. That is, if the inspection item number (check number) is represented by the k-th inspection item, the same processing from procedure T9 to procedure T17 is performed with k=1 to N.

After that, in a state where the check (N-1) inspection item has been input on the display device 15, that is, the input screen GA2 for check N is displayed, the processor 11 uses the voice input device 14 as in step T9. The utterance of the person is collected (T18). Processor 11 transmits this audio data to audio processing device 50 via communication circuit 13, access point 40 and network NW (T19). The control unit 61 of the voice processing device 50 receives voice data transmitted from the operation terminal 10 via the communication circuit 53 and stores the voice data in the memory 52 . The speech recognition unit 62 performs speech recognition on the speech data stored in the memory 52 (T20).

The control unit 61 determines whether or not the voice-recognized text data is an input for the previous check item (check (N-1) check item) (T21). For example, when the user utters "previous item NG" as a keyword, it is determined that the input is for the previous input item.

When the voice-recognized text data is an input for the inspection item of check (N-1), the control unit 61 transmits the corrected text data of check (N-1) via the communication circuit 53 and the network NW. It is transmitted to the operation terminal 10 (T22). When the processor 11 of the operation terminal 10 receives the text data of the corrected check (N-1) from the speech processing device 50 via the communication circuit 13, the access point 40 and the network NW, the processor 11 reflects this text data. Then, the inspection item input screen GA2 is updated (T23).

On the other hand, if the voice-recognized text data in step T21 is not an input for the check item of check (N-1), the control unit 61 selects the item to which the voice-recognized text data corresponds (here, the check item of check N). ) (for example, "OK") (T24). If the voice-recognized text data is not the input for the inspection item of check N, the control unit 61 waits until voice data is received in step 19 .

On the other hand, if the text voice-recognized in step T21 is input for the inspection item of check N, the control unit 61 generates an inspection result screen GA3 (see FIG. 7) (T25). The control unit 61 transmits screen data of the inspection result screen GA3 to the operation terminal 10 via the communication circuit 53 and the network NW (T26). The processor 11 of the operation terminal 10 receives the screen data of the inspection result screen GA3 from the voice processing device 50 via the communication circuit 13, the access point 40 and the network NW, and displays the inspection result screen GA3 on the display device 15 ( T27).

FIG. 7 is a diagram showing an example of the inspection result screen GA3 displayed on the display device 15. As shown in FIG. On the inspection result screen GA3, the employee ID, inspection date and time, and the inspection results of checks 1 to N are displayed in a list. In Fig. 7, employee ID: 123456, inspection date: 2018, month, day, check 1: engine check OK, check 2: brake disc check OK, check 3: accelerator pedal check OK, check 4: battery Confirmation of NG is displayed.

As described above, in the operation input system of Embodiment 1, operations from inputting the employee ID to checking the inspection items and displaying the inspection result screen can be performed only by simple utterances without requiring the user's hands. can be completed with In particular, the order of checks can be matched to the order of utterance without uttering the check number, and the number of utterances can be reduced. This simplifies the operation by speaking. Also, regardless of the speaking order, if the format of a word to be entered (e.g. 4-digit number) is different from the format of other words (e.g. 2-letter alphabet), the item name specifying the input target can be omitted. good.

In addition, when correcting the confirmation of the previous check item, the user utters "Previous item NG" as an example, but is not limited to this, and may utter "Check number ○○ NG". By inputting the check number, it is also possible to correct the input of the check number two or more before. In addition, the simple words uttered by the user used in the input operation are not limited to numbers, OK, and NG, but may be YES representing affirmation, NO representing negation, A, B, C representing rank, etc. good too. This reduces input errors.

As described above, the operation input method according to the first embodiment includes the step of collecting the voice uttered by the user while the employee ID input screen GA1 (first screen) is displayed on the operation terminal 10; a step of recognizing the collected voice; a step of determining whether or not the recognition result of the voice matches the input contents of the inspection items (confirmation items) shown on the employee ID input screen GA1; and a step of recognizing the voice. If the result matches the input content for the employee ID input item shown on the employee ID input screen GA1, switching the display from the employee ID input screen GA1 to the inspection item input screen GA2 (second screen); and a step of instructing the operation terminal 10 to display on the input screen GA2 the check item of the speech recognition result corresponding to the input item of the employee ID shown on the input screen GA1 of the employee ID.

As a result, according to the operation input method, the operation input system 5, or the operation terminal 10, even in a situation where it is difficult for the user to perform an operation using his hands (a state in which the operator's hands are occupied), the work can be performed. Since the inspection can be performed efficiently by a simple task of uttering a voice to the operation terminal 10 in order to record the inspection result of the vehicle, the convenience at the time of inputting the user operation can be improved.

Further, the operation input method is such that the recognition result of the voice uttered by the user is displayed on the inspection item input screen GA2 in a state in which N (N: an integer equal to or greater than 4) inspection item input screens GA2 are displayed on the operation terminal 10. a step of judging whether or not it matches the input content for the k-th inspection item (k: an integer satisfying 1 ≤ k ≤ (N-1)); a step of instructing the operation terminal 10 to display the result of voice recognition corresponding to the k-th inspection item on the inspection item input screen GA2 if it matches the input content for the k-th inspection item shown; have. This enables continuous input of inspection items, improving operability.

Further, in the step of recognizing the voice, if the voice recognition result does not match the input content for the k-th inspection item shown on the inspection item input screen GA2, the user uttered "previous item NG", " It includes a step of receiving a recognition process of a keyword (predetermined keyword) such as "check 3 NO" and the voice uttered again by the user for the k-th inspection item. This makes it possible to easily correct the input content of the previous inspection item even after the input has been completed.

Further, when the speech recognition result matches the input content for the N-th inspection item shown on the inspection item input screen GA2, the N inspection items and the recognition result of the voice uttered by the user for each inspection item are displayed. It further includes a step of instructing the operation terminal 10 to display the associated inspection result screen GA3 (recognition result). Thereby, the user can visually confirm the input contents of all inspection items in a list. Therefore, it becomes easier for the user to find an erroneous input, and input errors can be reduced.

In addition, the input contents for the inspection items from check 1 (first) to check N (Nth) shown on the inspection item input screen GA2 correspond to the employee ID input items shown on the employee ID input screen GA1. Associated with input content. Thereby, inspection items can be managed for each employee ID. In addition, the user can compare the input contents of the inspection items shown on the inspection item input screen GA2 with the contents of the inspection items assumed by the user, and can easily confirm whether the contents are correct or not.

In addition, keywords (predetermined keywords) such as "previous item NG" and "check 3 NO" uttered by the user are used to correct the input content for the previous check item (k-1th check item). It is text data (information) representing (correction). This makes it possible to easily correct the previous check item.

(Modification 1 of Embodiment 1)
In Embodiment 1, the case where the speech processing device 50 receives speech data from the operation terminal 10 and performs speech recognition is shown. explain.

8 and 9 are flowcharts showing an example of a speech recognition operation procedure according to Modification 1 of Embodiment 1. FIG.

In FIG. 8, when the operating terminal 10 is activated by a user's power-on operation or the like, the processor 11 of the operating terminal 10 starts speech recognition operation. The processor 11 displays an employee ID input screen GA1 on the display device 15 (S1). With the employee ID input screen GA1 displayed, the user speaks (for example, the number "123456"). The processor 11 picks up the user's speech with the voice input device 14 (S2).

The processor 11 stores audio data of the collected audio in the memory 12 . The speech recognition unit 25 of the processor 11 performs speech recognition on the speech data stored in the memory 12, and determines whether or not the speech has been recognized (S3). If the speech cannot be recognized, the processor 11 returns to step S2 and performs the sound pickup operation again. As a result, the user can speak the employee ID again, and can deal with the case where the sound cannot be picked up properly due to sudden noise or the like.

If the voice can be recognized in step S3, the processor 11 determines whether or not the voice-recognized text data is an input for the corresponding item (employee ID input item) (S4). If the input is for an employee ID input item, the processor 11 refers to the keyword table 12z stored in the memory 12, determines whether or not there is an employee ID corresponding to the text recognized by voice, and identifies the user's employee ID. Authenticate. Furthermore, when the authentication result of the employee ID is OK, the processor 11 picks up a plurality of inspection items corresponding to the employee ID registered in the keyword table 12z.

If it is determined in step S4 that the input item for the employee ID is not entered, the processor 11 returns to step S2 and performs the sound pickup operation again. Examples of cases where the input is not for the input item of the employee ID include that the authentication result of the employee ID is NG, or that the inspection item corresponding to the user's employee ID is not registered in the keyword table 12z. In these cases, the processor 11 may display on the display device 15 that the authentication result of the employee ID is NG. Moreover, the processor 11 may display on the display device 15 that the inspection item corresponding to the employee ID of the user is not registered.

The processor 11 generates an inspection item input screen GA2 based on the confirmation OK of the employee ID and a plurality of inspection items corresponding to the employee ID (S5). The processor 11 displays the inspection item input screen GA2 on the display device 15 (S6).

With the inspection item input screen GA2 displayed on the display device 15, the processor 11 picks up the user's speech with the voice input device 14 (S7), as in step S2. At this time, the user pronounces, for example, "OK" or "NG" for the inspection item of check 1 engine confirmation.

The processor 11 stores audio data of the collected audio in the memory 12 . The speech recognition unit 25 of the processor 11 performs speech recognition on the speech data stored in the memory 12, and determines whether or not the speech has been recognized (S8). If the speech cannot be recognized, the processor 11 returns to step S7 and performs the sound pickup operation again.

If the voice is recognized in step S8, the processor 11 determines whether or not the voice-recognized text data is an input for the corresponding item (inspection item of check 1) (S9). If the voice-recognized text data is not the input for the inspection item of check 1, the processor 11 determines whether or not the voice-recognized text data is the input for the previous input item (employee ID input item) ( S10). If the input item is not the employee ID, the processor 11 returns to the process of step S8. If the input is not for the employee ID input item, the processor 11 may display nothing, or may display on the display device 15 to prompt for re-input.

If the text data voice-recognized in step S10 is an input for an employee ID input item, that is, if it is a 6-digit number, the processor 11 determines that this input is a correction of the employee ID and corrects the input. The check item input screen GA2 is updated so as to reflect the employee ID (S11). After that, the processor 11 returns to the process of step S7.

If the text data speech-recognized in step S9 is the input for the check item of check 1, the processor 11 is configured to reflect the text data for the check item of check 1 (including the speech recognition result of the input of check 1). ), the inspection item input screen GA2 is updated (S12). In the updated inspection item input screen GA2, the input box by1 for the confirmation of the engine in Check 1 displays the characters "OK" (see FIG. 4).

Similar processing from steps S7 to S12 is repeated N times corresponding to the number of inspection items. That is, if the inspection item number (check number) is represented by the k-th inspection item, the same processing from steps S7 to S12 is performed with k=1 to N.

The processor 11 displays an inspection item input screen GA2 in which the check (N-1) item has been input on the display device 15 (S13). In this display state, the processor 11 picks up the user's speech with the voice input device 14 (S14), as in step S7. The processor 11 stores audio data of the collected audio in the memory 12 . The speech recognition unit 25 of the processor 11 performs speech recognition on the speech data stored in the memory 12, and determines whether or not the speech has been recognized (S15). If the speech cannot be recognized, the processor 11 returns to step S14 and acquires the user's speech again.

The processor 11 determines whether the text data obtained by voice recognition is an input (for example, "OK") for the corresponding item (inspection item of check N) (S16). If the voice-recognized text data is not an input for the check item of check N, for example, if the user utters "previous item NG", the processor 11 detects that the voice-recognized text data is the previous check item (check It is determined whether or not the input is for (N-1) inspection item) (S17).

If the voice-recognized text data is the input for the check N-1 input item, the processor 11 updates the check item input screen GA2 so as to reflect the corrected check (N-1) text data. (S18). After that, the processor 11 returns to the process of step S14. Note that if the text data voice-recognized in step S17 is not an input for the inspection item of check N-1, the processor 11 returns to the process of step S14. At this time, the processor 11 does not have to display anything, or may display on the display device 15 to prompt for re-input.

If the text data voice-recognized in step S16 is an input for the inspection item of check N, the processor 11 generates an inspection result screen GA3 (see FIG. 7) (S19). The processor 11 displays the inspection result screen GA3 on the display device 15 (S20). After this, the processor 11 terminates the voice recognition operation.

As described above, when the user speaks, the operation terminal 10 according to the first modification of the first embodiment performs speech recognition on its own and displays the speech recognition result. Therefore, the operation terminal can be used in places where there is no network environment. Cost reduction can be achieved by not using a network environment and not requiring an audio processing device.

(Embodiment 2)
In the first embodiment, input operations for inspection items are sequentially performed within one screen, but in the second embodiment, the screen transitions for each inspection item and input operations are performed. The configuration of the operation input system 5 according to the second embodiment is substantially the same as the configuration of the operation input system 5 according to the first embodiment. Therefore, by using the same reference numerals for the same components as those in the first embodiment, the description thereof will be simplified or omitted, and the different contents will be described.

FIG. 10 is a diagram showing transition of the voice input screen of the display device 15 according to the second embodiment. After being activated, the operation terminal 10 displays an employee ID input screen GA1 on the display device 15, as in the first embodiment. On the employee ID input screen GA1, a message ms1 of "enter employee ID" and an input box bx1 below it are displayed. The operation terminal 10 picks up the voice uttered by the user with the voice input device 14 and accepts the number contained in the voice as the employee ID. In the example of FIG. 10, the user uttered the voice "123456", and the voice recognition result "123456" is input to the input box bx1.

When the operation terminal 10 receives the recognition result of the voice of the employee ID, the operation terminal 10 authenticates whether or not the employee corresponding to the employee ID is a pre-registered employee. The screen data is received from 50, and the input screen GA12 of the check item of check 1 is displayed. On the upper side of the input screen GA12, the employee ID input by recognizing the user's voice picked up when the previous input screen GA1 was displayed is displayed. Check 1, inspection content ct1, and inspection result ef1 are displayed on the lower side of the input screen GA12. The contents of check 1 are confirmation of the engine. The inspection result is "OK" or "OK".

When the inspection result of Check 1 is confirmed, the operation terminal 10 receives the screen data from the voice processing device 50 and displays the input screen GA13 of the inspection item of Check 2. FIG. On the upper side of the input screen GA13, the result of check 1 input on the previous input screen GA12 is displayed. Check 2, inspection details ct2, and inspection results are displayed on the lower side of the input screen GA13. The contents of the check 2 are confirmation of the brake disc. The inspection result is "OK" or "OK". After that, the screen transition is performed in the same manner up to the input screen of the check N.

Next, operation procedures of the operation input system 5 according to the second embodiment will be described.

In the second embodiment, as in the first embodiment, an example in which the operator records the inspection result of the vehicle while the operator who inspects the object to be inspected (for example, the vehicle) has their hands occupied is taken as an example. Described as a use case. 11 and 12 are sequence diagrams showing the speech recognition procedure in the operation input system 5. FIG. Procedures that are the same as in Embodiment 1 are assigned the same procedure numbers, and descriptions thereof are omitted.

In FIG. 11, when the text data voice-recognized in step T5 is an input for the employee ID input item, the control unit 61 generates a check item screen GA12 for check 1 (T6A). The control unit 61 transmits the screen data of check 1 to the operation terminal 10 (T7A). The processor 11 of the operation terminal 10 displays the input screen GA12 of the inspection item of check 1 based on the received screen data (T8A).

If the text data voice-recognized in step T12 is an input for the employee ID input item, that is, if it is a six-digit number, the control unit 61 of the voice processing device 50 performs check 1 including the corrected employee ID. An inspection item input screen GA12 is generated (T12A). The control unit 61 transmits the screen data of the corrected inspection item input screen GA12 of the check 1 to the operation terminal 10 via the communication circuit 53 and the network NW (T13A). The processor 11 of the operation terminal 10 updates the input screen GA12 of the check item of check 1 based on the received screen data (T14A).

Further, when the text data voice-recognized in step T15 is an input for the check item of check 1, the control unit 61 of the voice processing device 50 displays the input screen GA13 of the check item of check 2 including the confirmation result of check 1. Generate (T15A). The control unit 61 transmits the screen data of the input screen GA13 of the inspection item of check 2 to the operation terminal 10 via the communication circuit 53 and the network NW (T16A). The processor 11 of the operation terminal 10 updates the input screen GA13 of the check item of check 2 based on the received screen data (T17A).

The same processing from procedure T9 to procedure T17A is repeated for several inspection item input screens. That is, if the inspection item input screen is represented by the m-th screen (m=2 to N), the same processing from procedure T9 to procedure T17A is performed for m=2 to N.

After that, if the text data voice-recognized in step T21 is an input for the inspection item of check (N-1), the control unit 61 of the voice processing device 50 includes the input of the corrected check (N-1). An input screen for inspection items for check N is generated (T21A). The control unit 61 transmits the screen data of the input screen of the check item of check N including the input of the corrected check (N-1) to the operation terminal 10 via the communication circuit 53 and the network NW (T22). Based on the received screen data, the processor 11 of the operation terminal 10 updates the input screen of the check item of check N in step T24 (T23).

Further, when the text data voice-recognized in step T21 is an input for the inspection item of check N, the control unit 61 of the voice processing device 50 displays the check result screen GA3 (FIG. 7) including the confirmation result of check N in step T25. reference) to generate the screen data. Since subsequent operations are the same as those in the first embodiment, description thereof is omitted.

Thus, in the operation input system 5 of Embodiment 2, the input operation can be completed without requiring the user's hand. Since the input screen of the inspection item displayed on the display device changes every time the inspection item changes, the user can perform visually easy-to-understand input operations. In addition, even if there are many inspection items, the user can immediately grasp the contents to be inspected next.

As described above, according to the operation input method of the second embodiment, the input screen GA12 (the m-th screen (integer satisfying 2≤m≤N, N: an integer equal to or greater than 4) for the inspection item of check 1 is displayed on the operation terminal 10, for example. ) is displayed, a step of determining whether or not the recognition result of the voice uttered by the user matches the input content for the inspection item of check 1 shown on the input screen GA12; If the input content for the inspection item of check 1 shown matches, the display is switched from the input screen GA12 (mth screen) to the input screen GA13 ((m+1)th screen) of the inspection item of check 2. and a step of instructing the operation terminal 10 to display, on the input screen GA13, the inspection item for Check 2 of the speech recognition result corresponding to the inspection item for Check 1 displayed on the input screen GA12.

This enables continuous input of inspection items, improving operability. In addition, the switching of the screen makes it easier for the user to notice that the input operation for the next inspection item has been performed.

Further, in the voice recognition step, if the voice recognition result does not match the input content for the inspection item of check 1 (confirmation item shown on the m-th screen) shown on the input screen GA12, the user speaks. It includes a step of receiving recognition processing of keywords (predetermined keywords) such as “previous item NG” and “check 3 NO” and the voice uttered by the user again for the inspection item of check 1. FIG. This makes it possible to easily correct the input contents of the previous item even after the input is completed.

Also, if the speech recognition result matches the input content for the inspection item shown on the inspection result screen GA3 (Nth screen), each of the input screens GA12 (second screen) to the Nth screen and a step of instructing the operation terminal 10 to display an inspection result screen GA3 (recognition result) in which the inspection items and the recognition results of the user's voice for each inspection item are associated with each other. Thereby, the user can visually confirm the input contents of all inspection items in a list. It becomes easy to find erroneous input, and it is possible to reduce input errors.

Also, the input contents for the inspection items shown on the input screen GA12 from the second input screen to the Nth input screen are associated with the input contents for the inspection items shown on the employee ID input screen GA1. Thereby, inspection items can be managed for each employee ID. In addition, the user can compare the input contents of the inspection items shown on the input screens GA12 and GA13 of each inspection item with the contents of the inspection items assumed by the user, and can easily confirm whether the input contents are correct or not.

In addition, keywords (predetermined keywords) such as "previous item NG" and "check 3 NO" uttered by the user are input to the inspection item displayed on the previous screen (m-1th screen). This is text data (information) representing the correction (modification) of the content. This makes it possible to easily correct the previous check item.

(Modification 1 of Embodiment 2)
In the second embodiment, the case where the voice processing device 50 receives voice data from the operation terminal 10 and performs voice recognition has been described. explain. The operation terminal 10 according to Modification 1 of Embodiment 2 has the same configuration as that of Embodiment 1. FIG. The same reference numerals are used for the same components as in the first embodiment, and the description thereof is omitted.

13 and 14 are flowcharts showing an example of a speech recognition operation procedure according to Modification 1 of Embodiment 2. FIG. The same step numbers are assigned to the step processes that are the same as those in Modification 1 of Embodiment 1. FIG.

In FIG. 13, when the operating terminal 10 is activated by a user's power-on operation or the like, the processor 11 of the operating terminal 10 starts speech recognition operation. The processor 11 displays an employee ID input screen GA1 on the display device 15 (S1). With the employee ID input screen GA1 displayed, the user speaks (for example, the number "123456"). The processor 11 picks up the user's speech with the voice input device 14 (S2).

The processor 11 stores audio data of the collected audio in the memory 12 . The speech recognition unit 25 of the processor 11 performs speech recognition on the speech data stored in the memory 12, and determines whether or not the speech has been recognized (S3). If the speech cannot be recognized, the processor 11 returns to step S2 and performs the sound pickup operation again. As a result, the user can speak the employee ID again, and can deal with the case where the sound cannot be picked up properly due to sudden noise or the like.

If the voice can be recognized in step S3, the processor 11 determines whether or not the voice-recognized text data is an input for the corresponding item (employee ID input item) (S4). If the input is for an employee ID input item, the processor 11 refers to the keyword table 12z stored in the memory 12, determines whether or not there is an employee ID corresponding to the text recognized by voice, and identifies the user's employee ID. Authenticate. Furthermore, when the authentication result of the employee ID is OK, the processor 11 picks up a plurality of inspection items corresponding to the employee ID registered in the keyword table 12z.

If it is determined in step S4 that the input item for the employee ID is not entered, the processor 11 returns to step S2 and performs the sound pickup operation again. Examples of cases where the input is not for the input item of the employee ID include that the authentication result of the employee ID is NG, or that the inspection item corresponding to the user's employee ID is not registered in the keyword table 12z. In these cases, the processor 11 may display on the display device 15 that the authentication result of the employee ID is NG. Moreover, the processor 11 may display on the display device 15 that the inspection item corresponding to the employee ID of the user is not registered.

The processor 11 generates an inspection item input screen GA12 for check 1 based on the authentication OK of the employee ID and a plurality of inspection items corresponding to the employee ID (S5A). The processor 11 displays the input screen GA12 of the inspection item of check 1 on the display device 15 (S6A). The input screen GA12 of the check item of check 1 displays the employee ID input on the input screen GA1 of the employee ID.

With the inspection item input screen GA2 displayed on the display device 15, the processor 11 picks up the user's speech with the voice input device 14 (S7), as in step S2. At this time, the user pronounces, for example, "OK" or "NG" for the inspection item of check 1 engine confirmation.

The processor 11 stores audio data of the collected audio in the memory 12 . The speech recognition unit 25 of the processor 11 performs speech recognition on the speech data stored in the memory 12, and determines whether or not the speech has been recognized (S8). If the speech cannot be recognized, the processor 11 returns to step S7 and performs the sound pickup operation again.

If the voice is recognized in step S8, the processor 11 determines whether or not the voice-recognized text data is an input for the corresponding item (input item for check 1) (S9). If the voice-recognized text data is not the input for the input item of check 1, the processor 11 determines whether or not the voice-recognized text data is the input for the previous input item (employee ID input item) ( S10). If the input item is not the employee ID, the processor 11 returns to the process of step S7. Note that if the input is not for the employee ID input item, the processor 11 may display nothing, or may display on the display device 15 to prompt re-input.

If the text data voice-recognized in step S10 is an input for an employee ID input item, that is, if it is a 6-digit number, the processor 11 determines that this input is a correction of the employee ID and corrects the input. The check item input screen GA12 of check 1 is updated so as to reflect the employee ID (S11A). After that, the processor 11 returns to the process of step S7.

If the text data voice-recognized in step S9 is an input for the check item of check 1, the processor 11 generates an input screen GA13 of the check item of check 2 (S9A). The processor 11 displays the input screen GA13 of the inspection item of check 2 on the display device 15 (S12A). The input screen GA13 of the check item of check 2 displays the confirmation result (for example, "OK") input on the input screen GA12 of the check item of check 1 (see FIG. 10).

Similar processing from steps S7 to S12A is repeated N times corresponding to the number of inspection items. That is, if the inspection item input screen is represented by the m-th screen (m=2 to N), the same processing of steps S7 to S12A is performed for m=2 to N.

After that, the processor 11 displays the input screen GA2 of the check item of check N on the display device 15 (S13A). The input screen GAN for the check item of check N displays the confirmation result (for example, "OK") input on the input screen of the check item for check N-1.

In this display state, the processor 11 picks up the user's speech with the voice input device 14 (S14), as in step S7. The processor 11 stores audio data of the collected audio in the memory 12 . The speech recognition unit 25 of the processor 11 performs speech recognition on the speech data stored in the memory 12, and determines whether or not the speech has been recognized (S15). If the speech cannot be recognized, the processor 11 returns to step S14 and acquires the user's speech again.

The processor 11 determines whether or not the text data that has undergone speech recognition is an input (for example, "OK") for the corresponding item (input item for check N) (S16). If the voice-recognized text data is not an input for the check N input item, for example, if the user utters "Previous item NG", the processor 11 determines that the voice-recognized text data is the previous input item (check N-1 input items) is determined (S17).

If the voice-recognized text data is the input for the input item for Check N-1, the processor 11 updates the check item input screen for Check N to reflect the corrected text data for Check N-1. (S18A). After that, the processor 11 returns to the process of step S14. It should be noted that if the text data voice-recognized in step S17 is not an input for the input item of check N-1, the processor 11 returns to the process of step S14. At this time, the processor 11 does not have to display anything, or may display on the display device 15 to prompt for re-input.

If the text data voice-recognized in step S16 is an input for the inspection item of check N, the processor 11 generates an inspection result screen GA3 (see FIG. 7) (S19). The processor 11 displays the inspection result screen GA3 on the display device 15 (S20). After this, the processor 11 terminates the voice recognition operation.

As described above, when the user speaks, the operation terminal 10 according to the first modification of the second embodiment performs speech recognition on its own and displays the speech recognition result. The operation terminal can be used in places where there is no network environment. Cost reduction can be achieved by not using a network environment and not requiring an audio processing device. In addition, every time the inspection item changes, the input screen of the inspection item displayed on the display device changes, so that the user can perform visually easy-to-understand input operations. In addition, even if there are many inspection items, the user can immediately grasp the contents to be inspected next.

(Modification 2 of Embodiments 1 and 2)
In the first and second embodiments, the speech processing device 50 performs speech recognition based on speech data transmitted from the operation terminal 10, generates screen data based on the result of this speech recognition, and transmits the screen data to the operation terminal 10. bottom. The operation terminal 10 received the screen data transmitted from the voice processing device 50 and displayed various screens on the display device 15 .

In Modified Example 2 of Embodiments 1 and 2, operation terminal 10 transmits voice data to voice processing device 50 and receives text data whose voice has been recognized from voice processing device 50 . Based on the received text data, the operation terminal 10 generates screen data for various screens (employee ID input screen GA1, inspection item input screen GA2, and inspection result screen GA3). to display.

As a result, the operation terminal can omit the process of performing voice recognition, and the voice processing device can omit the process of generating screen data. In addition, screen data with a large amount of data does not need to be communicated via the network, and the traffic of network communication can be reduced by reducing the amount of communication.

Various embodiments have been described above with reference to the drawings, but it goes without saying that the present disclosure is not limited to such examples. It is obvious that a person skilled in the art can conceive of various modifications, modifications, substitutions, additions, deletions, and equivalents within the scope of the claims. Naturally, it is understood that it belongs to the technical scope of the present disclosure. In addition, the constituent elements of the various embodiments described above may be combined arbitrarily without departing from the gist of the invention.

For example, in Embodiment 2 described above, one inspection item and one screen corresponded one-to-one. That is, one screen displayed the contents and input of one inspection item. Two or more inspection items may be associated with one screen, and the contents and inputs of two or more inspection items may be displayed on one screen. As a result, it is possible to reduce the processing of generating screens by the voice processing device and the operation terminal.

In each of the above-described embodiments, when inspecting a vehicle, the display of inspection items by the worker was exemplified. The present disclosure is similarly applicable to display of processes in which workers work at high places such as. In addition, if a smart speaker or a device linked to it that operates when a user speaks has a display function, the present disclosure can be similarly applied when displaying voice recognition results spoken by the user in the order of operation. is.

INDUSTRIAL APPLICABILITY The present disclosure is useful as an operation input method, an operation input system, and an operation terminal that improve convenience in inputting user operations even in situations where it is difficult for the user to perform operations using hands.

5 operation input system 10 operation terminal 11 processors 12, 52 memories 13, 53 communication circuit 14 voice input device 15 display device 25 voice recognition unit 50 voice processing device 51 processor 54 storage 61 control unit 62 voice recognition unit 63 keyword matching unit 541 keyword database

Claims (5)

a step of collecting a voice uttered by a user while the first screen is displayed on the operation terminal;
recognizing the collected speech;
a step of determining whether or not the speech recognition result matches the input content for the confirmation items shown on the first screen;
a step of instructing the operating terminal to switch display from the first screen to a second screen when the voice recognition result matches the input content for the confirmation item displayed on the first screen;
A recognition result of the voice uttered by the user is displayed on the m-th screen while the m-th screen (an integer satisfying 2≤m≤N, where N is an integer equal to or greater than 4) is displayed on the operation terminal. a step of determining whether or not it matches the input content for the confirmation item;
Instructing the operation terminal to switch the display from the m-th screen to the (m+1)-th screen when the voice recognition result matches the input content for the confirmation item displayed on the m-th screen. and
Manipulation input method. displaying the speech recognition result corresponding to the confirmation item shown on the first screen;
The operation input method according to claim 1. displaying the speech recognition result corresponding to the confirmation item shown on the m-th screen;
The operation input method according to claim 1 or 2. An operation input system in which an operation terminal having a voice input device and a display device and a voice processing device are communicably connected,
The operation terminal collects a voice uttered by the user with the voice input device while the first screen is displayed on the display device,
The operation terminal or the voice processing device recognizes the collected voice,
The speech processing device determines whether or not the speech recognition result matches the input content for the confirmation item shown on the first screen, and confirms the speech recognition result shown on the first screen. instructing the operating terminal to switch the display from the first screen to the second screen if the input content for the item matches;
The voice processing device is configured to recognize the recognition result of the voice uttered by the user while the m-th screen (integer satisfying 2≤m≤N; N: an integer equal to or greater than 4) is displayed on the operation terminal. It is determined whether or not the input content for the confirmation item shown on the m-th screen matches the input content for the confirmation item shown on the m-th screen, and if the speech recognition result matches the input content for the confirmation item shown on the m-th screen, instructing the operating terminal to switch the display from the screen to the (m+1)th screen,
operation input system. a voice input device that picks up a voice uttered by a user while the first screen is displayed on the display device;
a recognition unit that recognizes the collected voice;
a control unit that determines whether or not the speech recognition result matches the input content for the confirmation items shown on the first screen,
The control unit instructs the display device to switch the display from the first screen to the second screen when the recognition result of the voice matches the input contents for the confirmation items displayed on the first screen. death,
The control unit controls the m-th screen (an integer satisfying 2≦m≦N, where N is an integer equal to or greater than 4) to be displayed on the display device, and the recognition result of the voice uttered by the user is recognized as the m-th screen. If the speech recognition result matches the input content for the confirmation item shown on the m-th screen, the m-th screen instructing the display device to switch the display from the screen to the (m+1)th screen;
operating terminal.

Images