JP2021168020A - Voice input device - Google Patents

Abstract

To provide a device having a voice input function capable of being realized by various devices in particular, in voice input, and improving speed and accuracy of the voice input.SOLUTION: A voice input device which recognizes voice and converts it into character information, comprises: a voice recognition unit which recognizes inputted voice; a character conversion unit which converts the voice recognized by the voice recognition unit into a character string; a character string display unit which displays the character string converted by the character conversion unit; an auxiliary input display unit which displays a graphic character and/or a control character consisting of at least one of a period, a comma, a parenthesis and a function, on the character string display unit; and a sign insertion unit which inserts into the character string converted by the character conversion unit, the character selected from the graphic character and/or the control character displayed on the auxiliary input display unit.SELECTED DRAWING: Figure 2

Description

The present invention relates to a voice input device, and more particularly to a voice input device that can be implemented by various devices such as a computer, a smartphone, and a wristwatch-sized mobile terminal.

In information device devices such as computers, smartphones and wristwatch-sized mobile terminals, voice input is adopted as a method for inputting character information.
In such voice input, a voice recognition function is used in which a computer recognizes a human voice or the like and converts it into a character string.
Especially in recent years, the accuracy of this voice recognition function has been improved, and the probability of erroneous recognition has been significantly reduced.

However, if you want to document the spoken language as it is, you have to insert, for example, punctuation marks at the phrase breaks.
Therefore, it is conceivable to display the result of voice recognition on a display etc. for the time being, and while watching this, insert punctuation marks appropriately at the punctuation marks by operating the keyboard to create a document, but this is a lot before documenting. It takes a lot of time and effort and is not preferable for practical use.
Therefore, in the past, the following techniques have been proposed for inputting the punctuation marks and commas.

For example, in Patent Document 1 (Japanese Patent Publication No. 6-93221), when inputting a sentence by voice, the method of inputting by uttering "ten" for a comma and "maru" for a punctuation mark is a method of inputting a word. It obstructs the natural flow, and when creating sentences by dictation, it may also obstruct the flow of thought, so it automatically uses information such as utterance, tone, syntax and meaning before and after utterance. We are proposing a method for entering punctuation marks.
Specifically, the voice input device proposed in this document includes a means for recognizing an input voice, a means for dividing a character string represented by the recognized voice into a plurality of morphemes, and attributes of the plurality of morphemes. It is configured to include a means for detecting a morpheme located at the end of a sentence (a morpheme at the end of a sentence) and a means for inserting a kuten after the detected morpheme at the end of the sentence.

Further, in Patent Document 2 (Japanese Unexamined Patent Publication No. 2000-47688), a voice input method capable of automatically inserting punctuation marks by dictation of a natural flow and a recording medium for recording an implementation program of the voice input method are used. Whether or not there is a blank time for voice input exceeding a predetermined time after the voice input corresponding to the planned punctuation mark addition position while identifying the character string of the punctuation mark addition candidate from the character string recognized based on the input voice. We are proposing a voice input method that identifies and automatically adds one of the punctuation marks.

Further, in Patent Document 3 (Japanese Unexamined Patent Publication No. 2013-16138), as a display input method for a mobile phone, the display unit and the input unit are enlarged to improve the operability of information input and facilitate the information input. However, it has a display unit and a housing provided with a display-integrated touch panel device including a touch panel arranged on the display surface of the display unit and outputting the contact position when there is contact by an operator. In the kana input mode, the display-integrated touch panel device is equipped with a size suitable for the size of the operator's finger. ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, , Yu, yo, ra, ri, ru, re, ro, wa, o, n ", each hiragana character key, turbid point key, semi-turbid point key, reading point key, and punctuation key are all displayed. A display input method for a mobile phone in which characters are input using a key has been proposed.

Special Fair 6-93221 Japanese Unexamined Patent Publication No. 2000-47688 Japanese Unexamined Patent Publication No. 2013-16138

As mentioned above, various studies have been made on the input of punctuation marks and commas at the time of voice input.
However, the voice input device proposed in Patent Document 1 divides the character string represented by the recognized voice into a plurality of morphemes, and uses the attributes of the voice input device to be located at the end of the sentence (sentence end morpheme). Is detected, and then the punctuation marks are inserted. Therefore, it takes time to insert the punctuation marks, and it is difficult to input the voice quickly.

Further, the voice input method proposed in Patent Document 2 identifies whether or not there is a blank time of voice input exceeding a predetermined time after the voice input corresponding to the position where the punctuation mark is to be added, and is one of the punctuation marks. Was automatically added, so a blank time of a predetermined time was required, and it was also difficult to input voice quickly.

Further, in the display input method of the mobile phone proposed in Patent Document 3, in the kana input mode, each hiragana character key has a size suitable for the size of the operator's finger on the display-integrated touch panel device. , The voiced sound mark key, the semi-voiced sound mark key, the reading point key, and the punctuation mark key are all displayed, and it is disclosed that characters are input using the keys, but voice input is not performed.

Therefore, it is an object of the present invention to provide a device which can be implemented in various devices in particular for voice input and has a voice input function in which the speed and accuracy of voice input are improved.

The present invention relates to a known technique of "texting voice input", that "on / off of voice input" can be intermittently performed according to a necessary situation such as a long sentence or only one phrase, a misrecognized object or a synonym for the same sound. It is possible to correct and reconvert when an erroneous conversion occurs, and by touching the monitor screen while inputting only the main text, punctuation, parentheses, line breaks, etc. can be touched on the screen in the area where the screen is divided. It is possible to input without loss of time by using it together with voice input, and it is also possible to delete or move a part of the text, which enables voice input regardless of the presence or absence of a keyboard. In, the purpose is to realize accurate and maximum input speed.

In the present invention, in order to solve the above problems, a voice input device that recognizes a voice and converts it into character information, a voice recognition unit that recognizes the input voice, and a voice recognized by the voice recognition unit. A character conversion unit that converts to a character string, a character string display unit that displays the character string converted by the character conversion unit, and a graphic character consisting of at least any of punctuation marks, reading points, parentheses, and functions on the character string display unit. An auxiliary input display unit that displays and / or control characters, and a symbol insertion unit that is inserted into a character string converted by the character conversion unit by selecting a graphic character and / or control character displayed on the auxiliary input display unit. Provide a voice input device consisting of.

The voice input device of the present invention further includes a range designation unit that specifies the range of the character string displayed on the character string display unit, and a reconversion unit that reconverts the character string in the range specified by the range designation unit. It can also be configured in preparation.

The voice input device according to the present invention is a personal computer (hereinafter referred to as "computer") in which an input device such as a keyboard or mouse is generally used, or a tablet computer that does not include an input device such as a keyboard or mouse. Embodied in portable devices such as smartphones or portable translators (hereinafter referred to as "portable terminals"), or watch-sized portable terminals (hereinafter referred to as "smart watches") that have been made even smaller with the highest priority given to portability. can do.
However, some of these devices may have different operation methods depending on the accompanying devices such as input devices and the size of the screen.

In addition, since some tablets and smartphones are not given the properties and functions of the keyboard and mouse depending on the size and price range, in the description in this specification, the presence or absence of the keyboard and mouse functions is used. It is determined which operation method is closer to the embodiment.
Further, in the voice input device of the present invention, the auxiliary input display unit includes a pre-recognition screen including characters and symbols input at the beginning of a sentence and a recognizing screen excluding the characters and symbols input at the beginning of a sentence. Can be selected and displayed, and at the same time as voice recognition, a screen changing means for changing the pre-recognition screen to the recognizing screen can be provided. Further, it is provided with an editing / correcting unit for editing or correcting the input character string, and the editing / correcting unit can insert another character by voice input into a specified range of the input character. Further, it can be provided with an audio output unit that outputs a character string having a specified range as audio, and an enlarged display unit that enlarges and displays an arbitrary area displayed on the display device.

According to the voice input device of the present invention, it is possible to perform voice input with various devices in particular, and to have a voice input function with improved speed and accuracy of voice input.

Computer operation screen (pre-recognition screen) Computer operation screen (recognizing screen) Transition icon to voice input setting screen Computer operation screen (symbol display setting screen) Schematic diagram of multifunction mouse Computer operation screen (multifunctional mouse setting screen) Computer operation screen (edit / modify screen) Voice input flowchart Schematic diagram when inputting spreadsheet software Schematic diagram when entering text with a multifunction mouse Schematic diagram when using document input software Computer operation screen (computer program input setting screen) Schematic diagram of the stepping touchpad Appearance explanation and operation screen of mobile translator (initial screen) Operation screen of mobile translator (screen before recognition) Operation screen of mobile translator (symbol display setting screen 1) Operation screen of mobile translator (symbol display setting screen 2) Operation screen of mobile translator (symbol display setting screen 3) Mobile translator operation screen (recognizing screen) Mobile translator operation screen (edit / correction screen) Operation screen of mobile translator (rephrasing / reconverting procedure) Operation screen of mobile translator (insertion procedure) Operation screen of mobile translator (move / delete procedure) Operation screen of mobile translator (procedure for transition to pre-recognition punctuation marks) Explanation of punctuation marks in English and Chinese (pre-recognition screen, recognition screen, edit / correction screen) Operation screen of mobile translator (punctuation touch operation labor saving method) Input example to the corporate Web "inquiry page" Operation screen of mobile translator (special typeface input 1) Operation screen of mobile translator (special typeface input 2) Operation screen of mobile translator (special typeface input 3) Smartwatch operation screen (pre-recognition screen, recognition-progress screen, edit / modify screen) Smartwatch operation screen (symbol display setting screen) Smartwatch operation screen (SNS and phone number input) Smartwatch operation screen (Internet search) Smartwatch operation screen (confirmation of contents on edit / correction screen) Schematic diagram of reaching the target page on the Web with a smart watch Smartwatch operation screen (loupe mode) Smartwatch operation screen (OCR reading function and position setting)

Hereinafter, the voice input device according to the present invention will be specifically described.
The numbers in parentheses [] in the present specification indicate paragraph numbers in the present specification.
In particular, in the present embodiment, a voice input device embodied in a computer, a mobile terminal, and a smart watch will be described.
However, some of these devices may have different operation methods depending on the accompanying devices such as input devices and the size of the screen.
Therefore, in the following, the same functions will be explained together, and the different parts of each device will be explained separately with the proviso that "for computers", "for mobile terminals", and "for smart watches".

In the present specification, the "computer" is a device that uses a keyboard or a mouse as an external input device, and includes a tablet terminal with a mouse function.
In computers, the main purpose is to minimize the main text input time. The main text can be a device that simultaneously performs three processes for inputting voice, and it is also desirable to have a function of intermittently turning on / off the voice input from the viewpoint of maximizing the speed of the voice input processing.

Further, the "mobile terminal" is a smartphone-sized device, and in the present embodiment, a mobile translator will be described as an example.
The main purpose of the mobile terminal is to be able to input accurately without an external input device such as a keyboard, which enables highly accurate search on the Internet or the like.
In addition, since the translator can transmit the correct native language to the translation AI (that is, the same applies to "artificial intelligence" and the like), the resulting translation accuracy is improved.

A "smart watch" is a device whose main purpose is portability and simple input, and corresponds to a device that emphasizes response due to its excellent portability.
Similar to mobile terminals, the main purpose is to be able to input accurately without an external input device such as a keyboard, as a simple SNS transmitter (that is, the same applies to "Social Networking Service" and below), and to search accurately on the Internet. In the present embodiment, the function of transmitting the input information to the operator by voice without depending on the character display on the small screen is further provided.

First, in the present embodiment, the voice input itself activates the voice input function in the computer, mobile terminal, and smart watch by pressing the "voice input recognition button" (hereinafter referred to as "voice input button"). It will be in the voice reception state.
Then, the input person starts voice input by speaking.

In such a start process, it is an inexpensive practical method to set the computer so that the voice input is converted into text only while the button is pressed on the main body or the keyboard.
However, in order to achieve the purpose of shortening the operation time, it is desirable that the input device such as a mouse of the computer is provided with both a button that activates voice recognition only while the mouse is pressed and a sound receiving microphone.
This also has the advantage that the user can speak with the mouse close to his or her mouth by reducing the time loss of changing hands and considering the surrounding environment (noise, etiquette to others, etc.).

For mobile terminals and smart watches, there are no external input devices such as keyboards and mice, only the main body itself.
Therefore, the operation button itself is naturally provided on the main body, and depending on the usage environment, the main body itself may be used close to the mouth.
When the voice input (basically one phrase) is completed, the act of "releasing" the voice input button is to fix the text (in the case of a computer, it is synonymous with the enter key or "execute"), so that the input operation is quick. Can be realized.
Next, in each device, an embodiment when the voice input device is realized will be specifically described.

"First Embodiment: Computer"
The first embodiment is a voice input device formed by using a computer.
In particular, when configuring a voice input device using a computer, the voice input process is not started by separately starting specific software, but an operating system, a device driver, a BIOS (Basic Input / Output System), or the like. It is desirable to make it a function of the basic program of the computer so that it does not block or interfere with the software that was originally started.
In particular, the program for realizing the voice input device according to the present embodiment (that is, the voice input program) is incorporated into hardware as an operating system or the like, which meets the purpose of further speeding up.

The voice input device according to the present embodiment displays the pre-recognition screen 10 at the time of voice input shown in FIG. 1 on the computer display by pressing the voice input button.
FIG. 1 will be described as an example in which voice input is performed using the present invention when inputting an email body on a computer.
The arrangement of touch punctuation marks and the like can be customized by the operator as described in the column [0046] described later, so the items in the figure are merely examples.
The voice input button can be provided on a device (mouse or the like) connected to the computer or the computer itself, or can be displayed on the screen.
Further, the pre-recognition screen 10 is displayed on the frontmost side of the display, and further, the mail software screen to be voice input is displayed so as to be transparently displayed.
This "transparent display" makes it possible to perform voice input work while viewing the software to be input in the full screen display state, and does not impose any restrictions on the display size of the input object.
As the situation of FIG. 1, in the present embodiment, the pre-recognition screen 10 before the voice input of the text “The present invention relates to the voice input device” has already been completed and then the next sentence is voice-input is shown. ing.
That is, the pre-recognition screen 10 represents a standby screen in a state before recognizing even one voice for the next phrase.

In the present embodiment, the pre-recognition screen 10 has "number priority", "alphabet priority", "katakana priority", "no conversion", "(", """," @ "," calculation formula priority "," symbol and setting screen ". (These two items blink alternately in the same touch area.) Display the items "Caps Lock", "*", "Voice input Lock", "Word", "Quick color change", "Bold", and "Back one". A dotted line partition is provided between each item.
However, the partition is not essential, and it is sufficient if at least the designated area corresponding to the item can be specified.

In addition, when inputting a long sentence by voice input, there is a high possibility that the writing style will be adjusted as a result and the erroneous recognition will be corrected.
Therefore, on the pre-recognition screen 10, "voice input" as a screen touch area (designated area) for executing a voice input lock state (a state in which the device recognizes that the voice input button is held down) is displayed. The "Lock" region 11 is present.
This is to make the computer recognize the "hold and hold" of the voice input button, and to reduce the burden on the operator.
Voice input Lock After touching, even if the finger is released from the voice input button such as a mouse, the device side continues to recognize it as "the one that is kept pressed".
When the voice input touch part is pressed again, "touch operation recognition" is prioritized, and the moment the finger is released from the screen, the voice input button is also recognized synonymously with the released object.
Releasing the voice input button or touching the screen of the voice input Lock area once and then canceling it is the same as the enter key after inputting text with the keyboard, and the input text is confirmed.

However, the screen touch for releasing the voice input lock is not recognized as a command at the moment of touching, but at the moment when the finger is released from the screen after touching.
Even if the screen area of the voice input lock release is touched and the voice input button on the mouse side is pressed again and then the lock is released, the voice input ends, that is, "confirm", which is synonymous with the enter key, is entered. This is because it is possible to select that the hold state can be restored to the operation on the mouse side by performing the lock release operation after pressing the voice input button of the mouse again.
In addition, when inspecting and proofreading after inputting a long sentence over multiple lines by voice, when the inspection work is nearing the end, the operator side displays the lock on the screen in advance as a preliminary operation for unlocking the text confirmation operation. Since you can create a situation that you can touch, you can shorten the time when shifting to the confirmation operation, although it is a little.

Further, the pre-recognition screen 10 is configured so that touch items having a property that is grammatically not attached to the beginning of a sentence do not appear.
Punctuation marks such as "." And "," are not attached to the beginning of the sentence, and the "closed" side of parentheses and hook brackets is not attached to the beginning of the sentence.
Allocate the blank area to "number priority", "alphabet priority", "katakana priority", etc.
The definition of priority recognition is that when the device side AI that senses the voice input after the specified touch recognition converts the recognized voice into text, conversion candidates are converted into a list of numbers, alphabets, all katakana proper nouns, etc. A program that specifies the direction when making a decision is executed.

For example, when performing voice input such as "Submit vertically on A5 paper", there is a very high possibility that "A5" at the beginning of a sentence will be erroneously recognized as "English" in various current voice recognition devices. high.
Therefore, the operator touches the designated area 12 of "alphabet priority" on the screen, then pronounces "er", confirms "A" (* a conversion key if a is output), and then releases the voice input button. After fixing the text of A, press the voice input button again to touch (select) the designated area 13 of "number priority" on the screen, and then pronounce "go" to obtain the text of "5".
After that, the voice input can be continued as it is.
However, the priority recognition is a recognition limited to one phrase in the voice input after the touch operation.
This is because if the priority recognition continues to be applied, the operator will unintended voice input (misrecognition on the device side), which hinders natural voice input.
These are rarely used in computers that have a keyboard as an external input device, but they are indispensable selection methods for devices that are smaller than the size of a mobile terminal that do not have a keyboard as an external input device.

In addition, after touching (selecting) the designated area 14 of "calculation formula priority" on the screen, the operator simply pronounces "Sabutoru", and the calculation formula "Sabutoru" in spreadsheet software represented by Excel (registered trademark) is used. = Set the processing / function for the item so that the recognition of "SUBTOTAL" is automatically selected.
This is because the calculation formula input of spreadsheet software can be labor-saving.

Then, when the designated area 15 of "return by one" on the pre-recognition screen 10 is touched (selected) on the screen, the selected content of the punctuation mark is mistaken or the priority recognition is mistakenly pressed, and it is desired to cancel it. use.

By selecting the wallpaper color of your own computer, or when entering on the customer inquiry page of a company on the Internet, the background color and the color such as translucent punctuation accidentally become similar, making it difficult to see the translucent punctuation mark etc. There will also be situations where it becomes.
In this way, there may be situations where you want to change the color tone, so it is desirable to configure it so that you can also select the "quick color change" touch area 16 so that only the color tone can be easily changed.
In the present embodiment, as the quick change, the color tone of the touch portion such as the translucent punctuation mark can be changed to "pink", "light blue", "green", "orange" and the like.
If you touch the color directly, it will jump to that color, and if you touch "Area", it will return to "pink"->"lightblue"->"green"->"orange"->"pink".

Further, the “bold” touch area 17 is a touch that changes the typeface of the text generated by the voice input. For example, the screen is touched before starting the voice input of the phrase to be made bold, and the “voice without releasing” Continue typing and release the touch where you want to end bold. "This is how you specify the typeface.
In the example, it is arranged on the pre-recognition screen 10, but of course there may be cases where it is desired to obtain bold characters in the middle of voice input. It can also be placed in one of.
Similarly, touch areas such as italic characters and underline characters can be arranged on the setting screen instead of bold characters.
The same operation as clicking the icon for specifying the typeface when entering text with a normal keyboard, and the command to end the typeface change only needs to be "released", so "same again" with keyboard input It takes less time than "Cancel by clicking the icon".
If you want to specify the typeface for a long range, you can perform the pre-operation by clicking the icon with the mouse as usual.

As mentioned above, in this voice input device, texts (input characters) recognized by the device are piled up while talking by pressing the voice input button, and when the voice input button is released, input is performed in the same way as the enter key on the keyboard. The text can be fixed or fixed.
For the input by the voice input button, the process of starting the voice input process and executing the process of confirming the input character can be repeated.
By performing the minimum necessary operations in this way, it is possible to intermittently function the voice input only for the part that you want to convert into text in the software. You can increase the input speed by pressing the input button to input voice and using the ten keys as before when inputting numerical data.
Again, in order to achieve this, it is desirable to implement the voice input processing program on the computer as basic software such as an operating system.

Then, when voice recognition is started even for one voice, the explanation of the semi-transparent punctuation touch area (that is, the auxiliary input display unit) changes to the recognition screen 20 shown in FIG.
This is because a screen touch area of punctuation marks (such as "," and "." In Japanese) is required for the convenience of the grammar of each language. By surrendering the area of "number priority" 13, "alphabet priority" 12, "katakana priority") to punctuation marks "." 21, reading point "," 22, etc., the touch area can be changed and the same screen size. It effectively uses the number of divisions in.
That is, the voice input device according to the present embodiment includes a screen changing means for changing the pre-recognition screen 10 to the recognizing screen 20 at the same time as recognizing the voice, whereby the touch areas before the voice recognition and during the voice input can be changed. , Can be used efficiently within the same screen.

Also in the recognition screen shown in FIG. 2, the selection area of items consisting of symbols and punctuation marks that can be selected as an aid to voice input is superimposed on the display area of the input target, and the display screen of the input target on the background side is transparent. It is configured as a screen displayed as (semi-transparent).
Examples of the layout of the pre-recognition screen are "."",""Symbol""Conversion""(""""" Blank "" Line feed "") """""BS""Voice input Lock""Word" The items of "punctuation mark touch temporary deletion", "/ (slash)", and "back one" are displayed.

Visibility may be hindered by chance at the part where the color tone of the touch area of the translucent punctuation mark overlaps with a part of the screen to be input.
When you want to see the overlapping part in the middle of voice input, you can continue voice input with the semi-transparent punctuation marks disappearing from the screen while touching the screen touch area of "Punctuation touch temporary deletion".
The existence of this touch area makes it possible to erase the translucent punctuation marks from the screen without temporarily releasing the voice input button, that is, interrupting the voice input itself and confirming the text generated up to that point.
In most cases, the main usage is to release the touch immediately after confirming the overlapping part to be visually recognized and continue the voice input itself.

In addition, you can directly enter punctuation marks and parentheses between phrases with the keyboard without releasing the voice input button (voice input button is selected. The same applies below). By configuring it so that it does not depend on, the input speed and accuracy can be further improved.
The situation is that there is only one word, no punctuation, and frequent line breaks (for example, when you remember only the main points of the other party's talk and input voice instead of paper notes). This is because, depending on the form and content, it is possible to input the enter key repeatedly faster than pressing the screen touch or the voice input button each time.

In addition, the backspace key on the keyboard erases the mistaken part of the voice input without releasing the voice input button, so that the voice input can be continued.
Furthermore, once the voice input button is released and the text in the voice input is fixed, it is the same as the text input in the keyboard, and this can be corrected on the mouse / keyboard side.
Alternatively, it is also possible to add a voice input sentence to the original text by locating the end of the text input from the keyboard with the mouse and pressing the voice input button to utter.
If a phrase that is difficult to recognize by voice input appears, release the voice input button once (end the voice input), fix the text halfway, and enter only the part that is difficult to recognize from the keyboard. After that, if the voice input is restarted, the most efficient input processing can be performed.

When you want to change the arrangement and contents of touch items, there is a margin in the screen layout in the case of a computer, so after the voice input according to this embodiment becomes widespread, general spreadsheet software as shown in FIG. It is also desirable to place an icon that calls the system setting screen for voice input in the layout of the text input software.
An image diagram of the setting screen or the like after clicking the icon will be described later in the [0045] column.
Further, a description in which two touch areas of a word and a screen setting are mixed in the same place will be described later in the [882] column relating to a device having a size smaller than that of a portable translator that makes heavy use of this function.

In the computer, the default (initial setting) is 16 divisions to divide the touch area of the monitor screen, but the number of divisions itself can be reduced to at least 4 areas.
By matching the type and placement position of the touch area with a device with a small screen such as a smart watch described later, confusion is reduced when the same operator uses both devices, and the placement is memorized. Therefore, the transparency of punctuation touch is lowered in the setting screen explained in the previous and next sections, and the advantage of being able to operate with "blind touch" regardless of visual recognition is also taken into consideration.

After selecting the "setting screen" icon described in the [0043] column, after the transition from the setting screen shown in FIG. 4 (A) to (F), the software or Internet company that was temporarily displayed on the screen The screen to be input such as the inquiry page becomes a background and temporary input is interrupted (the software itself is not terminated or the Internet is not shut down).
FIG. 4A shows a state in which the screen is shifted to the setting screen while the pre-recognition screen is displayed, and the semi-transparent punctuation touch (that is, the auxiliary input unit) is referred to as “full screen size display” when voice input is used. By comparison, the display is reduced to about 80% toward the lower left, indicating a state in which blank areas are provided at the upper and right edges of the screen.
FIG. 4B shows a state in which a “touch area dedicated to various settings” appears in the blank area secured in FIG. 4A.
Although FIGS. 4 (A) and 4 (B) are shown as separate figures for explanation, screen transitions and developments are actually performed at the same time.
In the setting screen, in addition to the pre-recognition screen, there is always a selection point in the upper row for which punctuation touch screen to customize, such as the recognition in progress screen and the edit / correction screen, which will be explained in the future, and the target screen is immediately available. You can jump to the settings.
As the contents that can be changed, color tone selection, color tone shade, font size, number of divisions, etc. can be selected.
If the color tone and shading are changed on the pre-recognition screen, the recognition-in-recognition screen and the edit / correct screen are similarly linked and changed, and the color tone of the three punctuation touches does not differ.
Regarding word registration, the computer basically links to the dictionary tool of the IME option (registered trademark) and the registered word, and the computer does not register the word on this screen.

The arrangement and type of touch punctuation marks can also be changed (that is, customized) on the "setting screen" explained in the [0043] column.
^{This is because there are cases where specific symbols (eg, "m 2} ", "Σ", etc.) are frequently used depending on the needs of the user (professional business, etc.).
When using such symbols, even if the parentheses "("")" are used, the key brackets """""" used in the verbal tone are considered to be rarely used. ^{, You can select to assign symbols (eg "m 2} ", "Σ", etc.) to the parts of the key brackets """""".
If you want to change the touch contents, a predetermined time length press (approximately 2 seconds, the same below) one part of the reduced displayed existing touch area Then, since the color tone is inverted and then enter keyboard, for example, the "m ^2" , "Do you replace it with m ^2? Y / N" and the confirmation message has been confirmed from the computer side, is selected and determined by the "Y".
To change the position of the touch punctuation mark, press and hold the touch of the punctuation mark to invert the color tone, then pull it with your finger on the monitor and release it in the same way as the drag and drop operation with the mouse to change the touch position. can.
You can also delete the punctuation touch item by swiftly and greatly right-flicking off the screen.
The format that can be set in the touch area is not particularly limited, and is not limited to "one-character symbols such as \ and *".
When registering, you can directly enter "Voice Taro" on the keyboard and register it as one touch item if the number of characters is about the name.
However, the "back one" touch area on all touch screens can be repositioned but not erased.
Even if the position of the "Convert" touch area on the edit / modify screen described later can be changed, it cannot be erased.
Such customizations are programmed so that they are not accepted as "logical errors", as this is an operational contradiction described below.
FIG. 4C shows the “pre-decision state” in which “” (starting point of hook brackets) is ^{being changed to m 2 (square meter).}
FIG. 4 (D) shows the "pre-decision state" while "*" (rice mark) is being changed to "voice Taro".
FIG. 4 (E) shows the "calculation formula priority" area "a state in which the monitor screen is touched with a finger and the drag operation is being performed", and the positional relationship is switched with "bold" at the same time as the drop operation.
It also shows a state in which only "symbol items" are excluded from the area where "symbols and touch points for transition to the setting screen are mixed".
FIG. 4 (F) shows an operation image screen of the “long press time” newly appeared on the outer circumference in FIG. 4 (B).

In addition, although it is a rare case, it seems that there are cases where the dependence on the keyboard is reduced and the input frequency by touching the screen is increased. In that case, the arrow symbols "←" and "↑" are used instead of the cursor keys. The selection display area of "→", "↓", and "BS (backspace)" can also be selected on the setting screen.

Further, for a computer, since an expensive display (a monitor device with a screen touch recognition function) is required when selecting punctuation recognition by screen touch, the screen touch is not indispensable.
As a known technique, a multifunction mouse having many buttons is also provided, and this can be used as a substitute for a part of the screen touch area.
For example, on the setting screen of the multifunction mouse or the setting screen, as a specification applied only at the time of voice input, a function assignment that assigns each button of the multifunction mouse 50 to a specific item function (for example, punctuation marks, hook brackets, etc.). A screen (described later in FIGS. 6B to 6D) can also be provided.
For example, as shown in FIG. 5, the button 51 of the multifunction mouse 50 is set with a punctuation mark ".", The button 52 is set with a "line feed", the button 53 is set with a comma ",", and the button 54 is set with a "voice input button". Further, the mouse 50 is provided with a voice microphone 55.
An example of the operation will be described later in the [0059] column after the basic items of voice input on the computer are completed.

Further, by selecting "Do not display punctuation touch" in the setting screen for punctuation marks and the like shown in the above [0045] column, a semi-transparent punctuation mark area does not appear on the screen, and an input method that does not depend on screen touch punctuation marks. In other words, you can also use the punctuation input method that limits the types by just operating the right hand of the mouse button.
As a specific setting method, the touch area of "density" on the setting screen shown in FIG. 4 is selected, and the selection area 61 of the "density setting portion of translucent punctuation touch" shown in FIG. 6 (A) is touched. When the "Hide" density is selected in the selection bar, it starts with the appearance of an additional option of the check box 62 for specifying "Do not use monitor punctuation touch".
If the display density of punctuation touch is simply low, it is in the state of "blind touch" described in column [0044], and if a predetermined place on the monitor screen is touched, punctuation marks, parentheses, etc. are converted into text.
However, if the "Do not use screen touch" check box 62 is clicked, all touch inputs on the monitor screen will be judged to be invalid by the computer and will not be accepted.
However, after clicking the check box 62, a check box 63 for selecting a further additional option "multifunctional mouse setting" appears in the setting screen.
Further, when the check box 63 for selecting the "multifunctional mouse setting" is clicked, the screen transitions to the "which punctuation mark, parenthesis, etc." setting screen is assigned to each button of the multifunctional mouse shown in FIG. 6 (B).

Hereinafter, FIG. 6B is an explanation of the function assignment operation screen of the multifunctional mouse button, and the arrangement of each button matches the contents described in the [0048] column, and is a diagram described in the [0059] column described later. The example of 9 is described on the premise of the simplest usage with this arrangement.
As a setting operation, when the area of the left horizontal upper button 51 in FIG. 6 (B) is long-pressed (or right-clicked with the mouse) on the screen for a certain period of time, the color tone of the area and the button 51 in the mouse image diagram are reversed at the same time. At that point, after pressing the "." On the keyboard, the "Y / N" selection from the device side is shown and the decision is made with Y.
Similarly, after completing the desired setting work such as the left horizontal lower button 52 and the right horizontal upper button 53 in the screen shown in FIG. 6 (B), click "OK" at the lower right of the setting screen to overwrite the setting. It will be completed.
However, as the upper row of the button allocation setting screen (Fig. 6 (B)) has the notation "pre-recognition screen", "recognition-in-progress screen", and "edit / correction screen", each mouse button is on each screen (that is, "voice"). Different roles can be set for each situation during input.
In the process of the operator becoming familiar with the usage, the "pre-recognition screen" is entered with "priority recognition touches for letters and numbers", and the "recognizing screen" is entered in the middle of the text according to the grammar. By allocating it to hook brackets, etc., it is possible to efficiently use the limited number of multifunctional mouse buttons.
Moreover, the number of buttons of the existing multifunction mouse is not constant.
Therefore, it is necessary to change the number of divisions of the setting screen itself according to the number of mouse buttons.
In FIG. 6 (C) (the number of buttons in place of the punctuation touch area is 3, the number of divisions in the setting screen is 9) and FIG. 6 (D) (the number of buttons in place of the punctuation touch area is 5, the number of divisions in the setting screen is 11). The fact that the number of allocations themselves is different explains the image.
As a general rule, the basic function of the mouse, "click buttons on the upper left and right", is restricted so that punctuation touches and the like cannot be set, in order to avoid general operational confusion.
However, although the voice input button itself cannot be eliminated, it does not prevent the position from being changed (the button to be selected is different).
The mouse button layout settings on the edit / modify screen, which will be described later, are also included in the figure, but in computers, most of the "edit / modify by touching the screen with a finger" as explained in the [0091] column of the portable translator is used. Since this is not done, the postscript punctuation marks are input to the created text.
Example sentence: The explanation in this section is how to set the multifunction mouse.
Although FIG. 6D shows the recognition screen, it is assumed that the same button arrangement is set on the edit / modify screen.
Specify the position between "ha" and "many" with normal mouse operation, click the button 56 in FIG. 6 (D), enter "", and similarly between "su" and "no". Click the button 57 in FIG. 6 (D) at the position of, and enter """.
Result: The explanation in this section is how to set the "multifunctional mouse".
This is how to use "postscript punctuation".
By increasing the operation dependency on the mouse side, even when using a display without a screen touch recognition function, it is possible to input punctuation marks etc. at the time of voice input to some extent from the multifunction mouse side, so the entire computer device set It also has the advantage of lowering prices.
Further, when the operator is a person with a disability in one upper limb, it is advantageous that the voice input can be completed only by the multifunction mouse 50 without touching the screen (or keyboard input).

A monitor screen having a touch recognition function leads to an increase in the price of a set of computer devices as described above.
Also, for professionals who create long sentences such as writing, the overlapping display of the semi-transparent punctuation touch area and the input target screen itself causes eyestrain, and if the two-handed arrangement is as described in column [0020], The act of raising the left hand each time the screen is touched also causes fatigue.
Therefore, it is also desirable to configure the accessory devices such as smartphones and small tablet terminals to have independent roles such as punctuation touch.
The right hand does not change the coordinate specification operation with the mouse just by adding a voice input button, and the left hand performs typeface editing operations such as punctuation marks and line breaks on a smartphone etc. while still attached to the desktop, and voice input is performed instead of the keyboard. Is to say.
In addition, even when the operator does not want to touch the screen monitor emotionally (does not want to attach fingerprints or the like), a method such as punctuation touching with an accessory device such as a smartphone is effective.

Assuming that when inputting voice, while inputting text characters while inserting punctuation marks and adjusting the font by touching the screen, the operator may make a mistake or make a mistake on the recognition side of the device. Suppose you find it.
At that time, in the case of a computer, the voice input button can be released once to fix the text, and then the voice input can be interrupted because it can be easily corrected with the mouse / keyboard as described in the [0042] column. There is no need to do it.
In addition, when inputting an amount of money with many digits, it is conceivable to input by voice on the assumption that it will be corrected later by keyboard input.
Example sentence When you want to input "This car costs 2,159,784 yen including tax", the operator once voice-inputs "This car costs 10,000 yen including tax."
After that, the color tone is inverted by tracing the "10,000" part with a normal mouse operation, and the "10,000" once voice-input above is changed to "2,159,784" by inputting from the numeric keypad on the keyboard.
Even if there is "number recognition" in the [0032] column, it is faster to directly enter a simple enumeration of numbers with the numeric keypad in order to obtain the same result.

In this way, in a computer equipped with a voice input button together, the input method using the keyboard is not hindered at all, and all the input methods using only the keyboard as before are possible.
That is, since there is no situation in which voice input according to the present invention must be performed, there is no operational demerit, and the purchaser can gradually learn the text input method using voice input.

In voice input, when the device side recognizes something different from what the operator intended (open) (hereinafter referred to as "misrecognition"), and when the device side accurately recognizes the operator's voice, If the conversion candidate judged by AI from the text contents before and after is incorrect and a word different from the intention of the operator is selected (hereinafter referred to as "same-speech synonym"), it will decrease due to the evolution of equipment and AI. Although it is a thing, it has an unavoidable property due to the characteristics of voice input.

While inputting new text by voice using the method after the description in column [0038], input by voice according to the software or homepage format such as document input software, spreadsheet software, or the inquiry field of the company's homepage. However, if you find a false recognition or a synonym for the same sound in it, you can adjust the silence time for about 0.5 to 1 second while holding down the voice input button (adjustable on the setting screen, below. After a lapse of "a certain period of time"), the screen transitions to the edit / correction screen shown in FIG.
As a result, the original software such as document input software, spreadsheet software, and Web page screens that are the target of voice input will not be closed, but the computer itself (operating system side, etc.) will process the temporary input operation. Is cut off, and the color tone is extremely lightened on the monitor to make it "background".
On the edit / correction screen 70 of FIG. 7, the text currently being input is output to the monitor with semi-transparent characters of the same color as the punctuation touch (* transparency changes to a darker color tone) in a large font and wide characters. NS.
It should be noted that the edit / correction screen 70 of FIG. 7 is a specific example in which the text is created by voice input in the [0027] field, and then the correction input of the erroneously recognized portion is performed.
As an example of the punctuation touch to be displayed, from the upper left to the lower right, """"""(("")""linefeed"".""Backone"",""conversion""blank""BS"""Symbols" etc. are provided.
It means that the enlarged font screen of only the text during voice input becomes the "main part of visual recognition" rather than the software side that is temporarily backgroundized.
If the text does not fit on the screen due to the enlargement of the font and characters, the vertical bar 71 for position adjustment appears.
If the range of correction is short, such as erroneous conversion or correction of the same sound objection, the text is fixed once as described in column [0042] on the computer, and then re-input with a normal mouse and keyboard is quick. Therefore, the editing method in the editing / modifying screen will not be described here, but detailed explanations will be given in columns [0091] to [0108] regarding a device having a size smaller than that of a portable device that does not have a mouse / keyboard.

For moving / deleting text once fixed (after releasing the voice input button), unlike the method of devices of the size such as mobile terminals described later, direct operation with the mouse is faster as a correction operation. Therefore, the correction operation by touching the screen is not assumed.

The above processing will be re-explained as an overall flow of the voice input processing based on the flowchart shown in FIG.
That is, at the start of voice input, the voice input button provided on the computer or its peripheral device (mouse or the like) is turned on.
As a result, the "pre-recognition screen", which is the screen before the start of voice recognition, is output to the display.
This pre-recognition screen is displayed so as to make the input screen transparent on the input screen of the program that performs voice input.
Then, by specifying the item set on the pre-recognition screen, the process set in the area is performed.
For example, when a specified area of an item such as parentheses (starting point side) is selected, the selected (touched) content is input to the input screen of the program for voice input, and alphabetic characters, numbers, and katakana are prioritized. , Calculation formula priority, voice input lock, etc. When an item is selected, the process set in the item is executed.

Then, when the user's voice is emitted and voice recognition is started, recognition is being performed to insert symbols such as punctuation marks, parentheses (addition on the end point side) or line breaks, and control characters in the text being input by voice. The screen output is output in place of the pre-recognition screen.
When the user emits a voice while the recognition screen is displayed on the display, the voice recognition device recognizes the pronounced voice and inputs the voice to the screen on which the voice should be input as text.
Then, when the symbol or the control character is input during the voice input, the corresponding item on the recognition screen is selected / specified and added to the voice input text.
Through the above processing, it is possible to input symbols, control characters, line breaks, and other fonts at the same time as voice input using an input device such as a touch panel, keyboard, or mouse. As a result, the voice input itself is a text part. You can enter text quickly and accurately while keeping the natural flow of speaking only.

A concrete simple operation image diagram is shown below.
FIG. 9 is an input screen in spreadsheet software, and illustrates the intermittentness of the numerical value portion and the summary portion (voice input portion).
It is said that the fastest input method can always be selected from the intermittentness of voice input, such as using voice input for the description fields such as column B 3 rows and B column 4 rows, and using the numeric keypad for numerical data after column C 3 rows. Is.
Also, if you want to enter "actual value" and it is recognized as a homonym with "actual place", trace "ground" with the mouse to invert the color tone, then enter "ai" with the keyboard and press the conversion key. For example, the correction operation is easy.
The multi-function mouse input screen of FIG. 10 is a description of a method of allocating a part of punctuation touches to a multi-function mouse and creating text without touching punctuation marks on the screen.
In addition to the normal functions of the mouse, the button 51 with the kuten ".", The button 52 with the "line feed", the button 53 with the comma ",", the button 54 with the "voice input button", and the voice. The microphone 55 is assigned, and the underlined part of the text indicates the state in which the voice input button is held down.
Therefore, the unlined text is entered normally using the keyboard.
In this example sentence, "" (brackets), PDF and @ (add mark) correspond to it.
In addition, in order to simplify the explanation, the following description will be made on the assumption that no misrecognition or the like has occurred in this section.
The text input screen of FIG. 11 is a basic operation procedure in the document input software represented by a word processor. The main texts are all voice input and the underline is omitted.
The enter key mark 111 in FIG. 11 literally gives the same result with either the enter key on the keyboard or the line feed with punctuation touch.
The □ (blank) 112 in the figure indicates a blank, and the same result can be obtained with either the space key on the keyboard or the blank of the punctuation touch.
It should be noted that the line feed or blank "automatic continuous hitting determination while the touch is held down" is executed in the same manner as the automatic repeated hitting state while the key is held down on the keyboard.
Because of this definition, "special typeface input when voice input is performed while pressing" will be explained later in the device of the size smaller than the portable device of [0132]. Cannot be set.
All punctuation marks such as ",", ".", And "() parentheses" can be configured by touching punctuation marks according to the timing during voice input.
": (Colon)" 113 and "_ (underscore)" 114 can be input directly with the keyboard, or can be obtained by voice input of "Colon" and "Ada-ba" after touching the "symbol".
For special characters 115, you can also press the conversion key after entering "1" and "2" on the numeric keypad, or if the device has the ability to recognize by voice input, you can obtain "Ichimaru" and "Nimaru" just by voice input. You can also.

<Embodiment in Input of Electronic Medical Record>
As one of the present embodiments, a case where an electronic medical record input method is implemented will be described.

Traditionally, industries where input speed has been emphasized include journalists, newspapers, and writers, in addition to simple shorthand.
Unlike these special industries, in general industries, speed is required for "input of electronic medical records" in the medical industry.
In particular, long working hours of doctors working at large hospitals have become a social problem, and speeding up the work of writing medical certificates, referral letters and summaries, in addition to inputting medical records, contributes to reducing the burden on medical staff. It is a thing.

This method has the fastest input capability in a voice input device realized by using a computer as long as it does not overlap with other work (medical procedure, etc.) in a desktop environment.
If you want to force it, on the pre-recognition screen shown in Fig. 1, select items of "disease name priority" and "medical English abbreviation priority", and depending on the ability of AI, select "medical term priority" that puts them together. It is even better if there is an area, etc., and in order to correspond to different technical terms for each clinical department, fine adjustments such as increasing the number of words that can be registered compared to the general model can be sufficiently supported as a dedicated medical machine.

In the above-mentioned medical term priority, the transition of the conversion candidate in the reconversion key from the word utterance (recognition on the device side) or the conversion area touch on the screen is performed as follows.
Example 1: Voice input "Diemu" DM (English abbreviation for diabetes) → (Arrows indicate conversion touch each time) Type I diabetes (mainly congenital) → Type II diabetes (mainly lifestyle) → Diabetes (English) → Conversion returns to DM.
Example 2: Voice input "Erushi" LC (Lung cancer abbreviation) → Lung cancer (English) → Small cell lung cancer → Non-small cell lung cancer → Conversion returns to LC.
Example 3: Voice input "Lupus erythematosus" → Lupus erythematosus → Systemic lupus erythematosus → Systemic lupus erythematosus → SLE → Systemic lupus erythematosus (English) → Lupus erythematodes (Germany) → Convert to lupus erythematosus, etc. ..
This conversion pattern giving priority to medical terms is a known technique in electronic medical record systems, but in the current system, the device side may mistakenly recognize the medical terms unintentionally by the operator.
Since the priority recognition touch in this implementation is limited to "one phrase of the next voice input after touch recognition" in principle, there is an advantage that errors are less likely to occur throughout.

In addition, electronic medical records are used by multiple people in various situations, such as in hospitals.
According to each user, for example, when a doctor or the like creates a medical record or a medical certificate on a desk as described in the [0062] column, the "normal mode" is set, and a nurse or the like prepares a medical record in parallel during some treatment. When inputting, it can be set as "no touch mode" or the like.
This is because in the medical (or long-term care) field, you cannot take your hands off during medical treatment (or long-term care), and for convenience of hand hygiene (disinfected hands, changing diapers, etc.). This is because it is assumed that the mouse or touch screen cannot be touched depending on the situation.
Therefore, in order to follow such a scene, in principle, it is a method of operating the device without using the "front side" of the hand according to the situation when inputting voice.

In this no-touch mode, even when inputting to the electronic medical record, voice recognition is performed while the voice input button is pressed as described in columns [0027] to [0029], and when this is released, the same processing as the enter key on the keyboard is performed. The principle is the same.
However, such an operation of the voice input button restrains the hand of the caregiver or the procedure to be performed at the same time, so that the voice input state can be maintained by pressing the voice input button twice at the beginning (synonymous with the voice input Lock touch on the monitor). Then, if the voice recognition button is pressed again, the Lock state is released, and at the same time, the enter key operation is performed to solve the problem.

In addition, for the convenience of hand hygiene, the voice input device according to this embodiment has "place a button on the corner of a medical (nursing care) notebook computer that is easy to press from the back of the wrist with the back of the hand", "waist". "Provide a touch button on the belt that can be pressed with an elbow", or "Place a switch on the wrist (back side) of the opposite hand to attach a mobile terminal about the size of a smart watch and cross-touch the backs of both wrists" It is also desirable to configure the voice input switch so that it can be operated by the method.
Even in the Lock release operation described in the previous section, the finger (front side of the hand) is not used.

In addition, when inputting to an electronic medical record in such a medical processing situation, there is little need to input a long sentence, and the number of times to input punctuation marks by touching the screen or input line breaks / blanks to adjust the typeface is reduced. Become.
Therefore, when inputting voice to an electronic medical record in the no-touch mode, voice input processing in known technology, for example, after opening a beat interval from the text content to be recorded, words such as "ten", "maru", and "kaigyo" are entered. By speaking verbally, the speech recognition program on the device side changes so that it can be accompanied by the process of inputting punctuation marks and typesetting.
In other words, the device does not convert ",", ".", "Line breaks", etc., which are uttered slowly and clearly independently, into text, and the device recognizes them as typefaces and symbol commands.

Further, in the voice input processing for the electronic medical record, it is desirable that the confirmation processing (enter key input) and the operation corresponding to the end of the voice input are also executed by detecting silence for a certain period of time after the voice input of one phrase.
Since the situation such as daytime care and night care is different, this silence time detection is configured to be arbitrarily set in advance.
As a result, it is possible to avoid restraining the hand of treatment or care for the execution (enter key) and the operation of canceling the voice input described in the [0065] column.
It is also effective in preventing the conversation with the patient from being recorded meaninglessly and endlessly when the operator forgets to end the voice input process or unintentionally takes his / her hand.

Since the electronic medical record is a complete database for each patient and cannot be created in different software or files for each different input method at the same medical institution, the voice input method (for example, normal mode and no-touch mode) can be selected. It is desirable that it be done instantly.
Therefore, when the voice input button on the mouse side is pressed, it is automatically set to the normal mode, and when the corner of the notebook computer is pressed or the belt / wrist switch is pressed, it is automatically set. It is more efficient to define it on the hardware (including the operating system) side so that it will be in no-touch mode.
In addition, when developing a new notebook computer dedicated to medical treatment, it is advantageous for hand hygiene to be able to operate the mouse by specifying the position at the joint on the opposite side of the finger, so it is desirable to increase the size of the "touch pad". ..

<Embodiment in computer program input>
Similar to electronic medical records, there is computer programming (voice coding: voice coding) by voice input as an example in which priority recognition of a specific word and customization of touch punctuation marks can exert more capabilities than a dedicated machine.
This voice coding itself is a known technique, but the method according to this embodiment is realized by switching the semi-transparent punctuation touch area to one that is frequently used in program input.

In the text input in Japanese input, "Katakana priority" exists in FIG. 1 on the pre-recognition screen of [0027] as an example, but it is a character that is not used in a computer program.
Therefore, those areas are allocated to Example function (function: function) Example 2 ary (array: associative array).
Similarly, some punctuation marks (starting point of hook brackets and bold) are not used in the program, so they are assigned to things that are frequently used in the program, such as {} (curly braces) and; (semicolon).

In addition, "program priority" recognition of voice recognition is set, and the above-mentioned "Funkshon" and "Arei" are also supported by increasing the voice recognition rate on the device side.
Not limited to specialized program terms, the recognition rate of TRUE (true: normal value), FALSE (false: invalid value), etc. in Excel macro (registered trademark) also increases, but the calculation formula priority in the description in column [0033] Unlike, it does not automatically shift to the calculation formula format.

When the program work is started by voice input, the above-mentioned "program priority" will be continued for the time being.
In such a case, pressing the program priority touch area each time before utterance is a burden on the operator and does not meet the purpose of speeding up. Therefore, if the program priority touch area and the voice input Lock touch area are touched at the same time, ""Programpriority" is also locked, and it can be solved by constantly applying priority identification.
To cancel this, the voice input Lock area is touched again and immediately released in the same manner as described in the [0030] column.
In addition, although it is different from the case of the program and its use is rare, it is possible to continue to lock the specific recognition by the same simultaneous pressing touch even in katakana priority and alphabet priority.
This is because, for example, when creating a menu manuscript for Western food, there may be a situation in which katakana and English sentences are continuously input only in words, such as Carbonara (Pescatora) (genovese) (Arrabbiata).
This specific recognition lock continuation is a program separate from the special operation of adjusting the typeface such as bold and underscore by pressing and holding each priority recognition touch area in the device of the size smaller than the portable device described in the column [0132] described later. be.

Also, in programming, the hook brackets """and""" used for Japanese text input are punctuation touch areas that are not used.
Furthermore, by applying the fact that short sentences can be registered as punctuation touches as described in the [0046] column, it is effective to allocate almost all translucent punctuation touch areas to "symbols frequently used in programming languages and programs".
However, it is not randomly assigned to the punctuation touch area, but the semi-transparent punctuation touch area 121 on the pre-recognition screen shown in FIG. Semi-transparent punctuation marks on the recognition-in-progress screen shown in (B) The programming languages and symbols that are easily used for the "midpoints of program lines" are displayed in the touch area 122, and the semi-transparent punctuation marks on the edit / correction screen shown in FIG. 12 (C). The touch area 123 is allocated with an emphasis on programming languages and symbols that are easily used at the "end of the line of the program".
In the programming terms of the pre-recognition screen, the recognition middle screen, and the edit / correction screen, the punctuation touch areas such as "jump to the pre-recognition screen" and "jump to the edit / correction screen" may change depending on the program content. Correspond by setting.
The blank areas on each screen are areas for program terms and program punctuation marks that are arbitrarily customized by the operator.

Unlike the case where computer programming is performed by a general person on a general-purpose computer, more specialized programmers and data entry businesses (specialized punch input businesses) require the ultimate input speed, so it is necessary to switch input devices. Generally, it does not depend on mouse operation so that it is not performed.
Those industries use a dedicated keyboard with the numeric keypad centered and a large number of function keys added.
In addition, as for the programming language, multiple character colors such as pink, light blue, and yellow are displayed on the monitor at the same time.
Even if there is a quick color change function in the description in the column [0035], the screen touch punctuation marks hinder the visual recognition including the overlapping of the color tones as the degree of specialization increases, and as described above, the description in the column [0050]. Since there is no need to switch to a multifunctional mouse, the act of releasing the hand from the keyboard by touching the screen itself does not meet the purpose of speeding up the profession.
In such a case, it is only necessary to add a "voice input button" to the keyboard dedicated to the program (puncher), and only the function to input the programming language that is insufficient with the many function keys arranged by voice does not interfere with the screen display and it can be changed. It can contribute to the fastest programming work.
In this way, from general-purpose machines (individuals) to dedicated machines (professionals), a computer program environment with a voice input function suitable for the operator's purpose can be created by combining the presence or absence of screen touch and the presence or absence of a multifunction mouse. Can be provided.
Since programming will be introduced into compulsory education in Japan from 2020, it is an implementation method that can be expected to contribute to society.

<Embodiment in Input of Physically Handicapped Person>
As a special case, when used by a person with a disability in both upper limbs, a touch pad 130 that can be operated by stepping can be used as shown in FIG. 13 (A).
Punctuation marks can be selected by stepping on the touch, and by using a mouse provided with a position designation area 131 for instructing the movement of the pointer with the foot, it can be a tool for supporting input.
In the stepping touch pad 130 shown in FIG. 13A, it is desirable to set the reception time for the double-click process to be relatively long because the operation is based on the movement of the foot.
As shown in FIG. 13A, the left button area 132 and the right button area 134 correspond to the left button and the right button of a normal mouse.
In consideration of the case where only one leg is healthy, if the left end area 133 of the left button area 132 and the right end area 135 of the right button area 134 are specified, the designated state of the area is maintained, and by designating again, the said It can also be configured to release the specified state.
Also, from the viewpoint of space restrictions and cost, one mat can be switched between "a pad for punctuation touch processing and a pad mouse operation for processing as a normal mouse" for specifying symbols such as punctuation marks. It is also possible to provide a "punctuation mode switching" designated area 136 to be used.
After touching the designated area 136, the process shifts to the stepping touch pad in the punctuation mark mode shown in FIG. 13 (B), and input assistance such as punctuation marks can be performed in voice input of text.
However, unlike the pre-recognition screen in the [0027] column relating to the computer, "return to mouse operation" is provided as the designated area 137 for shifting to the stepping touch pad in the position designation area mode shown in FIG. 13 (B).
In addition, the stepping touch pad 130 can be effectively used even when right-sided / left-sided hemiplegia develops due to a stroke or the like.
In this case, by combining the multifunctional mouse shown in the above-mentioned column [0050] with the stepping punctuation pad, computer input and writing work can be performed at a speed close to that of a healthy person.
In modern society, being able to handle computers in the same way as healthy people is very effective in solving the social advancement and labor shortages of people with physical disabilities.

"Second embodiment: smartphone-sized mobile terminal"
A mobile terminal, particularly a smartphone-sized mobile terminal, will be described based on an embodiment of a voice input device formed by a more complicated mobile translator.
Therefore, as shown in the initial screen of the portable translator of FIG. 14, the voice input device according to the present embodiment includes both the native language input button 143 and the foreign language input button 141 of the other party.
Regarding the C button 142 located between the two buttons, "C" meaning clear and cancel, which is close to the function of the in-screen touch area "back one", and "C" meaning conversion, that is, change and conversion. It is also used, and its usage will be described later.
There are many existing mobile translators that can input by voice regardless of whether they are online or offline, but when inputting the native language side, it is possible to edit and modify without connecting a keyboard or mouse. This is the greatest feature of the voice input device according to the present embodiment.
If the recognition of the native language input side is wrong in the first place, the translation result will naturally have the wrong meaning, that is, it will give a fatal misunderstanding to the foreigner on the other side, so it will be the most required ability in the mobile translator.

FIG. 14 is a screen before the voice input language is selected (hereinafter referred to as “initial screen”).
In this voice input portable translator, national flags are displayed in parallel on each button to indicate the meaning of "what word and what word translation device".
It is also the screen when the foreign language type setting is changed or immediately after the power is turned on.
By touching the screen of the national flag portion 145 in this state, it is possible to move to the translation language selection setting screen.
The national flag itself is displayed in a state surrounded by a color that matches the "button color" of the native language and foreign language buttons, making it easy for the operator and the foreigner of the other party to understand the commonality with the buttons.
The box in the same color as the button and the national flag display itself are also displayed in semi-transparent color so that the explanation of the punctuation touch area in the pre-recognition screen is not invisible at all.
Further, by flicking the blank part other than the national flag to the left flick operation 146, the screen jumps to the system setting screen of the device, and in the case of the right flick operation 147, the presentation screen described in the [0117] column described later is jumped.

On this initial screen, if the Japanese button 143 of the voice input button is pressed when translating from Japanese to English, a hierarchical relationship occurs in the national flags displayed in parallel on the screen 144, and FIG. 15 shows. If you follow the screen example shown, you can visually understand the situation of translating from Japanese to English.
This is to make it easier to inform the foreigner of the other party of the changed state.

FIG. 15 shows the display of punctuation marks and the like in a state where the device does not detect even one voice.
As an example of arrangement, "number priority", "alphabet priority", "katakana priority", "no conversion", "(", """," symbol and setting screen "(these two items blink alternately in the same touch area). Yes) "@""Voice input Lock""Word""*""Returnone" etc.
This state is synonymous with the pre-recognition screen on the computer.
As for the symbol, it is a method to specify by voice input as "@" and "?"
For words, pre-register, select the designated area of "word" by touching the screen at the time of operation, and then predictive conversion is performed from the beginning of the corresponding word voice input, and "Y" whether or not this displayed word is correct. The / N "display unit appears on the screen, and the word is selected by, for example, touching the screen of" Y ".
Frequently used symbols and the like can be freely arranged on the setting screen as in the case of the [0046] column in the computer.
Further, even in the correction / editing screen shown in the [0108] column described later, some punctuation marks and symbols can be retrofitted even after the texts are piled up.

Since the portable translator is a dedicated machine and the background color is constant and does not change, a quick color change as described in the [0035] column of the computer is not required.
However, it is common to have "semi-transparent punctuation touch", and in the case of "color blindness" among visually impaired people, there is a possibility that they cannot be visually recognized if there is only one color of punctuation marks.
Therefore, you can select "pink", "light blue", "green", and "orange" tones and customize the punctuation touch in the same way as a computer.
For mobile translators (including smartphones) and devices of the following sizes, the area of each touch area on the screen is small, and the method of "clicking the icon that moves to the setting screen separately from the touch area" on the computer is used. Since it is difficult, the voice input itself can also serve as a command to the device body.
This will explain the method in the typewriter mode described in the [0123] column described later.

Unlike the dedicated icon click explained in the [0043] column of the computer, it is possible to apply a special process to one part of the touch area to move to the setting screen without using voice input.
The jump to the setting screen in the portable translator and its details will be described with reference to FIGS. 16 and 17.
FIG. 16A shows a state in which the touch area 161 of the “symbol and setting screen” is pressed and held (about 2 seconds, the same applies hereinafter) to jump (transition) to the “setting screen”.
The method and purpose of specifying different commands in the same touch area will be explained in the next section.
FIG. 16B shows a state in which the punctuation mark touch area is reduced to about 80% and the “touch icon dedicated to the setting screen” 162 is displayed in the space created by the reduction, as described in the computer [0045] column. ing.
Further, it shows a state in which the "recognizing screen" is selected from the upper row dedicated touch icons in the touch icon 162 dedicated to the setting screen, and the screen for dividing the touch area during voice recognition is called.
FIG. 16C shows a state in which the touch area 163 of the “conversion” is moved to the location of the touch area of the “symbol” by a drag and drop operation, and the procedure and the result are the same as in the computer example.
FIG. 16 (D) shows the screen transition when the touch area 164 of "color tone change" is selected.
FIG. 17A shows a situation in which the setting screen is called by an instruction to the device by voice input in the “typewriter mode” described in the column [0123] described later. In this example, the process is executed by selecting both the native language input button 143 and the foreign language input button 141 of the other party.
FIG. 17B shows a state in which the touch area 171 of “*” is long-pressed to accept movement or content change of the touch area.
FIG. 17C shows a situation in which voice input is restarted and "voice Taro" and his / her own name are set as a touch area.
The range specification and screen transition will be described in the [0091] column described later.
FIG. 17 (D) shows a state in which "decision" of the punctuation touch area is confirmed, and it is determined by touching "Y", and "going back one" is selected by touching "N". It is synonymous with punctuation.

Various migration methods and settings for customizing touch areas such as punctuation marks (customization of punctuation marks) The screen layout and the items that can be set (color tone, font, number of divisions, arrangement, etc.) will be described with reference to FIG.
In FIG. 18, the display positions of the key brackets (""), punctuation marks (.), Comma (,), parentheses (()), etc. can be exchanged by dragging and dropping, and the exclamation mark. After pressing and holding the (!) Display area, you can enter "@" with the voice input "at sign".
Also, after long-pressing the display area of "Word registration", after confirming "New / change?", Enter the kanji and reading kana by voice input to register the word.
The symbols and setting screens in FIG. 17 are blinking alternately as in the previous section, and if you touch and release them immediately, the recognition on the device side will change to "The next one phrase voice input is a symbol selection command". If the touch area at the same location is held down for a certain period of time or longer, the device recognizes it as a jump command to the setting screen.
The "combined use" is applied to such a limited touch area.
In addition, it is not essential to use both, and you can customize it by deleting the symbol in the setting screen (right flick operation after color tone inversion) and touching "Jump to setting screen" alone.
In addition, the touch area can be replaced or changed according to the method described in the [0046] column of the computer, but since there is no keyboard, new punctuation marks and word registration methods can be performed by pressing and holding the existing touch area for a certain period of time after reversing the color tone. Perform by voice input.
The method itself is the same as the rephrasing operation at the time of erroneous recognition described in the column [0091] described later.

In addition, if the operator makes a big mistake in the operation or the content / configuration of the voice input during the voice input, or if the operation is confused, touch the screen of the "back one" (or C button) designated area described later. Press and hold to return to the initial screen before the voice input button (the screen without punctuation touch in the national flag parallel state).
It has the meaning of "reset operation" to start over from scratch.

When the device recognizes even one voice as described in column [0038] on the computer, the flag display that has a hierarchical relationship disappears, and at the same time as the voice-recognized text is piled up, the recognition screen of FIG. 19 appears. As shown, the screen automatically transitions to the next punctuation candidate display screen used for voice input.
Examples of the arrangement are "."",""Conversion""(""""" Line feed "") """""Voice input Lock""Word""Symbol""Return by one".
If the native language text you want to input is completed accurately from the beginning by automatic conversion by AI of the device body with voice input, by releasing the voice input button, it will be sent to the translation AI via the Internet etc. You will have to wait for the translation result.
That is, the various correction and editing methods described below are explanations of "if there is a mistake found or a part to be corrected", and the present invention is applied to the current portable device for voice input that cannot be corrected or edited. Has no disadvantages in terms of response.
It should be noted that the text in FIG. 19 is displayed on the assumption that a recognition error similar to that in the [0055] column of the computer has occurred.
In addition, due to the nature of being able to make corrections, it is expected that the number of long text input cases will increase, and if the text accumulated by voice input does not fit on the screen, use the edit / correction screen described in the [089] column below. A "screen position selection vertical bar" similar to the frequently used scroll bar automatically appears.

When the operator determines that the entire composition of the sentence has failed, if the C button 142 is pressed and immediately released without releasing the voice input button, the "clear" process is executed and the entire sentence is deleted. The screen returns to the pre-recognition screen shown.
In other words, the upper limit relationship of the national flag and the punctuation touch display are maintained, but all the text input by voice until now disappears, and it means that it will be in a state of redoing.
This is different from the description in the [0084] column of the reset operation, and if you can operate to the situation of "rephrasing from the beginning" while continuing voice recognition, the mobile translator will not be repeatedly connected to the Internet environment, so go to the device. This is because it is advantageous in terms of load (connection error) and response.
If the voice input button is released after that, the screen returns to the initial screen in parallel with the national flags as described in the [0078] column, or if the foreign language button 141 is pressed, the pre-recognition screen for translating from English to Japanese is displayed immediately.
In this case, the connection with the translation AI may be cut, or another translation AI existing on the Internet may be reselected (reconnected) depending on the foreign language selected.
Further, when the foreign language button 141 is pressed, the top and bottom display of the screen is reversed as described in the [0120] column described later depending on the setting.

Even if you do not release your finger from the voice input button or release the voice input button after touching the screen in the selection area of "Voice input Lock", the voice input button continues to be pressed as described in the [0029] column on the computer. Defined as being.
This is to reduce the burden on the operator when inputting a long sentence like a computer, and the release method is also executed at the moment when the Lock is touched and released again as described in the [0030] column.

The portable translation device according to the present embodiment is edited as shown in FIG. 20 when the device detects a voice input silence state for a certain period of time in a state where texts are piled up to some extent, as described in the [0055] column of the computer.・ Automatic transition to the correction screen.
As an example of the arrangement of punctuation touch items, """""""."",""Backone""Conversion""?""!""Return to initial punctuation" from the upper left to the lower right. It is to be provided.
Even if the operator is confused about what to say, or even if he / she unintentionally transitions to this screen, if he / she continues to input voice, the screen will return to the recognition screen shown in FIG. It doesn't change to go up.

Unlike a computer, the portable translation device according to this embodiment does not have a function equivalent to a mouse as well as a keyboard. Therefore, on the edit / correct screen, the part to be corrected can be specified by pointing to the device main body screen (touch panel). You will touch the screen with.
Therefore, when the screen transitions to the edit / correction screen shown in FIG. 20, the font of the text input up to that point is automatically enlarged, and the layout changes to a wide space between characters.
Therefore, in the state of the recognizing screen shown in FIG. 19, even if all the text is contained in the screen, the font and the character spacing are enlarged and the auxiliary input display unit is displayed at the moment of transition to the edit / correction screen shown in FIG. With the advent of 203, there are many cases where the correction text input screen 201 cannot be accommodated, and the function of moving the display screen position by the vertical scroll bar 202 is indispensable.

For mobile devices, etc., whose screen size is smaller than that of a computer, the default (initial value) number of touch area divisions for the edit / correction screen is set to 9, and the number of areas is smaller than that of a computer. No touch is placed.
Therefore, if you want to unlock the lock (in a mobile translator, the act of completing the editing / correction work and sending it to the translation AI), use the voice input button on the mouse side described in the paragraph number [0029] column on the computer. By applying the operation to regain the right to release the lock, the lock can be released only by "pressing and releasing the voice input button (native language button)".
Of course, if the voice input Lock touch is not performed on the pre-recognition screen, etc., and the screen is transitioning to the edit / correction screen while holding down the voice input button from beginning to end, the transmission to the translation AI is "just release" the voice input button. Become.
In this situation where the number of touch areas decreases, the originally existing physical buttons are used as a substitute for the same operation.
This also applies to smart watches, which will be described later, in which the screen size is further limited.

Here, a specific method of editing / correcting will be described with reference to the screen transition diagram of FIG.
The example sentences are the same as those described in the [0055] column regarding the computer. In the following computer, the same operation described in the future can be performed by simply touching the screen without releasing the voice input button and using the keyboard. Everything is possible.
The case of editing "bathing" to "input" in the example sentence "editing such as" correction "and" rephrasing "even with voice bathing" in the voice input in the present embodiment will be described as an example.
As shown in FIG. 21, first, when changing "bathing" to "input" (that is, correcting erroneous recognition), the voice input is silenced on the recognition screen of FIG. 21 (A). , The edit / correction screen shown in FIG. 21 (B) is displayed.
Then, on the edit / correct screen, the operator first touches the screen with a finger to select the space between the start point 211 and the end point 212 of the erroneous phrase to be corrected.
As a result, the device side temporarily stores the text of the start point and the end point as the "target position" of the phrase to be exchanged.
At this moment, for languages with clear phonetic characters (such as Japanese), the entire specified range returns to hiragana.
This is so that the operator can visually recognize whether the word is a false recognition or a homonym.
The screen touch in the "back one" selection area erases incorrect punctuation marks, etc., and is also used when the screen touch operation at the start and end points is incorrect.
Then, when a new voice input is added as it is, the device side temporarily stores it as a "phrase to be overwritten".
When this new voice input type is piled up on the screen, the modified text input screen 201 of FIG. 21 (C) is displayed, and the entire text input so far disappears temporarily from the screen and is rephrased. Executes a process in which only text appears on the screen.
However, even if only "Nyuryoku" is spoken as a single phrase, the recognition by AI on the device side guesses the word from before and after the sentence, so the conversion accuracy is lowered.
Therefore, the operator predicts the content in which the target "input" text is likely to appear, such as "computer", and emits a voice.
After that, by touching the screen to select the start point 211 and the end point 212 between the characters in the input text part, "specifying the content of the phrase to be overwritten" is confirmed, and as shown in FIG. 21 (D), the initial "replacement" is performed. The correction result can be obtained by replacing it with the phrase part of the desired target position.
Even if you perform voice input in association with the target word, there is a possibility that homonyms will be selected or the position will be specified incorrectly by touching your finger. The touch area is set in the blank area at the bottom right, which is less likely to interfere with the visibility of the rephrased text.
Of the additional rephrasing voice inputs, the partial texts of "computer" and "do" outside the specified range are temporary to obtain "input", and as a result, they are ignored and the data is discarded. It is a thing.

Specific examples of rephrasing in English input are as follows.
Regarding the example sentence "Please tell me how to open it with this machine." ( Meaning, please tell me how to operate this machine), the device side says "Please tell me how to open it with this machine." Please tell me, meaning), and suppose that only a part of it was misrecognized.

In this case, after specifying the range of the "open it with" part, the operator abbreviates "How to operate the machine." (How to operate the machine) and the operator inputs voice again, and the recognition result to be obtained is displayed on the screen. When it is displayed, touch the screen of the start point and end point of the "operate" part to select it.
As a result, the first positioned "wrong phrase" and the second voice input "desired phrase" are exchanged to achieve the purpose.

Next, with reference to FIG. 21 (E), operations and processes when changing “etc.” to “etc.” (reconversion of homophones, etc.) will be described.
At that time, the designation of the start point 211 and the end point 212 of “etc.” is as described in the above-mentioned [0091] column, which is the target position to be corrected.
Then, after specifying the change target, press the "C button 142" (or "convert" touch if the conversion touch is set in the screen) to select it, and then select "change: change or conversion: conversion" (meaning of conversion). Executes the processing of.
On the edit / modify screen, the selection of the C button 142 or the like after the modification range is specified is set by the program to execute the "conversion" process, not the clear or cancel process at the time of the voice input process or the like.
These are the same processes as, for example, in text input on a computer, the input text is specified in a range with a mouse, the color tone is inverted, and then the conversion key is pressed.
Setting the "C button 142" independently is a function unique to a mobile translation-only machine that is frequently used, and in the case of a computer or smartphone, "conversion" is one of the touch areas of the keyboard or screen.

The conversion candidate pattern will be described.
It will be reprinted, but basically, after specifying the position of the start point and end point, the text of the selected area is "all open".
With the next conversion touch (or C button selection), "several examples of the next kanji candidates other than those selected by AI first" → "all katakana" etc. are converted.
For example, when you want to enter (purpose sentence) "I accidentally converted XX about this matter", the device side recognizes it as a homonym saying "I apologized for this matter and returned XX." In that case, if you select "Apologize" in the start / end point selection and select "Apologize" in the start / end point operation, it will be converted to the hiragana of "Ayame".
Then, the operator confirms this, presses the conversion touch (or C button) multiple times, converts "mistake" to "wrong" → "wrong" → "wrong", and the target phrase. When appears, press and hold the conversion touch (or C button) for a certain period of time to confirm.
Then, the same operation as described above is continued to the starting point touch for re-conversion of the erroneous recognition unit of the next "return".
Note that pressing and holding the C button as a conversion decision command is a different program from "reset and return to the initial screen" in the description in the [0084] column, and is not applied during conversion candidate selection.
If the target conversion candidate goes too far while pressing the conversion touch, the previous candidate can be returned by touching the screen in the "back one" area.
In that case, immediately after touching "Return by one", the conversion candidate for the returned purpose is displayed, and then the device side displays the (Y / N) option and decides with YES.
In the case of No, the conversion touch (or C button) of the homonym can be continued again.

There are so many candidates for general conversion in the current computer, and conversion candidates may continue to appear one after another, but in the case of mobile terminals, it will be an operational stress, so it is desirable to discontinue the candidates earlier than the computer. ..
This is because it is more efficient to execute the process of "adding and then moving / deleting in another way later" described in the columns [0102] to [0107] described later.

This function is limited to mobile translation-only machines (or smartphones with a translation application installed, etc.), but "Romaji in double quotation" is set in the conversion candidates.
In the current online AI translation, this conversion candidate is necessary when the "original expression of the mother tongue" is transmitted to the foreigner side by excluding it from the translation target.
AI is evolving day by day, and translation rules may change in the future.
To show an example sentence, if "This song title is pronounced as cherry in Japanese. The meaning is fruit cherry ." Is translated into English using the current AI translation, "This song name is pronounced cherry." in Japanese. the meaningis fruit cherries. ", and also when it is translated into Chinese," ▲ crawling ▼个曲name for Date ▲ Kataritomo ▼ sound ▲ for ▼ SakuraMomo. intention Shi water results basis SakuraMomo. ", and the intention I can't communicate.
Therefore, the input side of the day devised in the text added, repeating the conversion touch when you enter the first round of cherries, select the Roman "Sakuranbo" within double quotes "This Song is in Japanese pronounced" Sakuranbo " The meaning is fruit cherries. "And send it to AI translation.
Thereby, in English, "This song is pronounced" Sakuranbo "in Japanese. The meaning is fruit cherries. ", And also in the Chinese "▲ crawling ▼个曲name for Date ▲ Kataritomo ▼ sound ▲ for ▼" Sakuranbo ". Intention Shi Mizuhate manner SakuraMomo. ", and it is possible to obtain an accurate translation result.
After that, even if you click the voice output microphone icon of general translation software on the current Internet, you can hear "cherries " that are close to Japanese pronunciation, and you will reach your goal.
This is a conversion method that prevents the AI side from translating without permission even when introducing one's name.

The special conversion operation of homonyms will be described.
This is a method used when you want to create a sentence content consisting of homonyms at the lower level of conversion selection candidates, which is not often used on a daily basis.
Example sentence "I received a mission from the church." This is the purpose sentence.
Misrecognition Suppose that "selected by the church" and "delegation" have been recognized as "selected" due to misrecognition of the surrounding environment and equipment.
It is assumed that the operation of rephrasing "selection" → "delegation" is performed by the method described in the above column [0091].
Even if you rephrase it in various ways (expressions), you can't get "commission", which is rarely used in daily life. If only such candidates appear, a stalemate will occur.
Therefore, while holding down the conversion touch (or C button), the voice input of "se-n-ta-ku" is started with the utterance of "slowly and clearly only one word", and the utterance of the character "ku" is started. At the same time, when the conversion touch (or C button) is released, the hiragana of "Sentaku" is inverted in color and converted to text while waiting for conversion.
Similar to the state of inputting Japanese with normal keyboard input, it is the same as the state of waiting for the conversion key to be pressed from the beginning immediately after text input (text in the state of color inversion or wavy underline).
In this case, even with a portable translator or the like, candidates appear endlessly to the lower ranks of conversion candidates as in the case of ordinary computer keyboard input, and the desired "delegation" can be obtained.
The normal reconversion operation is performed in the edit / correction screen, but when inputting this "slowly and clearly only one word", it is performed in the voice recognition screen, so the conversion touch (or C button 142). Is also accepted directly on this screen.
If the purpose of the operation is "just return to hiragana", after returning to hiragana on the edit / correction screen, the text "remains in hiragana" is the same as when moving to another correction part or inputting additional voice. Is left behind and only shifts to another operation (touch) or text display of additional voice input, and it is applied to full hiragana, that is, "non-conversion touch" that does not exist on the recognizing screen by default. It is an operation method that doubles as a substitute.

If the operator wants to use these operations frequently, even if the C button is originally present in the portable translator, the computer can be customized and recognized in the same manner as the method described in the [0046] column. You can also add a "convert" touch area in the middle screen.
However, pressing the C button and the conversion touch at the same time leads to a contradiction operation. Therefore, for a device that has the C button separately, if the conversion touch exists as an additional setting on the recognition screen side, it is converted to the C button. The command right of "" is not given, and it is programmed to be a command button dedicated to clear / cancel.

If the conversion touch is not set by customization as in the previous section and is also used as the C button, normally, long pressing of the C button will result in a full clear operation as described in the [0084] column.
However, if the device detects a new voice input within about 2 seconds until "long press all clear" is executed, the command recognition of all clear is canceled and "slowly and clearly one word" is open. The recognition side of text conversion has priority.

Further, even in a device having a size smaller than that of a mobile terminal without a mouse, in the present invention, the text input by voice can be inserted as follows.
In the edit / modify screen shown in FIG. 20, when only one point of the text displayed in the display area 201 is specified by touching the screen and voice input is additionally started, the "insert phrase" process is executed. do.
As shown in FIG. 22 (A), for example, "The feature of this new model is that it can be rephrased and reconverted." If you want to add it to the position after the completed "re-conversion", keep the silent state for a certain period of time with the voice input button turned on, and edit / correct as shown in FIG. 22 (B). Display the screen.
Then, in the text displayed in the display area 201, after the character spacing 221 of "re-conversion" and "can be" is specified by a one-point touch with a finger, as shown in FIG. 22C, the text to be added " And various edits are pronounced and voice input is performed, and on this modified text input screen, the input start point 211 and the end point 212 in the textualized "and various edits" are specified.
As a result, as shown in FIG. 22 (D), the text of "and various edits" is inserted at the position specified by the screen touch (between the characters "reconvert" and "can"), and " The feature of this new model is that it can be rephrased, reconverted, and edited in various ways. "

The insert operation can also be performed by another method, which is an insert operation involving additional voice input and move / delete.
Example sentence (purpose sentence) "Hello, the fruits on the product shelves look very delicious. I want to buy one apple and one banana on display. As you can see, is my money enough?"
Equipment recognition text " Today , the fruits on the shelves look very delicious. I want to buy one apple and one banana on display. As you can see, my money is a tile ." It is assumed that two mistakes have occurred.
The first place "Hello" is converted to "Today", and the translation result is "Hello" → "Today".
The second place "sufficient" is erroneously recognized as "tile".
* These are actually based on misidentification examples in current voice input mobile translators.
Further, it is assumed that the operator thinks "I want to purchase lemon together " during the initial voice input.

While grasping the misrecognition, the operator first implements a method of continuously inputting "the lemon is also willing to purchase".
However, even if you say "lemon" alone, you cannot get "apple" and "connect between" and "like a banana". The target text you want to get with additional voice input is "and lemon".
Nouns fused with such sending kana and auxiliary verbs cannot generate AI by a single voice input.
For example, "orange and lemon juice" means that it is necessary to cut and paste "and lemon" from the additional utterances that the AI side can guess.
In this case, the operator said, "Hello (wrong today), the fruits on the shelves look very good. I want to buy one apple and one banana on display. As you can see, my money is as you can see. "Is it enough (wrong tile)?" Followed by voice input of "orange and lemon juice".

After that, the voice input button is changed from the silent state for a certain period of time to the edit / correction screen as shown in FIG.
After that, in addition to normal reconversion / rephrasing, editing / correction procedures that involve moving / deleting will be explained.
The first place of the error is to specify 2 points (start point / end point in the description in the [0091] column) and return to "hello" by full-scale conversion to reach the purpose. Although it is in color, it is canceled as defined in the description in the [0091] column when the starting point of the "tile" in the second error is touched.
The second error is the additional voice input after specifying two points, and since it is the same device when uttering the word "sufficient", there is a high possibility that it will be mistakenly recognized as "tile" again, so it is associated with "sufficiently". As described in the [0091] column, only the "sufficient" part is determined by specifying the start point and end point, and the purpose is achieved up to this point.

In the edit / correction screen as shown in FIG. 20, when the start point 211 / end point 212 of the text displayed on the correction text input screen 201 is selected by touching the screen, the screen touch at the end point is not separated for a certain period of time or more. And "Move / Delete specified phrase" is recognized as the reception status.
In this state, the color tone is inverted and the operator can determine that the character string can be moved / deleted.
It should be noted that, according to the definition described in the [0091] column after the end point touch for the purpose of moving only one word, even if all of them change once, the end point touch is not released and a certain period of time elapses, "movable / deleteable state". When becomes, it returns to the conversion candidate which AI originally selected.
In addition, although it is a different case from the example sentence, when a long sentence (all paragraphs, etc.) is targeted for the position specification of the start point and end point, it is recognized as the range specification of movement / deletion from the beginning of the operation, and it is within the specified range without waiting for a certain period of time. The tones of all text are inverted.
It is a range recognition only for the purpose of moving / deleting, and does not respond to subsequent conversion touches.
This is because even in a computer, even if a wide range (about one line or more) is specified and then the conversion key is pressed, it does not respond, and AI cannot generate conversion candidates.

In the case of the operation procedure to move "and lemon" in the example sentence, after moving to the edit / correction screen, operate the vertical bar to move the screen to the end, and in the display area, "(preamble omitted) ... Select the start point of "and lemon" in the "orange and lemon juice" part, maintain the state where the end point is specified, invert the color tone, and then change the character string of "and lemon" to "banana" and " By moving it between the letters "" and moving your finger off the screen, "and lemon" is inserted at the specified position.
It is the same operation as moving and placing the part after specifying the range with the mouse on the computer.
Alternatively, the operation image is similar to the "swipe operation" in the current tablet terminal or the like.

As a result, the additional voice input "orange juice" used to obtain "and lemon" is left behind, and "deletion" is required.
The operation method for deletion is the same as described in the [0105] column until the position of the end point is specified, the finger is not released, and the display is changed to the color tone inversion display indicating the acceptance of movement / deletion.
After that, by quickly and greatly performing a "right flick operation" on the relevant part, the selected text is "deleted" according to the image of throwing the selected phrase, and the blank part of the text is filled.

In addition, the characters such as "!" And "?" Existing in the auxiliary input display unit 153 on the outer periphery of the edit / correct screen are also moved by the movement method (in this case, long press) in which the mark is described in the paragraph number [0105] column after selection. (No time lapse is required), it can be used as a "postscript punctuation mark, etc." for the operation of pulling and placing the selected character.

Of course, according to the method described in the [0101] column, touch and specify only one point between the letters "na" and "o" of the banana, add "orange and lemon juice" by voice input, and only "and lemon". You can get the same result by selecting.
However, in the case of this method, two screen transitions occur in the edit / modify screen.
In addition, the voice input itself will be uttered in two steps after the transition to the edit / correction screen for the main text input and the additional input "orange and lemon juice".
By continuing to input voice as described in the [0103] column, screen transitions and utterances can be integrated, and as a result, the methods described in the [0103] and subsequent columns can save labor.
In addition, all voice input portable translators currently on the market cannot be rephrased, reconverted, edited, corrected, or added / removed, and if an error is found, voice input must be performed from the beginning.
Even if the operator discovers an erroneous recognition or a homonym during input by each method in the present invention, "the ability to correct any number of times later" can reduce the mental burden on the operator, and can be said to the end at once. As a result, it is faster than rephrasing or transitioning to the edit / correct screen each time.

In order to explain the above image of movement / deletion, the procedures (A) to (F) will be described separately with reference to FIG. 23.
Procedure (A): Specify the start and end positions of "Today" and make them all hiragana (do not press the conversion touch).
Procedure (B): Specify the start / end position of the "tile", and specify the start / end point only for the "sufficient" part obtained from the additional voice input "sufficient".
The tiles are replaced at the same time as the end point touch of "Ru". If the position of the start point / end point is specified incorrectly, the operation history is returned by one by touching "Back one".
Procedure (C): Designate the start and end points of "and lemon", and wait for the movable state (color tone inversion) without releasing the finger for specifying the end point position.
Step (D): Move between the banana "na" and the next "o" without releasing the finger of step (C).
Procedure (E): Specify the start and end points of the "orange juice" left by the additional voice input, wait for the color tone to be reversed without releasing the finger at the end point, and delete it by right flicking after the inversion.
Procedure (F): Pull "?" From the outer frame icon part and move it to the end of the sentence.

In the result of creating the accurate voice input sentence so far, if it is translated into English (however, the following example is an AI translation, so the translation result is not always the same), "Hello, fruit of the shelves" is very delicious. I wound like to buy one translated apple, one banana and one lemon. My money is as you see, is it.
When this is reversed by the current AI translation on the Internet, "Hello, the fruits on the shelf are very delicious. I want to buy an apple, a banana, and a lemon. I want to buy one. The money is exactly what you see, is that enough? "

Due to its characteristics, the portable translator cannot release the voice input button (or unlock it) until the completed sentence is obtained.
This is because after it has been sent to the translated AI on the Internet, it changes to a "foreign language text" that has left the operator's hands.
Therefore, we will explain how to deal with the need for "special proper nouns that combine letters, numbers, etc." at intermediate positions instead of at the beginning of the text.
For example, the case where the proper noun "A5 version" explained in the [0032] column about the alphabet priority and the number priority of the computer is put in the "middle part" of the whole sentence.
Example sentence (purpose sentence) "Because it will be used in the field, submit it in A5 version vertical writing."
In such a case, if it is a computer etc., there is no restriction that it cannot be released, so after inputting the voice of "I will use it in the field", release the button once and fix the text up to that point. If the voice input is restarted, "A" and "5" can be started at the beginning, so there is no problem in the input operation.
However, only the translator cannot do that due to the above restrictions, so the problem is solved by setting a special touch area that says "return to the initial punctuation mark".
When the voice input reaches "Because it is used in the field", the area of "Return to initial punctuation mark" in the edit / correction screen of FIG. 20 is touched to transition to the initial punctuation mark screen of FIG. 24.
This makes it possible to call up "priority recognition at the beginning of the text" that normally exists on the pre-recognition screen even during voice input of the text, and special proper nouns (facility names and artists that combine letters and numbers). You will be able to respond to names).
Since this is not a frequently used function, it is not allocated to the punctuation touch area of the initial value (default), and can be selected in the recognition screen or edit / correction screen by customization by the method described in the [0046] column or the like. Shall be.
Examples of placement are "Opening alphabet / Caps Lock241", "Opening number / Full-width / Half-width 242", "Opening non-conversion / Full-width 243", "Back one", "Opening Katakana / Full-width / Half-width 244", and "Punctuation mark normally". Return 245, "word registration," and so on.

Further, when inputting with the foreign language button, symbols such as control character grammar and punctuation marks displayed on the auxiliary input display unit 203 are changed according to the rules of the language.
Of course, when inputting voice in English, there is no need for a recognition touch area that prioritizes alphabets and katakana.
Instead, "Caps Lock" may require a designated area such as uppercase and lowercase letters grammatically.
In addition, there are three types of Chinese punctuation marks: ".", "," And ", (comma)".
For examples of such various patterns in foreign languages, the pre-recognition screen in the case of voice input in English is "Number priority" and "English abbreviation priority" as shown in FIG. 25 (A). "Caps Lock""$""(""""" symbol (symbol) "" Settings screen (setting screen) "(these two are mixed in the same area)" @ "" Voice input Lock "" Registered words (registration) It can be an auxiliary input display unit consisting of "words)", "*", and "Go back one step", and the recognition screen is ".""," As shown in Fig. 25 (B). Auxiliary input display consisting of ": (colon)""(""""" new line (line feed) "") """""Voice input Lock""Registeredwords""symbol""Go back one step" In addition, on the edit / correction screen, as shown in FIG. 25 (C), """""""."",""Go back one step""change or combination""?""!" It can be an auxiliary input display unit consisting of "-(hyphen)".
The definition of English abbreviation priority is the acronym of the alphabet "UN" automatically surrounded by double quotes when, for example, "United Nations" (meaning the United Nations) is input by voice after the touch. It is a program that raises the probability of being recognized as a proper noun abbreviated as.
Similarly, in the case of voice input in Chinese, the pre-recognition screen is as shown in Fig. 25 (D). """""(""<<""▲ 簽 ▼ collection (symbol)""▲ setting ▼ fixed screen" (these two are mixed in the same area) "@""▲ word ▼ sound ▲ transportation ▼ fixed (fixed) It can be used as an auxiliary input display unit consisting of "Voice input Lock)", "Note ▲ words ▼ (registered words)", "*", "Return 1 ▲ item ▼ Duties ▼ (Return one work)", and the recognition screen. As shown in Fig. 25 (E), "."","","""""(""<<""""")"">>""Notes ▲ words ▼ (registered words)""* It can be an auxiliary input display unit consisting of "▲ word ▼ sound ▲ transport ▼ fixed input (voice input Lock)", and on the edit / correction screen, as shown in Fig. 25 (F), "."",",""(""Return 1 ▲ item ▼ duty ▼"")""?""▲ conversion ▼ order (conversion)"":" can be an auxiliary input display unit.
In the case of voice input in a foreign language, in addition to English and Chinese, the punctuation marks and explanations displayed can be changed according to the selected language.
For example, in the case of Arabic, the "?" Mark is upside down, the "," is upside down, etc. Set.
All of them are just examples because each translucent punctuation touch can be customized as described in the [0046] column regarding the computer.

And even in the case of voice input in English or the like as described above, it is desirable to provide a conversion touch (or C button 142) instructing the re-conversion process.
This is because there is a homonym called "Homonyms" in English, although it is less necessary than in Chinese characters such as Japanese.
For example, "ad (advertisement)" and "add (add)", "ball (ball)" and "bowl (scream)" and the like.
In the current AI, which phrase is automatically selected from before and after the sentence, but once it is mistakenly selected by the AI side, it is meaningless to rephrase a short phrase, so the reconversion instruction function is a mobile translator. Then it becomes indispensable.

It should be noted that the current AI recognition is a little weak at determining the delimiter position of a long phrase.
Especially in translation between Japanese and other languages, the order of the subject and the object is different, so the influence on the translation becomes large.
In the method of the present invention, punctuation marks, especially "." And ". (Period)" can be input arbitrarily, and since the boundary of the phrase is fixed, the translation result of a long sentence is compared with the conventional portable translator. Has the advantage of being accurate.

Due to its peculiarity, it is desirable that the translation-only mobile terminal be equipped with the following functions and processing capabilities.

The first function is a process of recording the history of translation results in a storage BOX provided in a memory or the like of a device, assigning an order, arranging them, and calling them.
The main purpose is not to translate directly in ad lib, but to create and store the content of the presentation in advance.
In addition, even if ad-lib translation such as a question and answer from the other party is performed in the middle of the presentation, the save BOX is recalled, and when the presentation is resumed, the program is programmed so that the position of the progress before ad-lib translation is displayed. The flow is not interrupted, and it is effective for PR of products for foreigners and lessons for foreigners of sports instructors.
Therefore, the ordered texts are saved, the output processing of the saved texts is executed, and the voice input translation during the execution is also executed.
As an operation method for the device, on the initial screen described in the [0078] column, right-flick the blank part outside the national flag to move to the translation history correction screen (presentation group or order assignment work screen).

The second function is to improve the accuracy of the translated text created in advance. If you select the native language button while playing the translation history, the foreign language type text of the translation result will be sent back to the translation AI for retranslation. It is something to do.
For example, the result text translated from Japanese to English is used in reverse, and a process of retranslating it into Japanese and verifying it (hereinafter referred to as "reverse translation") is executed.
After confirming that no mistranslation has occurred, it can be recorded as the content of the presentation in the previous section, so that the accuracy of the finished product (that is, the translated text) is improved.
As a method of operating the device, if you press the native language button while replaying the translation result (touch the microphone icon to make the foreign language reverberate to the device side), a command to send the foreign language text back to the translation AI do.

The third function is to save labor in creating a presentation, so that the "original input text" in the native language can be recovered and re-edited even after the AI translation is performed.
In the direct translation to the other party (foreigner) by this implementation method, the main purpose is to complete the editing / correction work and send the "correct native language" to the AI translation, and then re-edit. There is no.
However, in the case of the presentation creation work described in the [0117] column, the room for creating a finished product with higher accuracy is functionally added by the translation result verification work in the previous section via the translation AI. Will be.
In addition, the accuracy of the current translation AI has improved, and unless it is a long sentence, it is rare that there are multiple inappropriate parts, and it is estimated that in most situations you want to correct only "one place you care about". Will be done.
When recreating a presentation, it is common for the operator to think of "another way of saying the same meaning" for the part that he / she wants to correct, and to "redo the voice input from the beginning."
In order to save labor, a touch area of "re-edit" is set on the screen displaying the text retranslated into Japanese text by the reverse translation in the previous section.
When the touch area of "Re-edit" is selected, the edit / correction screen of the native language text content that was originally input by voice before being transmitted to AI translation is reproduced in the voice input Lock state.
As a result, re-editing work can be started immediately only for the part to be corrected, which can save a lot of labor especially when continuously creating presentation contents.

The fourth function is a top-bottom reversal display in which the screen is flipped upside down on the assumption that the other party (for example, a foreigner) is located facing each other.
Since translators mainly deal with one-on-one, in actual use, there are overwhelmingly more situations where they are lined up face-to-face than when they are lined up in parallel.
This also applies to public institution counters and international airport counters.
When a foreigner wants to see the text of the translation result, or when he / she wants to touch the punctuation marks by himself / herself, it is troublesome to turn over the device each time, and while displaying the translation result or pressing the foreign language input button from the beginning, etc. Then, if a specification that allows the screen display to be set upside down can be selected, it is advantageous for visibility to the other party.
Therefore, it is desirable to implement an instruction part that executes a process of reversing the display contents on the setting screen and a program that executes this.
During the top-bottom reversal, the operator (owner side) may want to reconfirm the content (translation result), so a touch icon that temporarily cancels the top-bottom reversal and returns it to the normal position appears on the screen. Make it appear.

The fifth function is to use the device as a translator for creating the text of the SNS transmission function for foreigners.
A portable translator with a camera already exists as a known technology, but the result of converting the native language and foreign language into text as a set is not used as an ad-lib translation for a foreigner in front of you, and an unspecified number of SNS For transmission, for example, translation results of English and Chinese are automatically set and output in the Japanese text, so that it is used as a means of creating the original text such as SNS for PR advertisement of stores for inbound tourists.
In particular, if the machine is equipped with a camera, it is possible to directly transmit SNS, and even if the translator does not have a camera-equipped function, if only the text is transferred to the home computer or smartphone, the foreign country translated into the existing SNS transmission device as the native language. It is possible to attach a sentence with a set of words without installing a special function on the translator side.

The sixth function is the process of fetching esoteric words from a computer.
As described in the previous section, by connecting the portable translator to a computer or the like, it is possible to solve a text problem in which it is difficult to obtain a result by voice input.
For example, if there is a fictitious disease name of "congenital stasis dermatitis type II", even if correction, editing, movement, etc. are repeated, this result can be obtained only by voice input without a keyboard. It's not impossible, but it's very time-consuming.
Therefore, if the esoteric words entered earlier with the keyboard are sent from the computer side to the word dictionary (save BOX, etc.) on the mobile translation machine side and saved, the translator punctuation "word" can be touched on the screen. In voice input, since prediction candidates can be output only by inputting from "congenital ...", it is possible to achieve the purpose of text output in a voice input device for ultra-difficult words.

The seventh function is to set a "typewriter mode" that changes the properties of the device to a "text-only microphone with voice input" that does not translate when the voice input buttons of the native language and foreign language are pressed at the same time. Is.
This means that it is not operated as a translator, but only the "voice recognition function" of a portable translator or smartphone is taken out, and a computer that does not have a function corresponding to voice input is given the ability as a retrofit voice input microphone.
As a method of operating the device, by pressing and holding the native language and foreign language buttons at the same time, the device is automatically determined to start the mode.
In that case, touch punctuation is displayed on a native language basis.
Basically, it becomes continuous recognition, and with some exceptions such as the next section, the mode is continued unless it is reset by pressing and holding the C button.
As a method of substituting a smartphone for a voice input device, there is a remote mouse (registered trademark) of a known technology, but this is basically "unspoken", and correction / editing should be done with a computer screen and keyboard. It is a major premise.
In the present invention, it is possible to perform erroneous recognition, correction of homonyms, editing of typefaces, etc. in advance at the time of voice input work, and it is possible to send a somewhat prepared "completed text" to a computer. It has an advantage especially when working on the go (when not connected to a computer).
The created text is sent to your home computer via a network such as wired or wireless, or the Internet, proofread (adjusting the position of the cue, changing the font size, etc.), and then printed out. And you can get the finished product.
In the typewriter mode, there is no restriction associated with "sending to the translation AI" by releasing the voice input button (or releasing the Lock touch) described in the [0112] column.
Therefore, it is possible to release the voice input button once, fix the text, and then input the voice again. Therefore, the method of "returning to the initial punctuation point" explained in the [0112] column and the "to" explained in the [0102] column. In the method of obtaining "lemon", it seems that it is common to use a labor-saving method similar to that of a computer in which voice input is interrupted once, text is fixed, and then voice input is restarted after each priority touch.

The eighth function is to jump to each setting screen or a specific part of the presentation when a specific voice input (for example, "recognizing screen" or "presentation, 3 of A" is input) during this typewriter mode. It is also to have a recognition function as "instruction to the main body".
In the case of long-sentence voice input, it is rare that the long-sentence content overlaps with the "instruction keyword that speaks relatively slowly and clearly in words", so the device side ignores it and simply converts the long-sentence into text.
If you make a voice with the intention of giving an instruction to the main unit and it is erroneously recognized, you can correct or edit it by pressing the native language / foreign language button at the same time and noticing the silence for a certain period of time. The transition to the screen is the same as the normal transition of editing / modifying by silence for a certain period of time described in the [0091] column in which one of the buttons is held down.
If there is a jump function to the specified position of this presentation, it is possible to divide the content of the presentation according to the question content and type (habit in the case of sports, etc.) on the foreigner side.
After recognizing the jump to the setting screen or recognizing the command for the position designation jump of the presentation, the screen shifts to the screen waiting for replay of the presentation to achieve the purpose, and the typewriter mode itself is automatically canceled.

The ninth function is the application of the typewriter mode, which is used as a support tool for writing.
If you become deaf (caused by deafness, accident, stroke, phytotoxicity, etc.) after reaching a certain age, it is difficult to learn sign language later, so we are currently mediating written conversation. Will depend on volunteers to write summaries.
However, if you use this "typewriter mode", if you follow the rule that two people do not speak at the same time, the conversation will be transcribed endlessly, so it is also effective as a support tool for people with this disability.

The portable voice translator described (after column [0077]) does not necessarily have to be a dedicated machine, and may be constructed by introducing a translation application into a smartphone.
In such a translator, the basic number of screen divisions and the arrangement of punctuation touches, etc. are all the screen before recognition (see column [0080]) and the screen during recognition (see column [0085]) of the portable translator, and editing / editing. Each figure on the correction screen (see [0088] column) and the default (initial value) are the same.
Similarly, the content of the punctuation touch is customized by the method described in the above [0046] column, and as a result, an additional screen of "returning to the initial punctuation mark" is generated as described in the [0112] column.
However, unlike smartphones, smartphones do not have "three physical buttons", so a touch area corresponding to each button is added to the bottom of the punctuation touch area, resulting in "initial value divided into 15". Become.
Even if the screen is changed to the edit / correction screen during voice input, the three touch areas corresponding to the buttons do not change at all.
In other words, the virtual state where the native language button, foreign language button, and C button exist in between is always maintained, and by making sure that there is no difference in the operation method, you can own both devices, or you can own a smartphone from a dedicated device. This is also to prevent the need to re-learn the operation method when switching the translation device to the download version.

You can save "punctuation touch" by applying the method described in the above [0998] column, which obtains "all hiragana" by speaking "slowly, clearly, one word" while pressing the conversion touch (or C button 142). The method of converting is explained.
Not limited to the conversion touch (or C button), if you perform voice input while pressing and holding the "starting point touch" of punctuation marks before you start saying, the moment you release the starting point touch of the parentheses The end point (closed side) of parentheses is automatically added to.
This makes it possible to save labor by reducing the total number of touches.
In addition, such closing of parentheses and the determination of hiragana on the device side such as voice input while touching the conversion can be performed in duplicate.
An example sentence and an operation explanation are shown below.
As a precondition, it is assumed that when "cherry" is input by voice without designation, the device side selects "cherry" in katakana.
Hereinafter, FIG. 26 will be described.
The item in line A in FIG. 26 describes "touching the screen".
The item on line B indicates the voice input (voice of the operator) itself.
The item on line C indicates the (desired) text that will be generated.
The horizontal axis in FIG. 26 indicates a time series (timing).
Therefore, in FIG. 26, the text of the C line can be created by performing the operation of the A line and the pronunciation of the B line in chronological order from the left to the right of the drawing.

In the description in the [0055] column regarding the computer, the details of the editing / correction in the computer for which the explanation has been suspended will be described.
Various editing / corrections that do not depend on the keyboard in the present invention are not limited to the portable translator, but can be performed on a computer as well.
However, the fact that the computer does not have the restrictions described in the [0112] column of the portable translator is the same as the typewriter mode described in the [0123] column, and the position can be specified even if the font is small and the character spacing is narrow. Since there is a mouse that can do it, it is most realistic and hassle-free to interrupt the voice input once and perform the correction operation with the keyboard without performing a special operation method in most cases.
However, when "I want to insert a long sentence", there is an advantage in additional processing by voice input.

Also, in the case of creating a long sentence, it is not always possible to complete the text while proofreading the entire text by voice input in order, and it is a normal practice to look back at the whole text from the beginning and proofread it later.
Furthermore, if the email being created is temporarily saved, the format is naturally recorded in the email application format (small font, narrow character spacing).
If you want to overwrite such a small font style by voice input later, in the case of a computer, if you apply voice input to the object whose range is specified (color tone inversion) with the mouse, "overwrite", after specifying one point with the mouse When voice input is applied with, it is "insert", and there is no difference from the correction or insertion operation with the keyboard as defined in the above-mentioned [0042] and [0052] columns.

The typewriter mode voice input method described in the [0123] column can be used as it is for searching the Internet on a smartphone, inputting text into an inquiry column on a corporate homepage, or the like.
However, unlike the case of the computer in the previous section, the mouse does not exist on the machine for fixing to the mail application or company inquiry page once, or temporarily saving the mail and modifying the text whose format has changed. Unless the pinch-in / pinch-out operation, which is the basic operation of a smartphone, is considerably enlarged, the range specification touch with a finger cannot be performed in terms of visibility and position identification recognition on the device side.
As a specific operation method, an example of inputting with a smartphone in the "inquiry field" of the corporate homepage on the Internet will be described.
Since there is no need for translation, input is started in the typewriter mode described in the [0123] column.

Due to the size of the explanatory diagram, the display of the virtual native language button, the foreign language button, and the fixed touch area of the C button in the description in the [0126] column is omitted.
As shown in FIG. 27 (A), @ (add mark) and the like can also be obtained by inputting "Admark" by voice input after touching the touch punctuation mark or "symbol".
As shown in FIG. 27 (B), the e-mail address corresponds to uppercase and lowercase letters using alphabetic priority and Caps Lock, but since it is complicated to input each time, it is registered in the save Box by the method described in the [0122] column. , You can save labor by putting it in "Word registration" in advance.
As shown in FIG. 27 (C), by making full use of the above-mentioned method, temporary proofreading is completed in the edit / correction screen as much as possible, and the contents to be input are piled up.
As shown in FIG. 27 (D), the text once laid out on the homepage side to be input is considerably enlarged in the smartphone screen by the pinch-in / out operation, and the position designation touch is performed with the finger.
As shown in FIG. 27 (E), touching with a finger when the voice input button is not pressed (not waiting for voice input and no touch punctuation marks appear) is synonymous with mouse operation, and range specification (color tone inversion). (Waiting for re-entry or re-conversion), and in the example, specify the range of "Nice to meet you".
In this case, the definition of "returning to hiragana once" does not apply after specifying the starting point and ending point described in the [0091] column.
This is because the voice input has been completed and fixed, and the format has already changed to the corporate homepage side (software side, etc.), and the voice input is repeated again and the device side misrecognizes "Nice to meet you". Before fixing, change it to "Nice to meet you" with the virtual C button or conversion touch, and then fix it again.
As shown in FIG. 27 (F), after one point is specified (insertion position is specified), a new phrase is inserted by voice input.
As a tentative example, suppose you want to input a new text such as "or preventive method" between "how to deal with" and "teach".
In this case, since it is a "new voice input", unlike FIG. 27 (E), as described in the [0091] column, the "or preventive method" part has room for transitioning to the edit / correction screen.
In this way, even with a device that does not have a keyboard, when inputting a long sentence, it is possible to create a completed sentence after "proofreading work throughout the entire sentence after provisional completion".

In addition, in the typewriter mode, a method of inputting special typefaces such as bold / italic characters and underlines (underscores) by voice input will be described.
Hold down the C button (or conversion touch), slowly and clearly release the voice input for only one phrase, and release the C button (or conversion touch). As explained above, in each priority recognition touch area such as letters, numbers, and katakana on the setting screen, "speech recognition during long press is a touch of the role of adjusting the typeface" is separately set.
For example, it is synonymous with the situation where a bold or underlined icon is clicked with the mouse on a computer and then text input is started with the keyboard.
Furthermore, since the act of releasing the touch also serves as the operation of canceling the format change, it is possible to intermittently interrupt the computer faster than the computer on which the mouse exists.
If an erroneous recognition occurs while inputting this special typeface, the special typeface is maintained even if the erroneous text once fixed by releasing the voice input button is rephrased or reconverted.
It means that the device temporarily records the typeface at the end of the position specification operation for which the typeface is to be corrected, which is also the same as when the keyboard is input on the current computer.
An example sentence and an operation explanation are shown below.
As shown in FIGS. 28 (A) to 28 (D) in advance, there are several types of underlines (underlines) for number priority (speech recognition while pressing), eg, "all (select later)", "one underline", " Set "double underline", "dot underline", "wave underline", and "disappearing line".
Similarly, as shown in FIGS. 29 (A) to 29 (C), there are several types of typefaces for alphabetic priority (speech recognition while pressing), eg, "all (select later)", "bold", "italic", " Set "blank character", "ruby", etc.
Example sentence (Purpose format) To underline "Voice input device" and make "Typeface" bold in "Voice input device can also change typeface (* bold)", operate as follows (Fig. 30). See (A)).
While uttering the operation method, "Onsei Nyuroku Sochi", hold down the number priority touch.
FIG. 28 (D) shows a screen when a temporary jump is made to the typeface selection screen at the moment when the number priority touch is released at the same time as the utterance of “chi” in the pronunciation of the “device”.
Next, hold down the alphabetic priority touch while uttering "shotai".
FIG. 29 (C) shows a screen when a temporary jump is made to the typeface selection screen at the moment when the number priority touch is released at the same time as the utterance of "body: i".

The format change setting screen in the typewriter mode will be described with reference to FIG.
FIG. 28A shows a state in which the touch area of the “typeface” is selected.
FIG. 28B shows a situation in which the touch area of “priority recognition” for which the typeface is to be set is preset.
FIG. 28C shows options such as typefaces that can be selected.
Since there may be a situation where the touch is mistaken, the C button can be substituted as (cancel) even when the "back one" touch area is not on the screen.
FIG. 28D shows the underlined and other options that can be selected.
"All" is selected as the initial value, and the jump screen when "the priority touch area for typeface change is released at the same time as voice input" (the range specification for typeface change is completed) described in the [0132] column is displayed. , The screen is the screen excluding "all" from the screen.
FIG. 29A shows a situation in which another typeface change command is set in the touch area of “alphabet priority”.
FIG. 29B shows a state in which the “decorative character” is selected.
FIG. 29 (C) shows the options of the “decorative character”, and similarly to FIG. 28 (D), the initial value is “all”, and the range is specified and then “selected later”.
FIG. 29 (D) shows a state in which the two priority recognitions are applied as (underlined) and (decorative character) formatting commands.
Although it is colored for explanation, there is no change in color tone after setting.

Furthermore, if only one typeface is registered for each priority recognition touch, the options in FIGS. 28 (D) and 29 (C) in the previous section do not appear, and the special typeface is executed the moment the priority recognition touch is released. NS.
As an application example, in the case of only one typeface designation touch, if voice input is performed while pressing a plurality of typeface designation touches at the same time and both touches are released at the same time, special typefaces are applied in duplicate.
This is the same as the command for simultaneously multiplying the full hiragana and "" (closing the hooks in parentheses) described in the above column [0127].
Hereinafter, an example sentence and an operation explanation will be shown with reference to FIG. 30 (A).
Set only one type of "double underline" for number priority (voice input during judgment while pressing).
Set only one type of "italic" for alphabetic priority (voice input during judgment while pressing).
In the case of such a single setting, the screen transition of "Which format should be selected?" Does not occur at the moment when the priority recognition is released as in the previous section, and the format change is applied at the moment when the priority recognition is released.
Furthermore, when a special typeface is set in the area where the vertical writing of the typeface and the writing brush of the decorative character are prioritized in numbers and alphabetic characters, an extreme example as shown in Fig. 30 (B) ("Congratulations all staff" in vertical writing on the envelope, etc. ) Is also theoretically possible.
The definitions of lines A, B, and C in FIG. 30 are the same as those described in column [0127].
It should be noted that such a format change setting is exclusively for the typewriter mode described in the [0123] column in principle, because sending a special typeface to the translation AI may cause an error.

As one of the known techniques, the device recognizes the description field in which the scanned format (for example, history) is read by OCR (optical character recognition: Optical Character Recognition, the same applies hereinafter) as a "frame", and the frame is recognized by the keyboard. There is a method of entering text inside.
As a result, it is possible to create a finished product with printed text in the "originally handwritten part".
As described in the [0042] column relating to the computer in the present invention, there is no difference between the one in which text input is performed by the keyboard and the one in which voice input is performed, and the recognition from the input target software side is the same.
Therefore, by applying the connection with the computer and the typewriter mode described in the [0121] to [0123] columns, it is possible to input text by voice input in the above-mentioned "within the frame of the format such as resume". Become.

As a result, there is no keyboard and mouse, and text input is only uttered in the text, and punctuation marks, line breaks, blanks, corrections (rephrasing), reconversion, movement, deletion, format change, and formatting are performed by touching the small screen. Approximately all "correction, editing, typeface change, etc." such as input to a predetermined format becomes possible.
In the spread of the present invention, although it depends on the proficiency level of editing / correction by the operator's voice input and the original typing ability, not only a tablet terminal having a simple keyboard and a touch pad, but also a full-size keyboard and a mouse It will be possible to create texts faster than the computer on which the mouse exists, in other words, it will be possible to create a manuscript of a feature-length novel on the go with just one smartphone.
As of 2020, there are more than 200 million people, especially foreigners, who have smartphones because of their necessity, but do not have personal computers or tablet terminals, that is, they only have so-called mobile devices. People in such situations will have the opportunity to express themselves in their writing activities.
In addition, even if it is attached data in an email instead of paper, it is possible to present a well-formatted resume to a company in a situation where the fixation of disparities is becoming a social problem worldwide. It can also be expected to contribute to improving employment conditions.

So far, we have explained each method of editing and correction by just touching the screen in "portable devices without a mouse and keyboard" such as mobile translators and smartphones.
As for the computer, as described in the [0042] column, it seems that it is common to basically specify the coordinates of the corrected part with the mouse and overwrite it with the input of the keyboard.
However, even in the environment of computer use, the situation of communicating with foreigners by mediating foreign language translation AI will increase more and more in the future in accordance with the trend of globalization.
For example, there are video conferences with overseas offices, and if you don't need video, you can directly respond to friends who you met on a personal product transaction or overseas trip by e-mail in real time.
In particular, the situation where an interpreter can be requested at the individual level is not realistic, and naturally depends on the general translation AI on the Internet.
In this case, the restriction "The voice input button cannot be released and is completed in the edit / correction screen" described in the [0112] column also occurs in the computer.
Therefore, even in the edit / correction screen due to the temporary silence from voice input (see column [0055]), correct misrecognition and synonyms of the same sound while touching the computer monitor, send it to the foreigner, and send it to the other party. Will obtain text in a foreign language as character data via AI translation, and will also obtain voice information in its own language from its own computer if necessary.
In addition to such ad-lib exchanges, if you repeat the same explanation over and over, such as product transactions, it will be more convenient if you can pull out the "fixed phrase" each time. It is desirable that it can be added to the side as an additional function.
As described above, the present invention has an applicability that can contribute to "direct communication with a distant foreigner via the Internet" in addition to the portable translator for direct response.

"Third embodiment: a wristwatch-sized mobile terminal such as a smart watch"
Mobile terminals, especially wristwatch-sized mobile terminals such as smart watches, can be used to input specific keywords such as "weather forecast" and "alarm after 15 minutes" in addition to their own health status (monitoring vitals, etc.). Those that perform information processing are still provided.
Therefore, first, the superiority of configuring the voice input device with the smart watch in the present invention will be described.
In the current smart watch, remote setting is possible by synchronizing with an information processing device such as a smartphone, and in the present embodiment, configurations other than the synchronization setting will be mainly described.
As a known technique, touching a target icon on a small screen of a smart watch (hereinafter referred to as "board surface") or calling various functions by voice input exists as described above.
However, the "giving accuracy" in the present invention is only a merit, and there is no demerit because it does not interfere with the basic function of the current smart watch in terms of operation method.

Smartwatches have the ultimate portability due to their size and band fixing method, but the board is extremely small and it is difficult to obtain visual information.
For example, it is possible to purchase products on the Internet, but it is not particularly suitable for watching videos.
Therefore, in the present invention, as described in the previous section, an advantage is found in that even a short keyword can be "correctly input while being corrected".
For example, it is effective for information retrieval sites provided on the Internet, news searches on the Internet, and the like.
However, even if the target Web page is displayed with the targeted search keyword, it is still difficult to read the displayed text because the board surface is small as described in the previous section.
Therefore, by using OCR recognition, which is a well-known technology, and the voice output that accompanies it, the text of dictionaries / papers and net news on the Internet can be read aloud to the smartwatch side by pressing the voice output button. be.
That is, the Internet search is performed based on the voice input, and the operator can obtain the information necessary for pinpointing by the voice output on the device side.
The specific method will be described later in the column [0152].

Further, as a method of using the voice input device according to the present embodiment, it can be effectively used in an SNS mainly for exchanging information by short message.
With a smartwatch, the user does not notice the reception because it is worn by the user, and there is no need to worry too much about the typeface of the reply.
The punctuation marks used for voice input can be changed to a unique SNS stamp on the setting screen, and the manifestation of "Like" can be done quickly with a one-touch operation on the board.

In addition, although credit payment by mobile devices such as smartphones and associated smart watches is a known technology, it is a form of contracting after downloading software, and is limited to a specific payment service.
The voice input device according to the present embodiment displays the paid QR code (registered trademark) on the call board surface by accurate voice input each time in concerts, paid sports watching, ticket gates, and the like. It can be used for various types of gate passage authentication.
You can also use it when it is difficult to take out your smartphone from the work environment, or when one hand is occupied, you can change your luggage and answer the phone.
In this way, it is effective for sudden phone calls, performing device operations by voice input, accurate search, and relatively accurate short mail creation.

Rather, it is difficult to accurately create a long sentence from the viewpoint of visibility as described in column [0139], although the basic operation and processing are the same as those of a portable translator if it takes time and effort.
For long sentences, it is temporarily recorded as a draft created on the go, sent to a computer or smartphone via a network, etc., and editing / correction and final proofreading can be done later on a screen that is the size of a computer or at least a smartphone. It is more efficient to do it with a device that has.

FIG. 31 is a schematic diagram showing a voice input device using a wristwatch-sized mobile terminal (especially an embodiment in a round bezel type wristwatch that emphasizes natural design), and is a schematic diagram showing a smart watch according to the present embodiment. be.
In the mobile terminal according to this embodiment, when the voice input button is pressed and no sound is generated, the state of the pre-recognition screen shown in FIG. 31 (A) is obtained.
In particular, in the case of a round bezel type wristwatch, it is desirable to have a layout in which a blank part such as a virtual square or rectangle is provided in the center of the dial in order to secure the type visible part of the text as much as possible.
However, the solid line partitioning the dial area shown in FIG. 31A is for explaining the division of the area, and the boundary line is color-coded in the actual product.

Many of the round bezel-shaped watches shown in this embodiment have buttons at four corners due to their configuration, and even in this embodiment, a wristwatch having buttons at four corners (in the current general square smartwatch). There are models with buttons).
Unlike the above-mentioned operation / processing method for smartphone-sized mobile terminals, the processing and roles to be executed are shared by screen touches and buttons in order to secure as many touch areas as possible.
Since the process of pressing a button to execute a command also differs between the pre-recognition screen, the recognition-progress screen, and the edit / modify screen, a small display area 311 for "explanation of button operation" is provided on the board.
In order to execute the operation / process of "back one", the batten "x" 312 is displayed in a transparent color in the center.
If you want to re-enter from the beginning, such as when you make a mistake in punctuation, press and hold while touching the screen to "clear all resets and return to the initial screen (clock state in the case of a smartwatch)" is related to mobile translators. ] It is the same as the definition in the column.
The layout of the buttons and touches will be described in the order of upper left → upper right → lower left → lower right of the buttons, and then in the order of screen touch top → left → right → bottom.
Hereinafter, the same applies to the description of the recognition screen and the editing / correcting screen.
As an example of the initial arrangement, "screen enlargement", "voice input lock", "screen reduction", "voice input", "number priority", "alphabet priority", "symbol", and "quick color change" are set.

The screen transition of the method of customizing the punctuation touch area will be described.
In addition to accepting it as a command even if you say "Onsei Nyuryokusete" by voice input, you can press and hold one of the touch points to enter the setting screen as described in the [882] column of the mobile translator. It is also possible to arrange the touch area of the jump in the same area.
The most frequently used operation method is the "shortcut input method" that jumps to a specific setting screen by flicking the screen up / down / left / right without making a single voice while pressing the voice input button.
As shown in FIG. 32 (A), flicking 321 to the left jumps to the "system setting screen", and flicking 322 to the right jumps to the "color tone setting screen".
As shown in FIG. 32 (B), flicking up twice to change the arrangement of the "pre-recognition screen 325a", flicking up once to change the arrangement of "recognizing screen 325b", and flicking down once to change the arrangement of "screen 325a". Change the layout of the "edit / correction screen 325c" and flick down twice to jump to the setting screen of the "loupe 325d" described later.
FIG. 32C shows the layout of the color tone change screen as an example. A description of the button can be displayed near each button.

When voice recognition starts for one or more characters, the screen automatically transitions to the recognition screen shown in FIG. 31 (B), which is the same as the description in column [805] for a smartphone-sized mobile terminal, but there are few options such as punctuation marks. ..
The items in the auxiliary input display section shown in FIG. 31 (B) are just examples of initial values, and the item punctuation marks in the auxiliary input display section can be customized on the setting screen. The need for customization will be more important than that.
Examples of initial placement are "slide on screen", "symbol (including stamp)", "slide down screen", "voice input", ".", ",""Blank","linefeed", and x in the center (back one). Is set.
If the screen size is about the size of a smartphone, the screen position slide function will not appear unless the text shifts to the "edit / correction screen" other than the long text and the text does not fit on the screen. Because of its small size, the "slide vertical bar 313 at the screen display position" is present from the beginning at the start of recognition of one or more characters.

Then, when the operator finds an error in voice input, the silence is maintained for a certain period of time, so that the correction shown in FIG. Transition to the edit screen.
The area or button that specifies the "back one" process is the touch area or any of the four corners from the center of the board so as not to interfere with the screen touch operation that specifies the position of the start point, end point, etc. of the entered text. Move to the button.
Further, the board surface is provided with a "slide vertical bar 313 at the screen display position" for sliding the display area.
Although it is often used due to the small size of the board itself, the "slide vertical bar 313 at the screen display position" is displayed in a semi-transparent color on top of the text in order to secure the board display area as much as possible.
The operation of the "vertical bar 313" itself is different from the description in the [089] column regarding the smartphone-sized mobile terminal, and the forward / backward operation is performed on the button sides 314 and 315 of the watch body.
This is because when the vertical bar movement is operated by touching the screen, it is easy to mistake it as a touch for specifying the position of the start point / end point of the text because the board surface is small.
As an example of the initial arrangement, "slide up screen", "back one", "slide down screen", "voice input", ".", ",", "Conversion", and "?" Are set.

Frequently used functions and processes will be described with reference to the layout example shown in FIG. 33.
Such a call operation of the voice input screen can be performed by voice input by inputting words such as "Esuenuesu" and "Denwa".
As shown in FIG. 33 (A), in the SNS for transmitting and receiving short messages, a "stamp" or the like may be arranged instead of punctuation marks.
This greatly simplifies the time and effort required to enter pictograms and marks.
Further, calling with a smart watch is a generally known technique, and an example of dial display by touching the screen will be described with reference to FIG. 33 (B).
Basically, it seems that it is common to exchange within the pre-registered phone book data, but by applying the touch part of the screen division, it is possible to directly enter the phone number other than voice input. be.
Multiple numbers are assigned to one button or touch area. For example, if you press the upper right button once and release it immediately, "4" is selected, and if you press the button twice in a row, "5" is selected. ..
For the smallest round bezel wristwatch size, the face (touch panel, which is the dial) is divided into four parts.
Depending on the shape and screen size of the smartwatch (square smartwatch, etc.), the touch panel can be configured as a touch area divided into 6 to 9 parts.

The method of inputting the search key on the Internet will be described.
Currently, in various major search engines (search sites), the basic setting is that the cursor position is assigned to the search key input part of the top page at startup from the beginning, so the display of the search engine is displayed. The shortest voice input and search method is to continue to speak the content you want to search by voice input.
In the Internet search, when the Web browser is started, the cursor cursor is blinking in the search bar (search key input area) on the top page, so that the search key is input by voice as it is.
For example, you can directly switch to the search by inputting voice only for the purpose such as "Ohhhhhhhhh".
However, since the board surface is small, even if voice input is performed at the target search key location, an error occurs in the content of the search key (recognition result on the device side of voice input) in the size of the entire search key screen shown in FIG. 34 (A). I can't check if it's not.
This is because the screen layout of the target Web page or the like is configured on the assumption that the first half of the entire page including the search key is viewed on a horizontally long rectangular computer monitor.
Therefore, normally, it is not possible to confirm whether or not an error has occurred unless the contents of the search key are enlarged and displayed by operating the device side up to the display size shown in FIG. 34 (B).
Here, if the method described in the [0055] column of the present invention is applied to intentionally create a silent state without releasing the voice input button, the Web page (in the example, a fictitious search engine) side is light-colored and has a background. As shown in FIG. 35, the text side being input is enlarged and becomes the center of visual recognition, so that the content of the search key input by voice can be checked without any extra operation.
If there is no problem with the content, the search key is entered just by releasing the voice input button, and the overall image of the target Web page (corresponding to Fig. 36, for example, the relevant page of the "Net Fictitious Dictionary") screen is obtained as the search result. Will be.

However, smartwatches basically have a small screen size and are inferior in visibility in exchange for excellent portability.
Therefore, in order to eliminate the demerit of low visibility, a voice output function by OCR recognition is added.
However, contents that cannot be transmitted by voice, for example, even if the weather forecast is called from the Internet, "sunny" and "rain" are not texts, but are indicated by "sunny mark / rain mark" in most cases, and this voice output. Is not available, so there are situations where you have to rely on visual information, even temporarily.
Therefore, the "loupe (magnifying glass) mode" is set as a function that is activated by voice input dedicated to the smart watch.

That is, after selecting the voice input button, the mobile terminal is associated with activating the function corresponding to the recognized voice by inputting the voice specifying the function to be activated.
For example, if the function to be activated is (here, in particular, "loupe (meaning magnifying glass or magnifying glass) mode"), the voice of "looper mode" is emitted after the voice input button is selected.
When a mobile terminal such as a smart watch recognizes this utterance, the screen shown on the display screen of the loupe mode shown in FIG. 37 appears.
In the display screen of this loupe mode, the screen of an arbitrary area in the information display screen on the Internet or the like is enlarged and displayed.
The operation of moving the display area is performed by touching the loupe (magnifying glass) portion 371, which is pseudo-expressed in a semi-transparent color, on the surface of the smart watch.
For example, by sliding the area outside the loupe display area 372 (that is, the background), the display area inside the loupe can be moved quickly, the frame 371 of the loupe can be selected, and the outer frame of the selected loupe can be moved vertically, horizontally, and diagonally. When an operation such as pressing and tilting is performed, fine adjustment that moves slowly and two screen movement methods are combined for positioning.
Although it is in a virtual state, it is made easier for the operator to understand by imitating the natural movement when a magnifying glass or the like is actually placed on paper.
The double display in FIG. 37 is that the outside of the loupe is the majority of the Web page screen, but even if the characters are too fine to read, it is possible to intuitively know which part of the screen the loupe is in. This is to let the operator understand.

Further, it is desirable that the smart watch according to the present embodiment performs the audio output processing described in the above [0139] column.
However, with respect to the conventional audio output, if the audio is output from the beginning of the Web page without specifying the range, there is a great deal of waste up to the text.
The range 381 surrounded by a cross in FIG. 38, that is, a title, a cautionary note, an advertisement in a realistic Internet environment, and the like.
Therefore, by designating the start point of the audio output in the displayed screen, the audio is output from the specified location.
That is, the starting point is the position in the loupe where the starting point is held down (finger is stationary), and after the starting point operation, the explanation of the button that was previously "voice input" changes to "voice output button". ..
Then, when the "voice output" button is pressed, OCR recognition is performed from the displayed screen, and the device AI converts the recognized text into text and outputs the voice.
In other words, the moment the blank part 382 immediately before the "purpose" in FIG. 38 is pressed and held and the starting point in the loupe is determined, there is no need to search by voice input anymore, which is synonymous with reaching the target information. Yes, the "voice input button" at the lower right of the four corner buttons means that there is no problem even if the property changes to the "start button for voice output read from the relevant part".
When you have finished listening to the required information, select the "Back one" button to interrupt the audio output (press and hold the "Back one" button to return to the initial screen such as the clock display).
Further, after designating the starting point, the user can move further in the loupe mode and specify the position 383 again to "designate the end point", and "specify the range" when the audio output is started can be performed in advance.
When the range is specified, the content that has been listened to once is repeatedly played back by pressing the audio output button again.
For symbols that cannot be identified by the device side by OCR recognition (for example, a weather forecast mark), the voice output is skipped and the voice output of the next character text is continued.

Since the target of the screen display is a website on the Internet, the font and character spacing cannot be enlarged like the text on the edit / correction screen of a mobile translator (see [0088] column), so the positions of the start and end points. The specification by touching the board is not accurate, but the problem is that the part to be read out is only a number, at most one line off, and only the "voice output of the reading result" that does not require accuracy like translation. There are few cases of becoming.
In addition, by "moving the loupe slowly after enlarging an arbitrary area", even if audio output cannot be obtained, it is possible to grasp the contents of the weekly weather forecast mark as described in the [0150] column. Is considered to have no problem.

As described above, when a web page on the Internet is called by voice input, the loupe mode is often activated as a result in operation, so even if you do not input "looper mode" by voice each time, the loupe takes a certain amount of time. It is desirable to configure the mode to start automatically.
On the setting screen described in the [0145] column, it is possible to set the time for automatic startup, the inability to start automatically, and the like.
The longer the setting time until the automatic startup is executed, the longer it takes for the operator to "understand the entire layout" of the called Internet screen.
In the case where only familiar Web pages are called, it seems that it is common to set "start the loupe mode from the beginning, that is, wait time 0 seconds".
Since the automatic appearance of the loupe may hinder visibility (for example, when you want to see only the entire image or photo even on a small screen), the "lower left assuming the grip of the loupe" in the loupe silhouette that appeared. By touching the "bar-shaped part protruding from" and flicking it to the right, the displayed "loupe deletion operation" is set as the "operation to remove the loupe from the paper as an image".
If you want to check the details again after grasping the whole picture, you can re-appear the loupe by inputting "Rupemode" again.

In addition, the background color of Web pages on the Internet changes depending on the type of major search engine and the image color of a company's homepage.
Therefore, as a countermeasure when the color tone of the loupe itself is accidentally similar, a quick color change is set for the color tone of the translucent loupe as well as the quick color change described in the [0035] column regarding the computer.
However, since it is difficult to set a dedicated touch icon due to the narrowness of the board, the command operation of "color tone change" is performed by repeatedly hitting the "virtual loupe grip" protruding from the lower left of the loupe with a finger.
Similar to the other examples, this rotation is used to return to pink → light blue → green → orange → pink.

Even if you can enter the correct search key on your smartwatch, if it is recognized that you need to correct it by voice input, the operation itself is compared with a computer with a keyboard or a smartphone with a relatively large screen size. The smartwatch is the inferior in terms of the size and visibility of the operation unit.
Therefore, in order to reduce the amount of operations such as re-searching the same Web page, the last application used last time or the software used frequently can be called directly by inputting the proper noun by voice. This is a known technique.
As another existing example, there are many methods of ignoring the design of the clock itself and sprinkling the icons of search engines and frequently used applications on the board.
Even if the accurate search key input in the present invention does not impair the design and can call an accurate Web page or the like while keeping the small screen, when the radio wave condition is not good, the target Web is naturally used. You cannot call the page.
In addition, you have to search the same Web page many times, such as when you look at map information on the Internet or when you look at a long paper, every time you approach your destination or read it.
As mentioned at the beginning of this section, the operation itself is difficult compared to large devices, and the device has excellent portability but is used more frequently in situations where radio waves are poor, so the same Web page is displayed many times. It is desirable that the smartwatch body has a function to trim the entire image of the Web page screen (1 page for papers, etc.) so that there is no need to access it.
The operation is to "Satsuei" or "Torimingu" after adjusting the position of the map information (temporarily the walking route from the last station to the destination) screen on the Web or a whole page of a long paper on the board. Is executed by voice input.
Depending on the memory capacity of the device, the map information is one from home to the first station to get on, then one from the station to the destination station, and finally one from the arrival station to the destination. If you save the paper or repeat the work one after another from the second page onward, you can record all the target information "from the beginning to the end".
When using it, by calling the "save BOX" by voice input, the content to be viewed is not affected by the radio wave condition, and the trouble of input operation for search can be obtained.
The definition of one page is a range synonymous with "the first page when a print instruction is given" on the computer for the Web page being browsed, and if it is FIG. 38, it is the range of "the entire figure".
If you want the device to record by specifying only a part of the range on the page, once the loupe appears, the entire range you want to record is stored in the loupe, and then "Satsuei" or "Torimin" When the voice command operation of "Gu" is performed, only the contents in the loupe are trimmed (recorded).

When the saved data in the previous section is recalled and later reconfirmed in the loupe mode, the "continue function" that is further effective in terms of function will be described.
With map information, oneself will approach the destination, and if it is a long paper, reading (or listening) will proceed.
Of course, if you keep the smartwatch running, the battery of the smartwatch will be consumed, so it is important to reconfirm the map, and reading (listening to) the paper is also an example, but the end of the break time and the sound around you It may be necessary to interrupt the situation, such as changing to a situation where it cannot be put out (an unrelated person comes nearby).
As explained in the previous section, even if you recall from the save BOX with a simple voice input command, if the position of the loupe is the starting position each time, it will be up to the current position if it is a map, or up to the position that you have read if it is a paper etc. Extra operation to advance the loupe becomes indispensable.
Therefore, when the loupe mode is applied to the saved image data, the "position and magnification of the loupe at that moment" is also overwritten and saved when the screen is closed (touch back one). ..
As a result, "the progress of the process for the operator" is automatically recorded, which improves convenience when the call is made again.
Even when a paper or the like is read aloud by OCR recognition, it can be restarted from the middle position of the automatic slide of the loupe in the next section with one touch of a voice input button, so that the amount of operation is small and the continuation can be heard immediately.

An example of audio output in ORC recognition will be described from pages such as papers on the Internet.
Although it is a virtual dissertation page, in most cases, as explained in column [0152], an advertisement column and a cautionary note are posted at the beginning, which hinders the reading of the text.
After calling the loupe mode or automatically appearing after a certain period of time, it seems that it is common to specify the position at the "beginning of the text for which information is desired" (position 382 in FIG. 38) by enlarging or adjusting the position.
When you press and hold the beginning of the text to specify the start point and press the voice input button, you can obtain the "voice output" for OCR recognition, which is the purpose, but the visual information in the loupe is not neglected.
The loupe itself automatically slides with the phrase being read as the center position in accordance with the reading of OCR recognition.
However, if the loupe is enlarged considerably to specify the position, the characters in the loupe will flow too quickly and the contents will not be visible.
Therefore, even during audio output and automatic loupe slide, the enlargement / reduction buttons on the smartwatch body accept the operation.
This is because if the enlargement ratio of the loupe is reduced appropriately, better understanding can be obtained and the positional relationship of the reading portion of the entire layout can be grasped while visually recognizing the character information synchronized with the voice.
If you press Cancel (back one) once at the position 384 in the middle of the reading area of FIG. 38 to pause the reading and then recall the data from the same save box next time, the loupe exists at the enlargement ratio and position of FIG. 38. That is to say, if the voice input button (that is, the restart of the voice output) is pressed in that state, the loupe slides in synchronization with the voice output from the position 385 in the middle of reading in FIG. 38.

The voice input device of the present invention can be used as a voice input device using various electronic devices, and can be used as a voice input device that can be executed particularly on an electronic device having a liquid crystal screen, preferably a touch panel. ..

10 Pre-recognition screen 12 Designated area 17 Touch area 20 Recognizing screen 50 Multi-function mouse 61 Selection area 70 Editing / correction screen 71 Vertical bar 130 Stepping touch pad 130 Pad 141 Foreign language input button 142 C button 143 Native language input button 153 Auxiliary Input display section 201 Correction text input screen 211 Start point 212 End point 325a Pre-recognition screen 325b Recognizing screen 325c Editing / correction screen 371 Loupe (magnifying glass) section 372 Loupe display area

Claims (6)

A voice input device that recognizes voice and converts it into character information.
A voice recognition unit that recognizes the input voice and
A character conversion unit that converts the voice recognized by the voice recognition unit into a character string, and
A character string display unit that displays the character string converted by the character conversion unit, and
An auxiliary input display unit that displays graphic characters consisting of at least punctuation marks, commas, parentheses, and functions, and / or control characters on the character string display unit.
A voice input device including a symbol insertion unit that is inserted into a character string converted by the character conversion unit by selecting a graphic character and / or a control character displayed on the auxiliary input display unit.
A range specification unit that specifies the range of the character string displayed on the character string display unit, and
The voice input device according to claim 1, further comprising a reconversion unit that reconverts a character string in a range specified by the range designation unit.
The auxiliary input display unit can select and display a pre-recognition screen including characters and symbols input at the beginning of a sentence and a recognizing screen excluding the characters and symbols input at the beginning of a sentence.
The voice input device according to claim 1 or 2, further comprising a screen changing means for changing the pre-recognition screen to the recognizing screen at the same time as voice recognition.
Furthermore, it has an editing correction section that edits or corrects the input character string.
The voice input device according to any one of claims 1 to 3, wherein the editing correction unit inserts another character by voice input into a designated range or a designated position of the input character.
Furthermore, an audio output unit that outputs a character string with a specified range as audio,
The voice input device according to any one of claims 1 to 4, further comprising an enlarged display unit that magnifies and displays an arbitrary area displayed on the display device.
A voice input method according to any one of claims 1 to 5, wherein input to an electronic medical record or computer programming is performed using the voice input device.

Images