JP2019071104A - Input display device - Google Patents

Abstract

To enable a character string indicating the result of speech recognition to be displayed in a user's free layout.SOLUTION: An operation state detection unit 11 detects the state of an operation accepted by an operation acceptance unit 1 that accepts a trace-writing operation. A locus image generation unit 12 determines the locus of a line drawn by tracing from the trace-writing operation accepted by the operation acceptance unit 11, and generates a locus image that indicates the determined locus as a locus image of the line width that corresponds to the state of operation detected by the operation state detection unit 1. A display control unit 13 displays the locus image generated by the locus image generation unit 12 on a display 29, and causes a character string acquired by a character string acquisition unit 5 that acquires a character string that indicates the result of speech recognition to be displayed, on top of the locus image displayed on the display 29, using characters of a character size that corresponds to the line width of the locus image that is set by the locus image generation unit 12.SELECTED DRAWING: Figure 9

Description

  The present invention relates to an input display device for displaying a character string indicating a speech recognition result on a display.

Patent Document 1 below discloses an input display device for displaying a character string indicating a speech recognition result on a display.
In this input display device, when the user performs an operation of drawing a line using a pen-type input device, the start point and the end point of the line are detected, and a straight line connecting the start point and the end point is drawn on the display.
Thereafter, when the user inputs a voice using a microphone, the voice recognition process is performed to display a character string indicating the voice recognition result along a drawn straight line.

Unexamined-Japanese-Patent No. 2012-88969 (FIG. 2, FIG. 3, FIG. 6)

  Since the conventional input display apparatus is comprised as mentioned above, if a user performs operation which draws a line using a pen-type input device, the straight line which ties the start point of the line and the end point will be drawn on a display. Therefore, before the user inputs a voice using the microphone, it is possible to grasp in advance the position where the character string indicating the recognition result of the voice is displayed. However, since the line drawn on the display is limited to the straight line connecting the start point and the end point, there is a problem that a character string indicating a speech recognition result can not be displayed with a free layout of the user.

  The present invention has been made to solve the above-mentioned problems, and it is an object of the present invention to obtain an input display apparatus capable of displaying various character strings indicating speech recognition results with a free layout of the user. Do.

  The input display device according to the present invention includes an operation receiving unit for receiving a tracing operation, an operation state detecting unit for detecting a state of the operation received by the operation receiving unit, and a tracing operation received by the operation receiving unit. A locus image generation unit that determines a locus of a line drawn by scratching, and generates a locus image indicating the determined locus as a locus image of a line width corresponding to the state of the operation detected by the operation state detection unit; A character string acquiring unit for acquiring a character string indicating a speech recognition result, and displaying on a display a locus image generated by the locus image generating unit, generating a locus image of the character string acquired by the character string acquiring unit A display control unit for displaying the characters of the character size corresponding to the line width of the trajectory image set by the unit so as to be superimposed on the trajectory image displayed on the display Than is.

  According to the present invention, the display control unit displays the character string acquired by the character string acquisition unit on the display with the character having the character size corresponding to the line width of the trajectory image set by the trajectory image generation unit. Since the image is displayed superimposed on the image, there is an effect that it is possible to variously display a character string indicating the speech recognition result by the user's free layout.

It is a block diagram which shows the input display apparatus by Embodiment 1 of this invention. It is a hardware block diagram of the input display apparatus by Embodiment 1 of this invention. It is a hardware block diagram in case an input display apparatus is comprised by computer. It is a flowchart which shows the input display method which is the processing content of the input display apparatus by Embodiment 1 of this invention (the 1). It is a flowchart which shows the input display method which is the processing content of the input display apparatus by Embodiment 1 of this invention (the 2). It is explanatory drawing which shows the character attribute table memorize | stored in character management DB. It is explanatory drawing which shows a display process when a locus | trajectory image is produced | generated before the character string which shows the recognition result of audio | voice is produced | generated. It is explanatory drawing which shows a display process when the character string which shows the recognition result of an audio | voice is produced | generated before a locus | trajectory image is produced | generated. It is a block diagram which shows the input display apparatus by Embodiment 2 of this invention. It is a hardware block diagram of the input display apparatus by Embodiment 2 of this invention. It is a flowchart which shows the input display method which is the processing content of the input display apparatus by Embodiment 2 of this invention. It is explanatory drawing which shows the locus | trajectory image in which the line width was set according to the writing pressure of the user. It is explanatory drawing which shows the locus | trajectory image in which the line | wire width was set according to the speed to trace. It is explanatory drawing which shows the locus | trajectory image in which the line | wire width was set according to the writing pressure and tracing speed of the user which change on the way. It is a block diagram which shows the input display apparatus by Embodiment 3 of this invention. It is a hardware block diagram of the input display apparatus by Embodiment 3 of this invention. It is a flowchart which shows the input display method which is the processing content of the input display apparatus by Embodiment 3 of this invention. It is explanatory drawing which shows the color of the locus | trajectory image corresponding to the language of audio | voice. It is a block diagram which shows the input display apparatus by Embodiment 4 of this invention. It is a hardware block diagram of the input display apparatus by Embodiment 4 of this invention. 5 is a flowchart showing processing contents when an operation accepted by the operation accepting unit 1 is switched. FIG. 6 is an explanatory view showing an example of an operation for switching an operation accepted by the operation accepting unit 1; It is an explanatory view showing an example of a picture of a picture generated according to operation contents of picture writing.

  Hereinafter, in order to explain the present invention in more detail, a mode for carrying out the present invention will be described according to the attached drawings.

Embodiment 1
FIG. 1 is a block diagram showing an input display apparatus according to Embodiment 1 of the present invention, and FIG. 2 is a hardware block diagram of the input display apparatus according to Embodiment 1 of the present invention.
In FIG. 1 and FIG. 2, the operation receiving unit 1 is realized by the pointing device 21 and performs a process of receiving a tracing operation as a user operation.
Although a touch panel is used as the pointing device 21 in this first embodiment, the present invention is not limited to the touch panel. For example, a joystick, pointing stick (track point), touch pad, stylus, data glove, Trackballs, pen tablets, mice, light pens, joy pads, etc. can be used.

The character management DB 2 is realized by, for example, the memory 22 such as a RAM or a hard disk, and stores a character attribute table indicating the correspondence between character types, character sizes, character heights and character widths.
The character management unit 3 is realized by, for example, a character management processing circuit 23 constituted by a semiconductor integrated circuit on which a CPU (Central Processing Unit) is mounted, a one-chip microcomputer, or the like. The size of the character to be displayed on the display 29 is set in advance, the character management unit 3 refers to the character attribute table stored in the character management DB 2 and traces the character attribute information indicating the character height corresponding to the character size A process of outputting to the image generation unit 4 is performed.

  The locus image generation unit 4 is realized by, for example, an image generation processing circuit 24 configured by a semiconductor integrated circuit on which a GPU (Graphics Processing Unit) is mounted, a one-chip microcomputer, or the like. The locus image generation unit 4 determines the locus of the line drawn by the tracing from the tracing operation received by the operation reception unit 1 and carries out a process of generating a locus image indicating the locus. At this time, a locus image of a line width corresponding to the character height indicated by the character attribute information output from the character management unit 3 is generated.

The character string acquisition unit 5 includes a speech detection unit 6, a speech recognition dictionary DB 7, and a speech recognition unit 8, and executes a process of acquiring a character string indicating a speech recognition result.
The voice detection unit 6 is realized by, for example, the microphone 25 and detects the voice of the user.
The speech recognition dictionary DB 7 is realized by, for example, a memory 26 such as a RAM or a hard disk, and the speech recognition dictionary DB 7 is, for example, an acoustic model in which acoustic features of phonemes which are small units of human speech are described. It consists of a recognition dictionary in which speech recognition words are described.
The voice recognition unit 8 is realized by, for example, a voice recognition processing circuit 27 composed of a semiconductor integrated circuit mounted with a CPU, a one-chip microcomputer or the like, and recognizes voice detected by the voice detection unit 6 It has a speech recognition engine.
That is, the speech recognition unit 8 analyzes the speech detected by the speech detection unit 6, calculates acoustic features of the speech, and calculates the speech features from among the words described in the recognition dictionary of the speech recognition dictionary DB 7. A word having an acoustic feature closest to the acoustic feature is searched, and a process of outputting a character string indicating the word to the display control unit 9 as a character string indicating a speech recognition result is performed.

Although FIG. 1 shows an example in which the character string acquisition unit 5 includes the speech recognition dictionary DB 7 and the speech recognition unit 8, a data transmission / reception unit is mounted instead of the speech recognition dictionary DB 7 and the speech recognition unit 8. It may be
The data transmission / reception unit is, for example, a network communication device such as a network card capable of transmitting / receiving data to / from a voice recognition server (not shown) via a communication channel such as the Internet or LAN. Data indicative of the detected speech is sent to the speech recognition server.
The speech recognition server is equipped with a speech recognition engine that recognizes speech, and when data indicating the speech transmitted from the data transmission / reception unit is received, the speech is recognized and a character string indicating the speech recognition result is displayed. It transmits to the said data transmission / reception part.
When the data transmitting / receiving unit receives a character string indicating the recognition result of the voice transmitted from the voice recognition server, the data transmitting / receiving unit outputs the character string to the display control unit 9.

The display control unit 9 is realized by, for example, a display processing circuit 28 configured of a semiconductor integrated circuit on which a GPU is mounted, a one-chip microcomputer, or the like. The display control unit 9 performs a process of displaying the trajectory image generated by the trajectory image generation unit 4 on the display 29, and superimposing the character string acquired by the character string acquisition unit 5 on the trajectory image.
That is, when the locus image generation unit 4 generates a locus image before the character recognition unit 8 of the character string acquisition unit 5 outputs the character string, the display control unit 9 displays the locus image on the display 29. After that, the character string output from the voice recognition unit 8 of the character string acquisition unit 5 is superimposed and displayed on the trajectory image. On the other hand, when a character string is output from the voice recognition unit 8 of the character string acquisition unit 5 before the trajectory image generation unit 4 generates a trajectory image, the process waits until the trajectory image generation unit 4 generates a trajectory image. When the locus image is generated by the locus image generation unit 4, the locus image is displayed on the display 29, and the character string is superimposed and displayed on the locus image.
Further, the display control unit 9 superimposes and displays the character string on the trajectory image, and then executes a process of erasing the display of the trajectory image.
The character string temporary storage unit 10 is realized by, for example, the memory 30 such as a RAM or a hard disk, and temporarily stores the character string output from the voice recognition unit 8 of the character string acquisition unit 5.

In FIG. 1, components of the operation display unit 1, the character management DB 2, the character management unit 3, the locus image generation unit 4, the character string acquisition unit 5, the display control unit 9 and the character string temporary storage unit 10 which are components of the input display device. Although each is assumed to be configured by dedicated hardware, the input display device may be configured by a computer.
FIG. 3 is a hardware configuration diagram when the input display device is configured by a computer.
When the input display device is configured by a computer, the character management DB 2, the speech recognition dictionary DB 7, and the character string temporary storage unit 10 are configured on the memory 41 of the computer, and the operation reception unit 1, character management unit 3, trajectory image generation A program describing the processing contents of the unit 4, the voice detection unit 6, the voice recognition unit 8 and the display control unit 9 is stored in the memory 41, and the processor 42 executes the program stored in the memory 41. Just do it.
FIGS. 4 and 5 are flow charts showing an input display method which is the processing content of the input display apparatus according to the first embodiment of the present invention.

Next, the operation will be described.
FIG. 6 is an explanatory view showing a character attribute table stored in the character management DB 2.
In the character attribute table of FIG. 6, the character height and character width corresponding to the character size are described for each character type (for example, Mincho type, Gothic type, etc.).
In the character management unit 3, the size of characters to be displayed on the display 29 is set in advance, and the character management unit 3 refers to the character attribute table stored in the character management DB 2 and copes with the character size Character attribute information indicating the character height to be output is output to the trajectory image generation unit 4 (step ST1 in FIG. 4).
For example, if the character type of the character set in advance is “A” and the character size is “10”, character attribute information indicating that the character height is “5 mm” is output to the trajectory image generation unit 4 Be done.
Also, if the character type of the character set in advance is “B” and the character size is “5”, character attribute information indicating that the character height is “2 mm” is output to the trajectory image generation unit 4 Be done.
The character management unit 3 also outputs character attribute information indicating the character height and character width corresponding to the character size to the display control unit 9.

When receiving the character attribute information from the character management unit 3, the locus image generation unit 4 sets the line width of the locus image to be generated to the line width corresponding to the character height indicated by the character attribute information (step ST2).
For example, when the character height indicated by the character attribute information is "10 mm", if the upper and lower margins of the character are set to "0.5 mm", the line width of the locus image is "11 mm". Set
11 = 10 + 0.5 + 0.5 (mm)
Here, although the example which provides a margin in the upper part and the lower part of a character is shown, the character height which character attribute information shows, and the line width of a locus | trajectory image may correspond, without providing a margin.

In the first embodiment, it is assumed that the pointing device 21 that realizes the operation receiving unit 1 is a touch panel, and therefore, the user can write on the screen using his / her finger, an input pen for a touch panel, or the like. When the operation of is performed on the touch panel (step ST3: YES), the operation receiving unit 1 receives the tracing operation and outputs the tracing operation content to the locus image generating unit 4 (step ST4).
The operation content of trace writing corresponds to time-series data indicating continuous change of the position of the finger touching the touch panel or the position of the input pen (coordinates on the touch panel).

When the trajectory image generating unit 4 receives the operation content of the tracing from the operation receiving unit 1, it determines the trajectory of the line drawn by the tracing from the operation content (step ST 5).
As described above, the operation content of the tracing indicates the continuous change of the touch position on the touch panel. Therefore, by connecting the touch positions in time series, it is necessary to determine the locus of the line drawn by the tracing. Can.
When the locus image generation unit 4 determines the locus of a line drawn by tracing, it generates a locus image indicating the locus, and outputs the locus image to the display control section 9 (step ST6).
The line width of this locus image is the line width set in step ST2. In the above example, since the line width of the locus image is set to "11 mm", a locus image having a line width of "11 mm" is generated Do.

When the speech detection unit 6 detects the user's speech (step ST7: YES), the speech recognition unit 8 of the character string acquisition unit 5 executes the speech recognition process using the speech recognition dictionary DB 7 Then, a character string (text) indicating the recognition result of the voice is generated, and the character string is output to the display control unit 9 (step ST8).
The speech recognition process 8 by the speech recognition unit 8 is a well-known technique, and therefore detailed description is omitted. For example, the speech detected by the speech detection unit 6 is analyzed to calculate the acoustic feature of the speech, From the words described in the recognition dictionary of the speech recognition dictionary DB 7, a word having an acoustic feature closest to the calculated acoustic feature is searched, and a character string indicating the word is used as a character string indicating a speech recognition result There is a conceivable way to output.
In the first embodiment, for convenience of explanation, the user utters "Let's go to see cherry blossoms in spring", and let voice recognition unit 8 "go to see cherry blossoms in spring" It is assumed that a character string indicating "" is output.

Here, FIG. 7 is an explanatory view showing a display process in the case where a trajectory image is generated before a character string indicating a speech recognition result is generated.
FIG. 8 is an explanatory diagram showing display processing when a character string indicating a speech recognition result is generated before a trajectory image is generated.
When the trajectory image is output from the trajectory image generation unit 4 before the character recognition unit 8 of the character string acquisition unit 5 outputs the character string (in the case of YES in FIG. 5: step ST11), the display control unit 9 As shown in FIG. 7A, the locus image is displayed on the display 29 (step ST12).
Here, the display of the trajectory image may be such that the entire trajectory image may be simultaneously displayed, or an animation may be performed in which the display is sequentially performed from the top side of the character string to be displayed later.

The display control unit 9 waits until a character string is output from the speech recognition unit 8 of the character string acquisition unit 5 (step ST13: NO), and the user speaks as shown in FIG. 7 (b). Then, when a character string indicating a speech recognition result is output from the speech recognition unit 8 (step ST13: YES), the character string is superimposed on the trajectory image as shown in FIG. 7 (c). It displays (step ST14). The size of the character in the character string displayed on the trajectory image corresponds to the character height and character width indicated by the character attribute information output from the character management unit 3.
Here, the display of the character string may be made to simultaneously display all the characters constituting the character string, but in the form of an animation display in which the first character of the character string to the last character are displayed in order It may be.

As shown in FIG. 8A, the display control unit 9 speaks before the user performs a stroke operation, and before the trajectory image is output from the trajectory image generation unit 4, the display control unit 9 generates a character string acquisition unit. When a character string is output from the voice recognition unit 8 of 5 (step ST11 in FIG. 5: NO), the character string is stored in the character string temporary storage unit 10, and the trajectory image from the trajectory image generation unit 4 is It waits until it outputs (step ST15: in the case of NO).
Thereafter, when the trace image is output from the trace image generation unit 4 by the user performing the tracing operation, the display control unit 9 outputs a character string from the character string temporary storage unit 10 (step ST15: YES). Is displayed on the display 29 as shown in FIG. 8B, and the character string is displayed superimposed on the locus image. That is, the display control unit 9 superimposes and displays the locus image and the character string at the same position (step ST16). However, although the display of the locus image and the character string may be made to display the whole simultaneously, it may be in the form of animation display in which the characters are displayed sequentially from the top side of the character string.

When the display control unit 9 superimposes and displays the character string on the trajectory image, as shown in FIGS. 7 (d) and (e) and FIGS. 8 (c) and 8 (d), it erases the trajectory image and Are displayed (step ST17).
The deletion of the locus image may be performed so as to simultaneously delete the entire locus image, but an animation may be performed in which the deletion is sequentially performed from the top of the character string.
Here, it is assumed that the trajectory image and the character string are superimposed and displayed and then the trajectory image is erased after a predetermined time, but when the display of the character string is completed, the trajectory image is erased immediately. It may be.

  As apparent from the above, according to the first embodiment, from the operation reception unit 1 for receiving the tracing operation and the tracing operation accepted by the operation reception unit 1, the locus of the line drawn by the tracing. And a character string acquisition unit 5 for acquiring a character string indicating a speech recognition result, and the display control unit 9 generates a trajectory image. Since the locus image generated by the unit 4 is displayed on the display 29, and the character string acquired by the character string acquisition unit 5 is displayed superimposed on the locus image, the user's free layout is obtained. An effect of displaying a character string indicating a speech recognition result is achieved.

  Further, according to the first embodiment, when the display control unit 9 generates the trajectory image by the trajectory image generation unit 4 before the display control unit 9 outputs the character string from the voice recognition unit 8 of the character string acquisition unit 5. After displaying the locus image on the display 29, the character string output from the voice recognition unit 8 of the character string acquisition unit 5 is superimposed on the locus image and displayed, while the locus image generation unit 4 generates a locus image If a character string is output from the speech recognition unit 8 of the character string acquisition unit 5 before it is set, the process waits until the locus image generation unit 4 generates a locus image, and the locus image generation unit 4 generates a locus image Then, the locus image is displayed on the display 29 and the character string is displayed superimposed on the locus image. Therefore, when the user performs a tracing operation before speaking, the voice is voiced. Sentence that shows recognition result of It is possible to grasp the position at which the columns are displayed in advance. Therefore, the user can hold the display image of the character string. In addition, if the user speaks before performing the drag operation, if the drag operation is performed, the trajectory image and the character string are immediately displayed overlapping at the same position, so that the user has a highly responsive operation feeling. An effect that can be given to

Second Embodiment
Although the line width of the locus image is set to the line width corresponding to the character height of the character size set in advance in the first embodiment, it has been described according to the state of the drag operation by the user. The line width of the trajectory image may be set.

9 is a block diagram showing an input display apparatus according to Embodiment 2 of the present invention, and FIG. 10 is a hardware block diagram of the input display apparatus according to Embodiment 2 of the present invention.
9 and 10, the same reference numerals as those in FIGS. 1 and 2 denote the same or corresponding parts, and therefore the description thereof will be omitted.
The operation state detection unit 11 is realized by, for example, an operation state detection processing circuit 31 configured by a semiconductor integrated circuit or a one-chip microcomputer on which a CPU is mounted. Implement processing to detect the state.
That is, the operation state detection unit 11 senses, for example, the pressure (writing pressure) of the user's finger or the input pen on the touch panel as the pointing device 21 as the state of the operation received by the operation reception unit 1.

  The locus image generation unit 12 is realized by, for example, an image generation processing circuit 32 configured by a semiconductor integrated circuit on which a GPU is mounted, a one-chip microcomputer, or the like. Similar to the trajectory image generation unit 4 of FIG. 1, the trajectory image generation unit 12 determines the trajectory of the line drawn by the tracing operation from the drag operation received by the operation reception unit 1 and indicates the locus thereof. A process of generating a trajectory image is performed. At this time, unlike the trajectory image generation unit 4 of FIG. 1, a trajectory image of a line width corresponding to the state of the operation detected by the operation state detection unit 11 is generated.

The display control unit 13 is realized by, for example, a display processing circuit 33 configured by a semiconductor integrated circuit or a one-chip microcomputer on which a GPU is mounted, and the display processing similar to that of the display control unit 9 in FIG. carry out.
Further, the display control unit 13 changes the character size of the characters constituting the character string acquired by the character string acquisition unit 5 according to the line width set by the locus image generation unit 12, and changes the character size after the change. The character string is superimposed on the trajectory image and displayed.

In FIG. 9, the operation reception unit 1, the character management DB 2, the character management unit 3, the operation state detection unit 11, the trajectory image generation unit 12, the character string acquisition unit 5, the display control unit 13 and the characters which are components of the input display device Although it is assumed that each of the column temporary storage units 10 is configured by dedicated hardware, the input display device may be configured by a computer.
When the input display device is configured by a computer, the character management DB 2, the speech recognition dictionary DB 7, and the character string temporary storage unit 10 are configured on the memory 41 of the computer shown in FIG. , A program describing the processing contents of the operation state detection unit 11, the trajectory image generation unit 12, the speech detection unit 6, the speech recognition unit 8 and the display control unit 13 is stored in the memory 41, and the processor 42 is stored in the memory 41 You just have to run the program that
11 and 5 are flowcharts showing an input display method which is processing contents of the input display device according to the second embodiment of the present invention.
In FIG. 11, the same step numbers as in FIG. 4 indicate the same or corresponding parts.

Next, the operation will be described.
In the second embodiment, it is assumed that a touch pressure sensor for sensing writing pressure is mounted on a touch panel which is the pointing device 21.
The operation state detection unit 11 acquires the detection result of the writing pressure from the writing pressure detection sensor mounted on the pointing device 21, thereby setting the touch panel as the pointing device 21 as the state of the operation received by the operation receiving unit 1. The writing pressure of the user with respect to is detected (step ST21 in FIG. 11).

When the operation state detecting unit 11 detects the writing pressure of the user as the state of the operation received by the operation receiving unit 1, the locus image generating unit 12 detects the writing pressure of the user and the character attribute information output from the character managing unit 3. The line width of the trajectory image to be generated is set from the character height indicated by (step ST22).
For example, if the character height indicated by the character attribute information is set to the line width of the reference of the locus image and the writing pressure of the user is equal to or more than the reference value set in advance, the larger the writing pressure is, the larger An offset value is set, and a value obtained by adding the reference line width and the offset value is determined as the line width of the final trajectory image.
Final track image line width = reference line width + offset value On the other hand, if the user's writing pressure is smaller than the preset reference value, the larger the writing pressure is, the larger the offset value is set The value obtained by subtracting the offset value from the reference line width is determined as the line width of the final trajectory image.
Final track image line width = reference line width-offset value

The display control unit 13 displays a locus image and a character string on the display 29 according to the flowchart of FIG. 5 as in the display control unit 9 of FIG.
However, in the second embodiment, since the line width of the trajectory image is set according to the writing pressure of the user, the trajectory image generation unit 12 makes the line width of the final trajectory image thicker than the reference line width. In this case, the character size of the characters forming the character string is increased according to the line width of the final trajectory image, and the character string is displayed on the trajectory image.
For example, if the line width of the final trajectory image is set to "10 mm" when the reference line width (= the character height indicated by the character attribute information) is "5 mm" Change the character height to "10 mm" and display the character string on the trajectory image. Here, an example is shown in which the character height and the line width of the locus image match, but when providing margins at the upper and lower portions of the character, for example, assuming that both the upper and lower margins are "1 mm", The character height of the characters constituting the character string is changed to "8 mm", and the character string is displayed on the trajectory image.

In addition, when the trajectory image generation unit 12 makes the line width of the final trajectory image thinner than the reference line width, the display control unit 13 determines the character string according to the line width of the final trajectory image. Reduce the character size of the characters that make up and display the character string on the trajectory image.
For example, when the standard line width (= the character height indicated by the character attribute information) is "5 mm", if the line width of the final trajectory image is set to "3 mm", Change the character height to "3 mm" and display the character string on the trajectory image. Here, an example is shown in which the character height and the line width of the locus image match, but when providing margins at the top and bottom of the character, for example, it is assumed that both the top and bottom margins are “0.5 mm”. For example, the character height of the characters constituting the character string is changed to "2 mm", and the character string is displayed on the trajectory image.

FIG. 12 is an explanatory view showing a locus image in which the line width is set in accordance with the writing pressure of the user.
FIG. 12A shows a locus image in which a thick line width is set because the writing pressure of the user is equal to or greater than the reference value, and FIG. 12B shows that the writing pressure of the user is less than the reference value. For the purpose of illustration, a trace image in which a thin line width is set is shown.

In the second embodiment, an example is shown in which the operation state detection unit 11 detects the writing pressure of the user on the touch panel, which is the pointing device 21, as the state of the operation received by the operation reception unit 1; For example, as the state of the operation received by the operation receiving unit 1, the user may detect the speed at which the touch panel as the pointing device 21 is scanned.
As for the speed at which the touch panel is scanned, if the operation state detection unit 11 identifies a continuous change in the touch position on the touch panel from the operation content of the trace writing received by the operation reception unit 1, the unit time of the touch position on the touch panel It can be determined from the change of the hit.

When the operation state detection unit 11 detects the speed of tracing the touch panel as the state of the operation accepted by the operation acceptance unit 1, the locus image generation unit 12 indicates the tracing speed and the character attribute information output from the character management unit 3. The line width of the generated trajectory image is set from the character height.
For example, if the character height indicated by the character attribute information is set to the line width of the reference of the locus image, and the tracing speed is equal to or more than the preset reference value, the larger the tracing speed is, the larger the offset value. Set and add the standard line width and offset value to determine the final line width of the trajectory image.
Final track image line width = reference line width + offset value On the other hand, if the tracing speed is slower than the preset reference value, the larger the tracing speed is than the reference value, the larger the offset value is set, and the reference The line width of the final trajectory image is determined by subtracting the offset value from the line width of.
Final track image line width = reference line width-offset value

FIG. 13 is an explanatory view showing a trajectory image in which the line width is set in accordance with the tracing speed.
FIG. 13A shows a trajectory image in which a thick line width is set because the tracing speed is equal to or higher than the reference value, and FIG. 13B shows that the tracing speed is less than the reference value. It shows a trajectory image in which a thin line width is set.

In the second embodiment, an example is shown in which the writing pressure and the tracing speed of the user are uniform, but when the writing pressure and the tracing speed of the user change in the middle, the trace image generation unit 12 detects the line width. A trajectory image that changes in the middle may be generated.
FIG. 14 is an explanatory view showing a locus image in which the line width is set in accordance with the writing pressure and tracing speed of the user changing in the middle.
FIG. 14A shows a locus image when the tracing speed is gradually increased or when the user's writing pressure is gradually increased, and the line width of the locus image is gradually increased. There is.
FIG. 14B shows a trajectory image when the tracing speed and the writing pressure of the user are irregularly changing, and the line width of the trajectory image is thick or thin.

  As apparent from the above, according to the second embodiment, the operation state detection unit 11 for detecting the state of the operation received by the operation reception unit 1 is provided, and the trajectory image generation unit 12 includes the operation state detection unit 11. Since the locus image of the line width corresponding to the state of the operation detected by is generated, the effect of diversifying the display form of the character string can be achieved.

In the second embodiment, the locus image generation unit 12 receives the state of the operation (for example, the writing pressure of the user) received by the operation reception unit 1 by the operation state detection unit 11 and the characters output from the character management unit 3. Although the line width of the trajectory image to be generated is set based on the character height indicated by the attribute information, the operation accepting unit 1 is used without using the character height indicated by the character attribute information output from the character management unit 3 The line width of the trajectory image to be generated may be set only from the state of the operation accepted by the.
For example, a table storing the correspondence between the user's writing pressure (or tracing speed) and the line width of the locus image is prepared, and the locus image generating unit 12 determines the writing pressure of the user from the table (or It is conceivable to read the line width corresponding to the tracing speed) and set the line width to the line width of the locus image.

In the second embodiment, an example of generating a trajectory image of a line width corresponding to the state of the operation accepted by the operation accepting unit 1 has been described, but according to the state of the operation accepted by the operation accepting unit 1 The shading of the characters displayed on the display 29 may be changed. For example, as the writing pressure of the user is stronger or the tracing speed is faster, it is conceivable to make the shading of the characters darker. Further, the thickness of the characters displayed on the display 29 may be changed according to the state of the operation accepted by the operation accepting unit 1. For example, as the writing pressure of the user increases or the tracing speed increases, it is conceivable to make the thickness of the characters thicker.
In addition, according to the line width of the locus image generated by the locus image generation unit 12, the character height of the character displayed superimposed on the locus image is changed, but according to the line width of the locus image You may change the character type of the character which comprises a row | line.
For example, when the line width of the trajectory image is gradually thickened, it is conceivable to change the character type as character type A → character type B → character type C.

Third Embodiment
Although the display control units 9 and 13 display the trajectory image generated by the trajectory image generating units 4 and 12 on the display 29 in the first and second embodiments, the determination result of the language of the voice is acquired. The trajectory image may be displayed in a color corresponding to the language indicated by the determination result.

FIG. 15 is a block diagram showing an input display according to Embodiment 3 of the present invention, and FIG. 16 is a hardware block diagram of the input display according to Embodiment 3 of the present invention.
In FIG. 15 and FIG. 16, the same reference numerals as in FIG. 9 and FIG.
The language determination unit 14 of the character string acquisition unit 5 is realized by, for example, the language determination processing circuit 34 configured of a semiconductor integrated circuit or a one-chip microcomputer on which a CPU is mounted. The language discrimination processing of the voiced speech is performed, and the discrimination result of the language is output to the display control unit 15.

In the third embodiment, an example is shown in which the character string acquisition unit 5 has the language determination unit 14 mounted, but even if the data transmission / reception unit is mounted instead of the language determination unit 14 Good.
The data transmission / reception unit is, for example, a network communication device such as a network card capable of transmitting / receiving data to / from a language determination server (not shown) via a communication channel such as the Internet or LAN. Data indicative of the detected speech is sent to the language discrimination server.
The language discrimination server is equipped with a language discrimination engine for discriminating the language of speech, and when data indicating the speech transmitted from the data transmitting / receiving unit is received, the speech is recognized and the speech discrimination result of the speech is said Transmit to the data transceiver.
The data transmission / reception unit outputs, to the display control unit 15, the language determination result of the voice transmitted from the language determination server.

The display control unit 15 is realized by, for example, a display processing circuit 35 configured of a semiconductor integrated circuit mounted with a GPU, a one-chip microcomputer, or the like, and performs the same display processing as the display control unit 13 of FIG. carry out.
In addition, the display control unit 15 performs processing for displaying the trajectory image generated by the trajectory image generation unit 12 on the display 29 in a color corresponding to the language indicated by the determination result output from the language determination unit 14.

In FIG. 15, the operation reception unit 1, the character management DB 2, the character management unit 3, the operation state detection unit 11, the trajectory image generation unit 12, the character string acquisition unit 5, the display control unit 15 and the characters which are components of the input display device Although it is assumed that each of the column temporary storage units 10 is configured by dedicated hardware, the input display device may be configured by a computer.
When the input display device is configured by a computer, the character management DB 2, the speech recognition dictionary DB 7, and the character string temporary storage unit 10 are configured on the memory 41 of the computer shown in FIG. , A program describing the processing contents of the operation state detection unit 11, the trajectory image generation unit 12, the speech detection unit 6, the speech recognition unit 8, the language determination unit 14, and the display control unit 15 is stored in the memory 41; May execute a program stored in the memory 41.
FIGS. 17 and 5 are flowcharts showing an input display method which is the processing content of the input display apparatus according to the third embodiment of the present invention.
In FIG. 17, the same step numbers as in FIG. 11 indicate the same or corresponding parts.

Next, the operation will be described.
When the voice detection unit 6 detects the user's voice, the language determination unit 14 of the character string acquisition unit 5 performs language determination processing of the voice and outputs the determination result of the language to the display control unit 15 (FIG. 17). Step ST9). Since the speech language discrimination processing itself is a known technique, detailed description will be omitted.

The display control unit 15 displays the locus image and the character string on the display 29 in accordance with the flowchart of FIG. 5 in the same manner as the display control unit 13 of FIG.
However, unlike the display control unit 13 of FIG. 9, the display control unit 15 displays the trajectory image generated by the trajectory image generation unit 12 in a color corresponding to the language indicated by the determination result output from the language determination unit 14 Display on 29
In addition, the language determination unit 14 can not determine the language of the voice until the voice detection unit 6 detects the user's voice, and before the voice detection unit 6 detects the user's voice, trajectory image generation is performed. When the trajectory image is generated by the unit 12, the display control unit 15 does not display the trajectory image until the voice detection unit 6 detects the voice of the user. Also, until the voice detection unit 6 detects the voice of the user, the locus is displayed in gray color, and when the voice of the user is detected and the language discrimination result is obtained, the locus image is displayed in the color corresponding to the language You may do it.

FIG. 18 is an explanatory view showing the color of a trajectory image corresponding to the language of speech.
FIG. 18A shows the case where the speech language is Japanese, and the locus image is displayed in blue, for example.
FIG. 18B shows the case where the language of speech is English, and the locus image is displayed in red, for example.
Here, although the example which displays a blue locus | trajectory image if the language of an audio | voice is Japanese and the language of an audio | voice is English, a red locus image is displayed, it is only an example to the last. Needless to say, other colors may be displayed.

  As is apparent from the above, according to the third embodiment, the language determination unit 14 is provided which executes the language determination process of the voice detected by the voice detection unit 6 and outputs the determination result of the language. The control unit 15 is configured to display the trajectory image generated by the trajectory image generation unit 12 on the display 29 in a color corresponding to the language indicated by the determination result output from the language determination unit 14. The effect of being able to intuitively grasp whether the character is displayed is produced.

Fourth Embodiment
In the above first to third embodiments, after the locus image and the character string are displayed, the locus image is erased so that only the character string is finally displayed on the display 29. An image of a picture generated in accordance with the operation of drawing a picture may be displayed on the display 29.

FIG. 19 is a block diagram showing an input display device in accordance with Embodiment 4 of the present invention, and FIG. 20 is a hardware block diagram of the input display device in accordance with Embodiment 4 of the present invention.
19 and 20, the same reference numerals as those in FIGS. 15 and 16 denote the same or corresponding parts, and therefore the description thereof will be omitted.
The operation switching unit 16 is realized by, for example, an operation switching processing circuit 36 configured of a semiconductor integrated circuit mounted with a CPU, a one-chip microcomputer, or the like, and is used to switch an operation accepted by the operation accepting unit 1. As an operation, for example, when a long press operation of the pointing device 21 is detected, a process of switching the operation accepted by the operation accepting unit 1 is performed.
That is, when the operation switching unit 16 detects a long press operation of the pointing device 21 when the operation accepted by the operation accepting unit 1 is a tracing operation, the display 29 displays a picture on the operation accepted by the operation accepting unit 1. A process of switching to the operation of drawing a picture is performed. On the other hand, when the operation accepted by the operation accepting unit 1 is a pictorial writing operation, when a long press operation of the pointing device 21 is detected, processing is performed to switch the operation accepted by the operation accepting unit 1 to a tracing operation.

The picture image generation unit 17 is realized by, for example, an image generation processing circuit 37 configured of a semiconductor integrated circuit or a one-chip microcomputer on which a GPU is mounted, and the operation switched by the operation switching unit 16 is If it is a picture writing operation, processing for generating a picture image according to the picture writing operation accepted by the operation accepting unit 1 is performed.
The display control unit 18 is realized by, for example, a display processing circuit 38 configured of a semiconductor integrated circuit or a one-chip microcomputer on which a GPU is mounted, and the operation switched by the operation switching unit 16 is traced. In the case of the operation of (1), the display processing similar to that of the display control unit 15 of FIG.
If the operation switched by the operation switching unit 16 is a picture writing operation, the display control unit 18 performs a process of displaying the picture of the picture generated by the picture image generating unit 17 on the display 29.

In FIG. 19, the operation reception unit 1, the character management DB 2, the character management unit 3, the operation state detection unit 11, the locus image generation unit 12, the operation switching unit 16, the picture image generation unit 17 and the characters which are components of the input display device Although it is assumed that each of the column acquisition unit 5, the display control unit 18, and the character string temporary storage unit 10 is configured of dedicated hardware, the input display device is configured of a computer It is also good.
When the input display device is configured by a computer, the character management DB 2, the speech recognition dictionary DB 7, and the character string temporary storage unit 10 are configured on the memory 41 of the computer shown in FIG. The processing contents of the operation state detection unit 11, the trajectory image generation unit 12, the operation switching unit 16, the pictorial image generation unit 17, the voice detection unit 6, the speech recognition unit 8, the language determination unit 14, and the display control unit 18 are described. A certain program may be stored in the memory 41, and the processor 42 may execute the program stored in the memory 41.
FIG. 21 is a flowchart showing the contents of processing when the operation accepted by the operation accepting unit 1 is switched.

Next, the operation will be described.
When the operation switching unit 16 detects a long press operation as an operation for switching the operation accepted by the operation accepting unit 1 (in the case of YES in step ST31 of FIG. 21), a process for switching the operation accepted by the operation accepting unit 1 Is performed (step ST32).
FIG. 22 is an explanatory view showing an example of the operation for switching the operation accepted by the operation accepting unit 1.
FIG. 22 illustrates an example in which the operation for switching the operation accepted by the operation reception unit 1 is a long press operation, and for example, the same location of the touch panel as the pointing device 21 is preset. It is an operation in which the user's finger is touching continuously for more than one hour.
That is, when the operation switching unit 16 detects the long press operation of the pointing device 21 when the operation accepted by the operation accepting unit 1 at this point in time is a drag-and-write operation, the operation accepted by the operation accepting unit 1 thereafter An instruction to switch to the picture operation is output to the operation reception unit 1.
On the other hand, at the present time, when the operation accepted by the operation accepting unit 1 is a pictorial writing operation, when the long press operation of the pointing device 21 is detected, thereafter, the operation accepted by the operation accepting unit 1 is switched to the tracing operation. The command is output to operation accepting unit 1.

The operation reception unit 1 switches the operation to be received according to the command of the operation switching unit 16.
Therefore, in the state where operation accepting unit 1 switches the operation to be accepted to the operation of tracing (step ST33: YES), the operation of tracing is accepted as in the first to third embodiments, and tracing is performed. The content of the writing operation is output to the trajectory image generation unit 12 (step ST34).
Further, in a state where the operation receiving unit 1 switches the operation to be received to the operation of drawing (in the case of NO at step ST33), the operation of drawing is received and the operation content of the drawing is made to the picture image generating unit 17. It outputs (step ST35).

When the operation receiving unit 1 receives the tracing operation, the locus image generating unit 12 generates a locus image according to the contents of the tracing operation as in the second embodiment (step ST36).
When the operation reception unit 1 receives an operation of drawing, the picture image generation unit 17 generates an image of a picture according to the contents of the operation of the drawing (step ST37).
FIG. 23 is an explanatory view showing an example of an image of a picture generated in accordance with the operation content of the picture.
Since the process itself which produces | generates the image of a picture according to the operation content of a picture is a well-known technique, detailed description is abbreviate | omitted. In addition to the function of drawing a line, the operation content of the picture may include a function of changing the thickness or color of the line to be drawn, a function of erasing the line, and the like.

When the trajectory image generation unit 12 generates a trajectory image, the display control unit 18 displays the trajectory image and the character string output from the character string acquisition unit 5 on the display 29 as in the display control unit 15 of FIG. The process is performed (step ST38).
Further, when the picture image generation unit 17 generates a picture image, the display control unit 18 displays the picture image on the display 29 (step ST39).

  As apparent from the above, according to the fourth embodiment, if the operation switching unit 16 switching the operation accepted by the operation accepting unit 1 and the operation switched by the operation switching unit 16 are the operation of picture writing , And a pictorial image generation unit 17 that generates a pictorial image according to the pictorial operation accepted by the operation acceptance unit 1, and the operation switched by the operation switching unit 16 by the display control unit 18 is traced. If it is an operation, as in the display control unit 15 of FIG. 15, the locus image and the character string are superimposed and displayed, while if the operation switched by the operation switching unit 16 is a pictorial operation, picture image generation Since the picture of the picture generated by the unit 17 is displayed, the picture 29 can be displayed on the display 29 together with the character string.

In the fourth embodiment, when the operation switching unit 16 detects a long press operation of the pointing device 21 when the operation accepted by the operation accepting unit 1 is a drag-in operation, the operation accepted by the operation accepting unit 1 is detected. While switching to the picture writing operation, when the operation accepted by the operation accepting unit 1 is a picture writing operation, when the long press operation of the pointing device 21 is detected, the operation accepted by the operation accepting unit 1 is an operation of tracing. An example of switching is shown.
However, the switching method of the operation accepted by the operation accepting unit 1 is not limited to this, and the following switching method may be used.
For example, in normal times, the operation accepted by the operation accepting unit 1 is set to a pictorial operation, and when a long press operation of the pointing device 21 is detected, the operation switching unit 16 is accepted by the operation accepting unit 1 A command to switch the operation to the operation of tracing is output to the operation receiving unit 1 so that the operation of the writing operation can be received by the operation receiving unit 1.
When the tracing operation is finished, the operation accepted by the operation accepting unit 1 automatically returns to the pictorial operation, and thereafter, when the long press operation of the pointing device 21 is detected, the operation switching unit 16 operates again. A command to switch the operation accepted by the accepting unit 1 to the operation of tracing is output to the operation accepting unit 1 so that the operation of tracing can be accepted by the operation accepting unit 1.

  In the fourth embodiment, when the operation switching unit 16 detects the long press operation of the pointing device 21, it switches the operation accepted by the operation accepting unit 1, but the operation accepted by the operation accepting unit 1 at present However, information may be displayed on the display 29 indicating whether it is a tracing operation or a pictorial operation.

  In the scope of the invention, the present invention allows free combination of each embodiment, or modification of any component of each embodiment, or omission of any component in each embodiment. .

  The input display device and the input display method according to the present invention are suitable for the case where it is necessary to display the character string indicating the recognition result of the voice in a free layout of the user when displaying the character string on the display.

DESCRIPTION OF SYMBOLS 1 operation reception part, 2 character management DB, 3 character management part, 4 locus image generation part, 5 character string acquisition part, 6 speech detection part, 7 speech recognition dictionary DB, 8 speech recognition part, 9 display control part,
10 character string temporary storage unit, 11 operation state detection unit, 12 trajectory image generation unit, 13 display control unit, 14 language determination unit, 15 display control unit, 16 operation switching unit, 17 picture image generation unit,
18 display control unit 21 pointing device 22 memory 23 character management processing circuit 24 image generation processing circuit 25 microphone 26 memory 27 voice recognition processing circuit 28 display processing circuit 29 display 30 memory 31 operation state Detection processing circuit, 32 image generation processing circuit, 33 display processing circuit, 34 language discrimination processing circuit, 35 display processing circuit, 36 operation switching processing circuit, 37 image generation processing circuit, 38 display processing circuit,
41 memory, 42 processors.

Claims (11)

An operation reception unit that receives a tracing operation;
An operation state detection unit that detects the state of the operation received by the operation reception unit;
The trajectory of a line drawn by tracing is determined from the operation of tracing received by the operation accepting unit, and a locus image indicating the determined locus corresponds to the state of the operation detected by the operation state detecting unit. A trajectory image generation unit that generates a trajectory image of the line width;
A character string acquisition unit that acquires a character string indicating a speech recognition result;
The locus image generated by the locus image generation unit is displayed on a display, and the character string acquired by the character string acquisition unit is a character size according to the line width of the locus image set by the locus image generation unit. A display control unit configured to display the locus image superimposed on the locus image displayed on the display with the characters
An input display device comprising:   The trajectory image generation unit generates a trajectory image in which the line width changes in the middle if the state of the operation detected by the operation state detection unit changes in the middle. Input display device.   The input display apparatus according to claim 1, wherein the operation state detection unit detects a writing pressure of a user as the state of the operation.   The input display according to claim 3, wherein, when the writing pressure of the user is less than a reference value, the locus image generation unit sets the line width of the locus image to be generated to a line width thinner than the reference line width. apparatus.   The input display apparatus according to claim 1, wherein the operation state detection unit detects a speed at which a user follows a touch panel as the state of the operation.   6. The input display apparatus according to claim 5, wherein, when the tracing speed is less than a reference value, the locus image generation unit sets the line width of the generated locus image to a line width thinner than the reference line width. The input display apparatus according to claim 1 or 2, wherein the locus image generation unit changes the gradation of the characters forming the character string according to the writing pressure of the user or the speed at which the user traces the touch panel. .   The input display according to claim 1 or 2, wherein the locus image generation unit changes the thickness of characters forming the character string according to the writing pressure of the user or the speed at which the user traces the touch panel. apparatus.   The input display apparatus according to claim 1 or 2, wherein the locus image generation unit changes a character type of characters constituting the character string according to the writing pressure of the user or the speed at which the user traces the touch panel. . It further comprises a character management unit that outputs character attribute information indicating a character height to the locus image generation unit,
The trajectory image generation unit sets the line width of the trajectory image to be generated corresponding to the character height indicated by the character attribute information by the character management unit in addition to the state of the operation detected by the operation state detection unit. The input display device according to any one of claims 1 to 9, characterized in that:   The locus image generation unit sets the line width of the locus image based on a table storing a correspondence between the writing pressure of the user or the speed at which the user traces the touch panel and the line width of the locus image. The input display device according to any one of claims 1 to 9.

Images