JP2002244841A - Voice indication system and voice indication program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To support communications between a speaker and a user by providing the user with language information and non-language information from the speaker simultaneously. SOLUTION: The voice indication system 100 includes, for example the speaker 10, a computing device 1 of a portable computer, a permeable display device 2 of a permeable spectacle display, and the user 40. The voice indication system 100 is a system to support communication between the speaker 10 and the user 40 (for example, a hearing impairment). The computing device 1 recognizes voice inputted by the speaker 10 through a microphone or the like and converts the same into language information (verbal information, character string here), and outputs the character string as a result of the voice recognition on the permeable display device 2. The permeable display device 2 indicates the character string of language information inputted by the computing device 1 and is provided with a permeable part (permeable display) for providing non-language information (non-verbal information, for example, notions of parts of the speaker's face such as lip and eyes, reading, gesture, sign language, expression or some of those) from the speaker 10.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a voice display system and a voice display program, and more particularly to not only character string information (language information: verbal information) as speech recognition result data but also a speaker (speaker). The present invention relates to a voice display system and a voice display program capable of assisting communication between a speaker and a user using information (non-verbal information: non-verbal information) other than linguistic information such as facial expressions, lips, and gestures.

[0002]

2. Description of the Related Art In recent years, in a society of advanced information and super aging, people (for example, elderly people, hearing impaired people, etc.) who have low ability of sensory organs (for example, sight, hearing, etc.) to receive various information. Therefore, development of various auxiliary devices (for example, hearing aids and the like) for supplementing the capabilities of these sensory organs has been eagerly desired. Especially in human-to-human communication,
Voice plays a very important role, and various assistive systems for the hearing impaired are being studied. For example, in a lecture at a university, there is a method of quickly writing down the contents of the lecture and displaying subtitles around a speaker or a lecture blackboard (reference:
Masayuki Kobayashi, Yasushi Ishihara, Shun Nishikawa, Hidetomo Takahashi, Prototype of real-time subtitle presentation system with ruby; Tsukuba College of Technology Techno Report, 1996).

[0003]

However, the above technique requires the assistance of a third party, and it is assumed that, for example, it is difficult to use a hearing-impaired person when going out alone. As described above, there is no auxiliary device that can be used by a hearing-impaired person on a daily basis. Therefore, assistive devices for the hearing impaired are also useful for elderly deaf or completely deaf persons who will continue to increase in the future, and the need for such devices is expected to increase in the future.

On the other hand, in recent years, a so-called voice recognition method of recognizing an input voice and converting it into a character string and displaying the character string as a subtitle of a monitor display of a computer or a television has become widespread. This speech recognition method generally performs 50 to 6 when an unspecified speaker is recognized in a noisy environment (that is, a daily living space).
At present, only a recognition rate of about 0% can be secured, so that the use is limited to a specific use.

[0005] However, this situation is a result of the speech recognition system being regarded as a man-machine interface as in the past, and this speech recognition system is considered as a man-man interface for humans. Humans can infer missing information from contexts before and after communication without complete speech recognition of input speech. (See: Miki Saito, Using speech recognition technology for the hearing-impaired.) Research on man-man interface; Master's thesis, Graduate School of Engineering, Hokkaido University, 19992).

Here, a technique related to the present invention will be described. In the communication, the present inventors use non-verbal information (non-verbal information) such as movement of a face part such as lips and eyes, reading speech (reading the movement of lips), gesture, sign language, and facial expression in addition to the voice of the speaker. Is important, and this non-verbal information requires a very high level of technology to be used in a man-machine interface. We focused on the point that it can be easily obtained.

[0007] In addition, the hearing impaired (user) often learns the above-mentioned "reading utterance" or "sign language" conveyed by gesture, and it can be understood that the reading utterance, sign language, etc. can understand the speaker's words to some extent. is assumed. For this reason, a character string of speech recognition result (verbal information) and non-verbal information obtained from reading and sign language can be simultaneously received by a hearing-impaired person, and the character string of the speech recognition result is presented to a hearing-impaired person. Is required.

At this time, it is important to consider that the ability to understand sentence in reading and sign language greatly depends on the hearing loss and hearing loss of a hearing-impaired person. The recognition rate must be 10 even if the result of speech recognition by the device
It is not 0%, and in most cases, the character string contains an error. Therefore, if all the characters that are the result of the speech recognition are presented, it is assumed that the meaning is misunderstood. Is a point. For this reason, for example, the ratio of displaying a character string to be presented can be set by the hearing-impaired person according to the ability of the hearing-impaired person to understand a sentence in reading and sign language (that is, proficiency in reading and sign language). is necessary.

SUMMARY OF THE INVENTION In view of the above, it is an object of the present invention to provide communication between a speaker and a user by simultaneously presenting the user with linguistic information and non-linguistic information. It is another object of the present invention to simultaneously present non-verbal information based on reading and sign language and linguistic information as a result of voice recognition to a user according to the characteristics of the user and the performance of the voice recognition device. It is another object of the present invention to enable smooth communication, especially for a prematurely hearing-impaired person who has a low ability to understand sentences by reading and sign language.

[0010]

According to a first aspect of the present invention, there is provided a voice display system for assisting communication, comprising an operation for recognizing an input voice, converting the input voice into linguistic information, and outputting the linguistic information. Device, and a transmissive display device that displays the linguistic information and includes a transmissive unit for obtaining non-linguistic information from a speaker, wherein the arithmetic device includes a voice input unit that inputs the voice, A voice recognition unit for recognizing a voice input from a voice input unit, a layout setting unit for setting a display state on the transparent display device, and a voice recognition unit according to the setting of the layout setting unit. An output unit that outputs a result of voice recognition to the transparent display device.

According to a second solution of the present invention, an arithmetic unit for recognizing an inputted voice, converting it into linguistic information and outputting the same, and displaying the linguistic information and obtaining non-linguistic information from a speaker A voice display program used for a voice display system for assisting communication with a transmission type display device including a transmission portion for the voice input procedure for inputting the voice, According to a preset threshold value for the likelihood, when the likelihood is lower than the threshold value, the character string to be displayed is recognized as being displayed as non-characters. A voice display procedure for causing a computer to execute a voice recognition procedure for recognizing the read voice and an output procedure for outputting the result of voice recognition by the voice recognition procedure to the transparent display device. To provide the ram.

Further, the present invention displays language information (character string), which is speech recognition result data including an error, on a transmissive display device (glasses display) so that a user using the transmissive glasses display can use the information. In addition, non-verbal information including movement of facial parts such as the lips and eyes of the speaker, reading, gesture, sign language, facial expressions, etc., as well as the character string displayed in front of the speaker can be simultaneously viewed. Thereby, even if the user is a hearing impaired person, it is easy to understand the meaning of the speaker, and smooth communication between the user and the speaker can be achieved.

[0013]

Embodiments of the present invention will be described below in detail with reference to the drawings. FIG. 1 is a schematic configuration diagram of a voice display system 100 according to the present invention. The audio display system 100 includes, for example, an arithmetic device 1 that is a portable computer and a transmissive display device 2 that is a transmissive glasses display. The voice display system 100 is a speaker 1
This is a system for assisting communication between 0 and a user (for example, a hearing-impaired person) 40. The arithmetic unit 1 recognizes speech input by the speaker 10 via a microphone (not shown) or the like, converts the speech into verbal information (verbal information, a character string in this case), and uses the result of the speech recognition. A certain character string is output to the transmissive display device 2.
The transmissive display device 2 displays a character string, which is linguistic information input by the arithmetic device 1, and transmits non-verbal information (non-verbal information, for example, speaker 1) from the speaker 10.
0 lips, eyes and other facial movements, reading, gestures,
Including one or more of sign language and facial expressions: a transparent portion (transmissive display) for obtaining a sign (dotted arrow in the figure) is provided.

The arithmetic unit 1 includes, for example, a voice input unit 50,
A processing unit (CPU) 55, a voice recognition unit 60, a layout setting unit 90, and an output unit 95 are provided. The voice input unit 50
The voice of the speaker 10 is input. The speech recognition unit 60 performs, for example, speech recognition of the speech input from the speech input unit 50, and includes a database selection unit 70 and a likelihood threshold setting unit 80.

The database selection unit 70 may include, for example, one or a plurality of language databases (Kanji D) provided in the speech recognition unit 60 or the speech display system 100 as appropriate.
The degree of difficulty of the kanji stored in advance in B) (for example, the second grade of elementary school, the level of junior high school, etc., according to the JIS standard level) is determined by the vocabulary of the user 40 (how much kanji should be created. Will be determined). Thereby, when recognizing the voice of the speaker 10, a character string including kanji according to the vocabulary of the user 40 can be created. If a kanji corresponding to Hiragana is not recognized in the language database, all character strings are displayed on the transmissive display of the transmissive display device 2 as Hiragana and / or non-characters (for example, symbols).

When speech recognition is performed on a continuous speech from the speaker 10 stored in a speech database (not shown) using a language database, the speech recognition section 60 performs a word sequence and the likelihood of each word. Degree (for example, the likelihood of connection between words, which may be regarded as the distance between words; therefore, a high likelihood and a small distance between words are generally synonymous). Obtained as a recognition result.

The likelihood threshold setting unit 80 sets a threshold corresponding to the likelihood in order to present only a character string that has been correctly recognized among the recognition results of the speech recognition unit 60, for example. Specifically, when the likelihood of each word is high, it is assumed that the word is likely to be correctly recognized, and when the likelihood is low, it is assumed that the word is not likely to be correctly recognized. Therefore, by setting a threshold in the likelihood threshold setting unit 80, if the log likelihood of each word is larger than a certain threshold with respect to the word sequence of the recognition result, this word is presented, and the log likelihood is set to the threshold. If it is smaller, this word may not be presented (specific numerical values of the threshold will be described later). In addition, the speech recognition unit 60 not only presents a word whose log likelihood is smaller than a threshold value (the probability of connection between words is small and the distance between words is large).
It can be set appropriately so as to be displayed on the transmissive display of the transmissive display device 2 as non-characters such as symbols.

The layout setting section 90 sets the display state on the transmissive display device 2. For example, a layout character setting section 91, a color adjustment setting section 92, a size adjustment setting section 93, and a display section. A position adjustment setting unit 94 is provided. The number-of-presented-characters setting unit 91 can appropriately adjust the number of presented characters of a character string output from the arithmetic device 3 and displayed on the transmissive display device 2, for example. In addition, the presented character number setting unit 91 includes, for example, a proficiency folder indicating a correspondence between the proficiency of the reading and / or sign language of the user 40 and the number of presented characters (see FIG. 2).

Similarly, the color adjustment setting section 92 can appropriately adjust the color of the character string. Size adjustment setting section 9
3 can similarly adjust the size of the character string appropriately. Similarly, the display position adjustment setting unit 94 appropriately adjusts the display position of the character string and the focal length of the display (for example, the focal length at which the character string is displayed can be adjusted according to the distance of the speaker 10). Can be. The layout setting unit 90
The settings of the various setting units included in are set as appropriate by the user 40 himself (solid arrows in the figure). Thus, the layout of the character strings displayed on the transmissive display device 2 can be changed to a state desired by the user 40. The output unit 95 outputs a character string (verbal information) as a result of voice recognition by the voice recognition unit 60 to the transparent display device 2 according to the setting of the layout setting unit 91 by the user 40.

FIG. 2 is an explanatory diagram of the proficiency level folder 20. The proficiency level folder 20 is included in the number-of-presented-characters setting unit 91 in the layout setting unit 90 as described above.
For example, the reading and / or sign language proficiency 21 of the user 40
And the number of characters to be presented (ratio) 22. Here, "low, 80%", "normal, 60%", and "high, 40%" are stored in advance as the correspondence between the proficiency level 21 and the (number of) presented characters 22. It should be noted that the proficiency level 21 of this proficiency level folder 20 and the number of presented characters (ratio) 2
2 can be set as appropriate.

The user 40 (for example, a hearing-impaired person) uses the transmissive display of the transmissive display device 2 to make the speaker 10
Not only a character string (verbal information) as a result of speech recognition of the voice of
Non-verbal information can also be obtained. The user 40 inputs the proficiency level 21 of the user 40 into the arithmetic unit 1 using, for example, a proficiency level setting button (not shown). It should be noted that the proficiency level 21 of the user 40 himself is
, Not only the user 40 but also the maintenance person, family,
The setting may be made by an appropriate person such as a doctor.

If the user 40 has a high level of proficiency 21 in reading and sign language (this level of proficiency has a large individual difference),
It is assumed that communication with the speaker 10 will be performed smoothly by extracting nonverbal information. In this case, the character string functions as an auxiliary (or confirmation) for the user 40 (here, “high, 40
% ").

On the other hand, when the user 40 has a low proficiency level 21 in reading and sign language, for example, it is assumed that it is difficult to communicate with the speaker 10 using only nonverbal information. In this case, the user 40
It relies on verbal information to communicate with speaker 10 (here, "low, 80%"). In addition,
The voice display system 100 according to the present invention can also be applied to a kind of training system for improving the proficiency level 12 in reading and sign language of a hearing-impaired person. In particular,
As the proficiency level 21 for reading and sign language of hearing-impaired persons increases, the number of presented characters (ratio) 22 decreases, or the number of presented characters (ratio) 22 increases to confirm the proficiency level 21. May be.

FIG. 3 is a schematic explanatory diagram showing a use state of the audio display system 100 according to the present invention. A user 40 (not shown) in the audio display system 100 wears the transmissive glasses display 2. The user 40 speaks not only the verbal information (here, “character string or the like,..., Etc.”) displayed on the transmissive glasses display 2 but also the story acquired via the transmissive glasses display 2. Communication with the speaker 10 is performed using the non-verbal information (here, facial expression, mouth movement, gesture) of the speaker 10. In the figure, the verbal information is the speaker 10
Although drawn on top of this, this
, Etc., the “character string or the like, etc.” displayed on the transmissive glasses display 2 is the speaker 10
Because it appears to be displayed in front of the.

FIG. 4 is a flowchart of the voice display system 100 according to the present invention. First, the number-of-presented-characters setting unit 91 sets the (number of) presented characters 22 based on, for example, information on the proficiency level 21 of the user 40 input via a proficiency level setting button (not shown) (S20).
1). Here, the setting of the likelihood threshold and / or the selection of the kanji DB can be performed as necessary. Next, various settings of the color adjustment setting unit 92, the size adjustment setting unit 93, and the display position adjustment setting unit 94 included in the layout setting unit 90 are performed (S203). The voice from the speaker 10 is input to a voice input unit 50 via a microphone (not shown).
(S205).

The voice input in step S205 is voice-recognized by the voice recognition unit 60 (S207). The verbal information, which is the speech recognition result data in step S207, is output to the transmissive display device 2 via the output unit 95 (S209). Change of verbal information output to transmissive display device 2 (here, setting of (number of characters to be presented) 22; setting of likelihood threshold if necessary and / or selection of kanji DB; furthermore, step S203) It is determined whether various settings have been input (S211). When a change in verbal information is input from step S211, step S201 and / or step S201 are performed again.
Returning to 203, the setting of (the ratio of) the presented characters 22 and the setting of the likelihood threshold and / or the selection of the kanji DB as necessary, and the various settings in step S203 are performed. If a change in verbal information has not been input from step S211, a series of processing ends.

FIG. 5 is a view showing the results of an experiment which objectively shows the understanding of the meaning by the subject. However, here, the experiment shown in the conventional technology (see: Miki Saito, research on man-man interface using speech recognition technology for hearing loss; Master's thesis, Graduate School of Engineering, Hokkaido University, 19992) was performed. Therefore, the precondition of the present invention is that a human being has, for example, a character string recognition rate of 60%
If the degree is secured (incomplete verbal information), the character string can be inferred from the context and / or the recognition result before and after, and communication can be achieved as a result. " According to the idea of simultaneous presentation of non-verbal information by the present inventors, communication can be performed more smoothly. "

FIG. 5A is a diagram showing a change in the log likelihood threshold value and the presentation sentence. This log likelihood threshold and the change of the presentation sentence 3
0 includes, for example, the likelihood 30 and the presentation sentence 32. The speech recognition unit 60 calculates the recognition word and its likelihood. Next, only the recognized words corresponding to the likelihood larger than the preset threshold are presented. Note that words corresponding to likelihoods smaller than the threshold are displayed as non-characters (here, “?”). As a result, as the correspondence between the likelihood 30 and the presentation sentence 32, the number of words to be presented is increased by reducing the log likelihood threshold as illustrated.

FIG. 5B is a diagram showing a change in the accuracy of sentence understanding according to the log likelihood threshold. As a result of the experiment, as shown in the figure, the sentence comprehension is performed when the log likelihood threshold is “−2500”, that is, when the word recognition accuracy exceeds about 40%, common to all the subjects (A to F). Accuracy is increasing rapidly. In addition, individual differences are large among the hearing-impaired persons, and this is considered to be one of the factors due to the difference in the number of acquired vocabularies.

As described above, the precondition of the present invention is that “human is
For example, if the recognition rate of the character string of the voice recognition result is about 60% (incomplete verbal information), the character string can be inferred from the context and / or the recognition result before and after, and as a result communication can be achieved. "I can do it" objectively.

FIG. 6 is a diagram showing experimental results in the audio display system 100 according to the present invention. In this experiment, the incomplete verbal information obtained from the arithmetic unit 1 and the speaker 1
The non-verbal information obtained from 0 is simultaneously presented to the user 40 wearing the transmissive display device 2 to change the sentence comprehension.

In this experiment, three Japanese males aged 23 to 30 years without hearing impairment were used as subjects before having the hearing impaired cooperate. The three subjects have not received any particular training in reading so far. Further, the sentence to be presented was the same as the presented sentence 32 shown in FIG.

The non-verbal information presented together with the presentation sentence 32 includes digital video (Victor: GR-D).
(V1, 570,000 pixels). This video was taken by a 23-year-old Japanese man who read the correct sentence before voice processing and focused on the face at that time. This video was subjected to digital processing by a PC and superimposed on the subtitles of the presented text. The text subtitles were superimposed on the speaker's mouth after the speaker finished speaking and the mouth stopped moving. If the original sentences are the same,
Difference of likelihood in four stages (corresponding to plot position shown in FIG. 6)
Regardless, the video of the same face was used.

As an experiment, a first experiment (verbal information) for examining a change in semantic understanding of only verbal information and a second experiment for examining a change of semantic understanding in a state where not only verbal information but also a video sample, that is, non-verbal information are added, are performed. Perform two experiments. The contents of the first experiment are the same as those in FIG. 5, and the description is omitted. Here, the subjects were randomly divided into two groups A and B according to the original sentences. The subjects in groups A and B presented 100 sentences divided into four levels of likelihood for each of 25 sentences. Subjects in each group, for example, read presentation texts printed on paper in order, and if they could understand the meaning, responded to the content they understood. At that time, using a personal computer to which the test subject was accustomed, the text editor was hit with a keyboard. Note that the subject is instructed in advance so as to proceed in the order instructed and not to skip the presentation sentence or return to the previous presentation sentence.

In the second experiment (verbal information + non-verbal information), a transmission type HMD was used to present an image to a subject.
(OLYMPUS: Mediamask) was used. The subject plays the digital video, attempts to read from the facial image, then pauses when the subtitles appear, and once he understands the contents of the sentence, he types it into the text editor on the personal computer with the keyboard as before. I let you.

As an experimental procedure, the group A of the first experiment was tried, the second experiment was performed, and finally, the group B of the first experiment was tried. In addition, a break was inserted between each experiment according to the judgment of the subject. This experimental procedure was used in order to reduce any habitual use of the sentence caused by repeating the experiment because the original sentence was the same, and the result was easily affected by the concentration and motivation of the subject. In order to reduce some.

The graph shown in the figure shows the experimental result of the subject A. The horizontal axis of the graph is the log likelihood threshold (word recognition accuracy), and the vertical axis is the sentence understanding accuracy (%). The solid line with the square marker is the sentence comprehension accuracy in the second experiment. The solid line with the triangular marker is the average of the results of the groups A and B in the first experiment.

In the subject A, the log likelihood threshold is -2000
From -3000 (approximately 40% in recognition rate), the sentence comprehension accuracy sharply improves. In addition, when the log likelihood threshold value of the subject A decreases, the sentence comprehension accuracy generally increases. That is, in the subject A, the improvement of the sentence comprehension accuracy by adding the non-verbal information is clearly shown.

As described above, according to the audio display system 100 of the present embodiment, the linguistic information and the non-linguistic information by the speaker are simultaneously presented to the user, thereby assisting the communication between the speaker and the user. be able to. Also,
According to the characteristics of the user and the performance of the speech recognition device, non-verbal information based on reading and sign language and linguistic information as a speech recognition result can be simultaneously presented to the user. In addition, according to the voice display system 100, smooth communication can be achieved even for a mid-hearing person who has a low ability to understand a sentence in reading and sign language.

A voice display system according to the present invention includes a voice display method including a function for realizing each unit, a voice display program for causing a computer to execute each procedure, and a computer-readable recording medium storing the voice display program. , A program product that includes a voice display program and can be loaded into an internal memory of a computer, a computer such as a server including the program, a voice display device, or the like.

[0041]

According to the present invention, as described above, communication between the speaker and the user can be assisted by presenting the linguistic information and the non-linguistic information of the speaker to the user at the same time. Further, according to the present invention, non-verbal information based on reading and sign language and linguistic information as a voice recognition result can be simultaneously presented to the user according to the characteristics of the user and the performance of the voice recognition device. Also, the present invention, for example,
Even a half-hearing person who has a low ability to understand sentences by reading and sign language can communicate smoothly.

[Brief description of the drawings]

FIG. 1 is a schematic configuration diagram of a voice display system 100 according to the present invention.

FIG. 2 is an explanatory diagram of a proficiency folder 20.

FIG. 3 is a schematic explanatory diagram showing a use state of the audio display system 100 according to the present invention.

FIG. 4 is a flowchart of the voice display system 100 according to the present invention.

FIG. 5 is a view showing an experimental result that objectively shows the understanding of the meaning by the subject.

FIG. 6 is a view showing experimental results in the audio display system 100 according to the present invention.

[Description of Signs] 1 Arithmetic device 2 Transmissive display device 10 Speaker 40 User 50 Voice input unit 55 Processing unit (CPU) 60 Voice recognition unit 90 Layout setting unit 95 Output unit 100 Voice display system

 ────────────────────────────────────────────────── ─── Continuation of front page (72) Inventor Tatsu Ifukube F-term (reference) 5D015 KK02 LL03 LL05 LL07 5E501 AC37 CA37 CB15 CC11 EA01 EA21 FA45

Claims (9)

[Claims] 1. An audio display system for assisting communication, comprising: an arithmetic unit for recognizing an input voice, converting the input voice into linguistic information, and outputting the linguistic information; A transmissive display device including a transmissive unit for obtaining non-verbal information, wherein the arithmetic unit comprises: a voice input unit for inputting the voice; and a voice recognition unit for recognizing the voice input from the voice input unit. A layout setting unit for setting a display state on the transmissive display device; and an output for outputting a result of speech recognition by the speech recognition unit to the transmissive display device according to the setting of the layout setting unit. And a voice display system. 2. The method according to claim 1, wherein the layout setting unit sets one or more of the number of characters, a color, a size, a display position, and a display focal length of a character string displayed on the transmissive display device. Item 2. The audio display system according to item 1. 3. The method according to claim 1, wherein the non-verbal information includes at least one of a motion of a face portion such as a lip and an eye of the speaker, a gesture, a sign language, a reading, and a facial expression. The audio display system according to 1. 4. The layout setting section decreases or increases the number of displayed characters or the ratio thereof when the proficiency of the reading or sign language is high, and increases or reduces the number of displayed characters or the ratio thereof when the proficiency is low. 3. The method according to claim 2, wherein the values are set smaller.
Or the voice display system according to 3. 5. The audio display system according to claim 1, wherein the arithmetic device is a portable computer, and the transmission type display device is a glasses display. 6. The system according to claim 1, wherein the voice recognition unit converts the displayed character string into a kanji based on a language database selected according to a set kanji difficulty level.
6. The audio display system according to any one of claims 1 to 5. 7. The speech recognition unit includes a likelihood threshold setting unit that sets a likelihood threshold of a character string that is speech-recognized, and sets a likelihood threshold according to a threshold set in advance by the likelihood threshold setting unit. 7. The voice display system according to claim 1, wherein when the degree is lower than the threshold value, the displayed character string is displayed in non-characters. 8. A transmissive display including an arithmetic unit for recognizing input speech, converting it into linguistic information and outputting the same, and a transmissive unit for displaying the linguistic information and obtaining non-linguistic information from a speaker. A voice display program used for a voice display system for assisting communication with a device, comprising: a voice input procedure for inputting the voice; and a preset threshold value for the likelihood of a voice-recognized character string. When the likelihood is lower than the threshold value, a voice recognition for recognizing voice input from the voice input procedure is performed so as to recognize a displayed character string as non-character. A voice display program for causing a computer to execute a procedure and an output procedure of outputting a result of voice recognition by the voice recognition procedure to the transparent display device. 9. The voice according to claim 8, wherein the voice recognition step converts the displayed character string into Chinese characters based on a language database selected according to the set difficulty level of Chinese characters. Display program.