JP5340059B2 - Character information presentation control device and program - Google Patents

Description

  The present invention relates to a character information presentation control apparatus and program, and more particularly to a technique for presenting character information in a form that does not move the line of sight more than necessary to a person who visually recognizes character information.

  2. Description of the Related Art Conventionally, there is known an apparatus that presents character information by devising a display form so that character information can be easily read for a person who visually recognizes character information displayed on the screen (viewer) (Patent) Reference 1). This device inputs the voice signal of the speaker, converts the voice signal into character information, measures the volume and tone of the uttered voice, and displays the volume level and tone level in the form of character information. To do. Thereby, the viewer who saw the character information can visually understand the emotion of the speaker by the form of the character information. In this way, by presenting the character information with a devised display form, it is possible to convey the meaning reflected in the form of the character information to the viewer.

  On the other hand, in video production for TV programs, movies, education, enlightenment, etc., when an utterer such as an announcer, actor, teacher, or politician speaks to the camera while watching text information A method for presenting character information to a person is known. For example, there is a method using a campaign. This method presents text information written in a sketch book or the like held by the staff near the lens of the photographing camera to the speaker (see Non-Patent Documents 1 and 2). Furthermore, there is a display device that presents character information to a speaker by a display device installed near the lens of the photographing camera (see Patent Document 2).

  There is also a technique for presenting character information to a speaker using a prompter. This method presents text information to the speaker by projecting the text information to be presented to the speaker to a half mirror provided in front of the lens of the photographing camera (speaker side) via a display device. On the other hand, a speaker is photographed by transmission through a half mirror (see Patent Document 3). In addition, in a method using a prompter, there has been proposed a device that uses a screen that can electrically control the transmission state of light and that performs shooting or display in a time-sharing manner according to the transmission or non-transmission state (Patent Document). 4 and 5). As a result, it is possible to photograph the speaker by transmitting the screen while presenting the character information to the speaker.

  There is also a method of presenting character information using speech recognition (see Patent Documents 6, 7, and 8). The character information presentation method of Patent Document 6 adds an effect to a character string on a presentation screen. The character information presentation method of Patent Document 7 relates to the operation of the information presentation device by voice recognition. The character information presentation method of Patent Document 8 controls the display speed of subtitles and screens by voice recognition.

JP 2005-274791 A Japanese Patent No. 3478308 Japanese Patent No. 3565998 Japanese Patent Laid-Open No. 4-96581 JP-A-4-11485 JP 2005-208292 A JP-A-11-184671 JP 2003-29779 A

“Free encyclopedia“ Wikipedia ””, [online], [searched June 8, 2009], Internet <URL: http://en.wikipedia.org/wiki/campe> “Video Industry Glossary”, [online], [Search June 8, 2009], Internet <URL: http://www.geocities.jp/amukoris/dictionary01aka.htm#ka>

  The character information presentation control device according to Patent Document 1 described above visually conveys the speaker's emotion and the like to the viewer by reflecting the speaker's emotion and the like in the form of the character information. However, since the character information presented to the viewer is displayed from the left to the right and from the top to the bottom of the screen, in order for the viewer to recognize the character information, the line of sight depends on the position of the character information. It was necessary to move up and down.

  Also, in the method of presenting text information for the purpose of video production, in the method using a campaign, the campaign is placed near the lens of the camera, so that the speaker's line of sight is visible when the speaker visually recognizes the text information. However, there is a problem that the line of sight of the photographed speaker becomes unnatural. In addition, when the characters written on the campaign are handwritten, the characters are difficult to read and may be misread. In addition, there is a problem that it is difficult to read the characters written on the campaign because the vicinity of the photographic camera is usually dark with little light. Similarly, in the method using the display device according to Patent Document 2, when the speaker visually recognizes the character information, there is a problem that the line of sight is shifted and becomes unnatural.

  Further, in the method using the prompter of Patent Document 3, a hand manuscript read by a speaker is photographed by a prompter camera installed above the speaker, and an image of the hand manuscript captured by the prompter camera is captured by the photographing camera. Reflected by a half mirror installed in front of the lens and displayed on the optical axis of the taking lens. However, since the manuscript read by the speaker is composed of various formats such as vertical writing or horizontal writing, the line of sight of the speaker reading the manuscript follows characters arranged according to the format. For this reason, there has been a problem that the movement of the line of sight of the speaker is reflected on the photographing camera and the line of sight of the speaker becomes unnatural. In addition, the manuscript read by the speaker needs to be fixed on the desk or the like, but since a standing speaker cannot fix the manuscript, a prompter is used for a standing speaker. It was difficult.

  Similarly to the method using the prompter of Patent Document 3 in the methods using the devices of Patent Documents 4 and 5, the displayed character information is laid out on a plane in the screen, so that the line of sight of the speaker reading the manuscript is read. Had the problem of chasing characters and making the speaker's gaze unnatural. Further, in the apparatuses of Patent Documents 4 and 5, since the transparency of the screen is temporally controlled, there is a problem that the display image is projected intermittently and the display image becomes dark. Furthermore, since it is necessary to install a projection apparatus or to perform a light shielding treatment to prevent reflected light from the screen surface, there is a problem that the apparatus becomes large as a whole.

  The method of Patent Document 6 adds a scheduled effect based on an utterance to character information on a presentation screen for viewing by many people. On the other hand, the present invention presents character information in a form that does not move the line of sight more than necessary to a person who visually recognizes the character information. Therefore, the method of Patent Document 6 is basically different from the present invention in that it does not achieve the purpose of minimizing the movement of the line of sight.

  Since the technique of Patent Document 7 relates to a user interface using a user's voice, it is basically different from the present invention. The technique of Patent Document 8 relates to the reproduction control of character information using the speech recognition of the speaker, and presents the screen controlled by the speaker's speech to the speaker and a third party. . Therefore, this method is also fundamentally different from the present invention.

  Accordingly, the present invention has been made to solve the above-described problems, and the object thereof is to present character information that can reduce the line-of-sight movement of a person who visually recognizes character information and can improve the visibility of character information. To provide a control device and a program. It is another object of the present invention to provide a character information presentation control apparatus and program capable of presenting character information suitable for the utterance timing of the utterer with little movement of the sightline of the speaker in video production of the utterer. .

In the present invention, in order to solve the above-described problems, the following configuration is adopted. According to the first aspect of the present invention, in the character information presentation control device that controls the character information to be arranged in the virtual space, displayed on the screen, and presented to the user, the character information is divided to generate divided data for each phrase. The character information dividing unit to be divided, and the divided data divided by the character information dividing unit are set as a character group of a predetermined unit, and the number of mora is calculated from the character information of the character group unit for the character group unit, Based on the number of mora, an interval between consecutive character groups is set, and character information for each character group is centered from the back side to the near side in the virtual space at the center of the screen when viewed from the user , and a predetermined speed and After sequentially scrolling at the intervals, a new character group appears from the back side of the center, and the character group is scrolled toward the front side. Characterized by comprising character information presentation unit to eliminate the-loop.

  In the character information presentation control device according to claim 1, the character information presentation unit may display the character information in units of character groups in a superimposed manner before and after the character group. It is presented and presented at a character display rate so that character information located on the back side is transmitted.

  The character information presentation control device according to claim 2 is characterized in that the character display rate gradually increases from 0% toward the back side and is lower than 100%. It is set to reach a predetermined maximum value and gradually decrease to reach 0%.

Further, the invention of claim 4 is the character information presentation control device according to any one of claims 1 to 3 , wherein the character information presentation unit, when the end of the character group is a punctuation mark, A new character group is generated by inserting a blank at the beginning of the character group, and centering is performed on the new character group.

The invention according to claim 5 is the character information presentation control device according to any one of claims 1 to 4 , further comprising a mora dictionary in which a word and the number of mora of the word are stored, The character information presentation unit calculates the number of mora of words included in the character information, and registers the word and the number of mora in the mora dictionary.

Further, the invention of claim 6 is the character information presentation control device according to any one of claims 1 to 5 , further comprising an operation input unit for outputting a start control signal in accordance with the key operation of the user. The character information presentation unit inputs a start control signal from the operation input unit, starts the scroll at the input timing of the start control signal, and stops the scroll at a predetermined position of the character information. Features.

Further, the invention according to claim 7 is the character information presentation control device according to any one of claims 1 to 5 , and further outputs start and stop start control signals according to the user's key operations, respectively. An operation input unit, wherein the character information presentation unit inputs a start control signal from the operation input unit, and when the start control signal is a start signal, the scrolling is started, and the start control signal is a stop signal In this case, the scrolling is stopped.

The invention according to claim 8 is the character information presentation control device according to any one of claims 1 to 5 , wherein the user is a speaker who reads out the character information, and further, the speech by the speaker And a start control unit that outputs a start control signal according to the progress of the utterance in the character information based on the word sequence, and the character information presenting unit includes the start control unit A start control signal is input, scrolling for each character group is started at the input timing of the start control signal, and the scroll is stopped at a predetermined position of the character information.

Further, the invention of claim 9 is the character information presentation control device according to claim 8 , wherein the start control unit has a frequency between an audio signal of an utterance by the speaker and an audio frequency pattern recorded in advance. Matching and using a word dictionary stored in advance, speech recognition means for converting the speech signal into a word string, a word string converted by the speech recognition means, and a word string constituting the character information, The start control signal is output when it is determined that the start of a continuous character group constituting one sentence among the plurality of character groups constituting the character information is uttered by the speaker Action determining means, and the character information presenting section performs the scrolling for each sentence and inputs the start control signal from the start control section. Scrolling is started, and when the beginning of the continuous character group constituting the next sentence is displayed on the front side of the screen, and the next character group is displayed toward the back side, the scrolling is performed. It is characterized by stopping.

The invention according to claim 10 is the character information presentation control device according to claim 9 , wherein the one sentence has a punctuation mark at the end thereof.

Further, the invention of claim 11 is the character information presentation control device according to claim 8 , wherein the number of character groups displayed when the scrolling is stopped is the number of character groups displayed by the speaker. Depending on the delay time from the start of the reading to the start of the next scroll, so that the next scrolling is started before the reading of all the displayed character groups is completed. It is characterized by a number.

The invention of claim 12 is a character information presentation control program for causing a computer to function as the character information presentation control device according to any one of claims 1 to 11 .

  According to the present invention, character information in units of character groups is centered, sequentially scroll-displayed at a predetermined speed and a predetermined interval, a new character group appears from the back side in the center of the screen, and scroll-displayed toward the front side. Later, the character group disappeared. Thereby, it is possible to reduce the line-of-sight movement of the person viewing the character information and improve the visibility of the character information.

  Further, according to the present invention, the voice signal of the utterance by the speaker is converted into a word string, and the start control signal is generated from the word string to determine the progress of the utterance. The display is started and the scroll display is stopped at a predetermined position of the character information. As a result, in the video production of the speaker, it is possible to present character information suitable for the speaker's speech timing with little movement of the speaker's line of sight.

It is a figure explaining the example of the usage condition of a portable information terminal provided with the character information presentation control apparatus of Example 1. FIG. It is a block diagram which shows the structure of the portable information terminal containing the character information presentation control apparatus of Example 1. FIG. It is a flowchart explaining the character information presentation process of Example 1. FIG. It is a flowchart explaining the character information presentation control of Example 1. FIG. It is a figure which shows the example of character information. It is a figure which shows the example of division | segmentation data. It is a figure which shows the example of a character group. It is a figure which shows the positional relationship between the character group and virtual camera in virtual space. It is a figure which shows the space | interval of a character group, and a character display rate. It is a figure which shows the example of presentation of character information. It is a figure explaining the example of the usage pattern at the time of imaging | photography using the character information presentation control apparatus of Example 2. FIG. It is a block diagram which shows the structure of the character information presentation control apparatus of Example 2. FIG. It is a flowchart explaining the character information presentation process of Example 2. It is a flowchart explaining the character information presentation control of Example 2. It is a block diagram which shows the structure of a speech recognition means. It is a figure explaining the time relationship of utterance, speech recognition, and character information presentation.

  Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings. In the character information presentation control apparatus used in a situation not intended for shooting, the character information is centered on a predetermined character group unit according to a user operation, and the first embodiment has a predetermined size, interval, speed, and character display rate. The screen is continuously scrolled from the back side to the front side in the center of the screen. As a result, the person who visually recognizes the character information can recognize the centered character information in units of a predetermined character group in order as it moves from the back side to the near side in the center of the screen. Therefore, the visibility of character information can be improved. Further, in the second embodiment, in the character information presentation control apparatus used in a situation where photographing is intended, the character information is scroll-displayed in the same manner as in the first embodiment, and the reading of the speaker who reads the document is read out. In accordance with the progress timing, the character information of the manuscript is displayed in one sentence unit corresponding to the utterance break. Thereby, the gaze movement of the speaker is reduced, and the character information suitable for the speaker's utterance timing can be presented. Hereinafter, Examples 1 and 2 will be described in detail. Hereinafter, the character information is a character that is visually recognized by the user or a character that is uttered by the speaker, and means information that includes characters including punctuation marks and symbols.

[Example 1]
First, Example 1 will be described. The character information presentation control device according to the first embodiment is a device that presents character information in a situation where shooting is not intended. The character information presentation control device divides the entire character information presented in accordance with user operations into character information in predetermined character groups. And has a function of scrolling from the back side of the center of the screen toward the front side in units of character groups at a predetermined size, interval, speed, and character display rate.

  FIG. 1 is a diagram illustrating an example of a usage pattern of a portable information terminal including the character information presentation control device according to the first embodiment. The portable information terminal 2 has a small display and operation keys. According to an operation by the user 3, a portable information terminal 2 creates a document and transmits / receives an e-mail, or inputs a keyword to search a Web screen. And a character information presentation control device 1-1 (not shown) according to the first embodiment. The portable information terminal 2 outputs and displays character information such as received e-mail and searched Web data on a small display via a character information presentation control device 1-1 (not shown). The user 3 can visually recognize character information such as e-mail and Web data by using the small display of the portable information terminal 2.

  FIG. 2 is a block diagram illustrating a configuration of the portable information terminal 2 including the character information presentation control device 1-1 according to the first embodiment. The portable information terminal 2 includes a communication unit 60, a character information presentation control device 1-1, and a display unit 70.

  The communication unit 60 has a function of transmitting an e-mail created by the user 3, a function of receiving an e-mail, a function of transmitting a search word specified by a key operation of the user 3 and receiving Web data, and the like. . In addition, the communication unit 60 receives e-mail, Web data, and the like from the outside, extracts character information to be presented to the user 3, and outputs the character information to the character information presentation control device 1-1. The extracted character information is converted into character information of a predetermined form in the character information presentation control device 1-1 and output to the display unit 70 as a video signal of character information. The user 3 visually recognizes character information such as e-mail and web data received by the portable information terminal 2 on the display unit 70.

(Character information presentation control device)
The character information presentation control device 1-1 includes a storage unit 10, a character information dividing unit 20, an operation input unit 30, and a character information presentation unit 40. The character information presentation unit 40 includes a presentation control unit 41, a real-time CG drawing unit 42, and a mora dictionary 43. The character information presentation control device 1-1 receives character information from the communication unit 60, converts it into character information of a predetermined form, and outputs it to the display unit 70 as a video signal.

  FIG. 3 is a flowchart illustrating the character information presentation processing according to the first embodiment, and FIG. 4 is a flowchart illustrating the character information presentation control according to the first embodiment. Hereinafter, processing of each component of the character information presentation control device 1-1 shown in FIG. 2 will be described with reference to the flowcharts of FIGS.

(1. Preparation of character information)
The character information presentation control device 1-1 receives character information from the communication unit 60 and stores it in the storage unit 10 (step S301). FIG. 5 is a diagram illustrating an example of character information. The storage unit 10 stores, for example, the text information shown in FIG. 5 "Firstly, the establishment of a tax-related law that maintains a provisional tax rate such as gasoline tax."

(2. Generation of divided data)
The character information dividing unit 20 includes a Japanese dictionary (not shown), reads character information from the storage unit 10, and uses the Japanese dictionary to divide the character information into phrases and words and convert them into divided data. The divided data (phrase, word) is output to the character information presentation unit 40.

  Specifically, the character information dividing unit 20 uses an SVM (Support Vector Machine) learned by referring to information such as words, parts of speech, and paragraph breaks registered in a Japanese dictionary by a Japanese dependency analysis method. Using the pattern recognition method, the input character information is divided into phrases composed of a plurality of continuous words, and a marker such as <b> is inserted between the phrases (step S302). As a Japanese dependency analysis method, for example, a known Japanese dependency analyzer (see “CaboCha”, <http://chasen.org/~taku/software/cabocha/>) is used. In addition, the character information dividing unit 20 refers to word information registered in the Japanese dictionary by using a morphological analysis method, and divides the input character information into minimum unit morphemes (words), and between the words. A blank is inserted (step S303). As a morphological analysis method, for example, a known morphological analyzer (see “ChaSen”, <http://chasen-legacy.sourceforge.jp/>) is used.

  FIG. 6 is a diagram illustrating an example of divided data in a case where character information is divided into clauses and words. As shown in FIG. 6, the word is sandwiched between blanks (spaces), and the phrase is sandwiched between <b> markers. As described above, the character information dividing unit 20 inserts identification information such as a space before and after each phrase and word constituting the character information, and outputs divided data including the phrase, the word, and the identification information.

(3. Preparation for presentation of text information)
Next, the character information presenting unit 40 receives the divided data from the character information dividing unit 20 and prepares to present character information for outputting the character information to the display unit 70 as a video signal. In steps S304 to S306, a dictionary registration process for registering words constituting the divided data in the pronunciation dictionary (not shown in FIG. 2) and the mora dictionary 43 is performed. In step S307 to step S312, the divided data is divided into character groups for each phrase, and processing such as arranging the character groups for each phrase in the three-dimensional virtual space is performed.

(3.1 Pronunciation dictionary registration, mora dictionary creation)
The real-time CG drawing means 42 of the character information presenting unit 40 inputs the divided data from the character information dividing unit 20, takes out a word sandwiched between blanks included in the divided data, extracts the extracted word, the pronunciation dictionary, and the mora dictionary 43. Are compared with the words that have already been registered, and unregistered words are extracted (step S304). Then, the real-time CG drawing means 42 registers the extracted unregistered word in the pronunciation dictionary (step S305). Further, the real-time CG drawing means 42 creates a new mora dictionary 43 by manually or automatically setting the number of mora of the extracted unregistered word and registering the unregistered word and its mora number in the mora dictionary 43. (Step S306). The mora dictionary 43 stores a word and the number of mora of the word corresponding to each word. Here, when setting the number of mora manually, the real-time CG drawing means 42 outputs an unregistered word to the display unit 70 and registers the mora number input by the user's 3 key operation in the mora dictionary 43. On the other hand, when setting the number of mora automatically, phonemes (vowels, repellent sounds, prompting sounds and long vowels) are identified from unregistered words, and the number of mora is calculated by counting the phonemes, and the calculated number of mora is not registered. It is registered in the mora dictionary 43 together with the word. The number of mora is the number of beats necessary to pronounce a word, that is, the number of sound breaks. The mora dictionary 43 created in this way is used in step S309 described later.

(3.2 Character grouping)
The real-time CG drawing means 42 removes the blanks inserted between the words from the input divided data, and sets the phrase sandwiched between the markers <b> as one character group (step S307). ). FIG. 7 is a diagram illustrating an example of a character group. As shown in FIG. 7, the character group is a character string sandwiched between angle brackets. For example, the character group of the phrase “first” is composed of the characters “first” and the punctuation mark “,”. As described above, the real-time CG drawing unit 42 acquires a phrase from the divided data and sets a character group for each phrase. Note that the real-time CG drawing means 42 may set a character group of a predetermined unit instead of a character group for each phrase. A character group in a unit longer than the phrase may be set, or a character group in a unit shorter than the phrase may be set. In short, it may be a character group in a unit that is easy for the user 3 to visually recognize.

(3.3 Punctuation processing for centering)
The real-time CG drawing means 42 performs punctuation processing for character group centering (step S308). Specifically, the real-time CG drawing unit 42 determines whether or not the end of the character group is a punctuation mark such as “.” Or “,” for the character group. In the character group virtual space arrangement processing in step S310, half-width spaces are inserted at the beginning of the character group in order to center character information other than punctuation marks in the character group. This is because if you do not insert a single-byte space, even if you center the character group, character information other than punctuation in the character group will be shifted from the center to the left due to the punctuation, so insert a single-byte space. Thus, the character group composed of full-width character information is shifted to the right by a half-width as a whole, and as a result, character information other than punctuation marks is centered within the character group. For example, the real-time CG drawing means 42 determines that the end of the character group is a reading “,” for the character group “First,” and centers the character information “First” other than the reading in this character group. Insert a single-byte space at the beginning of the character group.

  On the other hand, if the real-time CG rendering means 42 determines that the end of the character group is not a punctuation mark, or the character group is removed in the character group virtual space placement process in step S310 described later, the character group is removed. When the character information included in the group is centered and arranged in the virtual space, the above-described processing for inserting a half-width space is not performed.

(3.4 Character information presentation interval setting)
The real-time CG drawing means 42 counts the number of mora for each character group, and sets the character group interval when the character groups are arranged in the three-dimensional virtual space in step S310 to be described later (step S309).

  Here, the interval between character groups is a spatial distance between a character group and the next character group to be arranged next in a three-dimensional virtual space. Since the number of character information (number of characters) included in the character group is different, the time for uttering the character group is different for each character group. For this reason, if all character groups are arranged at equal intervals in a three-dimensional virtual space, a character group with a small number of characters is presented after a character group with a large number of characters is presented. The user 3 who received the presentation feels that the interval between these character groups is short. Conversely, when a character group with a large number of characters is presented after a character group with a small number of characters is presented, the user 3 feels that the interval between these character groups is long. As described above, when all the character groups are arranged at equal intervals in the three-dimensional virtual space, there is a problem in that the sense of time changes depending on the length of the character symbol string constituting the character group. Arise. Therefore, the interval between the character groups is set based on the time required for speaking the character group, and the number of mora of the character group is used as a reference as the time required for speaking the character group.

  The real-time CG rendering means 42 extracts words from the character group with reference to the blank included in the input divided data, searches the mora dictionary 43 created in step S306, reads the number of words in the mora, and sets the character group. The number of mora of all the constituent words is summed to calculate the number of mora of the character group. The number of mora in the example of the character group shown in FIG. 7 is “first” is 2, “gasoline tax etc.” is 9, “provisional tax rate” is 9, “if you maintain” is 5, “do” is 2 "Tax-related law" is 11, and "Issued." Is 6.

  The real-time CG drawing unit 42 sets an interval between the character group and the next character group to be arranged next in accordance with the total number of mora calculated for the character group. For example, the real-time CG rendering means 42 sets the character group spacing to be proportionally larger when the character group has a larger number of mora, and proportionally increases the character group when the character group has a smaller number of mora. Set the interval between groups to be narrow. Note that the real-time CG drawing means 42 may set the interval between the character group and the front character group arranged in front based on the calculated number of mora of the character group. Further, the character group interval may be set based on the number of characters constituting the character group. Here, the interval between character groups is a ratio, and when actually presenting character information, the actual interval in the virtual space is defined according to units such as cm, pixels, grids and the like serving as a reference.

When the character group is arranged in the virtual space in step S310 to be described later, the sense of distance when the user 3 visually recognizes the character group changes depending on the scale, the shooting angle of view of the virtual camera, and the like. Therefore, the real-time CG drawing means 42 may set the character group interval by multiplying the sum of the number of mora by the coefficient k, for example, by the following equation (1).
Character group spacing = k × (total number of mora) (1)

Further, the real-time CG drawing means 42 multiplies the above-mentioned total number of mora by the interval coefficient k according to the overlapping state of the character information presented in the character information presentation processing in step S313 described later, the sense of depth, or the like. The intervals between character groups with a small number of characters or character groups with a large number of characters may be leveled. For example, the interval between character groups is set by the following equation (2).
Character group interval = k × ((total number of mora / 1.5) +2) (2)

  The real-time CG rendering unit 42 sets the character group interval using the above-described equation (2), so that the character group has a mora number less than a predetermined number, that is, for a character group having a relatively short character string. If the number of mora of the character group exceeds a predetermined number, that is, for a character group having a relatively long character string, the length may be set to be slightly wider than the interval corresponding to the shortness. It can be set slightly narrower than the corresponding interval. That is, the intervals between the character groups are leveled so that the user 3 can visually recognize the character information without feeling any trouble according to the overlapping state of the character information presented in the character information presentation processing in step S313, which will be described later, or the sense of depth. Can be

  As described above, the real-time CG drawing means 42 extracts the words from the character group and calculates the number of mora of the character group by summing up the number of mora for each word registered in the mora dictionary 43. Here, since there are infinite combinations of character information using Arabic numerals, it is difficult to register all the mora numbers in the mora dictionary 43, but the mora numbers of Chinese numerals can be registered in advance. Therefore, the real-time CG drawing means 42 may once convert the number of mora of the Arabic numerals into the Chinese numerals, read out the number of mora of the Chinese numerals from the mora dictionary 43, and add the number of mora for each Chinese numeral. Thereby, the number of mora of the character group can be calculated for any combination of numbers. For example, when “5236” is extracted from the character group, the real-time CG drawing means 42 converts the Arabic numeral “5236” into the Chinese numeral “five thousand three hundred thirty-six” and stores the number of mora from the mora dictionary 43. Are calculated as five (1) thousand (2) two (1) one hundred (3) three (2) ten (3) six (2) = 14 mora. Also, since punctuation marks are often “between” when a person reads an actual sentence, for example, “,” = 4 mora, “.” = 10 mora, and the number of mora is assigned to the punctuation marks. May be. As a result, the interval between character groups can be formulated so as to be close to the tempo when an actual sentence is read out.

(3.5 Character group virtual space layout)
Each character group is handled as one object by the user 3 or an operator who adjusts the character information presentation control device 1-1 and arranged in a three-dimensional virtual space (step S310). FIG. 8 is a diagram illustrating a positional relationship between a character group and a virtual camera in the virtual space. As shown in FIG. 8, it is assumed that a character group and a virtual camera are arranged in a three-dimensional virtual space. The video of the character group captured by the virtual camera is output as a video signal to the display unit 70, and the video is displayed. The image of the character group photographed by the virtual camera, that is, the screen displayed on the display unit 70 is displayed by a known three-dimensional graphic drawing software on the position of the virtual camera, the arrangement of the character group as a plate without thickness, the character display rate Or the like using a preset parameter such as. As shown in FIG. 8, it is assumed that a virtual camera arranged in a three-dimensional virtual space has an optical axis parallel to the bottom surface of the virtual space, or a slightly downward optical axis toward the bottom surface of the virtual space. The character group subjected to the punctuation processing in step S308 is directed from the front to the back with respect to the virtual camera so as to be perpendicular to the optical axis of the virtual camera or at a predetermined angle and at the same height from the bottom surface. The character groups are centered and arranged according to the character group interval set in step S309. The character groups arranged in this way are scroll-displayed toward the virtual camera (from the back side toward the front side).

  In this case, it is desirable that the character group be arranged so as to smoothly move up when scrolling toward the virtual camera. As described above, when the character group is arranged in parallel to the bottom surface of the virtual space, the line of sight of the user 3 may move downward when the character group is scroll-displayed on the display unit 70. However, when the character groups are arranged so as to smoothly move up, the character groups fade out near the center of the screen of the display unit 70 and disappear, and the line of sight of the user 3 does not move up and down. Thereby, the line-of-sight movement of the user 3 is further reduced, and the visibility of the character information can be further improved. When the punctuation processing of the character group is not performed in step S308 by the real-time CG drawing unit 42, the character information included in the character group from which the punctuation mark at the end of the character group is removed is arranged and centered.

(3.6 Setting of virtual camera position)
The user 3 or the operator is virtual at any position of a section position with a character display rate of 0% set in step S312 to be described later (a section position with a character display ratio of 0% set at the back position from the virtual camera). By setting the gazing point of the camera, the height of the virtual camera arranged in the three-dimensional virtual space is adjusted to a position slightly higher than the height of the character group arranged in step S310. Adjust the angle of view. In other words, the position of the virtual camera is set so that the character group can be photographed while looking down slightly (step S311). As a result, in the character information presentation process in step S313 described later, the display unit 70 is presented with character information in units of character groups as captured by the virtual camera in the three-dimensional virtual space. Although the character groups are overlapped by the display on the display unit 70, the character information can be visually recognized so that the context can be understood. That is, the character group photographed by the virtual camera becomes smaller in perspective as it is arranged in the back, so that the user 3 can intuitively understand the context of the character group.

(3.7 Character display rate setting)
The transmission rate (character display rate) of the character information for each character group arranged in the three-dimensional virtual space is set by the user 3 or the operator so as to change according to the distance from the virtual camera (step S312). . The character display rate is set to improve the visibility of overlapping character information. Specifically, for example, the character display rate is set to 0% at the position closest to the virtual camera, that is, to be in a completely transparent state, and the character display rate is set to gradually increase as the distance from the virtual camera increases. The character display rate is set to 80% at the maximum, so that the character group in the back of the character group can be seen through even when the visibility is highest. Then, the maximum character display rate of 80% is set to a certain section, and after that section, the maximum character display rate is set to gradually decrease. Finally, the character display rate is set to 0%, that is, the non-display state.

  FIG. 9 is a diagram illustrating character group intervals and character display rates. The arrows on the horizontal axis indicate the direction in which character information in character group units advances by scrolling, and the vertical axis indicates the character display rate. A graph having a mountain-shaped trapezoidal characteristic is the character display rate with respect to the character group position on the horizontal axis. The number between the character groups indicates the number of mora of the character group.

  Thus, the character display rate is set to the maximum 80% at a position away from the virtual camera by a predetermined distance, gradually decreases from the position toward the virtual camera, and is set to 0% at the nearest position. Therefore, in the character information presentation process in step S313 described later, the user 3 can visually recognize character information in character group units so that the display unit 70 gradually becomes thinner as it moves toward the front side. Further, since the character display rate is set to 80% instead of 100% at the maximum distance at a position away from the virtual camera, the user 3 visually recognizes the character group arranged behind the position. be able to.

  Note that the setting of various parameters in the character group virtual space layout (step S310), the virtual camera position setting (step S311), and the character display rate setting (step S312) has been described as being performed by the user 3 or the operator. However, it may be set as a preset by the manufacturer when the portable information terminal 2 is manufactured. In addition, the communication unit 60 of the portable information terminal 2 receives e-mail, web data, etc. in which these parameters are inserted from an external device, extracts the parameters from the e-mail, web data, etc. and stores them in the storage unit 10 The stored real-time CG rendering means 42 may read the parameters from the storage unit 10 and perform the settings in steps S310 to S312. Further, these parameters may be changed according to a change in viewing environment, own preference, etc., by the operation of the portable information terminal 2 by the user 3.

(4. Presentation of character information)
The character information presenting unit 40 receives a start control signal from the operation input unit 30 after the preparation for presenting character information in steps S304 to S312 is completed, scrolls the character group arranged in the virtual space, and performs virtual control. A video signal when the camera virtually captures character information in character group units is generated and output to the display unit 70 (step S313).

  FIG. 10 is a diagram illustrating a presentation example of character information displayed on the display unit 70. As shown in FIG. 10, the character information in units of character groups is displayed so as to be gradually enlarged from the back side to the near side in the center of the screen in a centered state and gradually thinned. Thereby, the user 3 can view the video when the virtual camera arranged in the virtual space virtually captures the character information in character group units on the display unit 70 in the form shown in FIG. The character information of all character groups displayed on the screen can be visually recognized.

(5. Character information presentation control)
Next, character information presentation control for scrolling and displaying character information in character group units on the display unit 70 will be described. This character information presentation control is performed by the user 3 operating a start control switch provided in the portable information terminal 2. The character information presentation unit 40 inputs a start control signal generated according to the operation of the start control switch from the operation input unit 30, and starts and stops scrolling of character information based on the start control signal. The scrolling of the character information is performed at a preset reproduction speed, and the reproduction speed is set by the user 3.

  The user 3 sets whether the stop point of scrolling is up to the punctuation point in the entire character information or the end of the entire character information. That is, the operation input unit 30 outputs a start start control signal to the character information presentation unit 40 according to the operation of the start start control switch by the user 3. The presentation control unit 41 of the character information presentation unit 40 outputs a control signal for starting scrolling to the real-time CG drawing unit 42 when a start control signal is started from the operation input unit 30. The real-time CG rendering unit 42 starts scrolling character information based on a control signal for starting scrolling output by the presentation control unit 41, and character information in units of character groups that are scrolled in a three-dimensional virtual space. Is generated by the virtual camera and output to the display unit 70 as a video signal. Thereby, the user 3 can visually recognize character information in units of continuous character groups that scroll from the back side of the center of the screen of the display unit 70 toward the front side. The character information of each character group is centered so that the center of the character group is located at the center of the horizontal axis of the screen, and new character group-unit character information appears from the back side of the center of the screen and is directed toward the front side. Disappears after scrolling.

  Then, when the scroll stop point is set as a phrase, the presentation control unit 41 scrolls when the character group including the phrase disappears from the front by scrolling, that is, when it is no longer displayed on the display unit 70. Is output to the real-time CG drawing means 42. The real-time CG drawing unit 42 stops scrolling based on the control signal for stopping scrolling output by the presentation control unit 41. In this case, the real-time CG drawing means 42 determines when the character group including the punctuation scrolls disappears from the front, that is, when it is no longer displayed on the display unit 70, and stops scrolling from the presentation control means 41. The scroll may be stopped without being based on the control signal. When the user 3 recognizes the stop of the scrolling by the display unit 70 and wants to read the subsequent character information, the start start control switch is operated again. Then, the operation input unit 30 outputs a start control signal for starting, and the real-time CG drawing means 42 continues to start scrolling character information based on the control signal for starting scrolling output by the presentation control means 41. To do.

  In addition, when the scroll stop point is set at the end of the entire character information, the presentation control unit 41 scrolls all character groups constituting the entire character information and disappears from the front, that is, the display. When it is no longer displayed on the unit 70, a control signal for stopping the scrolling is output to the real-time CG drawing means 42. The real-time CG drawing unit 42 stops scrolling based on the control signal for stopping scrolling output by the presentation control unit 41.

  In the above description, the stop point of scrolling is the punctuation point in the entire character information or the end of the entire character information. However, it may be the end of one sentence described in Example 2 described later. The stop point of scrolling is set by the user 3.

  Further, when the user 3 wants to pause the scrolling while the character information is scrolling, the temporary start control switch is operated. Then, the operation input unit 30 outputs a temporary start control signal to the presentation control unit 41. When the scrolling is stopped halfway, the presentation control unit 41 outputs a control signal for stopping the scrolling to the real-time CG drawing unit 42 at the input timing of the temporary start control signal from the operation input unit 30. On the other hand, when the presentation control means 41 stops scrolling after displaying up to the first appearing phrase, a control signal for stopping scrolling is displayed when the character group including the first appearing phrase scrolls and disappears from the front. Output to the real-time CG drawing means 42. The real-time CG drawing unit 42 stops scrolling based on the control signal for stopping scrolling output by the presentation control unit 41. Then, when the user 3 cancels the pause and determines that the subsequent character information is to be read, the start control switch is operated again. Thus, according to the intention of the user 3, the start of scrolling, automatic stop at a predetermined stop point, and temporary stop during scrolling can be realized.

  Referring to FIG. 4, when character information presentation control device 1-1 is in the presentation standby state (step S 401), presentation control means 41 of character information presentation unit 40 starts start control according to the operation of user 3. A signal is input from the operation input unit 30 (step S402). The real-time CG drawing unit 42 starts a character information presentation process, that is, a scrolling process, based on a start control signal from the presentation control unit 41 (step S403). Thereby, the character information in character group units is scroll-displayed on the display unit 70, and the user 3 can recognize the character information.

  The operation input unit 30 determines whether or not the playback speed has been changed by the user 3 (step S404). When it is determined that the setting has been changed (step S404: Y), the changed playback speed is presented as character information. The data is output to the unit 40 (step S405). On the other hand, when it is determined that the setting has not been changed (step S404: N), it is determined whether or not there has been a pause operation input by the user 3 (step S406). The setting of the playback speed can be changed not only in the state where the character information is presented, but also in the state where the presentation is on standby.

  If the operation input unit 30 determines in step S406 that there has been a pause operation input (step S406: Y), the operation input unit 30 outputs a pause start control signal to the character information presentation unit 40. Then, the real-time CG drawing unit 42 of the character information presentation unit 40 temporarily stops scrolling based on the stop control signal from the presentation control unit 41 (step S407). Thereby, the character information presentation control apparatus 1-1 will be in a presentation standby state. Then, when the presentation control unit 41 of the character information presentation unit 40 inputs a start control signal from the operation input unit 30 according to the operation of the user 3, the real-time CG drawing unit 42 controls the start from the presentation control unit 41. On the basis of the signal, the character information presentation process, that is, the scroll process is resumed (step S408). Then, the process proceeds to step S409. On the other hand, when the presentation control means 41 determines in step S406 that there is no pause operation input by the operation input unit 30 (step S406: N), the presentation control unit 41 proceeds to step S409.

  The presentation control unit 41 causes the real-time CG drawing unit 42 to stop scrolling at a predetermined position and stop the character information presentation process. That is, the real-time CG drawing means 42 performs the presentation control means 41 after scrolling to the character group including the punctuation in the whole character information or after scrolling to the character group at the end of the whole character information. The scrolling is stopped based on the stop control signal from (step S409). Then, the presentation control unit 41 determines whether or not the next sentence exists (step S410). When it is determined that the next sentence is present (step S410: Y), it is determined whether the scroll stop point set by the user 3 is a punctuation point in the whole character information or the end of the whole character information. Is determined (step S411), and when it is determined that the scroll stop point is a phrase, that is, when it is determined that there is a presentation waiting for each phrase (step S411: Y), step S401 Migrate to On the other hand, when it is determined that the scroll stop point is the end of the entire character information, that is, when it is determined that there is no standby for each phrase (step S411: N), the process proceeds to step S403. If it is determined in step S410 that the next sentence does not exist (step S410: N), the process ends.

  The communication unit 60 of the portable information terminal 2 controls the interval, playback speed, character group virtual space layout, virtual camera position, character display rate, etc., for e-mail, web data, etc. received from the outside. When a command is inserted, the real-time CG rendering means 42 of the character information presentation unit 40 extracts information between the commands and sets the character group interval so that the character interval according to the information is extracted. It may be. Further, the real-time CG drawing means 42 may extract the playback speed from the command and scroll using the playback speed. Further, the real-time CG drawing means 42 may extract each parameter of the virtual space arrangement of the character group, the virtual camera position, and the character display rate from the command, and set these parameters. Accordingly, an external device that transmits e-mail, Web data, and the like to the portable information terminal 2 can intentionally adjust the display control of the character information in the portable information terminal 2. That is, the character information presentation control device 1-1 of the portable information terminal 2 does not present uniform character information, but the external device has nuances (character groups) regarding character information assumed by the user who operates the device. , The scrolling speed (playback speed), the spacing, size, form, etc. of characters constituting the character information are reflected in the command, and the character information presentation control device 1-1 presents the character information based on the command. To do. As a result, the user 3 of the portable information terminal 2 that is the receiving side can read the nuance relating to the character information by the user operating the external device by the change of the scroll in the display unit 70.

  The character information presentation control device 1-1 according to the first embodiment has been described by taking the portable information terminal 2 as an example. However, the character information presentation control device 1-1 is not limited to the portable information terminal 2, but for a desktop computer. The present invention can also be applied to a display, a television receiver, and the like. In this case, the user 3 can perform an operation for scroll control of the character information using a mouse of the computer, a remote control of the television receiver, or the like.

  As described above, according to the character information presentation control device 1-1 of the first embodiment, the real-time CG drawing unit 42 of the character information presentation unit 40 transmits character information in character group units from the front side to the back side of the screen. In the form of being centered and laid out, it is scrolled from the back side toward the front side and displayed on the display unit 70. Thereby, the user 3 can visually recognize the character information scrolled to the near side for each character group. Character information in character group units is moved centered from the back side to the front side at the center of the screen at a predetermined size, interval, speed, and character display rate. The character information in character group units appears from the farthest center of the screen, scrolls and is displayed in the foreground, and then disappears from the display unit 70. Therefore, the user 3 naturally sees the center of the display unit 70, and the line-of-sight movement is reduced, and the visibility of the character information can be improved.

  In addition, the real-time CG drawing means 42 displays character information in units of character groups in an overlapping manner from the near side to the far side. In addition, the character information in character group units is displayed at a character display rate that allows the character information located on the back side to be transmitted without being blocked. Thereby, the line-of-sight movement of the user 3 is further reduced, and the visibility in the character information in a plurality of character groups can be improved.

  Further, the real-time CG drawing means 42 displays the character information of each character group from the front side to the back side of the screen at a character display rate having a mountain-shaped trapezoidal characteristic as shown in FIG. I did it. Thereby, the character information in character group units is displayed so as to gradually become thinner as it moves toward the front side. Moreover, since the maximum character display rate is not 100%, the user 3 can visually recognize the character information located behind the position where the character display rate is the highest. Therefore, the visibility of the entire character information displayed on the screen can be further improved.

  Further, the character information dividing unit 20 generates the divided data by dividing the entire character information into clauses and words, and the real-time CG rendering means 42 acquires the clauses from the divided data and sets the character group for each clause. did. As a result, the character information in character group units divided for each phrase becomes a judgment material for punctuation processing of character groups, processing for setting intervals, and processing for arrangement in a three-dimensional virtual space, and becomes a unit of scrolling. Moreover, since the user 3 visually recognizes the character information scroll-displayed on the display unit 70 for each phrase, the user 3 can read it at a glance without taking time.

  In addition, when the character group is composed of full-width character symbols, the real-time CG rendering means 42 inserts a half-width space at the beginning when the end of the character group is a punctuation mark. Thereby, since the character information contained in the character group is centered, the user 3 who visually recognizes the character information does not need to move the line of sight to the left or right according to the character group. Therefore, the line-of-sight movement can be further reduced, and the visibility can be further improved.

  In addition, the real-time CG drawing means 42 calculates the number of mora or the number of characters in the character information included in the character group, and sets the character group interval. Thereby, since the character group is scroll-displayed at intervals corresponding to the length of the character information, the visibility of the character information can be further improved.

  Further, when the word included in the character information is not registered in the mora dictionary 43, the real-time CG drawing unit 42 associates the word with the mora number determined based on the number of characters constituting the word. It was registered in the dictionary 43. Thereby, even when new words, coined words, and the like are included in the character information, new word information is registered in the mora dictionary 43. Therefore, the character group interval setting process using the mora dictionary 43 is performed. Done exactly.

  The operation input unit 30 outputs a start control signal according to the start of the start control switch by the user 3, and the real-time CG drawing means 42 starts from the presentation control means 41 that inputs the start control signal. On the basis of the control signal, scrolling of character information is started, and after scrolling to the point or end of the whole character information, it is stopped. Thereby, the scroll display according to the intention of the user 3 is realizable. Note that the stop position may be a predetermined position, that is, the end of a sentence or the end of a phrase, in addition to the point or end of the entire character information. For example, in the case of a heavy sentence or a compound sentence, it may be the end of each sentence constituting the heavy sentence or the compound sentence.

  Further, the operation input unit 30 outputs a pause start control signal in accordance with the operation of the pause start control switch by the user 3 while the character information is scrolling, and the real-time CG drawing means 42 pauses. On the basis of the stop control signal from the presentation control means 41 to which the start control signal is input, the scrolling is stopped halfway, or the scrolling is stopped by scrolling to the first appearing phrase. Further, the scrolling is resumed according to the operation of the start control switch started by the user 3. Thereby, according to the intention of the user 3, scroll display can be started and stopped.

[Example 2]
Next, Example 2 will be described. The character information presentation control device according to the second embodiment is a device that presents character information to a speaker in a situation where shooting is intended. In addition to the function of presenting character information as in the first embodiment, the character information is read aloud. It has a function of scrolling and displaying character information of a manuscript in units of one sentence corresponding to an utterance break in accordance with the reading progress timing of a certain speaker.

  FIG. 11 is a diagram illustrating an example of a usage pattern when shooting is performed using the character information presentation control apparatus according to the second embodiment. In this type of usage shooting system, the character information of the document presented from the character information presentation control device 1-2 is transmitted to the transmissive display device 6 provided in front of the lens of the shooting camera 7 in the studio of the television broadcasting station. This is a system in which the speaker 4 reads out the text information of the document displayed and displayed on the display device 6 and the photographing camera 7 photographs the speaker 4. The photographing camera 7 photographs the speaker 4 via the transmissive display device 6 and outputs the camera image of the speaker 4 to the television receiver 8.

  The character information presentation control device 1-2 outputs the character information of the manuscript read by the speaker 4 to the transmissive display device 6 in accordance with the reading progress timing of the speaker 4. Specifically, the character information presentation control device 1-2 receives the speech signal of the speech from the speaker 4 from the microphone 5, converts the speech signal into a word string, obtains the progress of the speech in the document, and determines the speaker 4 The character information of the original to be read out next is output as a video signal to the display device 6 in accordance with the progress timing. Thereby, the speaker 4 can recognize the character information of the original that the speaker 4 should read out next by the display device 6, and can read out the original at a predetermined speed according to the character information.

  The transmissive display device 6 is a prompter that includes a transmissive liquid crystal display panel that displays character information and a half mirror that reflects the character information transmitted through the transmissive liquid crystal display board toward the speaker 4. Since the distance between the speaker 4 and the photographing camera 7 is about 3 to 5 m, the speaker 4 can visually recognize the character information by the display device 6 provided in front of the photographing camera 7. The screen size needs to be 40 inches or more. Conventional prompters have a maximum display screen of about 24 inches. For this reason, the prompter used in the photographing system shown in FIG. 11 needs to be a display device 6 that combines a large display of 40 inches or more and a half mirror.

  The center of the half mirror in the display device 6 is made to coincide with the center of the optical axis of the photographing camera 7 where the speaker 4 is photographed. Thereby, the speaker 4 who reads out the character information on the display device 6 naturally sees the lens of the photographing camera 7. Therefore, since the line of sight of the speaker 4 photographed by the photographing camera 7 is not unnatural, the viewer who sees the video of the speaker 4 feels as if the speaker 4 is speaking to himself and feels strange. There is nothing.

(Character information presentation control device)
FIG. 12 is a block diagram illustrating a configuration of the character information presentation control device 1-2 according to the second embodiment. The character information presentation control device 1-2 includes a storage unit 10, a character information division unit 20, a character information presentation unit 40, and a start control unit 50. The character information presentation unit 40 includes a presentation control unit 41, a real-time CG drawing unit 42, and a mora dictionary 43. The start control unit 50 includes a voice recognition unit 51 and an operation determination unit 52. The character information presentation control device 1-2 divides character information of a document to set a predetermined character group, converts character information in character group units into character information of a predetermined form, that is, a three-dimensional virtual The character information of each character group arranged in the space is scrolled, and the video is taken by the virtual camera and output as a video signal. In addition, the character information presentation control device 1-2 receives a voice signal when the speaker 4 is reading a document, and performs scrolling of character information in units of character groups arranged in a three-dimensional virtual space. Based on the word string converted from the signal, control is performed to match the progress timing of the utterance.

  FIG. 13 is a flowchart illustrating the character information presentation processing according to the second embodiment, and FIG. 14 is a flowchart illustrating character information presentation control according to the second embodiment. Hereinafter, processing of each component of the character information presentation control device 1-2 illustrated in FIG. 12 will be described with reference to the flowcharts of FIGS.

(1. Preparation of manuscript)
In the storage unit 10 of the character information presentation control device 1-2, character information of the original read by the speaker 4 is stored in advance. The character information of the document is, for example, information composed of characters shown in FIG.

(2. Generation of divided data)
As in the first embodiment, the character information dividing unit 20 reads the character information of the original from the storage unit 10 (step S1301), and uses the Japanese dictionary to divide the character information of the original into phrases and words to obtain divided data. Conversion is performed (steps S1302 and 1303), and the divided data (sentence, word) is output to the character information presentation unit 40 and the start control unit 50. Note that steps S1301 to S1303 in FIG. 13 are the same as steps S301 to S303 in the first embodiment shown in FIG.

(3. Preparation for presentation of text information)
Next, as in the first embodiment, the character information presentation unit 40 receives the divided data (phrase, word) from the character information division unit 20 and prepares to output the character information to the display device 6 as a video signal. Do. In addition, the start control unit 50 inputs division data (sentence, word) from the character information division unit 20 and prepares for scroll control of character information in character group units. In steps S1304 to S1308, dictionary registration processing for registering words constituting the divided data in the pronunciation dictionary (not shown in FIG. 12) and the mora dictionary 43 is performed, and a language model and an acoustic model (in FIG. 12). Optimization (not shown) is performed. Also, in steps S1309 to S1314, processing is performed such as dividing the divided data into character groups for each phrase and arranging the character group phrases in a three-dimensional virtual space.

(3.1 Word dictionary registration, mora dictionary creation, language model adaptation, acoustic model adaptation)
The voice recognition means 51 of the start control unit 50 inputs the divided data (sentence, word) from the character information dividing unit 20, takes out the word from the divided data, and extracts the extracted word and the word registered in the pronunciation dictionary. The unregistered words are extracted by comparison (step S1304). Then, the voice recognition unit 51 registers the extracted unregistered word in the pronunciation dictionary (step S1305). Thereby, when a word such as a new word or coined word that is not registered in the pronunciation dictionary exists in the manuscript, it can be registered in the pronunciation dictionary in advance. Similarly, the real-time CG drawing means 42 of the character information presentation unit 40 extracts unregistered words for creating the mora dictionary 43.

  The real-time CG drawing means 42 sets the number of mora of the extracted unregistered word manually or automatically, and registers the unregistered word and the number of mora in the mora dictionary 43 to create a new mora dictionary 43 ( Step S1306). Since the number of mora and the mora dictionary 43 have already been described in the first embodiment, description thereof is omitted here. The mora dictionary 43 created in this way is used in step S1311 described later. Note that steps S1304 to S1306 in FIG. 13 are the same as steps S304 to S306 in the first embodiment shown in FIG.

  Further, the speech recognition means 51 pre-registers the language pattern of unregistered words in the language model in order to calculate the similarity (word score) between words from the viewpoint of language, and adapts the language model (step). S1307). In addition, the speech recognition means 51 inputs a speech signal of an utterance by the same speaker 4 in advance in order to calculate the similarity (acoustic score) between words as viewed from the acoustic plane, and unregistered words Are registered in the acoustic model to adapt the acoustic model (step S1308).

  Such adaptation of the language model and the acoustic model by pre-learning is performed when the speech of the speaker 4 can be recorded in advance. By adopting such adaptation, the voice recognition accuracy when the voice recognition unit 51 converts the voice signal into a word string can be improved. The voice recognition unit 51 converts the voice signal of the input document to 100%. Can be converted into a word string with a voice recognition accuracy close to. Therefore, it is possible to realize the presentation control of the character information based on the voice recognition output based on the reading voice of the original by the speaker 4, that is, on the word string converted by the voice recognition means 51. The processing of the speech recognition means 51 using the pronunciation dictionary, language model, and acoustic model will be described later.

(3.2 Character grouping) (3.3 Punctuation processing for centering) (3.4 Character information presentation interval setting) (3.5 Virtual space layout of character group) (3.6 Virtual camera position setting) (3.7 Character display rate setting)
These processes are the same as those in the first embodiment. That is, the real-time CG drawing means 42 sets a phrase as one character group from the input divided data (phrase, word) (step S1309), and the character group ends at “.” In order to center the character group. Alternatively, in the case of punctuation marks such as “,”, a half-width space is inserted at the beginning of the character group to perform punctuation processing (step S1310). The real-time CG drawing means 42 counts the number of mora for each character group using the mora dictionary 43, and sets the character group interval when the character groups are arranged in the three-dimensional virtual space (step S1311). Then, the character group is arranged in the three-dimensional virtual space (step S1312), and the position (height and angle of view) of the virtual camera in the three-dimensional virtual space is set (step S1313). In addition, the character display rate for each character group arranged in the three-dimensional virtual space is set so as to change according to the distance from the virtual camera (step S1314). Steps S1309 to S1314 in FIG. 13 are the same as steps S307 to S312 in the first embodiment shown in FIG.

(4. Presentation of character information)
The character information presenting unit 40 receives a start control signal from the start control unit 50 after the preparation for character information presentation in steps S1309 to S1314 is completed, and controls scrolling of the character group arranged in the virtual space. Generates a video signal when the character information of the character group is virtually photographed and outputs it to the display device 6 (step S1315). Note that step S1315 in FIG. 13 is the same as step S313 in the first embodiment shown in FIG.

(5. Character information presentation control)
Next, the character information presentation control for scrolling the character information in units of character groups in accordance with the speech progress timing of the speaker 4 who reads the document and outputting the video signal to the display device 6 will be described. This character information presentation control is a control in which the character information presentation unit 40 starts and stops scrolling. Based on the start control signal generated by the start control unit 50, the text information presentation control is performed at a preset reproduction speed for each sentence. Done. For example, it is performed at a reproduction speed of 8 mora / second. The reproduction speed is a speed at which character information in character group units moves from the back side toward the near side in the center of the screen, and the character group interval is determined based on the number of character groups. In other words, the playback speed of 8 mora / second is a speed at which, for example, an interval of character groups of 8 mora is moved in 1 second. The start control signal is displayed by scrolling the character information of one sentence that the speaker 4 should read out next in accordance with the progress timing when the speaker 4 is reading out the character information of the document displayed on the display device 6. As described above, it is generated based on the voice signal of the speaker 4 and the divided data of the original.

  Here, a sentence basically means a unit of linguistic expression representing a complete statement consisting of a subject and a predicate (one may be omitted), and means from the beginning to the end of a sentence. For example, it refers to character information from the beginning of a sentence to the first punctuation mark “.”, And is composed of a plurality of character groups. In the example shown in FIG. 5, “Firstly, the establishment of a tax-related law that maintains a provisional tax rate such as gasoline tax” is a single sentence. Further, one sentence includes each sentence constituting a heavy sentence or a compound sentence. In this case, the end of one sentence is not always a punctuation mark “.”.

  A speech recognition method is used to implement character information presentation control. FIG. 15 is a block diagram showing the configuration of the voice recognition means 51 in the start control unit 50 shown in FIG. The voice recognition means 51 includes a voice analysis means 53, a search means 54, a language model 55, an acoustic model 56, and a pronunciation dictionary 57, and inputs the voice signal when the speaker 4 who is shooting is reading a document. Then, using a speech recognition method, frequency matching is performed between the speech signal and a speech frequency pattern stored in advance, the speech signal is converted into a word, and the word string is output to the motion determination means 52. The language model 55 is a probability model learned from a large amount of language information, and the acoustic model 56 is a probability model learned from a large amount of speech data. The pronunciation dictionary 57 is a list in which word notation and pronunciation are associated with each other.

  The voice analysis means 53 inputs a voice signal from the microphone 5 that picks up the voice of the speaker 4, analyzes the voice signal, and extracts a feature amount of the voice signal as a feature vector. The search means 54 receives the feature vector of the speech signal from the speech analysis means 53, and sequentially lists candidates for the output series (word) to be connected based on the probability value (language score) obtained from the language model 55. To do. For each output sequence search candidate listed, the similarity between the feature amount in the acoustic model 56 of each phoneme corresponding to the pronunciation shown in the pronunciation dictionary 57 and the feature amount (feature vector) of the input speech signal. The degree (probability value) is calculated as an acoustic score. Then, the search means 54 outputs the output series that maximizes the product of the acoustic score and the language score to the action determination means 52 as a word string of speech recognition output. The action determination unit 52 generates a start control signal based on the word string of the document that is actually read out by the speaker 4 and converted by the voice recognition unit 51. And the character information presentation part 40 presents the character information of a character group unit to the display apparatus 6 based on this starting control signal, and scroll-displays it.

  Hereinafter, the character information presentation control will be specifically described. Referring to FIG. 14, when character information presentation control device 1-2 is in the presentation standby state (step S1401), operation determination unit 52 of start control unit 50 receives the word sequence of the speech recognition output from speech recognition unit 51. Is input from the character information dividing unit 20, and the word string at the beginning of the sentence (at the beginning of the sentence) in the divided data is compared with the word string in the speech recognition output (step S1402). In other words, the action determination unit 52 calculates a matching rate between the word string of the divided data and the word string of the speech recognition output, and determines whether or not the matching rate is equal to or higher than a predetermined threshold (step S1403).

  If it is determined that the matching rate is not equal to or higher than the predetermined threshold (step S1403: N), the operation determination unit 52 proceeds to step S1401. On the other hand, when it is determined that the coincidence rate is equal to or higher than the predetermined threshold (step S1403: Y), it is determined that the speaker 4 has started reading a predetermined sentence, and a start control signal is output. For example, when the threshold is 80%, five words from the beginning of one sentence in the divided data are compared with five consecutive words in the word sequence of the speech recognition output, and four or more words match. Sometimes, it is determined that the match rate is equal to or greater than a predetermined threshold. In this example, when four or more of five words from the beginning of a sentence match, it is determined that the threshold value is exceeded. However, in order to prioritize response speed, two of three words from the beginning of one sentence are determined. It is also possible to change in consideration of the voice quality of the speaker and the characteristics of the utterance, such as determining that the number of words or more matches is greater than the threshold.

  When the start control signal is input from the action determination unit 52, the presentation control unit 41 of the character information presentation unit 40 outputs a start control signal to the real-time CG drawing unit 42. The real-time CG drawing unit 42 starts the presentation process of character information in units of character groups for one sentence, that is, the scroll process of one sentence, based on the start control signal output by the presentation control unit 41 (step S1404). As a result, the character group unit character information for one sentence is scroll-displayed on the display device 6, and the speaker 4 can recognize the character information to be uttered.

  Based on the start control signal from the presentation control means 41, the real-time CG drawing means 42 performs characters from the back side of the center of the screen of the display device 6 toward the near side from the character information at the beginning of the sentence to the end of the character information. Sequential scroll display is performed for each group, and character information at the end of one sentence is displayed in front of the display device 6. Then, when the real-time CG drawing unit 42 disappears from the display device 6, the real-time CG drawing unit 42 stops the scroll process that is the character information presentation process based on the stop control signal from the presentation control unit 41 (step S 1405). In this case, when there is a new sentence next to the one sentence, the real-time CG drawing means 42 displays the character information of the start of the new sentence on the display device 6 based on the control signal from the presentation control means 41. Scroll to the front, display the next character information on the back side, and stop the scroll process.

  As a result, the speaker 4 displays the character information in units of a plurality of character groups from the beginning toward the back in order from the beginning of the one sentence to be read out after the reading of the one sentence currently being read out is completed. It can be recognized by the device 6. Therefore, the speaker 4 can continuously read out the original without leaving a space between the one sentence and the one sentence after completing the reading out of one sentence. Even if the speaker 4 does not continue reading the manuscript, such as after having finished reading one sentence or speaking an ad lib different from the contents of the manuscript, the speaker 4 starts from the beginning of the next sentence to be read out. Since the character information of a plurality of character groups is displayed on the display device 6, the next one sentence can be read out smoothly.

  The presentation control unit 41 determines whether or not the next sentence exists in the character information of the document (step S1406). When it is determined that the next sentence is present (step S1406: Y), the character information at the beginning of the new sentence is placed at the forefront of the display device 6 and the character information in units of the following character groups is placed at the back side. In the displayed state, the character information presentation control device 1-2 enters a presentation standby state, and the process proceeds to step S1401. On the other hand, if it is determined that the next sentence does not exist (step S1406: N), the process ends.

  By the way, when the speaker 4 starts reading a new sentence, the display device 6 has the character group at the beginning of the new sentence closest to the front, and the plurality of character groups that follow the character group toward the center back. It is displayed. When the speaker 4 starts reading a new sentence displayed on the display device 6, scroll display is started in steps S1402 and 1403 in FIG.

  However, the processing of the speech recognition means 51 shown in FIGS. 12 and 15 (processing for converting the speech signal of the speaker 4 into a word string) usually requires about 500 ms. Similarly, the process for generating the start control signal by the motion determination means 52 also requires a predetermined time since the start control signal is generated by calculating the coincidence rate between a plurality of words. That is, it takes time for the voice recognition process for converting the voice signal of the utterance by the speaker 4 into a word string and the generation process of the start control signal, and a delay corresponding to that time occurs. For this reason, after starting to read a new sentence displayed on the display device 6, the character information to be read next may be displayed on the display device 6 later than the reading timing.

  Therefore, in order to absorb the delay time from the start of reading the character information at the beginning of one sentence displayed on the display device 6 until the start of scrolling, that is, the display device 6 starts from the foremost and most Character information for a predetermined number of character groups needs to be displayed in advance on the display device 6 so that the real-time CG drawing means 42 starts scrolling before the reading of the back character information is completed. That is, at the stage of character information presentation preparation, it is necessary to arrange a predetermined number of character groups in the three-dimensional virtual space in step S1312 of FIG. Specifically, referring to FIG. 9, since the number of character groups in the range where the character display rate is not 0% is a predetermined number, the character group at the beginning of one sentence is in the foreground within the range where the character display rate is not 0%. It is assumed that the character group is arranged on the character information traveling direction side and the subsequent character group is arranged farthest from the character group. The reading time is longer than the delay time by the time when the speaker 4 starts reading from the first character group of one sentence and completes the reading of the innermost character group (reading time) and the delay time described above, that is, The number of character groups in the range where the character display rate is not 0% is determined in consideration of the character group interval and the reproduction speed set in step S1311 so that the reading to the end is not completed within the delay time. The character group is arranged in the three-dimensional virtual space.

  Thus, the character information at the beginning of a sentence is displayed toward the front side with an offset applied to the center back (center) of the screen. FIG. 16 is a diagram for explaining a time relationship between utterance, voice recognition, and character information presentation. In FIG. 16, there is a delay between the utterance of the speaker 4 and the voice recognition, “first” at the forefront of the screen, “gasoline tax etc.” “provisional tax rate” “maintain” in the back “Done” is displayed. When the speaker 4 starts reading a sentence of the manuscript, scroll display starts when the character group “provisional tax rate” advanced by two groups is read. Then, `` First '' disappears from the front, `` Tax-related law '' appears from the back of the center of the screen, `` Gasoline tax etc. '' at the front of the screen, `` Provisional tax rate '' at the back “To be maintained”, “Done”, and “Tax-related laws” are displayed. Then, the disappearance and appearance of character information in units of character groups are repeated by scrolling, and the displayed character information is sequentially updated.

  Therefore, after the speaker 4 starts reading the character information at the beginning of one sentence displayed on the display device 6, the scroll display is started before reading the character information displayed at the back of the screen. The character information that the speaker 4 should read out is not displayed on the display device 6 after the timing. That is, the speaker 4 speaks sequentially from the character information displayed on the foremost side, and when the speaker group is reading out the character group displayed in the center of the screen, the speech recognition unit 51 and the action determination unit 52 perform the speech. After the processing such as recognition is completed, the real-time CG drawing means 42 displays the character information of the next character group unit, and scrolls at a preset reproduction speed until the start of the next sentence is displayed.

  As described above, according to the character information presentation control device 1-2 of the second embodiment, the real-time CG drawing unit 42 of the character information presentation unit 40 transmits character information in character group units from the front side to the back side of the screen. In the form of being centered and laid out, it is scrolled from the back side toward the front side and displayed on the display device 6. Thereby, the speaker 4 can see the character information scrolling forward in character group units by the reflection of the half mirror of the display device 6. Character information in character group units is moved centered from the back side to the front side at the center of the screen at a predetermined size, interval, speed, and character display rate. The character information in character group units appears from the farthest center of the screen, disappears after scrolling and being displayed at the front. Accordingly, the speaker 4 naturally sees the center of the display device 6, and as in the first embodiment, the line of sight movement is reduced, and the visibility of the character information can be improved. Further, since the speaker 4 looks at the center of the photographing lens of the photographing camera 7, the line of sight of the speaker 4 is not unnatural, and the viewer who views the image of the speaker 4 is as if the speaker 4 is speaking. I feel like I'm speaking to myself and I don't feel uncomfortable.

  In addition, the same processing as in the first embodiment, such as the display mode of character information in character group units, punctuation processing for centering, and the setting of character group intervals, has the same effects as in the first embodiment.

  Further, the character information dividing unit 20 generates divided data by dividing the character information of the document into phrases and words, and the speech recognition means 51 uses the speech recognition method for the speech signal of the document read by the speaker 4. The motion determination means 52 compares the word sequence of the audio signal with the word sequence of the divided data, determines whether the start of one sentence already displayed on the display device 6 is read out, and the speech progress A start control signal corresponding to the output is output. The real-time CG drawing means 42 starts scrolling at the timing of the start control signal and stops at the end of one sentence. As a result, the speaker 4 does not need to advance or send the read-out manuscript by his / her hand, and can read out the manuscript at the utterance speed as his / her intention. Further, the character information of the document can be continuously obtained from the display device 6 not only in the sitting state but also in the standing state.

  In addition, the operation determination unit 52 compares the word string of the audio signal and the word string of the divided data, determines whether the start of one sentence already presented on the display device 6 is read out, and determines that the reading is performed. In this case, a start control signal is output. Further, the real-time CG drawing means 42 starts scrolling based on the start control signal from the presentation control means 41 at the timing of the start control signal, and the last character information of one sentence that the speaker 4 has started to read ( The punctuation mark “.”) Disappears from the display device 6, the character information at the beginning of the next sentence is displayed on the front side, and the subsequent character information is displayed toward the back side so that the scrolling is stopped. did. Thereby, as the speaker 4 starts reading a sentence, the text information of the sentence is scroll-displayed on the display device 6. When the reading of one sentence is completed, the character information in a predetermined number of character groups can be visually recognized on the display device 6 from the beginning of the next sentence. Therefore, the speaker can continue to read the manuscript without leaving a space between one sentence, and in this case, by recognizing the voice signal at the beginning of reading the next sentence read out, One sentence can be displayed continuously. In addition, it is possible to pause after the speaker finishes reading one sentence, or to speak an ad-lib different from the content of the original, and thereafter, the next sentence can be read out smoothly.

  In addition, the real-time CG drawing means 42 is based on the stop control signal from the presentation control means 41 in a state where the character information in units of character groups continuous from the beginning of one sentence to be read out by the speaker 4 is displayed on the display device 6. , Stop scrolling display. In this case, the number of character groups displayed on the display device 6 is provided by the real-time CG rendering means 42 after the speaker 4 starts reading the displayed character information and before completing the reading of all the character information. A number corresponding to the delay time from the start of reading to the start of scrolling is set so that scrolling is started based on a start control signal from the control means 41. That is, a predetermined number of character groups in the three-dimensional virtual space are set so that the real-time CG drawing means 42 starts scrolling based on the start control signal from the presentation control means 41 before the speaker 4 completes reading. Is placed. Thereby, the speaker 4 reads out the character information displayed on the display device 6, and the display device has the number of character groups in consideration of the delay time due to the process of determining the start of reading one sentence by recognizing the voice signal. Character information is displayed at 6. Therefore, before the speaker 4 finishes reading the character information displayed on the display device 6, scroll display is started, and the character information to be read out is displayed on the display device 6, always watching the display device 6. You can continue reading.

  The character information presentation control device 1-1 according to the first embodiment and the character information presentation control device 1-2 according to the second embodiment include a volatile storage medium such as a CPU and a RAM, a nonvolatile storage medium such as a ROM, and The computer is configured with an interface and the like. Each function of the character information division unit 20, the operation input unit 30 and the character information presentation unit 40 provided in the character information presentation control device 1-1, and the character information division unit 20 provided in the character information presentation control device 1-2, Each function of the character information presentation unit 40 and the start control unit 50 is realized by causing the CPU to execute a program describing these functions. These programs can also be stored and distributed in a storage medium such as a magnetic disk (floppy (registered trademark) disk, hard disk, etc.), optical disk (CD-ROM, DVD, etc.), semiconductor memory, or the like.

DESCRIPTION OF SYMBOLS 1 Character information presentation control apparatus 2 Portable information terminal 3 User 4 Speaker 5 Microphone 6 Display apparatus 7 Shooting camera 8 Television receiver 10 Storage part 20 Character information division part 30 Operation input part 40 Character information presentation part 41 Presentation control means 42 Real-time CG drawing means 43 Mora dictionary 50 Start control section 51 Speech recognition means 52 Operation determination means 53 Speech analysis means 54 Search means 55 Language model 56 Acoustic model 57 Pronunciation dictionary 60 Communication section 70 Display section

Claims (12)

In a character information presentation control device that performs control for arranging character information in a virtual space, displaying it on the screen, and presenting it to the user,
A character information dividing unit for dividing the character information and generating divided data for each clause;
Set the divided data divided by the character information dividing unit as a character group of a predetermined unit,
In the character group unit, the number of mora is calculated from the character information in the character group unit, and based on the number of mora , a continuous character group interval is set.
From the back side to the near side in the virtual space in the center of the screen as seen from the user, the character information in the character group unit is centered and scrolled sequentially at a predetermined speed and the interval,
A character information presentation control apparatus comprising: a character information presentation unit that causes a new character group to appear from the back side of the center and scrolls toward the near side, and then erases the character group. In the character information presentation control device according to claim 1,
The character information presentation unit
The character information of the character group unit is presented so as to be displayed in a superimposed manner before and after the character group, and presented at a character display rate so that the character information located on the back side is transparent. Character information presentation control device. In the character information presentation control device according to claim 2,
The character display rate is set so as to gradually increase from 0% toward the back side from the near side, reach a predetermined maximum value lower than 100%, and gradually decrease to reach 0%. A character information presentation control device characterized by that. In the character information presentation control device according to any one of claims 1 to 3 ,
The character information presentation unit
When the end of the character group is a punctuation mark, character information is generated by inserting a blank at the beginning of the character group to generate a new character group and centering the new character group Presentation control device. In the character information presentation control device according to any one of claims 1 to 4 ,
And a mora dictionary storing words and the number of mora of the words.
The character information presentation control device, wherein the character information presentation unit calculates the number of mora of a word included in the character information, and registers the word and the number of mora in the mora dictionary. In the character information presentation control device according to any one of claims 1 to 5 ,
Furthermore, an operation input unit that outputs a start control signal according to the user's key operation,
The character information presentation unit
Character information presentation control, wherein a start control signal is input from the operation input unit, the scroll is started at an input timing of the start control signal, and the scroll is stopped at a predetermined position of the character information. apparatus. In the character information presentation control device according to any one of claims 1 to 5 ,
Further, according to the key operation of the user, provided with an operation input unit for outputting start and stop start control signals, respectively.
The character information presentation unit
A start control signal is input from the operation input unit, and when the start control signal is a start signal, the scroll is started, and when the start control signal is a stop signal, the scroll is stopped. Character information presentation control device. In the character information presentation control device according to any one of claims 1 to 5 ,
The user is a speaker who reads out the character information,
Furthermore, the voice control signal of the utterance by the speaker is converted into a word string, and based on the word string, a start control unit that outputs a start control signal according to the progress of the utterance in the character information,
The character information presentation unit
A start control signal is input from the start control unit, scrolling for each character group is started at the input timing of the start control signal, and the scroll is stopped at a predetermined position of the character information. Character information presentation control device. In the character information presentation control device according to claim 8 ,
The start controller is
A voice recognition means for performing frequency matching between a voice signal of the utterance by the speaker and a pre-recorded voice frequency pattern, and converting the voice signal into a word string using a dictionary of words stored in advance; ,
Based on the word string converted by the voice recognition means and the word string constituting the character information, the beginning of a continuous character group constituting a sentence out of a plurality of character groups constituting the character information is And an operation determining means for outputting the start control signal when it is determined that the speaker has spoken,
The character information presentation unit
The scrolling is performed for each sentence, and at the timing when the start control signal is input from the start control unit, the scroll for the one sentence is started, and the start of the continuous character group constituting the next sentence is in front of the screen. The character information presentation control device is characterized in that the scrolling is stopped when the next character group is displayed toward the back side. In the character information presentation control device according to claim 9 ,
The character information presentation control device, wherein the sentence has a punctuation mark at the end thereof. In the character information presentation control device according to claim 8 ,
The number of character groups displayed when the scrolling is stopped is the number of character groups before the speaker completes reading all the displayed character groups after the speaker starts reading the displayed character groups. The character information presentation control device is characterized in that the number is set in accordance with a delay time from the start of the reading to the start of the next scroll so that the next scroll is started. Computer, character information presentation control program for functioning as the character information presentation control unit according to any one of claims 1 to 11.

Images