JP2015091077A - Information display device, method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information display technology capable of shortening the time required for grasping conversation contents further than the prior arts.SOLUTION: In an information display method of displaying information on a touch panel, a character information display part 4 displays character information corresponding to a voice on a second area that is a certain area on the touch panel. A character information duplication part 5 duplicates at least a portion of the character information that is displayed on the second area, on a first area different from the second area of the touch panel. An image information display part 6 displays image information that is inputted on the first area of the touch panel, on the first area together with the character information that is duplicated on the first area. A free description area information storage part 7 stores the character information duplicated on the first area and the image information that is inputted on the first area, in the vicinity of the character information duplicated on the first area while associating the character information and the image information with each other.

Description

  The present invention relates to a technique for recording and / or referring to a voice and a note accompanying the voice.

  In a multi-person conference, voice communication and information recording tools (such as a whiteboard) for information sharing are often used. Hereinafter, an example in which a whiteboard is used as an information recording tool will be described.

  The whiteboard frequently includes a word string spoken by the participant, and supplementary information about the word string is additionally written. The whiteboard thus created during the meeting is used not only for communication during the meeting but also for reflection after the meeting. At this time, the contents of the whiteboard may include a description that cannot be remembered in what context of the conversation.

  In a conference, audio information of the conference may be stored in addition to the whiteboard information. In this case, it is possible to search for and reproduce the sound of the corresponding portion of the portion of the whiteboard whose context is unknown. However, it is not easy to find the corresponding part when the meeting is long. Therefore, the technology for associating the information recording tool with the audio information is effective in improving the efficiency of the meeting content grasping work.

  A conference content grasp support technology that links an information recording tool and audio information has been proposed so far.

  Japanese Patent Application Laid-Open No. 2004-151561 describes a technique for associating audio information, video information, and sensor information acquired during a meeting based on time information and outputting the information on one screen. The sensor information includes, for example, character information and graphic information input with a touch panel and a drawing pen.

  Non-Patent Document 1 proposes a remote conference system that can handle multimedia including moving images, voices, characters, figures, images, and handwriting. The multimedia information is synchronized based on the time information and is output on one screen.

JP 2004-173058 A

Kazuo Watanabe et al., “Multimedia Distributed Enrollment Conference System MERMAID”, IPSJ Transactions, Vol.32, No. 9, 1991.

  Both Patent Literature 1 and Non-Patent Literature 1 associate voice and handwritten information. However, when speech and handwritten information are linked, if the conversation content related to the handwritten information takes a long time, all the related parts must be re-listened, and the time required for grasping the conversation content increases. is there.

  An object of the present invention is to provide an information display method, apparatus, and program that make it possible to shorten the time required for grasping the content of a conversation compared to the prior art.

  According to the information display method according to one aspect of the present invention, in the information display method on the touch panel, the character information display unit displays the character information corresponding to the sound in the second area which is the area where the touch panel is located. A display step, a character information replication unit that replicates at least part of the character information displayed in the second area in a first area different from the second area of the touch panel, and an image information display section The image information display step for displaying the image information input in the first area of the touch panel in the first area together with the character information copied in the first area, and the free description area information storage unit are copied in the first area. Free description area information storage step for storing the character information and the image information input in the first area in association with the character information copied in the first area , The has.

  According to the information display method according to another aspect of the present invention, in the information display method on the touch panel, the character information display unit displays the character information obtained by the speech recognition in the first area that is the area where the touch panel is located. The character information display step and the image information display include an image information display step of displaying the image information input in the first area of the touch panel in the first area together with the character information.

  By associating text information and image information associated with voice instead of voice and handwritten information, the conversation content related to the image information can be grasped at a glance from the character information, so the time required for grasping the conversation content has been conventionally increased. Can be shortened.

The block diagram for demonstrating the example of the information display apparatus of 3rd embodiment from 1st embodiment. The block diagram for demonstrating the example of the information display apparatus of 4th embodiment. The flowchart for demonstrating the example of the learning phase of 1st embodiment to 3rd embodiment. The flowchart for demonstrating the example of the reflection phase of 3rd embodiment from 1st embodiment. The flowchart for demonstrating the example of the learning phase of 4th embodiment. The flowchart for demonstrating the example of the reflection phase of 4th embodiment. The figure for demonstrating the example of the screen of a touchscreen. The figure for demonstrating the example of the screen of a touchscreen.

[First embodiment]
As shown in FIG. 1, the information display device according to the first embodiment includes a voice input unit 1, a voice information storage unit 2, a character information conversion unit 3, a character information display unit 4, a character information duplication unit 5, and an image information display unit. 6. A free description area information processing unit 7, a free description area information storage unit 8, a correspondence unit 9, and a character information storage unit 10 are provided, for example.

  The information display device has two different operation modes of a recording phase and a retrospective phase. Character information and related image information are recorded in the recording phase, and these recorded information are referred to in the retrospective phase. It is assumed that the user can appropriately switch between the recording phase and the reflection phase, for example, by operating a button. For example, when an information display device is used for a conference, the conference start and end buttons are arranged on the screen. That's fine. In addition, a configuration is possible in which a review button is added, and the operation mode can be shifted to the review mode by pressing the button during the conference.

  The information display method is realized by performing each process of the recording phase and the reflection phase illustrated in FIGS. 3 and 4 respectively.

<Recording phase>
The audio input unit 1 is a device that converts sound such as a microphone into an electric signal. For example, the voice at the time of the conference is input to the voice input unit 1 and converted into an electrical signal (step R1). The converted electrical signal is stored in the voice information storage unit 2 as voice information which is information about the voice (step R2). In the first embodiment, the audio information need not be stored.

  The character information conversion unit 3 generates character information corresponding to the voice input to the voice input unit 1 (step R3). For example, the character information conversion unit 3 generates character information by performing voice recognition based on the voice information read from the voice information storage unit 2. The generated character information is provided to the character information display unit 4. The generated character information is stored in the character information storage unit 10 (step R4).

  The character information conversion unit 3 may assign an utterance ID to each utterance divided in an arbitrary time interval that constitutes a voice. If there is information accompanying the utterance, the information may be included in the utterance ID. The information accompanying the utterance is, for example, a speaker ID for identifying the speaker who made the utterance, time information of the utterance, or the like. Speech separation may be performed automatically using speaker recognition technology, silent section detection technology, or the like, or sections may be manually separated. For example, a sentence with a punctuation as a delimiter is delimited as one utterance.

  The generation of the character information corresponding to the voice may be performed by a person such as a stenographer. In this case, the information about the text transcribed based on the speech becomes character information corresponding to the speech. In this case, the character information conversion unit 3 may not be provided in the information display device.

  The character information display unit 4 displays the character information corresponding to the voice generated by the character information conversion unit 3 in a second region that is a region where the touch panel is located (step R5). In this example, the second area is the voice text display unit 101 illustrated in FIG.

  When there is an utterance ID or an information speaker ID associated with the utterance, the character information display unit 4 displays the utterance ID or the information speaker ID associated with the utterance in addition to the character information in the second area. Good.

  In this example, as shown in FIG. 7, the touch panel includes a first area that is a free description area portion 102 that can be freely described, and a second area that is a speech text display portion 101 on which character information is displayed. Contains. When a touch pen or a finger touches the touch panel, the touch panel generates coordinate information of the contact portion, thereby generating a time series of coordinate information. Based on the time series of the coordinate information, for example, processing of the character information duplicating unit 5 and the image information display unit 6 described below can be performed.

  The character information duplication unit 5 duplicates at least a part of the character information displayed in the second area in the first area (step R6). Thereby, at least a part of the character information displayed in the second area is displayed in the first area. As shown in FIG. 8, the character information duplicating unit 5 performs processing by selecting a part of the text displayed on the voice text display unit 101, which is the second area, and touching the selected text as it is. This is performed when dragging and dropping from the display unit 101 to the free description area 102 which is the first area.

  In this case, after the coordinates of the text are determined by the drop, the text and the coordinates are provided to the free description area information processing unit 7. When the utterance ID is given, the utterance ID of the utterance including the text may be further provided to the free description area information processing unit 7.

  In addition, when characters, figures, symbols, etc. are input using the touch pen or finger in the first area of the touch panel, image information which is information about the image corresponding to the input and coordinate information of the image information are generated. The The image information display unit 6 displays the generated image information in the first area together with the character information duplicated in the first area (step R7). The generated image information and its coordinate information are provided to the free description area information processing unit 7.

  If the margin of the first area which is the free description area 102 is insufficient, the user can save the currently displayed free description area 102 and create a new blank page. Also good. At this time, an ID is assigned to each free description area 102. The coordinates of each free description area 102 are uniquely managed as (free description area ID-coordinate value).

  The free description area information processing unit 7 freely associates the character information copied to the first area with the image information input in the first area that is in the vicinity of the character information copied to the first area. It is stored in the description area information storage unit 8. In other words, the free description area information storage unit 8 stores the character information copied to the first area and the image information input in the first area that is in the vicinity of the character information copied to the first area. The information is stored in association (step R8).

  Specifically, the free description area information processing unit 7 combines the character information copied to the first area, the coordinate information of the character information, the image information, and the coordinate information of the image information into one free description segment. Group as. The free description area information processing unit 7 assigns a free description segment ID to each free description segment, thereby reproducing character information copied to the first area, coordinate information of the character information, image information, and the image Link the coordinate information of the information. If an utterance ID is assigned, the utterance ID may be included in the free description segment. When there are a plurality of pages in the free description area 102, the coordinate information is a unique value including the free description area ID.

  As this segmentation method, for example, a plurality of character information and image information continuously copied within a predetermined time, for example, at a short time interval, and a method of grouping these coordinate information into one group, or a center of a certain coordinate There is a method in which character information and image information included in a range of a figure having a predetermined shape are grouped. The figure having a predetermined shape is an annular figure such as a rectangle, a circle, or an ellipse. A certain coordinate is a coordinate of character information or a coordinate of image information. In this way, character information and image information that are close in time or space and their coordinate information are grouped into one group.

  After the segment is determined, the information about the free description segment (for example, the first character information, the coordinate information of the character information, the image information, the coordinate information of the image information) and the free description segment ID are the free description area information. It is provided and stored in the storage unit 8.

<Review Phase>
In the retrospective phase, it is assumed that image information and duplicated character information are displayed in the first area of the touch panel. When there are a plurality of free description area portions 102, the user can refer to the desired free description area portion 102 and image information by an operation such as screen scrolling, for example.

  It is assumed that image information is designated by the user by input in the first area of the touch panel. In this case, the associating unit 9 reads the character information stored in association with the designated image information from the free description area information storage unit 8 and reads the character information including the read character information into the character information storage unit. 10 is read and displayed in the second area (step S1).

  When the user touches the touch panel with a touch pen or a finger, coordinate information of the contact portion is generated. For example, corresponding character information is searched based on this coordinate information. That is, referring to the free description area information storage unit 8 using the generated coordinate information as a key, a free description segment having coordinate information corresponding to the coordinate information is searched. When a free description segment matching the condition is found, character information including character information included in the free description segment is displayed in the second area.

  Here, the free description segment having the coordinate information corresponding to the coordinate information is, for example, the coordinates specified by the coordinate information are the character information coordinates included in the free description segment or the image information included in the free description segment. This is a free description segment when it is included in a figure of a predetermined shape including the coordinates. The figure having a predetermined shape is an annular figure such as a rectangle, a circle, or an ellipse. The graphic having the predetermined shape may be the same as or different from the graphic having the predetermined shape when the free description segment is generated.

  The character information including character information included in the free description segment is, for example, all or a part of an utterance including character information included in the free description segment. When the utterance ID is assigned, the associating unit 9 reads from the character information storage unit 10 utterances having the same utterance ID as the utterance ID included in the free description segment, and all or part of the read utterances Is displayed on the voice text display unit 101 which is the second area.

  The association unit 9 may emphasize the character information when displaying the character information in the second area. The character information is emphasized, for example, by scrolling the character information and displaying it in a prominent place like the top of the screen, or by changing the text color or background color to highlight it.

  In this way, by associating character information and image information rather than voice and handwritten information, the conversation content related to the image information can be grasped at a glance from the character information, so the time required for grasping the conversation content has been conventionally increased. Can be shortened.

  In addition, as shown in FIG. 7, two areas are provided on the touch panel, character information is displayed on the speech text display unit 101 in principle, image information is displayed on the free description area unit 102, and information on different time axes is displayed. By realizing the browsing, flexible content grasping work becomes possible.

[Second Embodiment]
In the first embodiment, the character information described in the free description area 102 is presented. However, when the character information is generated by the voice recognition system, the character information may include an error. Therefore, in the second embodiment, not only character information but also audio information corresponding to the character information is presented to the user. Thereby, the user can grasp | ascertain character information and audio | voice information, and can grasp | ascertain the conversation content more correctly.

  Hereinafter, a description will be given centering on differences from the first embodiment. A duplicate description of the same parts as in the first embodiment is omitted.

  The information display apparatus according to the second embodiment further includes an audio reproduction unit 11 as indicated by a broken line in FIG.

<Recording phase>
In the first embodiment, the audio information need not be stored, but in the second embodiment, the audio information is always stored. That is, voice information that is information about the voice input from the voice input unit 1 is stored in the voice information storage unit 2. At that time, a voice information ID may be given to each voice information. In this case, the voice information is stored in the voice information storage unit 2 together with the voice information ID.

  When the utterance ID is given, the character information conversion unit 3 associates the utterance ID of each utterance with the voice information ID. Information regarding this association is also stored in the voice information storage unit 2.

  Other processes in the recording phase are the same as those in the first embodiment.

<Review Phase>
When the image information is designated by input in the first area of the touch panel, the associating section reads the character information stored in association with the designated image information from the free description area information storage section 8, and Character information including the read character information is read from the character information storage unit 10 and displayed in the second area, and the voice information corresponding to the character information stored in association with the designated image information is displayed as the voice information storage unit 2. The audio information read from the audio data is read by the audio reproduction unit 11 (step S1).

  For example, the associating unit 9 obtains the utterance ID of the utterance including the character information stored in association with the image information designated by the input to the touch panel from the free description segment stored in the free description area information storage unit 8. The voice information is acquired by reading the voice information having the voice information ID corresponding to the utterance ID from the voice information storage unit 2.

  The sound reproducing unit 11 is an electroacoustic transducer that generates sound based on an applied electric signal such as a speaker or a headphone.

  Other processes in the retrospective phase are the same as those in the first embodiment.

[Third embodiment]
In the first embodiment and the second embodiment, when an arbitrary portion of the first area which is the free description area portion 102 is designated, a corresponding portion of character information and / or voice information is presented to the user. However, in the case where the content of the meeting is grasped by following the text information that is a voice text, it is necessary to be able to refer to the corresponding image information from the text information.

  Therefore, in the third embodiment, when arbitrary character information displayed on the voice text display unit 101 is designated, the corresponding image information is further displayed. This makes it possible to refer to the corresponding image information while following the character information, leading to efficient content grasping.

  Hereinafter, a description will be given centering on differences from the first embodiment. A duplicate description of the same parts as those in the first embodiment and the second embodiment is omitted.

<Recording phase>
Other processes in the recording phase are the same as those in the first embodiment and the second embodiment.

<Review Phase>
When the character information is designated by the input in the second area, which is the voice text display unit 101 of the touch panel, the associating unit 9 stores an image stored in association with the character information included in the designated character information. Information is read from the free description area information storage unit 8, and the read image information is displayed in the first area (step S1). Of course, the associating unit 9 may read not only the image information but also the character information stored in association with the image information from the free description area information storage unit 8 and display it in the first area as necessary. .

  Specifically, the correspondence unit 9 reads an utterance ID corresponding to the designated character information from the character information storage unit 10 when certain character information is designated by an input in the second area of the touch panel. When the free description area 102 extends over a plurality of pages, the free description area ID is also read. Then, the associating unit 9 searches the free description segment including the same utterance ID as the utterance ID with reference to the free description area information storage unit 8 using the read utterance ID as a key. If a free description segment matching the condition is found, the character information and image information included in the free description segment are read from the free description area information storage unit 8 and displayed in the first area.

  The read character information and image information may be displayed at a coordinate position determined by the coordinate information included in the free segment, or may be displayed at another position. The read character information and image information may be displayed with emphasis. The emphasis of character information and image information is performed, for example, by highlighting the character information and image information by changing the color or background color, or by moving the character information and image information by animation.

  Other processes in the retrospective phase are the same as those in the first embodiment and the second embodiment.

[Fourth embodiment]
In the first to third embodiments, the character information conversion unit 3 is assumed to be a transcription by a voice recognition system or a stenographer. When a speech recognition system is used for the character information conversion unit 3, it is preferable that the time lag in which the speech is displayed as speech text is as small as possible. On the other hand, in order to improve the accuracy of speech recognition, technologies such as using complicated acoustic models and language models in speech recognition systems, or outputting highly accurate recognition results again using the recognition results once output are used. Sometimes. However, these techniques are computationally intensive and may cause a time lag before the speech text is output.

  Therefore, in the fourth embodiment, for example, a text information storage unit is used to quickly output speech text using a simple model at the time of a conference, and improve accuracy using a complex model or a recognition result after the conference ends. 10 to modify the voice text stored in Thereby, it is possible to reduce the time lag of the voice text generation at the time of carrying out the conference, and to use the voice text with high accuracy when looking back.

  Hereinafter, a description will be given centering on differences from the first embodiment. A duplicate description of the same parts as those in the first embodiment and the second embodiment is omitted.

  As shown in FIG. 2, the information display device according to the fourth embodiment includes a first voice recognition unit 31 and a second voice recognition unit 32 in place of the character information conversion unit 3. This is the same as the information display device of the embodiment.

  The information display method of the fourth embodiment is realized by performing each process of the learning phase and the reflection phase illustrated in FIGS. 5 and 6, respectively.

<Recording phase>
The first voice recognition unit 31 recognizes the voice input to the voice input unit 1 by the first voice recognition method, instead of the character information conversion unit 3, and sets it as character information corresponding to the voice (step R3). . The first speech recognition method is a simple speech recognition method. The generated character information is displayed on the character information display unit 4 and stored in the character information storage unit 10 as in the first embodiment.

  Other processes in the recording phase are the same as those in the first embodiment and the second embodiment.

<Review Phase>
The second voice recognition unit 32 recognizes the voice input to the voice input unit 1 by the second voice recognition method, generates character information corresponding to the voice, and the character information generated by the first voice recognition method Is replaced with the character information generated by the second speech recognition method (step S2). Here, it is assumed that the second voice recognition method has higher voice recognition accuracy and a larger calculation amount than the first voice recognition method.

  Specifically, the character information stored in the character information storage unit 10 is replaced with the character information generated by the second speech recognition method. Of course, the character information stored in the free description area information storage unit 8 may be replaced with the character information generated by the second speech recognition method from the character information generated by the first speech recognition method.

  When the utterance ID and the voice information ID are associated with each other, the character information may be replaced based on the information regarding the association. That is, the character information generated by the second speech recognition method for the sound information of a certain sound information ID is replaced with the character information of the utterance having the utterance ID corresponding to the sound information ID.

  The processing by the second voice recognition unit 32 is automatically performed before the look-back phase starts, for example, after the learning phase ends. Of course, the processing by the second speech recognition unit 32 may be performed after the user's explicit instruction.

[Modifications, etc.]
In the above-described embodiment, an example is described in which the audio at the time of the conference is input to the audio input unit 1, but other audio may be input to the audio input unit 1. In other words, the information display apparatus and method are not limited to use as a conference record.

  In the above embodiment, the character information generated by the character information conversion unit 3 or the first voice recognition unit 31 is displayed on the voice text display unit 101 which is the second region, but may be displayed on the first region. Good. In this case, the voice text display unit 101 that is the second region may not be provided on the touch panel.

  The various processes described above are not only executed in time series according to the description, but may also be executed in parallel or individually as required by the processing capability of the apparatus that executes the processes. Needless to say, other modifications are possible without departing from the spirit of the present invention.

  For example, the first to fourth embodiments may be appropriately combined. In other words, one certain information display device may have the function of the information display device of the first to fourth embodiments. In this case, it is assumed that the user can select which function of the information display device of the first embodiment to the fourth embodiment is used by, for example, a switch. Further, as one information display method, it may be possible to perform at least two methods of the information display methods of the first embodiment to the fourth embodiment. This makes it possible to grasp the contents more flexibly.

  When the above configuration is realized by a computer, the processing contents of the functions that each device should have are described by a program. By executing this program on a computer, the above processing functions are realized on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium. An example of a computer-readable recording medium is a non-transitory recording medium. Examples of such a recording medium are a magnetic recording device, an optical disk, a magneto-optical recording medium, a semiconductor memory, and the like.

  This program is distributed, for example, by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, this computer reads a program stored in its own recording device and executes a process according to the read program. As another execution form of the program, the computer may read the program directly from the portable recording medium and execute processing according to the program, and each time the program is transferred from the server computer to the computer. The processing according to the received program may be executed sequentially. The above-described processing may be executed by a so-called ASP (Application Service Provider) type service that realizes a processing function only by an execution instruction and result acquisition without transferring a program from the server computer to the computer. Good.

  In the above embodiment, the processing functions of the apparatus are realized by executing a predetermined program on a computer. However, at least a part of these processing functions may be realized by hardware.

DESCRIPTION OF SYMBOLS 1 Voice input part 2 Voice information storage part 3 Character information conversion part 31 First voice recognition part 32 Second voice recognition part 4 Character information display part 5 Character information replication part 6 Image information display part 7 Free description area information processing part 8 Free Description area information storage unit 9 Corresponding unit 10 Character information storage unit 11 Audio playback unit

Claims (11)

In the information display method on the touch panel,
A character information display step in which the character information display unit displays character information corresponding to the sound in a second region which is a region where the touch panel is provided;
A character information duplicating step, wherein a character information duplicating unit duplicates at least a part of the character information displayed in the second area to a first area different from the second area of the touch panel;
An image information display step for displaying the image information input in the first area of the touch panel in the first area together with the character information replicated in the first area;
The free description area information storage unit associates the character information copied in the first area with the image information input in the first area that is in the vicinity of the character information copied in the first area. A free description area information storage step for storing
Information display method. In the information display method on the touch panel,
A character information display step for displaying character information obtained by voice recognition in a first region which is a region where the touch panel is provided; and
An image information display step for displaying image information input in the first area of the touch panel in the first area together with the character information;
Information display method. In the information display method of Claim 1,
The character information corresponding to the speech is at least one of information about characters obtained by speech recognition of the speech and information about text transcribed based on the speech.
Information display method. In the information display method in any one of Claim 1 to 3,
A character information storage unit for storing character information corresponding to the voice;
In the case where image information is designated by the input in the first area of the touch panel, the associating unit reads character information stored in association with the designated image information from the free description area information storage unit. An association step of reading character information including the read character information from the character information storage unit and displaying the character information on the second region or the first region;
An information display method further comprising: In the information display method of Claim 4,
In the associating step, when character information is designated by input in the first area or the second area of the touch panel, an image stored in association with the character information included in the designated character information Further comprising reading information from the free description area information storage unit and displaying the read image information in the first area.
Information display method. In the information display method in any one of Claim 1 to 3,
A character information storage unit for storing character information corresponding to the voice;
A voice information storage step in which a voice information storage unit stores voice information which is information about the voice;
When image information is designated by the input in the first area of the touch panel, the associating unit reads character information stored in association with the designated image information from the free description area information storage unit. The character information including the read character information is read from the character information storage unit and displayed in the second area or the first area, and corresponds to the character information stored in association with the designated image information. Reading audio information from the audio information storage unit and reproducing the read audio information;
An information display method further comprising: In the information display method in any one of Claim 1 to 3,
A first voice recognition step in which the first voice recognition unit recognizes the voice by a first voice recognition method, which is a simple voice recognition method, and converts the voice into character information corresponding to the voice;
An information display method further comprising: In the information display method in any one of Claim 4, 5, and 6,
A first voice recognition step, wherein a first voice recognition unit recognizes the voice by a first voice recognition method to obtain character information corresponding to the voice;
A second voice recognition unit that recognizes the voice by a second voice recognition method having a higher voice recognition accuracy and a larger calculation amount than the first voice recognition method, and generates character information corresponding to the voice; A second voice recognition step of replacing the character information generated by the first voice recognition method with the character information generated by the second voice recognition method;
An information display method further comprising: In an information display device that displays information on a touch panel,
A character information display unit for displaying character information corresponding to voice in a second region which is a region with the touch panel;
A character information duplicating unit that duplicates at least a part of the character information displayed in the second area in a first area different from the second area of the touch panel;
An image information display unit for displaying the image information input in the first area of the touch panel in the first area together with the character information duplicated in the first area;
Free description area information storage that associates and stores character information copied in the first area and image information input in the first area that is in the vicinity of the character information copied in the first area And
An information display device. In an information display device that displays information on a touch panel,
A character information display unit that displays character information obtained by voice recognition in a first region that is a region of the touch panel;
An image information display unit for displaying the image information input in the first area of the touch panel in the first area together with the character information;
An information display device.   A program for causing a computer to realize the steps of the information display method according to claim 1.

Images