JP2005017949A - Document creation support system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a document creation support system that achieves high efficiency. <P>SOLUTION: The document creation support system comprises: a document input means that generates coordinate data or key data; a first storage means that stores first storage information obtained by adding time information to an input signal to be concerned with; a second storage means that records second storage information obtained by adding the time information to a sound signal taken in by a sound input means; and a means for retrieving the second storage information, which is stored in the related second storage means, from the first storage information of the first storage means through the intermediary of the time information to make the second storage information output from a sound reproduction apparatus; or a means for retrieving the first storage information, which is stored in the related first storage means, from the second storage information of the second storage means to make a display device display the first storage information. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a document creation support system, for example, a technique effective for use in recording a lecture or a lecture performed while drawing characters and figures on a board.
[0002]
[Prior art]
In order to record a lecture or the like in a document, it may be possible to document it using voice recognition software. The voice recognition technology is a technology that allows a computer to directly recognize words and sentences spoken by a person, and has attracted attention as a simple input means that replaces keyboard input and pen input that are currently used as input means. Examples of documents related to speech recognition include the following.
[0003]
[Non-Patent Document 1]
Y. Obuchi, A.A. Koizumi, Y. et al. Kitahara, J. et al. Matsuda, and T.M. Tsukada, Proc. EUROSPEECH '99, pp. 2023-2026, 1999
[Non-Patent Document 2]
Via Voice (IBM)
[Non-Patent Document 3]
Naturally Speaking (Dragon)
[0004]
[Problems to be solved by the invention]
Since the above dictation software tries to document everything recognized as voice data, it is not suitable for creating a record that summarizes the whole meeting minutes.
There is also a problem that an unexpected recording is created due to a voice recognition error. When creating a record in real time, such as at a lecture, there are many things that you may want to reconfirm where you missed. Therefore, we considered recording both audio data and document data. However, if each was recorded independently, the location where the recording error occurred was relied on memory, and the search was performed while reproducing the audio data, or a rough time was spent. The problem is that the search is performed while estimating and the efficiency is low.
[0005]
An object of the present invention is to provide a document creation support system that realizes high efficiency. The above and other objects and novel features of the present invention will be apparent from the description of this specification and the accompanying drawings.
[0006]
[Means for Solving the Problems]
The outline of a typical invention among the inventions disclosed in the present application will be briefly described as follows. Coordinate data or key data is generated by the document input means, first storage information obtained by adding time information to the input signal is stored in the first storage means, and time information is stored in the voice signal captured by the voice input means. The second storage information added is recorded in the second storage means, and the second storage information stored in the second storage means related thereto is changed from the first storage information of the first storage means through the time information as a medium. A search is made to output from the audio reproduction device, or the first storage information stored in the first storage means related to the second storage information of the second storage means is searched and displayed on the display device.
[0007]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 shows a block diagram of an embodiment of a document creation support system according to the present invention. In the document creation support system of this embodiment, two types of input means are provided. In this embodiment, a system configuration based on a personal computer / tablet input device is shown. A document to be recorded through an input I / F such as a pen, a keyboard, and a mouse is stored with a time stamp added by an input document recording module. Here, elements constituting the document include not only characters but also figures, pictures, deletion records, scene change information, and the like. On the other hand, the voice is input through the microphone simultaneously with the document input, and the voice data with the time stamp added from the voice input start time to the end time is stored by the recorder.
[0008]
When the stored document data is displayed on a display medium (display) such as a personal computer / tablet input device and the user makes a request to extract the stored document data using the pen / keyboard / mouse, it is instructed that there is a recording voice confirmation instruction. The time corresponding to the selected document position is taken out, the playback device is controlled from the control device, and the stored sound at the corresponding time is played back through the playback device.
[0009]
The elements constituting the document are characters, figures, pictures, etc., deletion records, scene change information, etc. composed of coordinate signals such as characters or figures drawn on the tablet or whiteboard. The pen input, key input, or mouse input is also used as an input for a search operation as described above. For example, when an audio signal corresponding to an explanation of a figure or the like is taken out by a user who creates a document on the screen of a display device that displays stored document data, the area on which the figure is drawn is pen-input on the board Alternatively, by moving the cursor on the display screen with a keyboard or mouse and instructing a part of the figure, the recording voice confirmation instruction is given, and the control device gives the voice signal from the time stamp of the stored voice signal. Is taken out and output from the speaker via the playback device.
[0010]
On the other hand, when creating a document by outputting the stored voice through the speaker by the playback device, it is possible to specify a part whose meaning is unknown from the voice alone, time information specification using a keyboard, etc., or storage voice specification using a GUI or the like. Corresponding characters and figures can be displayed on the display device. As described above, it is possible to reproduce and display the document input and the voice input in association with each other through the time stamp.
[0011]
As in this embodiment, a user who creates a document associates the recording created in real time with a keyboard of a personal computer, a touch panel such as a tablet PC or PDA, or a pen input system with the voice data on the spot at the time. Select a part of the record entered with the device and keyboard, touch panel, or pen input system, and create a document or graphic format record with the audio as the data source by using the playback device to associate the selected data with the time I want to check the document or graphic recording, such as when I missed a keystroke or pen speed that I could not hear at times, or when I missed the record, or when I want to check the document or graphic recording or the audio source data If a location is selected, the associated audio data can be played back at that time. , Sure can as to become documentation of confirmation or sound source data of the record of the graphic form or fill in the record leakage becomes easy.
[0012]
FIG. 2 shows an explanatory diagram of an embodiment of a method for accumulating handwritten information which is document input data in the document creation support system according to the present invention. Characters and figures drawn with a pen on a whiteboard or tablet are sampled from the coordinate data, and an absolute time (time stamp) is added to the header to form one element 1. A plurality of elements are grouped by the temporal distance and the spatial distance to form a group 1 corresponding to the character “A”. Such a group is further grouped and recorded in the recording module.
[0013]
FIG. 3 shows an explanatory diagram when a voice input is searched from a document input in the document creation support system according to the present invention. In the figure, an example in which a document that can be grooved at a fixed time, an object unit of both format records, and audio data are associated with each other is shown. The document data has one large group 1, and the large group 1 includes groups 1 to 3. Group 1 is composed of element 1 to element 3, group 2 is composed of element 1, and group 3 is composed of element 1 to element 4. When such a period of the large group 1 is designated, the voices 1 to 3 close to the period in which the large group 1 is included are searched from the voice data and are reproduced in the order of the voices 1 to 3.
[0014]
If any one of the specific groups 1 to 3 is specified instead of the large group 1, the sounds 1 to 3 closest to the specified group are searched and reproduced. If the sound 2 searched for the document data group 2 does not correspond to the document data, the previous sound 1 or the next sound 3 is designated and reproduced from the contents of the sound 2. To find the corresponding voice.
[0015]
In this way, the element (or group) corresponding to the time of the search key is searched from the stored document database in which all elements are stored. The search results are reproduced in order of time. When searching, what kind of information (audio, video, document, etc.) is required depends on the case, so add time information (preferably in the same format) to all data and search for any data at a specific time.・ It can be played back. In addition, as for the functions necessary for the search, a document corresponding to voice is rarely input at the same time, and the voice may be earlier or later as described above. This correction can only be done manually by hand, but as a support for the correction, there may be an expression method that can estimate the time lag just by looking, such as a speech recognition result (OK even if there is a recognition error). . It is also convenient to be able to prepare a function that can automatically determine the time lag from the voice recognition result and the content of the document and group it.
[0016]
FIG. 4 is an explanatory diagram for explaining the relationship between document input and voice input in the document creation support system according to the present invention. In the case of sentences, the units for adding time information to the document data may be character units, vocabulary units, sentence units, punctuation marks, space units, line feed units, and conversion processing units. Further, graphic information, deletion information, scene change information, etc. may be used as event occurrence units. In addition to this, an object unit that can be grouped at a fixed time regardless of the recording data format may be used. FIG. 4 shows an example of associating a document that can be grouped at a fixed time as described above, an object unit of both format records, and audio data.
[0017]
FIG. 5 shows a block diagram of another embodiment of the document creation support system according to the present invention. In this embodiment, the input side and the search side are shown separately. From the document input IF, coordinate data such as characters and figures drawn using a touch panel (tablet or whiteboard) is input as described above, and characters and symbol code data input using a keyboard or the like are input. .
[0018]
The coordinate data is converted into a coordinate sequence with a time stamp. This time-stamped coordinate string is TM · x1, y1, x2, y2,. Here, TM is a time stamp, and END is an end mark. These are recorded as the database 1 in the storage device. The character and symbol code data is also converted into a time-stamped key character string as described above. The time stamped key character string is TM · A · B · C · D... END as shown in the figure. Here, TM is a time stamp, and END is an end mark. These are recorded as the database 2 in the storage device. The audio signal is input by voice through a microphone, and a time stamp is added by a recording device with a time stamp and recorded as a database 3 in a storage device.
[0019]
The recorded data of the databases 1 and 2 are displayed on the screen of the display device. The recorded data in the database 3 is reproduced by an audio reproducing device. For example, when a document is created, the recorded data of the database 1 is displayed on the screen of the touch panel display device, and the area in which the corresponding graphic is drawn is touched, so that the graphic drawn in the area is displayed. The search time T is extracted from the added time stamp information, and an audio signal to which the corresponding time stamp is added is selected from the database 3 based on the search time T and reproduced.
[0020]
In addition, at the time of creating the document, the time stamp information added to the character string by specifying the corresponding character string with a pen or cursor while the recorded data of the database 2 is displayed on the screen of the display device. The search time T is extracted from the voice signal, and the audio signal with the corresponding time stamp added is selected from the database 3 based on the search time T and reproduced.
[0021]
Furthermore, at the time of creating the document, a portion of the audio signal recorded in the database 3 that is difficult to hear is recognized by speech, and the search time T is extracted from the corresponding time stamp information. Then, the database 1 or 2 is searched, and the corresponding graphic or key character string is displayed on the display screen.
[0022]
In this way, when creating a document with audio as a data source, when it is not audible when creating a record in the figure format, or when the key input or pen speed cannot keep up with the record omission, confirmation of the document, diagram format record, or audio source data If you want to check the document or graphic recording, you can play the associated audio data at that time, fill the record omission, check the graphic recording, or record the audio source data. You can check it.
[0023]
According to the document creation support system of the embodiment described above, the voice information recorded at the same time can be reproduced by directly pointing to an unclear point in the manually or keyed document. This makes it possible to complete an accurate document without any problems. PC (personal computer, the same applies hereinafter) and PDA users can be expanded by proposing a new recording format by associating input documents with audio data.
[0024]
The invention made by the present inventor has been specifically described based on the embodiments. However, the present invention is not limited to the above embodiments, and various modifications can be made without departing from the scope of the invention. Nor. For example, the document input means may take various embodiments, such as one that captures characters or figures as coordinate data, one that inputs a key code like a keyboard, or one that uses cursor or positional information of a mouse. it can. The audio signal may be added with a conversion function that eliminates silence or increases the playback frequency within the recognizable range of the sampling frequency during recording so that it can be heard quickly. Good. The present invention is a document creation support system that can be combined with a network shared whiteboard and voice chat, as in a PDA, notebook PC, tablet PC, electronic blackboard, video conference system, and electronic conference system (net meeting). Widely available.
[0025]
【The invention's effect】
The effects obtained by the representative ones of the inventions disclosed in the present application will be briefly described as follows. Coordinate data or key data is generated by the document input means, first storage information obtained by adding time information to the input signal is stored in the first storage means, and time information is stored in the voice signal captured by the voice input means. The second storage information added is recorded in the second storage means, and the second storage information stored in the second storage means related thereto is changed from the first storage information of the first storage means through the time information as a medium. By retrieving and outputting from the audio reproduction device, or by retrieving the first storage information stored in the first storage means related to the second storage information of the second storage means and displaying it on the display device, the document High efficiency of creation can be realized.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an embodiment of a document creation support system according to the present invention.
FIG. 2 is an explanatory diagram showing an embodiment of a method for accumulating handwritten information as document input data in the document creation support system according to the present invention.
FIG. 3 is an explanatory diagram when a voice input is searched from a document input in the document creation support system according to the present invention.
FIG. 4 is an explanatory diagram for explaining the relationship between document input and voice input in the document creation support system according to the present invention.
FIG. 5 is a block diagram showing another embodiment of the document creation support system according to the present invention.
[Explanation of symbols]
TM ... Time stamp, x1, y1, x2, y2 ... Coordinate data, A, B, C, D ... Key input information, END ... End mark.

Claims (3)

Document input means for generating coordinate data or key data;
First storage means for storing first storage information obtained by adding time information to the input signal captured by the document input means;
Voice input means;
Second storage means for storing second storage information obtained by adding time information to the voice signal captured by the voice input means;
A display device for displaying from the first storage information stored in the first storage means;
An audio reproduction device for reproducing an audio signal from the second storage information stored in the second storage means;
Using the time information as a medium, the second storage information stored in the second storage means related to the first storage information of the first storage means is retrieved and output from the audio reproduction device, or the second storage means And a control device for retrieving the first storage information stored in the first storage means related thereto from the second storage information and displaying it on the display device. In claim 1,
A document creation support system, wherein the coordinate data captured by the document input means is sampled and used as the input signal. In claim 2,
The coordinate data includes continuous coordinate data as elements, and a plurality of elements are grouped by time distance and spatial distance, and the time information is added.
The document creation support system, wherein the plurality of groups are further grouped to form a large group.

Images