JPS63201821A - Voice memorandum system - Google Patents

Abstract

PURPOSE:To transpose and edit partial contents of a memory by providing a voice input and recording/reproducing means and a means which allows recorded voice and display to correspond to each other. CONSTITUTION:A document stored in a picture memory 34 is displayed on a picture display 36 through a picture buffer 35 by the control of a picture control processor 30. The voice inputted as voice memorandum is inputted to a voice detecting part 26 and an input voice buffer 27 through an encoding part 24 of coding. A voice processing processor 20 finds the output of the detecting part 26 and stores the input voice signal of the buffer 27 in a voice memory 29. A control processor 11 executes correspondence between picture display and voice memorandum and the control of the whole of the system while communicating with processors 20 and 30 through a voice interface part 23 and a picture interface part 33. Information input from a keyboard 13 and input of a pointing device 14 are controlled by the processor 11 to allow voice and picture to correspond to each other.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a voice memo system, and particularly to a system suitable for adding voice memos to electronic documents.

[Conventional technology]

Conventional devices have features such as indicating the position where a voice memo has been added as a mark, displaying the length of the voice memo with a bar, and showing how far the voice memo has been played and how much is left. being done. Incidentally, related devices of this type include, for example, Japanese Patent Application Laid-Open No. 13730/1982 and Japanese Patent Application Laid-open No. 248056/1989.

[Problem that the invention seeks to solve]

When adding voice memos, you can't change what you say or change the level of explanation while watching the other person's response, as you would in a normal conversation. If you don't do it all together, it will be very difficult to listen to. For this reason, it is necessary to re-edit the voice memo once it has been added to make it easier to listen to by rephrasing, rearranging, or adding a part of the voice memo, but the above technology does not take this point into consideration.

Also, audio is a one-dimensional signal in terms of time, and unlike images, it is not possible to view the information at a glance and find out which information is correct, and it is necessary to go through each voice memo in order. Yes, it is not efficient. Regarding this point, a method has been proposed in which the playback of voice memos is spatially distributed using stereo technology to create a cocktail party effect, but this method is not effective. This is because the cocktail party effect is small under conditions where voices from the same speaker are heard simultaneously.

SUMMARY OF THE INVENTION An object of the present invention is to provide a means for easily adding a voice memo and efficiently retrieving a necessary voice memo.

[Means for solving problems]

The above purpose is achieved by adding a simple voice editing function that allows you to paraphrase a part of a memo or change its order, and a function that efficiently plays back the main parts such as the main sentences of each memo. Ru.

[Effect]

In order to achieve the above object, the present invention provides voice input and
In addition to recording and reproducing means, means for associating recorded audio and display, and various pointing and function selecting means, audio editing means and audio fast forward reproducing means are provided. Furthermore, this audio input, recording, and reproducing means is provided with means for detecting audio sections. Further, if a function is provided to inform the inputter that the end of the voice section has been confirmed after 1 to 2 seconds have passed after detecting the end of the voice section, it will be easier for the user to use. It goes without saying that the use of high-efficiency audio encoding means and decoding means for the recording means and reproducing means provides the greatest advantage in terms of memory capacity.

The editing means constitutes an easy-to-use editing means by combining audio input/recording/playback means, means for associating audio and display, pointing and various function selection means, and the like. When you select voice input or editing mode with the function selection means,
Set an editing screen near the screen of the document to which you want to add a voice memo. The editing screen can be realized in a so-called multi-window configuration. The editing screen is provided with an editing function that displays memo audio in units of sentences, etc., and allows insertion, replacement, deletion of audio, designation of key sentences, etc. in combination with pointing means.

The audio fast-forwarding means plays back only the main parts of the audio memo attached to the document one after another while displaying the playback position, and has several playback modes. The first one is to play about 1-2 seconds from the beginning of each memo, the second one is to speak quickly and also plays from the beginning for 1-2 seconds, and the third and fourth ones are to play key centimeters. Only the responses are played from the head at normal or rapid speed. By gradually lowering the reproduced sound level at the tail of each reproduced sentence and reproducing the next reproduced sentence overlappingly, natural and easy-to-listen fast-forward reproduction is possible.

Additionally, there is usually a mark that indicates where a voice memo has been added in a document, but if you do not use voice memos, you may want to prohibit the display of this voice memo mark or set the By using a function that changes the color or blinks the relevant part of a document instead of a mark to indicate the playback position, audio and document can be more directly and smoothly matched, and the target part Reducing the brightness of the outside display or making it slightly blurred to more naturally indicate the location of attention has a great effect on natural correspondence.

As the input and function selection means, a special light pen or a mouse may be used, and it goes without saying that the functions can also be realized by a menu selection method.

By using these various functions in combination, it is possible to realize an effective and easy-to-use voice memo system.

〔Example〕

An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram for explaining the present invention.

In FIG. 1, a document stored in an image memory 34 is transferred to an image buffer 3 under the control of an image control processor 30.
5 and is displayed on the image display 36. This configuration is realized as an image terminal with a normal multi-window function. The image memory 34 stores each segment image information necessary for multi-window display, and the image buffer 35 edits and outputs the segment image information to the display screen.

The voice input as a voice memo passes through a coding encoder 24 and is input to a voice detector 26 and an input voice buffer 27. When the audio processing processor 20 looks at the output of the audio detection unit 26 and determines that audio has been input,
The input audio signal in the input audio buffer 27 is stored in the audio memory 29 via the audio memory interface 28.

Reference numeral 21 denotes a memory section for the audio processing processor, in which programs and the like are stored. The audio detection unit 26 calculates the short-term power of the input audio, and when the short-term power reaches a predetermined threshold and continues for a certain period of time or more, it is determined that there is audio, and this technology is used to detect the threshold for a certain period of time or more. This can be realized using known technology.

The voice memo is reproduced under the control of the voice processing processor 20, from the voice memory 29, through the voice memory interface 28, through the output voice buffer 22, and then output through the copic decoding section 25.

The control processor 11 performs the correspondence between image display and voice memo and overall control via the voice processing processor 2 via the voice interface section 23 and the image interface section 33.
0 and the image control processor 30. Also record the order of memo creation in the correspondence table. Here, information input from the keyboard 13 and input from the pointing device 14 are also performed by the control processor 1.
1 manages the data in a unified manner and associates them in the same way as audio and images. 12 is a memory section for the control processor 11;
31 is a memory section for the image processing processor 7 processor 30;

The memory unit 11 also records a correspondence table that associates the voice memo with the image part.

First, the case where a voice memo is attached to a specific location on a displayed screen will be explained using FIG.

Using the pointing device 14, first specify the voice memo addition function from the menu at the bottom of the screen, and then specify the position (2) to which you would like to add the memo, and the voice memo editing screen B is displayed.

As the pointing device, devices that are already widely used such as a mouse can be used. The display position of the voice memo editing screen B is automatically determined depending on where on the screen the memo is attached. This can be achieved by roughly dividing the screen into several areas and determining the display position according to a predetermined table depending on which area the memo position (■) falls within.

As shown in the figure, the editing screen B displays two bars (auxiliary marks) surrounded by broken lines and two bars (audio recording marks) surrounded by solid lines, which are alternately arranged horizontally. This screen is an image segment that is separate from the original document data and is stored in a different location from the original document in the image memory, edited on a single screen buffer and displayed as a single screen. . When a memo voice is input, the inside of a bar surrounded by a solid line is filled in between each voice (usually a sentence with one voice). The control processor 11 creates a correspondence table between the position of the data to be marked with a memo on the screen and the voice memo, stores the table in the memory 12, and instructs the voice processor 20 to store the input voice in the voice memory 29. instruct. As shown in FIG. 3, this correspondence table has information on the order and position of each of the plurality of audio sentences that make up the memo.

An example of the correspondence table will be explained using FIG. (A) shows the status of voice memos on a screen as a whole, and (B) shows the status of individual voice memos. In (A), a series of numbers are attached to the voices displayed on the screen, and the information is arranged in the order of the numbers, and each voice memo has a number that indicates the memo number of the next voice memo. and a corresponding table pointer that points to a table (B) prepared for each voice memo indicating the status of each voice memo. The information of voice memo number 0 includes pointer information Po, which is the head information of the voice memo start, and the number n of memos.

When a voice memo number i is selected during editing or the like, the individual voice memo information table (B) corresponding to ti is selected based on the information t of the correspondence table pointer.

The individual voice memo information table (B) includes voice memo numbers i,
Voice X is created by the creation order number (in the creation order in the table (A)
(equivalent to memo creation order information on that screen), information regarding the position in the voice memory 29 where the voice information is actually stored, and an image for making a correspondence with the position on the screen to which this memo is added. It consists of information associated with information regarding the position of the image on the memory 34, the number N of audio sentences composing the audio memo, and information regarding individual audio sentences.

Information about each audio sentence includes the actual starting position where the audio is stored (this information may be created in the audio processing unit), an importance mark indicating whether the sentence is a key sentence, and the next audio sentence. It consists of a pointer indicating which one. By operating this pointer, the order within the voice memo can be edited.

If you use the information obtained by such a configuration.

It is easy to understand that the voice memo can be edited and the playback order can be changed arbitrarily, such as input order or screen position order.

Furthermore, there may be various modifications to the structure of the tables, such as moving a part of these tables to audio memory, image memory, etc., but the specific structure of the table does not limit the point of the present invention. is clear.

When an instruction to finish adding a voice memo is input using the pointing device 14 or the like, the editing screen disappears and a display is attached to the position of the screen where the memo was attached so that it can be seen that the voice memo has been attached. This display data is also recorded in the image memory 34 as an original segment separate from the original document. From this, if you want to display only the original document, it becomes possible to easily erase the display such as the addition of a voice memo.

Next, a case of editing in which the order of voice sentences within one voice memo is changed, inserted, or deleted will be explained.

When one editing function is selected from the menu, the menu screen changes to a menu of various editing functions, and an editing screen 13 similar to that for adding a voice memo is displayed in accordance with the specified position. The bar surrounded by a solid line is displayed with the inside filled in by the number of voice memos attached.

First, if you want to listen to and see the entire memo, select the playback function from the menu, and the bar at the position of the audio sentence to be played will blink and play in sequence.

If you want to erase a specific audio sentence, select the bar for that audio and the delete function, and that audio sentence will be erased, and subsequent filled-in bars will move up one by one.

If you want to insert an audio sentence, select the dashed line bar at the position you want to insert and the insertion function, and the subsequent filled bars move down one by one, and the one unfilled bar inside the solid line moves to the insertion position. will be displayed. If you say the audio sentence here, a voice memo will be inserted,
If you want to swap the positions of a certain audio sentence whose bar is being filled in, you can do so by selecting the filled-in bar that represents the audio sentence, the dashed line bar to which you want to move, and the movement function.

If you want to specify that a particular audio sentence is particularly important in the memo, you can do so by selecting Key Sentence Designation from the menu and specifying the bar that indicates the audio sentence that will be the key sentence.

Next, we added a function that responds to voice input to make it easier for people who add voice memos to say their notes. In Figure 1,
1 to 2 after the voice section detection unit 26 detects the tail of the voice section.
After a few seconds have passed, the control unit 11 instructs the image display processor 30 to display a response confirming that there has been a voice input. Based on this instruction, the image display processor 30 displays a confirmation display on a part of the display screen for several looms that rises and falls with a time constant of several tens of milliseconds.

As a display, there are methods such as changing the brightness or size of the display indicating the presence of the voice memo. Also, instead of displaying an image, you can instruct the audio processor and use audio output to
“Hi” or “Is that so?” It is also known from research by Ishii et al. that it is easier to respond if you respond within 1 to 2 seconds after uttering the voice. This is the one using the second ticket 2.

The results of these processes are all managed by the control processor 11, necessary correspondence tables etc. are modified each time, and commands regarding memory management for modification are issued to the image control processor 3o and the audio processing processor 20. , each processor performs necessary processing. These processes themselves can be easily realized using various techniques used in current multi-window display terminals, so they will be omitted here.

When listening to a voice memo, in addition to selecting the voice memo playback menu and specifying the position, a mode is provided in which the voice memo can be played back by simply instructing to display the voice memo directly.

Next, a case will be described in which fast-forward playback of audio is performed.

When the fast-forward playback function menu is selected, menus such as normal fast-forward, fast-forward playback, and fast-forward playback using key sentences are displayed. In addition to the screen position order, a menu of playback functions in the order of voice memo creation is also provided for the display order. In normal fast-forward playback, a voice memo of about 2 seconds is played from the top of each menu sequentially from the top left to the bottom right of the screen, and the corresponding location is displayed on the screen so that you can see the corresponding location as it plays. make it blink. Further, it is also possible to display a light shade in areas other than the vicinity of the target area.

In the recording order playback mode, the voice memo is played back in the order in which it was created, and all the memos are played back.This function is useful when it is necessary to reproduce the author and the order of ideas when creating the memo.

Fast playback speeds up the speed of the audio when playing back the voice memo.If the voice memo is recorded using a parameter encoding method such as the well-known PARCOR method, the reading timing of the parameters will be faster. It is possible to realize changes in the pitch of the voice, such as fast forwarding on a tape recorder, without being perceptible.

Fast-forward playback using key sentences does not start from the beginning of each memo, but from the beginning of the key sentence for a certain period of time (1 to 2
This is a method in which the data is played back in sequence (seconds) at a time.

In these fast-forward playbacks, if the memo is suddenly switched to the next memo at regular intervals, it becomes unnatural and extremely difficult to listen to. In the present invention, the volume of the preceding playback sound is gradually reduced near the end of its playback, and the playback sound of the next memo is overlaid on top of it, so that it is possible to listen to the voice memos one after another in a natural manner. . To achieve this,
In FIG. 1, the parameters that determine the audio level of the audio data taken out from the audio memory 29 are gradually reduced by the audio memory interface unit 28, read out at a high speed approximately twice the real time, and The preceding voice memo data written to the output buffer on one side of the side buffer is added to the data of the preceding voice memo written in the buffer memory on the other side by the adder provided in the output control unit 22, and the result is decoded by Copeic. The signal is sent to section 25 and reproduced and output as audio. Depending on the audio compression method, there are methods that do not allow addition using parameters, and in this case, the output control unit 22 and the decoding unit 25 may be configured to weight the multifunction device amplitude on the linear waveform or to add both. Needless to say, it is necessary to modify the order and structure of the system as appropriate to suit each system.

If you want to listen to only important memos, select the important memo selection menu to sequentially reproduce only the voice memos containing voice sentences designated as important.

In addition, a menu selection mode is provided in which only the original document information is displayed, excluding the voice memo-related display.

The main steps of the above editing process are expressed in PAD in the 4th WI.
Shown below. See Figure 3 for equivalent terms. Just as there are already various implementation methods for implementing a processing procedure using a pointing device, it goes without saying that this procedure also has various modifications.

It is clear that the claims of this patent are not affected by the means and procedures used to implement the procedure itself.

[Effects of the Invention] According to the present invention, it is possible to easily add an arbitrary voice memo to documents, images (medical images, etc.), and drawings (maps, design drawings, etc.) displayed on the screen, and easily modify them. This allows one person to easily handle information because the user can read the memo and listen to the summary of the memo efficiently.

[Brief explanation of the drawing]

FIG. 1 is a block diagram explaining one embodiment of the present invention, and FIG.
3 is a diagram for explaining the present invention in detail, FIG. 3 is a diagram showing correspondence between a voice memo and an image memo in the present invention, and FIG. 4 is a diagram showing the flow of editing processing of a voice memo in the present invention. . 11... Control processor, 13... Keyboard, 1
4... Pointing device, 20... Audio processing processor, 29... Audio memory, 30... Image control processor, 34... Image memory, 35... Image buffer, 36... Image display .

Claims (1)

[Scope of Claims] 1. In a voice memo method for attaching a memo by voice to a displayed predetermined image, means for specifying an arbitrary position on the image for attaching the voice memo; means for inputting a voice to be attached to a position; means for recording the input voice as a voice memo; means for associating the position on the image with the recorded voice memo; and editing the recorded voice memo. and means for reproducing the edited voice memo based on designation of a predetermined position on the image. 2. The voice memo system according to claim 1, wherein the means for editing the recorded voice memo includes means for displaying an editing screen. 3. In the voice memo system according to claim 2, the editing screen includes a plurality of marks indicating the recording of the voice in units of audio sentences, and an editing screen sandwiched between the recording marks. A voice memo method characterized by having an auxiliary mark for use. 4. The voice memo system according to claim 3, wherein the editing screen has a voice recording mark whose display state changes as the voice is actually input. Voice memo method. 5. In the voice memo method described in claim 1, the means for editing the recorded voice memo has a function of selecting and specifying important ones from among the voice sentences making up the voice memo. Voice memo method. 6. The voice memo system according to claim 1, wherein the means for editing the recorded voice memo has a function of rearranging the order of voice sentences constituting the voice memo. . 7. In the voice memo system set forth in claim 1, the means for editing the recorded voice memo has a function of inserting a new voice sentence at any position between the voice sentences constituting the voice memo. The voice memo method is characterized by: 8. In the voice memo system as set forth in claim 1, the means for editing the recorded voice memo has a function of erasing any of the voice sentences constituting the voice memo. Memo method. 9. In the voice memo method described in claim 1, the means for editing the recorded voice memo changes the display state of the voice recording mark as the voice text constituting the voice memo is deleted or inserted. Voice memo method with correction function. 10. In the voice memo system set forth in claim 1, the means for inputting the voice includes means for outputting confirmation that voice input has occurred 1 to 2 seconds after detecting the tail of the section of the input voice. The voice memo method is characterized by: 11. In the voice memo system as set forth in claim 1, the reproducing means includes fast-forward reproducing means for sequentially reproducing only a range of about 2 seconds from each of the series of voice memos on the display screen. Voice memo method. 12. The voice memo system according to claim 11, wherein the reproducing means includes means for correcting and reproducing the voice memo at a faster rate than the utterance rate of the input voice. 13. In the voice memo system according to claim 11, the reproduction means selects only voice sentences designated as important from among the voice sentences constituting the voice memo and performs fast-forward reproduction. Voice memo method with 14. In the voice memo system as set forth in claim 11, the reproduction means has means for gradually reducing the audio intensity of the tail portion of the previously reproduced voice and taking over to reproduction of the next voice memo. Voice memo method.