JP3704968B2 - Multimedia editing device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a multimedia editing apparatus that supports editing and processing of a presentation using multimedia.
[0002]
[Prior art]
Conventionally, when editing and processing a presentation using multimedia, the presentation that has already been created is played back in chronological order, and the part that needs to be edited and processed is detected and played back in parallel with that part. Editing was performed while considering attributes such as whether the multimedia information being a still image or a movie.
As an invention related to multimedia, Japanese Patent Laid-Open No. 6-176542 discloses that a script is associated with a sound converted into text, and a predetermined video portion is searched by associating the script with the video. A method is described.
[0003]
In JP-A-5-67184, when deleting multimedia managed by a segment, the user sets in advance the association between media and attributes (time shift enable / disable, disconnect enable / disable compressible). Describes an apparatus for referring to the information and checking the attribute of the media related to the multimedia to be deleted, and performing an editing process according to the attribute. Furthermore, Japanese Patent Laid-Open No. 7-182365 describes a method of extracting a keyword from a conversation during a conference by voice recognition, and searching for an important scene by the keyword to create a conference record.
[0004]
[Problems to be solved by the invention]
When editing locations for editing and processing multimedia presentations, the created presentation is played back in chronological order to detect the portion that needs to be edited and processed, and the starting point of that portion. Therefore, it takes a very long time to specify the end point while repeating the rewinding, fast-forwarding, and the like of the multimedia data. In addition, the editing work after specifying the editing part needs to be edited by taking into account the attributes of the plurality of multimedia information in the editing part. .
[0005]
On the other hand, according to the method described in JP-A-6-176542, a predetermined video portion can be easily searched. However, this publication has a problem that there is no description at all regarding the editing process after the video is searched, and the trouble of the editor cannot be reduced. Further, since the video portion that can be searched is only the portion associated with the script, there is a problem that it is not possible to search for a portion related to an utterance that is not described in the script.
[0006]
Further, according to the device described in Japanese Patent Laid-Open No. 5-67184, the editor does not require much trouble at the editing stage. However, in order to perform editing with this device, the user must insert media such as video before editing. It is impractical to associate with other media during playback and set media attributes. Further, according to the method described in Japanese Patent Laid-Open No. 7-182365, it is possible to easily search for a scene by using a keyword. However, since it does not contribute to the subsequent editing at all, the labor of the editor can be reduced. Can not. In addition, there is a problem in that it is impossible to search for portions other than the keyword.
[0007]
An object of the present invention is to provide a multimedia editing apparatus that can easily and appropriately edit a presentation in a short time.
[0008]
[Means for Solving the Problems]
The object is to provide information holding means for holding a plurality of video and audio media information, scenario holding means for holding a scenario file describing media information to be presented in a presentation, a presentation position of media information, and a presentation time, and a scenario file A voice recognition means for recognizing a character string from predetermined audio media information described in the above and creating text data of the character string; a time domain detection means for detecting a time domain in which the character string is emitted; and text data A multimedia-related information holding unit that holds the time region in association with each other, a display unit that displays text data, an instruction unit that specifies the text data that specifies an editing range, and a predetermined reproduction that is performed in the time region For determining the attribute information related to temporal changes for media information subject to editing evaluation A media attribute determination unit, and a scenario editing unit that edits a scenario file for the time region corresponding to the instructed text data based on the attribute information about the media information to be edited and evaluated that is reproduced in the time region. This is achieved by a multimedia editing device.
[0009]
DETAILED DESCRIPTION OF THE INVENTION
Before describing a multimedia editing apparatus according to an embodiment of the present invention, a presentation using multimedia will be briefly described. FIG. 1 shows an example of a presentation using multimedia.
In this presentation, a display screen 1 displays a moving image 2 representing a person (presenter) giving a presentation, a moving image 4 of a product explained in the presentation, and a still image 3 of a graph related to the product. In both cases, the presenter's audio 6 is output from the speaker 5. Thus, according to the presentation using multimedia, it is possible to obtain an effect as if listening to the presentation in an actual conference room.
[0010]
In such a presentation using multimedia, the reproduced video 2, video 3, video 4, audio 6, etc. are stored as multimedia data (media information), respectively. Playback is based on a scenario file that describes the playback position of media information, playback timing, playback effects, and the like.
The scenario file is created in advance by a presenter, and is distributed to each viewer's information processing apparatus by communication means via a network, for example.
[0011]
The multimedia editing apparatus according to the present invention is an apparatus for editing the scenario file as described above. Hereinafter, a multimedia editing apparatus according to an embodiment of the present invention will be described with reference to FIGS. First, a schematic configuration of a multimedia editing apparatus according to an embodiment of the present invention will be described with reference to FIG.
[0012]
In FIG. 2, a multimedia information storage unit 13 stores a plurality of media information such as video and audio. The scenario holding unit 11 holds a scenario file 11a in which media information used for presentation, a position at the time of reproducing the media information, a reproduction timing (time, time, etc.), an effect at the time of reproduction, and the like are described.
The scenario analysis unit 6 interprets the contents described in the scenario file 11a, extracts the media information described in the scenario file 11a from the multimedia information storage unit 13, and describes the described timing, the described processing method, and the like. Accordingly, if the media information is media information of a predetermined voice, it is output to the voice recognition unit 15, and if the media information is media information subject to editing evaluation other than the voice, it is output to the multimedia attribute determination unit 14. Here, the predetermined audio media information is audio media information for extracting text data as a reference for specifying the editing range, as will be described later, and any audio media described in the scenario file. It may be information, or may be audio media information designated by the user.
[0013]
The voice recognition unit 15 creates text data of a character string from the voice data in the voice media information input from the multimedia information storage unit 13, and outputs the text data to the multimedia related information holding unit 16. Further, the voice recognition unit 15 detects a time region in which the voice data corresponding to the created text data is reproduced, and outputs the time domain to the multimedia related information holding unit 16. In the present embodiment, the voice recognition unit 15 creates text data for each syllable break of voice data, that is, for each continuous voiced portion, and the multimedia related information holding unit 16 in units of each text data. Output to.
[0014]
Based on the media information input from the multimedia information storage unit 13, the multimedia attribute determination unit 14 edits media information that is reproduced simultaneously with the audio data corresponding to the text data created by the audio recognition unit 15. The attribute information of the media information to be evaluated is determined and output to the multimedia related information holding unit 16.
Here, the media information subject to editing evaluation refers to media information that should be considered for editing according to attribute information, as will be described later, and may be media information other than the audio media information. It may be media information directly designated, or media information indirectly designated by the user, such as media information other than specific media information.
[0015]
The attribute information is information regarding whether or not the media information changes with time. For example, in the case of video media information, it is information such as whether the image is a still image or a moving image. is there. As a method of determining attribute information, for example, when attribute information of multimedia information is described in the scenario file 11a, a method of determining by description of the scenario file 11a, or in the case of video media information There is a method of determining whether the image is a still image or a moving image by taking out the video at regular intervals and comparing the difference with the previous video.
[0016]
The multimedia related information holding unit 16 reproduces the text data input from the voice recognition unit 15, the time range in which the corresponding audio data is reproduced, and the voice data input from the multimedia attribute determination unit 14. Multimedia-related information in which attribute information related to the media information to be edited is associated.
The display / instruction unit 11 displays the text data held in the multimedia related information holding unit 16 on the display device 17a, and issues a selection instruction for the text data as an editing range from the user using the pen-type pointer 17b or the mouse device 17c. Accept and instruct the scenario editing unit 18 to edit the editing range corresponding to the instructed text data. The scenario editing unit 12 edits the scenario file 11a based on the multimedia related information of the multimedia related information holding unit 16 corresponding to the text data instructed by the display / instruction unit 11, and creates a new scenario file 19.
[0017]
Here, the information holding means referred to in the claims is configured by the multimedia information storage unit 13, the scenario holding unit is configured by the scenario holding unit 11, and the voice recognition unit and the time domain detection unit are the voice recognition unit. 15, the multimedia related information holding unit is configured by the multimedia related information holding unit 16, the display unit and the instruction unit are configured by the display / instruction unit 17, and the multimedia attribute determination unit is configured by the multimedia attribute determination unit. The scenario editing unit is configured by the determination unit 14 and the scenario editing unit 18 is configured.
[0018]
Next, processing operations for creating multimedia related information by the multimedia editing apparatus will be described with reference to FIG.
When the processing is started (step S101), the scenario analysis unit 12 extracts and analyzes the scenario file 11a from the scenario holding unit 11, and extracts information such as the position, playback timing, and display effect at the time of playback of a plurality of media information. (Step S102). Next, the scenario analysis unit 12 outputs predetermined audio media information to the audio recognition unit 15 based on the extracted information, and outputs media information to be edited and evaluated among other media information to the multimedia attribute determination unit 14. (Step S103).
[0019]
Thereafter, the voice recognition unit 15 determines whether or not the voice media information has been input to the end (step S104). If the voice media information has not been input to the end, the start point of the syllable of the part of the currently input media information is determined. Time information: t1 is acquired (step S105), and it is evaluated whether the part is a sound part or a silence part (step S106).
If the currently input part is a sound part, it is determined that the next syllable has not yet been reached, and the above processing steps S103 to S106 are repeated until the input part becomes a silent part.
[0020]
On the other hand, if the currently input part is a silent part, it is determined that a syllable has been reached, and the speech recognition unit 15 acquires time information t2 of the end point of the part (step S107). Recognize a character string with respect to the voice data of the separated sound part, create text data of the character string, and output the text data and time information t1 and t2 (time domain) to the multimedia related information holding unit 16 (Step S108). Next, the multimedia attribute determination unit 14 determines the attribute of the media information to be edited and evaluated that is played between times t1 and t2, and outputs the attribute information of the determination result to the multimedia related information holding unit 16. (Step S109). For example, when it is determined that all of the media information subject to editing evaluation is media information that does not change over time (for example, still images), attribute information “0” is output, and at least one piece of media information subject to editing evaluation is If it is determined that the media information changes with time (for example, a moving image), attribute information “1” is output.
[0021]
Next, the multimedia related information holding unit 16 associates the text data input from the speech recognition unit 15, the time information t 1 and t 2, and the attribute information of the media information input from the multimedia attribute determination unit 14. It is held as multimedia related information (step S110). Then, the above-described processing (steps S103 to S110) is repeated, and when it is determined in step S104 that the audio media information has been input to the end, the processing ends (step S111).
[0022]
FIG. 4 is a diagram for specifically explaining the processing operation for creating the multimedia related information. The figure shows video data a and video data b that are two attribute evaluation targets (editing evaluation targets), video data c that contains a video of a presenter that is not an attribute evaluation target (editing evaluation target), and editing standards. The example which processed about the presentation comprised from the audio | voice data of the presenter who performs is shown.
[0023]
According to the above operation, the presentation time area corresponding to the sound part P of the audio data is detected as in the areas 1, 2, 3,..., And the audio text data in this area is created. In addition, among these time areas, the video data a and video data b to be edited and evaluated are determined to have the attribute value “0” indicating that the still image areas 2 and 4 are areas that can be deleted. The other areas 1, 3, and 5 are determined to be attribute values “1” representing areas that cannot be deleted. Then, as shown in FIG. 5, these results correspond to the text data, the start time and end time indicating the time area corresponding to the text data, and the attribute information of the media information to be edited in that time area. And stored in the multimedia related information storage unit 16.
[0024]
Next, a processing operation for editing a scenario file for multimedia presentation by the multimedia editing apparatus will be described with reference to FIG.
When the processing is started (step S201), the display / instruction unit 17 extracts the text data from the multimedia related information holding unit 16 and displays it on the display 17a (steps S202 and S203), the pen pointer 17b, and the mouse 17c. An instruction event for ending the editing process or a text data selection instruction event (step S205) serving as a reference for the editing range is received from the user (step S204).
If an instruction event not to be edited is selected, the process ends (step S206). On the other hand, when the text data selection instruction event is selected,
The scenario editing unit 18 refers to the multimedia related information held in the multimedia related information holding unit 16 and checks the attribute information associated with the selected text data (step S207).
[0025]
When the attribute information represents media information that does not change with time (for example, a still image) (in this embodiment, “0”), the media information is all media information that does not change with time, and the media information is reproduced. Even if not, the naturalness of the presentation is maintained, and the scenario editing unit 18 edits the scenario file so as not to reproduce all the media information in the corresponding time domain (step S208). On the other hand, if the attribute information represents time-varying media information (for example, a moving image) (“1” in the present embodiment), the media information is not played back and the connection between the front and rear becomes poor and the presentation becomes unnatural. Therefore, the scenario editing unit 18 edits the scenario file so as not to reproduce only the audio data corresponding to the text data (step S209). Then, the above processing steps S202 to S209 are executed until an end event is selected by the user.
[0026]
In this way, the user can easily and quickly specify the part to be edited without playing the multimedia presentation, and the scenario so that the presentation is not hindered without considering the media information at the part to be edited. You can edit the file.
[0027]
The present invention is not limited to the above embodiment, and various modifications can be made.
For example, in the above embodiment, before the text data as the reference of the editing range is designated by the user, the attribute information of the media information to be edited and evaluated in the time domain corresponding to the text data is determined in advance. The scenario file can be edited in a short time when the text data is selected by the user. However, the present invention is not limited to this, and after the text data is designated by the user, the media information in the corresponding time domain can be edited. The scenario information may be edited by determining the attribute information.
[0028]
Further, in the above embodiment, attribute information including a plurality of pieces of media information to be edited and evaluated is detected, and a scenario file is edited for a plurality of media based on the attribute information. However, the present invention is not limited thereto, and attribute information may be detected for each piece of media information to be edited and evaluated, and the scenario file may be edited for each piece of media information based on each piece of attribute information.
[0029]
【The invention's effect】
As described above, according to the present invention, it is possible to easily and appropriately edit a scenario file for a presentation in a short time.
[0030]
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a presentation using multimedia.
FIG. 2 is a diagram showing a configuration of a multimedia editing apparatus according to an embodiment of the present invention.
FIG. 3 is a flowchart of a processing operation for creating multimedia related information of the multimedia editing apparatus according to the embodiment of the present invention;
FIG. 4 is a diagram illustrating a process of creating multimedia related information in the multimedia editing apparatus according to the embodiment of the present invention.
FIG. 5 is a diagram illustrating an example of multimedia related information of the multimedia editing apparatus according to the embodiment of the present invention.
FIG. 6 is a flowchart of a processing operation for editing a multimedia presentation scenario file of the multimedia editing apparatus according to the embodiment of the present invention;
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 11 Scenario holding part 12 Scenario analysis part 13 Multimedia information storage part 14 Multimedia attribute determination part 15 Speech recognition part 16 Multimedia related information holding part 17 Display / input part 18 Scenario editing part

Claims (7)

Information holding means for holding a plurality of video and audio media information;
A scenario holding means for holding a scenario file describing media information to be presented in a presentation, a presentation position of the media information, and a presentation time;
Voice recognition means for recognizing a character string from predetermined audio media information described in the scenario file and creating text data of the character string;
Time domain detection means for detecting a time domain in which the character string is emitted;
Multimedia-related information holding means for holding the text data and the time region in association with each other;
Display means for displaying the text data;
Instruction means for instructing the text data for specifying the editing range;
Multimedia attribute determination means for determining attribute information related to time change for media information to be subjected to predetermined edit evaluation in the time domain;
Scenario editing means for editing a scenario file for the time area corresponding to the instructed text data based on the attribute information about the media information to be edited and evaluated that is reproduced in the time area. Features multimedia editing device. The multimedia editing apparatus according to claim 1, wherein
The multimedia-related information holding unit holds the attribute information about the media information to be edited and evaluated that is reproduced in the time domain in association with the time domain. apparatus. The multimedia editing apparatus according to claim 1 or 2,
The multimedia attribute determining means determines attribute information as to whether the image is a moving image or a still image by detecting a change in the image every predetermined time for the media information of the image. apparatus. The multimedia editing apparatus according to claim 1 or 2,
The scenario file includes the attribute information related to the media information,
The multimedia editing apparatus, wherein the multimedia attribute determining means determines attribute information of the media information based on the scenario file. The multimedia editing device according to any one of claims 1 to 4,
The voice recognition means converts the continuous voiced portion in the voice media information into text data as a unit,
The multimedia-related information holding means associates the text data with the time domain in units of text data. The multimedia editing device according to any one of claims 1 to 5,
The scenario editing means edits a scenario file so as not to reproduce the time domain portion of all media information if all of the media information to be edited and evaluated has attribute information that does not change over time. A multimedia editing device characterized by this. The multimedia editing device according to any one of claims 1 to 5,
The scenario editing means edits the scenario file so that only the audio part corresponding to the text data is not reproduced if at least one of the media information to be edited is subject to attribute information that changes with time. A multimedia editing apparatus characterized by that.

Images