JP2009200959A - Data editing apparatus, data editing method, program and storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To perform high-level editing linking sounds and images. <P>SOLUTION: A data editing apparatus is applied for editing input sound data and image data. As a configuration, the data editing apparatus includes: a sound editing unit 40 for editing the input sound data; an image editing unit 40 for editing input image data; and a control unit 51 for controlling an editing state of image data in the image editing unit in accordance with results of editing the sound data in the sound editing unit. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a data editing apparatus and data editing method for editing image data and audio data, a program for executing the editing method, and a storage medium storing the program.

  2. Description of the Related Art Conventionally, various image and sound editing apparatuses that simultaneously edit sound data and image data (moving image data) have been developed. Various editing programs have been developed that are implemented in a computer device and edit audio and images.

  In conventional editing apparatuses and editing programs for editing images and sounds, it is possible to handle images and sounds in parallel, but as a general process for handling images and sounds in parallel, for example, start time and It was processing to the extent that images and sounds were handled in relation to each other, such as making end times uniform.

In Patent Document 1, when editing audio data and image data, the audio data is analyzed to detect a change point of the audio data, and the position of the detected change point is an image or audio edit point. There is a description about handling as an index point.
JP 2005-4820 A

  As a conventional process in which audio and moving images are linked, as described in Patent Document 1, processing that is handled as an editing point such as an index is common, and an advanced process that reflects the contents of audio in an image is common. Editing has never been done before.

  The present invention has been made in view of such a point, and an object thereof is to enable advanced editing in which sound and image are linked.

  The data editing apparatus of the present invention is applied to a data editing apparatus that edits input audio data and image data. The configuration includes an audio editing unit that edits the input audio data, an image editing unit that edits the input image data, and an image in the image editing unit according to the audio data editing result in the audio editing unit. And a control unit that controls the data editing state.

  Further, the data editing method of the present invention, the program to which the method is applied, and the medium storing the program include: an audio editing process for editing input audio data; an image editing process for editing input image data; Is to do. Then, a control process for controlling the editing state of the image data in the image editing process is executed according to the editing result of the audio data in the sound editing process.

  According to the present invention, it is possible to directly reflect the editing state of the audio data on the image data itself, thereby giving various effects to the image.

  According to the present invention, it is possible to directly reflect the editing state of the audio data on the image data itself, thereby giving various effects to the image. Accordingly, it is possible to perform advanced image editing in which voice editing and image editing that are impossible with a conventional editing apparatus are linked. For example, it is possible to realize advanced editing in which the content of an image is changed in accordance with execution of sound editing that changes the sound volume or tempo.

Hereinafter, an example of an embodiment of the present invention will be described with reference to the accompanying drawings.
FIG. 1 is a diagram illustrating a configuration example of an editing apparatus according to an example of the present embodiment.
The editing apparatus in FIG. 1 includes an image information input unit 11, an audio information input unit 12, and an additional information input unit 13. The image information input unit 11 includes a terminal for inputting moving image data, and performs input processing for image data handled by the editing apparatus. The audio information input unit 12 includes a terminal for inputting audio data, and performs input processing for audio data handled by the editing apparatus. The additional information input unit 13 receives data accompanying image data and audio data. Note that the data accompanying the image data or the sound data may be added to the image data or the sound data obtained by the image information input unit 11 or the sound information input unit 12 and input. Also, audio data and image data may be input as integrated data. As the data format of the image data, those encoded by various image data encoding methods can be applied. For example, each display element in the image can be converted into an object and individually encoded.

  The data input to the image information input unit 11, the audio information input unit 12, and the additional information input unit 13 are supplied to the encoding unit 20. The encoding unit 20 generates a header division method based on the respective data, and supplies the generated header information to the recording / reproducing unit 30 together with the audio data and the image data.

  The recording / reproducing unit 30 includes a recording medium 31 as a relatively large-capacity data storage unit such as a hard disk drive, and records image data, audio data, header information, and the like supplied from the encoding unit 20 on the recording medium 31. . Also, each data recorded on the recording medium 31 is reproduced.

  The reproduced data is supplied to the editing unit 40. The editing unit 40 performs an editing process on the supplied image data and audio data, causes the edited image data to be output from the image information output unit 61, and the edited audio data from the audio information output unit 62. Output. Further, the edited image data and audio data may be sent to the recording / reproducing unit 30 and recorded on the recording medium 31 again.

The image data edited by the editing unit 40 is supplied to the image information output unit 61 and output to the outside. The audio data edited by the editing unit 40 is supplied to the audio information output unit 62 and output to the outside. The image data and audio data reproduced from the recording medium 31 by the recording / reproducing unit 30 may be output from these output units 61 and 62.
When image data and audio data are output from the image information output unit 61 and the audio information output unit 62, if header information is added to these data, they are output as they are. Alternatively, when there is additional data that is not integrated with image data or audio data such as header information, the additional data may be output.

  Processing operations in these editing apparatuses are executed under the control of the control unit 51. In addition, an operation unit 52 that can be operated by a user who operates the editing apparatus is provided, and the control unit 51 determines an operation command of the operation unit 52 to perform an editing operation, recording / reproduction, and the like.

FIG. 2 is a diagram illustrating a configuration example of the encoding unit 20 of the editing apparatus of FIG. The image data input from the image information input unit 11 and the audio data input from the audio information input unit 12 are supplied to the synchronization information generation unit 21. Further, the additional information input to the additional information input unit 13 is supplied to the additional information generation unit 22, and the additional information generation unit 22 generates information to be added to the image data and audio data, and a part of the generated information Is supplied to the synchronization information generating unit 21.
Then, the synchronization information generation unit 21 supplies the supplied image data, audio data, and additional information to the header information generation unit 23 to generate header information of the image data and audio data. At this time, some data such as effect data supplied from the additional information generation unit 23 is also added to the header information. The header information is information on a portion added to the head portion of the packet data when image data or audio data is packetized in a predetermined unit. In the case of this example, the header information includes information related to the effect state of images and sounds.
The image data and audio data to which the header information is added are supplied to the recording / reproducing unit 30 for recording.

FIG. 3 is a diagram showing a configuration example of the editing unit 40 of the editing apparatus shown in FIG.
The editing unit 40 includes a header generation analysis unit 41 and a synchronization effect playback unit 42. The header generation / analysis unit 41 analyzes the header information of the data reproduced by the recording / reproducing unit 30, changes the analyzed header information as necessary, and returns the header information to the recording / reproducing unit 30 as new header information. To update. In this example, image and sound editing is executed by updating the header information. The header information is changed according to an instruction from the input unit 43. The input unit 43 is a part to which a command from the control unit 51 shown in FIG. 1 is input.

  The header information analyzed by the header generation analysis unit 41 is supplied to the synchronous effect reproduction unit 42, and the image data and the audio data reproduced by the recording / reproduction unit 30 are changed by the effect information in the header information, and the change is made. Image data and audio data are output from the image information output unit 61 and the audio information output unit 62.

FIG. 4 is a diagram showing an example of data packetized by the processing of this example. Image data and audio data are integrated into packets in predetermined units, and header information is added to the head of the data to which the image data and audio data are added.
As shown in the lower part of FIG. 4, the header information includes synchronization information (such as a time code) necessary for image and audio synchronization processing, image and audio pointers, effect information for image elements, and audio. Contains effect information for the element.

Here, the effect state of the image and sound according to the present embodiment will be described with reference to FIG.
In the present embodiment, the effects on the image elements include the position, size, depth, transparency, color, direction of movement, speed of movement, etc. of each display element in the image. Examples of the effects on the sound element include a sound image localization state, sound volume, pitch, tone color, and tempo.
In the present embodiment, each of these elements can be changed by the header information, and the header information is changed to a state corresponding to the editing by the editing unit 40.

Here, as shown in FIG. 5, the change between the image element and the sound element can have an interaction.
An example of the interaction will be described. For example, when the sound image localization state is changed, the position of the display element in the image is changed.
Further, when the volume is changed, the size or depth of the display element is changed.
Also, when the pitch or tone color is changed, the transparency of the display element is changed.
Further, when the tone color is changed, the display color of the display element is changed.
Further, when the tempo is changed, the direction or speed of the movement of the display element is changed.

Alternatively, a change opposite to the change from the sound element to the image element may be performed. That is, when the position of the display element in the image is changed, the sound image localization state is changed.
Further, when the size or depth of the display element is changed, the volume is changed.
Further, when the transparency of the display element is changed, the pitch or timbre is changed.
When the display color of the display element is changed, the timbre is changed.
Further, when the direction or speed of the movement of the display element is changed, the tempo is changed.
These interactions are examples, and are not limited to these examples.

Next, an actual processing example will be described with reference to the flowcharts in FIG.
First, header information, image data, and audio data are read from the recording / reproducing unit 30 to the editing unit 40 (step S11). Then, it is determined whether or not there has been an editing process for changing the volume of the audio data in the editing unit 40 (step S12). The editing for changing the volume is performed in response to a command from the control unit 51, for example, when the user who performs the editing operation operates the operation unit 52 of FIG.
When the editing for changing the volume is performed in the determination in step S12, the recording / playback unit 30 rewrites the volume data in the header information of the audio data instructed to be edited to the corresponding value data. Things are written (step S13).

Next, it is determined whether there is image data linked to the audio data (step S14). If there is image data, the enlargement ratio of some display elements of the image data can be changed. It is determined whether or not there is (step S15). If it can be changed, the enlargement rate of the corresponding display element is set to an enlargement rate (or reduction rate) corresponding to the increase rate (or reduction rate of the volume) of the volume. Then, the header information of the image is updated so that it is displayed at the set enlargement ratio (or reduction ratio), and the updated header information of the image information is rewritten to the recording / reproducing unit 30 (step S16).
Note that if there is no image data linked to the audio data in step S14 and if the enlargement ratio of a part of the image cannot be changed in step S15, the process ends without performing the processing of this example.

FIG. 7 is a diagram showing an example of changing the image when the volume is adjusted. FIG. 7A shows an image before change, in which people are arranged in front of the background.
In this state, when the volume increasing process described in the flowchart of FIG. 6 is performed, as shown in FIG. 7B, for example, only the displayed person is enlarged and the background size is changed. Do not use images. In this case, the enlargement rate of the person corresponds to the increase rate of the volume, for example. In the example of FIG. 7, all three displayed persons are enlarged at the same ratio, but a display element that matches a certain condition, such as a specific person, for example, a person displayed in the center or a person with movement, is displayed. You may make it enlarge only. A display element to be enlarged or reduced may be selected by a user operation.

In the example of FIG. 7, the person in the image is used as the display element to be enlarged. However, for example, the background may be handled as the display element so as to be enlarged or reduced.
For example, as shown in FIG. 8A, when there is an image displaying a person in a certain background, when editing is performed to adjust the volume of the audio data, As shown in FIG. 8B, only the background image may be reduced (or enlarged), and the display size of the person may not be changed.
Alternatively, all the display elements such as a person and the background image may be handled as being the same, and may be enlarged or reduced in a batch.

The flowchart of FIG. 9 is a diagram illustrating another processing example.
First, header information, image data, and audio data are read from the recording / reproducing unit 30 to the editing unit 40 (step S11). Then, it is determined whether or not there is an editing process for changing the pitch of the audio data in the editing unit 40 (step S21). The editing for changing the pitch is also performed by a command from the control unit 51 when the user who performs the editing operation operates the operation unit 52 of FIG.
When the editing for changing the pitch is performed in the determination of step S21, the recording / playback unit 30 rewrites the data related to the pitch in the header information of the audio data instructed to be edited with the corresponding value data. Is written (step S13).

Next, it is determined whether there is image data linked to the audio data (step S14). If there is image data, the transparency of some display elements of the image data can be changed. Whether or not (step S22). If it can be changed, the transparency of the corresponding display element is set to a transparency corresponding to the change in pitch. Then, the header information of the image is updated so as to display with the set transparency, and the updated header information of the image information is rewritten to the recording / reproducing unit 30 (step S16).
Note that if there is no image data linked to the audio data in step S14, and if the transparency of a part of the image cannot be changed in step S22, the process ends without performing the processing of this example.

The flowchart of FIG. 10 is a diagram showing still another processing example.
First, header information, image data, and audio data are read from the recording / reproducing unit 30 to the editing unit 40 (step S11). Then, it is determined whether the editing unit 40 has performed an editing process for changing the tempo of the audio data (step S31). The editing for changing the tempo is also performed by a command from the control unit 51 when the user who performs the editing operation operates the operation unit 52 of FIG.
When editing is performed to change the tempo in step S21, the tempo data in the header information of the audio data instructed to edit is rewritten to the corresponding value data for the recording / playback unit 30. Is written (step S13).

Next, it is determined whether there is image data linked to the audio data (step S14). If there is image data, the movement amount of some display elements of the image data can be changed. It is determined whether or not there is (step S32). If it can be changed, the movement amount of the corresponding display element is set to transparency corresponding to the change in tempo. Then, the header information of the image is updated so that it is displayed so as to move with the set movement amount, and the updated header information of the image information is rewritten to the recording / reproducing unit 30 (step S16). ).
Note that if there is no image data linked to the audio data in step S14, and if the movement amount of a part of the image cannot be changed in step S22, the process ends without performing the processing of this example.

  As described above, by editing the sound, the image is correspondingly changed, and advanced editing in which the sound and the image are linked is possible. Each of the processes shown in the flowcharts of FIGS. 6, 9, and 10 shows an example, and other audio elements and image elements may be linked. When changing which element, how much to change which image element may be set in advance in the editing apparatus, but may be freely selected by a user operation. The image data to be edited may be any image data. For example, in the case of image data encoded as an object for each display element, the individual display elements shown in FIGS. This is preferable because it can be easily changed.

In the processing so far, the processing for reflecting the sound editing on the image has been described. However, the processing for reflecting the image editing on the sound may be performed. That is, for example, as shown in the flowchart of FIG. 11, the header information, the image data, and the audio data are read from the recording / playback unit 30 to the editing unit 40 (step S41). Then, it is determined whether or not there has been an editing process for changing the image content in the editing unit 40 (step S42). Examples of the change in the image content include each effect processing shown in FIG. The editing for changing the image content is also performed by a command from the control unit 51 by, for example, a user who performs an editing operation operating the operation unit 52 of FIG.
If editing for changing the image content is performed in the determination in step S42, the image data in the header information of the audio data instructed to be edited is converted into data corresponding to the recording / playback unit 30. The rewritten data is written (step S43).

Next, it is determined whether there is voice data linked to the voice data (step S44). If there is voice data, it is determined whether the volume of the voice data can be changed (step S44). Step S45). If it can be changed, the volume of the corresponding audio data is set to a corresponding volume. Then, the audio header information is updated so that the audio is output at the set volume, and the updated header information of the audio information is rewritten to the recording / reproducing unit 30 (step S46).
Note that if there is no audio data linked to the image data in step S44 and if the volume cannot be changed in step S45, the process ends without performing the processing of this example.

As shown in FIG. 11, more advanced editing is possible by reflecting the editing of the image in the sound. Even when editing the image is reflected in the sound, if the sound data is individually objectized and encoded, for example, for each voice of the person who is talking, the object Each unit of audio data can be processed, and more advanced audio editing is possible.
The editing apparatus may be capable of both the process of reflecting the image editing on the sound and the process of reflecting the sound editing described in FIG. 6 and the like on the image. By doing so, the editing of the sound and the image is correspondingly changed with respect to the image and the sound, and very advanced editing is possible.

  In the above-described embodiment, an example in which a dedicated editing apparatus that edits sound and images (video) is described has been described. However, for example, the voice input unit of the present example is added to a personal computer apparatus that performs various data processing. A data storage unit (recording unit) provided in the computer device is used as a recording / playback unit after a board or card that performs processing corresponding to the video input unit or the like is mounted, and editing processing in the editing unit is performed in the computer device. It may be executed by the arithmetic processing means.

In this case, the execution of the editing process by the arithmetic processing means may be performed by executing software (program) installed in the computer device. The software in that case is a program that executes processing such as the flowchart of FIG.
Further, the program for executing such editing processing may be stored in various media such as a disk and a semiconductor memory and distributed, or distributed via various transmission media such as the Internet.

It is a block diagram which shows the example of whole structure of the editing apparatus by one embodiment of this invention. It is a block diagram which shows the structural example of the encoding part of the editing apparatus of one embodiment of this invention. It is a block diagram which shows the structural example of the editing part of the editing apparatus of one embodiment of this invention. It is explanatory drawing which shows the example of a data structure by one embodiment of this invention. It is explanatory drawing which shows the example of interaction of the audio | voice and image by the edit by one embodiment of this invention. It is a flowchart which shows the example of an edit process by one embodiment of this invention. It is explanatory drawing which shows the example of a display (example 1) by the edit of the image by one embodiment of this invention. It is explanatory drawing which shows the example of a display (example 2) by the edit of the image by one embodiment of this invention. It is a flowchart which shows the example of an edit process by other embodiment of this invention. It is a flowchart which shows the example of an edit process by other embodiment of this invention. It is a flowchart which shows the example of an edit process by other embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 11 ... Image information input part, 12 ... Audio | voice information input part, 13 ... Additional information input part, 20 ... Encoding part, 21 ... Synchronization information generation part, 22 ... Additional information generation part, 23 ... Header information generation part, 30 ... Recording Reproduction unit 31 ... recording medium 40 ... editing unit 41 ... header generation analysis unit 42 ... synchronization effect reproduction unit 51 ... control unit 52 ... operation unit 61 ... image information output unit 62 ... audio information output unit

Claims (9)

In a data editing device for editing input audio data and image data,
A voice editing unit for editing the input voice data;
An image editing unit for editing the input image data;
A data editing apparatus comprising: a control unit that controls an editing state of the image data in the image editing unit in accordance with a result of editing the audio data in the audio editing unit. The data editing apparatus according to claim 1, wherein
The audio data and / or the image data is data objectized for each audio or image element,
A data editing apparatus, wherein the editing state is individually controlled for each object data. The data editing apparatus according to claim 2, wherein
The control of the editing state is control for changing the display magnification of at least one object of the image data corresponding to the volume change by editing when the volume of the audio data is changed by editing. Data editing device. The data editing apparatus according to claim 2, wherein
The control of the editing state is control for changing the display transparency of at least one object of the image data corresponding to the volume change by editing when the volume of the audio data is changed by editing. Data editing device. The data editing apparatus according to claim 2, wherein
The control of the editing state is a control for changing the movement amount of at least one object of the image data corresponding to the volume change by editing when the volume of the audio data is changed by editing. Data editing device. The data editing apparatus according to claim 1, wherein
Further, the control unit controls the editing state of the audio data in the audio editing unit according to the editing result of the image data in the image editing unit. In a data editing method for editing input audio data and image data,
A voice editing process for editing the input voice data;
An image editing process for editing the input image data;
And a control process for controlling an editing state of the image data in the image editing process according to a result of editing the audio data in the sound editing process. In a program that executes processing for editing input audio data and image data by being implemented in a data processing device,
A voice editing process for editing the input voice data;
An image editing process for editing the input image data;
And a control process for controlling an editing state of the image data in the image editing process according to a result of editing the audio data in the sound editing process. A storage medium storing the program according to claim 8.

Images