JP4654947B2 - Movie processing apparatus and program thereof - Google Patents

Description

  The present invention relates to a moving image processing apparatus and a program thereof, and more particularly, to a moving image processing apparatus and a program thereof for determining an editing point of moving image data.

In recent years, digital cameras that can shoot moving images of subjects and record voices of subjects have become widespread.
In addition, when it is desired to edit (cut) the moving image data with audio, it is performed by selecting a frame of an image with a good separation and deleting moving image data before or after the selected frame.
In addition, the sound level in the moving image is displayed in a waveform along the time axis, and a fixed mark indicating the current playback position is displayed at a position corresponding to the moving image playback time on the time axis, so that the playback speed of the moving image is increased. In addition, a technique has been developed in which an edit time point can be easily found by moving and displaying a voice waveform as compared with the conventional technique (Patent Document 1).

Japanese Patent Laid-Open No. 2005-260391

However, according to the conventional technique, since editing is performed on the basis of an image, there is a problem in that sound is struck by editing (a state where sound is unnaturally interrupted).
Further, according to the technique of the above-mentioned Patent Document 1, it is possible to prevent the sound from being crowded, but it is necessary to find the editing time point while viewing the displayed sound waveform. For general users, There is a problem in that it does not know which sound (word) the sound waveform corresponds to, and it takes more time to edit, and in some cases it cannot be edited.

  Therefore, the present invention has been made in view of such conventional problems, and an object of the present invention is to provide a moving image processing apparatus and a program thereof that can easily determine an edit point that prioritizes sound.

To achieve the above object, a moving image processing apparatus according to the first aspect of the present invention comprises an acquisition means for acquiring moving image data with audio;
Display control means for displaying on the display means a moving image frame of the moving image data with audio acquired by the acquiring means;
Playback means for repeatedly playing audio in a predetermined range corresponding to the moving image frame displayed by the display control means;
Range setting means for arbitrarily setting the predetermined range of the audio to be played repeatedly;
Determining means for the user to determine the moving image frame displayed on the display means as an editing frame;
When the editing frame is determined by the determining means, processing means for processing the moving image frame before or after the determined editing frame;
A video processing apparatus comprising:
The range setting means includes
Including means for setting whether the voice is processed or not processed by the processing means, and includes setting the predetermined range based on the set side. It is characterized by.

To achieve the above object, a program according to the second aspect of the present invention is stored in a computer.
Acquisition procedure to acquire video data with audio,
A display procedure for displaying a moving image frame of the moving image data with audio acquired by the acquiring procedure on a display unit;
Reproducing procedure to repeat playback of the audio in a predetermined range corresponding to the video frame displayed by the display procedure,
A range setting procedure for arbitrarily setting the predetermined range of audio to be played repeatedly;
A determination procedure for a user to determine a moving image frame displayed on the display means as an editing frame;
When an edit frame is determined by the determination procedure, a processing procedure for applying a processing process to the determined video frame before or after the determined edit frame,
A program for executing
The range setting procedure includes:
Including a procedure for setting whether the voice is processed or not processed according to the processing procedure, and the predetermined range is set based on the set side. .

  According to the present invention, the acquisition means for acquiring the moving image data with sound, the display control means for displaying the moving image frame of the moving image data with sound acquired by the acquisition means on the display means, and the display control means Since the reproduction means for repeatedly reproducing the audio in a predetermined range corresponding to the moving image frame is provided, it is possible to easily determine an edit point with priority given to the audio.

Hereinafter, the present embodiment will be described in detail with reference to the drawings as an example in which the moving image processing apparatus of the present invention is applied to a digital camera.
[Embodiment]
A. Configuration of Digital Camera FIG. 1 is a block diagram showing a schematic electrical configuration of a digital camera 1 that implements the moving image processing apparatus of the present invention.
The digital camera 1 includes a photographing lens 2, a lens driving block 3, a CCD 4, a driver 5, and a TG (timing
generator) 6, unit circuit 7, image generation unit 8, CPU 9, key input unit 10, memory 11, DRAM 12, flash memory 13, image display unit 14, audio processing unit 15, and bus 16.

  The taking lens 2 includes a focus lens 2a, a zoom lens 2b, and the like that are composed of a plurality of lens groups. A lens driving block 3 is connected to the photographing lens 2. The lens driving block 3 includes a focus motor and a zoom motor that drive the focus lens 2a and the zoom lens 2b in the optical axis direction, respectively, and a focus motor driver that drives the focus motor and the zoom motor in accordance with a control signal sent from the CPU 9. And a zoom motor driver (not shown).

  The CCD 4 is driven by the driver 5 and photoelectrically converts the intensity of light of each color of the RGB value of the subject image at a constant period and outputs it to the unit circuit 7 as an imaging signal. The operation timing of the driver 5 and unit circuit 7 is controlled by the CPU 9 via the TG 6.

  A TG 6 is connected to the unit circuit 7, a CDS (Correlated Double Sampling) circuit that holds the imaged signal output from the CCD 4 by correlated double sampling, and an AGC that performs automatic gain adjustment of the imaged signal after the sampling. An (Automatic Gain Control) circuit and an A / D converter that converts an analog image pickup signal after the automatic gain adjustment into a digital signal, the image pickup signal output from the CCD 4 passes through the unit circuit 7 as a digital signal. It is sent to the image generation unit 8.

  The image generation unit 8 performs processing such as γ correction processing and white balance processing on the image data sent from the unit circuit 7, generates a luminance color difference signal (YUV data), and generates the generated luminance color difference. The image data of the signal is sent to the CPU 9. That is, the image generation unit 8 performs image processing on the image data output from the CCD 4.

The CPU 9 has a function of performing compression / decompression processing (for example, compression / decompression of JPEG format, MPEG format), repeat reproduction processing, moving image data and audio deletion processing of the image data sent from the image generation unit 8. A one-chip microcomputer for controlling each part of the digital camera 1.
In particular, the CPU 9 has a function of displaying a frame selected by the user from moving image data composed of a plurality of frames, and repeatedly playing back sound in a range based on the displayed frames.

The key input unit 10 includes a plurality of operation keys such as a shutter button, a power on / off key, a mode switching key, a cross key, and a SET key, and outputs an operation signal corresponding to the user's key operation to the CPU 9.
The memory 11 stores a control program (for example, a program necessary for repeat reproduction) necessary for the CPU 9 to control each unit and necessary data, and the CPU 9 operates according to the program.

The DRAM 12 is used as a buffer memory for temporarily storing image data sent to the CPU 9 after being imaged by the CCD 4 and also as a working memory for the CPU 9.
The flash memory 13 is a recording medium that stores compressed image data.

  The image display unit 14 includes a color LCD and its drive circuit, and displays the subject imaged by the CCD 4 as a through image when in a shooting standby state, and is read from the flash memory 13 and decompressed when a recorded image is played back. The recorded image is displayed.

The sound processing unit 15 includes a built-in microphone, an amplifier, an A / D converter, a D / A converter, an amplifier, a built-in speaker, and the like, and converts sound input to the built-in microphone into a digital signal when shooting a movie with sound. To the CPU 9. The CPU 9 sequentially stores the transmitted audio data in the buffer memory (DRAM 12), and records the stored audio data in the flash memory 13 in association with the captured moving image data.
In addition, the sound processing unit 15 emits sound based on the sound data from the built-in speaker during reproduction of the moving image with sound, and the CPU 9 displays the image recorded in association with the sound data on the image display unit 14. Display. Similarly, in the moving image editing mode, sound for a certain period including sound corresponding to the displayed frame is repeatedly emitted from the built-in speaker (repeat reproduction).

B. Operation of Digital Camera 1 The operation of the digital camera 1 in the embodiment will be described with reference to the flowchart of FIG.
When the power is turned on by the operation of the power on / off key of the user's key input unit 10, the CPU 9 determines whether or not the moving image editing setting mode is set by the user's operation of the mode switching key (step S1). .

If it is determined in step S1 that the moving image editing setting mode has been set, the CPU 9 sets the time designated by the user as the repeat playback time (step S2). At this time, the CPU 9 displays a screen for allowing the user to specify the time on the image display unit 14, and the user can specify the time by operating the cross key and the SET key while viewing the screen. The CPU 9 sets the repeat playback time according to the user's operation.
The repeat reproduction time is not the time for performing repeat reproduction, but the time for audio to be subjected to repeat reproduction. For example, when the repeat playback time is set to 5 seconds, the sound of 5 seconds (5 seconds when played back at the same speed as recording) is repeatedly played from the sound of the selected frame described later.

Next, the CPU 9 sets the speed designated by the user as the repeat reproduction speed (step S3). At this time, the CPU 9 causes the image display unit 14 to display a screen for allowing the user to specify the speed, and the user can specify the speed by operating the cross key and the SET key while viewing the screen. The CPU 9 sets the repeat reproduction speed according to the user's operation.
The repeat playback speed can be set in stages such as slow, slightly slow, normal, slightly fast, and fast. When normal is set as the repeat playback speed, playback is performed at the same speed as the voice at the time of recording.

Next, the CPU 9 sets the direction (front or rear) designated by the user as the repeat reproduction direction (step S4), and returns to step S1. The range that can be specified by the user is two types, “front” and “rear”. For example, when “front” is specified, the audio before the position of the time axis of the audio of the selected frame to be described later When repeat playback is performed and “after” is selected, the audio after the position of the time axis of the audio of the selected frame is played back repeatedly.
At this time, the CPU 9 displays a screen for designating the repeat reproduction direction on the image display unit 14, and the user can designate the repeat reproduction direction by operating the cross key and the SET key. The CPU 9 determines whether or not the repeat reproduction direction is designated based on the user operation.
The range of audio to be played back repeatedly is determined by the repeat playback time and the repeat playback direction.

On the other hand, when determining in step S1 that the moving image editing setting mode is not set, the CPU 9 determines whether or not the moving image editing mode is set (step S5).
If it is determined in step S5 that the moving image editing mode has not been set, the process proceeds to another process. If it is determined that the moving image editing mode has been set, moving image data with audio (such as a file name or thumbnail image) recorded in the flash memory 13 is determined. May be displayed as a list (or sequentially displayed) on the image display unit 14 (step S6).
Next, the CPU 9 determines whether or not moving image data has been selected by the user (step S7). At this time, the user can select moving image data by operating the cross key and the SET key, and the CPU 9 determines whether moving image data has been selected based on the user's operation.

  If it is determined in step S7 that the moving image data is not selected, the process stays in step S7 until it is selected. If it is determined that the moving image data is selected, the CPU 9 selects the first frame (moving image frame) of the selected moving image data. It selects and displays on the image display part 14 (step S8). At this time, the CPU 9 also displays a moving image frame selection bar on the image display unit 14. This moving image frame selection bar is displayed until an edit frame to be described later is determined. The moving image frame selection bar is a bar for allowing the user to select an arbitrary frame.

Next, the CPU 9 determines whether or not a predetermined time (for example, 3 seconds) has elapsed since the frame was displayed (step S9).
If it is determined in step S9 that a predetermined time has not elapsed since the frame was displayed, the CPU 9 determines whether or not a new frame (moving image frame) has been selected by the user (step S10). At this time, the user can select a frame by operating the cross key while looking at the displayed moving image frame selection bar, and the CPU 9 determines whether or not the frame has been selected based on the user's operation. To do.

If it is determined in step S10 that no new frame has been selected, the process returns to step S9 to determine whether a predetermined time has elapsed since the currently displayed frame was displayed.
On the other hand, if it is determined in step S10 that a new frame has been selected, the CPU 9 newly displays the selected frame on the image display unit 14 (step S11), and then returns to step S9 to display the newly displayed frame. It is determined whether or not a predetermined time has elapsed since the displayed frame was displayed.

On the other hand, if it is determined in step S9 that a predetermined time has elapsed since the frame was displayed, the CPU 9 starts repeat playback of the sound based on the displayed frame (the selected frame) (step S9). S12).
The repeat playback based on the selected frame is determined by the position of the audio time axis corresponding to the selected frame and the repeat playback time and repeat playback direction set in the moving image editing setting mode. This means that the audio in the range is played back repeatedly at the set repeat playback speed.

For example, before and after the sound of the selected frame is "Hello, Mr. Yamada. Long time no see, how are you?" A voice saying, "do."'S voice is Yamada, corresponding to the selected frame in the In some cases, if the repeat playback time is set to “2 seconds”, the repeat playback speed is set to “slightly slow”, and the repeat playback range is set to “previous”, the time axis of the audio corresponding to the selected frame is set. The sound between the position and the position two seconds before the position is repeatedly reproduced at a slightly slower speed. In this case, "Hello, Mr. Yamada." So that the sound is repeat play a slightly slower speed that.
Conversely, if the repeat playback range is set to “back”, the sound between the position of the time axis of the sound corresponding to the selected frame and the position 2 seconds after the position is slightly Repeat playback at slow speed. In this case, the voice “No. Long time no see” is played repeatedly at a slightly slower speed.

  Thus, since the user can set the repeat playback time, it is possible to repeat playback of a sound of an appropriate time for the user. In addition, since the user can set the repeat playback speed, the voice can be played back at a speed that is easy for the user to hear. In addition, since the user can set the repeat playback direction, it is possible to repeat playback of the sound in the direction desired by the user (front or back).

When repeat reproduction is started in step S12, the CPU 9 determines whether or not the frame (displayed) selected by the user has been determined as an edit frame (step S13). This determination is made based on whether or not an operation signal corresponding to the operation of the SET key is sent from the key input unit 10.
This edit frame refers to a frame (edit point) serving as a reference for determining a part from which moving image data is to be deleted.

If it is determined in step S13 that the selected frame is not determined as an edited frame, the CPU 9 determines whether or not a new frame has been selected by the user (step S14). As described above, the user can select a frame by operating the cross key while looking at the displayed moving image frame selection bar, and the CPU 9 determines whether or not the frame has been selected based on the user's operation. Judging.
If it is determined in step S14 that a new frame has not been selected, the process returns to step S13 to determine whether an edit frame has been determined. If it is determined in step S14 that a new frame has been selected, the CPU 9 After the newly displayed frame is newly displayed on the image display unit 14 (step S15), the process returns to step S9 to determine whether or not a predetermined time has elapsed since the newly displayed frame was displayed.

During the loop from step S8 to step S13, the user can select a frame and listen to the sound repeat reproduction, thereby determining a frame having a good voice separation as an editing frame.
On the other hand, if it is determined in step S13 that the selected frame is determined as an editing frame, the process proceeds to step S16.

Here, an example of a method for selecting a user frame will be described.
As described above, until the editing frame is determined after the first frame of the selected moving image data is displayed (during the loop from step S8 to step S13), the CPU 9 displays the moving image frame selection bar on the image display unit. The user can select a frame by operating the cross key while viewing this display.

FIG. 3A is a diagram illustrating a state of the moving image frame selection bar displayed on the image display unit 14.
The bar in the figure represents the time axis of the video, the leftmost is the first frame of the video data, the rightmost is the last frame of the video data, and the first and last frames of the video data in the meantime It shows a frame between frames. In other words, the more you go to the right, the more back you get.

Further, the triangular portion in the figure indicates the position of the currently selected frame, and the user can select a new frame by moving the triangular portion when the user operates the cross key.
For example, when the user presses the “→” key on the cross key once, the next selected frame (the currently displayed frame) is selected as the selected frame, and the user selects the cross key. When “←” is pressed once, the frame selected immediately before the currently selected frame is selected.

  If the cross key is pressed and held, the selected frame is skipped (for example, every 10 frames). For example, if you press and hold the “→” key of the cross key, the frames 10 frames, 20 frames, and 30 frames from the currently selected frame will be selected in sequence, and if you press and hold the “←” key, you will select the current frame. The 10 frames before, 20 frames before and 30 frames before the selected frame are sequentially selected.

Returning to the flowchart of FIG. 2, if it is determined that the editing frame has been determined in step S13, the process proceeds to step S16, where the CPU 9 stops displaying the moving image frame selection bar and deletes the range selected by the user (before the editing frame). Then, it is determined whether or not designation has been made. There are two deletion ranges that can be specified by the user: before the edit frame and after the edit frame. Here, the sound corresponding to the piece that has been determined as the editing piece is "Hello, Mr. Yamada. Long time no see, how are you?" And "do.".
At this time, the CPU 9 displays a screen for designating the deletion range on the image display unit 14, and the user can designate the deletion range by operating the cross key and the SET key. It is determined whether or not a deletion range has been designated based on the above operation.

If it is determined in step S16 that the deletion range is not specified, the process stays in step S16 until it is specified. If it is determined that the deletion range is specified, the moving image data and audio data in the specified deletion range are deleted (step S17). .
For example, if the range designated as the deletion range is after the editing frame, the moving image data to be deleted includes moving image data after the editing frame (not including the editing frame) and the moving image data (moving image frame). the corresponding voice to), in other words, "Hello, this is Yamada. Long time no see, how are you?" one of, "Hello, Mr. Yamada." than after the voice data, that is, "Long time no see, Genki The audio data after "?" Is deleted. This makes it "Hello, Mr. Yamada." Moving image data remains to be corresponding to the previous audio data and voice data.

C. As described above, in the embodiment, in the video editing mode, when a user selects a video frame and a predetermined time elapses, the position of the time axis of the audio corresponding to the selected video frame is set by the user. Since the sound within the range determined by the repeat playback time and the repeat playback direction is repeatedly played back at the repeat playback speed set by the user, the edit point giving priority to the sound can be easily determined.
In addition, since the user can set the repeat playback time, it is possible to repeat playback of sound for an appropriate time for the user. Further, since the user can set the repeat playback speed, the voice can be played back repeatedly at a speed that is easy for the user to hear. Furthermore, since the user can set the repeat playback direction, it is possible to repeat playback of the sound in the direction (front or back) in which the user can easily determine the edit point.

[Modification]
D. The above embodiment may be the following modes.
(1) In the above embodiment, the repeat playback range and the repeat playback direction are set separately to determine the repeat playback range. However, the repeat playback range may be determined directly.
For example, as shown in FIG. 3B, a repeat playback range designation bar is displayed as an image, and the user operates the cross key while viewing this display to set the repeat playback range (repeat playback time and repeat playback direction). Can be specified.

The bar in FIG. 3 (b) represents the time axis (in seconds). The position of 0 second indicates the position of the audio corresponding to the selected moving image frame, and the + direction is selected. The sound after the sound corresponding to the moving image frame is shown, and the-direction indicates the sound before the sound corresponding to the selected moving image frame.
Further, the triangular portion in the figure indicates a repeat reproduction range to be designated, and when the user operates the cross key, the triangular portion is moved and the reproduction position or the end position can be selected.
When the user operates the cross key “→”, the triangle moves to the right, and when “←” is operated, the triangle moves to the left.
For example, when there is a triangular portion at the position shown in the figure, the repeat playback time is 2.5 seconds, and the repeat playback direction is “back”.
When the user operates the SET key, the repeat playback time and the repeat playback direction are set based on the position where the current triangular portion is located.

(2) In the embodiment and the modification (1), only “front” or “back” can be set as the repeat reproduction direction, but “front and back” can also be set. You may do it.
When “front and back” is set, the audio before and after the audio corresponding to the selected moving image frame is repeatedly reproduced.
In this case, the ratio of the audio time before and after the repeat reproduction may be set. For example, when the front / rear ratio is 50:50, the playback time of the audio before the audio corresponding to the selected moving image frame is the same as the playback time of the subsequent audio, and the front / back ratio is 100: 50. In the case of 0, only the sound before the sound corresponding to the selected moving image frame is reproduced.
Further, in the setting in the modified example (1), the front and rear reproduction times and ratios can be set by adjusting the positions of the two triangular portions by providing the triangular portions on the + side and the − side, respectively. it can.

(3) In the above embodiment, only one moving image frame can be determined as an editing frame. However, two moving image frames may be determined as an editing frame.
In this case, the frame between the determined editing frames and the corresponding audio data are deleted.

(4) In the embodiment and the modification (3), either “front” or “rear” is selected as the method for setting the repeat playback direction. One of “remaining side (side not deleted)” may be selected.
In this case, it is particularly effective in the modification (3), and if “front” is set out of “front” and “rear” as the setting method of the repeat reproduction direction, as shown in FIG. Of the two editing frames, when the time axis determines the previous editing frame, the remaining sound is played back repeatedly, and when the time axis determines the subsequent editing frame, the deleted sound is played back repeatedly. However, such inconvenience does not occur if the “delete side” and “remaining side” are selected.

  (5) In the above embodiment and modification (3) or (4), the moving image data and the sound are deleted based on the determined editing frame. However, the present invention is limited to the deletion. However, the present invention may be applied even when image processing or sound processing is performed. That is, anything that specifies an edit point may be used.

  (6) In the above embodiment, the repeat playback time, repeat playback speed, etc. are set in the video editing setting mode (steps S2 to S4). It may be possible to set the repeat playback time, the repeat playback speed, and the like during the recording (in the loop from step S8 to step S13).

  (7) The digital camera 1 in the above embodiment is not limited to the above embodiment, and may be a device other than the camera, such as a mobile phone, a PDA, a personal computer, an IC recorder, etc. Any device capable of processing can be used.

1 is a block diagram of a digital camera according to an embodiment of the present invention. It is a flowchart which shows operation | movement of the digital camera 1 of embodiment. It is a figure which shows a moving image frame selection bar, and a figure for demonstrating a modification.

Explanation of symbols

1 Digital Camera 2 Shooting Lens 3 Lens Drive Block 4 CCD
5 Driver 6 TG
7 Unit circuit 8 Image generator 9 CPU
10 Key input unit 11 Memory 12 DRAM
13 Flash memory 14 Image display unit 15 Audio processing unit 16 Bus

Claims (2)

An acquisition means for acquiring video data with audio;
Display control means for displaying on the display means a moving image frame of the moving image data with audio acquired by the acquiring means;
Reproduction means for repeatedly reproducing a predetermined range of audio corresponding to the moving image frame displayed by the display control means;
Range setting means for arbitrarily setting the predetermined range of the audio to be played repeatedly;
Determining means for the user to determine the moving image frame displayed on the display means as an editing frame;
When the editing frame is determined by the determining means, processing means for processing the moving image frame before or after the determined editing frame;
A video processing apparatus comprising:
The range setting means includes
Including a means for setting whether the sound is processed by the processing means or the sound not processed, and the predetermined range is set based on the set side. Movie processing device. On the computer,
Acquisition procedure to acquire video data with audio,
A display procedure for displaying a moving image frame of the moving image data with audio acquired by the acquiring procedure on a display unit;
Reproducing procedure to repeat playback of the audio in a predetermined range corresponding to the video frame displayed by the display procedure,
A range setting procedure for arbitrarily setting the predetermined range of audio to be played repeatedly;
A determination procedure for a user to determine a moving image frame displayed on the display means as an editing frame;
When an edit frame is determined by the determination procedure, a processing procedure for applying a processing process to the determined video frame before or after the determined edit frame,
A program for executing
The range setting procedure includes:
Including a procedure for setting whether the voice is processed or not processed according to the processing procedure, and the predetermined range is set based on the set side. program.

Images