JP2007081466A - Image processing apparatus and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology of generating audio attached still picture movie by attaching proper audio data extracted from a moving picture file recorded on a recording medium to a still picture movie. <P>SOLUTION: An image processing apparatus disclosed herein includes: a reproducing means that reproduces a still picture file and the moving picture file from the recording medium recorded with the still picture file including still picture data and the moving picture file including a moving picture stream comprising moving picture data and the audio data; and generating means (401 to 404) that generate new moving picture stream by using the still picture data and the audio data in the moving picture file. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and method.

  In recent years, digital video cameras using a large-capacity recording medium such as a DVD have become widespread. Such digital video cameras generally have a moving image shooting function and a still image shooting function. For example, a moving image is encoded by the MPEG method and a still image is encoded by the JPEG method and recorded on a DVD.

  However, many DVD playback devices have only a moving picture decoder, and such playback devices cannot decode still images, and users cannot use still images.

  As means for solving the above problem, there is known a method in which still image data is converted into an MPEG I picture and newly recorded as a moving image stream (hereinafter, still image movie) (see Patent Document 1).

JP 2004-201170 A

  However, Patent Document 1 discloses means for creating and recording moving image data (still image movie) of only video from still image data, and does not disclose means for adding audio data.

  An object of the present invention is to create a still image movie with audio by adding appropriate audio data extracted from a moving image file recorded on the recording medium to the still image movie.

The image processing apparatus according to the present invention includes the still image file and the moving image file from a recording medium on which a still image file including still image data and a moving image file including a moving image stream including moving image data and audio data are recorded. And a generating means for generating a new moving picture stream using the still image data and the audio data in the moving picture file.
The image processing method of the present invention is an image processing method using a recording medium in which a still image file including still image data and a moving image file including a moving image stream including moving image data and audio data are recorded. A generating step of generating a new moving image stream using the still image data and the audio data in the moving image file.

  Since appropriate audio data can be easily added to still image data, it can be viewed as a still image movie with audio. In addition, the still image movie with sound can be recorded on a recording medium, and can be reproduced by a reproducing apparatus that does not include the still image reproducing means.

  Hereinafter, exemplary embodiments of the present invention will be described in detail by way of example. However, the constituent elements described in this embodiment are merely examples, and are not intended to limit the scope of the present invention only to them.

  FIG. 1 is a block diagram illustrating a configuration example of a digital video camera (image processing apparatus) according to an embodiment of the present invention. In FIG. 1, reference numeral 101 denotes an image pickup unit, 102 denotes a microphone (microphone), and 103 denotes a recording system circuit, which compresses and encodes image data and audio data obtained from the image pickup unit 101 and the microphone 102. Reference numeral 104 denotes a disk I / F (interface), which formats data output from the recording system circuit 103 into a predetermined format and records / reproduces the file data on / from the optical disk 105. Reference numeral 106 denotes an editing system circuit for creating a still image movie with sound, which will be described later. Reference numeral 107 denotes a playback system circuit for reproducing a data file from the disc 105 via the disc I / F 104. Reference numerals 108 and 109 denote image data and audio data output terminals, respectively. Reference numeral 110 denotes an LCD (liquid crystal display) that displays an image related to image data output from the imaging unit 101 and the reproduction system circuit 107. Reference numeral 111 denotes a system control unit (hereinafter referred to as a system controller) for controlling the operation of the digital video camera, and 112 denotes an operation unit including a power switch, a recording trigger, a playback switch, a stop switch, or a playlist editing switch.

  Next, the operation in the normal recording mode will be described. When the moving image shooting mode is set by the operation unit 112 and recording start is instructed by the recording trigger, the syscon 111 controls the recording system circuit 103 to start recording. The recording system circuit 103 encodes image data and audio data obtained from the imaging unit 101 and the microphone 102 and outputs them to the disk I / F 104 in a predetermined format. The disk I / F 104 adds additional information such as a header according to a predetermined file format to format the data string output from the recording system circuit 103 and records it on the disk 105.

  When the still image shooting mode is set by the operation unit 112 and the still image shooting is instructed by the recording trigger, the syscon 111 controls the recording system circuit 103 to start recording. The recording system circuit 103 encodes the image data obtained from the imaging unit 101 and outputs the encoded image data to the disk I / F 104 in a predetermined format. The disk I / F 104 adds additional information such as a header according to a predetermined file format to format the data string output from the recording system circuit 103 and records it on the disk 105.

  FIG. 2 is a diagram showing a configuration example of a main part of the recording system circuit 103 and its peripheral circuits. In FIG. 2, the image data from the imaging unit 101 is output to the image signal processing unit 201. The image processing unit 201 performs processing such as sub-sample according to the encoding format on the image data from the imaging unit 101 and outputs the processed data to the image data encoding unit 202. The image data encoding unit 202 encodes the image data from the image signal processing unit 201 according to the MPEG2 or JPEG standard, and outputs the encoded image data to the data encoding processing unit 206. Further, the image data encoding unit 202 generates thumbnail image data indicating a representative image of one data file and outputs the thumbnail image data to the data encoding processing unit 206.

  Also, audio data from the microphone 102 is output to the audio signal processing unit 203. The audio signal processing unit 203 converts the audio data from the microphone 102 into digital data, further performs processing such as noise suppression and amplification, and outputs the processed data to the audio data encoding unit 204. The audio data encoding unit 204 encodes the audio data from the audio signal processing unit 203 in accordance with the MPEG2 standard, and outputs it to the data encoding processing unit 206.

  The data encoding processing unit 206 converts the encoded image data and audio data into an MPEG2 transport stream or JPEG file format, and outputs the converted data to the disk I / F 104.

  Further, the metadata generation unit 205 generates metadata indicating shooting information such as shooting date and time and outputs the metadata to the data encoding processing unit 206. The data encoding processing unit 206 arranges these metadata so as to be recorded in the header or footer of the moving image, audio data file, or still image file, and outputs them to the disk I / F 104.

  In the digital video camera of the present embodiment, moving image data and audio data captured and input between a recording start trigger and a recording stop trigger are recorded as one moving image file. Further, at the time of still image shooting, the image data of one screen shot by one still image shooting instruction is recorded as a still image file.

  The data encoding processing unit 206 stores the metadata from the metadata generation unit 205 and the thumbnail image data from the image data encoding unit 202 in the header or footer of each file. In addition, a file name and the like are recorded in the header.

  Next, the operation during normal playback will be described. When the playback mode is set by the operation unit 112, the system controller 111 controls the playback system circuit 107 to display a thumbnail image of the data file recorded on the disc 105 on the LCD 110. The user confirms the thumbnail displayed on the LCD 110 and operates the operation unit 112 to select a file desired to be reproduced. When a file to be reproduced is designated, the syscon 111 controls the reproduction system circuit 107 to display metadata information regarding the file being reproduced on the LCD 110.

  The reproduction system circuit 107 reproduces a file designated from the disk 105 via the disk I / F 104, decodes image data and audio data in the file, outputs them from the output terminals 108 and 109, and reproduces the reproduced image. Is displayed on the LCD 110.

  FIG. 3 is a diagram showing a configuration example of the reproduction system circuit 107 and its peripheral circuits. In FIG. 3, the data reproduced by the disc I / F 104 is output to the data decoding processing unit 302. The data decoding processing unit 302 detects image data and audio data from the reproduced data sequence, and further detects metadata from the file header and footer. Then, the image data is output to the image data decoding unit 303.

  The image data decoding unit 303 decodes the image data output from the data decoding processing unit 302 and outputs the decoded image data to the image output unit 305. The image output unit 305 converts the decoded image data into a format suitable for processing by an external monitor or other external device, and outputs it to the output terminal 108 and the LCD 110.

  The audio data decoding unit 304 decodes the audio data output from the data decoding processing unit 302 and outputs it to the audio output unit 306. The audio output unit 306 converts the decoded audio signal into a format suitable for processing by an external device, and outputs it to the output terminal 109.

  Next, the function of creating a still image movie with sound according to the present embodiment will be described. As described above, the still picture movie described here is obtained by converting a plurality of still picture data into MPEG (Moving Picture Experts Group) type I pictures and handling them as new moving picture data. An I picture is a frame-encoded picture, and is an image that is encoded using only the frame image.

  In the present embodiment, there is provided means for extracting audio data from a moving image file that is expected to be most relevant to a still image and adding it when creating the still image movie.

Hereinafter, a method for extracting audio data and a process for creating a still image movie with audio will be described.
FIG. 4 is a diagram showing a configuration example of the editing system circuit 106 and its peripheral circuits in FIG. If the user operates the operation unit 112 to create a still image movie with audio or determines that the syscon 111 should create a still image movie with audio at a certain timing (for example, at the time of finalization) Instructs the next process. That is, the editing circuit 106 creates a still image movie with sound.

  The syscon 111 controls the editing system circuit 106 to create a still image movie with sound. First, the data analysis unit 401 analyzes still image data recorded on the disc 105. At this time, the data analysis unit 401 reads still image shooting time information from the metadata added to the still image data and records it in the internal memory. This is performed for all still image data.

  Next, the data analysis unit 401 detects whether or not a moving image file shot at the same time as the still image shooting time or a time close thereto is recorded on the disc 105. Thereby, the association between the still image and the moving image is performed. Note that it is possible to set a time range by operating the operation unit 112 by the user to determine which range is taken at a nearby time.

  In the above means, when there is no moving image associated with the still image, the user selects whether or not to include the still image in the still image movie by the operation unit 112. When included in a still image movie, the sound may be silenced, or music or sound effects may be added by other means.

  When a moving image file associated in this way exists on the disc 105, the data analysis unit 401 reproduces the related moving image file from the disc 105 via the disc I / F 104 and outputs it to the audio extraction unit 402. The audio extraction unit 402 extracts audio data associated with each still image from the reproduced moving image file. The voice extraction location may be automatic or manual. In the case of automatic, the voice extraction unit 402 randomly extracts voice for a predetermined time.

  In addition, when there are a plurality of still image files associated with the same moving image file, it is possible to collectively extract the audio for the time for reproducing those still images as a still image movie. For example, if the playback time per still image is 3 seconds and there are 3 still images associated with the same movie, 3 seconds × 3 = 9 seconds of continuous audio data is a moving image file. Extract from

  FIG. 5 is a diagram showing a state of association. Hereinafter, FIG. 5 will be described as an example. Three moving images I to III and five (5 screens) still images (1) to (5) are recorded on the disc. The data analysis unit 401 automatically associates a moving image captured at the same time as or in the vicinity of the still image capturing time with the still image. In the case of FIG. 5, the still image (1) is associated with the moving image I, the still images (2) and (3) are associated with the moving image II, and the moving image (4) is associated with the still image III. After that, the audio extraction unit 402 extracts audio data from the associated moving image file. The still image (5) is not associated because there is no moving image taken at the same time as the shooting time or in the vicinity thereof. Therefore, whether the still image (5) is not included in the still image movie, is included in the still image movie as a still image without sound, or is added to the still image movie by adding new audio data (music, sound effects, etc.) Is determined by the user's selection setting.

  In the above-described means, the audio data extracted from the moving image file is output to the still image movie creating unit 404 with audio. On the other hand, the data analysis unit 401 outputs a still image to the format conversion unit 403. The format converter 403 converts a JPEG still image into an MPEG I picture format. The converted data is output to the still image movie creation unit 404 with sound.

  In the still image movie creation unit 404 with audio, the audio data output from the audio extraction unit 402 and the still image data after I picture conversion output from the format conversion unit 403 are synthesized, and a new moving image stream (still image movie with audio is added). ). The audio-added still image movie creation unit 404 encodes the audio data and the I picture in the MPEG format to generate a new moving image stream.

  A new moving image stream (a still image movie with audio) is output to the recording circuit 103. The recording circuit 103 finally records a moving image file including a new moving image stream on the disc 105.

  In the above-described embodiment, MPEG and JPEG are described as examples of the moving image format and the still image format, respectively. Needless to say, the format is not limited to these formats.

  In the above-described embodiment, an optical disk is used as a recording medium, but the present invention is not limited to this. The present invention can also be applied to recording media such as hard disks and semiconductor memories.

  As described above, according to the present embodiment, a recording unit that records image data on a recording medium, a reproducing unit for image data recorded on the recording medium, and a moving image recorded on the recording medium A configuration is provided that includes means for generating a still image movie with sound from a still image. The recording medium records a still image file including still image data and a moving image file including a moving image stream including moving image data and audio data. As described above, the moving image file and the still image file can be associated from the shooting information such as the recording (shooting) time from one recording medium in which the moving image file and the still image file are recorded. Then, audio data is extracted from the moving image file. Then, it is possible to create a new moving image stream (a still image movie with audio) by converting still image data into an MPEG I picture and adding the extracted audio data. The still picture movie can also be recorded on the same recording medium.

  According to the present embodiment, it is possible to view a still image movie with sound by easily adding the optimum sound to the still image movie. In addition, the still image movie with sound can be recorded on a recording medium, and can be reproduced on a reproduction device that does not include a still image reproduction unit. In addition, a still image movie with sound can be easily generated and recorded from a moving image and a still image recorded on one recording medium.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed in a limited manner. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

It is a figure which shows the structural example of the digital video camera by embodiment of this invention. It is a figure which shows the structural example of a recording system circuit. It is a figure which shows the structural example of the reproduction | regeneration system circuit. It is a figure which shows the structural example of an edit type | system | group circuit. It is a figure which shows the mode of the correlation of a moving image and a still image.

Explanation of symbols

101 Imaging unit 102 Microphone 103 Recording system circuit 104 Disc I / F
105 Disc 106 Editing system circuit 107 Playback system circuit 108 Video output terminal 109 Audio output terminal 110 LCD
111 system control unit 112 operation unit 401 data analysis unit 402 audio extraction unit 403 format conversion unit 404 still image movie creation unit with audio

Claims (9)

Reproducing means for reproducing the still image file and the moving image file from a recording medium on which a still image file including still image data and a moving image file including a moving image stream including moving image data and audio data are recorded;
An image processing apparatus comprising: a generating unit that generates a new moving image stream using the still image data and the audio data in the moving image file.   The generating means generates the new moving image stream using audio data in a moving image file having a relationship with the still image data among the plurality of moving image files recorded on the recording medium. The image processing apparatus according to claim 1.   3. The generation unit generates the new moving image stream using a plurality of still image data recorded on the recording medium and audio data in a moving image file associated with each of the still image data. The image processing apparatus described.   The image processing apparatus according to claim 2, wherein the generation unit determines the presence / absence of the association using recording time information of the moving image file and the still image file.   The moving image data and audio data in the moving image file are encoded in a predetermined encoding format, and the generation unit encodes the still image data and the audio data in the predetermined encoding format. The image processing apparatus according to claim 1, wherein a new moving image stream is generated.   The generating means extracts audio data for a predetermined period from audio data in the moving image file for the still image data for one screen, and the audio data for the predetermined period and the still image data for the one screen are extracted from the extracted audio data for the predetermined period. The image processing apparatus according to claim 1, wherein a moving image stream for a predetermined period is generated.   The image processing apparatus according to claim 1, further comprising a recording unit that records a moving image file including a new moving image stream generated by the generating unit on the recording medium.   The image processing apparatus according to claim 1, wherein the generation unit converts the still image data into an MPEG I picture and generates the new moving image stream. An image processing method using a recording medium in which a still image file including still image data and a moving image file including a moving image stream including moving image data and audio data are recorded,
An image processing method comprising a generation step of generating a new moving image stream using the still image data and audio data in the moving image file.

Images