JP2004363828A - Data reproducing device, method, computer program, and computer-readable storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make an appreciator not feel uneasy and unpleasant even when a partial audio stream is absent when a scene formed by connecting a plurality of clips of, for example, a moving picture is reproduced and appreciated by preventing information from ceasing during a reproduction period of processed data. <P>SOLUTION: Stream data including an image stream and an audio stream, and a rendering clip generated by using the stream data are reproduced according to reproduction description data. In this case, if the rendering clip has no audio stream, the audio stream of the stream data is continuously reproduced in a reproduction period of the rendering clip as well as the reproduction of the rendering clip. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a data reproducing apparatus, a method, a computer program, and a computer-readable storage medium suitable for reproducing image data and audio data according to reproduction description data.
[0002]
[Prior art]
In recent years, devices handling digital image data such as digital VTRs and DVD players have become widespread, and moving images have been edited not only in studios but also at home due to advances in AV devices and computers.
[0003]
For example, in a video editing system called non-linear editing, moving image data of a moving image A (1301) and a moving image B (1302) as materials shown in FIG. 12 are edited to generate a moving image C (1303). In this way, in conventional nonlinear editing, moving image data as a material is cut out, cut out in the time direction by cutout, rearranged, and rendered with an effect such as a wipe effect between cuts, and rendered. A system for generating new moving image data is generally used.
[0004]
Also, moving image data generally has a large amount of information, and thus is often compression-encoded. If the material data is compression-encoded in this way, it is necessary to decode the data once and then edit it. And then encode again.
[0005]
However, in the above-described nonlinear editing system, processing of an additional effect such as wipe or dissolve is directly performed on image data, so that only the additional effect can be deleted later or replaced with another effect. In addition, it is not possible to distinguish between the portion that has become the material and the portion that has been subjected to the additional effect processing.
[0006]
In recent years, a description language for controlling reproduction processing of moving image data, audio data, and the like has been proposed. For example, SMIL (Synchronized Multimedia Integration Language) is one example. In SMIL, synchronous reproduction, sequential reproduction of image data such as moving images and still images, or reproduction processing of other media data such as text and audio can be instructed. In addition, it is possible to specify a special effect or the like to be applied when the display of the image data is switched. Furthermore, it is possible to add a video effect by using a reproduction description language, and the contents of the effect can be replaced by rewriting the reproduction description language.
[0007]
However, even if a free reproduction procedure and a special effect can be described using a reproduction description language such as SMIL, the reproduction as instructed may not be possible depending on the reproduction environment in which the reproduction processing is actually performed. is there. For example, the special effect described cannot be executed in real time depending on the reproduction environment, and the description about the special effect may be ignored. Therefore, it is impossible to describe a sophisticated and complicated video effect as a description method specification.
[0008]
As a solution to the above problem, for example, Patent Literature 1 discloses an information processing apparatus that processes information data and reproduction description data indicating a reproduction procedure of the information data, and newly uses a part of the information data. And a description data processing means for changing the reproduction description data so as to reproduce the processing data in place of the partial data in the information data. Has been made.
[0009]
With reference to FIG. 11, the data processing proposed in Patent Document 1 will be described. The playback procedure for realizing the effect AB (403) (for example, wipe) at the connection point between the moving image A (401) and the moving image B (402) shown in FIG. The reproduction end point of the moving image A and the reproduction start point of the moving image B are moved to predetermined positions, and a reproduction description in which the effect AB (403) (processed data) is inserted therebetween is described.
[0010]
[Patent Document 1]
JP-A-2003-218385
[0011]
[Problems to be solved by the invention]
However, the proposal in Patent Literature 1 mainly deals with processing of moving image data when generating new processing data using a part of information data. Not. Generally, moving image editing performs data editing and data processing mainly on moving images. In such a case, there is little compensation for reliably processing the audio data except when the audio data is edited and added by post-recording. In particular, very few simple video editing software supports editing and processing of audio data (audio stream). However, when a scene in which a plurality of clips of a moving image are connected is played back and viewed, if a part of the audio stream is missing, there is a possibility that the content may cause anxiety and discomfort to the viewer.
[0012]
The present invention has been made to solve the above-mentioned problems, and an object of the present invention is to prevent information from being interrupted during a reproduction period of processed data. Another object of the present invention is to propose to appropriately perform a control operation in an editing and reproducing process when a means for easily changing a reproducing procedure of information data as a material is applied.
[0013]
[Means for Solving the Problems]
A data reproducing apparatus according to the present invention is a data reproducing apparatus that reproduces a first data string including a plurality of types of data and a second data string generated using the first data string according to reproduction description data. When the second data string does not include a predetermined type of data among the plurality of types, data is reproduced according to the predetermined type of data during a reproduction period of the second data. .
[0014]
Another data reproducing apparatus of the present invention reproduces information data and processed data generated by using a part of the information data, instead of the part of the information data. A reproducing unit that reproduces from the recording medium according to reproduction description data that specifies a reproduction procedure of the information data and the processed data; a determining unit that determines that the information of the processed data is insufficient; and that the determining unit determines that the information is insufficient. In this case, the method is characterized in that the processing data is reproduced during the reproduction period of the processed data, and the control means controls the reproducing means so as to continuously reproduce the information data immediately before the reproduction of the processed data.
[0015]
Another data reproducing apparatus of the present invention reproduces the processed data from the information data and the processed data generated by using a part of the information data instead of the part of the information data. A reproducing means for reproducing from the recording medium in accordance with reproduction description data for specifying a reproduction procedure of the information data and the processed data; a judging means for judging information shortage of the processed data; and judging that the information is insufficient in the judging means. In this case, a control means is provided to reproduce the alternative information data together with the processed data during the reproduction period of the processed data.
[0016]
In the data reproducing method of the present invention, the reproducing means reproduces the processed data generated by using the information data and the partial data of the information data, instead of the partial data of the information data. A procedure for reproducing from the recording medium in accordance with reproduction description data for specifying a reproduction procedure of the information data and the processed data, a procedure for determining information shortage of the processed data by a determining unit, Controlling the reproducing means to reproduce the processed data during the reproduction period of the processed data and to continuously reproduce the information data immediately before the reproduction of the processed data when it is determined that the information is insufficient. It is characterized by points.
[0017]
In another data reproducing method of the present invention, the reproducing means converts the information data and the processed data generated by using a part of the information data into the processed data instead of a part of the information data. A step of reproducing from the recording medium according to the reproduction description data specifying the reproduction procedure of the information data and the processed data so as to reproduce data, a step of determining information shortage of the processed data by a determination unit, and a control unit A step of reproducing alternative information data together with the processed data during a reproduction period of the processed data when the determining means determines that the information is insufficient.
[0018]
The computer program according to the present invention, the information data and processed data generated using a part of the information data, the information data so as to reproduce the processed data instead of a part of the information data A process of reproducing from the recording medium in accordance with reproduction description data specifying a reproduction procedure of the processed data, a process of determining information shortage of the processed data, and a process of determining that the information is insufficient in the determination process. And reproducing the processed data during the reproduction period, and controlling the reproducing means to continuously reproduce the information data immediately before the reproduction of the processed data.
[0019]
Another computer program of the present invention reproduces information data and processed data generated by using a part of the information data, instead of the part of the information data. A process of reproducing from the recording medium in accordance with the reproduction description data specifying the reproduction procedure of the information data and the processed data, a process of determining information shortage of the processed data, and when it is determined that the information is insufficient in the determination process, A feature is that a process is executed by a computer to reproduce the alternative information data together with the processed data during the reproduction period of the processed data.
[0020]
The computer-readable storage medium of the present invention is characterized in that the computer program of the present invention is stored.
[0021]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing a configuration of an imaging apparatus 200 according to an embodiment to which the present invention is applied. In FIG. 1, reference numeral 201 denotes a lens for capturing an image, 202 denotes a solid-state sensor that converts an image into an electric signal, and 203 denotes a solid-state sensor. An A / D converter for converting an electric signal output from the digital camera 202 into digital data, an iris 205, and a motor 204 for driving the lens 201 and the iris 205.
[0022]
Reference numeral 209 denotes a microphone that captures audio, 210 denotes a microphone amplifier that amplifies audio, and 211 denotes an A / D converter that converts audio into digital data.
[0023]
Reference numeral 212 denotes an operation key for operating the imaging device 200. Reference numeral 213 denotes a display driver for converting an image into a display format, and 214 denotes a display such as an LCD or EVF for displaying an image. Reference numeral 215 denotes a speaker driver for outputting sound, and 216 denotes a speaker.
[0024]
An input / output terminal 217 inputs / outputs various information. Reference numeral 224 denotes various sensors. Reference numeral 101 denotes a storage medium, and 223, a socket for storing the recording medium 101. The recording medium 101 is detachable from the socket 223. As the recording medium 101, a memory card such as an MMC, an SSFDC, a compact flash (R), an SD, a PC card, or a hard disk is used. Alternatively, a magnetic (or magneto-optical) recording medium such as a CD-RW, FD, MD, or DVD-RAM is used. By making the shape of the socket 223 compatible with a plurality of types of recording media, a plurality of types of recording media can be used.
[0025]
230 is a CPU for controlling the system, 250 is a decoder block configured in the CPU 230, and 222 is a memory for the CPU 230 and the system. The decoder block 250 can be constituted by either firmware of the CPU 230 or hardware in the CPU 230, or both.
[0026]
Reference numeral 221 denotes a power supply for the imaging device 200, and power is supplied to each circuit block.
[0027]
225 is a recording / reproduction control circuit.
[0028]
Next, a recording operation in the imaging device 200 shown in FIG. 1 will be described. In the image recording operation, light from a subject incident through the lens 201 is converted into an electric signal by the solid-state sensor 202, sampled by the A / D converter 203, and converted into digital data. The digital data is input to the CPU 230 as a digital video signal such as luminance Y, color difference Cr, Cb, or RGB. The lens 201 can be moved by driving the motor 204 according to a control command of the CPU 230 by the autofocus function and the zoom function. A timing signal for extracting data from the solid-state sensor 202 is generated by the CPU 230. The iris 205 is controlled by driving the motor 204 from the CPU 230. When the lens 201 has an anti-vibration function (acceleration sensor, anti-vibration actuator, etc.), signals from the acceleration sensor and the position sensor of the anti-vibration actuator are input to the CPU 230, and the anti-vibration actuator is used as an anti-vibration control output signal. Drive control is performed by the motor 204.
[0029]
Further, the sound taken in from the microphone 209 is amplified by the microphone amplifier 210, sampled by the A / D converter 211, converted into digital data, and input to the CPU 230 as a digital sound signal.
[0030]
For the digital video signal, the CPU 230 performs basic processing such as color separation, white balance, gamma correction, and aperture correction, and additional processing such as image size, image quality adjustment, and position adjustment set by the operation keys 212. Then, the processed signal is compression-coded according to the set compression method and compression parameters.
[0031]
Further, the digital audio signal is subjected to additional processing such as sound quality adjustment set by the operation keys 212. Then, the processed signal is compressed according to the set compression method and compression parameter.
[0032]
The compressed video data and compressed audio data are formed as stream data together with the control data. The stream data format generates a program stream or a transport stream specified by ISO / IEC13818-1. The stream data is sent to the recording / reproduction control circuit 225 as recording data. The control data includes a header for forming stream data, management data, information on the image pickup apparatus itself, recording setting information such as images and sounds, additional photographing information such as photographing history and photographing environment information, a recording medium used, and the like. Media information, and time information such as recording time, code time, reproduction decoding time, reproduction display time, and the like.
[0033]
Further, the files are stored in a file system management system such as UDF (universal disk format), FAT (file allocation table), EXT (environmental audio extension) 2, and the like. Writing is performed on the recording medium 101 as a file with a file name of mpg. The file system means a structure system including management information for managing a file and file data. By sharing this file system, data can be recognized and recorded / reproduced by different recording media or recording / reproducing devices.
[0034]
FIG. 2 shows an example of a directory to be recorded. Below the root directory (Root), a file (media.txt) describing information of the recording medium and the like, a playlist file (playlist.sml) described later, and a content directory (content) for storing a group of recorded contents are stored. Below the content directory, information files (content.info) describing content information and the like, moving image contents (mov1.mpg, mov2.mpg), audio contents (aud1.mpg), still image contents (img.jpg), text contents ( text.txt) and so on.
[0035]
The recorded file mov3. mpg is registered in the content information file (content.info). Items to be registered include a file name, a file attribute, recording setting information, shooting additional information, recording time, reproduction decoding time, reproduction display time, copyright information, and the like.
[0036]
It is also registered in a playlist file (playlist.sml). The playlist is a script for reproducing the content, and describes a reproduction order, a reproduction procedure, a reproduction effect, and the like. In the present embodiment, the reproduction description language is described in SMIL. FIG. 3 is a description example of a playlist, and here, mov1. mpg, mov2. mpg, mov3. An example in which each moving image file of mpg is sequentially reproduced is shown.
[0037]
The data recorded as described above is sent to the external input / output terminal 217 as needed. Further, the image data before encoding is displayed on the display 214 through the display driver 213 for checking the recorded image.
[0038]
Here, the rendering clip is a clip created by rendering a video effect. As described in the conventional example, even if a free reproduction procedure and a special effect can be described using a reproduction description language such as SMIL, in a reproduction environment for actually performing reproduction processing, particularly in an embedded device such as a video camera. Aside from the fade effect and the resolve effect, special effects that require complicated processing such as a wipe effect and a mix effect may not be able to be processed in real time as instructed in real time. Therefore, by preparing the special effect as a clip in advance and playing it back normally, the processing load on the device can be reduced.
[0039]
The rendering clip is created by an external device such as a PC, or is created by an operator using the imaging apparatus 200 of the present embodiment. An example of a method for creating a rendering clip will be described with reference to FIG. When a rendering clip of the wipe (crossfade) effect of the period T3 is created between the time T1 of the clip 1 (stream 1) and the time T2 of the stream 2 (clip 2), from the time T1 of the clip 1 to the time T2 And the data of the clip 2 during the period from time T1 to time T2 are respectively decoded to extract the image signal and the audio signal. Then, the clip 1 signal is mixed to 100%, the clip 2 signal is set to 0%, the clip 1 signal is sequentially mixed to 0%, and the clip 2 signal is mixed to 100%. The mixed signal is encoded and rendered and clipped. This operation can be processed by a dedicated circuit or by software.
[0040]
FIG. 6 shows an example of a playlist using a rendering clip. The first line specifies reproduction of a clip of 5 to 21 seconds from the beginning of the V1 stream file. TailShift = "2s" means that the rendering clip insertion period is shifted forward by 2 seconds.
[0041]
The second line specifies that the RC rendering clip is to be inserted for 2 seconds, and systemInsert = “true” means that this clip is a rendering clip. The efxType element in the third line holds the type of the rendered video effect and the applicable range of the video effect, and indicates that the transition effect is rendered by setting transitionFilter = “true”.
[0042]
The fourth line specifies reproduction of a clip of 5 to 52 seconds from the beginning of the V2 stream file. headShift = “2s” means that the rendering clip insertion period is shifted backward by 2 seconds.
[0043]
With such a script, it is possible to specify that each clip of VI, RC, and V2 is sequentially reproduced for a predetermined period. In FIG. 6, the detailed transition effect is not described, but it is also possible to describe the transition effect type, the method, the period, and the like using the parameter element.
[0044]
Next, a reproduction operation in the imaging device 200 will be described. FIG. 4 is a diagram showing a detailed configuration of the decoder block 250 in the block diagram of FIG. 1, where 301 is a stream separator demux 1, 302 is a stream separator demux 2, 303 is a video stream switcher, and 304 is an audio stream. A switch 305 is a video decoder, 306 is an audio decoder, 307 is a switch, 310 is an input terminal of an encoded stream, 311 is an input terminal of a rendering clip stream described later, 312 is an output terminal of a video decoded signal, and 313 is an audio terminal. An output terminal 314 of the decoded signal is an input terminal of the audio signal.
[0045]
1 and 4, when the imaging apparatus 200 is in the reproduction mode, the content information and the playlist recorded on the recording medium 101 are read, and a list of the content recorded on the recording medium 101 and the playlist are obtained. Then, a consistency check of the content list, the playlist, and each content is performed. When a file operation (deletion, editing, or the like) is directly performed on the recording medium 101 by an external device, the content file registered in the content list or the play list is lost, or the content file is edited. When a playback operation is performed based on a content list or a playlist, there is no playback file or inconsistency in the playback time, which prevents smooth playback operation. Perform sex check. As an example of the consistency check, it is compared whether or not the content file and the information of the content file described in the playlist differ from the substance of each content file. The comparison items include the file entity, file attributes such as file size, update date and time, and type, and file playback time. It is also conceivable to provide means for arbitrarily setting comparison items. If an error occurs during the consistency check, modify the content list and playlist as much as possible.
[0046]
Then, the content list or the playlist is displayed on the display 214, and the operator selects a desired content or playlist. For example, when the play list shown in FIG. 6 is selected and playback is performed, the play list is read and the contents are analyzed. This play list includes a rendering clip according to the description of the transition filter = "true". In this case, first, the file RC is pre-read. Then, it is checked whether an audio stream exists in the RC file. If the RC file is a program stream, the presence / absence check of the audio stream is determined by checking the stream type (MPEG video, MPEG audio, etc.) described in the system header recorded at the head of the program stream. Can be.
[0047]
First, when an audio stream is present in the rendering clip, a stream 5 seconds after the beginning of the V1 file is read into the CPU 230, the stream is input to the terminal 310, and the system data and image data are input by the stream separator Demax 1 (301). , And separate into audio data. The switch 303 is connected to the position 1V, and the switch 304 is connected to the position 1A. The video data is decoded by the video decoder 305 via the switch 303 and output to the output terminal 312. The audio data is decoded by the audio decoder 306 via the switch 304 and output to the output terminal 313. Video and audio synchronization control uses a time stamp from the system data. The V1 file is reproduced up to 19 seconds, 2 seconds before, 21 seconds after the beginning.
[0048]
Before the reproduction of the V1 file is completed, the pre-read RC file is input to a terminal 311 and separated into system data, image data, and audio data by a stream separator Demax 2 (302). Further, at the timing when the total playback time becomes 19 seconds at the playback image output terminal 312 and the playback audio output terminal 313, switching of the switch 303 to the position 2V and switching of the switch 304 to the position 2A are performed. The video data is decoded by the video decoder 305 via the switch 303 and output to the output terminal 312. The audio data is decoded by the audio decoder 306 via the switch 304 and output to the output terminal 313. Video and audio synchronization control uses a time stamp from the system data. The RC file is played for 2 seconds.
[0049]
Further, before the end of the reproduction of the RC file, the V2 file is read from the beginning 5 seconds to 7 seconds after 2 seconds, the stream data is input to the terminal 310, and the system data and image data are input to the stream separator Demax 1 (301). Separate into data and audio data. Further, at the timing when the total playback time becomes 21 seconds at the playback image output terminal 312 and the playback audio output terminal 313, switching of the switch 303 to the position 1V and switching of the switch 304 to the position 1A are performed. The video data is decoded by the video decoder 305 via the switch 303 and output to the output terminal 312. The audio data is decoded by the audio decoder 306 via the switch 304 and output to the output terminal 313. Video and audio synchronization control uses a time stamp from the system data. The V2 file is reproduced up to 52 seconds from the beginning.
[0050]
The buffer memories in the video decoder 305 and the audio decoder 306 are used for adjusting the timing of the total playback time at the playback image output terminal 312 and the playback audio output terminal 313. Further, the switch 307 is connected to the position 3A, and the input of the audio input terminal 314 is invalid.
[0051]
Next, when there is no audio stream in the rendering clip, the stream 5 seconds after the beginning of the V1 file is read into the CPU 230, the stream is input to the terminal 310, and the stream separator Demax 1 (301) outputs system data, Separate into image data and audio data. The switch 303 is connected to the position 1V, and the switch 304 is connected to the position 1A. The video data is decoded by the video decoder 305 via the switch 303 and output to the output terminal 312. The audio data is decoded by the audio decoder 306 via the switch 304 and output to the output terminal 313. Video and audio synchronization control uses a time stamp from the system data. The V1 file is reproduced up to 21 seconds after the beginning.
[0052]
Before the reproduction of the V1 file is completed, the pre-read RC file is input to a terminal 311 and separated into system data and image data by a stream separator Demax 2 (302). Further, at the timing of the playback image output terminal 312, at the timing when the total playback time becomes 19 seconds, the switch 303 is switched to the position 2V. The video data is decoded by the video decoder 305 via the switch 303 and output to the output terminal 312. In this case, since there is no audio data in the rendering clip, the switch 304 does not perform switching at the position 1A. The audio data is decoded by the audio decoder 306 via the switch 304 from the output of the stream separator Demax 301, and output to the output terminal 313. Video and audio synchronization control uses time stamps from both system data. In this case, since the video and the audio are different streams, there is a difference in the real-time time stamp. Therefore, the audio time stamp is controlled to be relatively synchronized with the video time stamp. The RC file is played for 2 seconds.
[0053]
Further, before the end of the reproduction of the RC file, the V2 file is read from the beginning 5 seconds to 7 seconds after 2 seconds, the stream data is input to the terminal 310, and the system data and image data are input to the stream separator Demax 1 (301). Separate into data and audio data. Further, at the timing of the reproduction image output terminal 312, at the timing when the total reproduction time becomes 21 seconds, the switch 303 is switched to the position 1V. The video data is restored by the video decoder 305 via the switch 303 and output to the output terminal 312. The audio data is decoded by the audio decoder 306 via the switch 304 and output to the output terminal 313. Video and audio synchronization control uses a time stamp from the system data. The V2 file is reproduced up to 52 seconds from the beginning.
[0054]
The buffer memories in the video decoder 305 and the audio decoder 306 are used for adjusting the timing of the total playback time at the playback image output terminal 312 and the playback audio output terminal 313. The switch 307 is connected to 3A, and the input of the audio input terminal 314 is invalid.
[0055]
FIG. 5 illustrates the operation of the switch when the playlist of FIG. 6 is reproduced, and shows the file (clip) reproduction order of the V1 file, the V2 file, and the rendering clip RC. (1), (2), (3), and (4) explain the switching operation of the switches 303, 304, and 307 in FIG. 4, and (1) and (4) show the presence of audio data of a rendering clip. The operation of the switches 303 and 307 regardless of the presence or absence, (2) the operation of the switch 304 when audio data exists in the rendering clip, and (3) the operation of the switch 304 when audio data does not exist in the rendering clip. Action. In the rendering clip reproduction period T3, when sound is present, the switch 304 is connected to the position 2A, and when no sound is present, the switch 304 operates to connect to the position 1A.
[0056]
Next, a method different from continuing the audio of the previous stream when the audio stream does not exist in the rendering clip will be described. FIG. 8 illustrates the operation of the switch when the playlist in FIG. 6 is reproduced, and shows the file (clip) reproduction order of the V1 file, the V2 file, and the rendering clip RC. (1), (2) and (3) explain the switching operation of the switches 303, 304 and 307 in FIG. 4, and (1) and (3) show whether or not the audio data of the rendering clip exists. (2) is the operation of the switch 307 when there is no audio data in the rendering clip. When the playlist shown in FIG. 6 is reproduced, if audio data is present in the rendering clip, the operation is the same as described above. If there is no audio data, an audio signal serving as an alternative audio is input to the terminal 314, and a switch is performed. Switch 307 from position 3A to position 4A. That is, an alternative voice is used. The alternative sound is a sound file provided in the device by default for each special effect, a sound file prepared in advance by an operator, a sound file obtained from a recording medium or an external device by the device according to the special effect, etc. Use If the audio file has been encoded, it is decoded and then input to the terminal 314.
[0057]
The substitute file is an audio file, but as a substitute file, a text file, an image file, or the like is used.In the case of a text file, the text is processed as font data, and the image file is processed into a format that can be displayed in an image. , Can be mixed with the image output terminal 312 for display. Furthermore, if there is no audio stream, in addition to continuing the audio of the previous stream, if there is a transition effect type, method, period, etc. described in the parameter element, perform the corresponding processing as much as possible Is also possible. For example, if a process such as fade-out is described, the continuous audio stream is faded out.
[0058]
FIG. 10 is a display example when the present embodiment is applied, and 214 is a display in FIG. This display example is a display example in a case where an audio stream does not exist in a rendering clip to be used when a playlist including the above-described rendering clip is reproduced. In the display 501, the display of "rendering clip. RC is being reproduced" indicates that the rendering clip is being reproduced. The display "*** representative voice fade.mp3" indicates that the fade-in voice is replaced as a substitute voice. This indicates that the stream file mp3 is used.
[0059]
This indication tells which file stream is currently being played. In particular, when the audio stream of the rendering clip does not exist, the operator is notified and alerted. In this case, the information is meaningful for reporting the file stream automatically selected by the device. Further, during this display period, the information is effective as information for the operator to cancel the automatic selection and stop the reproduction.
[0060]
FIG. 7 is a flowchart illustrating play list reproduction in the present embodiment. If an instruction to start reproducing a moving image of a playlist is issued in step S401, the playlist described in SMIL or the like is analyzed in step S402, and the contents, the playback procedure, and the like are scheduled. In step S403, each circuit (or firmware) for reproduction (decoding) is initialized. In step S404, it is determined whether a rendering clip is used in the playlist.
[0061]
If the rendering clip is not used, the process moves to step S411. If so, the process moves to step S405 to preload the rendering clip. Then, a stream analysis of the rendering clip read in step S406 is performed to determine whether or not audio data exists (if the presence or absence of the audio information of the rendering clip is described in the content information file, By determining the information, the processing of step S404 and step S405 may be performed instead.) If audio data exists (step S407), Flag is set to "1" in step S410, and if audio data does not exist (step S407), flag is set to "0" in step S409.
[0062]
In step S411, reproduction (decoding) is started. In step S412, it is determined whether the playback stream to be decoded is a rendering clip. If it is not a rendering clip, the flow shifts to step S413 to perform normal reproduction (decoding process), and shifts to step S414. If it is a rendering clip, the process proceeds to step S420 to determine the flag.
[0063]
If the flag is “0”, the process proceeds to step S421 to display on the display unit information indicating that no audio data is present in the rendering clip and that the alternative processing is being executed. Then, the process 0 is executed in the step S423, and thereafter, the process shifts to the step S414. Process 0 is a process performed when there is no audio in the above-described rendering clip.
[0064]
If the flag is “1”, the flow shifts to step S421 to execute processing 1, and then shifts to step S414. Process 1 is a process in the case where audio is present in the above-described rendering clip.
[0065]
In step S414, if the reproduction is not completed, the process proceeds to step S412, and if the reproduction is completed, the process proceeds to step S450 and ends.
[0066]
The reproduction use file of the process 0 and the reproduction operation history can be automatically registered in a script language (as a playlist), a recording medium, or the like. The playlist registered in the history becomes an effective function because it becomes a playlist whose playback compatibility is compensated by another device.
[0067]
Also, in the present embodiment, the case where the missing information in the rendering clip is audio data has been described. Even when the missing data is image data, management data, and additional information data, the data or file before the rendering clip is continuously reproduced. Alternatively, it is possible to replace alternative data or files. In the above case, as the continuous reproduction or the substitute data or file, if the missing data is image data, the image file before the rendering clip. When the missing data is management data such as file date and time or additional information data, management data or additional information data before the rendering clip is used.
[0068]
By performing the above operation, in the present embodiment, when a playlist playback using a rendering clip is executed in an imaging device such as a consumer video camera, in the case of a rendering clip having no audio stream, Can be prevented.
[0069]
(Other embodiments)
It is to be noted that an object of the present invention is to supply a storage medium storing program codes of software for realizing the functions of any of the devices or systems of the above-described embodiments to the device or system, and to provide a computer (CPU Or, the MPU or the like) can read and execute the program code stored in the storage medium, and needless to say, it can be achieved.
[0070]
In this case, the program code itself read from the storage medium realizes the function described in the above embodiment, and the storage medium storing the program code and the program code constitute the present invention.
[0071]
As a storage medium for supplying the program code, a ROM, a floppy (R) disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, or the like can be used. .
[0072]
When the computer executes the readout program code, not only the functions described in the above-described embodiments are realized, but also an OS or the like running on the computer is used based on the instructions of the program code. It goes without saying that a case where some or all of the actual processing is performed and the functions described in the above embodiments are realized by the processing is also included in the embodiment of the present invention.
[0073]
Further, after the program code read from the storage medium is written into a memory provided on a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. It goes without saying that a case where a CPU or the like provided in a board or a function expansion unit performs part or all of actual processing and the functions described in the above-described embodiments are realized by the processing is also included in the embodiment of the present invention. No.
[0074]
It should be noted that the shapes and structures of the respective parts shown in the above-described embodiments are merely examples of the specific embodiments for carrying out the present invention, and the technical scope of the present invention is limited thereby. It must not be interpreted. That is, the present invention can be embodied in various forms without departing from the spirit or main features thereof.
[0075]
【The invention's effect】
As described above, according to the present invention, it is possible to prevent the information from being interrupted even if the processed data has insufficient information during the reproduction period of the processed data.
[0076]
In particular, in the case where the information data is viewing image and sound data including image data and sound data, comfortable image and sound viewing can be prevented.
[0077]
Further, if information is insufficient, the fact is displayed, so that the operator can easily grasp the reproduction state.
[Brief description of the drawings]
FIG. 1 is a diagram showing a configuration of an imaging device 200.
FIG. 2 is a diagram illustrating an example of a directory.
FIG. 3 is a diagram illustrating a description example of a playlist.
FIG. 4 is a diagram showing a detailed configuration of a decoder block 250.
FIG. 5 is a diagram for explaining the operation of the switch when the play list in FIG. 6 is reproduced.
FIG. 6 is a diagram illustrating an example of a playlist using a rendering clip.
FIG. 7 is a flowchart illustrating play list reproduction.
FIG. 8 is a diagram for explaining the operation of the switch when the play list in FIG. 6 is reproduced.
FIG. 9 is a diagram for explaining an example of a method of creating a rendering clip.
FIG. 10 is a diagram showing a display example on a display 214.
FIG. 11 is a diagram for explaining conventional data processing.
FIG. 12 is a diagram for describing an example of non-linear editing.
[Explanation of symbols]
200 Imaging device
201 lens
202 Solid-state sensor
203 A / D converter
204 motor
205 Iris
209 microphone
210 microphone amplifier
211 A / D converter
212 operation keys
213 Display Driver
214 display
215 Speaker driver
216 Speaker
217 External input / output terminal
221 power supply
223 socket
230 CPU
301 Stream Separator Demax
302 Stream Separator Demax
303 Video Stream Switcher
304 audio stream switcher
305 video decoder
306 audio decoder
307 switch
310-314 terminal

Claims (13)

What is claimed is: 1. A data reproducing apparatus for reproducing a first data string composed of a plurality of types of data and a second data string generated using the first data string in accordance with reproduction description data,
A data reproducing device that reproduces data corresponding to the predetermined type of data during a reproduction period of the second data when the second data sequence does not include the predetermined type of data among the plurality of types. . A reproduction procedure of the information data and the processed data so that the information data and the processed data generated by using a part of the information data are reproduced as the processed data instead of a part of the information data. Playback means for playing back from a recording medium according to playback description data specifying
Determining means for determining the lack of information of the processing data;
When the determination unit determines that the information is insufficient, the control unit controls the reproduction unit to reproduce the processed data during the reproduction period of the processed data and to continuously reproduce the information data immediately before the reproduction of the processed data. And a data reproducing apparatus. Reproduction of the information data and the processed data so that the processed data generated by using the information data and a part of the information data is reproduced instead of the part of the information data. Reproducing means for reproducing from a recording medium according to reproduction description data specifying a procedure;
Determining means for determining the lack of information of the processing data;
A data reproducing apparatus, comprising: a control unit for reproducing alternative information data together with the processed data during a reproduction period of the processed data when the determining unit determines that the information is insufficient. 4. The data reproducing apparatus according to claim 2, wherein the information data includes at least image data and audio data. 5. The data reproducing apparatus according to claim 4, wherein the determination unit determines whether audio data or image data is insufficient in the processed data. 6. The data reproducing apparatus according to claim 2, wherein when the information is determined to be insufficient, the information shortage is displayed. 7. The data reproducing apparatus according to claim 2, wherein the information data, the processed data, and the reproduction description data are recorded on an external recording medium. The data reproducing apparatus according to any one of claims 2 to 7, wherein the data reproducing apparatus is included in an image pickup apparatus including a unit for inputting audio and an image pickup unit for inputting an image. The information processing unit is configured to reproduce the information data and the processed data generated by using a part of the information data by the reproducing unit so that the processed data is reproduced instead of the part of the information data. A procedure for reproducing from a recording medium according to reproduction description data specifying a data reproduction procedure;
A step of determining information shortage of the processing data by a determining unit;
When the control means determines that the information is insufficient, the reproduction means reproduces the processed data during the reproduction period of the processed data and continuously reproduces the information data immediately before the reproduction of the processed data. And a procedure for controlling the data reproduction. By the reproducing means, the information data and the processed data generated by using a part of the information data, the information data and the processed so as to reproduce the processed data instead of a part of the information data A procedure for reproducing from the recording medium according to the reproduction description data specifying the reproduction procedure of the processed data;
A step of determining information shortage of the processing data by a determining unit;
A step of, when the control means determines that the information is insufficient, reproducing the substitute information data together with the processed data during a reproduction period of the processed data. A reproduction procedure of the information data and the processed data so that the information data and the processed data generated by using a part of the information data are reproduced as the processed data instead of a part of the information data. Processing to reproduce from a recording medium according to reproduction description data specifying
Processing for determining the lack of information of the processing data;
When it is determined in the determination process that the information is insufficient, the process of reproducing the processed data during the reproduction period of the processed data and controlling the reproducing unit to continuously reproduce the information data immediately before the reproduction of the processed data. And a computer program for causing a computer to execute the following. Reproduction of the information data and the processed data so that the processed data generated by using the information data and a part of the information data is reproduced instead of the part of the information data. A process of reproducing from a recording medium according to reproduction description data specifying a procedure;
Processing for determining the lack of information of the processing data;
A computer program for causing a computer to execute a process of reproducing alternative information data together with the processed data during a reproduction period of the processed data when it is determined that the information is insufficient in the determination process. A computer-readable storage medium storing the computer program according to claim 11.

Images