JP2006340066A - Moving image encoder, moving image encoding method and recording and reproducing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To allocate a large code amount to an important scene of a high viewing value by taking into consideration the degree of importance of semantic contents of moving images in re-encode dubbing by a moving image encoding system. <P>SOLUTION: When encoding video signals (110) and sound signals (111) supplied from the outside and recording them in a first recording medium (102), the video signals and the sound signals to be recorded are divided into respective prescribed sections, the feature of the sound of the sound signals in each section is extracted (143), and an encoding rate is decided on the basis of the feature of the sound indicated by the sound signals in each section (120, 121, 122, 123). When performing dubbing from the first recording medium (102) to a second recording medium (103), encoding is performed by the previously decided encoding rate (112, 113). <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention is, for example, a moving picture coding apparatus, a moving picture coding method, and a recording that can be applied to a case where a multimedia data file is recorded and dubbed by a video recording / playback apparatus such as a hard disk drive (HDD) built-in DVD recorder. It relates to a playback method.

  Here, the multimedia data file refers to a single file obtained by multiplexing video information encoded and compressed by, for example, the MPEG-2 system and audio information encoded and compressed by the AC-3 system. Further, dubbing is defined as a process of copying (or moving) a multimedia data file recorded on a first recording medium provided in the video recording / reproducing apparatus to a second recording medium.

  As general dubbing methods, two methods of “high-speed dubbing” and “re-encode dubbing” are widely known. “High-speed dubbing” is a dubbing method based on file copying as a basic principle. Dubbing can be performed at high speed, but the encoding conditions cannot be changed, such as redistribution of encoding rates and resetting of encoding parameters. . On the other hand, “re-encode dubbing” is a method in which after encoding / compressed video / audio information included in a multimedia data file recorded on the first recording medium is once decoded, a desired encoding rate or encoding parameter is set. On the basis of this, the encoding / compression is performed again and the data is recorded on the second recording medium. Therefore, in many cases, the dubbing process requires the same time as normal playback, but encoding conditions such as redistribution of encoding rates and resetting of encoding parameters can be changed. That is, in the re-encoding dubbing, the encoding rate can be redistributed according to the video scene.

  In the conventional re-encoding dubbing, when a multimedia data file is normally recorded on the first recording medium, the required amount of code of the input video (also referred to as encoding difficulty) is analyzed every certain interval, There is a moving image encoding device (also called a two-pass encoding method) that redistributes code amounts based on the required code amount analyzed during dubbing (see, for example, Patent Document 1). Similarly, there is also a moving image encoding apparatus that resets encoding parameters based on the required code amount (see, for example, Patent Document 2).

Japanese Patent Laid-Open No. 2002-232882 (page 5-6, FIG. 1) JP 2001-245303 A (page 3-4, FIG. 2)

  In the re-encoding dubbing (or two-pass encoding method) based on the moving image encoding method disclosed in the above-mentioned patent document, feature information is extracted from a video having an enormous amount of information. There is a problem that processing capability is required, and the circuit scale and the feature extraction algorithm become complicated. Regardless of the degree of content importance for the viewer, the code amount is redistributed and the encoding parameters are reset only from the degree of difficulty in encoding the image. Therefore, even if the scene is important for the viewer, such as a sports scoring scene or a movie climax, a large amount of code is not allocated, and block noise occurs at that location, the image is distorted, etc. There was a problem of causing image degradation. The degree of content importance for the viewer is called “viewing value (also referred to as importance level)”.

  Therefore, in the present invention, the degree of importance of the semantic content of the moving image to be encoded is calculated from the audio information, so that appropriate code distribution can be performed according to the viewing value at the time of re-encoding, and the high viewing value is important. An object of the present invention is to provide a moving picture coding apparatus capable of allocating a large amount of code to such a scene.

  The present invention divides the video signal and the audio signal into predetermined intervals and extracts audio feature extraction means for extracting audio features of the audio signal in each interval, and encoding based on the audio features in each interval. There is provided a moving picture coding apparatus comprising coding rate determining means for determining a rate, and means for encoding the video signal and the audio signal at the determined coding rate.

  According to the present invention, an index for determining an encoding rate at the time of re-encoding from only a speech physical quantity with a small amount of information (for example, speaker identification by speech recognition technology, frequency analysis, volume, voice attribute information, etc.) Therefore, even if the circuit scale is small, re-encoding that reflects the feature information can be performed.

Embodiment 1 FIG.
FIG. 1 is a system configuration block diagram of Embodiment 1 according to the present invention. In the figure,
The system control unit 101 performs integrated control of the entire moving image encoding apparatus 100.
In the first embodiment,
Although the hard disk drive 102 and the optical disk 103 are both recording means, in this embodiment, the hard disk drive 102 records multimedia data files and metadata files as dubbing source recording media (primary recording media). The optical disk 103 is used for recording a re-encoded multimedia data file as a dubbing destination recording medium (secondary recording medium).
Recording / reproducing of the file on the disk 103 is performed through the recording / reproducing drive 104.

  The moving image encoding apparatus 100 includes a buffer memory 115, a demultiplexer 130, a video decoder 131, an audio decoder 133, a video encoder 112, an audio encoder 113, a multiplexer 114, a video feature extraction unit 142, an audio feature extraction unit 143, and metadata generation. Unit 120, metadata analysis unit 121, recording rate map holding unit 122, and recording rate changing unit 123.

The buffer memory 115 is used to temporarily hold data read from the hard disk drive 102 and the optical disk 103 and data to be written.
The data held in the buffer memory 115 includes metadata in addition to multimedia data including a video stream encoded by MPEG-2 or the like and an audio stream encoded by AC-3 or the like.

  The demultiplexer 130 sequentially captures the multimedia data file captured in the buffer memory 115, and then separates it into an encoded and compressed video stream or audio stream for output.

The video decoder 131 decodes the video stream encoded by MPEG-2 or the like output from the demultiplexer 130 and outputs an output video signal 132.
The audio decoder 133 decodes the audio stream encoded by AC-3 or the like output from the demultiplexer 130 and outputs an output audio signal 134.
The monitor 140 receives the output video signal 132 output from the video decoder 131 and the output audio signal 134 output from the audio decoder 133, and performs video display and audio output.

The video encoder 112 encodes the video signal 110 input to the video encoding device 100 or the video signal reproduced from the hard disk drive 102 and decoded by the video decoder 131 using MPEG-2 to generate a video stream. To do. That is, as will be described later, the video encoder 112 encodes and compresses the input video signal 110 with MPEG-2 or the like when a program is recorded on the hard disk drive 102 to generate a video stream. On the other hand, when dubbing, the video signal reproduced from the hard disk drive 102 and decoded by the video decoder 131 is received and re-encoded.
The video feature extraction unit 142 performs a motion vector amount between frames, a change amount of a color histogram, an image of a video signal encoded by the video encoder 112 during recording or after recording and before dubbing. Extraction of features such as detection of people and faces using recognition methods.

The audio encoder 113 encodes the audio signal 111 input to the moving image encoding apparatus 100 or the audio signal reproduced from the hard disk drive 102 and decoded by the audio decoder 133 with an AC-3 or the like to generate an audio stream. To do. That is, as will be described later, the audio encoder 113 encodes the input audio signal 111 with AC-3 or the like when recording a program on the hard disk drive 102 to generate an audio stream. On the other hand, when dubbing, the audio signal reproduced from the hard disk drive 102 and decoded by the audio decoder 133 is received and re-encoded.
The audio feature extraction unit 143 performs coefficient value and frequency information after digital sampling on the audio signal encoded by the audio encoder 113 at the time of recording on the hard disk drive 102 or after recording and before dubbing. Feature amount extraction, such as detection of a change of a speaker, a change of a voice level, a change of a speaker using a voice recognition method, or a scene of applause.
The video feature extraction unit 142 can be configured as part of the video encoder 112, and similarly, the audio feature extraction unit 143 can be configured as part of the audio encoder 113.

The feature amount extraction in the video feature extraction unit 142 and the audio feature extraction unit 143 is performed for each predetermined segment synchronized with each other. This predetermined section corresponds to a segment based on a predetermined time interval or data capacity of the video signal or the audio signal.
This time interval corresponds to a video shot, for example. A video shot is composed of continuous frames divided at a predetermined time.

  The multiplexer 114 packetizes the video stream generated by the encoding in the video encoder 112 and the audio stream generated by the encoding in the audio encoder 113, multiplexes them together with the reproduction time information, and sequentially records them in the buffer memory 115.

The metadata generation unit 120 generates metadata based on the feature information output from the video feature extraction unit 142 and the audio feature extraction unit 143. The generated metadata 115 is written into the buffer memory 115 and further written into the hard disk drive 102.
If necessary, the data is read from the hard disk drive 102, written to the buffer memory 115, and further supplied to the metadata analysis unit 121 described later.
As metadata, the importance level for each section, for example, each video shot, is described, and playback time information is described together with this.

  The metadata analysis unit 121 acquires the reproduction time information and the importance level for each of the above-described sections described in the metadata, for example, for each video shot, after sequentially capturing the metadata captured in the buffer memory 115, At the time of re-encoding such as dubbing, re-encoding rate information corresponding to the reproduction time information is generated.

The recording rate map holding unit 122 holds the re-encoding rate information generated by the metadata analysis unit 121.
The recording rate changing means 123, when re-encoding such as dubbing, based on the re-encoding rate information recorded in the recording rate map holding unit 122, and further depending on the playback time information, the video encoder 112 and the audio encoder Each encoding rate of 113 is determined and output.
The video encoder 112 and the audio encoder 113 operate at the encoding rate supplied from the recording rate changing unit 123.

  The metadata generation unit 120, the metadata analysis unit 121, the recording rate map holding unit 122, and the recording rate changing unit 123 can be partially or wholly configured by software, and the software is the system control unit 101. It may be built in. Further, it is assumed that a memory (not shown) is used as appropriate for the process of generating the metadata and the recording rate map.

FIG. 2 shows a file structure diagram according to the first embodiment of the present invention. FIG. 2 shows a logical file structure in the hard disk drive 102 that is the dubbing source.
Reference numeral 20 denotes a root directory that is a directory structure at the highest level of a file structure that logically forms a hierarchical structure, reference numeral 21 denotes a multimedia directory that is a directory structure arranged in a lower hierarchy of the root directory 20, and reference numeral 22 denotes a multimedia directory 21. Similarly, a metadata directory having a directory structure arranged in a lower hierarchy of the root directory 20, an information management file 23 describing management information (including attribute information and playback time information) of programs recorded in the hard disk drive 102, Reference numeral 24 denotes a multimedia data file including at least one of a video stream or an audio stream obtained by encoding and compressing a video signal or an audio signal of a program and multiplexed with reproduction time information, and 25 denotes a backup of the information management file 23 or the like. Airu, 26 is a metadata file comprising a separate logical file and multimedia data file 24 includes associated feature data with and the multimedia data file 24.

Although an example in which the multimedia data file 24 and the metadata file 26 are arranged in separate directories is shown, the multimedia data file 24 and the metadata file 26 may be arranged in the same directory or the metadata file 26 may be arranged directly in the root directory. I do not care.
Further, the multimedia data file 24 and the metadata file 26 may be divided according to the number of programs, or may be divided into a plurality of parts due to file capacity limitations.

  Further, the metadata file 26 in the present embodiment is not limited to the data format, and may be a text format or a binary format. In addition, encryption processing may be performed to prevent tampering by third parties and leakage of information.

  Describe information about whether or not the metadata or metadata file 26 exists in the management information file 23, or if the metadata or metadata file 26 is described, whether or not it is a valid value. In other words, it is possible to quickly determine whether or not the metadata or the metadata file 26 is valid by referring to the information described in the storage medium 1.

  FIG. 3 is a metadata structure diagram according to the first embodiment of the present invention. FIG. 3 shows the data structure of the metadata file 26 recorded in the hard disk drive 102 as the primary recording medium. Hereinafter, description will be made with reference to the hierarchical structure shown in FIG. As shown in FIG. 3A, the metadata 30 is located in the highest hierarchy of the data structure.

  Next, as shown in FIG. 3B, the metadata 30 includes metadata management information 31a that collectively describes information of the entire metadata, and N video objects (N is an integer of 1 or more). (Hereinafter referred to as VOB) metadata information search pointers 31b-1 to 31b-N and N pieces of VOB metadata information 31c-1 to 31c-N.

  The data recorded in the multimedia data file 24 of FIG. 2 is divided into one or more VOBs. Each VOB may correspond to one program, and each VOB may be a unit divided by a file capacity limit. The VOB metadata information 31b-n (any one of 31b-1 to 31b-N) is prepared corresponding to each VOB in the multimedia data file 24 as shown in FIG. 3B. Is done. That is, the nth VOBn has corresponding nth VOBn metadata information 31c-n. When a table indicating the correspondence between the number of the VOB and the number of the VOB metadata information 31c-n or an offset amount is prepared, it is not always necessary that the numbers coincide with each other, and one VOB is used. A plurality of VOB metadata information (such as 31c-n) may be prepared, and one VOB metadata information 31c-n may be related to or correspond to a plurality of VOBs. Note that there may be no VOB metadata information (corresponding to 31c-n) in a VOB that does not have associated metadata.

  The nth VOBn metadata information search pointer 31b-n shown in FIG. 3B describes start address information of the nth VOBn metadata information 31c-n. The total number N of VOB metadata information 31c-1 to 31c-N is described in the metadata management information 31a.

  Next, as shown in FIG. 3C, each of the VOB metadata information 31c-n includes metadata general information 32a and video shot map information 32b.

  In the metadata general information 32a, VOB content information corresponding to the upper layer VOB metadata information 31c-n, start address information of the corresponding video shot map information 32b, and the like are described. Here, the VOB content information described in the metadata general information 32a includes a program name, a producer name, a performer name, a description, a broadcast date and time of a recorded program, a channel, and the like.

As shown in FIG. 3D, details of the video shot map information 32b include video shot map general information 33a and M (M is an integer of 1 or more) video shot entries 33b-1 to 33b-M. Each video stream or audio stream recorded in the multimedia data file 24 shown in FIG. 2 is divided into a plurality of video shots along the playback time axis. M video shot entries 33b-1 to 33b-M are prepared corresponding to the total number M of video shots in the VOB to be referred to. That is, the mth video shot has a corresponding mth video shot m entry 33b-m.
When a table showing the correspondence between the number of video shots and the number of video shot entries 33b-m and an offset amount are prepared, it is not always necessary for both numbers to match, and one video shot A plurality of video shot entries (such as 33b-m) may be prepared, and one video shot entry 33b-m may include a plurality of video shots (corresponding to video shots). The total number M of the video shot entries 33b-1 to 33b-M is described in the video shot map general information 33a.

As shown in FIG. 3E, the video shot entry 33b-m includes video shot start time information 34a, video shot end time information 34b, and a video shot importance level 34c.
The video shot start time information 34a is playback start time (presentation time) or start frame position information of a video shot obtained by fragmenting a video stream or audio stream recorded in the multimedia data file 24 into a plurality of pieces on the playback time axis.
The video shot end time information 34b is playback end time (presentation time) or end frame position information of the video shot.
The video shot importance level 34c is a numerical value assigned to the video shot and indicating the importance level of the content.
The video shot end time information 34b may be omitted when the time interval of each video shot with respect to the video shot reproduction time information 34a can be acquired separately.

The importance attached to the video shot importance level 34c is a name for convenience, and (a) the content of the video shot is the importance of the content based on the subjective evaluation (for example, it becomes higher in the highlight scene). May be,
(B) It may be a value corresponding to the duration of the cheering of the audio corresponding to the video shot,
(C) It may be a value corresponding to the intensity of the motion of the video shot in the screen,
(D) It may be a numerical value based on a physical measurement value or index that does not depend on subjective evaluation.
Among the above, (a) and (b) are referred to as the importance based on the subjective evaluation, and (c) and (d) are referred to as the importance corresponding to the physical change amount. Here, the “physical change amount” refers to a physical index that can be directly obtained from video data and audio data such as a motion vector, color histogram, and sound volume in image coding. On the other hand, the degree of importance based on subjective evaluation refers to a feature amount that is a meaning of a physical index directly obtained from video data or audio data, such as the fun of video and audio (for example, the madness of audience).

  FIG. 4 is a conceptual diagram of data synchronization in the first embodiment of the present invention. Here, the basic concept of the video recording format applied to DVD-R and DVD-RW media is described as an example. However, the present invention is not limited to the video recording format. The present invention can be widely applied to a storage media format in which a media data file and a metadata file can be synchronized based on reproduction time information.

In FIG. 4, 40 is program chain information described in the management information file 23 and describes the playback order of multimedia data in the multimedia data file 24, 41 is a playback unit defined by the program chain information 40, and N (N is an integer greater than or equal to 1) programs (only two of them are shown), 42a and 42b are playback units defined by the program 41, and one or more cells ("cell 1", " Cells 2 "), 43a and 43b are described in the management information file 23 and are videos for describing the reference destination of the actual video data or audio data corresponding to the reproduction time information (presentation time) designated in the cell 42. Object (VOB) information ("VOB1 information", "VOB2 information)", 44a 44b is a time map table for offsetting the reproduction time information (presentation time) defined by the VOB information 43 and converting it into address information of actual video data or audio data. 45a and 45b are stored in the multimedia data file 24. Video object units (hereinafter referred to as VOBU) that are subdivided into the minimum units for the video / audio reproduction system to access by multiplexing the actual video data or audio data described in the packet structure together with the reproduction time information. In the illustrated example, the time map table 44a for VOB1 includes VOBU1 to VOBUP, and the time table 44b for VOB2 includes VBU1 to VOBUQ.
Video shot 1 entry to video shot R entry and video shot entry 1 to video shot S entry indicated by reference numerals 33b-1 to 33b-R and 33b-1 to 33b-S are VOB1 metadata 31c- of FIG. 1 and part of the VOB2 metadata 31c-1, and corresponds to the video shot 1 entry 33b-1 to the video shot M entry 33b-M in FIG.

  FIG. 5 shows an example of the importance map and the encoding rate map generated by the metadata analysis unit 121 according to Embodiment 1 of the present invention. FIG. 5A shows an example of an importance map showing the transition of the importance level with respect to the playback time for a video object. FIG. 5B shows an example of a coding rate map showing a transition of a coding rate reference value set at the time of dubbing (re-encoding) with respect to a playback time for a video object.

  In the importance map (FIG. 5A), the horizontal axis 51 indicates the reproduction time, and the vertical axis 50 indicates the importance level. As shown in the figure, the importance level graph 52 continuously changes in a predetermined range (for example, [0, 1] or [0, 100]). The importance degree map upper limit value 53 indicates the upper limit value of the predetermined range, and the importance degree map lower limit value 54 indicates the lower limit value of the predetermined range. That is, the importance level graph 52 changes within the range from the importance map upper limit 53 to the importance map lower limit 54.

  The unit of time shown on the horizontal axis 51 is based on the values shown in the video shot start time information 34a and the video shot end time information 34b in FIG. 3, and the importance level is the value of the video shot importance level 34c. It shall be based on In other words, the graph of FIG. 5 shows one video shot (scene) from the video shot start time represented by the video shot start time information 34a to the video shot end time represented by the video shot end time information 34b. And one importance level (represented by the video shot importance level 34c) is plotted.

  A place where the importance level is set to a high value means a highlight scene (video shot) in the content based on a certain standard, and it can be said that the viewing value is high for the user.

  In the encoding rate map (FIG. 5B), the horizontal axis 56 indicates the reproduction time, and the vertical axis 55 indicates the re-encoding rate setting reference value during dubbing. As shown in the figure, the encoding rate setting graph 57 continuously changes between the encoding rate conversion upper limit value 58 and the encoding rate conversion lower limit value 59. Data representing the encoding rate conversion upper limit value 58 and the encoding rate conversion lower limit value 59 is supplied from the system control unit 101 to the metadata analysis unit 121 in advance and stored in the metadata analysis unit 121.

  As described above, when determining the encoding rate suitable for the video scene from the importance level, by efficiently allocating the bit rate for each scene within the upper limit / lower limit value set by the video encoding device, A multimedia data file recorded within a specified encoding rate can be generated. Further, by manipulating the upper limit value and the lower limit value, it is possible to control the influence of the average coding rate and importance level of the multimedia data file.

  At locations where the re-encoding rate is set to a high value, a high-definition video that is less prone to block noise and image distortion can be recorded by assigning a high encoding rate during re-encoding dubbing. .

Hereinafter, the operation of the moving image encoding apparatus 100 will be described.
First, an outline of a general recording process (a process similar to normal recording) in a video recording format or the like will be described with reference to FIG. 1, FIG. 2, FIG. 3, and FIG. In the first embodiment, the case of recording on the hard disk drive 102 as a dubbing source recording medium (primary recording medium) is described. Of course, the dubbing source recording medium (primary recording medium) is the optical disc 103. It does not matter.

  An input video signal 110 input from the outside is encoded by a video encoder 112 into an encoding / compression method such as MPEG-2 to generate a video stream. Similarly, the input audio signal 111 is encoded by the audio encoder 113 by an encoding compression method such as AC-3 to generate an audio stream. These video stream and audio stream are multiplexed by the multiplexer 114, and a multimedia data file 24 is generated. Thereafter, the multimedia data file 24 is sequentially written in the buffer memory 115 and recorded in the hard disk drive 102 under the instruction from the system control unit 101.

  The multimedia data file 24 is recorded in the directory structure shown in FIG. When recording the multimedia data file 24, attribute information and playback time information of the multimedia data file 24 are recorded in the information management file 23. Thereafter, a file is generated or data is updated so that the same information as the information management file 23 can be held in the backup file 25.

  Next, the recording process in the moving picture coding apparatus 100 according to the first embodiment will be described in more detail with reference to FIGS. Here, recording to the hard disk drive 102 will be described, but of course, recording to the optical disk 103 may be performed.

When dubbing according to the present embodiment is performed, a program is first recorded on the hard disk drive 102 which is a primary recording medium. When recording,
When the video encoder 112 encodes and compresses the input video signal 110 to MPEG-2 or the like, the video feature extraction unit 142 uses a motion vector amount between frames, a change amount of a color histogram, a person using an image recognition technique, Extraction of features such as detection of a face or the like is performed.
Similarly, when the audio encoder 113 encodes and compresses the input audio signal 111 to AC-3 or the like, the audio feature extraction unit 143 changes the coefficient value or frequency information after digital sampling, changes in audio level, or audio recognition. Features are extracted, such as speaker changes and applause scene detection using techniques.

The feature amount extracted by the video feature extraction unit 142 or the audio feature extraction unit 143 is supplied to the metadata generation unit 120 and analyzed, and the metadata generation unit 120 calculates the video shot importance level 34c.
For example, as a feature amount obtained by the speech feature extraction unit 143, when a speech signal identified as applause or cheer by a speech recognition technique continues for a long time, a numerical value corresponding to the duration is assigned as the importance. In addition, as a feature amount obtained by the video feature extraction unit 142, when there is a portion having a large motion vector amount, that is, a video signal with intense motion, a numerical value corresponding to the degree of the motion vector amount is assigned as an importance level. The importance is the video shot importance level 34c (FIG. 3E), and the reproduction time information of the portion where the feature amount is observed is the video shot start time information 34a and the video shot end time information 34b (FIG. 3E). )).
The above processing is repeated to generate a plurality of video shot entries 33b-1 to 33b-M to form metadata 30, which is used as a metadata file 26 via the buffer memory 115 and the metadata directory 22 of the hard disk drive 102. For example, writing is performed so as to be arranged at a predetermined logical position.

The generation of the metadata file 26 does not necessarily have to be simultaneously processed in real time for the recording of the multimedia data file 24, and at least the video shot importance level 34c is stored in the memory area of the metadata generation unit 120 or the system control unit 101. Alternatively, a method may be used in which necessary data including is stored, the metadata file 26 is generated later, and recorded in the hard disk drive 102.
Further, necessary data including the video shot importance level 34c is once recorded in the hard disk drive 102, and later read out (before dubbing described later) from the hard disk drive 102 to generate the metadata file 26, and the generated metadata The file 26 may be written in the hard disk drive 102.

  Further, the metadata file 26 according to the first embodiment is not limited to the data format, and may be a text format or a binary format. In addition, encryption processing may be performed to prevent tampering by third parties and leakage of information.

  Next, a dubbing process in the moving picture coding apparatus 100 according to the first embodiment will be described with reference to FIGS. Here, the processing when re-encoding dubbing the VOB in the multimedia data file 24 recorded on the hard disk drive 102 to the optical disk 103 will be described, but of course the dubbing from the optical disk 103 to the hard disk drive 102 may be performed. .

First, an outline of general re-encoding dubbing processing (processing similar to normal re-encoding) will be described with reference to FIGS. 1 and 2.
When re-encoding dubbing of a desired VOB (program) recorded in the hard disk drive 102 to the optical disk 103, first, the program recorded as the multimedia data file 24 in the hard disk drive 102 is demultiplexed by the demultiplexer 130 and the video decoder. 131, the audio decoder 133 decodes. Thereafter, the decoded output video signal 132 and output audio signal 134 are encoded by the video encoder 112 and the audio encoder 113, multiplexed by the multiplexer 114, passed through the buffer memory 115 and the recording / reproducing drive 104, The media data file 24 is written on the optical disc 103.

  Next, the dubbing process of the first embodiment will be described in detail with reference to FIGS. First, since the program 41 constituting the program in the hard disk drive 102 and the cell 42 constituting the program 41 can be known from the program chain information 40, each of the number of the VOB to be referred to and the reproduction start time and the reproduction end time of the cell are referred to. Presentation time (PTM) is fixed.

  The moving image encoding apparatus 100 reads out and buffers the metadata file 26 recorded in the hard disk drive 102 at any timing before the viewer instructs the dubbing process or after selecting a desired program as a dubbing target. By importing into the metadata analysis unit 121 via the memory 115, the data structure described in the metadata 30 can be referred to as appropriate.

  Here, an example in which the metadata file 26 and the multimedia data file 24 are configured as independent logical files has been described. For example, the metadata 30 is described in the data structure of the information management file 23, or the multimedia file 26 The data file 24 may be multiplexed and described.

  The metadata file 26 is composed of a logical file independent of the multimedia data file 24, so that it is not necessary to read all the multimedia data file 24, and only the metadata file 26 is read and analyzed. The scene part can be detected quickly.

  The metadata analysis unit 121 obtains the video shot start time information 34a, the video shot end time information 34b, and the video shot importance level 34c for each video shot for the program to be dubbed from the data structure described in the metadata 30. read out. Since the importance level in each unit time can be acquired from these pieces of information, the importance map shown in FIG. 5A can be generated.

  Thereafter, the metadata analysis unit 121 generates an encoded rate map (FIG. 5B), which is a reference value for rate setting at the time of dubbing, from the obtained importance map (FIG. 5A). The encoding rate map (FIG. 5B) shows a reference value for recording rate setting when re-encoding by dubbing, and a scene with a high importance level in the importance map (FIG. 5A). In this case, a high encoding rate value is set in the encoding rate map (FIG. 5B).

  When generating the encoding rate map (FIG. 5B) from the importance map (FIG. 5A), numerical conversion is required. For example, the importance level is set in a range of [0.0 to 1.0], while the encoding rate setting value is generally set in a range of [3 Mbps to 8 Mbps]. Since the dimension and setting range of the numerical values are different, the metadata analysis unit 121 encodes an encoding rate map that reflects the importance level within the range of the encoding rate upper limit value 58 to the encoding rate lower limit value 59 (FIG. 5B). )) Must be designed.

  In the moving image encoding apparatus 100, the description will be given assuming that the encoding rate conversion upper limit value 58 and the encoding rate conversion lower limit value 59 are set in advance, but these setting values are within a range that can be encoded by the video encoder 112. In this, the user may determine manually or the moving picture encoding apparatus 100 may determine automatically.

  When determining the value of the encoding rate setting graph 57, the range from the importance upper limit 53 to the importance lower limit 54 defined in the importance map (FIG. 5A) is set to the encoding rate conversion upper limit. The value is determined by converting the scale from the value 58 to the encoding rate conversion lower limit 59.

  For example, in FIG. 5A, it is assumed that the importance upper limit 53 is set to a value of 1.0, and the importance lower limit 54 is set to a value of 0.0. Further, it is assumed that 8.0 Mbps is set in the encoding rate conversion upper limit value 58 of the encoding rate map (FIG. 5B) and 3.0 Mbps is set in the encoding rate conversion lower limit value 59. In this case, the scale is changed in such a manner that the range of importance level 0.0 to 1.0 is projected onto the range of 3.0 Mbps to 8.0 Mbps in the coding rate map. That is, the recording rate setting value (X [i]) in video shot [i] is “importance level of video shot [i] × (encoding rate upper limit value 58−encoding rate lower limit value 59) / (importance level). For example, in the case of a scene with an importance level of 0.8, the encoding rate set value is converted to 7.0 Mbps. Will be.

  FIG. 5 shows a case where the range of the encoding rate conversion upper limit value 58 and the encoding rate conversion lower limit value 59 in the encoding rate map ((b) in the same figure) is set narrow, and the lower limit value is particularly greatly increased. When the encoding rate conversion upper limit value 58 and the encoding rate conversion lower limit value 59 are set as shown in the figure, the encoding rate setting graph 57 becomes a curve having setting values compressed in the vertical direction as shown in the figure. Since the encoding rate conversion lower limit 59 is greatly increased, the encoding rate setting graph 57 transitions at a high bit rate.

  In this way, by changing the setting values of the encoding rate conversion upper limit value 58 and the encoding rate conversion lower limit value 59, the influence level of the importance level and the average value of the encoding rate can be manipulated at the time of encoding rate design. can do. As a result, when the importance level as shown in FIG. 5 is compressed in the vertical direction, even if the importance level changes greatly in dynamic, an encoding rate value that is less affected by the importance level is obtained. Can be set. On the other hand, when the importance level is extended in the vertical direction, even if the width of change after the importance level is small, it can be dynamically reflected in the coding rate.

  When creating the coding rate map (FIG. 5B), information indicating the presence / absence of viewing value may be generated based on the physical change amount described above, and the rate design may be performed based on this information. The coding rate may be designed in combination with the physical change amount of the video (such as a motion vector or a color histogram). With such a configuration, it is possible to perform efficient encoding in consideration of the influence of the image encoding difficulty level.

  In this way, the metadata analysis unit 121 can create an encoding rate map (FIG. 5B). The encoding rate map generated here (FIG. 5B) is stored in the recording rate map holding unit 122.

  For the dubbing target VOB, the actual dubbing process is started when the encoding rate map (FIG. 5B) is determined. The VOB held in the multimedia data file 24 read from the hard disk drive 102 is supplied to the demultiplexer 130 via the buffer memory 115. The demultiplexer 130 separates the multimedia data into a video stream and an audio stream and supplies them to the video decoder 131 and the audio decoder 133, respectively, and decodes the encoded video stream and audio stream.

  The decoded output video signal 132 and output audio signal 134 are supplied to the video encoder 112 and the audio encoder 113. At that time, the recording rate changing unit 123 uses the encoding rate reference value for each video scene from the encoding rate map (FIG. 5B) stored in the recording rate map storage unit 122. The code amount of the encoder 112 and the audio encoder 113 is set. Therefore, it is possible to generate a video stream and an audio stream in which code amounts are appropriately allocated according to the semantic importance of the scene. Thereafter, as in normal recording, the data is multiplexed by the multiplexer 114 and recorded as the multimedia data file 24 on the optical disk 103 via the buffer memory 115 and the recording / reproducing drive 104. When writing the multimedia data file 24 to the optical disk 103, the metadata file 26 held in the buffer memory 115 may be written together.

Embodiment 2. FIG.
FIG. 6 is an explanatory diagram of an example of the coding rate map correction shown in the second embodiment. In FIG. 6, reference numeral 60 denotes an encoding rate setting graph (before correction) that holds encoding rate setting information before correction of the encoding rate, and 61 denotes an average encoding rate of the entire encoding rate setting graph (before correction) 60. The average coding rate (before correction) is shown. Reference numeral 62 denotes a coding rate setting graph (after correction) that holds the coding rate setting information after correction of the coding rate, and 63 denotes a target average coding rate that indicates the average coding rate after the coding rate is corrected. Show.

  FIG. 7 is an explanatory diagram of another example of the coding rate map correction in the second embodiment. In FIG. 7, reference numeral 70 denotes an encoding rate setting graph (before correction) that holds encoding rate setting information before correction of the encoding rate, and 71 denotes an average encoding rate of the entire encoding rate setting graph (before correction) 70. The average coding rate shown (before correction) is shown. Reference numeral 72 denotes an encoding rate setting graph (after correction) that holds the encoding rate setting information after correction of the encoding rate, 73 denotes a target average encoding rate that indicates the average encoding rate after the encoding rate correction, Reference numeral 74 denotes an encoding rate upper limit value indicating an upper limit value of a recording rate that can be set by the video encoder 112.

  6 and 7, the same reference numerals are given to the same configurations as those described in the first embodiment. In the second embodiment, a method for re-correcting the information of the coding rate map according to the capacity of the dubbing destination recording medium will be described. Other processes are the same as those in the first embodiment.

  An example (FIG. 6) of correcting the recording rate of the coding rate map in the second embodiment will be described in detail with reference to FIG. 1, FIG. 2, FIG. 3, FIG. 4, and FIG. The second embodiment is characterized in that the encoding rate map is re-corrected according to the dubbing destination free capacity.

  Normally, when dubbing to a recording medium having a fixed capacity such as an optical disk, it is necessary to encode at an appropriate average rate with respect to the free capacity of the dubbing destination recording medium. This is because a multimedia data file encoded at a recording rate that exceeds the free capacity of the dubbing destination cannot be finally recorded in the free capacity of the dubbing destination recording medium. On the other hand, if the recording rate is set too low, the recording medium cannot be used effectively and recording with high image quality and high sound quality cannot be performed.

  Before starting dubbing, it is assumed that the recording rate map holding unit 122 holds a coding rate map (FIG. 6) regarding the VOB held in the dubbing target multimedia data file 24 in advance. At the time of dubbing, the system control unit 101 acquires the free capacity of the optical disc 103 that is a dubbing destination recording medium. Then, a target average encoding rate 63 capable of allocating the largest bit rate is determined from the recording time of the dubbing target VOB in the multimedia data file 24 and the size that fits in the free capacity of the optical disc 103 from the dubbing destination free capacity. .

A method of calculating the target average coding rate 63 will be specifically described with reference to FIG. The explanation is based on the assumption that the average encoding rate (before correction) 61 is 6.0 Mbps, the dubbing destination recording medium free space is 4.5 GB (gigabytes), and the playback time of the dubbing target VOB in the multimedia data file 24 is 2 hours. Proceed. The target average encoding rate 63 can be calculated by dividing the free capacity of the dubbing destination recording medium by the recording time. That is, the target average encoding rate 63 is calculated as (4.5 G × 8 bytes) / (2 hours × 60 minutes × 60 seconds) = 5.0 Mbps.
This calculation is performed by the system control unit 101, and the target average encoding rate 63 obtained as a result of the calculation is transmitted from the system control unit 101 to the recording rate changing unit 123.

  If encoding is performed with the recording rate set at the average encoding rate 61 being exceeded, the capacity of the optical disc 102 as the dubbing destination recording medium is exceeded, so the average rate of the encoding rate setting curve 60 is the target average encoding rate. It is necessary to pull down the whole so that it becomes equal to 63. The encoding rate setting curve 60 is modified like the encoding rate setting curve 62 by shifting the whole downward. As described above, the whole may be shifted equally, or the rate may be designed while weighting the rate of change according to the importance. This correction is performed by the recording rate changing means 123. That is, the recording rate changing unit 123 corrects the recording rate held in the recording rate map holding unit 122 using the target average encoding rate 63 supplied from the system control unit 101.

  In this example, the method of automatically resetting the average rate according to the free space at the dubbing destination has been described, but the target average encoding rate 63 may be determined manually.

  Next, another example (FIG. 7) of correcting the recording rate of the coding rate map in the second embodiment will be described in detail with reference to FIG. 1, FIG. 2, FIG. 3, FIG. This example describes a case where the target average encoding rate is raised when the dubbing destination free capacity is sufficiently large. As a result, it is possible to perform encoding using the disk recording capacity efficiently with higher quality moving images.

  As described in the example of the recording rate correction of the encoding rate map in the second embodiment (FIG. 6), the encoding rate setting graph (before correction) 70 is displayed by the operation of the system control unit 101 and the recording rate changing unit 123. According to the target average encoding rate 73, the encoding rate setting graph (after correction) 72 can be corrected. However, when the encoding rate setting is corrected as described above, the encoding rate of a certain video scene may exceed the encoding rate upper limit 74. In that case, it is necessary to adjust the encoding rate of the video scene that exceeds the encoding rate upper limit value 74 so as to be within the encoding rate upper limit value 74. Therefore, in this example, the process of correcting the encoding rate curve 72 is performed again so that the corrected encoding rate value does not exceed the encoding rate upper limit 74.

  With respect to the encoding rate setting graph (after correction) 72, in a video scene that exceeds the encoding rate upper limit 74, it is reset to a value equivalent to the encoding rate upper limit 74 as shown in FIG. The code amount that has become excessive by this operation is redistributed to other video scenes that have not been corrected, and the overall code amount is adjusted so as not to change. The recording rate changing means 123 also performs correction including such limitation by the upper limit value 74. Therefore, data representing the upper limit value 74 is supplied from the system control unit 101 and stored in the recording rate changing unit 123.

  In this example, the case where the encoding rate upper limit 74 exists is described, but the present invention may also be applied to the case where there is a recording rate lower limit.

  With this configuration, the encoding rate can be controlled within a range in which the video encoder 112 can perform encoding. In the second embodiment, the encoding rate upper limit value 74 is determined due to the restriction of the hardware characteristics of the video encoder 112. However, the encoding rate upper limit value 74 is determined based on the restriction of the application standard to be recorded on the optical disc 103. It may be determined.

  Also, since the encoding rate reference value at the time of re-encoding calculated from the importance can be re-corrected according to the capacity of the recording medium of the dubbing destination, efficient code amount control according to the capacity of the dubbing destination medium Dubbing can be performed.

Embodiment 3 FIG.
FIG. 8 shows a conceptual diagram of summary dubbing in the third embodiment. In the figure, 80 is a threshold value that is a value for evaluating the importance level graph 81, 82 is an importance level graph after correcting the importance level of a scene below a specific value based on the threshold 80, 83 is A threshold 80 indicates a dubbing target section and a skip target section of the video scene. Other than the above, the configuration is the same as that described in the first embodiment, and the description thereof is omitted here.

  The third embodiment is characterized in that the dubbing execution section is determined according to the importance map shown in FIG. Specifically, when a value equal to or less than a certain threshold is set in the importance map, the dubbing of the section is not performed, and the jump to the start point of the next dubbing section is continued and the dubbing is continued.

  A dubbing control sequence of the moving picture coding apparatus 100 according to Embodiment 3 will be described with reference to FIG. When the video shot importance level 34c of the video shot related to the VOB to be dubbed becomes the importance level graph 81, the video shots below the threshold 80 are in the section from a1 to a2 and in the section from b1 to b2. The metadata analysis unit 121 or the system control unit 101 performs redesign as in the importance level graph 82 in which the importance level is reset to a specific value (for example, 0 is shown in FIG. 8) in the section. Thereafter, when dubbing is performed, the section where the importance level shows a specific value (for example, 0) is not performed, but jumps to the start point of the next section to be dubbed and continues dubbing. Note that the dubbing process is performed in the same manner as in the first embodiment. When the importance level is a specific value (for example, 0), data indicating that the encoding rate is zero is output from the recording rate changing unit 123. As a result, the video decoder 112 and the audio decoder 113 perform encoding. As a result, dubbing is skipped.

  By configuring so that the dubbing process is skipped in this way, it is possible to generate a multimedia data file excluding a portion with low viewing value at the time of re-encoding dubbing. For example, when the importance level graph is created so that the importance level is set to be low in a portion that is not related to the main program such as a CM (commercial message) section in television broadcasting, the importance level By referring to the level, it is possible to selectively dub only the main part of the program without the CM section.

  The above embodiment relates to a method and an apparatus for performing dubbing. However, it is read from one area of one recording medium, decoded and encoded, and transferred to another area on the same recording medium. The present invention can also be applied when recording, or when recording (overwriting) in the same area of the same recording medium.

In the above embodiment, the code amount is allocated based on both the video characteristics and the audio characteristics. By doing so, the degree of importance of the content of the video, Efficient encoding can be performed in consideration of the effect of image encoding difficulty, but instead of doing this, only extraction of audio features is performed, and code amount is allocated based on this. You may do it.
If the configuration is such that the amount of code is allocated only by the features of the speech, an index for determining the coding rate at the time of re-encoding can be generated from the data representing the features of the speech with a relatively small amount of information, so the circuit scale is large. Even if it is small, re-encoding reflecting the feature information can be performed.
Furthermore, in the above embodiment, the feature amount is extracted for each predetermined time segment such as a scene or video shot made up of one or more frames, but a segment (data amount made up of a predetermined amount of data). The feature amount may be extracted for each of the categories defined by (1).

BRIEF DESCRIPTION OF THE DRAWINGS It is a system configuration block diagram showing Embodiment 1 of the present invention. It is a file block diagram which shows Embodiment 1 of this invention. It is a metadata block diagram which shows Embodiment 1 of this invention. It is a data synchronization conceptual diagram which shows Embodiment 1 of this invention. It is the importance map obtained from the metadata which shows Embodiment 1 of this invention, and a re-encoding rate map. It is explanatory drawing of an example of the encoding rate map correction shown in Embodiment 2 of this invention. It is explanatory drawing of another example of the encoding rate map correction which shows Embodiment 2 of this invention. It is a summary dubbing conceptual diagram which shows Embodiment 3 of this invention.

Explanation of symbols

  20 root directory, 21 multimedia directory, 22 metadata directory, 23 information management file, 24 multimedia data file, 25 backup file, 26 metadata file, 30 metadata, 31a metadata management information, 31b-1 to 31b- N VOB metadata information search pointer, 31c-1 to 31c-N VOB metadata information, 32a metadata general information, 32b video shot map information, 33a video shot map general information, 33b-1 to 33b-M video shot entry, 34a Video shot start time information, 34b Video shot end time information, 34c Video shot importance level, 40 Program chain information, 41 Program , 42 cells, 43 video object information, 44 time map table, 45 video data and audio data, 50 importance map vertical axis, 51 importance map horizontal axis, 52 importance level graph, 53 importance map upper limit, 54 Importance map lower limit value, 55 Coding rate map vertical axis, 56 Coding rate map horizontal axis, 57 Coding rate setting graph, 58 Coding rate conversion upper limit value, 59 Coding rate conversion lower limit value, 60 Coding rate setting Graph (before correction), 61 average encoding rate (before correction), 62 encoding rate setting graph (after correction), 63 target average encoding rate, 70 encoding rate setting graph (before correction), 71 average encoding rate (Before correction), 72 Encoding rate setting graph (After correction), 73 Target Average coding rate, 74 Coding rate upper limit value, 80 Importance level graph, 81 Threshold value, 82 Importance level graph, 83 Dubbing control sequence diagram, 100 Moving picture coding device, 101 System control unit, 102 Hard disk drive, 103 Optical disk, 104 recording / reproducing drive, 110 input video signal, 111 input audio signal, 112 video encoder, 113 audio encoder, 114 multiplexer, 115 buffer memory, 120 metadata generation unit, 121 metadata analysis unit, 122 recording rate map holding unit 123 recording rate changing means, 130 demultiplexer, 131 video decoder, 132 output video signal, 133 audio decoder, 134 output audio signal, 140 monitor, 142 video feature extractor, 143 audio feature extractor.

Claims (12)

Audio feature extraction means for dividing a video signal and an audio signal into predetermined intervals, and extracting audio features of the audio signals in each interval;
Coding rate determining means for determining a coding rate based on the characteristics of the speech in each section;
A moving image encoding apparatus comprising: means for encoding the video signal and the audio signal at a determined encoding rate. The encoding rate determining means includes
Feature data representing the voice characteristics of each section and feature data generating means for generating the start position of each section;
Feature data analysis means for creating a coding rate map based on the feature data;
The moving image coding apparatus according to claim 1, further comprising coding rate generation means for generating and outputting data indicating a coding rate of each section based on the coding rate map.   3. The moving image encoding apparatus according to claim 2, wherein the feature data generation unit generates and holds feature information indicating presence / absence of viewing value based on a physical change amount in the audio signal.   The feature data analysis means sets a value within a range between a predetermined upper limit value and a lower limit value as an encoding rate when generating an encoding rate map from the feature data. 2. The moving image encoding apparatus according to 2. Means for recording the encoded video and audio signals on a recording medium;
The encoding rate generation means includes
The data indicating the coding rate is generated not only based on the coding rate map but also based on the free space of the recording medium used for recording the coded video signal and audio signal. The moving image encoding device according to 1. Video feature extraction means for extracting video features represented by the video signal of each section;
The encoding parameter determining means includes
The moving image encoding apparatus according to claim 1, wherein an encoding rate is determined based on not only the audio feature of each section but also the video feature.   7. The moving picture encoding apparatus according to claim 6, wherein the encoding parameter determination unit extracts the video feature based on a physical change amount of the video signal. Means for recording the encoded video and audio signals on a recording medium;
The code rate determining means includes
2. The moving picture encoding apparatus according to claim 1, wherein the encoding rate is determined in accordance with a free space of the recording medium used for recording the encoded video signal and audio signal. Means for recording the encoded video and audio signals on a recording medium;
When the feature amount representing the feature generated by the feature data generation unit has a value equal to or less than a preset threshold value, the recording of the video signal and the audio signal in the section to the recording medium is skipped. The moving picture encoding apparatus according to claim 1. An audio feature extraction step of dividing the video signal and the audio signal into predetermined intervals, and extracting an audio feature represented by the audio signal of each interval;
An encoding rate determining step for determining an encoding rate based on a feature of speech represented by the speech signal of each section;
A video encoding method comprising: encoding the video signal and the audio signal at a determined encoding rate. A first recording step of encoding a video signal and an audio signal supplied from the outside and recording them in a recording means;
An audio feature extraction step of dividing the video signal and the audio signal recorded in the recording means into predetermined intervals, and extracting audio features of the audio signal in each interval;
An encoding rate determining step for determining an encoding rate based on a feature of speech represented by the speech signal of each section;
A reproduction step of reading out and decoding the video signal and the audio signal from the recording means;
A recording / reproducing method comprising: a second recording step of encoding the decoded video signal and audio signal at the encoding rate determined in the encoding rate determining step and recording the encoded signal on the recording means. The recording means includes a first recording medium and a second recording medium;
In the first recording step, recording is performed on the first recording medium,
The recording / reproducing method according to claim 11, wherein recording is performed on the second recording medium in the second recording step.

Images