JP2011091859A - Video recording apparatus, video recording method, video and audio recording apparatus, and video and audio recording method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video recording apparatus, video recording method, video and audio recording apparatus, and video and audio recording method in which scene changes in video images can be extracted, in parallel with the recording of video images. <P>SOLUTION: The video recording apparatus includes: an encoder 101; an HDD 105; a stream control section 103 for controlling transfer of an encoded stream to the HDD 105; a scene change extracting section 110 for extracting scene changes in video data for each encoded unit of the encoded stream for the encoded stream controlled by the stream control section 103; and a recording control section 102 for acquiring an encoding completion notice for each encoded unit of the encoded stream from the encoder 101, and outputting a scene change extraction instruction to the scene change extraction section 101, when encoding completion notice is acquired. The scene change extracting section 110 is provided with a decoding unit. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a video recording apparatus capable of detecting a video scene change in parallel with video recording, and a scene change extraction method implemented by the video recording apparatus. Further, the present invention is a video / audio recording capable of detecting a commercial message broadcast sandwiched between the main parts of a recorded television broadcast program in parallel with the recording of the television broadcast program. It relates to the device.

  Conventionally, in a video recording apparatus that encodes and records a video, in order to search for a desired video scene from the recorded video, a corresponding scene is searched for by the human eye using special playback such as fast-forward or rewind. And had to spend a great deal of time and effort.

  In order to solve such a problem, a method for finding a scene change scene (hereinafter referred to as “scene change”) of a video has been proposed (for example, see Patent Documents 1 and 2). In Patent Document 1, in the pre-video signal processing apparatus in the preceding stage of the encoder that performs the encoding, the absolute value of the difference between the histograms of two videos that are temporally mixed is calculated, and the calculated absolute value has a predetermined threshold value. A method for determining that a scene change has occurred is shown. Japanese Patent Application Laid-Open No. 2004-228688 discloses a method for reading out video data already stored in a storage medium such as an HDD and extracting a scene change.

  Also, television broadcasts can be classified into “main program broadcast” and “commercial message broadcast” based on the contents. In the case of a movie program, the main program broadcast is a broadcast part based on video / audio data of the content of the movie itself. On the other hand, commercial message broadcasting is a broadcast part based on video and audio data provided by a client who has a contract with a broadcaster, such as an advertiser or an advertising agency, and is broadcast between the main program broadcasts. It is what is done. The commercial message broadcast also includes a broadcast for a broadcasting station to advertise itself. In the present application, “program main broadcast” is also referred to as “program main broadcast”, and “commercial message broadcast” is also referred to as “CM broadcast”. The relationship between the main part of the program and the CM broadcast is often sparse, and for viewers who want to concentrate on watching the main part of the program, the CM broadcast is efficiently deleted when recording or viewing the television broadcast. There is a demand to do or want to overlook.

  The conventional video / audio recording apparatus detects the audio system of television broadcasting, and identifies the main program and the CM broadcast based on the detected audio system. The audio system of the main part of the program is often monaural audio or bilingual audio (Japanese and English in a Western movie, etc.). In general, the audio system of CM broadcasting is often stereo audio. By utilizing such a difference, a section in which stereo audio is detected is determined as a CM broadcast section, and the CM broadcast section is not recorded (that is, deleted) at the time of recording, or the CM broadcast section is set at the time of viewing. The process of skipping (that is, skipping) was performed.

  The above-described conventional CM detection method does not function when the audio system of the main program is the same stereo sound as the audio system of the CM broadcast. As an improvement measure, there is a proposal for specifying a CM broadcast section by detecting a full-screen black frame image called “black frame” inserted at the boundary between the main program and the CM broadcast (see, for example, Patent Document 3). .) There is also a proposal for detecting a silent section that occurs at the boundary between the main program and the CM broadcast, and specifying the CM broadcast section from the appearance interval of the detected silent section (see, for example, Patent Document 4). There is also a proposal for extracting CM broadcasts using the frequency of occurrence of scene changes (see Patent Document 5). In addition, there is also a proposal for providing a user interface that skips (ie, skips) a fixed time such as 15 seconds by the user's button operation during viewing.

JP 2004-282318 A (Page 7, FIG. 2) JP 2002-064823 A (5th page, FIG. 1) JP-T 8-507633 (pages 12-15, FIG. 1) Japanese Patent Laying-Open No. 2003-47031 (page 5-8, FIG. 1) Japanese Patent Laying-Open No. 2000-16596 (page 7, FIG. 14)

  However, since the method disclosed in Patent Document 1 is a method of detecting a scene change by a pre-video signal processing device provided in the previous stage of the encoder, an encoder chip set in which the configuration from the video encoder to the encoder is integrated. However, it could not be applied to a video recording apparatus equipped with a scene change extraction result that is not output to the outside. Even when the encoder chip set outputs a scene change extraction result, the scene change extraction result changes every time the type of encoder chip set to be used changes. There was a problem that adjustments were required.

  The method disclosed in Patent Document 2 is a method for detecting a scene change from a video stream that has already been compressed and recorded on an HDD. Therefore, after the recording process is completed, a long time-consuming process is performed offline. It was necessary and not practical.

  Further, in the method described in Patent Document 3 for detecting a black frame and specifying a CM broadcast section, a black frame may not be inserted depending on the country, region, or broadcasting station (in fact, domestic In broadcasting, the insertion of black frames is not required by law), and there is a problem that the accuracy of automatic detection of CM broadcast sections cannot be made sufficiently high.

  Further, the method described in Patent Document 4 or 5 has a problem that it is difficult to specify the CM broadcast section with high accuracy because there are many silent sections and scene changes in the main program.

  Furthermore, the method of skipping 15 seconds by the user's button operation cannot cope with CM broadcasting having a non-default time length (for example, a program spot advertisement of 5 seconds) that is not in a multiple of 15 seconds. There is.

  Accordingly, the present invention has been made to solve the above-described problems of the prior art, and an object of the present invention is to provide a video recording apparatus capable of extracting a scene change of a video in parallel with video recording, and the video recording To provide a scene change extraction method implemented in an apparatus. Another object of the present invention is to provide a video / audio recording apparatus capable of extracting a CM broadcast section in parallel with the recording of a television broadcast program.

  The video recording apparatus of the present invention includes an encoding unit that encodes video data and outputs an encoded stream, a stream control unit that controls transfer of the encoded stream to the recording unit, and a code controlled by the stream control unit A video change point extracting means for extracting a scene change of the video data for each coding unit of the coded stream, and a coding completion notification for each coding unit of the coded stream from the coding means. And a recording control unit that outputs a scene change extraction instruction to the video change point extraction unit when the encoding completion notification is acquired. The video change point extraction unit decodes the encoded stream. It has a decoding means.

  The video / audio recording apparatus of the present invention includes a recording unit that records an encoded stream of video and audio, a stream holding unit that temporarily stores the encoded stream before recording the encoded stream in the recording unit, Video change point detection means for detecting a scene change from the video data of the encoded stream held in the stream holding means, silence detection means for detecting silence from the audio signal, and detected by the video change point detection means Temporary storage means for temporarily holding scene change information and silence information detected by the silence detection means, and a transition in which scene change and silence occur simultaneously from the scene change information and silence information held in the temporary storage means Transition point detection means for detecting points, and intervals between transition points detected by the transition point detection means are preset. CM broadcast section determination means for determining a section in which two or more transition point combinations that fall within a predetermined time range centered on the specified reference time continue as a CM broadcast section, and the CM broadcast section determination means And CM broadcast section information recording means for recording the time information of the determined CM broadcast section.

  According to the video recording apparatus and method of the present invention, the scene change extraction process is performed every time the video is encoded. Therefore, there is an effect that the scene change can be extracted in real time.

  According to the video / audio recording apparatus and method of the present invention, the CM broadcast section detection process is executed in parallel with the recording of the broadcast program, so that the CM broadcast section can be detected in real time.

It is a block diagram which shows the structure of the video recording device of Embodiment 1 of this invention. 3 is a block diagram illustrating a configuration of a scene change extraction unit of the video recording apparatus according to Embodiment 1. FIG. It is a block diagram which shows the structure of the scene change command control part of the video recording device of Embodiment 2 of this invention. 6 is a flowchart illustrating an operation of the video recording apparatus according to the second embodiment. 10 is a flowchart illustrating an operation of an API unit of a scene change command control unit of the video recording apparatus according to the second embodiment. 10 is a flowchart illustrating an operation of a state control unit of a scene change command control unit of the video recording apparatus according to the second embodiment. It is a block diagram which shows the structure of the decoding part of the video recording device of Embodiment 3 of this invention. (A) thru | or (D) are the figures for demonstrating the DCT coefficient reduction process of the video recording device of Embodiment 3. FIG. It is a block diagram which shows the structure of the decoding part of the video recording device of Embodiment 4 of this invention. It is a figure which shows the YUV area | region in the video recording device of Embodiment 5 of this invention. 10 is a flowchart illustrating an operation of the video recording apparatus according to the fifth embodiment. It is a block diagram which shows schematically the structure of the audiovisual recording apparatus of Embodiment 6 of this invention. 18 is a flowchart illustrating an operation of a transition point detection unit of the video / audio recording apparatus according to the sixth embodiment. FIG. 25 is a diagram illustrating an example of a transition point stack table of the video / audio recording apparatus according to the sixth embodiment. (A) thru | or (E) are the figures for demonstrating the CM broadcast area detection operation | movement in the audio video recording apparatus of Embodiment 6. FIG. 18 is a flowchart showing a CM broadcast section detection operation of the video / audio recording apparatus of the sixth embodiment. (A) thru | or (C) is a figure which shows the transition point detection method in the audiovisual recording apparatus of Embodiment 7 of this invention. It is a graph which shows the measurement result of the silence time length between CM in the audiovisual recording apparatus of Embodiment 8 of this invention. 29 is a graph showing a measurement result of silent time length when moving from CM to the main program in the video / audio recording apparatus of the eighth embodiment. 29 is a flowchart showing a CM broadcast section detection operation using a silent time length in the video / audio recording apparatus of the eighth embodiment. It is a flowchart which shows CM broadcast area detection operation using the scene change difference value in the video / audio recording apparatus of Embodiment 9 of the present invention. FIG. 25 is a diagram for explaining a CM broadcast section detection operation using a scene change difference value in the video / audio recording apparatus of the ninth embodiment. FIG. 38 is a diagram for explaining an operation in the video / audio recording apparatus of the tenth embodiment.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing a configuration of a video recording apparatus 100 according to Embodiment 1 of the present invention. As shown in FIG. 1, the video recording apparatus 100 includes encoding means that performs encoding in accordance with MPEG-2 (Moving Picture Experts Group) on input video data input from a tuner (not shown) or from the outside. , An HDD (Hard Disk Drive) 105 as recording means for recording the encoded stream generated by the encoding unit 101, and writing and reading the encoded stream to and from the HDD 105 stably. And a stream control unit 103 as stream control means for performing stream control. The video recording apparatus 100 also instructs the stream control unit 103 to temporarily store the encoded stream, the stream buffer 104 as a temporary storage unit, and the encoding unit 101 to start and end encoding. A recording control unit 102 serving as a recording control unit, and a scene change extracting unit 110 serving as a video change point extracting unit configured to extract a scene change from the encoded stream generated by the encoding unit 101. Although FIG. 1 shows the HDD 105 as the recording means, the recording means may be another information recording medium such as an optical disk such as a DVD or a magnetic tape. Further, the video recording apparatus 100 may be a DVD / HDD hybrid recorder. Furthermore, the video recording apparatus 100 is applicable not only to a home video recorder but also to various uses such as a personal computer, a video server, or a surveillance recorder for a surveillance system.

  MPEG-2, which is an encoding method by the encoding unit 101, supports a variable rate, and therefore, the amount of code (bit rate) generated per second varies greatly with time. Therefore, the stream control unit 103 temporarily stores the encoded stream in the stream buffer 104 so that the encoded stream does not overflow or underflow due to data writing to the HDD 105 and data reading from the HDD 105, and the HDD 105 enters the state of the HDD 105. Accordingly, the encoded stream is written to the HDD 105 and read from the HDD 105. The stream buffer 104 includes a ring buffer, and a valid data area is managed by a write position pointer and a read position pointer. That is, when data is written, writing is performed for the data size with the writing position pointer at the head, and the writing position pointer is advanced by the written data size. On the other hand, when reading data, reading is performed for the data size starting from the reading position pointer, and the reading position pointer is advanced by the read data size. However, when the end position of the ring buffer is reached, processing for returning the pointer to the head position is performed.

  FIG. 2 is a block diagram showing a configuration of the scene change extraction unit 110 shown in FIG. As shown in FIG. 2, the scene change extraction unit 110 includes a decoding unit 111 serving as a decoding unit that decodes an input encoded stream, and a histogram generation that generates a histogram of the decoded video data. The histogram generator 112 as means, the first histogram buffer 113 and the second histogram buffer 114 for holding the histogram generated by the histogram generator 112, the histogram stored in the first histogram buffer 113, and the second And a difference extractor 115 as a difference extracting means for obtaining a difference between histograms held in the histogram buffer 114. The scene change extraction unit 110 also includes a scene change determination unit 116 as a scene change determination unit that compares the difference value obtained by the difference extractor 115 with a preset threshold value, and a scene change start command from the recording control unit 102. And a scene change command control unit 120 as a scene change command control means for controlling the scene change extraction unit 110 based on a control signal such as.

Next, the recording operation of the video recording apparatus 100 will be described. When recording is started, the recording control unit 102 outputs a control signal _CEN to the encoding unit 101 to instruct the start of encoding, and the encoding unit 101 encodes the input video data _PIN . Start processing. For example, every time encoding of a GOP (Group of Pictures) that is an encoding unit is completed, the encoding unit 101 sends a GOP encoding completion notification I _EN to the recording control unit 102. The stream control unit 103 sequentially writes the encoded stream encoded by the encoding unit 101 with the write position pointer of the stream buffer 104 at the head. On the other hand, the stream control unit 103 reads the encoded stream from the read position pointer of the stream buffer 104 and writes it to the HDD 105 while confirming that the HDD 105 is in a write state. While the recording is being performed, the above operation is repeated. At the end of recording, the recording control unit 102 instructs the encoding unit 101 to end encoding, and the encoding unit 101 stops encoding. The stream control unit 103 writes all the encoded streams remaining in the stream buffer 104 to the HDD 105.

Next, a method for extracting scene changes while recording will be described. Recording control unit 102, for example, each time it receives a coding completion notice _{I EN} of the GOP from the coding unit 101 sends the scene change extraction command _{C SC} against the scene change extracting unit 110. The scene change extraction command _CSC is sent to the decoding unit 111 via the scene change control unit 120 in the scene change extraction unit 110 as a decoding start command _CDE . Decoding unit 111, an encoded stream _{P ST} via the stream control unit 103 calls from the stream buffer 104, starts decoding. The video data decoded from MPEG-2 to YUV by the decoding unit 111 is input to the histogram generator 112, and a histogram is created based on the count included in each YUV section of the video data by the histogram generator 112. . Here, the partitioning for generating the histogram is a method of partitioning a range of values that can be taken at a constant interval, or a method of partitioning in accordance with a magnitude relationship with respect to each threshold by setting a plurality of thresholds in advance. There is. Although the case where a histogram is created for all of the luminance component Y and the color difference components U and V has been described here, a method of generating a histogram for only the luminance component Y may be employed. In addition, although histograms are created for all components of the luminance component Y and the color difference components U and V, a method may be adopted in which weighting is applied to the color difference components U and V to roughen the division. Further, the method of dividing the histogram is not limited to the example described above, and other partitioning methods may be adopted.

ここで、Ｎは１フレーム内の画素数である。これによって、差異値ｄは０から１までの値をとり、ｉ番目のフレームの映像とｉー１番目のフレームの映像が同じ映像であれば差異値ｄは０になり、ｉ番目のフレームの映像とｉー１番目のフレームの映像の違いが大きければ差異値ｄは１に近づくことになる。 The histogram generated by the histogram generator 112 is alternately sent to and held in the first histogram buffer 113 or the second histogram buffer 114. The difference extractor 115 uses the histograms alternately stored in the first histogram buffer 113 and the second histogram buffer 114 to compare the current video data with the previous video data (for example, The video data of the current frame is compared with the video data of the previous frame), and the difference value is obtained. As a method for obtaining a difference value between two histograms, for example, there is a method of accumulating a difference between values of corresponding components of two histograms for all components. Here, the histogram of the i-th video is represented as H _{i
H i = {h i (1} ), h i (2), ···, h i (K)}
When a histogram H _i of i-th image, obtaining the difference value d between the histogram H _i-1 of the preceding i-1 -th image by the following equation. K is a predetermined positive integer.

Here, N is the number of pixels in one frame. Thus, the difference value d takes a value from 0 to 1, and if the video of the i-th frame and the video of the i-1th frame are the same video, the difference value d is 0, and the difference value d of the i-th frame If the difference between the video and the video of the i-1st frame is large, the difference value d approaches 1.

The difference value d obtained by the difference extractor 115 is sent to the scene change determiner 116 and compared with a threshold value E _TH set in advance by the scene change command control unit 120, and is a value larger than the set threshold value E _TH. if determined that the scene change, and sends the determination result R _SC in the scene change command controller 120. Furthermore, the judgment result, such as during recording end sends the _{R SC} in the recording control unit 102.

As described above, every time GOP encoding is completed by encoding section 101 (that is, in synchronization with completion of GOP encoding in Embodiment 1, scene change extraction section 110 performs scene change). the process for determining, by continuously be carried out during the period until completion of the recording start of the video, while recording a video stream P _ST, it is possible to extract a scene change of the video stream P _ST. In other words, the scene change extraction process is executed every time GOP encoding, which is the minimum unit of encoding, is completed, so that scene change extraction can be performed in parallel with encoding.

  Since scene change extraction is performed in real time, an encoder that does not have a scene change extraction function can be used for the encoding unit 101. Therefore, when selecting an encoder to be used in a video recording device, it is possible to select an encoder that does not have a scene change extraction function, which broadens the range of encoder selection and makes it possible to select an encoder that emphasizes cost and availability. .

  Also, in the method of extracting a scene change by analyzing an encoded stream already stored in an HDD or the like, a time for reading the encoded stream from the HDD as a storage device is required in addition to the scene change analysis time. It is said. However, in the video recording apparatus 100 according to the first embodiment, the writing and reading speed is faster than that of the HDD 105, and the stream is read from the stream buffer 104 (usually using a volatile semiconductor memory) that temporarily stores the stream. Data can be read out at high speed, resulting in faster processing time.

  Further, since the scene change extraction unit 110 is not affected by the operation of the encoding unit 101 (encoder), the scene change extraction reference can be maintained within a certain range even if the type or specification of the encoder is changed.

  In addition to the function of extracting a scene change in parallel with recording, the video recording apparatus of Embodiment 1 also has a function of directly analyzing an encoded stream. Therefore, scene change extraction can be performed by reading the encoded stream via the stream control unit 103 even for the encoded stream directly recorded on the HDD.

  In addition, the scene change function in the encoder chip, which is generally composed of hardware, is not used, but it can be configured only by the firmware of the recording / playback device, so the algorithm for scene change extraction is relatively easy. Or the state transition of the scene change extraction process and the threshold value can be dynamically changed according to the state of the recording / playback apparatus.

For example, during a period in which the input video data is CM video data, the scene change determination threshold _ETH is increased to make it difficult to determine that it is a scene change, so that scene change detection for the CM video data is not executed. A control method may be employed.

Also, focusing on the fact that there is a moment when the audio signal becomes silent in a scene change, when the audio signal is silent (that is, at a level lower than a predetermined threshold), the scene change determination threshold E _TH is reduced. Accordingly, a control method may be employed in which a scene change is detected in consideration of not only video data but also audio data, and scene change detection accuracy is improved.

  In the above description, the scene change extraction unit is 1 GOP, but the scene change extraction unit may be a plurality of GOP units. The scene change extraction unit may be one or more slice units or one or more macroblock units.

  In the above description, an example using software is shown as a method for performing the decoding process, but the method may be executed by hardware.

Embodiment 2. FIG.
The video recording apparatus according to the second embodiment of the present invention is different from the video recording apparatus according to the first embodiment only in the configuration and operation of the scene change command control unit 120 shown in FIG. Therefore, in description of Embodiment 2, FIG.1 and FIG.2 is also referred.

  The video recording apparatus according to the second embodiment performs the scene change extraction operation asynchronously with the GOP encoding process when performing the scene change extraction in parallel with the recording. This asynchronous encoding process is also simply referred to as “asynchronous process”. This asynchronous process can be executed even when the start timing of the scene change extraction process executed for the GOP for which the encoding process by the encoding unit 101 has been completed does not necessarily coincide with the completion timing of the encoding process. As an actual operation of the asynchronous processing, focusing on a certain GOP, the scene change extraction processing is not performed immediately after the encoding processing is completed, but the scene change extraction processing is executed after a while. .

  Generally, in an apparatus for recording video, such as an HDD / DVD hybrid recorder, an operation that increases the load on the CPU of the apparatus, such as playback or dubbing, may occur in parallel with the recording operation. In addition, actions that are performed irregularly by the user, for example, operations such as opening a DVD tray, displaying a function setting screen, and viewing an electronic program guide, cause a temporary increase in the CPU load of the apparatus. When a load increase occurs due to these operations, there is no guarantee that the CPU can continue to be occupied for the scene change extraction process, and it is desirable to delay the scene change extraction process for a while while waiting for the completion of other processes. There may be cases. On the other hand, the scene change extraction process should not interfere with the video recording function, which is the basic function of the apparatus. Therefore, it is desirable that the scene change extraction process has a lower priority than the recording process. . In order to realize such a function, the video recording apparatus of the second embodiment is configured so that the scene change extraction process can be performed asynchronously with the recording operation.

  FIG. 3 is a block diagram showing a configuration of the scene change command control unit 120 (shown in FIG. 2) of the video recording apparatus according to the second embodiment. As shown in FIG. 3, the scene change command control unit 120 includes an API (application program interface) unit 121 that receives a command from the recording control unit 102, a command buffer 122 that holds a plurality of received commands, a scene It has a state control unit 124 that manages the state transition of the change extraction unit and processes received commands, and a scene change extraction result buffer 123 for holding the scene change extraction result and error state and returning them to the recording control unit 102. ing. The video recording device of the second embodiment is different from the video recording device of the first embodiment in that the scene change extraction command issued from the recording control unit 102 is transmitted to the API unit 121 in the scene change command control unit 120. This is temporarily held in the command buffer 122. The number of commands that can be held in the command buffer 122 may be set according to the capacity of the stream buffer 104. For example, assuming that the size of the stream buffer 104 is 20 Mbytes shown in the first embodiment, if the encoded stream is about 15 seconds (ie, 15 pictures per GOP and a time length of 0.5 seconds), 30 GOP (Data) can be temporarily stored, and therefore, it is sufficient that 30 scene change extraction commands can be stored. If the command buffer 122 can hold 30 scene change extraction commands and 30 GOPs of data remain in the stream buffer 104, it is possible to extract scene changes retroactively from the latest GOP to the 30th previous GOP. This also means that scene change extraction processing delayed by 30 GOPs, that is, by 15 seconds is possible. The scene change extraction result buffer 123 of the scene change command control unit 120 can hold a plurality of scene change extraction results executed by the state control unit 124 of the scene change extraction unit 110, and the number of commands that can be held in the command buffer 122. It is sufficient that the same number can be held.

  As a result, the scene change extraction process of the preceding GOP has not been completed (for example, because the user has requested playback of a stream that has already been recorded, the system load increases and the scene change extraction process cannot be completed. In this case, even when the encoding of the next GOP is completed and a scene change extraction command is issued from the recording control unit 102, the new scene change extraction command is held in the command buffer 122. The scene change extraction process stored in the command buffer 122 can execute the scene change extraction process of the preceding GOP after the currently executed scene change extraction process is completed, so that the scene change extraction process is performed every time the GOP is encoded. This prevents the problem that the next scene change extraction process cannot be executed because it is not completed.

Next, scene change extraction processing that operates asynchronously with the recording operation will be described with reference to the flowcharts of FIGS. 4, 5, and 6. First, the operation of the recording control unit 102 will be described. When the video recording apparatus 100 is turned on, the system is activated (step S201), the system end is confirmed (step S202) and the recording start is confirmed (step S204), and the system end is requested. For example, the process proceeds to the system end (step S202). On the other hand, if there is a request to start recording, a recording start command is issued to the scene change command control unit 120 (step S205). After the recording start command is issued, the end of recording is confirmed (step S206). If the recording is ended, a recording end command is issued to the scene change command control unit 120 (step S207). In the recording state, when encoding in units of GOP is completed (step S208), a scene change detection command is issued to the scene change command control unit 120 (step S209). The GOP encoding detection (step S208) is continued until the recording is completed. The control command issued from the recording control unit 102 is processed in the API unit 121 of the scene change command control unit 120. When the scene change control command _CSC is issued, the vacancy of the command buffer 122 is checked (step S223), and if the command buffer 122 is vacant, the command is stored in the command buffer 122 (step S224). The scene change extraction result stored in the scene change result buffer 123 is then acquired from the scene change state control unit 124, including the case where it cannot be stored in the command buffer 122 (step S225). Finally, the scene change extraction result and whether or not the command is accepted are returned to the recording control unit 102 (step S226).

  On the other hand, when the scene change state control unit 124 is powered on (step S241), it enters a command waiting state (step S242). If the received command is the system end (step S243), the system end is performed (step S245). . If the command is a recording start command (step S252), a recording start process (step S253) such as securing a memory or initializing variables for recording start is executed, and a recording end command (step S254) is executed. If there is, for example, a recording end process (step S255) for releasing the secured memory and sending the scene change extraction result to the recording control unit 102 is performed. On the other hand, if the received command is a scene change extraction command, the encoded stream is decoded based on the head address in the stream buffer 104 of the encoded stream passed along with the scene change extraction command (step S246). A histogram is generated from the video data converted into YUV by the decoding unit 111 (step S247), and the histogram is stored in the first histogram buffer 113 (or the second histogram buffer 114) (step S248). A difference value between the generated histogram and the histogram of the previous video stored in the second histogram buffer 114 (or the first histogram buffer 113) is obtained (step S249), and if it is equal to or greater than a preset threshold value. If it is a scene change, it is determined (step S250). This result is stored in the result buffer 123 (step S251).

  When the scene change extraction process for one GOP is completed, the process proceeds to a command wait (step S242) to perform the next process. If a command has already been stored in the command buffer 122, analysis of the next GOP is immediately started, but if the next command buffer has not yet been set, the command waiting is continued (step S242). ).

  In the video recording apparatus of the second embodiment, the priority of the task or thread of the software module that performs scene change extraction is set lower than the module group that performs recording and reproduction. The reason for this is that the scene change extraction module is accompanied by MPEG software decoding processing, and therefore has a very high processing load. Therefore, although not shown, the time required to occupy the CPU for processing the software is extremely high compared to other modules. Because it grows. Therefore, if the priority of the scene change extraction module is equal to or higher than the priority of the recording / playback module, it takes up much of the processing time of the CPU during recording / playback, and the processing of the module related to video recording is This is because there is a possibility of causing a problem that delays and results cannot be recorded or reproduced. Furthermore, it can be assumed that the operation of a module that operates by user input, for example, a module that performs key input, screen display, and the like is also delayed by the scene change extraction module. Therefore, it is desirable that the priority of the scene change extraction module is lower than the priority of other modules. On the other hand, scene change detection cannot be performed unless scene change analysis is executed for GOPs generated by recording one after another. However, although the encoding process by the encoding unit 101 is performed in real time, the generated encoded stream is temporarily held in the stream buffer 104 for writing to the HDD 105. Therefore, if the scene change module processes the encoded stream on the stream buffer 104 while the encoded stream is held in the stream buffer 104, the scene change extraction process can be performed on all GOPs. .

  Actually, it is assumed that, while the scene change extraction operation is continuously executed, a system-intensive process such as a request for reproducing another program by the user or display of an EPG program guide is started. Try. When a scene change extraction of the preceding GOP is performed, a large load is applied and the progress of the process is not progressing so much, and the next GOP encoding completion notification is sent from the encoding unit 101 to the recording control unit 102. The recording control unit 102 immediately issues a scene change extraction command to the scene change command control unit 120 of the scene change extraction unit 110. The API unit of the scene change command control unit 120 looks at the empty space in the command buffer (step S223), and if there is a free space, stores the scene change extraction command in the command buffer. The scene change command control unit 120 only stores the scene change extraction command and immediately returns a return to the recording control unit 102. Further, if the scene change extraction of the preceding GOP is not completed by the completion of the encoding of the next GOP, a scene change extraction command for the next GOP is also stored in the command buffer. Thereafter, when the scene change extraction is completed, the scene change state control unit waits for a command (step S242), receives the oldest command from the command buffer, and starts executing the scene change extraction of the next GOP. Thereafter, when the system load becomes steady, the processing frequency of the scene change extraction process increases, and the delayed commands in the command buffer are sequentially executed. However, the maximum number allowed for this delay is 20 GOPs, and when there are more scene change processing requests, there is no command space in the command buffer, so no scene change processing is executed for the corresponding GOP. It will be. At this time, the recording control unit 102 may execute some error processing, and the processing is continued without performing error processing especially because the recording operation which is the main purpose of the apparatus is normally performed. May be.

  In addition to analyzing the stream temporarily held in the stream buffer 104 in this way, by buffering a command issued from the recording control unit 102, the video is not necessarily encoded every time it is encoded. It is not necessary to perform scene change extraction on the stream, and delay processing is possible. Therefore, it is not necessary to continuously process the encoding unit in the encoding process by the recording control unit 102 and the analysis unit in the scene change extraction unit 120, and therefore the encoding process and the scene change extraction process can be designed as independent processing units. .

  In addition, the recording control unit 102 and the scene change extraction unit 120 can operate independently, and the processing time of the scene change extraction unit 120 is made lower than that of the processing related to recording, thereby increasing the processing time. The operation of the scene change extraction unit 120 does not cause the adverse effect of delaying the process of the recording control unit 102 and consequently affecting the normal recording process.

  In addition, since the scene change extraction process can be delayed, if the scene change extraction process cannot be completed within the GOP playback time due to a temporary increase in system load, or the process does not go to the scene change extraction process. Even in this case, the scene change extraction process is skipped and not executed because the scene change extraction command stored in the command buffer 122 can be used to sequentially execute the extraction process on the previously encoded stream. The problem can be avoided.

  Note that the time or number of times that the scene change extraction process can be delayed is determined by the number of stream buffers 104 and command buffers.

  In the second embodiment, the points other than the above are the same as those in the first embodiment.

Embodiment 3 FIG.
The video recording apparatus according to the third embodiment of the present invention is different from the video recording apparatus according to the first embodiment in that the configuration of the decoding unit 111 of the video recording apparatus according to the first embodiment is specified. Therefore, in description of Embodiment 3, FIG.1 and FIG.2 is also referred.

  FIG. 7 is a block diagram showing a configuration of the decoding unit 111 in FIG. The video recording apparatus of the third embodiment is characterized in that the decoding order of the DCT (discrete cosine transform) coefficient is controlled in the decoding process when the scene change extraction process is performed from the encoded stream in FIG. . As illustrated in FIG. 7, the decoding unit 111 includes a pixel size detection unit 301 that obtains a pixel size from the header of the input encoded stream, and uses a DCT block based on the pixel size detected by the pixel size detection unit 301. An order determination unit 302 that determines a component (order) to be performed, a DCT coefficient extraction unit 303 that extracts a DCT coefficient while performing inverse quantization on the encoded stream to the order determined by the order determination unit 302, An IDCT unit 304 that performs inverse DCT from DCT coefficients, and an image shaping unit 305 that generates a reduced image by packing pixel data that has been discretized by inverse DCT performed with a reduced order. Yes.

  8A to 8D are diagrams for explaining DCT coefficient reduction processing of the video recording apparatus according to the third embodiment. 8A shows the encoded stream pixel size, FIG. 8B shows the DCT block (when 2 × 2 is used) and the DCT block (when 4 × 4 is used), and FIG. (C) shows a decoded DCT block (when invalidating other than 2 × 2) and a decoded DCT block (when invalidating other than 4 × 4), and FIG. 8 (D) shows a decoded image. . In the third embodiment, a video recording apparatus will be described in which the pixel size of a video to be encoded can be selected from 704 × 480 pixels and 352 × 240 pixels as desired by the user. Therefore, the stream input to the decoding unit 111 has two types of pixel sizes, that is, an encoded stream of 704 × 480 pixels or an encoded stream of 352 × 240 pixels. When an encoded stream is input to the decoding unit 111, the pixel size detection unit 301 first detects the pixel size of the encoded stream and sends the detected pixel size to the order determination unit 302. The order determination unit 302 determines the order De to be processed among the DCT coefficients included in the encoded stream according to the detected pixel size. For example, in the case of an encoded stream of 704 × 480 pixels, processing is set in advance to process up to degree 2 in the DCT block, while in the case of an encoded stream of 352 × 240 pixels, processing up to order 4. According to this order, DCT coefficient extraction section 303 and IDCT section 304 set the order to be processed by the DCT block, and perform decoding. Since the decoded video data output from the IDCT unit 304 is subjected to DCT coefficient decimation in units of DCT blocks, the decoded video pixels generated by connecting the decoded DCT blocks are discrete. It is a video. The pixel shaping unit 305 performs a process of filling the pixel gaps with respect to an image having discrete pixels. As a result, the decoded video is 176 × 120 pixels in the case of an encoded stream of 704 × 480 pixels, and 176 × 120 pixels in the case of an encoded stream of 352 × 240 pixels. Even though the pixel size is different, an image of the same size can be obtained. By sending the decoded video to the next histogram generator 112, the scene change can be detected as in the first embodiment.

  That is, in the third embodiment, a function for controlling the DCT coefficient is added to the decoding unit 111, so that even if the encoded stream is encoded with a different pixel size by the encoding unit 101, a scene change is performed. This indicates that the video used for extraction can be performed with the same pixel size, and that the upper limit of the frequency band of the video to be analyzed by scene change extraction can be made uniform. In addition, since scene change extraction can always be performed on an encoded stream having the same pixel size in the histogram generator 112 and later, the processing in the subsequent histogram generator 112 and later does the same processing without depending on the pixel size. be able to.

  By configuring as described above, it is possible to obtain video decoded to the same or similar pixel size even when the pixel sizes of the encoded streams are different.

  Also, even if the pixel size of the encoded stream is different, the scene change extraction process can be performed on the decoded video of the same pixel size, so that the stream having a different pixel size of the encoded stream can be performed. However, the same scene change evaluation method and evaluation standard can be used. As a result, even when the same video is recorded while changing the pixel size, the same time position can be detected as a scene change.

  Also, by controlling the DCT coefficients for encoded streams with different pixel sizes, the frequency band of the decoded image can be made uniform, so the same video can be recorded even when the pixel size is changed. The time position can be detected as a scene change.

  Further, since the higher resolution encoded data can be thinned out more, scene change extraction can be performed at high speed even for high resolution encoded data that requires processing time.

  Further, since the higher-order encoded data can be configured so that the high-frequency order can be thinned out, the higher-resolution encoded data that is relatively easy to generate high-frequency noise can effectively perform noise removal.

  Further, in order to generate the same decoded video regardless of the resolution, the memory size for storing the decoded video can be made the same.

  In addition, since a thinning process is performed for a high-resolution video that originally requires a large memory size to store the decoded video, a memory area that should be prepared for the decoding process can be reduced.

  The third embodiment is the same as the first and second embodiments except for the points described above.

Embodiment 4 FIG.
The video recording apparatus according to the fourth embodiment of the present invention is different from the video recording apparatus according to the first embodiment in that the configuration of the decoding unit 111 of the video recording apparatus according to the first embodiment is specified. Therefore, in description of Embodiment 4, FIG.1 and FIG.2 is also referred.

  FIG. 9 is a block diagram showing the configuration of the decoding unit of the video recording apparatus according to Embodiment 4 of the present invention. With reference to FIG. 9, a description will be given of a process of limiting the slices to be decoded in the decoding process when the scene change extraction process is performed from the encoded stream. The decoding unit 111 of the video recording apparatus according to the fourth embodiment includes a pixel size detection unit 301 that obtains a pixel size from the header of the input encoded stream, and scene change extraction from the pixel size detected by the pixel size detection unit 301. The slice to be processed is used slice determining unit 306, the header detecting unit 307 that analyzes the header of the encoded stream, the inverse quantizing unit 308 that performs inverse quantization of the encoded stream, and the frequency domain. And an IDCT unit 304 that performs inverse DCT processing on the coefficient values.

Next, the operation of the scene change extraction process that operates in conjunction with slice reduction will be described. The pixel size detection unit 301 detects the pixel size, and the use slice determination unit 306 determines a slice to be used. Here, for example, when there are only two types of resolutions of the encoded stream generated by the encoding unit 101, 720 × 480 pixels and 352 × 240 pixels, when the number of pixels in the vertical direction is 480, 5 slices, ie
(16 pixels × 5 slices) × 2 = 80 pixels × 2
On the other hand, when the number of pixels is 240, the upper and lower slices, that is,
(16 pixels × 2 slices) × 2 = 32 pixels × 2
Not to use. Here, the number of pixels that are not used differs depending on the resolution, but in the case of 480 pixels, 10 slices are not used for all 30 slices, so that 1/3 is not used, whereas 240 pixels. In this case, since 4 slices are not used for all 15 slices, 4/15 is not used, and it can be seen that almost the same ratio is targeted for scene change extraction.

In this way, the number of slices used in the used slice determination unit 306 is obtained, the header of the encoded stream is analyzed by the header detection unit 307, and a slice that is not used by the previous use slice determination unit 306 each time a slice header is extracted. If it is the determined slice, it jumps to the next slice header, and if it is a slice to be used, the corresponding slice is sent to the inverse quantization unit 308. The inverse quantization unit 308 performs inverse quantization on the encoded stream, and the IDCT unit 304 performs inverse DCT. With the above configuration, only the slice to be used is decoded. As a result, if the vertical resolution is 480 pixels, the video having the vertical resolution of 320 pixels is decoded, while if the vertical resolution is 240 pixels, the video having the vertical resolution of 172 pixels is decoded. You can ask for a video. The histogram generator 112 generates a histogram for a pixel at the same position as an effective slice area for the video that has been reduced and decoded in the vertical direction as described above by the decoding unit 111. In order to derive the difference value d obtained by the difference extractor 115, N (number of pixels in one frame) representing the number of pixels in the frame is used. In the fourth embodiment, since the number of pixels is reduced in the vertical direction, the number N of pixels in the frame is corrected as follows. First, if the original is a vertical resolution of 720 pixels, the reduced vertical resolution is 320 pixels and the horizontal resolution is 720 pixels.
N = 320 × 720 = 230400
It becomes. Further, in the case of a vertical resolution of 240 pixels, if the reduced vertical resolution is 172 pixels and the horizontal resolution is 352 pixels, N = 172 × 352 = 60544
Make corrections. Using the difference value d obtained after performing this correction, the scene change determination unit 116 can perform the same scene change determination as in the first embodiment.

  In the above description, whether or not to perform decoding is set in units of slices, but an area used for scene change may be set in units of pixel positions or macroblocks. In this case, since the position where the scene change is detected can be specified, the detection accuracy can be improved. Also, when setting whether or not to perform scene change extraction for each pixel, it is possible to perform decoding for all the pixels and take the histogram of the area to be extracted after decoding and the difference value of the histogram. If the unit of detection is designated in units of macroblocks, the header detection unit or decoding processing unit can control whether or not to execute processing in units of macroblocks, so that it can be configured not to perform unnecessary decoding. , Processing speed can be improved.

  With the above configuration, the pixel size for decoding and scene change extraction can be reduced in units of slices, so that the processing speed can be increased.

  Further, since the size of the decoded image is reduced by reducing the slices, the memory size to be used is also reduced.

  In addition, if the target of the scene change is a broadcast program, it is information that is less relevant or less important by configuring so that slices at the top and bottom of the video are not used. The scene change can be extracted without being influenced by the time display, the text display of the special news, or the telop.

  In addition, by specifying the execution of decoding and scene change extraction processing in units of macroblocks, it becomes possible to detect scene changes only for any one or more macroblocks, and scene change extraction with a specified position can be performed. It becomes possible.

  The fourth embodiment is the same as the first to third embodiments except for the points described above.

Embodiment 5 FIG.
The video recording apparatus of the fifth embodiment is different from the video recording apparatus of the first embodiment in the operation of the difference extractor 115. Therefore, in description of Embodiment 5, FIG.1 and FIG.2 is also referred.

  In the fifth embodiment, a method of switching a scene change determination criterion according to a scene change extraction target video will be described. FIG. 10 shows a YUV distribution of image pixel values. In general, YUV is expressed by 256 values, the luminance signal Y takes values from 0 to 255, and the color difference signals U and V take values from −128 to 127. At this time, an area that is almost white and an area that is almost black by human eyes are indicated by W and B in FIG. In the fifth embodiment, when the video to be extracted is a video having a large number of regions close to white as a whole (hereinafter also referred to as “white video”), the difference value is calculated in a region other than the region W. On the other hand, in the case of an image having a large number of areas close to black as a whole (hereinafter also referred to as “black image”), the two images to be compared are calculated by a method of calculating a difference value in an area other than area B. The purpose is to detect scene changes accurately even for similar images.

FIG. 11 is a flowchart showing the operation of the difference extractor 115 according to the fifth embodiment. First, the i-th frame (hereinafter also referred to as “i-frame”) and the i−1-th frame (hereinafter also referred to as “i-1 frame”) from the first histogram buffer 113 and the second histogram buffer 114. A histogram is acquired (step S301). Next, the number of pixels in the white area W of each frame is counted (step S302). The numbers of white pixels C _Wi and C _Wi−1 in the i frame and the i−1 frame are expressed by the following equations.

Next, determine the cumulative difference _{r W} of the white area W by the following equation (step S303).

On the other hand, for the black region B, the numbers of black pixels C _Bi and C _Bi−1 are similarly obtained by the following equation (step S304).

Moreover, determining cumulative difference _{r B} of the black region B by the following equation (step S305).

Next, if both of the i frame and the i-1 frame satisfy the following conditional expression, it is determined that the video has a lot of white areas (also referred to as “white video”) (step S306).
_{C W _MAX> C Wi> C} W _MIN and _{C W _MAX> C Wi-1} > C W _MIN
Above, as the color area determining conditions often video close to white as a whole, the number of pixels in the white region W is greater than the lower limit C _W _MIN, with the proviso that is less than the upper limit C _W _MAX. In this way, not only is the number of white pixels satisfied more than the lower limit, but the reason why the upper limit is set is that when the number of white pixels is larger than a certain value, the number of pixels other than the area close to white is very large. This is to prevent the scene change of the entire image from being determined with pixels other than white, which is very small relative to the total number of pixels.

If the above condition can be satisfied, a difference value is obtained for the area excluding the white area W by the following equation (step S307).

On the other hand, if it is not determined that the image has many areas close to white, it is determined whether the image has many areas close to black according to the following conditional expression (step S308).
C _B _MAX> C _Bi > C _B _MIN and C _B _MAX> C _Bi-1 > C _B _MIN
Above, as conditions for determining the color of the region is large image close to black, the number of pixels the black region B is greater than the lower limit C _B _MIN, with the proviso that is less than the upper limit C _B _MAX. If the above condition can be satisfied, a difference value is obtained for the area excluding the black area B by the following equation (step S309).

によって差異値を求める（ステップＳ３１０）。 On the other hand, if it is not determined that the image has many areas of colors close to white and the image has many areas of colors close to black, the difference value derivation formula described in the first embodiment is used.

To obtain the difference value (step S310).

  Using the difference value d obtained as described above, the scene change determination unit 116 determines the scene change in the same manner as the method described in the first embodiment.

  By the above operation, if both of the two images to be compared are images having many areas of colors close to white, the change in the image is compared in the area excluding the white area W, while the two images to be compared are both close to black. If the image has many color areas, the change in the image can be compared in the area excluding the black area B, so that the scene change of the image excluding the specific color area can be extracted.

  For this reason, it is possible to detect a scene change even in a scene in which an image with many color areas close to white as a whole or an image with many color areas close to black continues.

  In particular, in a joint between commercial programs (CMs) of a broadcast program, a company name or product that is different on a white background (or black background) in the same way from a scene where the company name or product name is shown on a white background (or black background) When switching to a scene where the name is shown, scene change determination is performed in the area excluding the white background and black background. As a result, it can be detected as a scene change from the area of the company name and product name, and the CM is separated. It can operate very effectively to detect.

  The fifth embodiment is the same as the first to fourth embodiments except for the points described above.

Embodiment 6 FIG.
FIG. 12 is a block diagram schematically showing the configuration of the video / audio recording apparatus 20 according to the sixth embodiment of the present invention. As shown in FIG. 12, the video / audio recording apparatus 20 includes a tuner 2 that selects a broadcast program to be recorded from a broadcast wave received by an antenna 1 and a video that digitizes an analog video signal output from the tuner 2. The A / D converter 3, the audio A / D converter 4 that digitizes the analog audio signal output from the tuner 2, and the digital video signal is compression-encoded to MPEG-2 (Moving Picture Experts Group-2). A video encoder 5 as an encoding means, an audio encoder 6 as an encoding means for encoding a digital audio signal into, for example, linear PCM (Linear Pulse Code Modulation) or Dolby Digital (Dolby Digital), and encoded Video and audio signals It has a multiplexer (hereinafter also referred to as “Mux”) 7 that multiplexes into one encoded stream, and a video encoder 5, audio encoder 6, and recording controller 8 that controls Mux 7.

  In addition, the video / audio recording apparatus 20 includes a hard disk drive (HDD) 11 as a recording unit for recording the encoded stream, and a stream as a stream control unit for stably writing the encoded stream multiplexed by Mux7 to the HDD 11. The control unit 9, the stream buffer 17 for temporarily storing the encoded stream, the file system unit 10 for saving each generated encoded stream as a file in the HDD 11, and the program name of the generated encoded stream In addition to the generation date and time, the navigation unit 12 generates and manages address information for realizing special reproduction and CM section information described later.

  Furthermore, the video / audio recording apparatus 20 includes a scene change detection unit 13 as a video change point detection unit that detects a scene change scene of video, a silence detection unit 14 as a silence detection unit that detects a silent section of audio, a scene A transition point detector 15 as a transition point detecting means for detecting a transition point at which a change and a silent section occur at the same time, and a transition point detecting section 15 that holds information on a scene change and a silent section in order to detect the transition point A transition point matching buffer 19 as a temporary storage means provided in the CM, a CM filter 16 as a CM broadcast section determination means for detecting a CM section from a plurality of transition points based on the periodicity inherent to the CM, and the CM filter 16. And a transition point stack table 18 for storing transition points to be analyzed.

  FIG. 12 shows the HDD 105 as the recording means, but the recording means may be another information recording medium such as an optical disk such as a DVD or a magnetic tape. The video / audio recording apparatus 20 may be a receiver or a DVD / HDD hybrid recorder that receives and records a broadcast program. Furthermore, the video / audio recording apparatus 20 is applicable not only to a home video recorder but also to various uses such as a personal computer, a video server, or a surveillance recorder for a surveillance system.

  The file system unit 10 performs data management for making it easy to access the write information by making the information to be written in the HDD 11 into a file. The file system unit 10 allows the stream control unit 9 and the navigation unit 12 to specify a file without worrying about the direct address on the HDD 11 when a stream or information is written to or read from the HDD 11. Thus, reading and writing can be performed in order from the top or in order from the designated position.

  MPEG-2, which is an encoding method by the video encoder 5, corresponds to a variable rate, and therefore, the amount of code (bit rate) generated per second varies greatly with time. Therefore, the stream control unit 9 temporarily stores the encoded stream in the stream buffer 17 so that the encoded stream does not overflow or underflow due to data writing to the HDD 11 and data reading from the HDD 11, and the HDD 11 is in the state of the HDD 11. Accordingly, the encoded stream is written to the HDD 11 and read from the HDD 11. The stream buffer 17 is composed of a ring buffer, and a valid data area is managed by a write position pointer and a read position pointer. That is, when data is written, writing is performed for the data size with the writing position pointer at the head, and the writing position pointer is advanced by the written data size. On the other hand, when reading data, reading is performed for the data size starting from the reading position pointer, and the reading position pointer is advanced by the read data size. However, when the end position of the ring buffer is reached, processing for returning the pointer to the head position is performed.

  Next, an operation for recording a broadcast program will be described. When the user manually presses the recording start button or the recording start time set in advance by the recording timer is reached, the tuner 2 selects a designated channel from the broadcast wave received by the antenna 1. The video and audio analog signals output from the tuner 2 are sent to the video A / D converter 3 and the audio A / D converter 4, respectively, and converted into digital video signals and digital audio signals. The recording control unit 8 instructs the video encoder 5 and the audio encoder 6 to start encoding and the Mux 7 to start multiprocessing when recording starts. Then, the digital video signal and the digital audio signal output from the video A / D conversion unit 3 or the audio A / D conversion unit 4 are sequentially compressed and encoded by the video encoder 5 and the audio encoder 6, and further, the mux 7 Multiplexing is performed on one program stream, and recording is performed on the HDD 11 while the stream control unit 9 controls the stream.

  The video encoder 5 sends a GOP encoding completion notification to the recording control unit 8 every time encoding of GOP (Group of Pictures) which is an encoding unit is completed. The stream control unit 9 sequentially writes the encoded stream encoded by the video encoder 5 and the audio encoder 6 and multiplexed by Mux 7 with the write position pointer of the stream buffer 17 as the head. On the other hand, the stream control unit 9 reads the encoded stream from the read position pointer of the stream buffer 17 and writes it to the HDD 11 while confirming that the HDD 11 is in a write state. While the recording is being performed, the above operation is repeated. At the end of recording, the recording control unit 8 instructs the video encoder 5 and the audio encoder 6 to end the encoding, and stops the encoding. After the encoding is stopped, the stream control unit 9 writes all the encoded streams remaining in the stream buffer 17 to the HDD 11.

  Next, scene change detection and silence detection will be described. The recording control unit 8 sends a scene change detection command to the scene change detection unit 13 every time GOP encoding completion is received from the video encoder 5. The scene change detection unit 13 reads the GOP data that has been encoded from the stream control unit 9 and detects a scene change of the video. As an example of the detection method, there is the following method. First, the I picture is decoded from the read GOP unit data, and a histogram is created. The histogram is compared with the histogram of the previous I picture, and the sum of the absolute value differences of each element value of the histogram is defined as the scene change difference value, and the scene change difference value is larger than a preset threshold value. If it is a scene change, it is determined. In the following description, to simplify the description, it is assumed that the number of pictures constituting a GOP is fixed to 15 and the GOP time is 0.5 seconds. Therefore, the time when the scene change occurs can be easily expressed by the number of GOPs.

  The audio A / D converter 4 converts the analog audio signal into a linear PCM signal that is a digital audio signal. The silence detection unit 14 determines that the period is a silence interval if the state where the amplitude of the input linear PCM signal is equal to or less than a preset threshold value continues for a preset time. Here, if the state in which the amplitude of the input linear PCM signal is equal to or less than a preset threshold value continues for 300 milliseconds or more, it is determined that a silent section has occurred. The reason for requiring that the state of the amplitude of the linear PCM signal be low for a certain period of time or more as a condition of the silent section is to prevent instantaneous silence from being determined as a silent section, and for each occurrence of silence. This is because if the frequency of the CM broadcast section detection operation described later increases, the processing load on the system increases, which may hinder operations other than the CM broadcast section detection. When it is determined that it is a silent section, the silent start time is calculated from the cumulative number of samples from the start of recording. Further, if the amplitude of the linear PCM signal becomes larger than a preset threshold value while the silence period is continued, it is determined that the silence period has ended, and the silence end time is similarly determined from the accumulated number of samples. Is calculated.

  FIG. 13 is a flowchart showing the operation of the transition point detector 15 of the video / audio recording apparatus 20 shown in FIG. The transition point detection unit 15 switches the operation based on the scene change information sent from the scene change detection unit 13 or the silence information sent from the silence detection unit 14. For example, the operation of the transition point detection unit 15 is started, and the scene change detection unit 13 performs a scene change detection operation (step S101). The scene change information includes a scene change detection flag, a GOP number, and a scene change difference value. If it is sent to the transition point detection unit 15, the scene change information detected in step S101 is stored in the transition point matching buffer 19 in the transition point detection unit 15 (step S102).

  On the other hand, when a silence section is detected by the silence detector 14 (step S103), a silence detection flag, a silence occurrence time, and a silence end time are sent to the transition point detector 15 as silence section information. The silence information related to the silence section determined in step S103 is stored in the transition point matching buffer 19 in the transition point detection unit 15 (step S104). When a silent section is detected, a transition point is detected on the transition point matching buffer 19, that is, a time position where a scene change and silence occur simultaneously is detected (step S105). From the viewpoint of processing efficiency, the transition point is detected every time a silent section is detected. In order to obtain the product information of the scene change detection section and the silent section, it is necessary to detect the silent section that is less frequently generated than the scene change detection. It is because it is suitable. The transition point matching buffer 19 temporarily holds the latest scene change and silent section information. If both match, the matched information is sent to step S106, which is the next process. Scene change information and silent section information, and previous scene change information and silent section information are discarded. Also, the scene change information and the silent section information are discarded when a predetermined time has elapsed. By such control, even if the buffer size is finite, the coincidence between the scene change and the silent section can be detected in real time. Therefore, a transition point can be detected on the transition point matching buffer 19 based on the scene change GOP number for a scene change that occurs between the start time and end time of the silent section. Then, the GOP number of the scene change is used as a value representing the transition point time. The transition point selected in this way, the difference value of the matched scene change, and the silent time length are sent to the next CM filter 16 (step S106), and the transition point stack table 18 for storing transition points in the CM filter 16 is sent. And the operation of the CM filter 16 is executed (step S107).

  Next, a method for detecting a CM broadcast section by the CM filter 16 shown in FIG. 12 will be described. As shown in FIG. 12, a transition point stack table 18 for storing information about transition points is provided in the CM filter 16. FIG. 14 is a diagram illustrating an example of the transition point stack table 18. Since the transition point stack table 18 is updated during the CM broadcast section detection operation that is executed in parallel with the recording, the transition point stack table 18 has a ring buffer configuration. Therefore, the processing range for CM broadcast section detection is from the read pointer to the front of the write pointer. In the transition point stack table 18, a transition point GOP number indicating the time position of the transition point, a concatenation end GOP number indicating the time position of the last transition point when the subsequent transition point is regarded as a CM, and the like follow. A transition point counter indicating the number of transition points, a silent time length at the transition point, a scene change difference value at the transition point, and an valid / invalid flag indicating validity / invalidity of the transition point are provided.

  The transition point GOP number is the GOP number where the scene change is detected. In the sixth embodiment, since the accuracy of scene change detection is 1 GOP, time can be expressed by the cumulative number of GOPs from the start of recording.

  The connection end GOP number is the GOP number of the last transition point in the CM broadcast section based on a certain transition point. In FIG. 14, in the case of the transition point GOP number “10”, CMs are continuously detected at the GOP numbers “10”, “40”, and “101”, so the GOP number “101” becomes the connection end GOP number. Become. When the transition point detection unit 15 detects the transition point 33g, the GOP number ‘101’ is connected to the GOP number ‘130’, and the transition point 33b to the transition point 33g is determined to be a CM broadcast section. Further, when there is no transition point subsequent to the transition point GOP number, the connection end GOP number has the same value as the transition point GOP number.

  The transition point counter indicates the number of transition points when CMs are connected at transition points in the CM broadcast section.

  The silent time length indicates the length of the silent section at the transition point in milliseconds.

  The scene change difference value indicates the change amount of the scene change at the transition point from 0 to 1. The larger the scene change difference value, the larger the amount of change in the video.

  The valid / invalid flag is a flag indicating that it is a processing target range in the transition point stack table, and is set to “1” when the transition point is registered. In the processing of the CM filter 16, for example, “40” is set. As described above, the transition point connected to the preceding transition point is changed to invalid '0'. Further, when a certain transition point is out of the processing target range, for example, when “10”, “40”, “101”, “131” is detected as a CM section, “6” before “10”, By disabling all transition points '62' and '83' between '10' and '131' to '0', the processing speed is increased and the memory size required for the transition point stack table is reduced. Can do.

  The CM filter 16 compares transition points that are elements of the transition point stack table 18, and the transition point interval is equivalent to a predetermined reference interval, for example, 15 seconds, 30 seconds, 60 seconds, or 90 seconds. Will be extracted. At that time, the CM length may be deviated from 15 seconds due to a detection error, a variation in the length of the CM at the frame level, and the like. For example, an allowable error is provided for 1 GOP. Therefore, an effective interval as the interval of the previous transition point can be expressed by the number of GOPs, and a GOP having a tolerance of ± 1 GOP such as 30 GOP ± 1 GOP, 60 GOP ± 1 GOP, 120 GOP ± 1 GOP, and 180 GOP ± 1 GOP. It can be expressed by the number of In the CM filter 16, the interval between transition points where the interval between the transition points coincides with the GOP interval set in advance as described above is set as the CM candidate interval. Furthermore, when there are two or more CM candidate intervals in succession, the section is set as a CM broadcast section.

  15A to 15E are diagrams for explaining the CM broadcast section detection operation in the video / audio recording apparatus 20 of the sixth embodiment. A method for identifying between CM broadcasts will be described with reference to FIGS. A case where the silent sections 31a,..., 31m are detected by the silent detector 14 will be described. On the other hand, it is assumed that scene changes 32a,..., 32o exist as shown in FIG. For the silent section, the silent occurrence time and silent end time are obtained, and for the scene change, the GOP number where the scene change has occurred is obtained. In the sixth embodiment, since the number of GOP frames is fixed, the time can be uniquely expressed by the GOP number. The time of silence and scene change is compared, and when silence and scene change occur at the same time, this point is taken as a transition point. Here, as shown in FIG. 15C, transition points 33a,..., 33i are detected. Further, the transition points 33a,..., 33i that match the CM interval are determined as CMs. Here, the interval from the transition point 33b to the transition point 33c is 30 GOP, the interval from the transition point 33c to the transition point 33f is 61 GOP, and the interval from the transition point 33f to the transition point 33g is 29 GOP. Are detected as CMs, and three CMs continue in succession. Therefore, it is determined that the transition points 33b, ..., 33g are CM broadcast sections. Further, the transition point 33i is detected when 201 GOP has elapsed from the transition point 33g, but the interval from the transition point 33g to the transition point 33i exceeds 180 GOP which is the maximum value of the preset CM broadcast section. It can be determined that the CM has ended, indicating that the CM broadcast section detection has been completed.

  Next, the CM detection processing procedure by the CM filter 16 will be described with reference to FIG. When the transition point detection unit 15 detects a transition point where a scene change and a silent section occur simultaneously, the information is sent to the CM filter 16 to start the operation of the CM filter 16 (step S401). Information on the detected transition point is stored at the end of the transition point stack table 18 in the CM filter 16 (step S402). Information stored in the transition point stack table 18 in the CM filter 16 includes a valid / invalid flag, transition point GOP number, connection end GOP number, transition point counter value that is the number of transition points detected as consecutive CMs, and detection. This is a scene change difference value indicating the silent time length of the transition point and the change amount of the scene change. Next, the reading base point of the transition point stack table 18 is set (step S403). In FIG. 15C, if, for example, the processing of the transition point 33a has been completed last time, the next transition point 33b is set as the first base point this time. Next, the end point side of the transition point is set (step S407). In FIG. 15C, the transition point 33c is the end point. If there is no transition point as the end point in the transition point stack table 18 (step S408), the next base point is acquired (step S404). In FIG. 15C, the base point is the transition point 33b when the base point is the transition point 33a and the end point is the transition point 33i. If the next base point is not found (step S404), the CM detection process is terminated (step S406). When the base point and the end point are determined, the time length from the base point to the end point is obtained (step S409). In FIG. 15C, when the base point is the transition point 33b and the end point is the transition point 33c, the difference between the base point '10' and the end point '40' (that is, the transition point interval) is 30 GOP. It is. If this time length matches one of the CM reference times of 30 GOP ± 1 GOP, 60 GOP ± 1 GOP, 120 GOP ± 1 GOP, and 180 GOP ± 1 GOP (step S 410), the base end connection end GOP number is connected to the end point connection. Replace with the end GOP number (step S411). Further, the valid / invalid flag at the end point is invalidated (step S412), and the transition point count of the base point is incremented by one (step S413). If it is determined in step S410 that it is not the CM reference time length, it is next determined whether or not the time length between the base point and the end point exceeds the maximum CM reference time length of 180 GOP ± 1 GOP (step S410). In S414), if it is determined that the number has exceeded, it is determined as the CM broadcast section from the base point to the connection end GOP number, and information on the CM broadcast section is sent to the recording control unit 8 (Step S415). Further, the base point of the next CM broadcast section detection process is set to the transition point next to the final transition point of the current CM broadcast section, and the valid / invalid flag of the transition point before the detected final transition point is invalidated ( Step S416).

  By continuously executing the above steps during the recording operation, the CM broadcast section of the broadcast program to be recorded can be detected in real time. The time information of the CM broadcast section detected by the CM filter 16 is once passed to the recording control unit 8 and transmitted to the navigation unit 12 that handles the management information of the recorded program, and is filed by the file system 10 and CM broadcast section information. It is recorded on the HDD 11 that functions as a recording means.

  By configuring the video / audio recording apparatus 20 as described above, the CM broadcast section can be detected while recording the broadcast program.

  In addition, when an encoder that does not have a scene change detection function is used, it is usually necessary to analyze a recorded program after the recording is executed. However, in the configuration of the present invention, the CM broadcast section is recorded in parallel with the recording operation. Since detection can be performed, the range of encoder selection is widened, and as a result, it is possible to select an encoder with an emphasis on cost and availability.

  Furthermore, it is also possible to detect a CM broadcast section for a program recorded in the HDD 11 by sending a stream from the stream control unit 9 to the scene change detection unit 13 and the silence detection unit 14. However, in that case, silence detection must use a method of analyzing from a stream in the same manner as a scene change.

  In addition, the scene change detection result and the silence detection result are recorded in the transition point matching buffer 19 which is a temporary storage area, and the transition point is detected in real time by comparing on the storage area of the transition point matching buffer 19. Therefore, the CM broadcast section can be detected in parallel with the recording. Even if the scene change and the silent section occurring at the same time are notified to the transition point detection unit 15 with a certain time difference, the information is held in the transition point matching buffer 19, so that one of the information is missed. It is possible to detect the CM broadcast section without any problem.

  When a scene change is detected, only registration in the transition point matching buffer 19 which is a temporary storage area is executed. On the other hand, when silence is detected, not only registration in the transition point matching buffer 19 but also transition point matching is performed. By detecting a scene change that occurs simultaneously with silence on the buffer 19 and further performing a detection process for the CM broadcast section, the processing can be performed in synchronization with silence detection with a small number of detections. The CM broadcast section can be detected with a short processing time and a low system load.

  Further, not only can the CM broadcast section detection unit be a GOP unit, but not a frame unit, for example, the processing frequency and processing load can be reduced, and the CM length may differ by several frames for each CM. In addition, the CM broadcast section can be detected without being affected by the difference in length of each CM.

  In the above description, the scene change detection unit is 1 GOP, but the scene change detection unit may be a plurality of GOP units. Moreover, the scene change detection unit may be set to an arbitrary number of frames.

  In the above description, an example using software is shown as a method of performing scene change and silence detection processing, but it may be executed by hardware. In addition, the sound directly analyzes the output of the audio A / D conversion unit 4, but silence may be detected by analyzing from the encoded stream.

Embodiment 7 FIG.
In the seventh embodiment, a method of selecting a transition point to be determined as a boundary between CMs when there are a plurality of transition points near the boundary between CMs will be described. Usually, a telop such as the name of a company or a product to be advertised is often temporarily displayed before and after the CM. For this reason, in the vicinity of the boundary between CMs, silence and scene changes occur at the same time, and transition points often occur continuously several times. On the other hand, in the detection of the CM broadcast section, the detection is performed with a certain width with respect to the reference CM length in order to compensate for the CM time length error, detection delay, and detection variation. For example, in the sixth embodiment, detection is performed within a range of 30 ± 1 GOP in order to detect a CM of 15 seconds. However, there may be a plurality of transition points in the detection width of ± 1 GOP, and a guideline for selecting one transition point from candidate transition points is necessary. Therefore, in the seventh embodiment, three selection criteria serving as guidelines for selecting one transition point from candidate transition points are shown.

  FIGS. 17A to 17C show candidate transition points and selected transition points. In the first method, as shown in FIG. 17A, the next transition point is selected so that the transition point interval is ± 0 GOP from the reference time, such as 30 GOP, 60 GOP, and 90 GOP. It is.

  In the second method, as shown from GOP number “30” to GOP number “59” in FIG. 17B, the interval between transition points selected in a certain CM boundary detection is 29 GOP (= 30 GOP-1GOP). In this case, as shown from GOP number “60” to GOP number “90” in FIG. 17B, the interval of the selected transition point in the next CM boundary detection is centered on 31 GOP (= 30 GOP + 1 GOP). It is a method to choose. This is because when the selected transition point is temporarily deviated from the reference value (corresponding to GOP number '59' in FIG. 17B), the next transition point is given priority to the GOP that cancels the deviation. This is to be selected (corresponding to GOP number “90” in FIG. 17B). That is, there are a plurality of transition points where a detected scene change and a detected silent section occur simultaneously within a predetermined time range (within ± 1 GOP) centered on a preset reference time (for example, 30 GOP). When detected, the CM filter 16 determines that the sum of the time length of the CM determined immediately before and the time length of the CM determined next is the reference time set in advance (for example, 30 GOP). It is determined that a transition point having a large degree close to an integral multiple of is a transition point for defining a CM broadcast section.

  The third method is a method of selecting the accumulated time so that it becomes ± 0 GOP as much as possible. As shown in FIG. 17 (C), after the selected transition point is set to GOP numbers “0”, “29”, and “58”, the GOP number “88” in which the GOP interval is 30 ± 0 GOP is obtained. GOP number '89' with GOP interval 30 + 1GOP is selected without selecting GOP, and GOP number '120' is selected without selecting GOP interval 30GOP-1GOP or 30GOP ± 0GOP at the next detection. By doing so, the shift of the transition point as a whole becomes ± 0 GOP. That is, there are a plurality of transition points where a detected scene change and a detected silent section occur simultaneously within a predetermined time range (within ± 1 GOP) centered on a preset reference time (for example, 30 GOP). When detected, the CM filter 16 defines a transition point having a large degree of approaching the determined CM broadcast section as an integral multiple of a preset reference time (for example, 30 GOP) as the CM broadcast section. Judge as a transition point.

  By adopting the second method or the third method, correction processing can be performed even when the transition point is once deviated from the boundary position between the original CMs due to a detection error or the like. Broadcast segment detection accuracy can be improved.

  In addition, by preferentially selecting transition points at which transition point deviations become ± 0 GOP as a whole, the detected CM broadcast section comes close to the reference time, and the detection accuracy of the CM broadcast section is improved.

  Further, this is an error for detecting one CM by weighting the detected transition point so that the deviation from the reference time of the cumulative value of the cumulative time number or the GOP number becomes ± 0 GOP. While ± 1 GOP is continuously executed in the CM broadcast section, it can be accumulated as the number of CMs increases, that is, the CM to be detected can be detected by deviating from the original CM boundary or even deviating from the original CM boundary. The problem of disappearance can be reduced.

  In the seventh embodiment, points other than those described above are the same as in the sixth embodiment.

Embodiment 8 FIG.
In the eighth embodiment, a method for improving the detection accuracy of the CM broadcast section using the silent time length at the transition point will be described.

  In general, in a TV broadcast program, there is a silent part of about 0.5 seconds before and after one CM, and in the vicinity of the boundary between the CM and the CM, the sum of the silent parts of both CMs is about There is a silent part of 1 second (≈0.5 seconds × 2). On the other hand, when moving from the program main part to the CM and when moving from the CM to the main part of the program, the silent part on the main part of the program is often short, and the sum of both silent parts is about 0.6 seconds of silence. Only the part enters.

  FIG. 18 is a graph showing a result of measuring the time length of a silent portion between a certain CM and the next CM in a broadcasting section including any 20 programs and 591 CMs actually broadcast. . As shown in FIG. 18, most of the silent part between CMs exists between 0.8 seconds and 1.4 seconds.

  FIG. 19 is a graph showing the result of measuring the time length of the silent part when the CM broadcast section ends and the program shifts to the main part in the broadcast section of any 20 programs as in FIG. As shown in FIG. 19, when moving from the CM broadcast section to the main part of the program, most of the time length of the silent part exists between about 0.4 seconds and about 1.2 seconds. I understand. In other words, if the silent time length at the transition point shown in Embodiment 6 is between 0.4 seconds and 0.8 seconds, or 1.4 seconds or more, the CM is almost finished and the main program It can be determined that it has shifted to.

  FIG. 20 is a flowchart showing a method for detecting a CM broadcast section using the silent time length between CMs. In FIG. 20, the same steps as those in FIG. 16 are denoted by the same reference numerals. The basic CM broadcast section detection method in the eighth embodiment is the same as that in the sixth embodiment, but in the eighth embodiment, the silent time length of the transition point is verified in step S420, and the CM In the state where two or more are detected, as described above, if the silent time length does not fall within the range from 0.8 seconds to 1.4 seconds (that is, the first specified range), it is determined that the CM has ended. Has been added. In addition, even when a CM broadcast section is detected, at least one silence period at each transition point of the CM broadcast section, and a silence portion having a time length within a range from 0.8 seconds to 1.4 seconds Is added (step S420), and if a silent part having a time length within a range from 0.8 seconds to 1.4 seconds is not included, a process not adopted as a CM broadcast section is added. ing.

  By configuring as described above, the end of the CM broadcast section and the start position of the main part can be detected, and the detection accuracy of the CM broadcast section can be improved.

  In particular, if a scene change and silence occur at the same interval as the CM interval after moving from CM to the program main part, the beginning part of the main part of the program may be erroneously detected as a CM, but the CM ends due to the length of silence. The position can be detected with high accuracy, the occurrence frequency of a situation in which the beginning of the main part of the program is erroneously detected as CM can be reduced, and as a result, the detection accuracy of the CM broadcast section can be improved.

  Furthermore, if a plurality of scene changes and silent sections occur simultaneously in the main part of the program, and the interval is the same as the occurrence interval of the scene change and silent section in the CM broadcasting section, the main part of the program is the CM broadcasting section. Although there is a case of erroneous detection, since it is a condition that the silent time length matches the silent time length between CMs, it is possible to reduce the frequency of occurrence of a situation in which the main program is erroneously detected as a CM.

  In the eighth embodiment, the points other than the above are the same as those in the sixth and seventh embodiments.

Embodiment 9 FIG.
In the ninth embodiment, a method of detecting a CM broadcast section that can improve detection accuracy using a scene change difference value indicating a change amount of a scene change will be described.

  FIG. 21 is a flowchart showing a CM broadcast section detection method using the scene change difference value in the video / audio recording apparatus of the ninth embodiment. FIG. 22 is a diagram for explaining a CM broadcast section detection method using a scene change difference value in the video / audio recording apparatus according to the ninth embodiment. FIG. 22 shows the detected CM broadcast section and transition points 51a,..., 51f located at the boundary of each CM in the CM broadcast section. In general, there is little visual causal relationship between the commercial and the main program or between the commercial and the commercial, so the amount of change in the scene change is extremely large. In the ninth embodiment, the program main part is mistakenly used because the scene change difference value between the CM and the main part of the program and between the CM and the CM is larger than the scene change difference value in the main part of the program. A method for reducing false detections that are regarded as CMs will be described.

  The operation shown in the flowchart of FIG. 21 is based on the operation shown in the flowchart of FIG. 16 in the sixth embodiment, and some judgments and processes using the scene change difference value are added. In FIG. 21, the same or corresponding steps as those in FIG. 16 are denoted by the same reference numerals.

  First, the CM filter 16 determines that the time length from the transition point as the base point to the transition point as the end point is the same as the CM reference time length (for example, 15 seconds, 30 seconds, 60 seconds, 90 seconds, etc.) ( After step S410), it is determined whether the base point and end point are the head CM and the time length is 60 seconds or 90 seconds (step S430). If the conditions match, the base point scene change difference value and the end point scene change difference value are compared (step S431). If the base point scene change difference value is large, it is determined that the CM is started from the base point position, and step S411 is performed. Return to. In FIG. 22, the scene change difference values at the transition point 51a and the transition point 51b are compared, and it is shown that the larger scene change difference value is determined as the boundary between the main program and the CM.

  On the other hand, if it is determined in step S431 that the scene change difference value at the end point is larger than the scene change difference value at the base point, the CM filter 16 starts CM from the end point position, that is, between this base point and the end point. Determines that the main program has been continued.

  Similarly, the CM filter 16 determines whether the CM is the final CM at the end of the CM and whether the time length is 60 seconds or 90 seconds (step S432). And the scene change difference value at the end point are compared (step S433). If the scene change difference value at the end point is larger than the scene change difference value at the base point, it is determined that the main program has been started from the end point position, and the process proceeds to step S435. FIG. 22 shows that the scene change difference value at the transition point 51e is compared with the scene change difference value at the transition point 51f, and the larger scene change difference value is determined as the boundary between the CM and the main program.

  On the other hand, if it is determined that the base scene change difference value is larger than the end point scene change difference value, the CM filter 16 has started the main program from the base point position. Judged to be in the main story. In this way, in the case of a CM that is long such as 60 seconds or 90 seconds at the start of the CM or at the start of the main part and is broadcasted less frequently, the main part of the program depends on which of the video change amount is greater before or after the section. And the boundary of CM. The beginning and end of the main part of the program may be edited units such as a group of episodes and one corner. In this case, there is a possibility of being erroneously detected as a CM. In the section detection method, the boundary between the CM and the main part of the program can be accurately detected by finding the CM boundary having a larger video change.

  In step S435, the scene change difference value of all transition points is checked and compared with a second threshold value that is larger than the threshold value used for the scene change determination, and the scene change value of at least one transition point in the CM broadcast section is compared. If the difference value exceeds the second threshold value, the process proceeds to step S415 where it is handled as a CM broadcast section. On the other hand, if the difference value does not exceed the second threshold, the CM broadcast section is invalidated (step S436). Referring to FIG. 14, if the threshold value of the scene change difference value is 0.9, the scene change difference values at the transition points 33b and 33g out of the transition points 33b, 33c, 33f, and 33g exceed the threshold value 0.9. Therefore, the period from the transition point 33b to the transition point 33g is determined as the CM broadcast section.

  As described above, the boundary between the program main part and the CM can be determined by using the scene change difference value for selecting the boundary between the program main part and the CM in the CM at the boundary between the program main part and the CM. Alternatively, it is possible to reduce the occurrence frequency of a situation in which the last is erroneously detected as a CM.

  In addition, if at least one of the scene change difference values does not exceed a certain threshold within the detected CM broadcast section, it is determined that the program is not a CM so that the main program having a smaller video change amount than the CM is a CM. The frequency of occurrence of erroneous detection can be reduced.

  In the ninth embodiment, points other than those described above are the same as those in the sixth to eighth embodiments.

Embodiment 10 FIG.
In the tenth embodiment, when there is a CM of several seconds at the beginning of a recorded stream and the main part is recorded subsequent to the CM, a method of detecting the fragmented CM of the several seconds is shown. . Usually, the video / audio recording apparatus 20 requires a certain period of time from the start of recording to the start of actual recording, and for this reason, the recording start time set in advance several seconds back. Are often configured to start recording. Further, there is no guarantee that the clock held in the video / audio recording apparatus always indicates the correct time. Therefore, for example, even if a broadcast program is scheduled to start at PM 9:00 and the main part at PM 9:00 is started, the program may be recorded at the beginning of the recorded stream. Embodiment 10 shows a method for extracting a fragmented CM recorded at the beginning of a recorded stream.

  The tenth embodiment will be described with reference to FIG. It is assumed that a transition point, which is a point at which scene change and silence occur simultaneously, is detected from a recorded stream. The transition point 60c finally generated in the forced CM conversion section 61 in FIG. 23 is determined as the boundary between the CM and the main part, and the portion between the recording start position 62 and 60c is regarded as the CM. The compulsory CM conversion section is determined according to the specifications of the video / audio recording apparatus. For example, if recording is started from 5 seconds before the scheduled recording time, the compulsory CM conversion section may be set to 5 seconds. Further, an error of the internal clock of the video / audio recording apparatus, for example, 2 seconds may be taken into consideration, and may be set to 7 seconds. Here, the final transition point 60c of the forced CM section is adopted as the boundary between the CM and the main part so that the time width of the forced CM section is substantially equal to the difference between the actual recording start time and the scheduled recording time. This is because the transition point occurrence frequency in the CM is considerably higher than the transition point occurrence frequency in the main part.

  As described above, even if the video / audio recording apparatus starts recording from an unnecessary CM before the main part in order not to start recording at a preset scheduled recording time, an unnecessary CM before the main part is detected. can do.

  Even if the internal clock of the video / audio recording apparatus does not keep accurate time, it is possible to detect unnecessary CMs before the main part.

  In the tenth embodiment, the final transition point in the compulsory CM conversion section is adopted as the boundary between the CM and the main part. Instead of the compulsory CM conversion section, for example, from the recording start time as the forced CM conversion time, for example, It is also possible to use a time point when 5 seconds have passed and adopt the transition point closest to the forced CM time as the boundary between the CM and the main part. In FIG. 23, 60d and CM are used as the boundaries of the main part.

  Further, the CM broadcast section can be detected by selectively combining the configurations of Embodiments 6 to 10.

DESCRIPTION OF SYMBOLS 100 Video recording device, 101 Coding part, 102 Recording control part, 103 Stream control part, 104 Stream buffer, 105 HDD, 110 Scene change extraction part, 111 Decoding part, 112 Histogram generator, 113 1st histogram buffer, 114 second histogram buffer, 115 difference extractor, 116 scene change determiner, 120 scene change command control unit, 121 API unit, 122 command buffer, 123 scene change extraction result buffer, 124 state control unit, 301 pixel size detection unit , 302 degree determination unit, 303 DCT coefficient extraction unit, 304 IDCT unit, 305 image shaping unit, 306 used slice determination unit, 307 header detection unit, 308 inverse quantization unit, 1 antenna, 2 tuner, 3 B A / D conversion unit, 4 audio A / D conversion unit, 5 video encoder, 6 audio encoder, 7 multiplexer (Mux), 8 recording control unit, 9 stream control unit, 10 file system unit, 11 HDD, 12 navigation unit , 13 Scene change detection unit, 14 Silence detection unit, 15 Transition point detection unit, 16 CM filter, 17 Stream buffer, 18 Transition point stack table, 19 Transition point matching buffer, 20 Video / audio recording device, 31a, ..., 31m Silence Section, 32a, ..., 32o Scene change, 33a, ..., 33i Transition point, 34a, ..., 34c Detected CM, 35 Detected CM broadcast section, 51a Transition point at time of transition from main program to CM broadcast section 51b, ..., 51e Transition point between CMs, Transition point at the time to migrate to the main program from 1f CM broadcast section.

Claims (19)

Encoding means for encoding video data and outputting an encoded stream;
Recording means for recording the encoded stream;
Stream control means for controlling transfer of the encoded stream to the recording means;
Video change point extraction means for extracting a scene change of video data for each encoding unit of the encoded stream with respect to the encoded stream controlled by the stream control means;
A recording control unit that obtains an encoding completion notification from the encoding unit for each encoding unit of an encoded stream, and outputs a scene change extraction instruction to the video change point extracting unit when the encoding completion notification is acquired; Have
The video recording device, wherein the video change point extraction means includes decoding means for decoding an encoded stream. The video change point extraction means includes:
Histogram generating means for generating a histogram of the decoded video data;
A first histogram buffer and a second histogram buffer for alternately holding the generated histogram for each predetermined number of pixels;
Difference extraction means for obtaining a difference value between the histogram held in the first histogram buffer and the histogram held in the second histogram buffer;
The video recording apparatus according to claim 1, further comprising: a scene change determination unit that compares the difference value obtained by the difference extraction unit with a predetermined threshold value. Recording means for recording the encoded stream;
Stream control means for controlling transfer of the encoded stream to the recording means;
Video change point extraction means for extracting a scene change of video data for each encoding unit of the encoded stream with respect to the encoded stream controlled by the stream control means;
Temporary storage means for temporarily storing the encoded stream input to the stream control means, the data writing speed and the data reading speed being faster than the recording means,
The scene change extraction by the video change point extraction unit is executed on the encoded stream read from the temporary storage unit. Recording the encoded stream in a recording means;
A stream control means for controlling transfer of an encoded stream to the recording means;
The video change point extracting means includes a step of extracting a scene change of the video data for each encoding unit of the encoded stream controlled by the stream control means;
The step of extracting the scene change includes:
Decoding means for decoding the encoded stream;
When the recording control means obtains an encoding completion notification for each encoding unit of the encoded stream from the encoding means for encoding the video data and outputs the encoded stream, the video when the encoding completion notice is acquired Outputting a scene change extraction instruction to the change point extraction means. The step of extracting the scene change includes:
A histogram generating means generating a histogram of the decoded video data;
Holding the generated histogram alternately in a first histogram buffer and a second histogram buffer for each predetermined number of pixels;
A step of obtaining a difference value between the histogram held in the first histogram buffer and the histogram held in the second histogram buffer;
5. The video recording method according to claim 4, wherein the scene change determination unit includes a step of comparing the difference value obtained by the difference extraction unit with a predetermined threshold value. Recording the encoded stream in a recording means;
A stream control means for controlling transfer of an encoded stream to the recording means;
The video change point extracting means includes a step of extracting a scene change of the video data for each encoding unit of the encoded stream controlled by the stream control means;
The step of extracting the scene change includes:
Temporarily storing the encoded stream input to the stream control means in a temporary storage means having a data writing speed and a data reading speed faster than the recording means;
The video recording method, wherein extraction of a scene change by the video change point extraction unit is executed on an encoded stream read from the temporary storage unit. Recording means for recording an encoded video and audio stream;
A stream holding unit for temporarily holding the encoded stream before recording the encoded stream in the recording unit;
Video change point detection means for detecting a scene change from the video data of the encoded stream held in the stream holding means;
Silence detecting means for detecting silence from the audio signal;
Temporary storage means for temporarily holding the scene change information detected by the video change point detection means and the silence information detected by the silence detection means;
From the scene change information and silence information held in the temporary storage means, a transition point detection means for detecting a transition point where a scene change and silence occur simultaneously;
A section in which two or more combinations of transition points in which a transition point interval detected by the transition point detecting means is within a predetermined time range centered on a preset reference time continues continuously is a CM broadcast section. CM broadcast section determination means for determining,
A video / audio recording apparatus comprising: CM broadcast section information recording means for recording time information of a CM broadcast section determined by the CM broadcast section determination means. Recording means for recording an encoded video and audio stream;
A stream holding unit for temporarily holding the encoded stream before recording the encoded stream in the recording unit;
Silence detecting means for detecting silence from the audio data of the encoded stream held in the stream holding means;
Temporary storage means for temporarily holding the silence information detected by the silence detection means as a transition point;
A section in which two or more transition point combinations in which the interval of silent information held in the temporary storage means falls within a predetermined time range centered on a preset reference time continues is determined as a CM broadcast section. CM broadcast section determination means to perform,
A video / audio recording apparatus comprising: CM broadcast section information recording means for recording time information of a CM broadcast section determined by the CM broadcast section determination means. Recording means for recording an encoded video and audio stream;
A stream holding unit for temporarily holding the encoded stream before recording the encoded stream in the recording unit;
Video change point detection means for detecting a scene change of the video from the video data of the encoded stream held in the stream holding means;
Temporary storage means for temporarily holding scene change information detected by the video change point detection means;
A section in which two or more transition point combinations in which the interval of scene change information held in the temporary storage means falls within a predetermined time range centered on a preset reference time continues continuously is a CM broadcast section. CM broadcast section determination means for determining,
A video / audio recording apparatus comprising: CM broadcast section information recording means for recording time information of a CM broadcast section determined by the CM broadcast section determination means. When a plurality of transition points are detected within a predetermined time range centered on the reference time used by the CM broadcast section determination means,
The CM broadcast section determining means is:
If the transition point interval determined immediately before by the CM broadcast section determination means is longer than the reference time by a first value, the transition point interval determined by the CM broadcast section determination means is then set as the reference time. The transition point to be shortened by the first value is preferentially selected as the transition point for defining the CM broadcast section,
When the transition point interval determined immediately before by the CM broadcast section determination means is shorter than the reference time by a second value, the transition point interval determined by the CM broadcast section determination means is then set as the reference time. The video / audio recording apparatus according to claim 7, wherein a transition point that is longer than the second value is preferentially selected as a transition point for defining a CM broadcast section. When a plurality of transition points are detected within a predetermined time range centered on the reference time used by the CM broadcast section determination means,
The CM broadcast section determining means determines a transition point at which a time interval from the first transition point of the CM broadcast section to each detected transition point is closest to an integer multiple of the reference time. The video / audio recording apparatus according to claim 7, wherein the video / audio recording apparatus is determined to be a transition point for defining. Recording a video and audio encoded stream in a recording means;
Before the step of recording the encoded stream in the recording means, temporarily holding the encoded stream in the stream holding means;
A step of detecting a scene change from video data of an encoded stream held in the stream holding unit;
A step in which silence detecting means detects silence from the audio signal;
Temporarily storing the scene change information detected by the video change point detection means and the silence information detected by the silence detection means in a temporary storage means;
A transition point detecting means for detecting a transition point where a scene change and silence occur simultaneously from the scene change information and silence information held in the temporary storage means;
The CM broadcast section determination means continues two or more combinations of transition points where the interval of transition points detected by the transition point detection means falls within a predetermined time range centered on a preset reference time. Determining a section to be a CM broadcast section;
Recording the time information of the CM broadcast section determined by the CM broadcast section determination means in the CM broadcast section information recording means. Recording a video and audio encoded stream in a recording means;
Before the step of recording the encoded stream in the recording means, temporarily holding the encoded stream in the stream holding means;
Silence detecting means for detecting silence from the audio data of the encoded stream held in the stream holding means;
Temporarily holding the silence information detected by the silence detection means in the temporary storage means as a transition point;
The CM broadcast section determination means continues two or more combinations of transition points where the interval of the silence information held in the temporary storage means falls within a predetermined time range centered on a preset reference time. Determining a section as a CM broadcast section;
Recording the time information of the CM broadcast section determined by the CM broadcast section determination means in the CM broadcast section information recording means. Recording a video and audio encoded stream in a recording means;
Before the step of recording the encoded stream in the recording means, temporarily holding the encoded stream in the stream holding means;
A step of detecting a video scene change from video data of an encoded stream held in the stream holding unit;
Temporarily holding scene change information detected by the video change point detection means in a temporary storage means;
The CM broadcast section determination means continues two or more combinations of transition points where the interval of the scene change information held in the temporary storage means falls within a predetermined time range centered on a preset reference time. Determining a section to be a CM broadcast section;
Recording the time information of the CM broadcast section determined by the CM broadcast section determination means in the CM broadcast section information recording means. The step of determining the CM broadcast section includes:
The transition point interval determined immediately before by the CM broadcast segment determination unit when a plurality of the transition points are detected within a predetermined time range centered on the reference time used by the CM broadcast segment determination unit. Is longer than the reference time by the first value, the CM broadcast section determination means sets the transition point interval determined by the CM broadcast section determination means next to the reference time by the first value. Select the transition point to be shortened preferentially as the transition point for defining the CM broadcast section,
A transition point interval determined immediately before by the CM broadcast section determination means when a plurality of the transition points are detected within a predetermined time range centered on the reference time used by the CM broadcast section determination means. Is shorter than the reference time by the second value, the CM broadcast section determination means sets the transition point interval determined by the CM broadcast section determination means next to the reference time by the second value. 13. The video / audio recording method according to claim 12, further comprising a step of preferentially selecting a transition point to be lengthened as a transition point for defining a CM broadcast section. The step of determining the CM broadcast section includes the step of determining the CM broadcast section when a plurality of transition points are detected within a predetermined time range centered on the reference time used by the CM broadcast section determination means. The transition point at which the time interval from the first transition point of the CM broadcast section to each detected transition point is the closest to an integral multiple of the reference time is the transition point for defining the CM broadcast section. The video / audio recording method according to claim 12, further comprising a determining step. Recording a video and audio encoded stream in a recording means;
Before the step of recording the encoded stream in the recording means, temporarily holding the encoded stream in the stream holding means;
The silence detection means detects the silence occurrence time and the silence time length from the encoded stream audio data held in the stream holding means, and determines that silence has been detected when the silence time length continues for a specified time. Steps,
Temporarily storing the detected silence information as a transition point in a temporary storage means;
A combination of transition points in which the interval between transition points including silence information held in the temporary storage means is within a predetermined time range centered on a preset reference time is continuous. Determining a CM broadcast section as a CM broadcast section and determining that the CM continues if the silent time length of the third and subsequent transition points is within the second specified range;
Recording the time information of the CM broadcast section determined by the CM broadcast section determination means in the CM broadcast section information recording means.   If there is no transition point whose silent time length is within the third specified range among the transition points of the CM broadcast section determined by the CM broadcast section determination means, the time information of the CM broadcast section is displayed as the CM broadcast section. 18. The video / audio recording method according to claim 17, wherein in the step of recording in the information recording means, the CM broadcast section is not recorded in the CM broadcast section information recording means.   In the step of determining the CM broadcast section, the detected transition point is determined as a CM broadcast section from the start of recording a stream to a transition point that occurs at the end within a specified time. The video / audio recording method according to any one of claims 12 to 18.

Images