JP2008301340A - Digest creating apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate digest contents in which a playback speed of a scene in which a title is being displayed or being played back as audio, is changed in accordance with the scene, in video contents. <P>SOLUTION: The number of characters in title data contained in video contents, a title display starting time to start displaying a title, and a title display ending time to end displaying the title are detected. Furthermore, regarding audio data contained in the video contents, its kind is determined and its fundamental frequency, volume, an audio starting time to start playing back audio, and an audio ending time to end playing back the audio are detected. A playback speed of the video contents is determined from the number of title characters, the title display starting time, the title display ending time, the fundamental frequency, the volume, the audio starting time and the audio ending time. On the basis of the determination, digest contents of the video contents are generated. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a video content digest creation device, and more particularly to a digest creation device that summarizes content such as a television program and generates digest content for efficient viewing.

  With the generalization of digital broadcasting and Internet broadcasting and the widespread use of DVD (Digital Versatile Disk) recorders and the like, long-time recording of TV programs and so-called time-shifted reproduction are generally performed. However, since the user's television viewing time is limited, it is not always possible to secure time for viewing all recorded television programs. In order to view a recorded television program in a short time, if the content of the television program is simply played back at a high speed, both important scenes and other scenes will be played back at high speed uniformly. With this method, the user may not be able to fully understand the content. Therefore, for a user, when viewing a recorded television program, an important issue is how to balance viewing time reduction and accurate grasp of the program contents.

As a technology for this purpose, Patent Documents 1 to 4 extract only scenes judged to be important by analyzing data such as video, audio, and subtitles included in the contents, and play back these important scenes by connecting them. A technique for creating a digest, or a technique for easily allowing traffic between scenes by a user operation is shown. Patent Document 5 discloses a technique for reproducing a scene determined to be important at a normal speed and reproducing a scene that is not so at high speed. Patent Document 6 discloses a technique for temporarily stopping playback of a scene including a subtitle according to the number of subtitle characters in order to prevent the user from losing the understanding of the subtitle content due to high-speed playback.
Japanese Patent Laid-Open No. 2000-23062 Japanese Patent Laid-Open No. 2002-344871 JP-A-2005-115607 Japanese Patent Laid-Open No. 11-55603 Japanese Patent Laid-Open No. 2005-252372 JP-A-9-200700

  In the case of content such as dramas, the scenes where the lines enter are often important. For this reason, in order to efficiently view the entire content of this kind in a short time, the part of the speech can be played at a speed as close to normal as possible to make it easy to hear the speech, and the rest of the content can be played as fast as possible. preferable. However, since the user must be able to understand the content even during high-speed playback, the playback speed naturally has an upper limit. In order to view the content more efficiently in a short time, the scene where the speech enters is played at the highest speed within the range where the user can hear the content of the speech, and the scene where the speech does not enter is a speed that can understand the development of the scene. Therefore, it is important to play at a higher speed than the scene where the dialogue enters.

  However, in the technique shown in Patent Document 4, a scene containing a speech is extracted, but how to reproduce the scene is left to the user's operation. For this reason, the user must specify whether to perform high-speed playback, normal playback, or skipping, and the operation becomes complicated.

  In the technique disclosed in Patent Document 6, the playback speed is controlled based only on the number of subtitle characters, and thus the playback speed is unconditionally reduced for long subtitles. Even if the speech speed is low and the playback speed can be increased, this technique is not suitable for efficient shortening of the viewing time because the playback speed decreases if there is a long subtitle.

  In view of these problems, the present invention creates a digest with a playback speed changed for each scene of content within a range in which the user can understand the content, thereby reducing the overall viewing time and efficient content viewing. It is a problem to realize.

  In order to solve this problem, a digest creation device according to the present invention is a digest creation device that generates digest content summarizing video content, and detects the number of subtitle characters and subtitle display time displayed in the video content. A subtitle analysis unit, a scene extraction unit that calculates a speech speed from the number of subtitle characters and the subtitle display time, and determines a playback speed of the video content at the subtitle display time based on the speech speed, and the scene extraction unit And a digest generation unit that generates digest content according to the playback speed determined by.

  Here, the digest generation unit may generate digest content that is played back at a constant speed except for the caption display time.

  Here, the digest generation unit may generate digest content to be played back at a higher speed according to the elapsed time from the start of the video content except for the caption display time.

  Here, the digest generation unit may generate digest content that reproduces only the caption display time at the reproduction speed determined by the scene extraction unit.

  In order to solve this problem, a digest creation device according to the present invention is a digest creation device that generates digest content summarizing video content, and detects the number of subtitle characters and subtitle display time displayed in the video content. A subtitle analysis unit; an audio analysis unit that analyzes audio of video content reproduced during the subtitle display time and detects at least one of a fundamental frequency and a volume; and an audio reproduction time; the number of characters of the subtitle; and audio A scene extraction unit that calculates an utterance speed from a reproduction time and determines a reproduction speed of video content in the audio reproduction time based on at least one of the fundamental frequency and the volume and the utterance speed; and the scene extraction unit A digest generation unit that generates digest content according to the determined playback speed. .

  Here, the digest generation unit may generate digest content that is played back at a constant speed except for the audio playback time.

  Here, the digest generation unit may generate digest content to be played back at a higher speed according to the elapsed time from the start of the video content except for the audio display time.

  Here, the digest generation unit may generate digest content that reproduces only the audio reproduction time at the reproduction speed determined by the scene extraction unit.

  Here, the scene extraction unit may determine the playback speed of the video content during the caption display time based on the utterance speed and the genre of the video content.

  Here, the scene extraction unit may determine the playback speed of the video content during the audio playback time based on at least one of the fundamental frequency and volume, the speech rate, and the genre of the video content.

  Here, the scene extraction unit determines the playback speed of the video content during the audio playback time based on at least one of the fundamental frequency, the volume, the speech rate and the audio type of the audio playback time. Also good.

  Here, the number of characters of the subtitle may be the number of characters of the kana when all the characters of the subtitle are expressed in kana.

  Here, an input processing unit may be provided that detects the number of subtitle characters from subtitles superimposed on a video signal included in the video content and transmits the detected number of subtitle characters to the subtitle analysis unit.

  Here, the scene extraction unit may determine a playback speed corresponding to the speech speed of each stage based on the speech speed divided into a plurality of stages.

  Here, the scene extraction unit may determine the corresponding playback speed based on the speech speed divided into a plurality of stages, the fundamental frequency divided into the plurality of stages, and the volume divided into the plurality of stages. Good.

  According to the present invention as described above, the digest content can be created by calculating the playback speed suitable for the emotion and excitement included in the speech for each scene of the video content. As a result, the user can not only watch the video content in a short time, but also can prevent the overlooked scene from being missed or missed, and can efficiently view the video content with an accurate understanding of the content of the video content.

(First embodiment)
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing a configuration of a digest creation apparatus according to the present embodiment. The digest creation device 1 includes a storage unit 11, an input processing unit 12, a caption analysis unit 13, an audio analysis unit 14, a scene extraction unit 15, a digest generation unit 16, a control unit 17, and an output interface (I / F) 18. ing.

  The storage unit 11 stores, for example, video content data such as a TV program and a movie, a subtitle table, an audio table, a playback speed table, a digest table, and digest content that summarizes video content, which will be described later. Specifically, the storage unit 11 is a semiconductor memory, a hard disk drive, an optical disk drive, or other storage device.

  The input processing unit 12 receives video content data from a broadcast wave or a network represented by the Internet by wire or wireless, and stores it in the storage unit 11. The input processing unit 12 includes a tuner, a network adapter, and the like. Note that the video content data may be received by the input processing unit 12 via a bus such as USB or IEEE 1394, or via a removable medium such as a hard disk or a memory card.

  The subtitle analysis unit 13 analyzes the video content data stored in the storage unit 11, and displays the subtitle display start time at which each subtitle included in the video content data starts, the subtitle display start time at which subtitle display ends, and the number of subtitle characters Are detected and calculated, and the subtitle display start time, the subtitle display end time, and the number of subtitle characters are stored in the storage unit 11 as a subtitle table to be described later. The subtitle display start time to the subtitle display start time is called subtitle display time.

  The audio analysis unit 14 refers to the caption table stored in the storage unit 11 and analyzes the audio data of the video content data stored in the storage unit 11 to generate an audio table described later. The sound analysis unit 14 classifies the sound data reproduced during the caption display time held in the caption table into several sound genres. Furthermore, the voice analysis unit 14 detects a voice start time Ts at which the voice or sound effect is reproduced, a voice end time Te to end, a fundamental frequency of the voice, and a volume. The voice genre, voice start time, voice end time, fundamental frequency, and volume are recorded in the storage unit 11 as a voice table to be described later. Note that the period from the voice start time to the voice end time is called a voice playback time.

The scene extraction unit 15 extracts the audio start time Tss and the audio end time Tse from the audio table stored in the storage unit 11. Further, the scene extraction unit 15 extracts the number Nc of subtitle characters displayed in the audio playback time from the subtitle table stored in the storage unit 11. Then, using the number of subtitle characters Nc and the length of time during which the voice from the voice start time Tss to the voice end time Tse is played, the speech speed Sv at each voice playback time is calculated by the following formula.
Sv = (Tse−Tss) / Nc
Further, the scene extraction unit 15 uses the playback speed table to specify the playback speed for each voice playback time based on the speech speed and the basic frequency and volume held in the voice table, and generates a digest table to be described later. The data is stored in the storage unit 11.

  The digest generation unit 16 refers to the digest table stored in the storage unit 11 to generate content digest data, and records the generated digest data in the storage unit 11.

  The control unit 17 controls each operation of the caption analysis unit 13, the audio analysis unit 14, the scene extraction unit 15, and the digest generation unit 16. The control unit 17 further reads digest data from the storage unit 11 as appropriate, and outputs the digest to the outside through the output interface 18.

  Here, an MPEG2 transport stream is assumed as a form of video content data in the present embodiment. FIG. 2 is a diagram showing a data structure of the MPEG2 transport stream in the present embodiment. The video content data includes a subtitle stream made up of subtitled PES (Packetized Elementary Stream), an audio stream made up of audio PES, a video stream made up of video PES, a data stream of data attached to content data, and a PCR (Program Clock Reference) stream. Contains. PCR is time information that is a reference time on the transmitter side and also serves as a reference for a time stamp. The time for displaying or outputting the actual data of subtitles, audio, and video included in the video content data is described in each PES as a time stamp. In this figure, subtitle PES has subtitle data and time stamps written therein, audio PES has audio data and time stamps written therein, and video PES has video data and time stamps written therein. Each of the above streams is multiplexed to constitute one transport stream. The details of the MPEG2 transport stream and each PES are defined in ISO / IEC13818-1 MPEG2 SYSTEMS.

  When playing back video, audio, and subtitles from the MPEG2 transport stream, the content playback device sets the internal clock to the time described in PCR. Then, the content playback apparatus separates each PES of video, audio, and subtitles from the stream, and when the time of the internal clock becomes the time of the time stamp described in each PES, the data described in the PES is Decode and display or output.

  Next, caption PES will be described. FIG. 3 is a diagram illustrating a configuration of a caption PES included in a caption stream of video content data. One caption PES includes a time stamp and caption data. The caption data includes partial caption data including character data and partial caption data including control data for controlling decoration and display of the character data. For example, the caption PES 20 in FIG. 3 includes a time stamp 21 and caption data 22, and the caption data 22 includes partial caption data 23 to 26. The time stamp 21 describes the time when the display of the character data included in the caption PES 20 is started. Part subtitle data 23 holds a "Hello" as the character data to be displayed as subtitles. The partial caption data 24 holds “TIME” and “10” which is the display maintenance time Td as control data. “TIME” instructs to maintain the display of the character data included in the previous partial caption data 23, and “10” indicates the length of time for maintaining the display. The partial caption data 25 holds “CS” as control data. “CS” instructs to delete the display of the displayed subtitles. The partial caption data 26 holds “Thank you” as character data to be displayed as a caption. Since the subtitle PES data is read sequentially from the left in the figure, the time stamp 21 to the partial subtitle data 26 are read in order and executed in order.

  A procedure for executing the contents of the caption PES will be described below. In the case of the caption PES 20 shown in FIG. 3, first, the time at which the caption data becomes valid is calculated with reference to the PCR in the video content data and the time stamp 21 in the caption PES 20. Then, to display the character data "Hello" in the portion subtitle data 23 at the time that the calculated. "Yes and, further to the next part subtitle data 25" The next part subtitle data 24 to the control data "TIME" "10 from the phrase that CS", the display of the first character data "Hello" and maintained for 10 seconds, display Erase 10 seconds after the start time. Then, to display the end time of the character data "Hello", to display the character data of the next part subtitle data 26, "Thank you". The display end time of “Thank you” is the display start time of the character data “Gokugenyo” included in the partial subtitle data 33 of the next subtitle PES30. This time is described in the time stamp 31 of the caption PES 30.

  Next, the caption table will be described. FIG. 4 is a diagram illustrating a caption table according to the present embodiment. The caption table holds a caption display start time, a caption display end time, and the number of caption characters for each character data included in one partial caption data. For example, in FIG. 1, subtitle display start time 00:01:15, subtitle display end time 00:01:22, and subtitle character count 3 are held for certain character data.

  Next, the voice table will be described. FIG. 5 is a diagram showing an audio table according to the present embodiment. The audio table holds the genre of audio data, the audio playback start time, the audio playback end time, the audio fundamental frequency, and the volume that are played between the subtitle display start time and the subtitle display end time of the subtitle table. ing. In the caption table of FIG. In the audio data reproduced between the subtitle display start time 00:01:15 shown in FIG. 1 and the subtitle display end time 00:01:22, No. 1 whose audio genre is “SPC” in FIG. No. 1 with the same audio genre as “EFF”. There are two data. Here, the audio genre is “APP” for cheering / going, “EFF” for sound effects, “SCR” for screaming, “SPC” for commentary / speaking, “MSC” for music. ". Note that the classification of the audio genre does not necessarily have to be as described above, and may be a classification that clearly indicates whether or not there is a possibility that a speech is included.

  In the subtitle table of FIG. The data corresponding to the caption data at the caption display start time 00:01:48 shown in FIG. 2 does not exist in the audio table in FIG. 5 because there is no audio data corresponding to the caption display.

  Next, the playback speed table will be described. FIG. 6 shows a playback speed table of the present embodiment. In this figure, one playback speed is designated by the combination of the fundamental frequency and volume of the audio data and the speech rate. For example, if the basic frequency of audio data is high and the volume is high, listening is likely to be difficult if the speech speed is high, so the playback speed is designated as the normal playback speed of 1.0 times speed, and playback is performed if the speech speed is low. Since it is easy to hear even if the speed is increased, the playback speed is set to 1.5 times speed. In this way, the playback speed table is for designating the playback speed for each voice playback time from the fundamental frequency, volume, and speech speed.

  Next, the digest table will be described. FIG. 7 is a diagram showing a digest table of the present embodiment. This figure is a table in which the parts for which the reproduction speed is specified are summarized based on the reproduction speed table of FIG. FIG. Four portions 1 to 4 are shown as portions where the playback speed is specified. The portion from 1 audio start time 00:01:15 to playback end time 00:01:19 indicates that playback is performed at 2.0 times speed.

  The operation of the digest creating apparatus 1 having such a configuration will be described with reference to the flowchart shown in FIG. First, the input processing unit 12 receives video content data (S11) and records it in the storage unit 11. Next, the caption analysis unit 13 analyzes the video content data to generate a caption table (S12), and records the caption table in the storage unit 11. Subsequently, the audio analysis unit 14 analyzes the audio data in the video content data only for the caption display time described in the subtitle table, generates an audio table (S13), and records the audio table in the storage unit 11. . The scene extraction unit 15 extracts the voice start time, the voice end time, the fundamental frequency, and the volume, which are data in the voice table, and calculates the speech rate. Further, the scene extraction unit 15 refers to the playback speed table based on the data of the voice table and the speech speed, and designates the playback speed for the voice playback time. A digest table holding the voice start time, voice end time, and playback speed at this time is generated (S14) and stored in the storage unit 11. Thereafter, the digest generation unit 16 refers to the digest table to generate digest content, and stores the generated digest content in the storage unit 11 or outputs the digest content to the outside via the output interface 18 (S15).

  Hereinafter, the operation of the caption analysis unit 13 in S12 of FIG. 8, the operation of the audio analysis unit 14 in S13, and the operation of the scene extraction unit 15 in S14 will be described in detail.

<Operation of Subtitle Analysis Unit 13>
9A and 9B are flowcharts illustrating the operation of the caption analysis unit 13. The caption analysis unit 13 first analyzes the data of the video content (video stream) shown in FIG. 2 to search for and acquire the first and second PCRs. Then, the stream start reference time is approximately calculated from the time difference between these two PCRs, the position difference within the stream, and the position within the stream of the first PCR (S21). That is, assuming that the times of the first and second PCRs are Tp1 and Tp2, respectively, and the positions in the stream are Pp1 and Pp2, respectively, the head reference time Ts is obtained by the following equation.
Ts = (Tp1 * Pp2-Tp2 * Pp1) / (Pp2-Pp1)

  In the following description, partial subtitle data including character data is distinguished using a count value n (n: integer). That is, when the counter value n = 0, it indicates that character data has not been detected yet, and the count value n = 1 is given to the first detected character data. Using the count value n, the display start time and display end time of each character data are represented by Tsc (n) and Tce (n), respectively.

When the head reference time Ts is determined, the caption analysis unit 13 searches for the caption PES in the video content data in order from the head of the stream (S22). Subsequently, it is determined whether or not a caption PES is detected (S23). If the subtitle PES20 in FIG. 3 is detected as the first subtitle PES, the time stamp Tpts described in the time stamp 21 in the subtitle PES20 is extracted, and the analysis time Tcc that is a candidate for the display start time and the display end time. The initial value is calculated (S24). The initial value of the in-analysis time Tcc is obtained from the difference between the time stamp Tpts of the caption PES 20 and the head reference time Ts by the following formula.
Tcc = Tpts-Ts

  If the time stamp Tpts is not specified in the detected caption PES, the first caption is validated based on the position of the caption data in the video content data based on the PCR and the start reference time Ts. The time may be calculated and used as the analysis time Tcc.

  Next, partial subtitle data 23 is first read from the subtitle data 22 of the detected subtitle PES20 (S25). Then, the read partial subtitle data 23 is analyzed to determine whether data is included (S26). If no data is included, the process returns to step S22 to search for the next caption PES.

  It is determined whether the data of the partial caption data 23 is character data (S27). Since the data of the partial caption data 23 is character data, the number of characters of the character data is calculated and held as the number of characters Nc (S28). Subsequently, it is determined whether or not the counter value n is greater than 0 and the display end time Tce (n) has not been determined (S29). Here, since the counter value is the initial value n = 0, the counter value is incremented to n = 1, and the analysis time Tcc is set as the display start time Tcs (1) (S31).

  Subsequently, the next partial caption data 24 is read (S25). It is determined whether or not the data of the partial caption data 24 is “TIME” (S32). Since the partial caption data 24 includes “TIME” as the control data and “10” as the display maintenance time Td, it is updated by adding 10 seconds of the display maintenance time Td to the analysis time Tcc (S33). Next, partial subtitle data 25 is read (S25). It is determined whether or not the data of the partial caption data 25 is “CS” (S34). Since the partial subtitle data 25 is “CS” which is control data, the process proceeds to the next step, and it is determined whether the counter value n is greater than 0 and the display end time Tce (n) has not been determined (S35). ). Since the counter value is n = 1 and the display end time Tce (1) is not yet determined, the analysis time Tcc is set as the display end time Tce (1), and the retained character count Nc is the character count Nc (1). (S36). Up to this point, the display start time Tcs (1), the display end time Tce (1), and the number of characters Nc (1) are determined for the counter value n = 1.

  Subsequently, the next partial caption data 26 is read (S25). Since the partial caption data 26 is character data, the number of characters of the character data is calculated and held as the number of characters Nc (S28). It is determined whether or not the counter value n is greater than 0 and the display end time Tce (n) has not been determined (S29), the counter value is n = 1, and the display end time Tce (1) has already been determined. Therefore, the counter value is incremented to n = 2, and the analysis time Tcc is set to the display start time Tcs (2) (S31).

  Subsequently, partial subtitle data is read (S25), and it is determined whether there is data (S26). Since there is no partial caption data following the partial caption data 26 in the caption data 22, it is determined that there is no data, and the process returns to S22 to search for the next caption PES (S22). Subsequently, it is determined whether or not a caption PES is detected (S23). When the next caption PES30 is detected, the time stamp Tpts described in the time stamp 31 of the caption PES30 is extracted, and the analysis time Tcc is calculated ( S24). Next, partial subtitle data 33 is read from the subtitle data 32 of the detected subtitle PES 30 (S25). The read partial subtitle data 33 is analyzed to determine whether data is included (S26), and it is determined whether the data of the partial subtitle data 33 is character data (S27). Since the partial subtitle data 33 includes character data, the number of characters of the character data is calculated and held as the number of characters Nc (S28). Subsequently, it is determined whether or not the counter value n is greater than 0 and the display end time Tce (n) has not been determined (S29). Here, since the counter value is n = 2 and the display end time Tce (2) is undecided (YES in S29), the analysis time Tcc is set as the display end time Tce (2), and the number of characters held Nc is the number of characters Nc (2) (S30). Up to this point, the display start time Tcs (2), the display end time Tce (2), and the number of characters Nc (2) are determined for the counter value n = 2. Subsequently, the counter value is incremented to n = 3, and the analysis time Tcc is set to the display start time Tcs (3) (S31).

  Subsequently, partial subtitle data is read (S25), and it is determined whether there is data (S26). Since there is no partial subtitle data following the partial subtitle data 33 in the subtitle data 32, it is determined that there is no data, and the process returns to S22 to search for the next subtitle PES (S22). By repeating such processing, the caption PES 40, the caption PES 50, and all subsequent captions PES are similarly analyzed, and the display start time, the display end time, and the number of caption characters of the caption table are sequentially determined.

  When the analysis of all subtitles PES is completed and there are no more subtitles PES to be detected, it is determined whether the counter value n is greater than 0 and whether the display end time Tce (n) is undetermined (S37). Since the current counter value n is n> 0 and the display end time Tce (n) is undecided, the end time of the stream is substituted for the display end time Tce (n), and the retained character count Nc is the number of characters. Nc (n) is set (S38). Since the display start time, the display end time, and the number of subtitle characters have been determined for the final character data, the subtitle display start time Tcs (k) (k is an integer from 1 to n) and the display end time Tce. (K) and the number of subtitle characters Nc (k) are recorded in the storage unit 11 (S39), and the series of subtitle table generation flow is terminated. If the current counter value n is n> 0 and the display end time Tce (n) is not yet determined in S37, the caption table is immediately recorded in the storage unit 11 (S39), and a series of caption table generation is performed. End the flow.

  Note that, when calculating the number of subtitle characters, the subtitle analysis unit 13 may detect the actual number of characters by performing kana conversion in the case of kanji. For example, in the case of character data “I'm fine”, there are 4 characters. However, if the reading is converted to kana, it becomes “genki” and becomes 5 characters. In this case, the number of subtitle characters in the character data is considered to be 5.

<Operation of the voice analysis unit 14>
FIG. 10 is a flowchart showing the operation of the voice analysis unit 14. First, the audio analysis unit 14 sequentially reads the caption display start time Tcs and the display end time Tce from the first data, starting from the first data (S51). Subsequently, it is determined whether or not all data of the caption table has been read (S52). The display start time Tcs detected from the caption table data is substituted into the analysis time Tcc (S53). Then, the audio data of the audio PES corresponding to the position of the analysis time Tcc in the video content data is analyzed, and the type of audio (audio genre), utterance end time, fundamental frequency, and volume are detected (S54).

  Here, the voice analysis compares the voice data with the voice waveform of the voice data sample of each voice genre (support / 喝采, sound effects, screaming, commentary / speech, music, etc.), and classifies them into genres with similar waveforms. It is common.

  The analysis result of S54 is stored in the storage unit 11 as voice table data (S55). It is determined whether or not the audio end time obtained in S54 exceeds the display end time Tce indicated in the caption table (S56). If the display end time Tce is exceeded, the process returns to S51 and the next data in the caption table. The display start time Tcs and the display end time Tce are read out. On the other hand, if the display end time Tce is not exceeded, the voice end time obtained in S54 is set as the analyzing time Tcc (S57), and the process returns to step S54.

<Operation of Scene Extraction Unit 105>
FIG. 11 is a flowchart showing the operation of the scene extraction unit 105. First, the scene extraction unit 15 reads the first audio genre, audio start time, audio end time, fundamental frequency, and volume from the audio table (S61). Subsequently, it is determined whether all data in the voice table has been read (S52). If the data of the voice table has been read to the end, the process ends. However, since only the first data has been read, the process proceeds to S63. It is determined whether or not the read sound genre is other than the sound effect “EFF” or the music “MSC” (S63). If the sound genre is the sound effect “EFF” or the music “MSC”, the process returns to step S61 to read the next data. . On the other hand, if the read audio genre is other than the sound effect “EFF” or music “MSC”, the read audio start time and audio end time are stored in the storage unit 11 as digest table data (S64). That is, the period from the voice start time to the voice end time when the voice genre is a sound effect or music is not regarded as a period with speech and is not a target to be described in the digest table.

  Next, the number Nc of subtitle characters from the audio start time to the audio end time read in S61 is read from the subtitle table, and the read subtitle character number Nc is divided by the period length, or the period length is divided by the read subtitle character number Nc. As a result, the speech rate is calculated (S65).

  Further, referring to the playback speed table, the playback speed is extracted based on the calculated speech speed and the basic frequency and volume read in S61, and this playback speed is added to the voice start time and voice end time stored in S64. The data is stored in the storage unit 11 as digest table data (S66).

  Thereafter, the processing returns to S61, the next data is read from the voice table, and the same processing is repeated. In step S62, it is determined whether or not the voice table data has been read to the end. If the data has been read to the end, this process ends. At this time, the storage unit 11 stores the digest table shown in FIG.

  The digest generation unit 16 refers to the digest table generated through the above processing, generates digest content from the video content data stored in the storage unit 11, and stores the digest content in the storage unit 11. The digest content at this time is to reproduce the portion described in the digest table at a specified reproduction speed. On the other hand, the part not described in the digest table is not played at all, is played at a constant speed, is played at a higher speed as the time of the part is longer, and is played at a higher speed as the program progresses. is there.

  Note that the digest of the present embodiment is to reproduce a part other than the part described in the digest table at a constant speed, but it is also possible to extract only the part described in the digest table and generate the digest content. This makes it possible to generate digest content for a shorter time. Furthermore, the playback speed of the described part can be regarded as an index indicating the importance, and only the part with a low playback speed can be extracted as the important part to generate the digest content.

  According to the digest creation device of the present embodiment, digest content with a playback speed suitable for a scene can be generated based on the fundamental frequency, volume, and speech rate. For this reason, the user can not only listen to the speech but also miss an important scene where the performer's emotions are uplifted, and can efficiently watch in a short time.

  In the playback speed table of the present embodiment, the basic frequency, volume, and speech speed parameters are divided into two levels, high, low, and large, and combinations are made, and the playback speeds corresponding to the combinations are designated. However, it is not limited to two stages and may be divided into more stages. Also, a playback speed table in which the respective combinations are changed may be used, or a playback speed table in which priorities are assigned to parameters, and values are determined in order from parameters with higher priorities to specify playback speeds.

(Second Embodiment)
Hereinafter, a second embodiment of the present invention will be described with reference to the drawings. FIG. 12 is a diagram showing a configuration of the digest creation device according to the present embodiment. The digest creation device 2 includes an input processing unit 12, a digest generation unit 16, an output interface (I / F) 18, a storage unit 21, a caption analysis unit 23, a scene extraction unit 25, and a control unit 27. The input processing unit 12, digest generation unit 16, control unit 17, and output interface (I / F) 18 are the same as those shown in FIG. 1 of the first embodiment. Further, the caption table shown in FIG. 13 used in the following description is the same as that shown in FIG. 4 of the first embodiment.

  The storage unit 21 stores, for example, video content data such as a television program and a movie, a subtitle table, a playback speed table, a digest table, and digest content summarizing the video content, which will be described later. Specifically, the storage unit 21 is a semiconductor memory, a hard disk drive, an optical disk drive, or other storage device.

  The caption analysis unit 23 analyzes the video content data stored in the storage unit 21, and displays the caption display start time at which the display of each caption included in the video content data starts, the caption display end time at which the caption display ends, and the number of caption characters , And the caption display start time, caption display end time, number of caption characters, and speech rate are stored in the storage unit 21 as a caption table to be described later. The subtitle display time is from the subtitle display start time to the subtitle display start time.

The scene extraction unit 25 extracts the subtitle display start time Ts, the subtitle display end time Te, and the number of subtitle characters Nc from the subtitle table stored in the storage unit 21. Then, the speech rate Sv is calculated by the following formula using the number Nc of subtitle characters and the length of time during which subtitles are displayed from the subtitle display start time Ts to the subtitle display end time Te.
Sv = (Te−Ts) / Nc
Furthermore, the scene extraction unit 15 specifies the playback speed from the caption display start time to the caption display end time using the speech speed and the playback speed table, generates a digest table described later, and stores it in the storage unit 21. To do.

  The control unit 27 controls each operation of the caption analysis unit 23, the scene extraction unit 25, and the digest generation unit 16. The control unit 27 further reads the digest content from the storage unit 21 as appropriate and outputs it to the outside through the output interface 18.

  Next, the playback speed table will be described. FIG. 14 is a diagram showing a playback speed table of the present embodiment. In this figure, speaking speeds are classified into three, low, medium and high, and one playback speed is designated for each speaking speed. For example, if the utterance speed is high, the listening speed is likely to be difficult, so the playback speed is designated as the normal playback speed of 1.0 times speed, and if the utterance speed is medium, the playback speed can be heard even if the playback speed is slightly increased. Is specified at 1.5 times speed. Furthermore, if the speech rate is low and slow, the playback speed is specified as 2.0 times speed. Such a playback speed table is for designating the playback speed of the video content with respect to the speech speed.

  Next, the digest table will be described. FIG. 15 is a diagram showing a digest table of the present embodiment. This figure shows No. in the caption table of FIG. 1-No. 5 is a table obtained by adding a reproduction speed based on the reproduction speed table of FIG. In this table, for example, No. 1 shows that the subtitle display start time 00:01:15 to the subtitle display end time 00:01:22 is reproduced at 2.0 times speed.

  The operation of the digest creating apparatus 2 having such a configuration will be described below. First, the input processing unit 12 receives video content data and records it in the storage unit 21. Next, the caption analysis unit 13 analyzes the video content data, generates a caption table as in the first embodiment, and records the caption table in the storage unit 21. Subsequently, the scene extraction unit 105 extracts the subtitle display start time, the subtitle display end time, and the number of subtitle characters, which are information in the subtitle table, and calculates the speech rate. Furthermore, the scene extraction unit 105 detects a playback speed corresponding to the speech speed calculated with reference to the playback speed table, and designates the playback speed during the caption display time. At this time, the subtitle display start time, subtitle display end time, and playback speed are recorded in the storage unit 21 as a digest table.

  Thereafter, the digest generation unit 16 generates a digest content of the video content with reference to the digest table, and stores the generated digest in the storage unit 11 or outputs it to the outside via the output interface 18. As described above, the digest content can be generated based on the caption display time and the number of caption characters without analyzing the audio data.

  A playback speed table may be prepared for each program genre. For example, a playback speed table may be prepared for each genre of video content such as sports, dramas, and news, and the apparatus may use the playback speed table in accordance with the genre of the video content. Further, for example, a playback speed table may be prepared by dividing a genre into finer categories such as sports for baseball, soccer, sumo, etc., drama for suspense, SF, action, etc., and the playback speed table may be used according to user designation. Even in this case, the digest table described in the embodiment of the present invention can be generated, and a digest of video content can be generated.

  In addition, when the number of subtitle characters to be displayed is large, reading the subtitle makes it easier to understand the content, so the playback speed may be slightly higher than the predetermined playback speed. In the first embodiment, the playback speed is determined based on the fundamental frequency, the volume, and the speech speed. In the second embodiment, the playback speed is determined based on the speech speed. It may be used for speed determination.

  The video content data may be an analog AV signal or the like. In the case of an analog AV signal or when captions are superimposed on the video signal, the input processing unit 102 may extract caption data from the video frame by image recognition. Thereby, even when there is no subtitle stream, the subtitle display start time and display end time can be calculated. That is, the present invention provides the same effect as described above for the content of the analog video signal.

  The digest creation device according to the present invention is useful for DVD recording / playback devices, digital television devices, mobile phones, portable content players, car navigation devices, and the like.

It is a figure which shows the structure of the digest production apparatus in the 1st Embodiment of this invention. It is a figure which shows the data structure of the MPEG2 transport stream in the 1st Embodiment of this invention. It is a figure which shows the data structure of the caption stream in the 1st Embodiment of this invention. It is a figure showing the caption table in the 1st Embodiment of this invention. It is a figure showing the audio | voice table in the 1st Embodiment of this invention. It is a figure showing the reproduction speed table in the 1st Embodiment of this invention. It is a figure showing the digest table in the 1st Embodiment of this invention. It is a figure which shows the flowchart of operation | movement of the digest production apparatus in the 1st Embodiment of this invention. It is a figure which shows the flowchart of operation | movement of the caption analysis part in the 1st Embodiment of this invention. It is a figure which shows the flowchart of operation | movement of the caption analysis part in the 1st Embodiment of this invention. It is a figure which shows the flowchart of operation | movement of the audio | voice analysis part in the 1st Embodiment of this invention. It is a figure which shows the flowchart of operation | movement of the scene extraction part in the 1st Embodiment of this invention. It is a figure which shows the structure of the digest production apparatus in the 2nd Embodiment of this invention. It is a figure showing the caption table in the 2nd Embodiment of this invention. It is a figure showing the audio | voice table in the 2nd Embodiment of this invention. It is a figure showing the digest table in the 2nd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1, 2 Digest production apparatus 12 Input processing part 11, 21 Accumulation part 13 Subtitle analysis part 14 Voice analysis part 15, 25 Scene extraction part 16 Digest generation part 17, 27 Control part 18 Output I / F

Claims (15)

A digest creation device that generates digest content summarizing video content,
A subtitle analysis unit for detecting the number of subtitle characters and subtitle display time displayed in the video content;
A scene extraction unit that calculates the utterance speed from the number of subtitle characters and the subtitle display time, and determines the playback speed of the video content at the subtitle display time based on the utterance speed;
A digest creation device comprising: a digest generation unit that generates digest content according to the playback speed determined by the scene extraction unit.   The digest creation device according to claim 1, wherein the digest generation unit generates digest content that is played back at a constant speed except for a caption display time.   2. The digest creation device according to claim 1, wherein the digest generation unit generates digest content to be played back at a higher speed according to an elapsed time from the start of video content except for the caption display time.   The digest creation device according to claim 1, wherein the digest generation unit generates digest content that reproduces only the caption display time at the reproduction speed determined by the scene extraction unit. A digest creation device that generates digest content summarizing video content,
A subtitle analysis unit for detecting the number of subtitle characters and subtitle display time displayed in the video content;
An audio analysis unit that analyzes audio of the video content played during the caption display time and detects at least one of a fundamental frequency and a volume and an audio playback time;
A scene extraction unit that calculates an utterance speed from the number of characters of the subtitles and an audio playback time, and determines a playback speed of video content at the audio playback time based on at least one of the fundamental frequency and volume and the utterance speed When,
A digest creation device comprising: a digest generation unit that generates digest content according to the playback speed determined by the scene extraction unit.   The digest creation device according to claim 5, wherein the digest generation unit generates digest content that is played back at a constant speed except for a voice playback time.   The digest creation device according to claim 5, wherein the digest generation unit generates digest content to be played back at a higher speed in accordance with an elapsed time from the start of video content except for the audio display time.   6. The digest creation device according to claim 5, wherein the digest generation unit generates digest content that reproduces only the audio reproduction time at the reproduction speed determined by the scene extraction unit.   The digest creation device according to any one of claims 1 to 4, wherein the scene extraction unit determines a playback speed of the video content during the caption display time based on the utterance speed and the genre of the video content.   The said scene extraction part determines the reproduction speed of the video content in the said audio | voice reproduction time based on the at least 1 of the said fundamental frequency and a volume, the said speech rate, and the genre of the said video content. The digest creation apparatus according to item 1.   The scene extraction unit determines a playback speed of video content during the audio playback time based on at least one of the fundamental frequency and volume, the speech rate, and the type of audio during the audio playback time. The digest production apparatus of any one of these.   The digest creation device according to any one of claims 1 to 11, wherein the number of characters of the subtitle is the number of characters of the kana when all the characters of the subtitle are expressed in kana.   13. The input processing unit according to claim 1, further comprising: an input processing unit that detects a number of characters of the subtitle from a subtitle superimposed on a video signal included in the video content, and transmits the detected number of characters of the subtitle to the subtitle analysis unit. The digest creation device according to item. The scene extraction unit
The digest creation device according to claim 1, wherein a reproduction speed corresponding to the utterance speed of each stage is determined based on the utterance speed divided into a plurality of stages. The scene extraction unit
6. The digest creation device according to claim 5, wherein a corresponding playback speed is determined based on an utterance speed divided into a plurality of stages, a fundamental frequency divided into a plurality of stages, and a volume divided into a plurality of stages.

Images