JP4086886B2 - Movie playback apparatus, movie playback method and computer program thereof - Google Patents

Description

  The present invention relates to the field of moving image reproduction technology accompanied with audio reproduction.

  Conventionally, for example, in a video playback device with audio playback, such as a video tape recorder, a double speed playback function and high-speed fast-forward are provided so that the user can view the entire video in a short time when playback is performed. Functions are provided.

  In addition, in a video tape recorder that is a typical moving image playback device, in recent years, when executing double-speed playback of a recording medium, a first audio section in which sound energy is equal to or higher than a predetermined threshold and less than the predetermined threshold The second audio section is detected and continuously reproduced while performing pitch conversion of the audio signal in the first audio section, so that the reproduced audio is somewhat less for the user while eroding the second audio section. A technique that can reproduce a storage medium at a double speed while being accompanied by a reproduced sound that can be understood easily has been proposed.

  However, if the partial pitch conversion processing of the audio signal is performed as described above, the synchronized relationship between the audio and the video cannot always be maintained when the moving image is reproduced. And the reproduced sound cannot be synchronized, the reproduction is unnatural for the human sense, and the user may feel uncomfortable.

  Further, for example, in JP-A-10-32776, JP-A-9-243351, etc., a silence state is detected based on voice energy, and sounds other than the detected silence state are regarded as a voice section emitted by a person. Thus, a technique for summarizing moving pictures has also been proposed. However, in a moving image in which a voice uttered by a person is dominant throughout the entire program, such as a news program, background noise can be detected to a certain extent although detection of a voice section uttered by a person based on voice energy is possible. This method is not practical in an environment where music or background music exists.

  Furthermore, in the prior art prior to the above-mentioned patent publication, there have been proposed many techniques for performing sound detection and moving image reproduction considering the detected sound, most of which are performed by thresholding sound energy. Audio is detected. There is a problem caused by the ambiguity of the Japanese language. “Human voice” is also called “speech”, and general sounds including human voice are also called “speech”. It is inappropriate to collectively refer to threshold processing of sound energy in the prior art as true “voice detection”.

  In Japanese Patent Laid-Open No. 9-247617, a “singularity” is obtained by calculating an FFT (Fast Fourier Transform) spectrum of an audio signal, and a “feature point such as audio information” is detected, and its volume is analyzed. Technology has been proposed. However, in the method using the FFT spectrum, when a sound signal to be reproduced includes so-called background music having a broadband spectrum distribution, a voice uttered by a person is detected therefrom. It is difficult.

  As described above, in the conventional video playback with audio, there is a problem that the detection of the audio section is convenient and inaccurate as described above, and further, the summary of the video using the detection result and the double speed are generated. When performing playback, there is a problem that the synchronization relationship between video and audio cannot be maintained during playback.

  Further, in recent years, media in which information of utterance contents is multiplexed with moving image data and audio signals by so-called subtitles or close captions, or inserted in another region band has appeared. Even when such media is played back, when creating a summary of a moving image or using double speed playback using the detection result of the audio section, there is a problem that the synchronization relationship between video and audio cannot be maintained during playback.

  In general, it is known that it is not easy for a user such as an elderly person or a child to master various devices, and it is difficult to comprehend the contents of voices emitted at a high speed. Therefore, for such a user, the optimum conditions for reproduction differ from those of general users when performing quick viewing (shortened reproduction) of contents such as double speed reproduction in a moving image reproduction apparatus such as the above-described tape recorder.

  Furthermore, even for users with low dynamic visual acuity, users with low hearing ability for fast voices, or foreign users who do not use the voice to be played as their native language, the video playback device as described above allows quick viewing (shortening) of contents such as double speed playback. When performing (playback), the optimum conditions for playback differ from those of general users.

  Therefore, the present invention accurately detects a voice section generated by a person and maintains a synchronized relationship between video and audio faithfully according to the detected voice section, and greatly reduces a user's viewing time. An object of the present invention is to provide a moving image reproduction method and a computer program thereof.

  According to one aspect of the present invention, there is provided a moving image reproducing apparatus capable of reproducing moving image data including an audio signal and sub information at high speed, and represents a person's utterance period based on the sub information included in the moving image data. Based on the moving image data and the determination means for determining the first voice section and the second voice section other than the first voice section, the first voice section plays the reproduced voice at a predetermined speed at which the user can grasp the contents. While the high-speed moving image playback is performed, the second sound section is a fast-playing playback means for performing high-speed moving image playback at a speed higher than the predetermined speed, the length of the first sound section and the playback speed of the section, And a presentation means for calculating the required time required for the high-speed moving image playback based on the length of the second audio section and the playback speed of the section and presenting the calculated required time to the user. Play video Location is provided.

  According to another aspect of the present invention, there is provided a moving image reproduction method capable of reproducing moving image data including an audio signal and sub information at high speed, wherein a human speech period is determined based on the sub information included in the moving image data. Based on the determination step of determining the first voice section to be represented and the second voice section other than that, and the moving image data, the first voice section is reproduced at a predetermined speed at which the user can grasp the content. While the high-speed moving image reproduction is performed, the second audio section includes a quick-playing step of performing high-speed moving image reproduction at a speed higher than the predetermined speed, a length of the first audio section, and a reproduction speed of the section. And a presentation step of calculating a required time for the high-speed moving image playback based on the length of the second audio section and the playback speed of the section, and presenting the calculated required time to the user. Video re A method is provided.

  According to the present invention described above, a user's time required for browsing is greatly shortened while accurately detecting a voice section generated by a person and faithfully maintaining a synchronization relationship between video and audio according to the detected voice section. Provided is a moving image reproducing apparatus, a moving image reproducing method, and a computer program thereof.

  Hereinafter, an embodiment of a moving image reproducing apparatus according to the present invention will be described in detail with reference to the drawings.

  First, an outline of the operation of the moving image playback apparatus in the present embodiment will be described with reference to FIG.

  FIG. 1 is a diagram illustrating a conceptual diagram of a moving image quick-view algorithm in the moving image reproducing apparatus according to the present embodiment.

  As shown in FIG. 1, the moving image playback apparatus according to the present embodiment is roughly divided into a moving image quick index creation unit 100 and a moving image quick playback unit 200.

<Quick movie index creation unit 100>
In the moving image quick index creation unit 100, moving image data read from the moving image data storage unit 10 is converted into video data (video signal), audio data (audio signal), and sub information in the video / audio / sub information separation process (step S101). To be separated.

  When the audio signal is subjected to audio section reading processing (step S102) and audio section correction processing (step S103), scene change point information is not included in the sub information for the video signal. Then, a video change degree calculation process (step S106) and a scene change point detection process (step S107) are performed, and when the sub information includes scene change point information, the scene change point reading is performed. Processing (step S105) is performed. In the quick-play playback section correction process (step S104), fast-play playback section information is generated, and the generated information is stored in the moving image quick-view index storage unit 11.

  That is, in the audio section reading process (step S102), based on the information obtained by separating the moving image data in the video / audio / sub-information separation process (step S101), the “information about the content of human occurrence” and the “display timing” "Information" is read out from the moving image data as a result of reading the voice section. Here, the display timing information includes a display start timing, a display end timing, and a section length.

  In the voice segment correction process (step S103), based on the voice segment reading result, a new reproduction is performed by integrating a plurality of adjacent voice segments so that a person (user) is not uncomfortable at the time of voice reproduction. The corrected voice section information is acquired by correcting the voice section representing the person's utterance period (hereinafter referred to as “person's voice section” or section A).

  For example, as a bad aspect in high-speed video playback, when the interval between two adjacent sections A is narrow, at the time of video playback, the voice is played at such a speed that a person can hear and understand the contents. In addition to performing double-speed playback (for example, double-speed playback), the content of a section that is not a human voice section (hereinafter referred to as section B) can be grasped by viewing the playback video when the video is played back. When the playback is performed at a high speed of 2 times, the change is severe and it is difficult for a general user to hear.

  Therefore, in the present embodiment, in the speech segment correction process (step S103), the interval between the human speech segments is taken into account, and when the predetermined condition satisfies the certain interval, a plurality of speech segment groups are integrated. This eliminates the difficulty of hearing. Here, as the predetermined condition, for example, it is easiest to set that the interval of the human voice section is equal to or less than a predetermined threshold value.

  Further, in the video change degree calculation process (step S106), the video data obtained in the video / audio / sub-information separation process (step S101) is processed between frames described in Japanese Patent Laid-Open No. 2000-235539. Image change information is generated by calculating the similarity between frames by performing a similarity comparison process.

  Generally, when there is a video transition in video data including audio signals and the audio section starts immediately after that, the video at the beginning of the scene is played back at a high speed for a moment when playing back the video. Since the reproduced video by the double speed reproduction is performed at a speed that can be heard and grasped by a person, the user feels uncomfortable that the video flickers.

  Therefore, in the present embodiment, when scene change point information is included in the sub information, in the scene change point reading process (step S105), a scene change point group (scene change point information) is obtained from the sub information. When the scene change point information is not included, in the scene change point detection process (step S107), for example, the scene change point disclosed in the preceding Japanese Patent Application Laid-Open No. 2000-235539 is filed. By adopting the detection technique, a scene change point group (scene change point information) is detected based on the video change information obtained in the video change degree calculation process (step S106).

  Then, in the quick-play reproduction section correction process (step S104), when the distance is less than or equal to a predetermined threshold at a time earlier than the beginning of the voice section after the voice section correction process in step S103 Furthermore, by replacing the head of the voice section with information corresponding to the scene change point acquired in step S105 or step S107, the user's uncomfortable feeling can be removed.

  Since the processing in each of the above steps can be performed at extremely high speed, in the present embodiment, the moving image data with audio / sub-information read from the moving image data storage unit 10 is read when the moving image is reproduced by the moving image quick reproduction unit 200. The contents of the moving image data to be reproduced are stored in advance in a memory buffer (not shown) by acquiring the above-mentioned “information on the content of human occurrence” before the actual reproduction of the moving image. Without analysis, the process of generating the preview playback section information by the movie preview index creation unit 100 and the movie playback process of the movie preview playback unit 200 using the generated preview playback section information and movie data are performed on the entire movie (that is, , The content to be played back) is executed in pseudo real time (that is, executed by pseudo parallel processing) Chromatography The may be efficiently Hayami whole (full length), a short time of the desired video content.

<Quick movie playback unit 200>
Next, in the movie quick-view playback unit 200, the playback video is played back using the display 12 and the playback audio is played back using the speaker 13 in the movie fast-play playback process (step S107). At the time of moving image reproduction by the moving image quick reproduction process, the time required for reproduction is displayed in step S108 based on the quick reproduction section information read from the moving image quick index storage unit 11, and the step corresponding to the display is performed. By integrating the feedback of the user-desired playback condition set in S109 and the playback condition based on the user profile 14, final setting of the fast-playing playback condition is performed, and based on the set fast-playing playback condition. The moving image reproduction of the moving image data read from the moving image data storage unit 10 is performed.

At that time, in this embodiment,
-For section A, double-speed playback with voice is performed at a speed that allows the user to grasp the content when the user hears the played voice,
-For the section B, high speed double speed reproduction is performed within a range in which the user can grasp the contents by viewing the reproduced video.

  As a result of an experiment by the applicant of the present application, the double-speed playback in the above section A, that is, playback at a speed at which a person can hear and grasp the contents, is up to double speed, preferably about 1.5 times speed in the experiment. I know. On the other hand, for the section B, playback is performed at a high magnification rate within a range in which the user can see the content of the playback video, but according to the results of experiments by the applicant of the present application, it is empirically up to 10 times the speed. It has been found that it is better to set the speed to 5x or higher.

  It is known that when the section B is played back at high speed at a high speed, generally, a sound of “curl curl” is produced. Therefore, in step S107, when the section B is played back at high speed, the user does not want to hear such a sound. In some cases, the sound reproduction may be muted by muting, or the volume during reproduction may be reduced.

  Regarding the playback speed of the section A, the playback speed of the section B, and the volume at the time of playback, the simplest implementation method is to determine how to process the sound in advance in the moving image quick playback process (step S107). There is a method that allows the user to set the playback speed in a variable manner.

  However, in general, it is not easy for a user such as an elderly person or a child to master various devices, and it is known that it is difficult to understand the content when high-speed audio playback is performed, It is preferable to perform double speed reproduction at a slightly lower magnification without performing troublesome speed adjustment. Similarly, users with weak visual acuity regardless of age (visually impaired), especially dynamic visual acuity and hearing, particularly users with weak early hearing ability (hearing impaired), or reproduced speech in their native language. It is known that foreign users and the like who do not want to understand the content are difficult to understand when high-speed audio reproduction is performed, and there is an optimum reproduction speed for these users.

  Therefore, in the present embodiment, information such as the user's age and language, understandable language, visual acuity, hearing ability, and the like, and attribute information about the user such as a standard reproduction condition preferred by each user are stored in the user profile 14 in advance. In addition, by referring to the profile 14 in the quick video playback process (step S107), the voice section (section A) representing the human voice period and the human voice section are excluded according to the target user. It is possible to determine the playback speed of each section (section B) and perform quick movie playback that makes it easy to understand the content according to the individual.

  In addition, as described above, when the sound is muted or the volume is reduced during high-speed double-speed playback in the section B, it is possible to comfortably respond to individual users by describing such settings in the profile 14 in advance. It is possible to perform quick movie playback.

  Furthermore, for elderly people and users with handicaps in moving visual acuity, it may be out of the viewpoint of the original quick-playing, but the playback speed of the section A is set slower than the normal speed and the playback speed of the section B Is set to a speed equal to or higher than the normal speed, while the user concerned can perform low-speed playback capable of grasping the audio content of the section A, as a whole, in a short time required compared to the case of low-speed playback of all sections, It is possible to view a moving image (that is, moving image data stored in the moving image data storage unit 10).

  In addition, for users who are handicapped in understanding the content of early speech and users who are not fluent in the language of speech content, the playback speed of section A may be slower than the normal speed, although this may be out of the viewpoint of fast-playing. In addition to the setting, the playback speed of the section B is set to 10 times speed, preferably 5 times speed or more, and the slow playback that enables the user to grasp the audio content of the section A is performed, but the entire section is played back at a low speed as a whole. It is possible to view the moving image (that is, the moving image data stored in the moving image data storage unit 10) in a shorter time than the case. Here, whether or not the language of the audio content is proficient is determined based on the identification information stored in advance in the above-described profile 14 (special language in Table 4 described later), the language type information of the audio included in the video to be played back, This can be done by comparing.

  As a procedure for selecting the user profile 14, for example, a user profile list is displayed on the profile selection screen displayed on the display 12, and the user profile 14 is selected according to the operation of the remote control terminal (not shown) by the user. In addition, an automatic profile selection method using a personal recognition technique such as fingerprint, voiceprint, or face recognition may be employed.

  By the way, in the case of performing the fast-view playback optimal for each user as described above, what length of the original video can be viewed in advance is determined by the user who is going to make a quick look at the free time. This is important information.

  Therefore, in this embodiment, in step S108, the playback time of the section A is calculated by dividing the total length of the section A by the playback speed, and for the section B, the section B is calculated by dividing the total length by the playback speed. The playback speed is calculated, the sum of these two calculated values is obtained as the time required for quick viewing, and is presented to the user together with the time required for reproducing the original moving image at the same magnification. Furthermore, by designating the playback speed of the section A and the playback speed of the section B so that these playback times are within the desired playback time when the user sees them, the playback time becomes closer to the user's desired playback time. It is possible to adjust to.

  By the way, the time required for the quick movie playback that is automatically calculated using the profile 14 in step S108 as described above is related to the preset user profile 14 and the desired playback speed designated by the user. In step S109, the user who has watched further designates the playback speeds of the sections A and B in step S109, thereby obtaining the time (playback speed information) required for the desired movie quick-playback playback. If it is set, the previous operation information is stored in the profile by automatically storing the set playback speed information after confirming with the user automatically or within the set required time. This makes it possible to perform easy-to-understand moving image quick-play according to the preferences of individual users.

  Further, in the user profile described above, it is further specified in advance how to process the volume during playback of the section B, or if the user specifies through a predetermined man-machine interface, the specification is made. Thus, it is possible to perform easy-to-understand moving image quick-play according to individual user's preference while reflecting the volume information.

<Details of the operation of the video playback device>
Hereinafter, details of the operation of the moving picture reproducing apparatus according to the present embodiment outlined above will be described. In the following description, fast-playing playback section information is created and created as an index for fast-playing recorded video data (video data including audio signals and sub-information) stored in the video data storage unit 10. A case will be described as an example in which the information is used to perform quick-view playback of the moving image data.

  In the present embodiment, as described above, the post-processing after the video / audio separation process in step S101 is roughly divided into a video quick-look index creation process by the video quick-look index creation unit 100 and a video fast-play playback by the video fast-play playback unit 200. There is processing.

  The moving image data with audio / sub-information in the present embodiment is content in which video information, audio information, and sub-information are multiplexed. Examples of such information-type media include DVDs and digital TV broadcasts. Can be mentioned.

  In the present embodiment, the sub-information is various types of information different from video and audio itself, such as moving image segment information, scene change information, subtitle information, time information, and the like.

  In the present embodiment, in the following description, “information related to the content of human occurrence” is used as such sub-information, but there are, for example, captions, close captions, and the like. The phoneme string notation obtained from the recognition result of the voice can be adopted.

  Here, subtitles and close captions correspond to the audio content in sync with the video signal so that people with hearing disabilities and those who do not understand the language they are listening to can enjoy the content. The content including such sub-information has a voice section (section A) representing a person's utterance period manually or automatically or semi-automatically before providing the content. The content of the utterance of the person in the determined individual voice section is determined as additional information (“sub information” in the present embodiment) to the content by hand and / or by performing voice recognition processing. Generally described.

  Also, in such content, subtitles or close captions in a language type different from the audio content produced by the original person of the content, sub-information described in the content as described above, and further, Or, it is generally written after being translated into a target language by automatic translation.

  Such sub-information such as subtitles and close captions is generally accompanied by section information for indicating a display period at the time of video playback, and this section information includes a human voice section (section A). It can be understood as representing.

  Therefore, in the present embodiment, the audio section (section A) of the person included in the moving image data is detected for the moving image data with audio / sub information including the sub information in the above form.

  In the present embodiment, as described above, the post-processing after the video / audio separation process in step S101 is roughly divided into a video quick-look index creation process by the video quick-look index creation unit 100 and a video fast-play playback by the video fast-play playback unit 200. There is processing.

<Quick movie index creation unit 100>
(Detection of human voice segment)
FIG. 2 is a flowchart showing an outline of the human voice segment detection process performed by the moving image quick index creation unit 100 in this embodiment, and shows the detailed procedure of the voice segment reading process (step S102) described above. .

  In step S201, it is determined whether there is any video stream that has not been read yet in the video / audio / sub-information separation process performed in step S101 described above. finish.

  In step S202, since it is determined that there is a video stream that has not yet been read in step S201, the video stream is read into a buffer (not shown). In step S203, the sub information included in the read video stream is As information on the content of human occurrence, seek subtitles, close captions, phoneme string notation obtained as a result of speech recognition, or speech detection result information, and obtain the resulting information as a speech segment (speech segment information) ).

  Here, from among the read information, which information is selected as the voice section information in step S203 may be selected from the existing information with high accuracy of the contents, for example, subtitle> cropping. -Selection may be made in the order of caption> phoneme string notation> speech detection result information.

  Table 1 is a table exemplifying the sub information read as the voice section information. In this example, as the section A, the start time (start point) and end time (end point) of each section with respect to the sound generation sections 0 to 2 are shown. Is read as pair information.

(Correction of human speech section)
Details of the above-described speech segment correction processing (step S103) will be described. In step S103, correction is performed by integrating a plurality of sound sections located in the vicinity on the time axis as one sound section so that the user who listens to the reproduced sound at the time of moving image quick reproduction does not feel uncomfortable.

  Here, the reason for correcting the speech section information acquired by the above-described speech section detection processing (FIG. 2) is that, for example, when the interval between two sections A located in the vicinity on the time axis is narrow, the section A When the playback is performed at high speed and high speed within the range that allows the person to grasp the content by watching the playback video, while performing the double speed reproduction with the sound at a speed at which the person can grasp the content by listening to This is because the reproduction mode changes drastically and is difficult for the user to hear.

  In addition, from the viewpoint of video decoder and playback processing, as an example, the change in speed in a short section has a large processing overhead, and the playback operation temporarily stops, resulting in jerky playback. In addition to being observed in an experiment by the applicant of the present application using the company's DirectShow, the same phenomenon can be seen in many other video playback means.

  Therefore, in the present embodiment, when the interval between two voice segments (section A) located closest on the time axis is equal to or smaller than a threshold value (Th3 in FIG. 3), these voice segments are integrated. The correction by doing. In determining this threshold value, for example, a scene in which conversation is performed is assumed, and the extent to which the conversation is established is experimentally determined and used as the threshold value. The processing procedure in this case will be described with reference to FIG.

  FIG. 3 is a flowchart showing an integrated correction process performed for a voice interval with a short interval in the present embodiment.

  In the figure, in step S301, among the plurality of previously detected sections A, the section A first positioned on the time axis is read as the speech section of interest. If there is no speech section of interest, this processing is performed. Ends.

  In step S302, it is determined whether or not there is a speech section to be focused on next (section A). If there is no speech section to be focused on, this processing is terminated. The processes from S303 to S307 are repeated.

  In step S303, since it is determined in step S302 that there is a voice section of interest next, voice section information representing the voice section (section A) is read. Here, the voice section information is information in which the start point and end point of the voice section are paired.

  In step S304, the interval between the two sections A, that is, the distance (time interval) between the end point of the preceding voice section (currently focused voice section) and the start point of the next voice section on the time axis. To determine whether this distance is equal to or smaller than a predetermined threshold value Th3.

  In step S305, since it is determined in step S302 that the interval between the two sections A is equal to or less than the predetermined threshold Th3, these two voice sections are integrated into one voice section. More specifically, in the voice segment information of the integrated voice segment, the start point of the previous voice segment and the end point of the next voice segment are set by the processing in this step.

  In step S306, the integrated voice section is set as the voice section of interest (section A), and the process returns to step S302.

  In step S307, since it is determined in step S302 that the interval between the two sections A is larger than the predetermined threshold value Th3, the currently focused voice section is stored as it is as one corrected voice section information. In S308, the next speech segment is set as a speech segment to be focused on as a processing target, and the process returns to step S302.

  Such integration processing is repeated until there is no voice section (section A) to be handled.

(Correction of human voice section using scene change point information)
In general, when there is a video transition in video data including an audio signal and section A starts immediately after that, the video is played after the video at the beginning of the scene is played at a high speed for a moment. Since the reproduced video by the double speed reproduction is performed at a speed that can be heard and grasped by a person, the user feels uncomfortable that the video flickers.

  Therefore, in the present embodiment, for example, the scene change point group detected in step S107 by adopting the scene change point detection technique disclosed in the preceding Japanese Patent Application Laid-Open No. 2000-235539 by the applicant of the present application, or Of the scene change points read out from the sub-information in step S105, it is earlier in time than the beginning of the voice section after the voice section correction processing, is closest, and the distance is below a certain threshold value. If a scene change point exists, correction is performed to replace the beginning of the audio section with information corresponding to the scene change point, thereby eliminating the user's uncomfortable feeling during quick-playback. In this case, the threshold value for determining the neighborhood is a value corresponding to the overhead when shifting from the high-speed playback state to the double-speed playback with sound at a speed at which a person can hear and grasp the contents.

  FIG. 4 is a flowchart showing the audio section integration correction process performed using the scene change point in the present embodiment, and shows the details of the quick-view playback section correction process (step S104).

  In the figure, first, in step S401, the first scene change point on the time axis from the scene change point group (scene change point information or scene change position information) detected in the scene change point detection process (step S107). Read (A).

  The scene change point information is normally described in units of frames. In this step, however, the scene change point information is converted into time information based on the frame rate and then compared with the audio section information. That is, in the algorithm of this embodiment, in order to obtain the nearest scene change point from the start point of the voice section, two consecutive scene change point information is used. Here, for convenience of explanation, the previous scene change point is used. Assuming that the point is A and the next scene change point is B, in step S401, the time of the scene change point is stored in the direction of A.

  In step S402, it is determined whether or not there is audio section information that has not been read. If there is no audio section information that has not been read, the process ends. If there is audio section information that has not been read, one audio section information is read in step S403.

  In step S404, it is determined whether or not there is scene change point information that has not yet been read. If there is no scene change point information, the voice section information that has already been read in step S403 is updated as corrected voice section information as it is in step S405. Remember.

  In step S406, since it is determined that there is scene change point information not read in step S404, the scene change point information is read as scene change point information B.

  In step S407, it is determined whether or not the scene change point A is located on the time axis before the start point of the current voice segment read in step S403. Since there is no need for correction, the speech segment information is updated and stored as corrected speech segment information.

  In step S408, since it is determined in step S407 that the scene change point A is located before the start point of the voice section of interest, the scene change point A exists at a distance within the threshold Th4 from the start point of the voice section. If it is not within the threshold value Th4, the information of the scene change point B is copied to the scene change point A in step S409, so that the next scene change point is set as the determination target. Prepare to do.

  In step S410, since it is determined in step S408 that the scene change point A exists at a distance within the threshold Th4 from the start point of the voice section of interest, the scene change point B is behind the start point of the voice section. If it is not located behind, the process proceeds to step S409.

  On the other hand, if it is determined in step S410 that the scene change point B is located behind the start point of the voice section, the scene change point A is the start point and the end point of the voice section is the end point in step S411. Is updated and stored as corrected audio section information, and in step S412, the information of the scene change point B is copied to the scene change point A, so that the next scene change point is prepared for determination. I do.

  That is, according to the determinations in step S407, step S408, and step S410 described above, the scene change point A is located in front of the start point of the currently focused audio section, is in the vicinity of the threshold value Th4, and is the most audio. Only after it is confirmed that the point is close to the start point of the section, the processes in steps S411 and S412 are performed.

  If it is determined in step S410 that the scene change point B is not behind the start point of the audio section, the scene change point B is set in the corrected audio section from the currently set scene change point A. Since it can be determined that it is more suitable as a starting point candidate, in step S409, the information of the scene change point B is copied as a new scene change point A, so that the next scene change point is prepared for determination. The process returns to step S404. However, since the scene change point A in this case already satisfies the requirements of step S407 and step S408, step S407 and step S408 may be passed and the determination of step S410 may be performed suddenly.

  The corrected voice section information acquired by the procedure of the voice section integrated correction process (FIG. 4) described above is stored in the moving image quick index storage unit 11 as the quick playback section information in the schema illustrated in Table 2. .

  Table 2 is a table exemplifying the scene change detection result in this embodiment. As an example, the result of converting the frame where the scene change point is detected into seconds based on the frame rate (30 frames / sec) is stored. Has been.

  Table 3 exemplifies the corrected speech segment detection results in the present embodiment. Based on the results shown in Table 2 and Table 1, the integration of speech segments using scene change points is shown. The processing result when the correction processing (FIG. 4) is performed with the threshold Th4 = 2000 mSec is shown.

  Referring to Tables 1 and 2, for voice period 0 and voice period 2, the period before the start points 60000 mSec and 400000 mSec of each voice period and within the threshold value Th4 within 2000 mSec There is no scene change. Also, for voice section 1, before 1500 mSec of start point 102000 mSec and within 2000 mSec, as scene change points, scene change ID = 2 (start time 100000 mSec) and scene change ID = 3 There are two points (start time 101000mSec), but since the nearest one is selected according to the algorithm shown in FIG. 4, 101000mSec with scene change ID = 3 is selected as a result, and this is reflected in Table 3. Yes.

<Quick movie playback unit 200>
The quick video playback process (step S107) performed by the quick video playback unit 200 performs double-speed playback with voice at a speed at which a person can hear and grasp the content of the voice segment (section A). For a section (section B) that is not a person's voice section, playback is performed at a high speed and a double speed within a range in which the user can grasp the content by viewing the playback video.

  In recent years, the moving image playback environment has been improved. For example, when a Microsoft DirectShow module is used, it is possible to perform continuous playback by designating the speed of an arbitrary section. By using a module having such a function, it is possible to realize a change in the playback speed of an arbitrary section relatively easily, and in that case, what is important is how to change the speed.

  FIG. 5 is a flowchart showing the moving image quick-view playback process in the present embodiment.

  In FIG. 8, in step S601, the user selects a desired one from the above-described user profile 14, and as a specific procedure thereof, for example, a user as illustrated in FIG. -A display screen including a profile list may be displayed, and a user may select a desired profile from among them using a remote control terminal or the like.

  That is, the specification of a user desired profile in the user profile list shown in FIG. 7 is illustrated in FIG. 9 according to, for example, a remote control terminal provided with an operation button for profile selection, which is pressed by the user. Such a menu display screen is displayed, and the user designates a desired profile using the profile selection operation buttons of the remote control terminal while viewing the screen. Of course, user profile selection can also be done by automatic profile selection using personal recognition technology such as fingerprints, voiceprints, and face recognition. This is because it is always possible to specify the correct profile. You can prevent troubles such as making mistakes, changing other people's profiles, and looking into the contents.

  In addition, when a user profile is newly registered, when the “new registration” button is designated with a pointer device on the display screen of FIG. 7, the display illustrated in FIG. 8 is used to input a profile name and other attributes. A screen appears.

  That is, FIG. 8 is a diagram exemplifying a display screen for user profile registration. In the initial state, contents other than the identification name and age are filled with the reference value, and a unique identification name is obtained by an input operation by the user. Only the location where the input of the age needs to be changed is changed, and after passing the appropriate check of the predetermined input value range, the profile is changed to the user profile 14 in response to the user pressing the “OK” button. It is newly registered.

  Further, when the user desires to change the contents of a desired profile, it is displayed in response to pressing the “change” button on the display screen shown in FIG. 7 and selecting the desired profile on the display screen shown in FIG. In the display screen of FIG. 8, after changing the information content of the item desired to be changed, the “OK” button may be pressed.

  Further, when the user desires to delete the desired profile, the user presses the “delete” button on the display screen shown in FIG. 7, selects the desired profile on the display screen shown in FIG. 9, and then clicks the “OK” button. Just press it.

  When the “cancel” button is pressed on the display screens shown in FIGS. 7 and 8 described above, processing (profile registration, change, and deletion) corresponding to the selection operation and input operation so far is performed. The process ends without

  Next, in step S602, it is determined whether the profile selected in step S601 exists in the user profile 14, and if it exists, the target profile is read from the user profile 14 in step S603, If it does not exist, the playback speeds of the sections A and B and the volume during playback of the section B, which are preset as reference values, are read in step S606. Here, an example of the data schema of the user profile is shown in Table 4.

  Table 4 is a table illustrating a user profile in the present embodiment. The reference value may be stored as shown in profile ID = 0. In this case, the playback speed of the section A is 1.5 times faster, the playback speed of the section B is 10.0 times faster, and the reference of the volume during playback of the section B The value is 0 (ie audio mute). This value is used as the reference value used when registering a new user profile.

  Also, in the data schema 1 of the user profile in Table 4, “None” indicates that no value is set. Conversely, if a value is set, playback is performed with the highest priority on that value. In Table 4, “Good” and “Poor” in the visual acuity and hearing fields represent the dynamic visual acuity and the ability to hear early voice regardless of the age of the person.

  In general, the older people become harder to hear, the lower the speed at which they understand words, and the lack of language skills makes children unable to understand when they play voice at high speeds. Many.

  Based on these circumstances, templates for the playback speed of the section A suitable for the age of the healthy person and the playback speed of the section B are prepared in advance, and these speeds are based on the age stored in the user profile 14. To decide.

  However, despite being adolescents, people with weak visual acuity and fast voice hearing, or languages other than their native language (for example, Japanese language) because of foreigners can not catch up with their understanding, etc. There may be unrelated causes. For this reason, in the present embodiment, the characteristics of visual acuity and hearing ability are described as in the user profile exemplified in Table 4, and if there are these settings, this is given priority, and the playback speed of section A, and The playback speed of section B is determined to be low.

  In such a case, for elderly people and users with weak moving vision, it may be out of the viewpoint of the original quick-playing, but the playback speed of the human voice section (section A) is determined to be slower than the normal speed. However, by making the playback speed of a section (section B) that is not a human voice section equal to or higher than the same speed, the user concerned can perform the slow playback that can grasp the voice content of the section A, It is possible to view a moving image in a time faster than the low speed playback of the section.

  Also, for users with weak hearing ability for fast-spoken voices and users who cannot understand in fast-spoken Japanese because of foreigners, the playback speed of section A is determined to be slower than the normal speed, and the playback speed of section B As for the whole, while performing low-speed playback that can grasp the audio content of the section A by setting the same playback speed as that of a healthy person of that age, as a whole, the video is played at a faster time than the low-speed playback of all the sections. It becomes possible to browse.

  As described above, in the present embodiment, the speed determination process for the user profile is performed in advance for a normal person's age-appropriate section A playback speed and section B playback speed template, dynamic visual acuity, and a symptom with a weak early hearing. Because of the foreigners, we make comprehensive judgments taking into account the situation where we cannot catch up with the quick Japanese.

  In the present embodiment, whether or not the language of the audio content is proficient is determined based on whether or not the proficiency stored in the user profile 14 is proficient, or language type information for specifying the native language, and the reproduction target This is done by comparing the language type information of the audio content included in the video. In recent years, digital content such as a DVD and digital media such as a digital BS have stored language type information that specifies the language of audio content, and recently, the program content is electronically stored from an EPG (electronic program guide) or the like. It is practical to use this information because it is available. Even if this information is not available, even in a terrestrial TV program, the default setting is native language, and in bilingual voice, the main voice is usually the native language and the sub voice is a foreign language. What is necessary is just to estimate based on a law.

  In step S604, the playback speed of section A and the playback speed of section B are determined based on the user-desired profile read in step S603. Details of the processing in this step will be described with reference to FIG.

  FIG. 6 is a flowchart showing details of the process in step S604 (FIG. 5) in the flowchart showing the quick-motion video reproduction process in the present embodiment.

  In the figure, first, in step S601, the profile previously selected by the user is read from the user profile 14, and in step S602, the optimum section according to the age of the healthy person is obtained according to the age of the user acquired from the read profile. By referring to a template in which the playback speed of A and the playback speed of section B are set, the playback speed of section A and the playback speed of section B for the user are provisionally determined.

  In step S603, it is determined whether or not the profile read in step S601 describes that the moving body visual acuity is weak. If this is described, in step S604, the playback speed of section A and the section B are determined. Both playback speeds are updated to values lower than the reference value. Therefore, it is desirable to store this value in the profile in advance.

  In step S605, since it is determined in step S603 that the dynamic visual acuity is not described in the profile, it is determined whether or not it is described in the profile that the hearing ability of fast speech is weak. If so, only the playback speed of the section A is updated to a low value in step S606. Therefore, it is desirable to store this value in the profile in advance.

  In step S607, since it is determined in step S605 that it is not described that the hearing ability of the fast voice is weak in the profile, it is determined whether the language type information of the voice content included in the moving image data to be reproduced is available. If it is determined that it is available, the process proceeds to step S608. If it is not available, the process ends.

  In step S608, the language type information of the audio content included in the moving image data to be reproduced is obtained, and the obtained language type information is compared with the good language information described in the currently selected profile. If the two types of information match, the process ends. If they do not match, only the playback speed of section A is updated to a low value in step S609. Therefore, it is desirable to store this value in the profile in advance.

  That is, in the series of processes shown in FIG. 6, if none of Step S603, Step S605, and Step S608 is reached, the playback speed of Section A and the playback speed of Section B temporarily determined in Step S602 are set. It will be adopted as it is.

  If the dynamic visual acuity or early voice hearing is excellent despite being old or young, or vice versa, use the menu for changing the playback speed of section A and the playback speed of section B. It is preferable to configure such that these values can be changed. In this case, the user appropriately changes the playback speed of the section A and the playback speed of the section B while watching the playback video, and automatically or prompts the user to confirm the set playback speed information. By storing the profile in the profile corresponding to the user, it becomes possible to perform easy-to-understand moving image quick reproduction corresponding to each user while reflecting the previous operation information.

  Note that if it is simply performed without using the above-described profile, for example, instead of the processing in each step of Steps S601 to S604 and Step S606, the playback speed of the section A is changed from 0.5 times speed to 2 times speed. An embodiment is assumed in which the user can variably set the playback speed of B between 2 × speed and 10 × speed using an operation menu.

  By the way, when the section B is played back at high speed at a high speed, the sound “curl” is heard. However, if the user does not want to hear the sound, the sound playback is muted when the section B is played. Alternatively, an embodiment in which the volume is changed to a small volume is envisaged. Such a setting is also described in advance in the profile read in step S603, and when it is determined that there is no profile in step S602 when it is determined that the profile has the highest priority at the time of moving image playback, in step S606 A preset reference volume is adopted. Of course, if it is performed more simply, for example, an embodiment may be assumed in which how to quickly process the moving image quick playback processing determines the audio playback level of the section B in advance.

  With the configuration as described above, in this embodiment, designation of the playback speed of the section A and the playback speed of the section B, or both, and the sound level of the section B can be made to each user by using the user profile. Optimal reproduction can be easily realized.

  Next, in step S605, the corrected audio section information is read from the video quick reference index storage unit 11, and in step S607, the playback time of section A is calculated by dividing the total length of section A by the playback speed, and section B is calculated. In the same way, the playback speed is calculated in the same manner as above, and the time required for the user to look quickly is calculated by adding these two values. The calculated time required for quick viewing is presented to the user using the display 23 or the like.

  In step S608, it is determined whether or not the user who has recognized the quick playback time in step S607 is satisfied with the time using an input operation to the remote control terminal or the like, and the user is satisfied with this determination. In this case, in step S610, the reproduction target moving image stored in the moving image data storage unit 10 is reproduced in accordance with the reproduction speeds of the sections A and B and the sound reproduction level of the section B set by the above-described processing. .

  In step S609, since it is determined in step S608 that the user is not satisfied, the playback speed of section A and section B and the audio playback level of section B can be changed so that the user's desired playback time can be accommodated. By providing the man-machine interface, adjustment is made so that the reproduction time desired by the user himself who is not satisfied with the profile and the standard setting is close, and the process returns to step S607.

  Also, as another embodiment corresponding to step S609, the user-desired playback speed can be changed for each section while watching the video playback based on the playback speeds currently set for section A and section B. A configuration is also possible in which the time required for quick viewing according to the configuration is calculated and presented so that the playback time desired by the user himself who is not satisfied with the profile or the standard setting is adjusted.

  By the way, in relation to the user profile and the user's desired speed instruction, the user who has seen the time required for the quick video playback in step S607 can set the time interval A and When these settings are adjusted / changed using a man-machine interface capable of changing the playback speed of B, the value after the adjustment / change may be used as a reference value. Therefore, in such a case, when “Yes” is selected after prompting confirmation by the user automatically or on the confirmation screen illustrated in FIG. 10, the playback speed adjusted / changed by the user is selected. By storing the information in the profile corresponding to the user, it is possible to perform easy-to-understand quick-playing of the moving image according to the user while reflecting the previous operation information in the subsequent moving-image playing.

  In the above-described embodiment, after the user confirms the calculated time required for quick playback, the playback speed of section A and the playback speed of section B are changed so as to be within the user's desired playback time. The configuration example in which the user who is not satisfied with the profile or the standard setting adjusts the playback time to be close to the desired playback time has been described. However, the present invention is not limited to this configuration. The playback speed of A and the playback speed of section B are configured to be changeable, the time required for quick reference according to the setting is recalculated, and this is presented to the user so that the playback time desired by the user himself / herself There are also embodiments that adjust to be close to.

  In the present embodiment, the voice section information has been described as a pair of a start point and an end point, but may be information including a start point and its section length, or an end point and a section length.

  As described above, according to the above-described embodiment, the synchronization relationship between the video and the audio is not broken, and at the time of the video quick playback, all the voices produced by the person are reproduced at a speed at which the contents can be grasped. The section (section B) that does not include the uttered voice is played back at a higher speed. This makes it possible to rationally reduce the total browsing time at the time of quick movie playback compared to the case where the same size playback is performed.

  In addition, according to the present embodiment, the playback speed of the section A and the playback speed of the section B can be easily set to playback speeds suitable for individual users by using the user profile 14, and the section B The volume during playback can also be set to be suitable for the user.

  Furthermore, according to the present embodiment, by displaying the time required for quick playback in advance or during playback of a moving image, a user who is not satisfied with this can specify the playback speed of section A and the playback speed of section B. Thus, it is possible to adjust the time required for the fast-playback optimal for the user, and the information set by the adjustment can be updated and stored in the profile corresponding to the user. Video playback.

(Other embodiments)
The present invention described using the above-described embodiments as an example may be applied to a system including a plurality of devices, or may be applied to an apparatus including a single device.

  In the present invention, a software program that realizes the functions of the flowcharts described in the above embodiments is directly or remotely supplied to a system or apparatus that operates as the above-described moving image reproducing apparatus, and a computer of the system or apparatus is provided. Is also achieved by reading and executing the supplied program code. In that case, as long as it has the function of a program, the form does not need to be a program.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. That is, the claims of the present invention include the computer program itself for realizing the functional processing of the present invention.

  In this case, the program may be in any form as long as it has a program function, such as an object code, a program executed by an interpreter, or script data supplied to the OS.

  As a recording medium for supplying the program, for example, floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card ROM, DVD (DVD-ROM, DVD-R) and the like.

  As another program supply method, a client computer browser is used to connect to an Internet homepage, and the computer program of the present invention itself or a compressed file including an automatic installation function is downloaded from the homepage to a recording medium such as a hard disk. Can also be supplied. It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. In other words, a WWW (World Wide Web) server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the claims of the present invention.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who have cleared predetermined conditions. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  In addition to the functions of the above-described embodiments being realized by the computer executing the read program, the OS running on the computer based on the instruction of the program is a part of the actual processing. Alternatively, the functions of the above-described embodiment can be realized by performing all of them and performing the processing.

  Furthermore, after the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion board or The CPU or the like provided in the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

It is a figure showing the conceptual diagram of the moving image quick look-up algorithm in the moving image reproducing apparatus which concerns on this embodiment. 4 is a flowchart illustrating an outline of a process for detecting a human voice section performed by the moving image quick index creation unit 100 in the present embodiment. It is a flowchart which shows the integrated correction process performed with respect to the audio | voice area with a short space | interval in this embodiment. It is a flowchart which shows the audio | voice area integrated correction process performed using a scene change point in this embodiment. It is a flowchart which shows the moving image quick-reproduction process in this embodiment. It is a flowchart which shows the detail of a process of step S604 (FIG. 5) among the flowcharts which show the moving image quick view reproduction | regeneration processing in this embodiment. It is a figure which illustrates the display screen for user profile selection. It is a figure which illustrates the display screen for user profile registration. It is a figure which shows the example of the user profile in this embodiment. It is a figure which illustrates the display screen which urges | confirms whether the value adjusted and changed is used as a reference value at the time of the next or subsequent video playback when a user who is not satisfied with the time required for the quick video playback is presented. .

Claims (3)

A video playback device capable of playing back video data including audio signals and sub information at high speed,
Determination means for determining a first voice section representing a person's utterance period and a second voice section other than the first voice section based on the sub-information included in the video data;
Based on the moving image data, the first audio section performs high-speed moving image reproduction with a reproduction sound at a predetermined speed at which a user can grasp the contents, while the second audio section is based on the predetermined speed. Quick playback means for high speed video playback at high speed,
Based on the length of the first audio section and the playback speed of the section, and the length of the second audio section and the playback speed of the section, the time required for the high-speed video playback is calculated and calculated. Presenting means for presenting the required time to the user;
Equipped with a,
The quick-view playback means, when the user performs an operation for changing the playback speed of the first and second audio sections in response to the presentation means presenting the required time, the playback speed after the change. The moving picture reproducing apparatus further comprising adjusting means for adjusting the required time based on the above . A video playback method capable of playing back video data including an audio signal and sub information at high speed,
A determination step of determining, based on the sub-information included in the moving image data, a first voice section representing a person's utterance period and a second voice section other than the first voice section;
Based on the moving image data, the first audio section performs high-speed moving image reproduction with a reproduction sound at a predetermined speed at which a user can grasp the contents, while the second audio section is based on the predetermined speed. A fast-playing process that performs high-speed video playback at high speed,
Based on the length of the first audio section and the playback speed of the section, and the length of the second audio section and the playback speed of the section, the time required for the high-speed video playback is calculated and calculated. A presentation process for presenting the required time to the user;
Equipped with a,
In the quick-view playback step, when a change operation of the playback speed of the first and second audio sections is performed by the user in response to the presentation of the required time in the presentation step, the playback after the change A moving image reproducing method comprising an adjusting step of adjusting the required time based on speed . A computer program for causing a computer to execute the moving image reproducing method according to claim 2 .

Images