JP4455644B2 - Movie playback apparatus, movie playback method and computer program thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately detect a phase of voice uttered by a person and to remarkably shorten the time required for browsing by a user while maintaining, with fidelity, a video and audio synchronized relation in accordance with the detected voice phase. <P>SOLUTION: In a motion picture reproducing apparatus, based on sub information contained in motion picture data, a phase A representing a human uttering period and a phase B other than the phase A are determined. On the bases of the relevant motion picture data, in the phase A, equal-speed reproduction or high-speed motion picture reproduction accompanied with reproduced audio is performed at such a predetermined speed that the user can grasp contents (e.g., speed 1.5 to 2 times as high as the equal-speed reproduction), and in the phase B, on the other hand, high-speed motion picture reproduction accompanied with at least small volume of reproduced audio or silent high-speed motion picture reproduction is performed at higher speed than the relevant predetermined speed (e.g., speed 5 to 10 times as high as the equal-speed reproduction). In that case, the speed of motion picture reproduction can be adjusted in accordance with user attribute information registered in a user profile 14. <P>COPYRIGHT: (C)2008,JPO&amp;INPIT

Description

  The present invention relates to the field of moving image reproduction technology accompanied with audio reproduction.

2. Description of the Related Art Conventionally, in a video playback device that accompanies audio playback, such as a video tape recorder, the user can view the entire video (that is, the entire content to be played back) in a short time during playback. Therefore, a double-speed playback function, a high-speed fast-forward function, and the like are provided.

  In addition, in a video tape recorder that is a typical moving image playback device, in recent years, when executing double-speed playback of a recording medium, a first audio section in which sound energy is equal to or higher than a predetermined threshold and less than the predetermined threshold The second audio section is detected and continuously reproduced while performing pitch conversion of the audio signal in the first audio section, so that the reproduced audio is somewhat less for the user while eroding the second audio section. A technique that can reproduce a storage medium at a double speed while being accompanied by a reproduced sound that can be understood easily has been proposed.

Japanese Patent Laid-Open No. 10-32776 JP-A-9-243351 Japanese Patent Laid-Open No. 9-247617

  However, when partial pitch conversion processing of audio signals is performed as described above, the synchronization relationship between audio and video cannot always be maintained during video playback (video quick playback). For example, a person in the played video Therefore, since the reproduced video and the reproduced audio cannot be synchronized, the reproduction is unnatural for the human sense, and the user may feel uncomfortable.

  Further, for example, in JP-A-10-32776, JP-A-9-243351, etc., a silence state is detected based on voice energy, and sounds other than the detected silence state are regarded as a voice section emitted by a person. Thus, a technique for summarizing moving pictures has also been proposed. However, in a moving image in which a voice uttered by a person is dominant throughout the entire program, such as a news program, background noise can be detected to a certain extent although detection of a voice section uttered by a person based on voice energy is possible. This method is not practical in an environment where music or background music exists.

  Furthermore, in the prior art prior to the above-mentioned patent publication, there have been proposed many techniques for performing sound detection and moving image reproduction considering the detected sound, most of which are performed by thresholding sound energy. Audio is detected. There is a problem caused by the ambiguity of the Japanese language. “Human voice” is also called “speech”, and general sounds including human voice are also called “speech”. It is inappropriate to collectively refer to threshold processing of sound energy in the prior art as true “voice detection”.

  In Japanese Patent Laid-Open No. 9-247617, a “singularity” is obtained by calculating an FFT (Fast Fourier Transform) spectrum of an audio signal, and a “feature point such as audio information” is detected, and its volume is analyzed. Technology has been proposed. However, in the method using the FFT spectrum, when a sound signal to be reproduced includes so-called background music having a broadband spectrum distribution, a voice uttered by a person is detected therefrom. It is difficult.

  As described above, in the conventional video playback with audio, there is a problem that the detection of the audio section is convenient and inaccurate as described above, and further, the summary of the video using the detection result and the double speed are generated. When performing playback, there is a problem that the synchronization relationship between video and audio cannot be maintained during playback.

  In general, it is known that it is not easy for a user such as an elderly person or a child to master various devices, and it is difficult to comprehend the contents of voices emitted at a high speed. Therefore, for such a user, the optimum conditions for reproduction differ from those of general users when performing quick viewing (shortened reproduction) of contents such as double speed reproduction in a moving image reproduction apparatus such as the above-described tape recorder.

  Furthermore, even for users with low dynamic visual acuity, users with low hearing ability for fast voices, or foreign users who do not use the voice to be played as their native language, the video playback device as described above allows quick viewing (shortening) of contents such as double speed playback. When performing (playback), the optimum conditions for playback differ from those of general users.

  Therefore, the present invention accurately detects a voice section generated by a person and maintains a synchronized relationship between video and audio faithfully according to the detected voice section, and greatly reduces a user's viewing time. An object of the present invention is to provide a moving image reproduction method and a computer program thereof.

  According to one aspect of the present invention, there is provided a moving image reproducing apparatus capable of reproducing moving image data including an audio signal at a high speed, wherein the first audio representing a human speech period based on the audio signal included in the moving image data. Based on the moving image data, the voice section determination means for determining the section and the other second voice section, and based on the moving image data, the first voice section has a predetermined speed at which the user can grasp the content, and a high speed with reproduced voice. While performing the moving image reproduction, the second audio section includes a fast-playing reproduction means for performing high-speed moving image reproduction at a speed higher than the predetermined speed, and the quick-playing reproduction means includes the length of the first audio section and the length thereof. Based on the playback speed of the section, the length of the second audio section, and the playback speed of the section, the time required for the high-speed moving image playback is calculated, and the calculated required time is presented to the user. Be Image playback device is provided.

  According to the present invention, while accurately detecting a voice section generated by a person and maintaining the synchronization relationship between video and audio faithfully in accordance with the detected voice section, video playback that significantly reduces the time required for viewing by the user Provision of an apparatus, a moving image reproduction method, and a computer program thereof is realized.

  Hereinafter, an embodiment of a moving image reproducing apparatus according to the present invention will be described in detail with reference to the drawings.

First, an outline of the operation of the moving image playback apparatus in the present embodiment will be described with reference to FIG.

  FIG. 1 is a diagram illustrating a conceptual diagram of a video quick-view algorithm in the video playback device according to the present embodiment.

  As shown in FIG. 1, the moving image playback apparatus according to the present embodiment is roughly divided into a moving image quick index creation unit 100 and a moving image quick playback unit 200.

<Quick movie index creation unit 100>
In the moving image quick index creation unit 100, the moving image data read from the moving image data storage unit 10 is separated into video data and audio data (audio signal) in the video / audio separation process (step S101). Then, an audio segment estimation process (step S102) and an audio segment correction process (step S103) are performed, and a video change degree calculation process (step S105) and a scene change point detection process (step S106) are performed on the video data. Then, the fast-playing playback section information is generated by the fast-playing playback section correction process (step S104), and the generated information is stored in the moving image quick-view index storage unit 11.

  That is, in the audio section estimation process (step S102), the audio signal obtained in the video / audio separation process (step S101) is filtered by a low-pass filter to obtain the zero crossing point of the audio signal. When the small segment group having the zero crossing point at the start point and the end point is formed and the signal energy of the adjacent small segment is small, the small segment is combined with the immediately preceding small segment, thereby A small segment is determined. Here, the zero crossing point is a point where the waveform of the filtered audio signal crosses the zero level which is the reference signal level.

  For each small segment determined in this way, attribute information indicating its property is given as a label. This label always includes a label for the voice pitch, and the voice pitch segment also has voice pitch period information (details will be described later).

  In the present embodiment, when voice is detected, the segment group having the voice pitch label is used as a base, and the distance between segments having the voice pitch label adjacent to each other (that is, the time interval, the same applies hereinafter) is thresholded. By using the continuity of the voice pitch period information individually contained in the voice pitch segment, and further considering both the continuity of the voice pitch period and the distance between the segments, a discrete voice pitch label can be created. By integrating a plurality of small segments as one segment, a speech section is accurately detected.

  Further, in the voice segment correction process (step S103), the person (user) does not become uncomfortable at the time of voice reproduction based on the processing result (voice segment detected from the voice signal) in the voice segment estimation process (step S102). As described above, correction of a voice section representing a person's utterance period (hereinafter referred to as “person's voice section” or section A), which is newly reproduced by integrating a plurality of adjacent voice sections, is performed. Thus, corrected speech section information is acquired.

  For example, as a bad aspect in high-speed video playback, when the interval between two adjacent sections A is narrow, at the time of video playback, the voice is played at such a speed that a person can hear and understand the contents. In addition to performing double-speed playback (for example, double-speed playback), the content of a section that is not a human voice section (hereinafter referred to as section B) can be grasped by viewing the playback video when the video is played back. When the playback is performed at a high speed of 2 times, the change is severe and it is difficult for a general user to hear.

  Therefore, in the present embodiment, in the speech segment correction process (step S103), the interval between the human speech segments is taken into account, and when the predetermined condition satisfies the certain interval, a plurality of speech segment groups are integrated. This eliminates the difficulty of hearing. Here, as the predetermined condition, for example, it is easiest to set that the interval of the human voice section is equal to or less than a predetermined threshold value.

  In the video change degree calculation process (step S105), the similarity comparison process between frames described in Japanese Patent Laid-Open No. 2000-235539 is performed on the video data obtained in the video / audio separation process (step S101). The video change information is generated by calculating the similarity between frames by performing.

  Generally, when there is a video transition in video data including audio signals and the audio section starts immediately after that, the video at the beginning of the scene is played back at a high speed for a moment when playing back the video. Since the reproduced video by the double speed reproduction is performed at a speed that can be heard and grasped by a person, the user feels uncomfortable that the video flickers.

  Therefore, in the present embodiment, in the scene change point detection process (step S106), for example, by adopting the scene change point detection technology disclosed in the prior Japanese Patent Application Laid-Open No. 2000-235539 by the present applicant, A scene change point group (scene change point information) is detected based on the video change information obtained in the change degree calculation process (step S105).

  Then, in the quick-play reproduction section correction process (step S104), when the distance is less than or equal to a predetermined threshold at a time earlier than the beginning of the voice section after the voice section correction process in step S103 Furthermore, by replacing the head of the voice section with information corresponding to the scene change point detected in step S103, the user's uncomfortable feeling can be removed.

<Quick movie playback unit 200>
Next, in the movie quick-view playback unit 200, the playback video is played back using the display 12 and the playback audio is played back using the speaker 13 in the movie fast-play playback process (step S107). At the time of moving image reproduction by the moving image quick reproduction process, the time required for reproduction is displayed in step S108 based on the quick reproduction section information read from the moving image quick index storage unit 11, and the step corresponding to the display is performed. By integrating the feedback of the user-desired playback condition set in S109 and the playback condition based on the user profile 14, final setting of the fast-playing playback condition is performed, and based on the set fast-playing playback condition. The moving image reproduction of the moving image data read from the moving image data storage unit 10 is performed.

At that time, in this embodiment,
-For section A, double-speed playback with voice is performed at a speed that allows the user to grasp the content when the user hears the played voice,
-For the section B, high speed double speed reproduction is performed within a range in which the user can grasp the contents by viewing the reproduced video.

  As a result of an experiment by the applicant of the present application, the double-speed playback in the above section A, that is, playback at a speed at which a person can hear and grasp the contents, is up to double speed, preferably about 1.5 times speed in the experiment. I know. On the other hand, for the section B, playback is performed at a high magnification rate within a range in which the user can see the content of the playback video, but according to the results of experiments by the applicant of the present application, it is empirically up to 10 times the speed. It has been found that it is better to set the speed to 5x or higher.

  It is known that when the section B is played back at high speed at a high speed, generally, a sound of “curl curl” is produced. Therefore, in step S107, when the section B is played back at high speed, the user does not want to hear such a sound. In some cases, the sound reproduction may be muted by muting, or the volume during reproduction may be reduced.

  Regarding the playback speed of the section A, the playback speed of the section B, and the volume at the time of playback, the simplest implementation method is to determine how to process the sound in advance in the moving image quick playback process (step S107). There is a method that allows the user to set the playback speed in a variable manner.

  However, in general, it is not easy for a user such as an elderly person or a child to master various devices, and it is known that it is difficult to understand the content when high-speed audio playback is performed, It is preferable to perform double speed reproduction at a slightly lower magnification without performing troublesome speed adjustment. Similarly, users with weak visual acuity regardless of age (visually impaired), especially dynamic visual acuity and hearing, particularly users with weak early hearing ability (hearing impaired), or reproduced speech in their native language. It is known that foreign users and the like who do not want to understand the content are difficult to understand when high-speed audio reproduction is performed, and there is an optimum reproduction speed for these users.

  Therefore, in the present embodiment, information such as the user's age and language, understandable language, visual acuity, hearing ability, and the like, and attribute information about the user such as a standard reproduction condition preferred by each user are stored in the user profile 14 in advance. By referring to the profile 14 in the quick video playback process (step S107), the human speech section (section A) and the section excluding the human speech section (section B) according to the target user. ) Can be determined for each video, so that it is possible to perform a quick video playback that is easy to understand for each individual.

  In addition, as described above, when the sound is muted or the volume is reduced during high-speed double-speed playback in the section B, it is possible to comfortably respond to individual users by describing such settings in the profile 14 in advance. It is possible to perform quick movie playback.

  Furthermore, for elderly people and users with handicaps in moving visual acuity, it may be out of the viewpoint of the original quick-playing, but the playback speed of the section A is set slower than the normal speed and the playback speed of the section B Is set to a speed equal to or higher than the normal speed, while the user concerned can perform low-speed playback capable of grasping the audio content of the section A, as a whole, in a short time required compared to the case of low-speed playback of all sections, It is possible to view a moving image (that is, moving image data stored in the moving image data storage unit 10).

  In addition, for users who are handicapped in understanding the content of early speech and users who are not fluent in the language of speech content, the playback speed of section A may be slower than the normal speed, although this may be out of the viewpoint of fast-playing. In addition to the setting, the playback speed of the section B is set to 10 times speed, preferably 5 times speed or more, and the slow playback that enables the user to grasp the audio content of the section A is performed, but the entire section is played back at a low speed as a whole. It is possible to view the moving image (that is, the moving image data stored in the moving image data storage unit 10) in a shorter time than the case. Here, whether or not the language of the audio content is proficient is determined based on the identification information stored in advance in the above-described profile 14 (special language in Table 4 described later), the language type information of the audio included in the video to be played back, This can be done by comparing.

  As a procedure for selecting the user profile 14, for example, a user profile list is displayed on the profile selection screen displayed on the display 12, and the user profile 14 is selected according to the operation of the remote control terminal (not shown) by the user. In addition, an automatic profile selection method using a personal recognition technique such as fingerprint, voiceprint, or face recognition may be employed.

  By the way, in the case of performing the fast-view playback optimal for each user as described above, what length of the original video can be viewed in advance is determined by the user who is going to make a quick look at the free time. This is important information.

  Therefore, in this embodiment, in step S108, the playback time of the section A is calculated by dividing the total length of the section A by the playback speed, and for the section B, the section B is calculated by dividing the total length by the playback speed. The playback speed is calculated, the sum of these two calculated values is obtained as the time required for quick viewing, and is presented to the user together with the time required for reproducing the original moving image at the same magnification. Furthermore, by designating the playback speed of the section A and the playback speed of the section B so that these playback times are within the desired playback time when the user sees them, the playback time becomes closer to the user's desired playback time. It is possible to adjust to.

  By the way, the time required for the quick movie playback that is automatically calculated using the profile 14 in step S108 as described above is related to the preset user profile 14 and the desired playback speed designated by the user. In step S109, the user who has watched further designates the playback speeds of the sections A and B in step S109, thereby obtaining the time (playback speed information) required for the desired movie quick-playback playback. If it is set, the previous operation information is stored in the profile by automatically storing the set playback speed information after confirming with the user automatically or within the set required time. This makes it possible to perform easy-to-understand moving image quick-play according to the preferences of individual users.

  Further, in the user profile described above, it is further specified in advance how to process the volume during playback of the section B, or if the user specifies through a predetermined man-machine interface, the specification is made. Thus, it is possible to perform easy-to-understand moving image quick-play according to individual user's preference while reflecting the volume information.

<Details of the operation of the video playback device>
Hereinafter, details of the operation of the moving image reproducing apparatus according to the present embodiment outlined above will be described. In the following description, fast-playing playback section information is created as an index for performing fast-playing with respect to recorded video data stored in the video data storage unit 10, and the video data is created using the created information. An example of performing fast-playing will be described.

  In the present embodiment, as described above, the post-processing after the video / audio separation process in step S101 is roughly divided into a video quick-look index creation process by the video quick-look index creation unit 100 and a video fast-play playback by the video fast-play playback unit 200. There is processing.

<Quick movie index creation unit 100>
FIG. 2 is a block diagram showing an algorithm for detecting a speech section (section A) representing a person's utterance period performed in the moving image quick index creation unit 100. The AGC (auto gain control) 21, low-pass filter 22, zero It comprises an intersection detection unit 23a, 23b, an audio segmentation unit 24, an audio pitch detection unit 25, an audio labeling unit 26, an audio energy calculation unit 27, and an audio interval determination unit 28.

  FIG. 3 is a flowchart showing an outline of processing based on the algorithm shown in FIG. 2. The procedure for detecting the section A will be described with reference to this flowchart. First, in step S301, the audio signal is divided into a plurality of small segments. In step S302, voice labeling representing the acoustic features of these small segments is performed. At this time, in step S303, a robust vowel candidate is detected by detecting the voice pitch, and finally, in step S304, a human voice section (section A) is determined based on the voice pitch detection result.

  That is, the audio signal separated from the moving image data by the video / audio separation process (step S101) is normalized by the AGC (auto gain control) 21. As the configuration of the AGC 21, a known configuration may be adopted, and a configuration for normalizing a registered audio signal with reference to a sound having a maximum signal level throughout the entire audio signal can be adopted.

  The normalized audio signal is filtered by the low-pass filter 22 to branch into an audio signal component having a band suitable for analysis processing performed later and an original audio signal having a band necessary for unvoiced consonant recognition. .

(Voice segmentation)
First, the audio signal that has passed through the low-pass filter 22 has a zero-crossing point obtained by the zero-crossing point detection unit 23a. Provisionally divided. This process corresponds to step S301 in FIG.

  Here, the reason why the low-pass filter 22 is used for segment division is that the reference of the small segment is a unit such as unvoiced consonant, voiced consonant, and voice pitch, and if there is an influence of high frequency, the unvoiced consonant will be adversely affected .

Now, the audio segmentation unit 24 divides the audio signal into small segments on the basis of the zero crossing point tentatively obtained for the audio signal. The small segment is divided into the following two conditions Rule 1: Small The start and end points of the segment are zero crossings,
Rule 2: When the energy of a small segment is small, it is combined with the immediately preceding small segment.

  Voice energy P is applied to a small segment f (x) starting from X1 and ending at X2,

なる数式（１）を満たすものと定義する。

It is defined that the following mathematical formula (1) is satisfied.

  When the calculated sound energy P is equal to or less than the predetermined threshold Eth1, the small segment f (x) that is currently targeted is integrated into the small segment immediately before it. Note that the sound energy P may be calculated using the square energy of f (x) instead of the accumulation of the absolute values of the small segments f (x) according to Equation (1).

  FIG. 4 is a diagram for explaining small segment combining processing performed in the present embodiment.

  In FIG. 4, FIG. 4A illustrates an audio signal level in which a plurality of zero cross points are obtained by the zero cross point detector 23a. Further, in FIG. 4B, the detected zero crossing points and the plurality of small segments set by applying the above-described rule 1 and rule 2 are indicated by individual vertical lines and are indicated by arrows. The two small segments shown indicate that they have been merged into one small segment according to rule 2 described above.

(Audio labeling process)
The zero-crossing detector 23b obtains the average number of zero-crossings in which the speech signal waveform whose speech energy has been normalized by the AGC 21 intersects the reference zero level, and the speech energy calculator 27 obtains the average energy. For each small segment, the audio labeling unit 26 calculates a start point, an end point, an average number of zero crossings, and an average energy, and stores these calculated values as feature quantities of the small segments. This process corresponds to step S302 in FIG.

  However, the average number of zero crossings and the average energy are calculated by the following formula using the segment length SegLen.

・ (Average number of zero crossings) = (Number of zero crossings of original audio signal included in small segment) / SegLen,
・ (Average energy) = (Energy of audio signal with low-pass filter included in small segment) / SegLen
It is.

  Further, the small segments are classified into five categories, and labels representing the categories are given. In this embodiment, types of labels that can be assigned to individual small segments include silence, unvoiced consonants, voiced consonants, voice pitch, and noise.

  Next, which label corresponds to the small segment currently focused on is determined by the procedure shown in FIG.

  FIG. 5 is a flowchart showing the process of audio labeling performed in the present embodiment, and shows the procedure of the process performed by the audio labeling unit 26.

  In step S501, the average zero crossing number AveZeroCrossRate and the average energy AveEnergy are read as the feature quantities of the target small segment (small segment to be processed).

  In the present embodiment, the following threshold values are provided as label determination conditions, and these threshold values are all constants.

・ Threshold value for maximum silent energy: SileceEnergyMax,
・ Minimum energy threshold for unvoiced consonants: ConHEnergyLow,
・ Maximum energy threshold for unvoiced consonants: ConHEnergyMax,
・ Minimum energy threshold for voiced consonants: ConLEnergyLow,
-Maximum energy threshold for voiced consonants: ConLEnergyMax,
・ Minimum zero-crossing threshold for unvoiced consonants: ConHZeroCrossRateLow,
・ Maximum zero-crossing threshold for voiced consonants: ConLZeroCrossRateMax,
However, SileceEnergyMax> ConHEnergyLow shall be satisfied.

In step S502, it is determined whether the feature amount read in step S501 satisfies a predetermined silence condition. Here, the silence label condition is
-((AveEnergy <SileceEnergyMax) AND (AveZeroCrossRate <ConHZeroCrossRateLow)), or ((AveEnergy <ConHEnergyLow) AND (AveZeroCrossRate> ConHZeroCrossRateLow)),
And In step S503, when the silent label condition is satisfied, a silent label is stored in association with the small segment of interest.

On the other hand, if the silence label condition is not satisfied in step S502, it is determined in step S504 whether the feature amount read in step S501 satisfies a predetermined unvoiced consonant label condition. Here, the unvoiced consonant label condition is
・ (ConHEnergyLow <AveEnergy <ConHEnergyMax)
・ (AveZeroCrossRate> ConHZeroCrossRateLow)
And In step S505, if the unvoiced consonant label condition is satisfied, the unvoiced consonant label is associated with the small segment of interest and stored.

  In step S506, since the feature amount read in step S501 does not satisfy the above-described silence label condition and unvoiced consonant label condition, an attempt is made to detect the voice pitch. Is assigned to the small segment group (step S507). The pitch detection will be described later in detail.

  Here, the reason for assigning the audio pitch label to the small segment group is that there is a possibility that the small segments will be integrated in the pitch detection described later. This is because they are integrated into one in step S508 and a pitch label is given thereto. At this time, the segment in which the voice pitch is detected is a vowel mainly accompanied by vocal cord vibration.

If the voice pitch cannot be detected in step S506, voiced consonant label condition determination is performed in step S509. At this time, the voiced consonant label condition is
・ (ConLEnergyLow <AveEnergy <ConLEnergyMax)
・ (AveZeroCrossRate <ConLZeroCrossRateMax)
And In step S510, when the above voiced consonant label condition is satisfied, the voiced consonant label is stored in association with the focused small segment.

  In step S511, since the above-described conditions are not satisfied, the noise label is stored in association with the small segment of interest.

  Here, the process from segmentation of audio signal waveforms to labeling will be described with reference to the example shown in FIG.

  FIG. 6 is a diagram for explaining a processing process from segmentation of audio signal waveforms to labeling in the present embodiment.

  More specifically, FIG. 6A shows an audio signal waveform after the low-pass filter. FIG. 6 (b) shows a state where the audio signal waveform shown in FIG. 6 (a) is divided into small segments with reference to the zero crossing point, and the thick vertical line shown in FIG.

  FIG. 6 (c) shows the result of voice labeling and segmentation. The narrow vertical lines in the figure represent segment delimiters, and the thick vertical lines represent the remnants of the integrated small segments. Yes. In FIG. 6 (c), it can be seen that a plurality of small segments divided as shown in FIG. 6 (b) are integrated into one pitch segment, and each segment has a given label. It is shown.

(Audio pitch detection)
Next, the operation of the voice pitch detection unit 25 will be described with reference to FIGS. This process corresponds to step S303 in FIG.

  FIG. 9 is a flowchart showing the voice pitch detection process in the present embodiment, and shows a processing procedure performed by the voice pitch detection unit 25.

  In step S901, the zero crossing point information of the audio signal waveform after the low-pass filter is obtained. Then, the voice pitch is obtained by verifying the similarity of waveforms with the zero crossing point as a reference.

  FIG. 7 is a diagram illustrating an audio signal waveform for explaining the audio pitch detection processing in the present embodiment.

  In the present embodiment, the reference zero-crossing point is the start point of a waveform having a positive value when viewed in the time direction, and in the example of FIG. 7, the reference zero-crossing points are X1, X2, and X3.

  In step S902, in the case illustrated in FIG. 7, the partial waveform having the zero crossing point X1 as the starting point, the zero crossing point X2 as the ending point is f (x), the zero crossing point X2 is the starting point, and the zero crossing point X3 is the ending point. The partial waveform is determined using g (x) as an initial reference.

  In step S903, it is determined whether or not an unprocessed speech segment (speech segment) exists. If it exists, the process proceeds to step S904, and if not, the process ends.

  In step S904, a pitch extraction process for reporting the presence / absence of a voice pitch and its segment range is performed. Here, the timing to report is the timing when the voice pitch segment is interrupted or the case where the pitch for the partial waveform f (x) is not found. The pitch extraction process in step S904 will be described later in detail with reference to FIG.

  In step S905, it is determined whether or not there is a voice pitch. If it is determined that there is a voice pitch, the voice pitch segment information is stored in association with the target voice section (voice segment) in step S906. On the other hand, if there is no voice pitch, the process returns to step S903.

  Here, the pitch extraction process performed in step S904 will be described in detail with reference to FIG.

  FIG. 10 is a flowchart showing details of the process in step S904 (FIG. 9) in the flowchart showing the voice pitch detection process in the present embodiment.

  In the figure, in step S1001, g (x) with respect to the set f (x) is set. In step S1002, the length of the set f (x) is checked, and if it is too long to exist as a pitch, it is determined that there is no voice pitch corresponding to the f (x), and step S1002 In S1003, a new part having the end point of the f (x) as the start point and the end point of the waveform having a negative value in the time direction and having the negative value closest to the start point An audio segment f (x) is set, and the segment of f (x) that has been focused on until now is reported not to be a pitch segment.

  Further, in step S1004, the length of the focused f (x) is checked, and if it is so short that it cannot exist as a pitch, in step S1005, the end point of the focused f (x) is set as the starting point. In addition, among the zero crossing points that are the end points of the waveform having a negative value when viewed in the time direction, a partial speech segment having the end point closest to the start point (end point of f (x)) as the end point is the f ( By integrating at the end of x), the process returns to step S1001 as new f (x).

  On the other hand, in step S1006, a dissimilarity calculation with g (x) is performed on the focused f (x) that has passed the checks in steps S1002 and S1004. The dissimilarity calculation performed in this step is calculated using the following dissimilarity evaluation function.

That is, if the absolute value of the difference between f (x) and g (x) at time Xf of the partial speech segment f (x) is Δ (Xf), X1 ≦ Xf ≦ X2 and Xg = X2 + (Xf− X1)
Δ (Xf) = | f (Xf) −g (Xg) |
It is expressed. Even in this case, based on the square of the difference instead of the absolute value of the difference between f (x) and g (x),
Δ (Xf) = [f (Xf) −g (Xg)] × [f (Xf) −g (Xg)]
It is also good.

  And furthermore,

と表すことができる。

It can be expressed as.

  In step S1007, it is determined whether or not the dissimilarity calculated as described above is equal to or greater than the threshold value ETh. If DiffSum ≧ ETh, the process returns to step S1005, and if DiffSum <ETh, the sound is more precise. In order to perform pitch detection, in step S1008, the positions of f (x) and g (x) are corrected so that the subsection with the largest energy is the end of the pitch segment.

  FIG. 8 is a diagram for explaining a procedure for updating the pitch detection reference performed in the audio pitch detection process according to the present embodiment. Correcting the pitch reference in the small section with the largest energy is reasonable because the small section is a waveform generated at a timing immediately after the vocal cord vibration.

  In step S1009, the pitch detection counter is reset to 0. In step S1010, the dissimilarity calculation is performed in the same manner as in step S1006 described above. In step S1011, the calculated dissimilarity is compared with the threshold value ETh. Is performed in the same manner as step S1007 described above.

  If the calculated dissimilarity is greater than or equal to the threshold value ETh as a result of the comparison in step S1011, the process proceeds to step S1013. If the dissimilarity is smaller than the threshold value ETh, the process proceeds to step S1014.

  In step S1013, it is determined whether the voice pitch is detected twice or more. If it is less than twice, the voice segments are integrated in step S1005 described above. If the voice pitch is detected twice or more, the voice pitch segment is detected. In step S1015, a new zero-crossing point closest to the starting point that is the end point of the waveform having a negative value when viewed in the time direction is set as the end point in step S1015. A segment f (x) is set and a pitch segment range indicating that a pitch segment has been detected is reported.

  In step S1014, the pitch detection count is incremented, the current g (x) end point is set as the start point, and among the end points of the waveform having a negative value when viewed in the time direction, the nearest zero crossing point to the start point is determined. A new partial speech segment f (x) is set as the end point, and the zero crossing point that is the closest to this partial speech segment f (x) and has a negative value in the time direction is defined as the end point. A new g (x) to be set is set, and the process returns to step S1010.

  The voice pitch segment acquired by the voice pitch detection process (FIGS. 9 and 10) described above is stored in a memory (not shown) in order to be used by the voice section determination unit 28 in the subsequent stage.

(Voice section judgment)
Next, the speech segment determination unit 28 determines the speech segment (section A) of the person using the speech pitch segment acquired by the speech pitch detection process. This process corresponds to step S304 in FIG.

  In general, in the case of a pure human voice, a vowel occupies most of the voice section, and thus a segment having a pitch appears long and stably. On the other hand, it has been experimentally found that when there is BGM, it is affected by its temperament, but when a person's voice energy is somewhat larger than the energy of BGM, it is not so much affected. In addition, when the voice energy is not sufficiently larger than the energy of GBM in a certain partial section, an accurate pitch does not appear in the partial section.

  In many cases, a consonant is accompanied immediately before the vowel, but no pitch appears even in the case of a consonant not accompanied by vocal cord vibration, and the duration is a short plosive with a duration of 10 ms or less. Even long friction sounds are on the order of several tens of ms. In addition, there is a case where silence occurs immediately before the occurrence of a plosive or the like.

  Therefore, the segment in which the voice pitch is obtained is not only due to factors outside the device but also due to the factor of the human voice itself. Even in such a case, the front and rear or the entire pitch period should be considered. Therefore, it is necessary to integrate the calculation results of the voice pitch periods of the partial sections and further determine the voice section (section A) of the person by utilizing the characteristics of the voice.

  FIG. 11 is a flowchart showing a speech segment determination process in the present embodiment, and shows a processing procedure performed by the speech segment determination unit 28.

  In the figure, first, in step S1101, a segment group having continuous silence, unvoiced consonant label, voiced consonant label, or noise label is combined into one segment.

  Further, in step S1102, continuous pitch label segments are obtained, and these are combined to obtain an average pitch period of the plurality of segments. This integrated pitch segment is referred to as an “integrated pitch segment”.

  In step S1103, the segment associated with the noise label sandwiched between the integrated pitch segments is obtained. In step S1104, the average pitch period variation rate of the integrated pitch segments at both ends of the segment is equal to or less than a threshold value Th1. If this condition is satisfied, these segments are integrated into one integrated pitch segment in step S1105. By this processing, even if a BGM having a large energy overlaps a part of a pitch segment, that is, a vowel, it can be corrected.

  By the way, since there is no single consonant in most cases, usually consonant is often accompanied backward or forward. This is called a CVC (Consonant Vowel Consonant) model.

  In step S1106, the voice segment is obtained by integrating the unvoiced consonant segment, the voiced consonant segment, and the pitch segment based on the CVC model. Details of the processing in step S1106 will be described with reference to FIG.

  FIG. 12 is a flowchart showing details of the processing in step S1106 (FIG. 11) in the flowchart showing the speech segment determination processing in the present embodiment.

  In step S1201, the first integrated pitch segment is set as a reference integrated pitch segment. Next, in step S1202, the next integrated pitch segment of the reference integrated pitch segment is obtained.

  Further, in step S1203, it is determined whether a voiced consonant segment or an unvoiced consonant segment exists between the two integrated pitch segments. If there is not, an integrated pitch segment next to the reference integrated pitch segment is determined in step S1206. It is determined whether or not it exists. If it does not exist, the process ends. If it exists, the reference integrated pitch segment is updated in step S1207.

  On the other hand, if it is determined in step S1203 that a voiced consonant segment or an unvoiced consonant segment exists between the two integrated pitch segments, whether or not the interval Dist between the two integrated pitch segments is equal to or less than the threshold value Pimax1. Determination is made in step S1204. If the distance Dist is equal to or smaller than the threshold value Pimax1, in step S1205, the end points of the two integrated pitch segments are stored as the voice segments of the person having the end point and the start point.

  Here, the threshold value Pimax1 is preferably a consonant having the longest normal duration, for example, a duration sufficiently longer than the duration of unvoiced friction sound / S /, etc. In addition to consonant segments, silence segments may exist. The reason for this is that, among the unvoiced consonants, a burst sound or a rubbing sound may cause a short silence before utterance.

  After the speech section storage in step S1205, in step S1206, it is determined whether there is an integrated pitch segment next to the reference integrated pitch segment, and if not, the process ends. In step S1207, the reference integrated pitch segment is updated, and the above-described steps are repeated until the end condition in step S1206 is satisfied. However, the integrated pitch segment information and the average pitch information are stored without being discarded for the next processing.

  On the other hand, as a result of comparing the average pitch periods of the two integrated pitch segments in step S1204, if the period variation rate is greater than a certain threshold value Pimax1, the processes from step S1206 described above are performed.

  Here, the description returns to the flowchart of FIG. In step S1107, in order to consider the case of a VV (Vowel-Vowel) structure such as “Ao” that does not take a CVC structure, two segments having a silence segment or a noise segment between or adjacent to each other are based on the VV model. A speech segment is obtained by integrating pitch segments.

  Here, the speech section detection processing performed in step S1107 will be described in detail with reference to FIG.

  FIG. 13 is a flowchart showing details of the processing in step S1107 (FIG. 11) in the flowchart showing the speech segment determination processing in the present embodiment.

  In step S1301, the foremost integrated pitch segment is set as a reference integrated pitch segment. Next, in step S1302, the next integrated pitch segment of the reference integrated pitch segment is obtained.

  Further, in step S1303, it is determined whether the interval Dist between the two integrated pitch segments is equal to or smaller than a threshold value Pimax2. If the interval Dist is larger than the threshold value Pimax2, the process proceeds to step S1306, where the sense Dist is the threshold. If it is equal to or less than the value Pimax2, the process proceeds to step S1304.

  In step S1304, if the average pitch period variation rate of the two integrated pitch segments is equal to or smaller than a certain threshold value Th2, a segment sandwiched between the two integrated pitch segments is stored as a voice section in step S1305. At this time, a silence segment or a noise segment may exist between the two integrated pitch segments in order to increase resistance to disturbance.

  Then, after storing the speech section in step S1305, in step S1306, it is determined whether there is an integrated pitch segment next to the reference integrated pitch segment, and if present, the process is terminated. In step S1307, the reference integrated pitch segment is updated, and the process is repeated until the end condition in step S1306 is satisfied.

  On the other hand, as a result of comparing the average pitch periods of the two integrated pitch segments in step S1304, if the period variation rate is larger than the threshold value Th2, the process proceeds to step S1306 described above and the same processing is performed.

  Even if a noise label is generated by BGM or the like included in the speech with reference to the segment in which the pitch is detected in this way, in the present embodiment, the noise label is applied in the above-described speech section determination processing. However, the integrated pitch segments are integrated by taking into account the continuity of the average pitch period of the integrated pitch segments before and after that, and further, by introducing the CVC model, unvoiced consonant segments and voiced consonant segments exist between them. By integrating the integrated pitch segments into speech sections and further determining the speech sections by combining the two integrated pitch segments in consideration of the VV model, it is possible to extract speech sections that are resistant to disturbances using the features of speech.

(Correction of human speech section)
As described above, in the voice section correction processing performed after detecting the human voice section (section A), the time axis is set so that the person does not feel uncomfortable when listening to the reproduced voice based on the processing result. Then, correction is performed by integrating a plurality of speech sections located in the vicinity as one speech section. The reason for this is, for example, when the interval between two sections A located close to each other on the time axis is narrow, while performing the double speed reproduction with sound at a speed at which a person can grasp the contents by listening to the section A, the section B On the other hand, if playback is performed at a high magnification and double speed within the range in which a person can grasp the contents by viewing the playback video, the playback mode changes drastically, making it difficult for the user to hear.

  In addition, from the viewpoint of video decoder and playback processing, as an example, the change in speed in a short section has a large processing overhead, and the playback operation temporarily stops, resulting in jerky playback. In addition to being observed in an experiment by the applicant of the present application using the company's DirectShow, the same phenomenon can be seen in many other video playback means.

  Therefore, in the present embodiment, when the interval between two voice segments (section A) located closest on the time axis is equal to or smaller than a threshold value (Th3 in FIG. 14), these voice segments are integrated. The correction by doing. In determining this threshold value, for example, a scene in which conversation is performed is assumed, and the extent to which the conversation is established is experimentally determined and used as the threshold value. The processing procedure in this case will be described with reference to FIG.

  FIG. 14 is a flowchart showing an integrated correction process performed on a voice interval with a short interval in the present embodiment. This process is a process performed by the voice segment determination unit 28 and represents details of the voice segment correction process (step S103) described above.

  In the figure, in step S1401, the section A that is first positioned on the time axis among a plurality of sections A detected previously is read as a speech section of interest. If there is no speech section of interest, this processing is performed. Ends.

  In step S1402, it is determined whether or not there is a speech section of interest (section A). If there is no speech section of interest, the present process is terminated. The processing from S1403 to S1407 is repeated.

  In step S1403, since it is determined in step S1402 that there is a next voice section of interest, voice section information representing the voice section (section A) is read. Here, the voice section information is information in which the start point and end point of the voice section are paired.

  In step S1404, the interval between the two sections A, that is, the distance (time interval) between the end point of the preceding voice section (currently focused voice section) on the time axis and the start point of the next voice section. To determine whether this distance is equal to or smaller than a predetermined threshold value Th3.

  In step S1405, since it is determined in step S1402 that the interval between the two sections A is equal to or less than the predetermined threshold value Th3, these two voice sections are integrated into one voice section. More specifically, in the voice segment information of the integrated voice segment, the start point of the previous voice segment and the end point of the next voice segment are set by the processing in this step.

  In step S1406, the integrated speech section is set as the speech section of interest (section A), and the process returns to step S1402.

  In step S1407, since it is determined in step S1402 that the interval between the two sections A is larger than the predetermined threshold value Th3, the currently focused speech section is stored as it is as one corrected speech section information. In S1408, the next speech segment is set as a speech segment to be focused on as a processing target, and the process returns to step S1402.

  Such integration processing is repeated until there is no voice section (section A) to be handled.

(Correction of human voice section using scene change point information)
In general, when there is a video transition in the video data including the audio signal, and the section A starts immediately after that, when the video is played back, the video at the beginning of the scene is played back at a high speed for a moment. Since the playback video by the double speed playback with sound is performed at a speed that can be heard and grasped by a person, the user feels uncomfortable that the video flickers.

  Therefore, in the present embodiment, for example, by adopting the scene change point detection technique disclosed in the prior Japanese Patent Application Laid-Open No. 2000-235539 by the applicant of the present application, among the detected scene change point groups, the voice interval correction is performed. If there is a scene change point that is earlier in time than the beginning of the processed voice section, nearest to the distance, and the distance is equal to or smaller than a certain threshold, the beginning of the voice section is set to the scene change point. By performing the replacement with the information corresponding to, the user's uncomfortable feeling at the time of quick playback is removed. In this case, the threshold value for determining the neighborhood is a value corresponding to the overhead when shifting from the high-speed playback state to the double-speed playback with sound at a speed at which a person can hear and grasp the contents.

  FIG. 15 is a flowchart showing the audio section integrated correction processing performed using the scene change point in the present embodiment. This process is a process performed in the audio section determination unit 28, and represents details of the above-described quick-view reproduction section correction process (step S104).

  In the figure, first, in step S1501, the first scene change point on the time axis from the scene change point group (scene change point information or scene change position information) detected in the scene change point detection process (step S106). Read (A).

  The scene change point information is normally described in units of frames. In this step, however, the scene change point information is converted into time information based on the frame rate and then compared with the audio section information. That is, in the algorithm of this embodiment, in order to obtain the nearest scene change point from the start point of the voice section, two consecutive scene change point information is used. Here, for convenience of explanation, the previous scene change point is used. Assuming that the point is A and the next scene change point is B, in step S1501, the time of the scene change point is stored toward A.

  In step S1502, it is determined whether or not there is audio section information that has not been read. If there is no audio section information that has not been read, the process ends. If there is audio section information that has not been read, one audio section information is read in step S1503.

  In step S1504, it is determined whether there is scene change point information that has not yet been read. If there is no scene change point information, the voice section information that has already been read in step S1503 is directly updated as corrected voice section information in step S1505. Remember.

  In step S1506, since it is determined that there is scene change point information not read in step S1504, the scene change point information is read as scene change point information B.

  In step S1507, it is determined whether or not the scene change point A is positioned before the start point of the currently focused voice section read in step S1503 on the time axis. Since there is no need for correction, the speech segment information is updated and stored as corrected speech segment information.

  In step S1508, since it is determined in step S1507 that the scene change point A is located before the start point of the current voice segment, the scene change point A exists at a distance within the threshold Th4 from the start point of the voice segment. If it is not within the threshold value Th4, the information of the scene change point B is copied to the scene change point A in step S1509, and the next scene change point is set as the determination target. Prepare to do.

  In step S1510, since it is determined in step S1508 that the scene change point A exists at a distance within the threshold Th4 from the start point of the current voice section of interest, the scene change point B is behind the start point of the voice section. If it is not located behind, the process proceeds to step S1509.

  On the other hand, if it is determined in step S1510 that the scene change point B is located behind the start point of the voice section, the scene change point A is the start point and the end point of the voice section is the end point in step S1511. Is updated and stored as corrected audio section information, and in step S1512, the information of the scene change point B is copied to the scene change point A, so that the next scene change point is prepared for determination. I do.

  That is, the scene change point A is located in front of the start point of the voice section of interest and is in the vicinity of the threshold value Th4 or less and the voice of the most voice according to the determinations of the steps S1507, S1508, and S1510 described above. Only when it is confirmed that the point is close to the start point of the section, the processing in steps S1511 and S1512 is performed.

  If it is determined in step S1510 that the scene change point B is not behind the start point of the audio section, the scene change point B is set in the corrected audio section from the currently set scene change point A. Since it can be determined that it is more suitable as a starting point candidate, in step S1509, the information of the scene change point B is copied as a new scene change point A to prepare for the next scene change point, and then Thus, the process returns to step S1504. However, since the scene change point A in this case already satisfies the requirements of Steps S1507 and S1508, Step S1507 and Step S1508 may be passed and the determination in Step S1510 may be performed suddenly.

  The corrected voice section information acquired by the procedure of the voice section integrated correction process (FIG. 15) described above is stored in the moving image quick index storage unit 11 as the quick playback section information in the schema illustrated in Table 1. .

  Table 1 is a table exemplifying the scene change detection result in this embodiment. As an example, the result of converting the frame where the scene change point is detected into seconds based on the frame rate (30 frames / sec) is stored. Has been.

  Next, Table 2 is a table exemplifying the detection result of the voice section in the present embodiment, and one voice section is expressed by a start point and an end point.

  Table 3 is a table exemplifying the corrected voice section detection result in the present embodiment. Based on the result shown in Table 1 and the result shown in Table 2, voice section integration using scene change points is shown. The processing result when the correction processing (FIG. 15) is performed with the threshold Th4 = 2000 mSec is shown.

  Referring to Tables 1 and 2, for voice period 0 and voice period 2, the period before the start points 60000 mSec and 400000 mSec of each voice period and within the threshold value Th4 within 2000 mSec There is no scene change. For voice section 1, before 1500 mSec of start point 102000 mSec and within 2000 mSec, as scene change points, scene change ID = 2 (start time 100000 mSec) and scene change ID = 3 There are two points (start time 101000 mSec), but the nearest neighbor is selected according to the algorithm shown in FIG. 15. As a result, 101000 mSec with scene change ID = 3 is selected, and this is reflected in Table 3. Yes.

<Quick movie playback unit 200>
The quick video playback process (step S107) performed by the quick video playback unit 200 performs double-speed playback with voice at a speed at which a person can hear and grasp the content of the voice segment (section A). For a section (section B) that is not a person's voice section, playback is performed at a high speed and a double speed within a range in which the user can grasp the content by viewing the playback video.

  In recent years, the moving image playback environment has been improved. For example, when a Microsoft DirectShow module is used, it is possible to perform continuous playback by designating the speed of an arbitrary section. By using a module having such a function, it is possible to realize a change in the playback speed of an arbitrary section relatively easily, and in that case, what is important is how to change the speed.

  FIG. 16 is a flowchart showing the moving image quick-view playback process in the present embodiment.

  In the figure, in step S1701, the user selects a desired one from the above-described user profile 14, and as a specific procedure, for example, a user as illustrated in FIG. -A display screen including a profile list may be displayed, and a user may select a desired profile from among them using a remote control terminal or the like.

  That is, the user-desired profile designation in the user profile list shown in FIG. 18 is illustrated in FIG. 20 in accordance with, for example, a remote control terminal provided with an operation button for profile selection, which is pressed by the user. Such a menu display screen is displayed, and the user designates a desired profile using the profile selection operation buttons of the remote control terminal while viewing the screen. Of course, user profile selection can also be done by automatic profile selection using personal recognition technology such as fingerprints, voiceprints, and face recognition. This is because it is always possible to specify the correct profile. You can prevent troubles such as making mistakes, changing other people's profiles, and looking into the contents.

  In addition, when a user profile is newly registered, when the “new registration” button is designated by a pointer device on the display screen of FIG. 18, the display illustrated in FIG. 19 is used to input a profile name and other attributes. A screen appears.

  That is, FIG. 19 is a diagram exemplifying a display screen for user profile registration. In the initial state, contents other than the identification name and the age are filled with reference values, and a unique identification name is obtained by an input operation by the user. Only the location where the input of the age needs to be changed is changed, and after passing the appropriate check of the predetermined input value range, the profile is changed to the user profile 14 in response to the user pressing the “OK” button. It is newly registered.

  Further, when the user desires to change the contents of a desired profile, it is displayed in response to pressing the “change” button on the display screen shown in FIG. 18 and selecting the desired profile on the display screen shown in FIG. In the display screen of FIG. 19, after changing the information content of the item desired to be changed, the “OK” button may be pressed.

  Further, when the user desires to delete the desired profile, the user presses the “delete” button on the display screen shown in FIG. 18, selects the desired profile on the display screen shown in FIG. 20, and then clicks the “OK” button. Just press it.

  When the “cancel” button is pressed on the display screens shown in FIGS. 18 and 19, the processing (profile registration, change, and deletion) corresponding to the selection operation and input operation so far is performed. The process ends without

  Next, in step S1702, it is determined whether the profile selected in step S1701 exists in the user profile 14, and if it exists, the target profile is read from the user profile 14 in step S1703. If it does not exist, the playback speeds of the sections A and B and the volume during playback of the section B, which are preset as reference values, are read in step S1706. Here, an example of the data schema of the user profile is shown in Table 4.

  Table 4 is a table illustrating a user profile in the present embodiment. The reference value may be stored as shown in profile ID = 0. In this case, the playback speed of the section A is 1.5 times faster, the playback speed of the section B is 10.0 times faster, and the reference of the volume during playback of the section B The value is 0 (ie audio mute). This value is used as the reference value used when registering a new user profile.

  Also, in the data schema 1 of the user profile in Table 4, “None” indicates that no value is set. Conversely, if a value is set, playback is performed with the highest priority on that value. In Table 4, “Good” and “Poor” in the visual acuity and hearing fields represent the dynamic visual acuity and the ability to hear early voice regardless of the age of the person.

  In general, the older people become harder to hear, the lower the speed at which they understand words, and the lack of language skills makes children unable to understand when they play voice at high speeds. Many.

  Based on these circumstances, templates for the playback speed of the section A suitable for the age of the healthy person and the playback speed of the section B are prepared in advance, and these speeds are based on the age stored in the user profile 14. To decide.

  However, despite being adolescents, people with weak visual acuity and fast voice hearing, or languages other than their native language (for example, Japanese language) because of foreigners can not catch up with their understanding, etc. There may be unrelated causes. For this reason, in the present embodiment, the characteristics of visual acuity and hearing ability are described as in the user profile exemplified in Table 4, and if there are these settings, this is given priority, and the playback speed of section A, and The playback speed of section B is determined to be low.

  In such a case, for elderly people and users with weak moving vision, it may be out of the viewpoint of the original quick-playing, but the playback speed of the human voice section (section A) is determined to be slower than the normal speed. However, by making the playback speed of a section (section B) that is not a human voice section equal to or higher than the same speed, the user concerned can perform the slow playback that can grasp the voice content of the section A, It is possible to view a moving image in a time faster than the low speed playback of the section.

  Also, for users with weak hearing ability for fast-spoken voices and users who cannot understand in fast-spoken Japanese because of foreigners, the playback speed of section A is determined to be slower than the normal speed, and the playback speed of section B As for the whole, while performing low-speed playback that can grasp the audio content of the section A by setting the same playback speed as that of a healthy person of that age, as a whole, the video is played at a faster time than the low-speed playback of all the sections. It becomes possible to browse.

  As described above, in the present embodiment, the speed determination process for the user profile is performed in advance for a normal person's age-appropriate section A playback speed and section B playback speed template, dynamic visual acuity, and a symptom with a weak early hearing. Because of the foreigners, we make comprehensive judgments taking into account the situation where we cannot catch up with the quick Japanese.

  In the present embodiment, whether or not the language of the audio content is proficient is determined based on whether or not the proficiency stored in the user profile 14 is proficient, or language type information for specifying the native language, and the reproduction target This is done by comparing the language type information of the audio content included in the video. In recent years, digital content such as a DVD and digital media such as a digital BS have stored language type information that specifies the language of audio content, and recently, the program content is electronically stored from an EPG (electronic program guide) or the like. It is practical to use this information because it is available. Even if this information is not available, even in a terrestrial TV program, the default setting is native language, and in bilingual voice, the main voice is usually the native language and the sub voice is a foreign language. What is necessary is just to estimate based on a law.

  In step S1704, the playback speed of section A and the playback speed of section B are determined based on the user-desired profile read in step S1703. Here, the details of the processing in this step will be described with reference to FIG.

  FIG. 17 is a flowchart showing details of the processing in step S1704 (FIG. 16) in the flowchart showing the quick-view movie playback processing in the present embodiment.

  In the figure, first, in step S1801, the profile previously selected by the user is read from the user profile 14, and in step S1802, the optimum section according to the age of the healthy person is obtained according to the user's age acquired from the read profile. By referring to a template in which the playback speed of A and the playback speed of section B are set, the playback speed of section A and the playback speed of section B for the user are provisionally determined.

  In step S1803, it is determined whether the profile read in step S1801 describes that the moving body visual acuity is weak. If this is described, the playback speed of section A and section B are determined in step S1804. Both playback speeds are updated to values lower than the reference value. Therefore, it is desirable to store this value in the profile in advance.

  In step S1805, since it is determined in step S1803 that the dynamic visual acuity is not described in the profile, it is determined whether or not it is described in the profile that the hearing ability of fast speech is weak. If yes, only the playback speed of the section A is updated to a low value in step S1806. Therefore, it is desirable to store this value in the profile in advance.

  In step S1807, since it is determined in step S1805 that it is not described in the profile that the hearing ability of fast voice is weak, it is determined whether the language type information of the audio content included in the moving image data to be reproduced is available. If YES in step S1808, the process advances to step S1808; otherwise, the process ends.

  In step S1808, the language type information of the audio content included in the moving image data to be reproduced is obtained, and the obtained language type information is compared with the good language information described in the currently selected profile. If the two types of information match, the process ends. If they do not match, only the playback speed of section A is updated to a lower value in step S1809. Therefore, it is desirable to store this value in the profile in advance.

  That is, in the series of processing shown in FIG. 17, if none of Steps S1803, S1805, and S1808 is hit, the playback speed of section A and the playback speed of section B temporarily determined in step S1802 are set. It will be adopted as it is.

  If the dynamic visual acuity or early voice hearing is excellent despite being old or young, or vice versa, use the menu for changing the playback speed of section A and the playback speed of section B. It is preferable that these values can be changed. In this case, the user appropriately changes the playback speed of the section A and the playback speed of the section B while watching the playback video, and automatically or prompts the user to confirm the set playback speed information. By storing the profile in the profile corresponding to the user, it is possible to perform easy-to-understand moving image quick reproduction corresponding to each user while reflecting the previous operation information.

  If it is simply performed without using the above-described profile, for example, instead of the processing in each step of steps S1701 to S1704 and step S1706, the playback speed of section A is changed from 0.5 times to 2 times speed. An embodiment is assumed in which the user can variably set the playback speed of B between 2 × speed and 10 × speed using an operation menu.

  By the way, when the section B is played back at high speed at a high speed, the sound “curl” is heard. However, if the user does not want to hear the sound, the sound playback is muted when the section B is played. Alternatively, an embodiment in which the volume is changed to a small volume is envisaged. Such a setting is also described in advance in the profile read in step S1703, and when it is determined that there is no profile in step S1702 when it is determined in step S1702 that the profile has the highest priority at the time of quick playback of the movie, in step S1706 A preset reference volume is adopted. Of course, if it is performed more simply, for example, an embodiment may be assumed in which how to quickly process the moving image quick playback processing determines the audio playback level of the section B in advance.

  With the configuration as described above, in the present embodiment, designation of the playback speed of the section A and the playback speed of the section B, or both, and the sound level of the section B can be made for each user by using the user profile. Optimal reproduction can be easily realized.

  Next, in step S1705, the fast-playing playback section information, which is the audio section information corrected in the fast-playing playback section correction process (step S104), is read from the video quick-view index storage unit 11, and in step S1707, the total length of section A is read. Is divided by the playback speed to calculate the playback time of section A, and similarly for section B, the playback speed is calculated, and the time required for the user to calculate is calculated by adding these two values. The calculated time required for quick viewing is presented to the user using the display 23 or the like.

  In step S1708, it is determined whether or not the user who has recognized the quick playback time in step S1707 is satisfied with the time by using an input operation or the like to the remote control terminal, and the user is satisfied with this determination. In this case, in step S1710, the reproduction target moving image stored in the moving image data storage unit 10 is reproduced in accordance with the reproduction speeds of the sections A and B and the sound reproduction level of the section B set by the above-described processing. .

  In step S1709, since it is determined in step S1708 that the user is not satisfied, the playback speed of section A and section B and the audio playback level of section B can be changed so as to be within the user's desired playback time. By providing the man-machine interface, adjustment is made so that the playback time desired by the user himself who is not satisfied with the profile and the standard setting is desired, and the process returns to step S1707.

  As another embodiment corresponding to step S1709, the user-desired playback speed can be changed for each section while watching the video playback based on the currently set playback speeds of section A and section B. A configuration is also possible in which the time required for quick viewing according to the configuration is calculated and presented so that the playback time desired by the user himself who is not satisfied with the profile or the standard setting is adjusted.

  By the way, as to the relationship between the user profile and the user's desired speed instruction, in step S1707, the user who has seen the time required for the quick movie playback can set the interval A and When these settings are adjusted / changed using a man-machine interface capable of changing the playback speed of B, the value after the adjustment / change may be used as a reference value. Therefore, in such a case, when “Yes” is selected after prompting confirmation by the user automatically or on the confirmation screen illustrated in FIG. 21, the playback speed adjusted / changed by the user. By storing the information in the profile corresponding to the user, it is possible to perform easy-to-understand quick-playing of the moving image according to the user while reflecting the previous operation information in the subsequent moving-image playing.

  In the above-described embodiment, the number of zero crossings and the sound energy are used as the sound labeling process, but the specific processing procedure is not necessarily limited to the above algorithm, and a known feature amount is used or different. A label determination algorithm may be used.

  That is, the purpose of the voice detection processing according to the above-described embodiment is to divide the voice signal into a plurality of rational voice segments (voice sections) using the zero crossing information of the voice signal subjected to the low-pass filter. At that time, after detecting the voice pitch by waveform processing and performing the voice labeling, the voice characteristics such as the CVC voice model are used with reference to a predetermined voice pitch that always accompanies a vowel that occupies most of the human voice. By integrating the plurality of audio segments, even when a disturbance such as BGM is included in the audio signal, a process for recovering the disturbance is included.

  Therefore, there are no restrictions on how to implement the AGC 21 and the low-pass filter 22, and the audio labeling is not necessarily limited to the algorithm of this embodiment, and a different label determination algorithm may be used.

  Also, in the determination process (FIG. 11) performed by the speech segment determination unit 28, a process for obtaining a speech segment by integrating unvoiced consonant segments or voiced consonant segments and pitch segments, which is performed in step S1106. , The order of the processing for obtaining a speech section by integrating two pitch segments having a silence segment or a noise segment adjacent or between them, which is performed in S1107, is not limited to the above-described embodiment, An algorithm that processes these processes in parallel may be used.

  In the embodiment described above, as a procedure for selecting a user profile, the user appropriately designates a profile selection screen using the remote control terminal, and himself / herself is selected from the user profile list displayed on the display 12. However, the present invention is not limited to this configuration. For example, a configuration that prevents operations such as changing or deleting another user's user profile using a password may be employed.

  In addition, automatic profile selection methods using personal recognition technologies such as fingerprints, voiceprints, and face recognition are also conceivable. In these cases, it is necessary to prevent operations such as changing or deleting another user's profile with a password. It is convenient.

  In the above-described embodiment, the user confirms the calculated time required for quick playback, and then changes the playback speed of section A and the playback speed of section B so that the playback time is within the user's desired playback time. The configuration example in which the user who is not satisfied with the profile or the standard setting adjusts the playback time to be close to the desired playback time has been described. However, the present invention is not limited to this configuration. The playback speed of A and the playback speed of section B are configured to be changeable, the time required for quick reference according to the setting is recalculated, and this is presented to the user so that the playback time desired by the user himself / herself There are also embodiments that adjust to be close to.

  As described above, according to the present embodiment, the basic voice utterance mechanism produced by a person is vocal cord vibration, so-called voice pitch. By extracting this from the voice signal, a useful voice section is obtained, and Detecting the voice section of the person, using that section, without breaking the synchronization relationship between the video and audio, at the time of video quick playback, while all the voice that the person uttered is played at a speed that can grasp the content, A section (section B) that does not include human-generated speech is played back at a higher speed. This makes it possible to rationally reduce the total browsing time at the time of quick movie playback compared to the case where the same size playback is performed.

  In addition, according to the present embodiment, the playback speed of the section A and the playback speed of the section B can be easily set to playback speeds suitable for individual users by using the user profile 14, and the section B The volume during playback can also be set to be suitable for the user.

  Furthermore, according to the present embodiment, by displaying the time required for quick playback in advance or during the playback of a moving image, a user who is not satisfied with this can specify the playback speed of section A and the playback speed of section B. Thus, it is possible to adjust the time required for the fast-playback optimum for the user, and the information set by the adjustment can be updated and stored in the profile corresponding to the user. Video playback.

(Other embodiments)
The present invention described using the above-described embodiments as an example may be applied to a system including a plurality of devices, or may be applied to an apparatus including a single device.

  In the present invention, a software program that realizes the functions of the flowcharts described in the above embodiments is directly or remotely supplied to a system or apparatus that operates as the above-described moving image reproducing apparatus, and a computer of the system or apparatus is provided. Is also achieved by reading and executing the supplied program code. In that case, as long as it has the function of a program, the form does not need to be a program.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. That is, the claims of the present invention include the computer program itself for realizing the functional processing of the present invention.

  In this case, the program may be in any form as long as it has a program function, such as an object code, a program executed by an interpreter, or script data supplied to the OS.

  As a recording medium for supplying the program, for example, floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card ROM, DVD (DVD-ROM, DVD-R) and the like.

  As another program supply method, a client computer browser is used to connect to an Internet homepage, and the computer program of the present invention itself or a compressed file including an automatic installation function is downloaded from the homepage to a recording medium such as a hard disk. Can also be supplied. It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. In other words, a WWW (World Wide Web) server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the claims of the present invention.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who have cleared predetermined conditions. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  In addition to the functions of the above-described embodiments being realized by the computer executing the read program, the OS running on the computer based on an instruction of the program is a part of the actual processing. Alternatively, the functions of the above-described embodiment can be realized by performing all of them and performing the processing.

  Furthermore, after the program read from the recording medium is written to a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion board or The CPU or the like provided in the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are also realized by the processing.

It is a figure showing the conceptual diagram of the moving image quick look-up algorithm in the moving image reproducing apparatus which concerns on this embodiment. It is a block diagram showing the algorithm for audio | voice area (section A) detection of the person's utterance period performed in the animation quick reference index preparation part 100. FIG. It is a flowchart which shows the outline of the process based on the algorithm shown in FIG. It is a figure explaining the joining process of the small segment performed in this embodiment. It is a flowchart which shows the audio | voice labeling process performed in this embodiment. It is a figure explaining the process from the segmentation of the audio | voice signal waveform in this embodiment to labeling. It is a figure which illustrates the audio | voice signal waveform for description of the audio | voice pitch detection process in this embodiment. It is a figure explaining the update procedure of the pitch detection reference | standard performed by the audio | voice pitch detection process in this embodiment. It is a flowchart which shows the audio | voice pitch detection process in this embodiment. It is a flowchart which shows the detail of a process of step S904 (FIG. 9) among the flowcharts which show the audio | voice pitch detection process in this embodiment. It is a flowchart which shows the audio | voice area determination process in this embodiment. It is a flowchart which shows the detail of the process of step S1106 (FIG. 11) among the flowcharts which show the audio | voice area determination process in this embodiment. It is a flowchart which shows the detail of a process of step S1107 (FIG. 11) among the flowcharts which show the audio | voice area determination process in this embodiment. It is a flowchart which shows the integrated correction process performed with respect to the audio | voice area with a short space | interval in this embodiment. It is a flowchart which shows the audio | voice area integrated correction process performed using a scene change point in this embodiment. It is a flowchart which shows the moving image quick-reproduction process in this embodiment. It is a flowchart which shows the detail of a process of step S1704 (FIG. 16) among the flowcharts which show the moving image quick view reproduction | regeneration processing in this embodiment. It is a figure which illustrates the display screen for user profile selection. It is a figure which illustrates the display screen for user profile registration. It is a figure which shows the example of the user profile in this embodiment. It is a figure which illustrates the display screen which urges | confirms whether the value adjusted and changed is used as a reference value at the time of the next or subsequent video playback when a user who is not satisfied with the time required for the quick video playback is presented. .

Claims (16)

A video playback device capable of playing back video data including audio signals at high speed,
A voice segment determination means for determining a first voice segment representing a person's utterance period and a second voice segment other than that based on a voice signal included in the video data;
Based on the moving image data, the first audio section performs high-speed moving image reproduction with reproduced sound at a predetermined speed at which the user can grasp the contents, while the second audio section is higher than the predetermined speed. With the quick-playing means for high-speed video playback,
With
The quick-view playback means calculates the time required for the high-speed video playback based on the length of the first audio section and the playback speed of the section and the length of the second audio section and the playback speed of the section. A moving image reproducing apparatus characterized by calculating and presenting the calculated required time to a user. A video playback device capable of playing back video data including audio signals at high speed,
A voice segment determination means for determining a first voice segment representing a person's utterance period and a second voice segment other than that based on a voice signal included in the video data;
Based on the moving image data, the first audio section performs high-speed moving image reproduction with reproduced sound at a predetermined speed at which the user can grasp the contents, while the second audio section is higher than the predetermined speed. With the quick-playing means for high-speed video playback,
With
The quick-view reproduction means targets the user who can use the moving image reproduction apparatus, according to the attribute information related to a specific user registered in the user profile in which attribute information related to each user is registered. A moving picture reproducing apparatus, wherein the reproducing speed of two audio sections is automatically determined.   3. The moving image reproducing apparatus according to claim 1, wherein the quick-view reproduction unit performs moving image reproduction with reproduced sound having a volume lower than the volume in the first audio interval in the second audio interval.   3. The moving image reproducing apparatus according to claim 1, wherein the quick-view reproducing unit performs moving image reproduction with no sound in the second audio section. 4.   The quick-view playback means, when the operation for changing the playback speed of the first and second audio sections is performed by the user in response to presenting the required time, based on the playback speed after the change, 2. The moving image reproducing apparatus according to claim 1, further comprising adjusting means for adjusting the required time. The quick-view reproduction means includes
Based on the length of the first voice section and the playback speed of the section, and the length of the second voice section and the playback speed of the section, which are automatically determined according to the attribute information about the specific user, In addition to calculating the time required for video playback and presenting the calculated required time to the user, the operation for changing the playback speed of the first and second audio sections is performed by the user in accordance with the required time. The moving image playback apparatus according to claim 2, further comprising an adjusting unit that adjusts the required time based on the playback speed after the change.   2. The moving image reproducing apparatus according to claim 1, wherein the predetermined speed at the time of high speed moving image reproduction in the first audio section is 1.5 to 2 times as fast as constant speed reproduction.   The quick-playing unit responds to the identification information when the attribute information related to the user registered in the user profile includes identification information indicating that the user is an elderly person, a visually impaired person, or a hearing impaired person. When the high-speed moving image playback is performed for a user who performs the above-described operation, the playback speed of the first audio section is slower than the same speed and the playback speed of the second voice section is higher than the same speed. The moving image reproducing apparatus according to claim 2.   The quick-playing means includes identification information indicating the user's language in the attribute information related to the user registered in the user profile, and the identification information and language type information included in the video data. If they do not match, when performing the high-speed video playback for the user corresponding to the identification information, the playback speed of the first audio section is slower than the normal speed, and the playback speed of the second audio section is 3. The moving image reproducing apparatus according to claim 2, wherein the moving image reproducing apparatus is performed at 5 times speed to 10 times speed. In the user profile, attribute information about individual users is registered for a plurality of users who can use the video playback device,
The moving image reproduction according to claim 2, wherein the quick-view reproduction means acquires attribute information about the specific user from the user profile in accordance with a selection operation of the specific user or based on a personal authentication technique. apparatus. 3. The moving image reproducing apparatus according to claim 2, further comprising attribute information changing means capable of changing attribute information relating to a specific user registered in the user profile by the specific user. A video playback method for playing back video data including audio signals at high speed,
A voice section determination step of determining a first voice section representing a person's utterance period and a second voice section other than the first voice section based on a voice signal included in the video data;
Based on the moving image data, the first audio section performs high-speed moving image reproduction with reproduced sound at a predetermined speed at which the user can grasp the contents, while the second audio section is higher than the predetermined speed. In the fast-play playback process to perform high-speed video playback,
With
In the quick-view playback step, the time required for the high-speed video playback is calculated based on the length of the first audio section and the playback speed of the section, and the length of the second audio section and the playback speed of the section. A moving image playback method characterized by calculating and presenting the calculated required time to a user. A video playback method for playing back video data including audio signals at high speed,
A voice section determination step of determining a first voice section representing a person's utterance period and a second voice section other than the first voice section based on a voice signal included in the video data;
Based on the moving image data, the first audio section performs high-speed moving image reproduction with reproduced sound at a predetermined speed at which the user can grasp the contents, while the second audio section is higher than the predetermined speed. In the fast-play playback process to perform high-speed video playback,
With
The quick-playing step automatically determines the playback speed of the first and second audio sections according to the attribute information related to the specific user registered in the user profile in which the attribute information related to each user is registered. A video playback method characterized by the above.   14. The moving image reproduction method according to claim 12 or 13, wherein, in the quick-playing step, moving image reproduction with reproduced sound having a volume lower than the volume in the first audio interval is performed in the second audio interval.   14. The moving image reproduction method according to claim 12 or 13, wherein in the quick-view reproduction step, the moving image reproduction is performed without sound during the second audio section.   A computer program for causing a computer to execute the moving image reproduction method according to any one of claims 12 to 15.

Images