JP2009004994A - Digest picture information creating method, digest picture information creating program, and picture apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To automatically generate digest information of the digest picture suited to users by interpreting taste of users without input of suggesting method and parameters. <P>SOLUTION: Characteristic information is extracted from picture data in accordance with level of the acoustic surround and keywords. Manipulation information of users is recorded and manipulation importance is obtained for the time of the relevant picture by evaluating the scoring points. A scene importance information is generated for the scene to regenerate the digest picture that is determined with parameters for designating the regeneration time of the digest from the picture. Next, correlation between the manipulation importance information and scene importance information is evaluated to determine combination of parameters for increasing a degree of correlation. The optimum scene importance information is generated by generating the scene importance information and tentative update of parameters. Finally, digest regenerating position information is generated in accordance with the scene importance information generated and the regeneration time of digest. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a digest video information creation method, a digest video information creation program, and a video apparatus. When creating digest video information for displaying a digest video as a summary from given video data such as a televised video. The present invention relates to a digest video information creation method, a digest video information creation program, and a video apparatus that can create digest video information for a digest that matches a user's viewing preference without complicated operations by the user.

  In recent years, it has become possible to record videos of a large number of broadcast channels for a long time by improving the performance of video information equipment. In addition, the spread of satellite broadcasting and the like has increased the number of broadcast channels, and the number of videos provided to viewers is increasing more than doubled. As a result, there is a need for viewers to efficiently select a video that suits their preference from among a large number of videos.

  As a solution to this, it has been proposed to provide a digest video creation function in a receiver or a recording device. For example, Patent Document 1 discloses a method of creating a digest video by extracting an important part of the video from the video volume and telop display.

  Further, Patent Document 2 discloses a method of digitizing based on rule information created according to user's preference and creating a digest video based on the numerical evaluation.

  Further, it is clear that the preference for the digest video to be created is different for each user, and it is necessary to create so that a different part for each user is a combination of video parts constituting the digest video. Conventionally, this method is based on a symbol sequence (character string, audio feature time series, video feature time series, etc. or a symbol sequence obtained by encoding them) specified by the user in the video and audio in the video and the attribute data. The search result has been reproduced as a digest desired by the user by searching for an existing symbol string or searching for the event information described above.

JP 2000-23062 A JP 2000-261754 A

  However, with these conventional techniques, it is difficult to automatically interpret each user's preference and create a digest. That is, in Patent Document 1, since a digest is simply created by using telop or acoustic information as a digest target, it is good to some extent in fields such as live sports broadcasting, but in other fields, it is not necessarily suitable for the user. It is not always possible to create an appropriate digest. In Patent Document 2, rules and parameters are introduced to evaluate the importance of the situation in a baseball game, numerical video information is created, and a digest video is finally created. However, since such a rule lacks versatility, it must be prepared for each type of video, and there is no guarantee that the digest video created by the video will match the user's preference. In addition, digest generation by search requires the user to input a symbol string or parameter that expresses the preference explicitly.

  In the digest video information creation method of the present invention, first, feature information is extracted from video data of a video. As the feature information, the size of the sound excitement of the video data, a keyword displayed in the video telop, and the like are considered.

  On the other hand, by recording operation information such as playback and fast-forwarding that the user has instructed on the video, the importance of the operation is obtained with respect to the time of the video by scoring evaluation for such operation. .

  Also, a parameter for designating the digest playback time from the video is introduced as calculation content information. This is, for example, a value that adjusts the threshold of sound excitement, the number of keyword evaluations, the start time of playback, and the end time. These may be provided for each user, for each user, for each genre, and for each genre.

  Then, scene importance level information that is the importance level of each scene at the playback time evaluated by the feature information is generated with respect to the scene at the playback time to be played back as a digest.

  Next, the degree of correlation between the operation importance level information and the scene importance level information is evaluated, and a combination of parameters for increasing the correlation level is determined.

  Then, scene importance information is generated and parameters are updated for a while to generate optimum scene importance information.

  Finally, digest playback position information is generated based on the generated scene importance level information and digest playback time.

  According to the above means, when creating a video digest, it is possible to automatically create digest video information for a digest suitable for the user's preference by interpreting the user's preference without inputting an explicit method or parameters. A digest video information creation method can be provided.

  Hereinafter, each embodiment will be described with reference to FIGS. 1 to 36.

Embodiment 1
Hereinafter, a first embodiment will be described with reference to FIGS.

In the present embodiment, a method is described up to the procedure of creating digest video information of a digest according to user preferences using a user's operation history and reproducing video data as a digest. In the present embodiment, a description will be given in the case where digest video information is created based on the acoustic climax from the audio data included in the video. This acoustic excitement is an effective method for creating a digest video for a video that includes the cheer of the audience in the audio data, such as a sports video on TV. For example, in the vicinity of a scene scored (goal) in a soccer game or when a good play occurs, it is often used with a focus on raising cheers as a natural reaction of the audience. Digest video information is created by selecting a portion with such a high degree of excitement and cutting the video. It is also assumed that the user specifically views a similar exciting scene part. This way of viewing is generally not to watch the whole video when watching after recording, but to shorten the time by watching the important part by fast-forwarding or skipping the unimportant part. This is a natural way to watch. In other words, when creating digest video information of sports video in this way, it is effective to place special emphasis on sound excitement. However, in this embodiment, a digest video suitable for the characteristics and preferences of the user is provided. To do. Therefore, in this embodiment, when the user repeats viewing, the viewing behavior is evaluated, and the viewing habits are analyzed, the preference pattern is taken into consideration by analyzing the viewing habits, and the parameters of the sound excitement level and the playback section are set. By fine-tuning, an optimum digest playback section is sought.
(I) Configuration of Video Device that Plays Digest Video First, the configuration of the video device that plays the digest video according to the first embodiment will be described with reference to FIGS. 1 and 2.

FIG. 1 and FIG. 2 are block diagrams showing the overall configuration of a video apparatus that reproduces a digest video according to the first embodiment.
The video device for reproducing the digest video shown in FIG. 1 includes a recording / playback unit 180 and a digest video information creation unit 100.

  The recording / playback unit 180 inputs an external broadcast video 190 and media video 191 via the input unit 181. The input is recorded in the video information recording unit 184 or sent to the decoding unit 185 under the control of the recording / playback control unit 183. Further, the video already recorded in the video information recording unit 184 may be sent to the decoding unit 185. These controls are performed by the recording / playback control unit 183 according to an operation performed by the user using the operation unit 182. This user operation can be output from the recording / playback control unit as operation information 187 described later. At this time, the signals may be sequentially output almost simultaneously with the operation or may be output collectively after being recorded in the video information recording unit 184. The output of the decoding unit 185 is displayed as a video on the display device 186 via the recording / playback control unit 183 and provided for viewing by the user. The video data in the video information 110 recorded in the video information recording unit 184 includes accompanying moving image data, audio data, and symbol data. The video information includes attribute information for explaining the video data such as the genre and title of the video data. The video data may be either the encoded data obtained from the input unit 181 or the data decoded by the decoding unit 185, and the same applies to the following description. Furthermore, an authentication unit 188 may be provided and managed in order to identify and display a user and manage video data.

  Furthermore, an exchange unit 120 may be provided to exchange video data and various types of information managed by each unit with the outside via the network 170 or the external medium 160. The operation of the recording / playback unit 180 is not different from that of a general video recording device including a VTR (Video Tape Recorder), a DVD (Digital Versatile Disc) recorder, an HDD (Hard Disk Drive) recorder, and a PC.

  As shown in FIG. 1, the digest video information creation unit 100 includes a calculation content management unit 101, an operation information analysis unit 102, a comparative evaluation unit 103, a feature extraction unit 111, a scene importance calculation unit 114, and a reproduction position information generation unit. 115 and digest creation control unit 120, and handles data of action score table 104, calculation content information 105, operation importance level information 106, feature information 112, scene importance level information 114, and reproduction position information 116.

  The calculation content management unit 101 manages the calculation content information 105 and is a part that accesses and updates the calculation content information 105.

  The operation information analysis unit 102 is a part for setting / updating the operation importance 106 with reference to the operation information 187 sent from the recording / playback unit 180 and the action score table 104.

  The comparative evaluation unit 103 is a part that refers to the operation importance level information 106 and the scene importance level information 114 and updates the correlation level under the current parameters.

The feature extraction unit 111 is a part that extracts the feature information 112 from the features of the video information 110 sent from the recording / playback unit 180.
The scene importance level calculation unit 114 is a part that calculates the scene importance level information 114 under the condition of each parameter.

  The playback position information creation unit 115 is a part that generates playback position information 116 from the scene importance level information 114.

  The control unit 120 is a part that controls each unit of the digest video information creation unit 100 and gives an instruction.

  The data structure handled by the digest video information creation unit 100 and the details of the processing of each unit will be described in detail later.

  In addition, the video device that reproduces the digest video shown in FIG. 2 is intended to create digest video information by software. The video device 3801 here is, for example, a PC with a TV function. 1 may be realized by a general device such as a DVD / HDD recorder or a television.

  As shown in FIG. 2, the video apparatus that reproduces the digest video in this example includes a CPU 3812, a memory 3810, and a storage unit 3824. Then, the same function as the digest video information creation unit 100 of FIG. 1 may be executed by causing the CPU 3812 to execute a program.

  At this time, various kinds of information for creating the digest video information, that is, the action score table 104, the operation importance level information 106, the calculation content information 105, the feature amount information 112, the scene importance level information 114, and the reproduction position information 116 are stored. The data may be stored in the storage unit 3824.

Further, the recording / playback unit 180 is also configured in the apparatus, that is, executed by the display unit 3813, the input unit 3820, the authentication unit 3822, the operation unit 3823, the operation input unit 3830, the operation panel 330, and the exchange unit 3825. Each processing program for recording and reproducing and displaying the video to be generated and creating user operation information may be operated on the CPU. At this time, the video information 110 and the operation information 187 may be stored in the storage unit 3824. In addition, input from devices such as the remote controller 341, the keyboard 342, and the mouse 343 may be performed from the operation input unit, or various types of information stored in the storage unit from the exchange unit may be exchanged with the outside.
(II) Information and Data Structure Related to Digest Video Next, information and data structure related to the digest video according to the first embodiment will be described with reference to FIGS.
FIG. 3 is a diagram illustrating an example of data stored in the video information recording unit 184.

  In the example shown in FIG. 3, a video management table 201 and individual video i information 210... Are stored. These are shown as video information 110 in FIG. The video management table 201 includes an ID that is an identifier of a video to be managed, a recording date and time, a recording start time and an end time, and a video acquisition destination. Here, in the case of broadcasting, the acquisition destination of the video is described as such if it is a TV channel, a provider, or an external medium. Also, the recorded video is managed for each ID, and video data and video attributes are recorded as video i information 210, 220,. The video data 211 is digital data including moving images, sound, character information, and the like in the MPEG-2 TS (Transport Stream) format, for example, for TV digital broadcasting. The video attribute 212 is information that describes the corresponding video, for example, EPG (Electric Program Guide: electronic video table) that is generally distributed for TV broadcasting, and information that describes the video included in the video data. Etc. This content attribute includes, for example, the genre to which the video belongs, the title of the video, the audio mode included in the video, the presence or absence of subtitles, the description of the video content, and the like. These pieces of video information can be acquired with reference to the ID for each video.

Next, the operation information 187 will be described with reference to FIG.
FIG. 4 is a diagram illustrating an example of the operation information 187.

  The operation information 187 is history information that the user has operated the content on the video apparatus. Here, it is assumed that each user can manage and each operation can be accumulated as a history. Each operation information includes, for example, the date and time when the operation was performed, the action indicating the type of operation, the content ID, the parameters of those actions, and the temporal position of the video displayed when the operation was performed. (For example, the relative time when the start of the video is 0) is included.

Next, the feature information 112 will be described with reference to FIG.
FIG. 5 is a diagram illustrating an example of the feature information 112 according to the first embodiment.

  The feature information 112 is information extracted from the video information 111 in order to create digest video information. In this embodiment, the climax data is extracted from the acoustic data based on its power.

  In this example, the program feature management table 610 of the video 1 feature information 601 indicates that the sound excitement information 611 is recorded for the program whose program ID is 1. The sound climax information 611 may be registered in the management table as a set of the start time and end time and the climax degree so as to cover the entire video, and in particular, only the place where the climax degree is large is extracted. You may enter.

Next, the operation importance level information 106 will be described with reference to FIG.
FIG. 6 is a diagram illustrating an example of the operation importance information 106.

  The operation importance level is an evaluation of the importance level based on an operation performed for each video segment of each user. The operation importance information 106 is managed as user i operation importance 910, 920,... For each user of the video, and is managed as video i importance 911. If the operation importance level information 911 of the user 1 and the video 1 in the figure is described, first, the genre of the video 1 (in this figure, “sport (soccer)”) information is recorded. Further, the entire time of the video may be covered by combining the item number, the start time, the end time, and the importance, and in particular, only the place with the higher importance may be extracted and entered. .

Next, the action score table 104 will be described with reference to FIG.
FIG. 7 is a diagram illustrating an example of the action score table 104 according to the first embodiment.

  The action score table 104 defines scoring for each operation in order to calculate the operation importance. Here, the contents operated by the user include “load”, “play”, “jump”, “fast forward”, “fast reverse”, etc., and the score for each is registered as a map. In the example shown in FIG. 7, 1 point is given when “reproduction” is performed, and 0 points are given otherwise.

  As another example, for example, 1 point may be set for “playback”, −0.3 point for “fast forward”, 0.3 point for “fast reverse”, and 0 point for others. This score is based on the assumption that the user is particularly interested in the playback part when the video is played back, and is not interested in the playback part when the video is fast-forwarded and the fast-forward operation is performed. It is assumed.

Next, the calculation content information 105 will be described with reference to FIGS. 8A and 8B.
FIG. 8A is a diagram showing parameter information 1501 of the calculation content information 105.
FIG. 8B is a diagram illustrating an example of each parameter value of the calculation content information 105 according to the first embodiment.

  The calculation content information 105 is information related to calculation for creating digest video information. For example, in the present embodiment, there is parameter information for calculating scene importance. In the parameter information 1501 shown in FIG. 8A, “acoustic excitement” of the present embodiment and other parameters based on “keyword” are shown as classifications related to the calculation method.

  As parameters by “sound climax”, there are a threshold value th of sound swell and ts and te which are offset times that define a playback section so as to widen a section exceeding the threshold. It is assumed that there are steps for searching for the optimum value by changing the minimum value and the maximum value respectively. These will be described in detail later.

  The parameter by “keyword” will be described in the following second embodiment.

  Furthermore, as shown in FIG. 8B, the calculation content information 105 is also set with values of individual parameters. The parameter values include common basic calculation contents 2601 for the entire parameters, calculation contents for each common genre, which is information for each genre, and calculation contents information for each user. The calculation content information of each user is also divided into basic part information and genre-specific information.

  The calculation content information held by each user is a user-dependent part, that is, there is user i calculation content information for each of the user 1 (2010), the user 2 (2020), the user 3 (2030), etc.

  Each of them is divided into genre-based calculation contents and genre-independent basic calculation contents, and these are recorded video images that have already been recorded or newly input video information. If the attribute is obtained as a genre, the content of calculation for each genre is used, and in other cases, the digest video information is created using the content of basic calculation.

  Processing for generating the scene importance level information 114 from the values of these parameters will be described later.

  The parameter can be created from a certain video and used later to create digest video information for another video. At this time, when the user operates using a new video and plays the video, the user's preference suitable for as many videos as possible is extracted as a parameter by analyzing these parameters again. It is desirable to keep it.

Next, the scene importance level information 114 will be described with reference to FIG.
FIG. 9 is a diagram illustrating an example of the scene importance level information 114.

  The scene importance level is an index for evaluating the position where the final digest is played back. The scene importance level is defined for each video playback unit (start time, end time) (scene).

Next, the reproduction position information 116 will be described with reference to FIG.
FIG. 10 is a diagram illustrating an example of the reproduction position information 116.

The playback position information 116 indicates the playback position of the video when the video is finally played back by digest based on the scene importance information 114, and is passed to the recording / playback unit 180 to play back the video. It is data.
(III) User's video viewing image and operation information 187
Next, a user's video viewing image will be described with reference to FIG.
FIG. 11 is a diagram illustrating an image when the user is viewing a video.

  When the user views the video, the video device 301 generates an image to be presented to the user on the display unit 186 and displays it on the display device 190 as shown in FIG. On the display device 190, a video display area 302 for moving images, a video selection area 320 that enables video selection, a locator area 303 for grasping the video time being played back, and a user inputs a playback operation. It is assumed that an operation button area 310 for performing an operation, a digest function instruction area 350 for performing an instruction regarding digest reproduction, and the like are displayed.

  Further, the video device 301 shown in FIG. 11 has the recording / playback unit 180 and the digest video information creation unit 100 shown in FIG. 1, and the device panel 330 has an operation switch. The video apparatus 301 can be connected to a remote controller (hereinafter abbreviated as “remote controller”) 341, a keyboard 342, and a mouse 343, which correspond to the operation unit 182 in FIG. It is assumed that information can be input.

  In the digest function instruction area 350, a playback method change button 350 for changing from normal playback to digest playback, a genre change input button 351 for changing a genre when creating a video digest, and a method change for changing the method of creating a digest. Each component of the input button 352 or the digest time input button 353 for changing the approximate time of the digest video is displayed. Further, these components may be arranged on the remote controller 341. In FIG. 11, a playback method change button 345 is arranged on the remote controller 341.

  When the user views video data that has already been recorded in the video information recording unit 184, first, the desired content is designated. For example, a reduced display image of a still image or moving image that is a part of video data, a part of a video management table, and a part of video attributes are displayed in the video units 321, 322, 323 and the video selection area 320. The user uses the operation unit 182 such as a switch on the remote control 341 or the operation panel 330 of the apparatus main body to select one desired video from the operation unit 182. In addition, an instruction can be input from the keyboard 342 or the mouse 343, and a desired portion may be selected and determined using the pointer 316 displayed on the screen in the display device. As for the video selected in this manner, a moving image is displayed in the video display area 302, and although not shown, audio data is presented from a speaker, so that the user can view the video. The video is operated by using the same operation unit 182 as described above. For example, by selecting the operation button 310 in the display of FIG. 3, fast playback 311, stop 312, pause 313, playback 314, and fast forward 315 can be performed to change the playback state. In addition, in order to visualize and present to the user the entire position during playback, the display object called the locator 304 displayed in the locator area 303 is displayed near a part called a seek bar 305 as shown in the figure. May be. In the case of the seek bar 305 shown in FIG. 11, the leftmost corresponds to the start time of the video and the rightmost corresponds to the end time of the video, and the locator moves between them to recognize the time when the user is playing back. Is possible. In addition, it is assumed that the playback position can be jumped by manually moving the locator according to the user instruction from the operation unit. Further, the total video recording time and the current playback time may be displayed (306), and the playback position may be jumped by changing the playback time using the instruction unit.

  As described above, these user operations are input from the operation unit 182 to the recording / playback control unit 180 and the display state is changed. This content is output to the digest video information creation unit 100 as the operation information 187 shown in FIG. Is done. This output may be sequentially output at the time of user operation, or may be output collectively after recording.

  Hereinafter, a specific example of the output operation information will be described with reference to FIG.

According to the user 1 operation history 410 shown in FIG. 4, the user 1 selects the video with the video ID 1 at around 22:01 on [# 1] December 18, 2006 (the video unit in FIG. 11). 321) was selected (read from the video information recording unit 104 and put into a reproducible state). Next, at [# 2] 22:01, a playback instruction was given and playback was started. At this time, the content position starts from “00: 00: 00.000” which is the start position, and the speed is 1.0, that is, normal reproduction. Next, at [# 3] 22:02, that is, after playing for about 46 seconds (“00: 00: 46.822”), the position “0: 10: 43.024” is operated by operating the locator. In other words, the playback position was moved from “0 minute 46 seconds 822” to “10 minutes 43 seconds 024”. Furthermore, the reproduction was started from [# 4] "0: 10: 43.024". Next, at [# 5] 22:07, the position “0: 16: 01.130” is rewinded during playback, and the playback speed at that time is −5, that is, 5 times the normal speed. It started moving in the start direction of the image while playing. As described above, the history of viewing while the user 1 performed a series of operations on the video of ID1 is output as operation information.
(IV) Digest Video Information Creation Processing Next, digest video information creation processing according to the first embodiment will be described with reference to FIGS.

(IV-1) Overview of Digest Video Information Creation Processing First, an overview of digest video information creation processing will be described with reference to FIGS. 12 and 13.
FIG. 12 is a general chart showing processing up to creation of scene importance information in digest video information creation processing.
FIG. 13 is a flowchart showing processing for creating reproduction position information in digest video information creation processing.

  First, processing up to creation of scene importance information in digest video information creation processing will be described with reference to FIG.

  The entire control process is performed under the control of the digest control unit 120.

  First, an end threshold value L for a correlation change, which will be described later, is determined in advance (step 501).

  Next, when video data is input during or after recording (step 502), feature extraction processing is performed (step 503).

  The feature extraction process is a process in which the feature extraction unit 111 extracts the feature information 111 shown in FIG. 5 from the video data. This feature extraction processing will be described in detail later.

  Next, while the user is viewing (step 504), an operation history analysis process is performed (step 505).

  The operation history analysis process is a process for updating the operation importance level information 106 according to the user's operation on the video apparatus. This operation history analysis process will be described in detail later.

  Next, the type of feature information for creating digest video information is determined (step 506). In the present embodiment, the type of feature information is based on the sound excitement information.

  Next, a parameter value for creating digest video information is initialized (step 507). The processing related to the parameters will be described later in detail, but here, it is assumed that there are N.

  Further, as described in FIG. 8A, as the parameters due to “sound climax”, the threshold value th of sound swell and the offset time ts and te that define the playback section so as to widen the section exceeding the threshold as variables. Is used.

  Next, the maximum value Cmax of the degree of correlation when the parameter that matches the portion reproduced well by the user is adjusted by simulation is initialized to 0 (step 508).

  Then, a loop for determining parameters one by one is entered (step 509).

  Next, the value of the current parameter is set in the calculation content management unit 101 as parameter designation processing (step 520). The parameter specification process will be described in detail later.

  Then, based on the currently set parameters, the scene importance calculation unit 113 performs scene importance information creation processing (step 521). The scene importance level information creation process will be described in detail later.

  Next, the comparative evaluation unit 103 performs a comparative evaluation process (step 522).

  The comparative evaluation process is a process for obtaining the correlation degree while evaluating the current scene importance level and operation importance level. This comparison evaluation process will be described in detail later.

  Then, the digest video information control unit 120 acquires the correlation degree calculated by the comparison evaluation process in step 522 as c (step 523), compares it with Cmax that is the maximum value of the correlation degree (step 524), and performs correlation. If the degree c is not smaller than Cmax, the difference between the correlation degree c and Cmax is compared with the end threshold L of the correlation degree change (step 525), and the difference between the correlation degree c and Cmax is the correlation degree. If it is greater than the change end threshold L, the parameter update process at this point is performed (step 526), and the process ends. The parameter update process will be described in detail later.

  Also, when the correlation c is smaller than Cmax, or when the difference between the correlation c and Cmax is larger than the correlation change end threshold L, the calculation is performed when the difference is larger than the cutoff threshold. When the parameter change processing of the parameter in the content management unit 101 is performed (step 530) and the parameter analysis currently being analyzed is completed (step 531), the next loop is taken to analyze the next parameter ( Step 509) If the parameter analysis is not completed, the analysis is continued again from the scene importance level information creation process (step 521) by the parameter change process and the changed parameter. The parameter change process will be described in detail later.

  Next, processing until creation of reproduction position information in the video information creation processing will be described with reference to FIG.

  The process of creating the reproduction position information is a process in which the reproduction position information creating unit 115 generates the reproduction position information 116 based on the process shown in the chart of FIG.

  First, a playback time is set as to how long a digest is to be created (step 3600). The playback time may be specified by a user who views the digest, or a standard value of the apparatus may be determined.

  Next, the scene importance level information 114 is acquired (step 3601), and is rearranged in descending order of importance (step 3602).

  Next, the playback position information and the variable T for recording the total playback time are initialized to 0 (step 3603), and the scene end time and start time are obtained in the order of the rearranged scene importance information ( Step 3604). These differences are the respective scene lengths, and the sum of the current T and the scene length is compared with the specified playback time (step 3610), and is added to the playback position information as a playback location for a time shorter than the specified playback time. (Step 3611), the scene length is added to the total time T (Step 3612). As described above, the reproduction position information 116 is created within the range in which the evaluation and addition of the scene evaluation and addition do not exceed the designated reproduction time, and the items are rearranged in the order of the start time so as to be convenient for reproduction (step 3605). Registration is made in the information information 116 (step 3605). In this way, the playback position information 116 can be created in the order of scenes whose scene importance level information is highly evaluated and within a range that does not exceed the playback time.

  Based on the playback position information 116 generated in this way, the recording / playback control unit 183 plays back the digest video by playing back the video in units of scenes from the start position to the end position. The user can view and listen through the one display unit 186 and the display device 187.

(IV-2) Details of Feature Extraction Processing Next, details of the feature extraction processing 503 will be described with reference to FIGS. 5, 14 and 15 already taken up.
FIG. 14 is a conceptual diagram when generating sound excitement information as an example of feature extraction processing.
FIG. 15 is a flowchart illustrating a process for generating sound excitement information as an example of the feature extraction process.

  The feature extraction processing is processing for extracting the feature information 111 shown in FIG. 5 from the video data by the feature extraction unit 111. The feature information 111 extracted by this processing is recorded in the video information recording unit 184 or recorded. Video data obtained directly from the reproduction control unit 183 (including accompanying moving image data, sound data, and symbol data. The feature information 111 is encoded data obtained from the input unit 181. Any of the data decoded by the decoding unit 185 may be used.

  In order to execute the feature extraction process, the video information 110 is first input to the feature extraction unit 111. A physical feature for the video data is calculated from the video data 211 (FIG. 3) in the video information, or a symbol feature based on symbol processing is generated and stored as the feature information 112. Here, the physical feature is a statistic calculated from video data based on a physical principle. The physical feature expresses the characteristics of the video data, but does not necessarily match the importance of the video in accordance with the viewer's feeling. In addition, the symbol feature is a symbol string included in video data, for example, a new symbol string obtained by decomposing part-of-speech such as a noun or a verb from a character string explaining a video such as subtitles, or a method for reading a character string. Phonetic symbol string.

  The feature by the sound rise of the image handled in the present embodiment belongs to the physical feature. Examples of symbol features are to be dealt with in the following second embodiment.

  If the physical feature and the symbol feature are already attached to the video data 110 and sent, they may be stored directly as feature information 112 or input directly to the feature extraction unit 111.

Next, the case where the feature extraction unit 111 extracts and applies the climax data based on the power of the acoustic data as the feature will be specifically described as an example.
As already shown in FIG. 5, the feature information 112 is recorded in the program feature management table 610 of the video 1 feature information 601 with the sound excitement information 611 for the program whose program ID is 1. As shown in FIG. 14, the input acoustic data for feature extraction includes sample data sampled (sampled) at sufficiently short intervals (acoustic samples). And the mean square of these values is obtained in advance. This is called acoustic power information 701. At this time, in order to reduce the data amount, a process of converting (down-sampling) into a predetermined long sampling period by a method such as a moving average method may be performed.

  Hereinafter, the processing when generating the sound excitement information will be described with reference to FIG.

  Specifically, FIG. 15 shows a process of creating the degree of climax 710 from the acoustic power 701 shown in FIG. In this processing, from the time series of the input acoustic power, a parameter τ (702 in FIG. 7) is used to output as a feature that takes a degree of climax of sound between 0 and 1. Note that the movement section length τ is determined in advance.

  First, while shifting the sound time t from 0 until the sound power reaches its maximum time, Pmin that gives the lowest value among the sound power values in the section from t to t + τ is selected as the value at time t (step 801). ).

  This value is between the minimum value and the maximum value of the sound power, but is normalized so as to correspond to 0 to 1, and this is set as the sound rise value at time t. (Step 802). This value is set in the item of the sound climax information 611 (FIG. 6) at the time of the sound climax at time t and all sound times (to be exact, the time of (maximum value−τ)). The process ends (step 803). Note that these processing flows are all finished when the acoustic peak value is passed to the acoustic peak information 611 (FIG. 6) when the acoustic time is completed. The processing shown in the flowchart of FIG. 15 may be continued again after passing to the information 611 (FIG. 6) and finishing the calculation at each time.

(IV-3) Details of Operation History Analysis Processing Next, details of the operation history analysis processing will be described with reference to FIGS. 4, 6, and 11 and FIGS.
FIG. 16 is a flowchart showing details of the operation history analysis processing 505.
FIG. 17 is a diagram illustrating a process of constructing operation importance information.
FIG. 18 is a diagram illustrating an example of a result of constructing operation importance information.

  The operation history analysis process 505 is a figure calculated for the video at each time by scoring each operation of the operation information 187 shown in FIG. 4 based on the action score table 104 shown in FIG. 6 is for explaining the operation importance level information 106 shown in FIG.

  The process of creating the operation importance level information 106 by the following operation history analysis process 505 will be described on the assumption that there is no change by the user. However, the process may be performed so as to cope with the change of the user. .

  In the following description, it is assumed that the user 1 operation history 410 in FIG. 4 is input to the operation information analysis unit 102.

  In addition, although description will be made on the assumption that the input user operation information 187 (the user 1 operation history 410 in FIG. 4) is input collectively, it may be input sequentially for each operation.

  First, when the operation importance level information 106 of the video 1 is already present (step 1001), the operation importance level information 106 is acquired (step 1002). If there is no operation importance level information for the video, initialization such as memory reservation is performed (step 1003). While there is a user's playback operation (step 1004), the operation information is acquired (step 1005). First, the operation “load” of item number 1 is acquired.

  If it cannot be acquired or is not finished (step 1006), it is checked whether the recorded video ID has been changed (except for the first time) (step 1007). As a result, when the ID is changed, that is, when a different video is loaded, the current operation importance level information is updated (step 1020), and the process returns to the first process. If the images are the same (or the first time), the score for the operation is acquired from the action score table 104 (step 1008).

  Below, it demonstrates according to the specific example of the action score table 104 shown in FIG.

  In FIG. 7, 1 point is given when “reproduction” is performed, and 0 points are given otherwise.

  Therefore, the score is 0 for “load” of item number 1 in the user 1 operation history 410 of FIG. Since the previous content does not exist (step 1009), the start time and the score are recorded as the current content, and the previous content is updated as “0: 00: 0.0000” and “0”, respectively ( Step 1010), the next operation acquisition is performed.

  The next item number 2 “play” is acquired (step 1005), and it can be seen from the action score table 104 that the video position is “0: 00: 00.000” and the score is 1 (step 1008). Since the previous content is “load” and the score is 0, no scoring is necessary (step 1009), and for the current “playback”, the previous position is set to the video position “0: 00: 0.0000”. As the previous score “1”, the previous content is updated (step 1010), and the next operation is acquired.

  The next item number 3 “jump” is acquired (step 1005), and it can be seen from the action score table 104 that the video position is “0: 00: 046.822” and the score is 0 (step 1008). Since the previous content was “reproduction” and the score was one point, it is understood that scoring is necessary (step 1009). Here, “0: 00: 00.000” is obtained from the previous contents, and the current video position (jumped position) is “0: 00: 046.822”. It can be seen that the section where “” has been executed has a start position of “0: 00: 0.00.000” and an end position of “0: 00: 046.822”. Therefore, this section is scored “1”, and the positional relationship of the images is compared. Since there was no operation importance before the previous playback, scoring is performed with no relevant importance information (step 1030) and no division necessary (step 1031).

  As described above, the evaluation of the importance of the operation related to the previous operation is evaluated when the current operation is known. As described above, the end position is known even in the case of “play”. This is because the next operation. Also, the segment of the video giving the operation importance level is divided in order to evaluate the operation importance level of the section highly when “playback” is accumulated.

  Then, as shown in FIG. 17A, a portion 1210 having a start “0: 00: 0.0000”, an end “0: 00: 046.822”, and a score “1” is created. Then, as the previous contents, the previous position is updated as the video position “0: 00: 046.822” and the score is “jump” score “0”.

  The next item number 4 “play” is acquired (step 1005), and it can be seen from the action score table 104 that the video position is “0: 10: 43.024” and the score is 1 point (step 1008). Since the previous content is “jump” and the score was 0, no scoring is necessary (step 1009), and for the current “playback”, the previous position is the video position “0: 10: 43.024”. As the previous score “1”, the previous content is updated (step 1010), and the next operation is acquired.

  The next item number 5 “fast reverse” is acquired (step 1005), and it can be seen from the action score table 104 that the video position is “0: 16: 01.130” and the score is 0 (step 1008). Since the previous content was “reproduction” and the score was one point, it is understood that scoring is necessary (step 1009). Here, the position “0: 10: 43.024” is obtained from the previous contents, and the current video position (the position where the fast reverse operation has been performed) is “0: 16: 01.130”. It can be seen that the section where “reproduction” is performed has a start position of “0: 10: 43.024” and an end position of “0: 16: 01.130”. Therefore, this section is related to the video position of the score “1”. Related importance information is acquired since there was no operation importance before the previous playback (step 1030), but there is no overlapping part in relation to this time, so there is no need for division (step 1031). I will do it.

  Then, as shown in FIG. 17B, a portion 1211 having a start “0: 10: 43.024”, an end “0: 16: 01.130”, and a score “1” is created. Then, as the previous contents, the previous position is updated as the video position “0: 16: 01.130”, and the score is updated as the score “0” of “fast reverse” (step 1010).

  The next item number 6 “play” is acquired (step 1005), and it can be seen from the action score table 104 that the video position is “0: 01: 12.824” and the score is 1 point (step 1008). Since the previous content is “fast rewind” and the score is 0, no scoring is necessary (step 1009), and for the current “playback”, the previous position is set to the video position “0: 01: 12.824”. "And the previous score" 1 ", the previous content is updated (step 1010), and the next operation is acquired.

  The next item number 7 “jump” is acquired (step 1005), and it can be seen from the action score table 104 that the video position is “0:03: 10.3111” and the score is 0 point (step 1008). Since the previous content was “reproduction” and the score was one point, it is understood that scoring is necessary (step 1009). Here, the position “0: 01: 12.824” is obtained from the previous contents, and the current video position (the position where the rewinding is performed) is “0: 03: 10.311”. It can be seen that the section where “reproduction” is performed has a start position of “0: 01: 12.824” and an end position of “0: 03: 10.311”. Therefore, this section is scored as “1”, the positional relationship of the video is compared, and related importance information is acquired (step 1030).

  Here, the operation importance of the section where the overlapping part 1210 exists as shown in FIG. 17B is acquired, and it is divided in consideration of the overlapping part with the current part (step 1031) and scored. As a result, as shown in FIG. 17 (c), the section “0: 00: 00.000” to “0: 01: 12.824” is the portion 1220 of the score “1” and the section “0:01:12”. .824 ”to“ 0: 02: 46.822 ”is the portion 1221 of the score“ 2 ”, and the interval“ 0: 02: 46.822 ”to“ 0: 03: 10.311 ”is the score“ 1 ”. As a portion 1220, a section giving operation importance information is divided. Then, as the previous contents, the previous position is updated as the video position “0: 16: 01.130”, and the score is updated as the score “0” of “fast reverse” (step 1010).

  In this way, by analyzing user operation information one after another, the operation importance level information for each video is constructed, and the section that is finally reproduced has a higher score, and is considered as important as the section that the user thinks is desired Will be built as. FIG. 18 shows an example in which operation importance is given for each video time in this way.

(IV-4) Details of Scene Importance Information Creation Processing Next, details of the scene importance information creation processing will be described with reference to FIGS. 9, 19 and 20 already taken up.
FIG. 19 is a conceptual diagram of a method for creating the scene importance level information 114.
FIG. 20 is a flowchart showing processing of a method for creating scene importance information 114 according to the first embodiment.

  As shown in FIG. 9, the scene importance level information 114 is an index for evaluating the position where the final digest is played back. The scene importance level is calculated for each video playback unit (start time, end time). It was what was defined.

  The scene importance calculation unit 113 creates scene importance information from the sound excitement information as a feature extracted from the video data.

  For example, as described in the left part of FIG. 19, a time 1722 when the degree of climax 1701 of the sound climax information with respect to the time axis 1702 of the video exceeds the threshold th (1720) and a time 1723 again below th are obtained. A start time s1 (1731) and an end time e1 (1732) are obtained by adding a time ts (1721) and te (1724) before and after the interval, and obtaining an end time e1 (1732). A value normalized so that the maximum value becomes 1 is set as a scene importance level M1 (1750).

  Hereinafter, this scene importance level generation processing will be described with reference to the flowchart of FIG.

  First, the threshold value th defined by the processing already described in the parameter information 1501 (FIG. 8A) of the computation content information 105 managed by the computation content management unit 101 and the offset times ts and te that define the playback section are set. Obtain (step 1801).

  Also, the sound excitement information 611 (FIG. 5) is acquired from the feature information 112 (step 1802). Assuming that the acoustic climax information is arranged in ascending order of time, the degree of climax is acquired in order with respect to the total number E of item numbers (step 1803) and compared with th (step 1810). The degree is recorded (step 1811), and the start time at that time is recorded in the variable s (step 1812). From the next item number (step 1813), when the degree of swell exceeds M (step 1820), M is updated (step 1821), and while not falling below th (step 1822), comparison is made one after another. If the climax is less than th, the start time at that time (equal to the previous end time) is recorded in the variable e (step 1823). Then, scene importance level information with s-ts as the start time, e + te as the end time, and M as the importance level is added (step 1824), and the loop is completed. (In this flow, j) is assigned to the item number (also i) of the upper loop (step 1825), and the analysis is continued. After finishing the processing of the sound excitement information for all the item numbers, the scene importance level information normalized (step 1804) based on the maximum value of the scene importance level is created (step 1725), and the process ends.

  The normalization based on the maximum value of the scene importance is, for example, that each scene importance is divided by the maximum value so that it falls within the range of 0 to 1.

(IV-5) Details of Comparison Evaluation Process Next, the comparison evaluation process 522 will be described with reference to FIGS. 21 and 22.
FIG. 21 is a diagram for explaining the concept of the comparative evaluation process 522.
FIG. 22 is a flowchart showing the comparative evaluation process 522.
The comparative evaluation process is a process for obtaining the correlation degree while evaluating the current scene importance level and operation importance level. The correlation degree means the correlation degree between the scene importance degree and the operation importance degree, and is also an index for adjusting the parameter so that this value becomes large.

  First, the concept of the comparative evaluation process 522 will be described with reference to FIG. 21. The graph 1711 in FIG. 21A is the scene importance level information 114 created by the scene importance level calculation unit 113, and is shown in FIG. A graph 1301 is the operation importance level information 106 created by the operation information analysis unit 102, and the time axis of the video is compared vertically.

  At this time, as explained in the left part of both figures, regarding the section T1 (2021) surrounded by the end time e1 (1732) from the start time s1 (1731) detected as one section of the digest video, the maximum in the section For the rectangular area obtained by multiplying the operation importance I1 (2011) and T1 by the black portion p1 (2001) created by the operation importance graph, that is, a part of the operation importance item number included in this section Find the sum of products of importance and time. In relation to all the scene importance level information, a ratio between the total of the operation importance level part and the total of the scene importance level information with respect to the total sum of the rectangular areas is obtained, and this is used as the correlation level. This degree of correlation is obtained by the following (Formula 1).

ここで、シーン重要度情報は、Ｎ個あるものとしている。

Here, it is assumed that there are N pieces of scene importance information.

  Hereinafter, the comparative evaluation process 522 will be described with reference to FIG.

  First, the operation importance level information 106 is acquired (step 2101), and then the scene importance level information 114 is acquired (step 2102).

  Next, variables C and P are initialized to 0 (step 2103). For all item numbers of the scene importance level information 114 (step 2104), start time s and end time e are acquired (step 2120). Next, the item number x including the time s is searched from the operation importance information 106 (step 2122), and the item number y including the time e is searched (step 2122). Next, using the operation importance information (x), the operation importance information (y), and the sandwiched operation importance I, the product sum p of the time and the importance from time s to e is calculated ( Step 2123) is added to P (Step 2124). Further, I * (es) is added to C as the aforementioned rectangular area (step 2125). These are obtained with respect to the item numbers of all the scene importance information, and if C is 0 (step 2111), the correlation degree is set to 0 (step 2142), otherwise P / C is output as the correlation degree (step 2112). ) To finish the process.

(IV-5) Parameter-related processing and correlation degree adjustment Next, parameter-related processing and correlation degree adjustment will be described with reference to FIGS.
FIG. 23 is a diagram for explaining adjustment of a correlation degree by adjusting a parameter.
FIG. 24 is a flowchart showing the parameter specifying process 520.
FIG. 25 is a flowchart showing the parameter change process 530.
FIG. 26 is a flowchart showing the parameter update process 526.

  First, the significance of adjusting the degree of correlation by adjusting parameters will be described with reference to FIG.

  Specifically, in the present embodiment, the parameter r shown in FIG. 23 is a threshold value th of a sound excitement that is a parameter due to “acoustic excitement”, and ts and te that are offset times that define a playback section.

As shown in FIG. 23, when the parameter r (2402) is calculated from the minimum value r_min (2413) to the maximum value r_max (2415) with the change amount Δr (2414), the correlation degree c (2401) is obtained. When r ^* (2412) that maximizes the degree exists, it is updated as a value suitable for the user preference with respect to r, and the scene importance degree information 114 is obtained. Then, by obtaining this with respect to all other parameters, a parameter group is selected for more optimal scene importance level information 114.

  Next, the parameter designation process 520 will be described with reference to FIG.

  The parameter designation process 520 is a parameter value initialization process.

  First, when the processing content and the parameter to be adjusted are given to the operation content management unit 101, the variable of the corresponding parameter is acquired from the parameter information 1501 of the operation content information 105 shown in FIG. 8A (step 1601), and the minimum value is obtained. The maximum value and the amount of change are acquired (step 1602). Then, the parameter value is set to the minimum value (step 1603).

  Next, the parameter changing process 530 will be described with reference to FIG.

  The parameter change process 530 is a process for changing a parameter value.

  The calculation content management unit 101 adds the amount of change to the current parameter value to obtain a new parameter value (step 2501).

  If the parameter value exceeds the predetermined maximum value (step 2502), the parameter is set to be analyzed (step 2504). Otherwise, the parameter value is stored. (Step 2503).

  Next, the parameter update processing 530 will be described with reference to FIG.

  The parameter update process 530 is a process of feeding back the current parameter value to the calculation content information 105.

  When going to the route of this parameter update processing 530, since the correlation c is the maximum, the current parameter value is acquired (step 2301) and recorded in the calculation content information 105 as the calculation content for each user / category. (Step 2302).

  When parameters are related to each other, the analysis of the entire parameter group is regarded as one unit, and many units may be collected to generate the scene importance level information 114 and perform optimum parameter adjustment based on the correlation degree. .

Further, in this embodiment, the analysis was only for the digest creation video information creation method due to the sound excitement. However, in combination with other creation methods, a comprehensively optimal creation method or a plurality of combinations are performed. A parameter group may be obtained. At that time, as shown in the calculation content information shown in FIG. 8B, the obtained parameters are classified and stored in the calculation content information 105 for each user and each genre to which the video belongs.
(V) Digest Video Reproduction Process Next, the digest video reproduction process will be described with reference to FIG.
FIG. 27 is a diagram illustrating a display example of a seek bar when digest video playback is performed.

  As described above, in the present embodiment, the digest video information is obtained by analyzing the user operation, quantifying and evaluating the user operation, obtaining the scene importance level, and adjusting the parameters without an explicit instruction from the user. Create Therefore, digest video information in which user preferences are automatically extracted can be created. When playing back the digest video, the playback position information 116 shown in FIG. 10 is transferred to the recording / playback control unit 183 of the recording / playback unit 180 for playback.

When playing back a digest video, for example, when viewing the video on the display device 190 of FIG. 11 and playing back the digest video when the user operates the digest playback button 350, the entire position during playback is visualized and presented to the user. A portion to be played back by the seek bar 305 displayed in the locator area 303 to be displayed may be displayed. The example of the seek bar display when performing digest video playback shown in FIG. 27 shows two examples in which the playback part is different when different digest methods and parameters are used in the same video (2701, 2702). Here, the leftmost corresponds to the start time of the video, and the rightmost corresponds to the end time of the video, and the black bands (2710 to 2714, 2721 to 2728) are sections that are extracted and reproduced as digests therein. There is. For example, when viewing a video belonging to the genre of “sports (soccer)”, when the present embodiment is implemented, if there is no calculation content (2010 to 2030) for each user before the user operation, FIG. It is assumed that the digest video section 2701 is reproduced one after another using the “sport (soccer)” parameter of the calculation content 2002 by common genre. Thereafter, regardless of whether digest playback is performed or not, when user 1 performs a playback operation, genre-based calculation content 2012 of user 1 calculation content information 2010 is created. Then, using the digest video information creation method according to the present embodiment, the parameter changes according to the operation of the user 1 and, as a result, an item such as item number 1 in the genre calculation content 2012 and the parameter are created. Therefore, when the user 1 performs the digest playback of “sports (soccer)”, the digest video section before viewing using this parameter (2701) is different from each other, or a different section is played back. Becomes possible (2702). In addition, since the calculation method and parameters are managed for other genres or other users, it is possible to create digest video information by calculation suitable for each.
(V) Features of the present embodiment According to the present embodiment, an instruction to an explicit digest generation method and parameters from the user is analyzed by analyzing the playback operation information so far on the video that is subject to user preference. It is possible to automatically adjust the playback position of the video digest without requiring input.

  In particular, in the present embodiment, since the upsurge by voice is adopted as the feature information, it is suitable for live broadcasting of sports as a video field.

  In addition, it is possible to evaluate the preference according to the section that has been viewed repeatedly, the playback speed, and the playback method as an analysis method. In addition, by making the action score table precise, due to changes in playback speed or changes in playback method, such as watching a section with high preference or watching with a high playback speed and skipping a section with low preference It becomes possible to evaluate the preference in detail.

  Also, it is possible to create digest video information using the creation method obtained by analysis not only for the video to be analyzed from the reproduction operation information but also for other video.

  Furthermore, by analyzing each reproduction operation information with different preferences for each user, it is possible to create digest video information suitable for each user.

  In addition, by analyzing the reproduction operation information for each genre to which the video belongs, it becomes possible to reflect the user preference specific to each genre.

[Embodiment 2]
Hereinafter, the second embodiment will be described with reference to FIGS. 28 to 34.

  In the first embodiment, attention is paid to the rise of sound with respect to video data, and it is extracted as feature information.

  On the other hand, in the present embodiment, a personal name part is extracted from text data such as telop and subtitle data in a moving image included in a video, a part of video data in which a desired personal name appears is searched, and before and after Digest video information is created by using the video of the section as a section constituting the digest video. Digest video information is created by this person name search. When a player is switched and displayed as in a sports video on TV, or when there are multiple characters, you want to watch the part where a specific person appears as a digest video. This is an effective method. For example, it is effective when the characters are changed in the vicinity of a scene appearing for each batter in a baseball game or for each corner such as variety. Assuming this embodiment, it is assumed that the user views a part where a specific person appears. This viewing method is generally a natural method when not viewing the entire video when viewing after recording, but viewing the important part to shorten the time. That is, as a method of creating digest video information for a video with many characters as described above, it is possible to place importance on the portion where the person name is displayed, but in this embodiment, the user repeats viewing. A further object is to create a digest video suitable for the viewing method of the user. In the following description, character string information is subtitles (or closed captions) for displaying the character string information separately from the moving image in the video data so that the characters can be visually recognized by being superimposed on the moving image as necessary. However, it may be a telop (or superimpose) embedded in a moving image by a technique such as computer graphics. In this case, by using a general OCR (Optical Character Recognition) technique or the like using an image processing method, it is possible to handle the same as subtitles. Similarly, you may handle the character string obtained by using a general voice recognition technique etc. from an acoustic.

  Here, the example of the video information 110 in the video information recording unit 184 of FIG. 2, the example of the image when the user watches the video of FIG. 11, and the example of the operation information of FIG. 4 are the same as the first embodiment. Assume that they are similar.

  Further, the process of acquiring the operation information 187 when the user views the video data is the same as that in the first embodiment.

  Furthermore, the process up to creating the scene importance information of the digest video information creation process shown in FIG. 12 is the same as in the first embodiment, and only different parts in each process will be described in detail. The rest is easy to touch.

The parts that are different from the first embodiment in particular are the feature extraction process 503, the parameter initialization process 507, and the scene importance level information creation process 521, which will be described in detail below.
(I) Information and Data Structure Related to Digest Video First, information and data structure related to the digest video according to the second embodiment will be described with reference to FIGS. 8A and 8B and FIGS. 28 and 29 already described.
First, the feature information 112 according to the second embodiment will be described with reference to FIG.
FIG. 28 is a diagram illustrating an example of the feature information 112 according to the second embodiment.

  In the example shown in FIG. 28, it is indicated that the feature information 112 includes caption person name feature information.

In the video 1, a subtitle in the video data, for example, a character string “Sato keeps the ball and feeds to Tanaka” is displayed in the section “0: 00: 05.255 to 0: 00: 08.822”. When there is subtitle data such as that, the information is extracted, and as shown in FIG. 28, the subtitle personal name information entry of the video feature management table 3010 stored in item number 1 of the subtitle personal feature information 3011 is Pointing to the subtitle personal name characteristic information 3011.

  In the caption personal name feature information 3011, the start time, the end time, and the personal name are registered with respect to the personal names extracted in time order.

  Next, the calculation content information 105 according to the second embodiment will be described with reference to FIGS. 8A, 8B, and 29. FIG.

  The parameter information 1501 used in this embodiment is a parameter for which the calculation method of item number 2 is “keyword”. As parameters for processing by keywords, among the number of keywords registered in order, numKwd is the number of keywords from the top of the digest used for digest creation, kwd [] is an array that holds registered keywords, and an offset that defines the playback range Suppose that there are ts and te which are time.

  For numKwd, ts, and te, a maximum value, a minimum value, and a change amount are determined, respectively.

Next, the calculation content information 105 according to the second embodiment will be described with reference to FIG.
FIG. 29 is a diagram illustrating an example of each parameter value of the calculation content information 105 according to the second embodiment.

The calculation content information 105 newly added in the present embodiment is keyword importance information that defines importance for each keyword. This is recorded together with the calculation content information 105 shown in FIG. 8B. The keyword importance information is held for each genre for each user, and holds keywords that the user seems to like in each genre. The keyword importance information may be recorded as genre-independent keyword groups as common keyword information 3111. In addition, the user may change, add, or delete contents independently, and the method is not particularly limited in this embodiment. In the keyword importance, item numbers, keywords, and importance are recorded. It is assumed that the keywords are not duplicated and are managed with item numbers in descending order of importance.
(I) Details of Feature Extraction Processing First, details of the feature extraction processing 503 will be described with reference to FIGS. 30 and 31.
FIG. 30 is a conceptual diagram when subtitle person name information is extracted from subtitle data in video data.
FIG. 31 is a flowchart illustrating a process for extracting caption person name information as an example of the feature extraction process.

  The feature extraction processing 503 of this embodiment is processing for extracting caption person name information from caption data in video data and generating caption person name feature information for each video in the feature information 112 shown in FIG.

As shown in FIG. 30, a subtitle in the video data, for example, a character string “Sato keeps the ball and feeds to Tanaka” is a section “0: 00: 05.255-0: 00: 08.822”. If there is subtitle data to be displayed on the screen (step 2901), the feature extraction unit decomposes the character string into words by analyzing a general morpheme (the smallest unit having meaning in the language) (step 2902). Thereby, it is separated into verbs, particles, nouns, etc., and can be classified into personal names in particular (step 2903). Based on this, the appearing section and the extracted person name (step 2904) are registered in the caption person name information as one feature of the video data (step 2905). By accumulating these steps for all the video data, it is possible to use subtitle personal name information for digest creation.
(II) Generation of Operation Importance Level Information Operation importance level information 105 can be generated by performing the operation history analysis process 505 of the first embodiment based on a playback operation when a user views a video. .
(III) Details of Parameter Initialization Process Next, the parameter initialization process will be described with reference to FIG.
FIG. 32 is a flowchart showing parameter initialization processing 507 according to the second embodiment.

  In the following description, it is assumed that the video to be picked is video 1, the user is user 1, and the genre is “sports (soccer)”.

  First, caption person name feature information 3011 (FIG. 28), operation importance information 1011 (FIG. 6)), and keyword importance information 3113 (FIG. 29) are acquired (steps 3201 to 2033). At this time, if the user, the current video ID, and the genre to which the video belongs are known, the genre information suitable for each is acquired, and if not, the respective common information 3111 is acquired.

  Then, with respect to all the personal name information of the subtitle personal name feature information 3011 (step 3204), the operation importance information relating to the section is obtained, and the maximum importance is set to z (step 3210). For example, the section “00: 00: 05.255” (start time) to “00: 00: 08.822” (end) for item number 1 “Sato” in the caption name feature information (2904) of video 1 in FIG. 6), in the user 1 and video 1 importance 911 in FIG. 6, the item number 3 and the item number 4 are related to this section, and the maximum importance “5” of the two is substituted for z. .

  If no keyword has been registered in the keyword importance information 3113 (step 3221), the current person name is registered as a keyword, the number of keywords W is set to 1 (step 3215), the importance is set to z (step 3216), Move processing to the name of the person.

  If there is a registered keyword and W is 1 or more (step 3211), a duplicate keyword is searched. That is, with respect to the registered keyword importance information, the current person name and the keyword are compared (step 3220), and if they are the same, if z is greater than the currently registered importance (step 3221), z is set. The importance level is updated (step 3222), and the process moves to the next person name.

  If it is not the same even when compared with all keywords, the current person name is newly registered as a keyword, the number of keywords is increased by 1 (step 3213), the importance is set to z (step 3214), and the next Move processing to the name of the person.

When all the names have been processed, the contents are rearranged in descending order of importance of the completed keyword importance information (step 3205), and the keyword importance information is updated in the operation content information (step 3206). The process ends. It should be noted that the keyword and the importance can be updated by this parameter initialization process for another program of the same genre.
(IV) Details of Scene Importance Creation Processing Next, details of the scene importance creation processing will be described using FIG. 33 and FIG.
FIG. 33 is a flowchart showing processing of a method for creating scene importance information 114 according to the second embodiment.
FIG. 34 is a view for explaining the concept of a method for creating scene importance information 114 according to the second embodiment.

  Parameters for creating the scene importance level information 114 of the present embodiment include the number numKwd from the top of the keyword importance level information used for the search, the start time offset ts and the end time offset for adjusting the time segment for playback. te.

  First, in the process of creating the scene importance information 114, numKwd, ts, te corresponding to the user and the genre of the video is acquired from the calculation content management unit 101 (step 3301), and the keyword importance information is acquired. To do. Here, the acquired parameter is shown in item number 3 of the genre calculation content 2612 in FIG. 8B. The keyword importance level information is the genre-specific keyword importance level information in FIG.

  In addition, the subtitle personal name feature information regarding the corresponding video for which digest video information is to be created is acquired. The caption personal name feature information is shown by 3011 in FIG.

  Next, with respect to the upper number of keyword importance information by the number of numKwd, the corresponding keyword is searched from the personal names of the caption personal name feature information (step 3304). First, the current keyword is searched from the subtitle personal name feature (step 3310). For each of the subtitle person information searched there (step 3311), the start time -ts of the subtitle person name is substituted for the variable s, the end time + te is substituted for the variable e, and the importance of the keyword is substituted for the variable q. (Step 3321), a new scene importance element to be added is prepared. Next, in the scene importance level information, elements relating to the section s to e are acquired (step 3321), and the prepared scene importance level elements (s, e, q) are added to each of them (step 3322). Then, it is determined whether the element needs to be divided or changed. When division or change is necessary because the times overlap, when either the start time or the end time is applied to a new element, these are adjusted so that there is no overlap on the time axis. In addition, when a new element is sandwiched between the start time and the end time, it can be adjusted so as to sandwich the new element by dividing it into two and changing each time. In addition, when the start time and end time are included in the section of the new element, (1) start of the new element to start of the previous element, (2) start of the previous element to the start of the previous element The adjustment can be divided into the end, (3) the end of the previous element to the end of the new element. However, these are adjusted as appropriate so as to have a higher importance (step 3331). That is, when a new element is included in the previous element and the importance of the previous element is higher, it is not necessary to leave the previous element as it is and add it as a new element. In this way, when a new element is necessary, the element is registered (step 3332).

More specifically, referring to FIG. 34, for example, when the keyword “Sato” with importance q1 (3410) is searched for and found (3440, 3441), the start time 3412 and the end time 3413 of the personal name feature are related to the case 3440. Each of offset ts (3411) and te (3414) is obtained as s1 (3431) and e1 (3432) so as to widen the section. If there is no overlapping existing scene importance in this part, the scene importance is registered with (s1, e1, q1) as elements (3405). For the “Sato” part of the case 3441, the same item is prepared as the element (s2, e2, q1), but “Suzuki” (3442) having the importance q2 (3415) is already new as shown in the figure. If it is registered so as to overlap, the start time of the existing scene importance item (3407) is changed to e2 (3434) so as to leave “Sato” having high importance, and a new element ( s2, e2, q1) are registered. In this way, the scene importance level is registered so that a part with a high importance level remains. Finally, normalization is performed so that the importance becomes 1 at the maximum (step 3305), the scene importance information is updated (step 3306), and the process is terminated.
(IV) Storage of information about parameters In this way, the feature extraction processing of caption personal names related to video, the parameter initialization processing for obtaining keyword importance, and the scene importance information creation processing by human name search using these are shown in FIG. By adapting to each process of the digest video information creation process shown in, it is possible to set parameters for digest video information creation centering on the part of the video data where the desired person name appears based on the user's playback operation Become. These parameters and keyword importance information are classified and stored in the calculation content information 105 for each user and for each genre to which the video belongs (FIG. 8B, in addition, FIG. 29).
(V) Features of the present embodiment In the present embodiment, keywords flowing in a telop or the like are employed as feature information. Therefore, taking a person's name as a keyword is suitable for digest reproduction of sports videos of players interested in the user. Also, if the name of a person is a celebrity, it can also be used to play a digest of an entertainment-related video. In addition, if a keyword is a specific topic, it can be applied to news and the like.

[Embodiment 3]
Hereinafter, the third embodiment will be described with reference to FIG.
FIG. 35 is a diagram showing an example of the action score table 104 according to the third embodiment.

  The action score table 104 of the first embodiment was scored independently for one operation, as shown in FIG. 7, but the action score table 104 of the present embodiment is This is an example configured based on the transition between the previous operation and the current operation. In the action score table 104 of FIG. 7 of the first embodiment, an example in which a score is given in the case of reproduction is shown, but one point is given regardless of what operation was performed before that. The action score table 104 of this embodiment considers the transition of operations. For example, when fast-forwarding or jumping is performed after playback, it is considered that the user has watched an uninteresting part at the end of playback. Decrease (1401), how much the user feels to fly by the speed of fast forward / reverse (1402), and consider the case where the speed has changed among them (1403) Therefore, it is possible to evaluate the operation more accurately considering the user's preference.

[Embodiment 4]
Hereinafter, the fourth embodiment will be described with reference to FIG.
FIG. 36 is a flowchart showing the playback position determination process.

  In the first embodiment, the example in which the scene importance level information 114 is generated while adjusting the parameters has been described. However, in the present embodiment, the scene importance level information 114 and the reproduction position information 116 are simply determined based on the given parameters. A process of creating a digest video and playing back a digest video will be described.

  The parameters here are the calculation content by genre of the calculation content information 105 in FIG. 8B and the keyword importance information in FIG. 29 in the calculation content information (105) determined in the above embodiments. Further, the calculation content information 105 is transmitted via the exchange unit 120 shown in FIG. 1 via the network 170 and the external medium 160 such as a memory or a CD-ROM, to the external server 171 or other You may acquire from an apparatus. This makes it possible to share digest creation parameters that are generally popular, parameters to create digest video information with users of other video devices, or the digest video information that is normally used in other video devices. , Digest playback becomes possible.

  In this playback position determination process, first, the processing content and playback time for completing digest playback are acquired from the user through the operation unit 182 (steps 3501 and 3502). The processing content is an instruction of which video digest video information is to be created.

  Next, parameters relating to the user, the video to be played back, and the genre to which the user belongs are acquired (step 3503), and the scene importance calculation unit 113 uses the scene importance calculation unit 113 to create scene importance information using the selected method. Information creation processing (FIG. 20 or FIG. 33) is performed. Next, using the obtained scene importance level information 114 (1901 in FIG. 19), the playback position information creation unit 115 performs the video information creation process (playback position information generation process) shown in FIG. 13 (step 3505). ).

  In this embodiment, a parameter for creating digest video information created by an external video device or computer is received via a network or external media, and by using it, the parameter optimized externally is used. Digest video information can be created. For this reason, the apparatus on the receiving side reduces processing for creating digest video information.

It is a block diagram which shows the whole structure of the video device which reproduces | regenerates the digest image | video which concerns on 1st embodiment (the 1). It is a block diagram which shows the whole structure of the video device which reproduce | regenerates the digest image | video which concerns on 1st embodiment (the 2). 5 is a diagram illustrating an example of data stored in a video information recording unit 184. FIG. It is a figure which shows an example of the operation information 187. It is a figure which shows an example of the characteristic information 112 which concerns on 1st embodiment. 6 is a diagram illustrating an example of operation importance information 106. FIG. It is a figure which shows an example of the action score table 104 which concerns on 1st embodiment. It is a figure which shows the parameter information 1501 of the calculation content information 105. FIG. It is a figure which shows an example of each parameter value of the calculation content information 105 which concerns on 1st embodiment. It is a figure which shows an example of the scene importance information 114. FIG. 6 is a diagram illustrating an example of reproduction position information 116. FIG. It is the figure which showed the image when the user is viewing video. It is a general chart which shows a process until the scene importance information of a digest video information creation process is created. It is a flowchart which shows the process which produces the reproduction | regeneration position information of a digest video information creation process. It is a conceptual diagram when producing | generating sound excitement information as an example of a feature extraction process. It is a flowchart which shows a process when producing | generating sound excitement information as an example of a feature extraction process. 10 is a flowchart showing details of operation history analysis processing 505; It is a figure explaining the process of constructing operation importance information. It is a figure which shows the example of a result which constructed | assembled operation importance information. It is a conceptual diagram of the method of creating the scene importance information 114. It is a flowchart which shows the process of the method which produces the scene importance information 114 which concerns on 1st embodiment. It is a figure explaining the concept of the comparative evaluation process 522. It is a flowchart which shows the comparative evaluation process 522. It is a figure explaining adjusting a parameter and adjusting a correlation. It is a flowchart which shows the parameter designation | designated process 520. It is a flowchart which shows the parameter change process 530. It is a flowchart which shows the parameter update process 526. It is a figure which shows the example of a display of a seek bar when performing digest image | video reproduction | regeneration. It is a figure which shows an example of the characteristic information 112 which concerns on 2nd embodiment. It is a figure which shows an example of each parameter value of the calculation content information 105 which concerns on 2nd embodiment. It is a conceptual diagram when subtitle person name information is extracted from subtitle data in video data. It is a flowchart which shows a process when extracting subtitle person name information as an example of a feature extraction process. It is a flowchart which shows the parameter initialization process 507 which concerns on 2nd embodiment. It is a flowchart which shows the process of the method which produces the scene importance information 114 which concerns on 2nd embodiment. It is a figure explaining the concept of the method of producing the scene importance level information 114 concerning 2nd embodiment. It is a figure which shows an example of the action score table 104 which concerns on 3rd embodiment. It is a flowchart which shows a reproduction position determination process.

Claims (10)

In a digest video information creation method for playing back a video digest played on a video device,
(1) a feature information section extracting feature information from video data of video;
(2) a step in which the operation unit obtains operation information of an operation on the video viewed by the user;
(3) The operation information analysis unit generates operation importance information that is importance at each time of the video evaluated by the operation information by scoring each operation of the operation information;
(4) The scene importance calculation unit determines a scene that is a playback time interval in the video for playback as a digest by using one or more parameters, and the scene is evaluated by the feature information. Generating scene importance information, which is the importance in each scene,
(5) The digest control unit includes a step of evaluating a correlation between the operation importance information and the scene importance information, and determining a combination of parameters for increasing the correlation.
The digest control unit generates optimal scene importance information for digest playback by repeating the steps (4) and (5),
A digest video information creation method, wherein a playback position information generation unit generates digest playback position information based on the generated scene importance information and digest playback time. The feature information of the video is sound excitement information,
2. The digest video information creation according to claim 1, wherein the parameter is a combination of a threshold value for judging the scene of the sound excitement and a value for adjusting a start time and an end time of a scene reproduced as a digest. Method. The video feature information is keyword information about the video,
The parameter is a combination of a number of registered keywords used for calculating the scene importance and a value for adjusting a start time and an end time of a scene reproduced as a digest. The digest video information creation method according to 1.   The scoring of each operation of the operation information in the step (3) is performed using an action score table evaluated by a viewing method, viewing frequency, viewing speed, or viewing method, transition of viewing speed for a video. The digest video information creation method according to claim 1, wherein: The parameter is determined by the first video,
2. The digest video information creation method according to claim 1, wherein scene importance level information for a second video different from the first video is created by the first video according to a determined parameter. The parameter is stored for each user,
2. The digest video information creation method according to claim 1, wherein when the scene importance information is generated in the step (4), a user is specified and scene importance information for each user is generated. Further, the method includes a step of acquiring a genre to which the video belongs from attribute information of the video or information input from a user,
The digest video information creation method according to claim 1, wherein the parameter is held for each genre of the video.   The digest video creation method according to claim 1, wherein the parameter is transmitted via a network or an external medium. In a video device to play a video digest,
A recording / playback unit for recording / playback the video;
A digest video information creation unit for reproducing the digest,
The recording / playback unit
A video storage unit for storing data relating to the video;
An operation unit for inputting an operation instruction to the device from the user;
A recording / playback control unit for controlling the recording / playback operation of the video;
A display unit for displaying the video to the user,
The digest video information creation unit
A digest control unit for controlling each unit;
An operation information analysis unit for analyzing the operation information of the user;
A feature extraction unit for extracting features from video data of the video;
A scene importance calculation unit that generates scene importance information that is importance in each scene evaluated by the feature information with respect to a scene that is a playback time interval in the video to be played back as a digest;
A comparative evaluation unit that evaluates the correlation between the operation importance of the operation importance information and the scene importance of the scene importance information;
A playback position information creating unit that creates playback information for playback of a video digest based on the scene importance level information and playback time;
The digest control unit repeatedly calls the scene importance calculation unit and the comparison evaluation unit to determine a combination of parameters for determining a reproduction time for reproduction as a digest from the reproduction time of the video. A video apparatus characterized in that the degree of correlation evaluated by is increased. In a program for creating digest video information for playing back a digest of a video executed on a computer or video device and played back on the video device,
(1) a function of extracting feature information from video data of a video;
(2) a function of acquiring operation information of an operation for video viewed by the user;
(3) a function of generating operation importance information that is importance at each time of the video evaluated by the operation information by scoring each operation of the operation information;
(4) The scene importance calculation unit determines a scene that is a playback time interval in the video for playback as a digest by using one or more parameters, and the scene is evaluated by the feature information. A function for generating scene importance information, which is the importance in each scene,
(5) a function of evaluating a correlation between the operation importance information and the scene importance information and determining a combination of parameters for increasing the correlation;
By repeating the execution of the functions (4) and (5), optimal scene importance information for digest playback is generated,
A program for generating digest playback position information based on the generated scene importance level information and digest playback time.

Images